[HN Gopher] A serverless email server on AWS using S3 and SES
___________________________________________________________________
A serverless email server on AWS using S3 and SES
Author : st_goliath
Score : 323 points
Date : 2021-04-30 12:52 UTC (10 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| the_arun wrote:
| Very creative solution! Building an enterprise product using this
| will be fun!
| someguy101010 wrote:
| I use this myself for forwarding emails and signing up for random
| sites. Pretty easy to set up and 0 overhead to maintain. One of
| these days I'll set up a ui around it to send emails. Having both
| set up and maintained an email server "the old fashioned way" and
| using this, the over head on this is way less than postfix et al.
| This only goes for receiving email though, sending email with
| this is a biiit clunky, and doesn't support attachments (yet). If
| you have an aws account and a spare domain give this project a
| shot! It has a surprising amount of utility.
| kadoban wrote:
| If you only want forwarding, you could look at
| https://www.nearlyfreespeech.net/services/email . You'd have to
| do a bit of math to figure out the total price (depending on if
| you want/need DNS there to make it work), but it is _really_
| cheap and easy.
|
| (not affiliated in any way, just a customer)
| gruez wrote:
| >Pricing
|
| >$1 per active CodePipeline (must run at least once a month to be
| considered active)
|
| I'm not too versed in how aws services work, but how many active
| codepipelines are you expected to have running every month? A few
| paragraphs up it says that there are 3x CodePipelines in total,
| but they might not be all active every month. If all of them are
| active, then that would come out to $3/month, which seems a bit
| pricey considering you can get an actual hosted email plan for
| $5[1]. If you're looking to save money you can go even
| lower[2][3]
|
| [1] https://www.fastmail.com/pricing/
|
| [2] https://purelymail.com/pricing
|
| [3] https://tutanota.com/pricing/
| rahimnathwani wrote:
| I was hoping this would include a lambda set up to respond to
| incoming requests on ports 25/465/587, but all the SMTP parts
| (both server/inbound and client/outbound) are being handled by
| Amazon SES, not by the serverless functions.
| OJFord wrote:
| It's not possible. I wanted to do that with mine (similar thing
| to OP) - not actually for SMTP but for IMAP or even POP3, but
| you can't with Lambda - you can only reach it (from outside
| AWS) with HTTP(S).
|
| (I use a Lambda to forward to another provider currently, and
| use their client. I plan to write my own to replace that;
| ultimately I'd just like to run a JMAP server on a Lambda, that
| you _could_ do.)
| unilynx wrote:
| There's another much more important limits on AWS SES that the
| article doesn't mention if you try to do this for normal email,
| the 10MB message size limit, which translates to about a 6MB
| maximum attachment size.
|
| It works fine for transactional mail and most newsletters, but
| quite a few users expect to be email larger files than that.
| philliphaydon wrote:
| They shouldn't. Most ISP email providers don't support more
| than 10mb.
| edoceo wrote:
| gmail is 25MB, yandex is 30MB.
| jeffbee wrote:
| Gmail is more than 25MB. It appears to be 150MiB.
| [disclosure: I was involved in the development of raising
| this limit, years ago]. $ nc
| aspmx.l.google.com 25 220 mx.google.com ESMTP
| ba11si4162422plb.407 - gsmtp EHLO foo
| 250-mx.google.com at your service, [[IP address elided]]
| 250-SIZE 157286400 [...]
| edoceo wrote:
| hm. when I did a Google search, the instant-answer
| snippet shows 25MB
| jeffbee wrote:
| "Some documentation may be out of date" is practically
| Google's motto.
| philliphaydon wrote:
| ?!? I said ISP emails, which a lot of people still use.
| Outlook/Hotmail are 10mb though. And outlook client is 20mb
| received.
| edoceo wrote:
| I'm only trying to help. Like, every PM has used gmail
| regard it as "normal" - so if you're different, you're
| abnormal.
|
| also, if your PMs listen to reason here's a site with a
| table
|
| https://www.outlook-apps.com/maximum-email-size/
|
| which show the 10MB size at the low-end but still used by
| a huge carrier
| superkuh wrote:
| I'm confused, does this use different IP addresses constantly?
| How is this mailserver able to send email without an IP based
| reputation? Is it just that Amazon is so giant other mega-corp
| email providers are wary to blacklist their IP range?
|
| If so this is a wonderful way to send spam without ever worrying
| about IP reputation. I imagine it will be abused and then
| blacklisted quickly.
| onefuncman wrote:
| The Lambda doesn't send or receive email directly, it uses SES.
|
| SES handles this in a delegated way via DKIM signatures (Domain
| Keys Identified Mail).
| superkuh wrote:
| The domain signing with DKIM still doesn't get you a valid IP
| reputation. It may help to gradually build one if it's the
| same IP over and over, but it's still a brand new IP suddenly
| sending mail from $domain. If it weren't an amazon IP it'd
| certainly get refused or at least marked as spam and hidden
| after acceptance.
|
| Just another example of one set of rules for human people and
| another for corporate people. Either pledge yourself to a
| corporate feudal lord (amazon here) or suffer.
| maerF0x0 wrote:
| @st_goliath does David Gatti know you posted? I would love to
| hear from the creator :)
| adontz wrote:
| From one side, great work!
|
| From another side, this looks much more complex, that getting an
| email service with catch-all address and use rules/filters to do
| the same. I'd programmatically manage rules/filters and folders,
| rather than this entire system. All email providers I have used
| on scale (Google, Microsoft, Rackspace) support this. I'd still
| have a usual email service with SMTP and IMAP, rather than
| incompatible custom JSON APIs and unknown failure modes.
|
| What is really interesting is cost on scale. Because if it is
| cheaper to send/receive 100M emails though this, than it's worth
| trying. However this is exactly the part not covered at all. And
| if you are sending not a lot of emails, maybe pay $50 to major
| email provider and spend your time on something more important?
| irae wrote:
| This is a brilliant architecture.
|
| Everyone here talking about no email clients, IMAP and AWS
| restrictions is missing the point. Although you could use this as
| a personal account, or as a company. This fully automated system
| is a perfect ground layer to a number of applications with a very
| cleaver API.
|
| Say you are a small startup, and you will need some email
| functionality? Give your web server access to writing to outbox
| and you can send email. Use the sender with + rules and you get
| replies with customerId, threadId etc. Need to show email
| interactions in your service UI? Subscribe to the notification
| channels. Need customer support platform, it is almost built-in.
|
| If you don't see value in this solution, you probably never tried
| to setup infrastructure to receive email. In fact, at $work, I
| developed a very similar architecture and it is running
| flawlessly for 15 months or so. We just don't do all this
| git+lambda craziness, we just had already setup EC2 that we added
| some routes.
| danielvinson wrote:
| As somebody who has been an early engineer at many startups, I
| would never use this in almost any case because a managed
| solution is just better. If I can pay somebody else to do
| something hard with zero maintenance so I can focus on shipping
| a good product, I'm going to do that every time.
| femto113 wrote:
| > Give your web server access to writing to outbox and you can
| send email.
|
| SES is already in the mix, so why not just use it directly to
| send emails from your web server?
| StreamBright wrote:
| This is how we handle notification emails and some incoming
| emails as well. There are few gotchas only (using separate
| domains is one of those)
| eljimmy wrote:
| Am I reading this correctly, you have to upload a JSON file to an
| S3 bucket to send an email? That's not really email then, is it?
| fnord77 wrote:
| it wouldn't be hard to have a JS ui that presents an email
| form, but converts that to JSON and posts it to S3.
| bdcravens wrote:
| Sure, but pretty much every other email option out there has
| protocol support. Without that, I think it's a stretch to
| call this an email server. (Though there are a few that are
| only accessible via their client, such as Hey)
| [deleted]
| ydant wrote:
| SES supports using SMTP to send emails - so I don't know why
| this particular setup would prohibit that.
|
| https://docs.aws.amazon.com/ses/latest/DeveloperGuide/send-e...
| eljimmy wrote:
| > This flow was designed to take advantage of the S3 trigger
| system and break each action into a small Lambda.
|
| The email flows from S3 => Lambda => SES so the underlying
| design seems to prohibit sending of emails directly from SES.
|
| It sounds like SES could still be used directly, but you then
| need to add in authorization and user management if you're
| accessing it externally.
|
| It's a cool project and I can envision it being used for
| internal tooling for automated email notifications or
| something of that nature. Just doesn't seem practical for
| users.
| weego wrote:
| AFAIK Amazon provides no guarantees that s3 file triggers
| will actually run, though they 'probably always will'
| philliphaydon wrote:
| I setup a super super basic version of this which basically
| allows me to accept emails from one of my domains and forwards it
| to my real email.
|
| (Cos I use AWS DNS and don't have the ability to forward like I
| do in name cheap)
| oblib wrote:
| I dunno... I run my own email server and it feels to me like the
| giant gap in the ease of getting your own email server (or
| services) up and running that you can connect to a web app still
| hasn't closed much here, especially for me because I don't use
| AWS and I don't want to spend the time to learn.
|
| My experience with 3rd party email services is they're also a
| pita to use and that can become huge when they merge or get
| bought up by a competing service or die, or raise their prices.
|
| With the web browser we can configure very simple emails that
| they can send with a link to click via the user's email client
| but as a web app developer what I really need is to be able to
| send a html email that way.
|
| Maybe web browser and email app makers could add a way to do that
| by letting apps request permissions and users approve it for a
| specific web app. That would really make life easier for me and
| my apps users.
| rdsedmundo wrote:
| I built something on those lines, although way simpler in terms
| of functionalities and resources used.
|
| All I wanted was to receive/send emails from a custom address
| (@acme.com) using Gmail, but I didn't want to pay Google Suite
| for that.
|
| So I used SES+S3+Lambda to receive emails and forward then to a
| Gmail alias (email+acme@gmail.com) so I can attach a filter on
| them using a label. And for sending emails using the address, I
| simply authenticated it on Gmail using the "Send mail as" option
| using AWS SES SMTP credentials.
| hrez wrote:
| This is as "serverless" as gmail. SES is literally Simple Email
| Service. Now do that without SES.
| ctippett wrote:
| | Now do that without SES.
|
| Why, so they can prove they know how to host an email server?
| That's not the point of the OP.
| hrez wrote:
| So they can say it's their service. Using somebody else's
| email service isn't their "serviceless". Of course there is a
| question of where you draw the line. But IMHO, using
| somebody's email service is that line here. Otherwise it's
| not far from using gmail or aws WorkMail for that matter.
| JackGardner wrote:
| This is really interesting! Currently trying to solve a similar
| problem thing myself, so having a look through to learn from your
| solution.
|
| I'd recommend having a look at AWS CDK to manage the AWS resource
| creation, is much nicer than raw CloudFormation:
| https://github.com/aws/aws-cdk/
|
| For my solution to this I've also then got a frontend ontop of my
| backend to setup view mailboxes, email list per mailbox, email
| viewer (read/unread status), and email sending. I'm doing the
| same with S3/SES/Lambda + DynamoDB for email
| sending/recieving/processing + email metadata in DynamoDB. Mine
| is currently WIP and needs some real auth (using a URL API key,
| not the most secure...) + email replying + tidying up. Will
| definitely open source my solution and post to HN when ready.
| jedahan wrote:
| I'd love to see a fork/branch that does the same as the linked
| repo, but using the cdk for comparison.
| JackGardner wrote:
| Yeah potentially I could extract the relevant CDK parts of my
| project out and combine with the code here to produce what
| you're asking for. I'll try and set aside some time and have
| a look.
|
| Thanks for the suggestion!
| joana035 wrote:
| One micro instance with Postfix is more than enough and can beat
| this setup not only in performance but in simplicity.
|
| What a year to be alive ...
| ungrateful-dead wrote:
| In cost? Running postfix.org mail sever on a VM must have a
| nonzero cost, with the additional need to maintain the
| underlying OS.
| caymanjim wrote:
| EC2 micro instances are free.
| beagle3 wrote:
| but are often blacklisted as SMTP sources, unfortunately.
| joana035 wrote:
| Not really, only if it the IP was used to delivery spam.
|
| But IP reputation can be established with a bit of warm
| up and human interaction.
| fxleach wrote:
| It's just a cloud server. Let's call it what it is. "Serverless
| email server"? Come on...
| hk1337 wrote:
| It's a nice POC but I wouldn't actually use it.
| mark242 wrote:
| When the documentation says "this may cost you money" what they
| really mean is "this will cost you a lot of money the moment your
| email address gets onto any spam lists". The number of lambda
| invocations will go through the roof, and even if you're able to
| catch 90% of the spam before dumping messages into S3, the cost
| of putting a message into a bucket will make your bill skyrocket
| periodically as spammers temporarily find their way past SES's
| built-in filters, which happens every so often.
| maerF0x0 wrote:
| depends how you define a lot of money.
|
| How much would this cost to receive 30k messages per month?
| mark242 wrote:
| Assuming a default message size of ~10kbytes the SES pricing
| is about $6 for the connections plus the chunks. Your lambda
| pricing is negligible and the S3 pricing gets you another
| dollar or so.
|
| But you're operating at the wrong scale-- 30k messages to a
| domain can happen in an hour by a spammer doing a directory
| attack. The point is that this solution is extremely open to
| abuse that happens all day every day over SMTP. Without a
| fixed-cost firewall to reject messages and only allow clean
| messages through to this solution, you are going to have
| escalating costs.
| megous wrote:
| That's a lot of money for a LKML subscription. :)
| lyjackal wrote:
| Previous discussion in January 2020
| https://github.com/0x4447/0x4447_product_s3_email
| awinder wrote:
| This is awesome & I totally intend to play with it :D
|
| Do you have any plans / any ideas on feasibility to plop an imap
| server on top of the s3 store? I have really wanted a serverless
| aws solution and this moves the needle on work I did not wanna do
| so I'm reinterested again. Something like an s3 object change ->
| dynamodb index might work well for latency concerns and keep the
| serverless spirit alive. I just have such a starting problem on
| "building/learning imap intricacies when it may never work".
| fooblat wrote:
| At that point why not use Amazon WorkMail[0]? Then you don't
| have to maintain anything at all.
|
| 0. https://aws.amazon.com/workmail/
| accelbred wrote:
| Workmail doesnt have wildcard addresses and you pay per
| address which is why I was looking at something like this.
| awinder wrote:
| Uh because I didn't know workmail came out, thank you kind
| human.
| fareesh wrote:
| I feel like the current methods for spam filtration are getting
| so many false positives that it makes email unnecessarily
| difficult for users with good intentions.
|
| I also don't know how much of this is unethical behaviour by the
| big email providers, forcing you to sign up for their cloud email
| solution instead of doing your own.
|
| It sure would be nice to be able to run one's own email server
| without having to stress over whether the recipient got your
| email.
| jahewson wrote:
| You can make it easy to send email or you can reduce spam but
| you can't do both.
| GuB-42 wrote:
| A serverless server?
|
| It is a great illustration of how the term "serverless" has
| shifted from literally no server (ex: sqlite database) to
| "somebody else's computer".
| redwall_hp wrote:
| My understanding of it has been "look, I reinvented cgi-bin."
| oblio wrote:
| Is cgi-bin highly available and will someone else have to
| wake up at night if it crashes?
| fooblat wrote:
| Yes and no, respectively.
|
| A cgi-bin is as highly available as the web infrastructure
| hosting it, which as the cgi-bin author you don't need to
| know anything about.
|
| A cgi-bin cannot "crash" because there is nothing long-
| running to crash. It is only run on demand.
|
| I think it is easy to make the case that CGI was the proto-
| serverless api and that the serverless offerings we have
| today are an evolution of the CGI approach.
|
| edit: typos
| mprovost wrote:
| And cgi-bin was really just an evolution of inetd (or
| ucspi-tcp if you worshipped at the altar of djb). You
| didn't used to have to worry about binding to ports or
| long running processes. Something would just send you a
| message on stdin and you replied on stdout.
| g8oz wrote:
| That depends on you and the SLAs you have in place with
| your cgi-bin provider.
| jamespo wrote:
| sure, cgi-bin with a big load balancer
| falcolas wrote:
| It can be, and not necessarily. `cgi-bin` can be as
| stateless as you want it to be. Just as stateless as
| lambdas, or other forms of "serverless" functions.
|
| And thus, it can be distributed as well as any other
| stateless function.
| vidarh wrote:
| I ran an e-mail service with 2m users. The web frontend was
| written in C++ (no, I don't want to go back, but we had a
| pretty nice custom framework) all served up as a CGI,
| behind a load balancer.
|
| If our code crashed, the next request would just get served
| as normal. If Apache crashed or a frontend crashed, another
| one would just take its place. High availability cgi-bin is
| trivial.
| yukinon wrote:
| I don't think there's anything wrong with reinventing
| something to be more accessible.
| hutrdvnj wrote:
| I guess a Heroku hosted application counts as serverless then.
| xtracto wrote:
| I was doing serverless pages back in late 90s when I uploaded
| PHP scripts to "website providers".
| themarkers wrote:
| true serverless
| dividedbyzero wrote:
| Why "true"? I don't have to worry about servers and
| infrastructure with Lambdas either.
| fxleach wrote:
| Now those are called "cloud" servers ( _gasps, awe_ )...
| adrianpike wrote:
| original lambda
| intergalplan wrote:
| "Your function runs from scratch on each request, with some
| intelligent caching for performance reasons."
|
| _Gestures at LAMP and PHP-FPM_
| benbristow wrote:
| Different thing though.
|
| When you uploaded your PHP script, you uploaded it to a
| single server, and it ran on a single server. Unless your
| provider were doing some kind of load balancing (doubt it).
|
| With serverless you upload your 'script' to the 'cloud' and
| then it will run on any number of servers, usually assigned
| to one when it dry-runs. Then it'll unload from the server
| afterwards. And therefore unless you have a bottleneck (like
| a database server) then you can essentially scale as far as
| the cloud will let you.
| innocenat wrote:
| Scaling: upload file to another server.
| asperous wrote:
| At least with Mediatemple's shared hosting, they had a
| "grid" service that would host your site on a cluster of
| servers and claimed to be able to handle large bursts of
| traffic.
|
| It worked because each page request ran a complete cgi php
| script to generate the page view and then terminated. There
| was no long running server.
| disgrunt wrote:
| It worked. But I remember Media Temple being slower than
| their competition. Seemed to have to pay a performance
| penalty for the "grid".
| throwaway667555 wrote:
| This argument is so tiring... if we all agreed to change the
| word "serverless" to idk "xirverous" it would have a pretty
| clear meaning and clear distinctions from "server" and "local
| to the machine". Xirverous technologies are an innovative, new,
| unique, and interrelated class of technologies. Thank you for
| your consideration.
| throwaway894345 wrote:
| > serverless server?
|
| This isn't a contradiction. "Server" is an overloaded term and
| the "server" in "serverless" refers to a different meaning of
| "server" than the "email server" bit. "Serverless" means there
| is no physical or virtual hosts to manage, you just supply the
| request handler i.e., the email _server_. It 's amusing because
| it _sounds like_ a contradiction, but it 's not one.
| mgkimsal wrote:
| "means there is no physical or virtual hosts to manage"
|
| We're just swapping one type of management for another,
| really. Becoming proficient in all the permutations of AWS
| stack, for example, to
|
| * make sure you don't get overbilled,
|
| * have appropriate memory/limit resources on your lambdas,
|
| * have correct backup/redundancy on s3,
|
| * make sure all your IAM policies are correct and not
| allowing malicious actors, etc.
|
| and more... it's ... just a different set of things to worry
| about. Yay - I don't have a 'server' that can go down. I now
| just have to make sure I don't get billed $87k because I
| forgot to click the correct button.
| kixiQu wrote:
| Billing exists for servers as well.
|
| Servers need to be configured as do lambdas.
|
| Servers also have backup needs.
|
| Servers also present security problems of access.
|
| It's not that hard to configure a lambda to run in just as
| limited a way as a physical machine.
|
| Not having to worry about MCEs or disk failure patching
| makes the concerns less like a "different set" and more
| like a _subset_ of things to worry about relative to
| managing servers.
|
| I know most of this stuff is really well trod-over, but
| from your comment I think one'd get the impression that
| people are switching just because of a trend or something,
| not because there's an actual layer of management they're
| paying to have outsourced.
|
| (I would acknowledge that view of it being a subset rather
| than a different set is invalid once you're debugging
| performance at a gritty level, where cold starts etc. etc.
| introduce their own equivalent layer of complication, but
| most people doing most things never need to)
| twistedpair wrote:
| Amen.
|
| How many times did you update your Node.js AWS Lambdas or
| GCP Cloud Functions because of the Linux kernel CVE of
| the week? You didn't because all you're responsible for
| is your few lines of Node.js logic that kept on scaling
| and humming along. The cloud vendor cares for the rest.
| vinay_ys wrote:
| Just because you didn't doesn't mean they did and you
| didn't actually have any vulnerabilities. There is no
| such provable attestation of security in serverless
| model.
| throwaway894345 wrote:
| > Lambda provides support for these runtimes by
| continuously scanning for and deploying compatible
| updates and security patches, and by performing other
| runtime maintenance activity.
|
| https://docs.aws.amazon.com/whitepapers/latest/security-
| over...
|
| I'm not sure if that addresses your concern (maybe you're
| worried they're lying or they have a bug in their
| process)?
| stonesweep wrote:
| I disagree with your assessment, email is a _service_ and not
| a _server_. A service can run on a server, or ephemerally as
| in this article (which still has servers underneath it, you
| 're just not running them). S3 and SES are services running
| on servers.
| throwaway894345 wrote:
| > email is a service and not a server.
|
| Those aren't mutually exclusive, which is my whole point.
| One definition of "server" is "a service in a network".
| Another definition is "a computer in a network". So you can
| have a server running on a server without contradiction. If
| the user isn't responsible for managing the underlying
| computer, then you have a serverless server.
|
| > which still has servers underneath it, you're just not
| running them
|
| Of course, "serverless" means you don't have to worry about
| the underlying servers, not that they don't exist. Perhaps
| you were only clarifying and not nerd-sniping, but this
| particular nit is so boring and predictable in every
| serverless thread.
| stonesweep wrote:
| > One definition of "server" is "a service in a network"
|
| Can you link to something where this is a common accepted
| definition? I'm a systems engineer by trade, talk to a
| bajillion people about all sort of things and we just
| don't call a "service in a network" a server in parlance.
|
| I was not nerd-sniping and could care less about
| serverless as a term, I'm specifically talking about
| calling a "service" a "server" in this chat. I feel this
| is presenting something as accepted definition which does
| not match my experience in the field.
| throwaway894345 wrote:
| > Can you link to something where this is a common
| accepted definition? I'm a systems engineer by trade,
| talk to a bajillion people about all sort of things and
| we just don't call a "service in a network" a server in
| parlance.
|
| This seems to be a common point of confusion for people
| who aren't native English speakers. Oxford English
| Dictionary defines "server" as:
|
| > a _computer or computer program_ which manages access
| to a centralized resource or service in a network.
|
| Similarly, a quick bit of Googling turned up [this][1]
| which isn't authoritative, but indicates that "server"
| can mean either hardware, VM, or software services.
|
| [1]: https://stackoverflow.com/a/13118205/483347
| stonesweep wrote:
| > This seems to be a common point of confusion for people
| who aren't native English speakers
|
| Where did this come from and what value does it have,
| other than being condescending to non-native English
| speakers? I am a native speaker and we're having a
| discussion in my native tongue about words in my native
| language about work I do as a profession.
|
| > Similarly, a quick bit of Googling turned up [this][1]
|
| I do not accept Stackoverflow as an authoritative source
| for anything. Useful? Yes, great for finding random
| solutions to random problems. Authoritative source on
| terminology used in the industry I work? Nah.
| reaperducer wrote:
| _Where did this come from and what value does it have,
| other than being condescending to non-native English
| speakers?_
|
| I don't think he was being condescending, so there's no
| need to be offended on behalf of other people.
|
| I think he was suggesting that part of the linguistic
| confusion comes from how the tech industry has become so
| global that different words and phrases are exchanged
| between cultures, but within the tech sphere.
|
| For example, it's not uncommon to hear the phrase "Do the
| needful" in places like Seattle, even though the phrase
| originated elsewhere and was imported by tech workers.
|
| _I do not accept Stackoverflow as an authoritative
| source for anything._
|
| Good call. I'm with you there.
| throwaway894345 wrote:
| > Where did this come from and what value does it have,
| other than being condescending to non-native English
| speakers?
|
| What is condescending? I'm observing that it's a common
| problem among non-native English speakers. It seems like
| you're taking offense on behalf of others and unduly so.
|
| > I am a native speaker and we're having a discussion in
| my native tongue about words in my native language about
| work I do as a profession.
|
| This is a common idiom among English-speaking IT
| professionals. If you're not familiar, that's fine. Now
| you know.
|
| > I do not accept Stackoverflow as an authoritative
| source for anything
|
| In a minute of Googling, I found several random sources
| on the Internet that indicate that the term is overloaded
| precisely as I described. One of those sources was the
| Oxford English Dictionary. I think that suffices to
| demonstrate that this is a common idiom, but I can't
| force you to be persuaded. -\\_(tsu)_/-
| Liquid_Fire wrote:
| I'm surprised you need a source for this. Literally half
| of the server software out there refers to itself as
| "<thing> server".
|
| Apache [1] "The Number One HTTP Server On The Internet".
| This is not referring to the machine hosting, it's
| referring to the software that you run to provide a
| service.
|
| Postfix [2] "mail server"
|
| IIS [3] "Web server"
|
| I could go on...
|
| [1] https://httpd.apache.org/
|
| [2] http://www.postfix.org/
|
| [3] https://www.iis.net/
| imwillofficial wrote:
| You are incorrect, see man pages.
| stonesweep wrote:
| This makes no sense (too abstract of a statement), but
| here let's try this: $ man systemd |
| head -4 | tail -1 systemd, init - systemd
| system and service manager
|
| An email daemon (Postfix, Exim, Dovecot, UW-IMAP,
| Sendmail, etc.) are services running on a server. An
| "email server" would be a server running a daemon to
| provide email service in this example.
| imwillofficial wrote:
| Makes plenty of sense: https://man7.org/linux/man-
| pages/man1/smtp.1.html
|
| "SMTP transaction against an SMTP server"
| ape4 wrote:
| This makes me think there should be a Linux tool to
| extract the Nth line of a file. With options to get the
| N-Mth lines, etc
| ipaddr wrote:
| There are many. Sed is fast. This will give you lines 11
| to 500
|
| sed -n '11,500p' < file.txt
| stonesweep wrote:
| Man pages can be tricky as the content can vary distro by
| distro (how up to date is your copy of man-pages, what
| changes have happened, etc.) - this technique is pretty
| portable to extract the synopsis of a man page
| specifically due to the format it uses (most times I'd
| use like grep -A3 -B2 foo /some/file, e.g.).
| ape4 wrote:
| Ugly way to get the first indented line: `man systemd |
| grep '^ ' | head -1`
|
| Which is the summary
| mprovost wrote:
| Calling cut with a newline as the field delimiter works:
|
| man systemd | cut -d $'\n' -f 4
|
| Works with multiple lines too:
|
| cut -d $'\n' -f 10-20,50-60
| xdennis wrote:
| > "Serverless" means there is no physical or virtual hosts to
| manage
|
| While true, that's not the defining characteristic of
| "Serverless" because PaaS also does not require server
| management.
| megous wrote:
| It's indistinguishable from a regular web hosting. You just
| upload script to some ftp and don't manage anything either.
| You just configure links to other services like DNS, or DB if
| that's needed.
|
| It's just now the same thing is done via some
| proprietary/custom cli tools and/or APIs not via a standard
| protocol and a trusty old clients that are already in your
| distro, and it's branded differently.
|
| APIs are good for some uses. That's usually lacking with
| hosting providers. Other than auto-scaling I can also easily
| bankrupt myself with if I make a mistake, I don't see much
| qualitative difference. Only in the details.
| twistedpair wrote:
| Cloud 2.0 "serverless" often expects:
|
| - Just add code/logic
|
| - Scales to zero (zero calls to your endpoints, $0/mo)
|
| - Scales up automagically to match load
|
| - You don't do anything to provision/patch servers (e.g.
| FaaS)
|
| Classic PHP shared hosts don't meet this definition because:
|
| - You paid a flat monthly hosting fee (e.g. Dreamhost)
|
| - You only scaled to 1 node (need more, too bad)
|
| True, you didn't usually need to mess with PHP.ini or Apache
| server installs and configs, but you still didn't get the
| same benefits as modern serverless.
| throwaway894345 wrote:
| I'm a little confused about why you're responding to me. I
| think I agree with everything you're saying, but I'm not
| sure where the comparison with PHP hosting is coming from.
| Are you sure you responded to the right thread?
| antihero wrote:
| I wonder if you could, for reasons, make a set-up where you
| upload your PHP files to an S3 bucket, and then have a API
| Gateway or Lambda which basically just serves the files
| from the bucket, and if they are PHP, executes them with a
| Lambda PHP runtime.
|
| So you hit /index.php -> checks S3 bucket for index.php ->
| executes that with lambda and gets string of response ->
| returns it to original lambda -> serves up to client.
| darkwater wrote:
| > This isn't a contradiction.
|
| Yes it is. The "server" in "serverless" means no need to low-
| level manage/provision a server (computer) and its lifecycle.
| While "server" in "A serverless email server" means actually
| "service" as in "A serverless email service on AWS using S3
| and SES". Which makes much more sense. The linked repo lets
| you have an "email server" just like GSuite gives you a
| "serverless email server".
| throwaway894345 wrote:
| I don't understand. You insist it's a contradiction but
| then go on to reiterate my own argument which demonstrates
| that it is not, in fact, a contradiction but rather using
| the same term ("server") to refer to two distinct concepts
| ("computer" vs "service")?
| cortesoft wrote:
| So if you get an omelette at a restaurant, I can call it an
| eggless omelette because I didn't have to cook the eggs
| myself?
| unethical_ban wrote:
| If "egg" was an overloaded term that meant the noun "egg"
| and the noun "process of cooking an egg for an omelette",
| then yes, you could.
| throwaway894345 wrote:
| This would make a lot of sense if the de-facto restaurant
| experience involved you cooking your own eggs, especially
| if the egg-cooking experience was notoriously laborious
| _and expensive_.
|
| If we swap "egg" and "cooking" with "VM" (or "bare metal"
| if it suits you better) and "maintaining", suddenly
| "serverless" doesn't sound so ridiculous.
| thehappypm wrote:
| "Serverless" is a ridiculous stupid term. You'd think it meant
| something like Photoshop, some system that doesn't rely a
| server, something that lives locally. Nope. It's just a server
| that someone else manages 90% of.
| specialp wrote:
| No we call those a more succinct term: local processes. We
| call "serverless" for processes running not locally that do
| not require standing up a server that you care about for, are
| billed by usage only ($0 for none, infinity for infinite),
| and you are not concerned where or how it is running. Are
| there computers listening for requests somewhere that someone
| is managing that is doing this? Yes. But from my frame of
| reference it is a service, not a server. I know a lot of
| people on here get hung up on it, but it is the term and it
| is not going to go away.
| barbazoo wrote:
| And a million other moving parts:
| https://raw.githubusercontent.com/0x4447/0x4447-product-s3-e...
| bambam24 wrote:
| Google serverless to start learning about it
| ionwake wrote:
| I literally had one of the heads of a crucial flight system
| project loudly exclaim to me in front of the senior managers.
| "It's serverless - there is no server!"
|
| To which I honestly, just kept my mouth shut and stood aside.
| navaati wrote:
| Maybe what he meant is "there is no server _that we are
| responsible for_ ", which is where the benefit lie for them.
|
| (not commenting on the "serverless" name for what used to be
| called "mutualised services"...)
| dec0dedab0de wrote:
| _To which I honestly, just kept my mouth shut and stood
| aside._
|
| I can never keep my mouth shut in these situations. Sometimes
| it leads to promotions, but I suspect it puts a ceiling on
| how high I can climb the corporate ladder.
| ncphil wrote:
| Spent the first 10 years of my IT career driving towards
| greater control over the systems we managed, because "the
| reason the service is down is that someone else (usually a
| vendor) didn't do their job" was never an excuse. The last
| 5-10 years I've found myself ceding more and more control
| to cloud service vendors (like Microsoft). Less work for me
| ("raise a premium case"). Less stress too ("they're working
| on it, nothing for us to do"). Now I finding myself hoping
| I'll be retired before the pendulum swings back again.
| ionwake wrote:
| I just figured there was some "short term"
| misunderstanding. As always I looked like an idiot but hey
| in a good place with good people youll all just move onto
| the next topic and know the details will work themselves
| out regardless who is right.
| EGreg wrote:
| Serverless has become a stupid marketing term, just like
| Facebook hijacked the words "Friend" and "Like".
|
| I look forward to true serverless software - that is either
| peer to peer or uses a distributed back-end with encryption.
| Now THAT is the future!
|
| Serverless just means "we spin up a server for you", similarly
| to how "the cloud" is just a euphemism for "extreme
| centralization of hosting myriad clients under the control of
| one company".
| smolder wrote:
| It's more than that, slightly. A "serverless" application
| doesn't have a specific piece of hardware supporting it
| except in the context of a single operation. The hardware is
| all abstracted away and allocated from a pool on demand. I
| suppose it comes from "serverless architecture", where there
| are just other distributed services (not specific servers,
| virtual or otherwise) executing your code.
|
| I agree it's not a term well suited to its popular use, but
| such is life. Commercial interests are constantly mucking up
| the language to suit their own ends. E.g. Discord is abusing
| the term "server" to mean a virtual meeting space. "Organic"
| w.r.t. food has a really contrived official criteria, unlike
| in chemistry. Regular people muck up the language in
| frustrating ways, too. See "literally".
| reaperducer wrote:
| _Serverless just means "we spin up a server for you"_
|
| Thank you for "spin up a server," as in fire up the hard
| drives, rather than "stand up a server," as in... I dunno.
| libria wrote:
| We went through this when people first started calling
| quadcopters "drones".
|
| It's just a term used to describe a server configuration that
| has different billing/operational characteristics than
| traditional always-on.
|
| And like the quadcopter-drone, I hope everyone losing their
| shit over this can either accept "serverless" or submit and
| market a new term that conveys this difference soon.
| diveanon wrote:
| This comment shows up on every thread about lambdas, and
| despite the FUD it still proves its value to me every single
| day.
| lhoff wrote:
| Something link that could be useful as disaster recovery action.
| I case of problems with the Mail infrastructure redirect the DNS
| to something like that and incoming e-mails won't get lost in the
| meantime. The standby cost for something like that is much lower
| then a independent secondary e-mail system.
| megous wrote:
| Why redirect anything? You can have backup MX records. This is
| already a solved problem. And backup system can be quite
| lightweight, if it only needs to store emails for the time
| primary is down and then push them to primary once it's up
| again.
| sequoia wrote:
| Totally cool project! I can't quite figure out where in the repo
| the code is, or maybe this repo just contains the CF configs? If
| so a link to the lambdas etc. would be useful.
| habibur wrote:
| Reading it I get that AWS now by default restricts email by
| - 200 mails per day max. 1 per second. - Can only send
| mail to verified addresses. Not sure how they verify though.
| - Can only send mail from the domain you own and verified.
|
| All these are good steps. Guess that will gradually take AWS IPs
| out of all those mail blacklists.
|
| And while we are on the topic, I find IP based mail blacklist
| services such as BRBL obsolete and possibly harmful. Now that we
| have domain verification services like DKIM SPF etc. Blacklist
| the domain. Not ip addresses as most institutions don't own their
| IPs like in the past. Most of all are hosted, and that IP might
| get assigned to someone else a few days later.
| nickjj wrote:
| > 200 mails per day max. 1 per second
|
| They've been limiting max sends per day for a long time. I set
| up an account around 4-5 years ago and they had a limit back
| then. I forgot the exact amount but it was 200 or less.
|
| But, they do make it pretty easy to get the limit raised. I
| filled out a form and about a day later they bumped my limit to
| 50,000 per day and I think I only requested 10k per day. The
| form required basic information like providing a reason why I
| wanted a higher limit and a link to my main site where I was
| collecting emails from. It seemed to be human reviewed.
| donmcronald wrote:
| They must just give 50k as the baseline. I asked for the
| minimum, which I think was 200, and they gave me 50k. I was
| surprised because my plan for handling bounced email was "by
| hand." Lol.
| dnag wrote:
| Here's an illustrated guide to getting out of SES Sandbox:
| https://docs.sendwithses.com/how/send-email/how-to-increase-...
| whoknew1122 wrote:
| RE: verification
|
| AWS sends an email to the destination address and ask them to
| click a link to receive emails from SES.
|
| There's no way for the email sender to verify an address. The
| email recipient has to click on the link in the email SES sends
| them.
| rhooke wrote:
| These restrictions are only for the SES sandbox mode. You fill
| in a form to get out of sandbox mode and then you can send many
| more per day (still only 14/second or so).
|
| You only verify addresses you want to _send from_ (for testing
| you might verify one to try sending _to_ as per the sandbox
| rules). You wouldn 't get customers to verify their addresses
| here, as that would let you send email on their behalf.
|
| See here:
| https://docs.aws.amazon.com/ses/latest/DeveloperGuide/reques...
| bmhin wrote:
| All very good info that as far as I know is all pretty unique
| to SES regarding the default/sandbox vs the upgraded
| production capabilities. The set up makes a lot of sense to
| prevent abuse and it works just fine out of the box for dev
| purposes. The production version also operates way more how
| you'd actually expect an email service to work.
|
| All that said, I don't know of any other services that have
| this "production" upgrade that is needed other than maybe
| just the general service limits (which have always seemed
| more of a "protect you from yourself" situation) so it can be
| a touch confusing.
| acdha wrote:
| Sandbox mode is also great for low-volume internal mail
| like notifications from cron.
|
| My guess is that the main argument for sandbox mode is that
| it requires you to explicitly say you don't spam and will
| handle reports, giving them carte blanche to yank your
| account if you do. Given how slimy email marketing is I
| suspect someone at AWS didn't want to deal with people
| lying to their support people all of the time.
| throwaway823882 wrote:
| And once you are out of sandbox mode, it scales pretty
| well. We regularly send millions of mails in 24hr
| periods. Some gotchas come in when you need to set up
| DKIM/etc and other providers may need to handle mail for
| the same domain.
| ydant wrote:
| Verifying emails just involves SES sending an opt-in /
| verification link to the recipient email address in question.
| [deleted]
| pirsquare wrote:
| "I find IP based mail blacklist services such as BRBL obsolete
| and possibly harmful"
|
| Fully agree to this. I've always wonder why spam filters don't
| place more importance on domain reputation. So many senders
| runs on the same IP for deliveries and it's really silly to
| blatantly block someone based on IP.
|
| For example, $89.95/mo is the starting price to get a dedicated
| IP for sending with Sendgrid.
| dec0dedab0de wrote:
| If the smtp server is allowing spam to be sent through it,
| then it is not trustworthy. It doesn't matter which domain it
| was sending on behalf of. Though the real reason is that
| we've only had spf records and what not for a relatively
| short amount of time, so most systems are designed from the
| perspective that you can't trust the domain a message is
| claiming to be from.
| tryauuum wrote:
| It is hard to fool an IP-address-based blocking system.
| Because it's easy to fake a domain name, but hard to fake an
| IP-address
| cutthegrass2 wrote:
| SES is pretty good at monitoring your email usage, such that it
| maintains a "reputation" score calculated by monitoring the
| hard bounce rate generated by your account. The premise being
| that high bounce rates (AWS define this as >5%) strongly
| indicate you're not managing your email recipient list properly
| i.e. sending untargeted SPAM.
| manishsharan wrote:
| >> I find IP based mail server blacklist services such as BRBL
| as obsolete and possibly harmful. Now that we have domain
| verification services like DKIM SPF etc. Blacklist the domain.
| Not the ip
|
| Agree with you 100%. I want to run a mail server on my AWS
| server and not use Yahoo so that Verizon doesnt snoop on my
| mail and insert creepy ads that seem like emails. But this is
| impossible as my mail would get dropped into a blackhole.
| acdha wrote:
| > Guess that will gradually take AWS IPs out of all those mail
| blacklists.
|
| Are you thinking of EC2? SES has never had a problem with this
| in my experience.
|
| Regarding verification, you can verify any email address where
| you can receive a link long enough to click on it. Domain
| validation is only needed if you want to send mail from
| arbitrary addresses without that step.
| dfsegoat wrote:
| Individual SES IP's have been [temporarily] blacklisted
| before. We hit this issue earlier this year with one spam
| filter/service, and you can find plenty of past examples in
| the SES forums.
|
| e.g.
|
| https://forums.aws.amazon.com/thread.jspa?threadID=233001
|
| https://forums.aws.amazon.com/thread.jspa?threadID=323992
| acdha wrote:
| Sorry, I should have said significant problems. If you send
| email on anything like a large-scale basis you'll get on
| some kind of blacklist at some time - all you need is one
| user sending email to someone with a twitchy spam reporting
| reflex. What I was thinking about, though, is that it's
| usually transient - e.g. out of hundreds of thousands of
| emails I've never had one bounce that way instead of being
| delayed, which is better then self-hosting on a long-time
| class B over a similar timeframe.
| mark242 wrote:
| > I find IP based mail blacklist services such as BRBL obsolete
| and possibly harmful. Now that we have domain verification
| services like DKIM SPF etc. Blacklist the domain.
|
| SPF and DKIM require opt-in by the sending domain, of which the
| majority of non-commercial non-US outgoing MTAs do not do. IP-
| based blocklists, including Spamhaus PSBL Mailspike etc, are
| all valuable to catch IP addresses that should not be directly
| connecting to your MTA. I agree that signing up for an outgoing
| MTA service such as Sendgrid makes some IP-based checks
| obsolete but that's not where the majority of spam is coming
| from. Botnets continue to be the number one source of junk
| email and will likely continue as we add more and more insecure
| doorbells, garage openers, refrigerators, etc, to our home
| networks.
| OJFord wrote:
| > Botnets continue to be the number one source of junk email
| and will likely continue as we add more and more insecure
| doorbells, garage openers, refrigerators, etc, to our home
| networks.
|
| Surely that's an argument _against_ blocking IPs?
|
| Unless I suppose you argue they're predominantly in homes;
| and homes are predominantly not running (intentional, well-
| behaved) email servers, so sorry-not-sorry those who are.
| mark242 wrote:
| Yes. The days of running a legitimate MTA at home are
| unfortunately over. Many US-based consumer ISPs block port
| 25 entirely except for connections to their own MTAs, but
| for the ones who don't, very very many of those IP
| addresses wind up on policy-based blacklists -- eg "this
| network is a bunch of consumer addresses and should never
| be connecting directly to an MTA"
| megous wrote:
| Running from home IP sucks for other reasons too, like
| non-static IP addresses, or lack of public IPv4/use of
| CGNAT.
|
| I certainly would not risk my emails being delivered to
| someone else's computer just because IP address changed,
| and DNS still points to the previous one.
|
| But having the server and data at home is still possible
| via VPN.
| ttul wrote:
| IPs are still a precious commodity, hence the threat of being
| listed by a high quality blocklist like Spamhaus provides an
| effective degree of encouragement for senders to clean up their
| act.
|
| Domain blocklists also exist, but the stick isn't as sharp,
| because domains are a dime a dozen.
| diveanon wrote:
| IPv6 has entered the chat.
| sethammons wrote:
| Meh. Something like 95% of mail servers wont work with IPv6
| last I checked. Source: I helped build and maintain an MTA
| that moved over a trillion emails last year.
| sneak wrote:
| Note that using the + address "trick" in email addresses is
| silly: everyone knows it and can strip it out with a simple
| regex. It doesn't accomplish the main things that it exists to
| do, which are to a) make it non-obvious what your username for a
| site is and b) make it obvious where a spammer got your email.
| caymanjim wrote:
| The main thing it exists to do is to route email to folders or
| special handlers. It has nothing to do with spam or
| obfuscation, and predates the concept of spam.
| bambam24 wrote:
| By default, you can't send emails to unverified addresses. If
| you'd like to be able to send (as opposed to just receiving),
| you'll need to reach out to AWS to remove this limitation from
| your account.
| Exuma wrote:
| All the 'client' images on your site are 404.
| throwaway894345 wrote:
| I was trying to set up Amazon Cognito (an authentication service)
| recently which basically needs a _verified_ email address to send
| users emails for user verification and password reset, and I was
| shocked that Amazon has no basic email hosting service. Of course
| it's easy enough to work around for my use case, but I was
| surprised nonetheless.
| mvanbaak wrote:
| They have workmail.
| throwaway894345 wrote:
| I saw that, but it was like $5/user/month and geared toward
| employee email rather than automation (my use case is my
| hobby projects for which my goal is <$5/month in total).
| stevehawk wrote:
| What's extra annoying is that they need a verified email
| address and can't use a verified domain via SES.
| andrew_ wrote:
| I'd love to see the cost difference between this solution and
| running an EC2 instance with a traditional lite mail server (like
| perhaps courier).
| wefarrell wrote:
| Email from EC2 instances will be blocked because of the high
| amount of spam that comes from those IPs.
| andrew_ wrote:
| Interesting, thanks for sharing. Not sure why folks downvoted
| the comment, it being a plea for more information.
| joana035 wrote:
| Yeah, you have to request amazon to remove the throttling
| from your network.
|
| It took few year for aws to document that they have this
| limitation on their network.
|
| https://aws.amazon.com/premiumsupport/knowledge-
| center/ec2-p...
| r2d2klapa wrote:
| This is great. I tried a similar approach but couldn't use it as
| a transactional email server because of this (taken from OP's
| readme):
|
| > By default, you can't send emails to unverified addresses. If
| you'd like to be able to send (as opposed to just receiving),
| you'll need to reach out to AWS to remove this limitation from
| your account.
|
| I reached out AWS to remove this limitation but they refused my
| request 3 times claiming that my use case would impact the
| deliverability of their service.
| ttul wrote:
| AWS is not wrong. The instant that you stand up a service that
| allows anyone to send email, spammers will take advantage of it
| and spam the world. This ruins your reputation and you get
| blocked everywhere.
| greatgib wrote:
| I'm a little bit disappointed by the solution of this article.
|
| They start with: "This stack was created out of frustration due
| to the fact that to this day there's no easy way to have a full
| email server without the overhead of installing and configuring
| all servers needed to handle incoming and outgoing messages."
|
| Ok, so the point is that it is too complicated/frustrating to
| deploy an email server.
|
| But look at their solution in "what will deploy"... in the graph
| there are over 40 entities! In the following they list 13
| different aws components needed.
|
| And all of that not even really doing the core feature of email
| sending/receiving that is provided by SES.
|
| Maybe you can deploy it with a single cloudformation recipe, but
| then it is so so many systems that you will have to watch and
| check for issues, quota, policies, usage billing and
| configuration.
|
| And you are not even in control, you have to go lick aws ass for
| easing the restrictions on the number of emails sent and to who.
| Plus there are probably a number of drawbacks and hidden limits
| that are not exposed to you by aws and that you will discover the
| hard way.
|
| For exemple, looking at the description I'm wondering, how do you
| do coherent backups at a random time? Knowing the quality of
| service of aws api, what do you do when there are random crash,
| network errors and co in the middle of the pipelines/lambdas?
|
| In my opinion this is a good exemple of an over-engineered
| solution.
| dahfizz wrote:
| There are a class of engineers who have very little Linux
| experience, but use AWS extensively. For them, "Setting up a
| server" is always going to be more complex and frustrating than
| using the platform that they are familiar with, even when the
| AWS setup they come up with to replace a single server is way
| more complex to an "outsider".
|
| Personally, I don't think there is an excuse to not learn
| Linux. It's usually the right tool for the job, and you can use
| tooling like ansible and AWS instances to get your stack cloud
| based, stateless, automated, yada yada buzzword . But I can't
| really blame people for using the tools they are comfortable
| with.
| antihero wrote:
| Frankly I don't understand why they didn't use this using
| CDK.
| gazarullz wrote:
| When all you have is a hammer ...
| init wrote:
| Cool project! I designed a similar system many years ago with SES
| and the AppEngine data store instead of S3.
| kureikain wrote:
| This is great but it's a bit misleading.
|
| It _only_ store email on S3, that mean you cannot read email with
| your email client.
|
| Seem thing with outgoing, you upload email to s3 in a certain
| format to be send out, not smtp protocol
|
| This is more like an API for sending and receiving. You cannot
| say generate SMTP credential and drop in your mail client.
|
| Also, one of the limitation with AWS SES is you cannot sent FROM
| or RETURN-PATH to anything, even in production mode, you can only
| set frm or return-path to verified domain.
|
| I have an internal AWS SES/Lambda stacks that I used for email
| forwarding(only receiving) which use SES to accept email, trigger
| a lambda and use SES to send a copy to a personal inbox. Plan to
| cleanup it a bit and release
| femto113 wrote:
| You could probably cobble together a POP server on top of S3
| pretty easily (e.g. using something like
| https://github.com/femto113/node-pop3-emitter which I threw
| together a couple years ago for a related purpose). I'm not
| sure if you can get it to work in Lambda though because the
| POP3 protocol needs sockets, but you could definitely get it to
| work in a simple container in Fargate (so still "serverless").
| jpalomaki wrote:
| Back in the days I remember storing emails in similar way and
| reading using an email client. It was called "maildir" [1].
|
| Emails were stored as files in folder structure. I think this
| goes quite nicely with the unix ideas as you could use many
| standard tools to work with the messages.
|
| [1] https://en.m.wikipedia.org/wiki/Maildir and
| OJFord wrote:
| I do something similar to OP [0] - and currently forward to
| another provider (single address, but that's only the
| 'envelope' To; so it doesn't matter, I preserve the original
| whatever.to.address).
|
| That's mostly an artefact of transitioning from that provider
| to my own on AWS, but I stopped working on it for some time and
| never continued. My plan was/still is to write a client that
| can read 'Just a Bunch of .Eml' format, and then I can point
| that at S3 (or via S3FS or whatever). Everything that exists
| for mailbox, maildir, and what-have-you assumes you need it to
| act as an MTA (I think that's right) and do the actual
| receiving too; and is difficult/impossible to stop that and say
| 'no, here's a directory full of email, just be a reader'.
|
| Longer-term plan/idle thought is to have a JMAP server on
| Lambda. (IMAP isn't possible, because it has to look like/be
| HTTP to reach the Lambda.)
|
| [0] - https://github.com/OJFord/amail
| kureikain wrote:
| Yes, I used something like that as well.
|
| But, are you sure you can set "From" header to anything(even
| for email/domain that aren't verified with SES). So that on
| receiver side(gmail for example) it will show the original
| sender.
|
| In my account(in production mode, not being sandbox), I
| couldn't. When I attempted to set "From" header(or "Return-
| Path") I got these error 554 Message
| rejected: Email address is not verified. The following
| identities failed the check in region US-WEST-2: can-we-
| overwrite-this@my-domain.com, Test random
| <raindom@gmail.com>...
|
| On their example doc, using SMTP credential, they also
| mention this: https://docs.aws.amazon.com/ses/latest/Develope
| rGuide/exampl...
| OJFord wrote:
| No, you can't set it to anything, you'll fail
| DKIM/DMARC/SPF. Just leave it as it is!
|
| You just wrap the original email in an 'envelope' with
| envelope-to (you @ other provider) and envelope-from
| (whatever @ verified with SES).
|
| You can see this here, if it helps at all, though it's a
| bit DSL-y: https://github.com/OJFord/amail/blob/c602f4d36c1
| bb9df4da1744...
| kureikain wrote:
| By envelope-from (whatever @ verified with SES), do you
| mean the `MAIL FROM` command in SMTP protocol or do you
| mean the `header FROM` in the email header?
|
| I don't know how `send_raw` work under the hood in Rust,
| but without a way to set `from` header. how do
| recipient(say an @gmail.com) address show the original
| from too?
|
| Did I miss something here? It would be great if somehow
| the email is forward as it's in so it appear in my inbox
| and all the information(header) is retain.
|
| > just wrap the original email in an 'envelope'
|
| It would be great if you can help clear my mind on how to
| do that. Say I got an email as this:
| FROM: original-from DKIM: dkim sign Return-
| Path: etc Other header: Body of email
| appear here
|
| How can I use AWS SES to forward that email as-is to my
| @gmail.com account for example while still keep in the
| original header intact(so DKIM/DMARC still work. SPF will
| be break but it's fine for me)
| kureikain wrote:
| I download and run your Rust code. And apparently it's
| same as others. You _cannot set_ the FROM header to any
| value, therefore cannot forward the email as it 's by
| retaining the original header. thread
| 'main' panicked at 'Failed to send test:
| Permanent(Response { code: Code { severity:
| PermanentNegativeCompletion, category: MailSystem,
| detail: Four }, message: ["Message rejected: Email
| address is not verified. The following identities failed
| the check in region US-WEST-2: random@gmail.com"] })',
| src/main.rs:41:10
|
| note: run with `RUST_BACKTRACE=1` environment variable to
| display a backtrace
| hnlmorg wrote:
| In fairness, you could write support for those in lambda too if
| you wanted.
|
| As a proof of concept though, this is interesting.
| StreamBright wrote:
| I would argue that an email client that read email from s3
| instead instead of a mailbox is preferable.
| alfiedotwtf wrote:
| > This stack was created out of frustration due to the fact that
| to this day there's no easy way to have a full email server
| without the overhead of installing and configuring all servers
| needed to handle incoming and outgoing messages.
|
| Looking at diagram, I'm not sure Postfix + Cyrus is more
| complicated than what you've got here.
|
| Congrats though, I'm sure this will help a lot of people wanting
| to manage mail themselves. Just not sure how you're going to get
| mail accepted by others from constantly changing (and possibly
| blacklisted) Amazon IPs without using trusted smart hosts.
| goodpoint wrote:
| You spotted the issue: maintaining a mailer requires constant
| monitoring of blocklists and contacting the list owners to be
| unblocked.
|
| It required a fixed ipaddr and takes months to gain trust.
|
| Using random AWS IP addresses is a lost cause.
| xenophonf wrote:
| > _Just not sure how you 're going to get mail accepted by
| others from constantly changing (and possibly blacklisted)
| Amazon IPs without using trusted smart hosts._
|
| Are you thinking about EC2 public IP addresses here? That
| service uses a different IP range than SES, which you can
| verify by comparing https://ip-ranges.amazonaws.com/ip-
| ranges.json to `dig amazonses.com TXT`. In my experience, we
| haven't had any issues with SES emails getting blocked, but you
| must set up DMARC properly and respond to bounces quickly.
| ignoramous wrote:
| We are building a similar serverless no-code inbox (with SES and
| S3) not for outgoing emails, but to only receive them [0].
|
| Our service doesn't require a user to sign-up with any existing
| identity (like email or phone number), and so, when they make
| payments, we send the invoices from the payments-processor to an
| email (corresponding to the customer-id) on our subdomain that
| goes to SES which promptly plonks it into an S3 bucket set up to
| life-cycle it out after 60 days, encrypting it with keys
| associated with customer-id. This encryption serves us well when
| the customer chooses to rotate / delete their pseudo-anonymous
| identity (and the keys along with it), as then all data in our
| systems associated with their id (the archived emails, for
| example) is essentially tombstoned.
|
| The reason we prefer serverless over other solutions is not
| because of its scale or low-cost, but high-availability.
|
| [0]
| https://docs.aws.amazon.com/ses/latest/DeveloperGuide/receiv...
| praveen9920 wrote:
| Great initiative but I see one major concern: Spam filtering.
| Most of the big mail providers provide spam mail filtering out of
| the box without which your inboxes are bombarded with spam. This
| can be probably solved by spam checking by adding additional
| lambda, now you need more compute per mail, translating to more
| cost per mail, which makes it easy for malicious parties to send
| mail bombs and bankrupt you.
| verst wrote:
| The actual email sending / receiving part is handled by Amazon
| SES here. Similarly you could use Sendgrid in that place. This
| project is a wrapper around those. It also doesn't aim to provide
| an email inbox - this is for programmatic handling of email.
|
| Now the interesting part to me is using the email address local
| part to create an email address hierarchy upon receipt.
|
| This project stores incoming emails to S3 and the '+' in the
| email address indicates subfolders from left to right.
|
| That's a very creative way to organize emails for processing.
| tonfreed wrote:
| I wrote a catch-all lambda for SES too forward anything I sent
| it. That was 5 hours of my life I'm never getting back
| lisper wrote:
| This is a very misleading title. It is not serverless. It uses
| SES which stands for Simple Email Service, which is provided by
| (of course) servers. It's just that they are servers that Amazon
| maintains for you. It is no more "serverless" than any other
| hosted email provider.
|
| This is not to say that this is without value. Having your emails
| end up in an S3 bucket is tremendously handy. But it's not
| serverless.
| Diederich wrote:
| The word 'serverless' is certainly a bit misleading.
|
| By your definition, is there any possible architecture that is
| truly 'serverless'?
| lisper wrote:
| That depends on what you mean. If you mean, "Is it possible
| to have SMTP without a server?" the answer is no, by
| definition. SMTP is a client-server protocol. The same is
| true of IMAP and POP.
|
| If you mean, "Is it possible to have something that has an
| email-like UI/UX without a server" the answer is yes. But it
| won't interoperate with what we call email today, because
| that is fundamentally based on SMTP.
|
| In fact, the original email was serverless because it only
| worked on a single machine. That protocol is actually still
| in use today. Cron jobs error logging message of last resort
| is to send email to root@localhost, which is done without any
| networking at all.
| kennu wrote:
| I'm not sure what the point is, since this solution doesn't give
| you an IMAP server to connect to with your email client. Also the
| solution seems a bit complex with all those CodePipelines and
| CodeBuilds. I think all you really need is one or two Lambda
| functions if you're just going to store stuff in S3.
| karlkatzke wrote:
| We use exactly this solution internally at $work to handle our
| customer support, inbound sales, and other 'generic' group
| email that gets read by software and eventually gets to humans.
| It all goes through a separate inbound path and domain vs. our
| corporate email domain. Our outbound email from all of our apps
| goes through sendmail locally on the app servers and then out
| via SES.
| garettmd wrote:
| All you'd really need is a file explorer that works with S3.
| Create a new file in the TMP folder, and it gets sent out. View
| received emails in the appropriate folders. It would also be
| trivial to setup an app or web form for creating the JSON file
| and uploading to S3.
___________________________________________________________________
(page generated 2021-04-30 23:00 UTC)