(had to dug my comment from under a flagged parent)
I self-hosted for well over 20 years, I did not throw the towel and I do not plan to. Self-hosting is a sign of pride. Neither my government nor my Prime Minister nor even my Ministry of Interior or Foreign Ministry can host their own email.
Last time I checked, only State Security self-hosted.
I was probably lucky, but I rarely had delivery problems. The last one was a couple years ago with Microsoft swallowing my emails and it was due to the combination of a fairly old exim and a TLS certificate verification quirk at *.protection.outlook.com. I found a fix in the form of a configuration option somewhere on SO.
In all fairness, there is very little maintenance involved, and whenever I have to do maintenance work, I take the opportunity to learn something new. Like this year, I decided to finally replace my aging Debian jessie setup by Arch Linux, and I rewrote all cron jobs as systemd timers.
I must admit that when I send a really important email, I check the mail server log if it went off without errors, but this does not bother me as checking logs manually once in a while is a good thing anyway.
Lastly, a piece of advice: treat self-hosting like a hobby and learn to enjoy it.
Oh and the very last thing: the person who designed Exim configuration for Debian deserves a special place in hell for all the hours wasted. If you set up Exim on Debian, just figure out how to use the upstream exim config and adapt it to your needs.
My first email usage was at University, pre-WWW. After that I briefly used some ISP email service, but that was on a time of very limited storage and POP only accounts, so I started hosting my own email even before having an always-on internet connection, using a relay and dynamic DNS to receive email when online. Now a days, I use a small VPS to route and receive email, but final destination and storage is on my home server. Over the years, I had, like others here, to ask Outlook and other providers to unblock my IP or domain, but it has been rare.
I really don’t want to live in a world where only two or three companies run email for the entire world, and this is my little act of resistance.
outlook.com keeps sending me dmarc reports with failed dkim... while every single other provider gives pass to all domains. at this point I don't even care anymore.
They have a crappy internal DNS caching server in the email infra that times out early and returns NXDOMAIN for timeouted requests, causing permfail for DKIM instead of tempfail as RFC suggests in case of DNS timeouts. This crap has been going on for years.
> treat self-hosting like a hobby and learn to enjoy it.
This is why I have stepped away from a lot of my self hosting. I have turned my attention/time elsewhere. Apparently though the time/money balance is shifting a bit again, so it may be worth it to go back.
My biggest hesitance to self hosting email specifically is dealing with spam. What does that look like these days and do you have any pointers to share?
> My biggest hesitance to self hosting email specifically is dealing with spam. What does that look like these days and do you have any pointers to share?
Postfix can easily be configured to reject incoming emails from senders without a reverse DNS mapping for their IP address, which makes it reject a lot of spam.
For spammers with reverse mapping greylisting still works fine, they almost never retry.
Certain commercial spammers (hello China :-0) use software which can be filtered with a just one rule matching their sending software, which is "nice" enough to display its name in their mail headers.
And last but not least spamassassin / rspamd work fine to filter whatever comes through.
In the end I get less than 10 spam emails per week. And these go into a separate mailbox filtered by good old procmail, based on spamassassin's ratings. I check the spam inbox maybe once a week for false positives and more often than not the box is empty.
I use a combination of DNSBL and SpamAssassin. Nowadays Rspamd is supposed to be better than SpamAssassin, but SpamAssassin has served me well enough so far, and I haven't gotten around to trying out Rspamd. When a spam email gets past SpamAssassin, I copy it to a special folder, which gets processed by a cron job to train SpamAssassin on it (sa-learn).
Overall the mail server is very low maintenance. I had to add SPF and DMARC a couple years ago (DKIM isn't necessary) and integrate TLS with letsencrypt (just a few lines in a config file), and sometimes a Debian upgrade requires reviewing the configuration (several years apart as well). There's really not that much to do.
rspamd is my go to solution. Out of the box you get a lot of protection. I use Exim as my MTA but I suggest you use Postfix if you are starting from scratch, only because you will find a lot more write ups on it.
The biggest issue is getting an IP address which is not in the banned lists. IP reputation is key along with SPF and do not send spam!
In the UK a "business" static IP address is sometimes/usually/probably/might be OK. If you are unfortunate then it is already in the lists and you can check that out at point of sign up.
You might look into IPv6 too. I managed to do the Hurricane Electric IPv6 email thing on my home connection for a laugh. That was a few years ago. It seems I need to do something more to get to Guru status.
I’m not sure that there is any pre made product for this, but I’ve been playing around with LLMs to identify spam, or just generally sorting emails for you. And even the self hosted models seem to be pretty good at classifying emails even without external information like spam blacklists or IP reputation.
Email for me is a critical service, and the reasons I stopped self hosting after about 15 years is:
1. Because I couldn't ensure consistent backup and restore with regular monitoring,
2. no disaster recovery plan and in doing so it'd be more expensive than going through another email provider,
3. not always on top of security (my friend that I colo'd with also ran an email server and his system was struck with ransomware (with no backup [except a copy of email via thick client] or DR); I seemed to get away unscathed because I was using FreeBSD which generally less of a target).
I agree that it is little maintenance, but once you're off the happy path, it can be a huge pain in the arse and devastating.
email has easily one of the best responses to failure modes ever and its ancient!
Most smtp daemons will put outbound emails in a queue and run the queue. If the other end is unavailable then it will generally retry on a schedule with some sort of increasing period and then give up after a week or so.
You can easily define multiple inbound relays via your MX records which predate SRV and generic TXT and are supported everywhere.
I've run a lot of other people's email, including my own vanity domains for decades. It really isn't rocket science.
Google and MS and Co really don't screw you around if you follow the rules and that largely involves only SPF being compulsory and the rest (DKIM n that) are nice to have. If you do send spam then you will be crucified and rightly so.
Email is not a critical (its important) service because of course you have several other means of communication starting off with the SIP n RTP server you also run ... 8)
I had a client domain banned by Gmail due to a missing DKIM, even though they had fewer than 1000 emails per month and SPF was correctly set up a decade ago. The bounce message explicitly said they are bouncing because DKIM is missing.
I agree with that aspect of DR; I guess I was more thinking of availability, in that I can probably handle a few hours of not receiving emails, but if it goes longer than a day or so then I'd be pretty miffed. Like I said it's all doable, but it requires a lot of effort, and is probably best not left to someone running a one man show, and once you have more than one person you likely now have to deal with trust and expenses.
I've run a lot of other people's email, including my own vanity domains for decades. It really isn't rocket science.
Again, so have I, and as I said the happy path is always easy, it's when things go wrong, and I'm not even talking about IP reputation or any of the usual issues that people bring up running email.
Email is not a critical (its important) service
Really depends; I still have many services such as banking where I need to use auth codes, also a lot of security is tied to my email in terms of private comms and recovering services.
Suppose your email service went down and the people you run email for complain, do you tell them "oh don't worry it's not a critical service, you can still communicate over other mediums"? Would that work for say gmail?
I have been self-hosting for about 25 years.
I remember the protection.outlook.com issue.
Once there was an issue with a bank that tried to do encryption, but used an expired certificate. But once I told them what the problem was, and that it was a problem for paying customers, they actually fixed it.
Being able to check the server log can be very useful. E.g. to tell someone that their mail was delivered to a served using their domain name, with that IP-address at that time.
I'm thinking of self-hosting email sending for my applications. Does anyone know if, with DMARC/DKIM, email reputation moved from the IP to the domain? If I can make sure only my server can send mail from my domain, shouldn't the sending IP then be irrelevant?
The sending IP remains very relevant; it may be in a third-party blacklist (RBL) or site local blacklist due to prior spam from said IP or even nearby IP(s). Let's have a look through /var/log/maillog... okay that didn't take long.
Spammers can setup DMARC, and have too many domains, so blocking by IP or ASN remains relevant (no legit email from that spammy country? Ban the country!). Reverse DNS is also important, as spammers have sent too much spam (shocking, I know) that some users complain about, a lot, so: no valid reverse DNS, no service. IP addresses or domains that are "too new" may also be a problem, or some sites will want you to fill out random webforms or talk to their support idiots (Hi, Microsoft! No, me logging into some cloud thing of yours was utterly irrelevant to the problem), and all this and more amounts to a lot of rakes you need to not step on to get email setup right.
Yes, I self-host email. Gmail was routing OpenBSD mailing list traffic to the spam "folder", and self-hosting that email was easier than fighting with some rink-a-dink web UI.
Oh, one time about half the customers were in Google and the other half in Microsoft and Google and Microsoft were having some mail snit so yeah good luck getting some of those mails through. That took a while to clear up, and what can you do?
I used to do this. What finally killed it wasn't reputation, it was the fact that I needed 100% uptime or risk losing messages, getting my address blacklisted, etc. Email is supposed to be resilient to down time (retries, trying each MX record, etc.) but I found that large mail providers tend to just bounce and walk away.
Worse, GitHub (back in 2016 and 2018) would mark a recipient as "unavailable" after a single bounce, refusing to send any more notifications to that address. They since improved the situation and their support was actually very helpful and responsive here, but it's pretty clear that modern SMTP senders have an expectation that recipients will be "always online" that didn't exist when the protocol was invented.
I have a feature (called greylisting) whereby my server intentionally rejects the first mail it receives from a domain.
I have never had anyone claim that their mail has not been delivered to me, and I get a lot of mail.
Retry is built in to the spec, and if you’re really worried you can put a second “receive” SMTP server on the internet with a lower priority, and have it backhaul with LMTP.
———
Email was designed in a time where hosts were not perpetually connected to each other.
This is fearmongering. My mails always got resent after some hours or a day. It's absolutely NOT possible to tell if the problem is on your side, senders side or somewhere in between why a mail is not delivered once and no standard server config would simply toss it.
Host your own mail. I get 99% deliverability with 0 repuation since i do dkim and spf correct.
Don't be distracted by the "complexity" - if you config right it's totally doable.
>I get 99% deliverability with 0 repuation since i do dkim and spf correct.
Your anecdote of success doesn't matter to the others that correctly configured DKIM/SPF and still don't get their emails delivered to Gmail/Outlook/Yahoo/etc. E.g. : https://news.ycombinator.com/item?id=32715437
One of the reasons for hard-to-diagnose sending failures is that Gmail/Outlook have "extra invisible rules" that override correct DKIM/SPF settings because spammers and phishers also have correct DKIM/SPF. So they use extra heuristics such as "ip reputation" etc.
And even after one gets it working, e.g. "submit some form" to Microsoft and wait a few days to get things unblocked... the deliverability may break again because of another "invisible heuristic".
EDIT to reply: >No, that's because your relay overwrites part of the header which makes dkim strict break. Change to relaxed or don't modify the header on your relay.
Delivery reliability can still break without using a relay.
In fact, this unreliability of 100% self-hosting at home is why some self-hosters split it into a hybrid setup and add an external relay for outgoing SMTP and only keep self-hosting for receiving email.
No, that's because your relay overwrites part of the header which makes dkim strict break. Change to relaxed or don't modify the header on your relay.
Outlook business will accept your mail, Outlook private may filter, but the rates fluctuate so heavy i suspect its rules based on user behaiviour/interests. I dono, cant have both spamfree inbox and 0 false positives.
> Q: If your server(s) is/are offline for a few hours, why would you "lose messages"?
They said...
>> Email is supposed to be resilient to down time (retries, trying each MX record, etc.) but I found that large mail providers tend to just bounce and walk away.
I take that to mean that if your server isn't availble to receive the mail at the time it is first offered, it won't be retried later. That wasn't the case (for most mail) when I gave up on self hosting 10 years ago, but it's plausible.
It's not reasonable. Mail not deliverable is not the same as house burned down, recipient moved unknown or sth, it simply means the letter was not received. Who and why messed up is unknown, thus NO mail server will mark you down after a single attempt.
Here is my advice to anyone wanting to test out self-hosting email. Start by using your self-hosted email to sign-up for accounts. You don't have to use the email address for your personal correspondence
Use Mail-in-a-box to get started [1]. You can literally set it up in a couple of hours by following the instructions and everything should just work.
After a few years, you can think about switching your personal correspondence to your new email.
I can recommend Stalwart [1] which is a complete mail service contained in a single binary, that doesn't really have any external dependencies, and is really easy to install and update.
I've looked (and tried) a few other projects in the past, but Stalwart was the easiest to setup, and I haven't had any issues with it so far.
Wow! I was just about to comment how email is the one thing where I wish something that didn't follow the unix philosophy existed. Exactly due to this, it is easy to set up a mail server but it is hard to think of all the things around it: spam, fishing, dmarc, dkim, spf, etc.
This looks really nice, especially also for saas projects.
It’s also what Thunderbird is using to build their paid email hosting. Seems like a very ambitious project mostly done by a single person – impressive!
I've been running MIAB for a few years now with generally good success as an outgoing sender using a rented cloud machine and a "clean" reputation IP. I've had to email the Microsoft postmaster on one occasion when my emails weren't reaching Outlook users, but they were surprisingly helpful and it's been working fine for years now. It's a good learning exercise in setting up stuff like DKIM/SPF/DMARC.
That said - receiving account sign-up emails is the absolute biggest pain in the backside with Mailinabox! The greylisting anti-spam feature relies on bouncing unknown senders and waiting for a retry. The trouble is, many legit sites just don't bother retrying. So email verification for new accounts and 2FA-type stuff often takes ages to come through, if at all. MIAB stubbornly has no easy, mail user-facing way to temporarily disable spam filtering and it's a real PITA at times.
Modern email providers, especially ones offered by ISPs often have the same problems that people criticize self-hosted providers for. Even Google has problems. For example, I regularly order via companies that use Shopify. Now, all of the shopify emails are going straight to spam in Gmail, despite constantly marking them as not spam. (These even pass dmarc/spf/dkim etc, so who knows what's going on here.)
Email delivery and receiving is not hard, but it's inevitably going to be imperfect, no matter the provider you use. There are so many bad actors out there, it's surprising that it works as well as it does.
> These even pass dmarc/spf/dkim etc, so who knows what's going on here.
Those have nothing to do with being spam, right? Spam is about content, those are about authenticity. Anybody can send authentic trash, or unauthenticated gold.
> For example, I regularly order via companies that use Shopify. Now, all of the shopify emails are going straight to spam in Gmail, despite constantly marking them as not spam. (These even pass dmarc/spf/dkim etc, so who knows what's going on here.)
There's a pretty good chance this is because Shopify is sending a lot of email users mark as spam, or is using the same mail server as someone who does. Then you marking them as not spam gives them a better score but the sender's reputation is still so bad that it can't break the threshold to stay out of the spam folder.
I have self hosted my email for about twenty years; fr about ten or fifteen I just forwarded everything to Gmail but had to revert to local ( started with local mail in emacs, but switched to imapd to solve the airplane ticket in the airport issue) because so much important stuff was marked as spam. Like in the middle of a conversation between me and on other person their reply to my email (which I always bcc:ed ack to myself) would disappear. Self hosted is much better. It took few iteration to get spf etc working.
That behaviour is the whole problem. If you use a self hosted or small time email provider you're much less likely to have email blocked or filtered by aggressive anti-not-gmail filters.
Hilarious Gmail addresses send tonnes of spam so filtering by provider doesn't do much there days anyway. But Google insists to continue
Bizarrely, I also find Gmail's spam algo is actually oversensitive to marketing emails from companies these days, which I never thought was something I would complain about. But like you said its super annoying when I actually want the emails.
Seems like we had the opposite problem 10ish years ago. But now the pendulum has swung a bit too far in the other direction.
Ultimately most of the spam I get these days is actually from individuals doing low volume cold outreach from personal email addresses...not companies sending bulk. The new gmail unsubscribe feature works great for marketing emails but is worthless against cold email spam -- which somehow rarely ever lands in spam.
Where do people self-host these emails? When email self-hosting is talked about, my thoughts wander to Fastmail, Migadu, etcetera (I use one like these), but I quickly realise that's not it. On those lines, I do not believe these mail self-hosting folks are talking about some VPS, or server from some provider, or even AWS, et al., either — not self-hosting enough. It must be a computer/server always running at their home/basement/or so (with whatever power/Internet backup setup they have—or maybe not, as they might find it acceptable if something was missed/dropped). So is it that? And if that's what it is, then what is that mail self-hosting home setup of yours? What all have you got there? Just curious, I doubt I can go through that, as my patience gives in even trying to set up a VPS for a seedbox when it is time for the first maintenance/tweak.
- More of anti-UCE, with postscreen (greylisting, DNSBL and DNSWL checks), policyd-spf, body_checks, check_sender_access, check_client_access, postscreen_access_list.
- Setting "home_mailbox = Maildir/", to keep mail in user directories and in the Maildir format (which seems to be less prone to corruption than mbox is, and well-supported by MUAs).
- Leaving TLS defaults, except for the paths. I used to set mandatory TLS, but then ran into some servers not using it, and figured that I do not trust the involved servers more than channels between them anyway (especially the servers that do not support TLS). Being overly strict with allowed protocol versions (or even ciphers) also reduces compatibility, while for encryption it is better to rely on OpenPGP.
- I do set Dovecot (for both IMAP and SMTP submission); the recent configuration change did not seem like a big deal to me, and it was documented, so I found it easy to update. It is nice to be able to use email from a server (and that ability does not go away with Dovecot), but a local MUA also has its advantages.
- Registered at dnswl.org, to improve deliverability in some cases.
I've been selfhosting for like dunno 10-15 years. Cheap kimsufi box, opensmtp, dovecot, later then rspamd, done. Never really had a problem. At one point telekom.de blocked my mailserver. I contacted them via postmaster@telekom.de (or something) explaining that while kimsufix boxes are notorious for shady stuff, this is actually a legit mailserver and they whitelisted me shortly after (yeah I was surprised too how smooth that went). So, yeah, can't confirm all the troubles everyone seems to get on about. However I do own the kimsufi box (and the corresponding IP) for a long time now, so maybe I'm just lucky.
I've been self-hosting my email for over 10 years now (I'm going to link a bunch of my old comments on old email HN threads). I have fallen back to using Amazon's SES to send because all of Digital Ocean's IP blocks suddenly got marked as bad and I don't have enough volume to improve a new IP reputation - https://news.ycombinator.com/item?id=39891262, https://news.ycombinator.com/item?id=38471262
But as others here have suggested greylisting is extremely helpful in this space as legitimate servers should always retry. Well only my power company is the exception and they will fall back to sending paper bills, but even Gmail falls foul for them. It's also one big reason I'm not worried about up to a week downtime. But I have two email servers, a receiving and a storage server, the receiving is cattle and I car re-deploy in minutes if needed. - https://news.ycombinator.com/item?id=38512732
On greylisting I would say using https://github.com/stevejenkins/postwhite (even if it's very old and not actively maintained) has proven very important for the annoying 2FA emails, I strongly contend that email isn't suitable for this use case but that's another conversation)
I missed an incoming message (fortunately an unimportant one) from Amazon SES recently, since its 54.240.27.30 address was listed by bl.spamcop.net: Amazon kept trying different addresses while running into greylisting, until it tried that address and was rejected. Possibly it is less of an issue when sending between large providers (e.g., Amazon to Gmail), but apparently still not a perfect solution to ensure message delivery.
And use it only for important things that won't be delivered if you don't send them from Google.
Receive only (e.g. account signup)? Use your own account. Not important anyway? Use your own account. Your recipient needs the email more than you? Use your own account. None of the above? Use a mainstream provider for that email only.
But this is why I'm using Amazon's SES for sending my very low volume emails now. It's never been blocked, if they ever start causing me grief I can switch to a new email relay service in a matter of minutes if necessary.
I think the following is a better guide for someone looking for a complete setup that includes an IMAP server and that can be used with regular email clients like Thunderbird:
I set up my own server more or less following the above guide, but eschewed the database in favor of plain text files. I wanted to keep things simple since I am the only user, but the above guide should scale to big enterprise setups.
I also use this guide, but I switched it to PostgreSQL instead. The recent upgrade to Trixie brought a new Dovecot with breaking changes to its configuration. That was a bit of a pain to resolve, but everything is working fine now.
Self Plug-in: We are currently beta testing Hyvor Relay [0], a self-hosted alternative for sending emails. We are focusing more on observability (monitoring DKIM/SPF, periodically querying DNSBLs) and DNS automation.
A simple docker compose up can get a reasonably working setup [1]
I have a writeup in german about self-hosting current and with debian trixie on https://krei.se/Doc
If you do it yourself and do it correct it's a pleasure. I have automatic updates with automatic reboot, tailored systemd to make sure all is well and status reports per mail - total bliss, easy 2-3 years, with trixie now even 5 until you have to touch it again.
It's mature software.
Host yourself! The peace of mind and control is totally worth it.
Right. You better not self-host like it's 1984 because that would also mean you're an open relay. And vulnerable for pretty much anything you can think of.
Those wore the days :-) I remember playing on a University lab with half a dozen Unix workstations, sending an email with the path of server1!server2!server3 etc and hearing the email flowing from server to server by the noise of the disks!
Assuming this is not hosted on your home system, since ISPs may block the ports and also most of the dynamic ips allocated are blacklisted, the issue with postfix is that its difficult to have a single set and forget config if you intend to use it on multiple internal machines, like for getting your root email on each system to one mailbox. Ideally you want a single main.cf for all your internal machines and for the outgoing/incoming mailhost to be determined solely by your mx or internal dns alias, but this is next to impossible with a single postfix config without getting mail loops on the system that is the mailhost. Exim and sendmail at least separate out the submit config from the rest of the configuration.
Also you would be insane to try this without fail2ban or something similar. Postfix does a reasonable job of handling attackers but it does so quietly -- so you may not see the activity.
Say I want to test the waters for selfhosting email, and I already have my how domains setup with SaaS like Google workspace and equivalent. Is there a way to setup mx records so that both google and my own server gets email for a while? This would be useful to test the waters over a few months before fully migrating
Not with MX but, look at google's split domain documentation. You can either have them handle the domain and forward you a copy, or you can have your own domain be the primary and forward to google. I have been using the latter for a few years now since not all of the users in the domain are using Google Workspace. They have a special address for forwarding to so you don't get into a loop. It has been working flawlessly for us.
You can set up a lower-priority MX to point to Google, so if your server fails, then email is delivered to Google. But if your server is misconfigured and returns permanent 5xx errors for legitimate emails, then it won't work, and the emails won't be delivered to Google.
No easy answer here. Individual MTAs or a cluster of them typically live under one unique domain. In your scenario, you'd have to point your existing records (or just MX) to your self-hosted instance, and have your self-hosted instance relay/autoforward to Gmail under a different domain. This might entail simply setting your Gmail back to @gmail.com.
Not really, SMTP relays will only send messages once, to one server.
But it’s not receiving that is the problem, that is generally fine, if ports are open at ISP / network level. It is the sending that is often tricky. Sending email on the other hand can be done from multiple servers (if SPF correctly configured) And nothing prevents you from sending email directly from your own relay. You could try that, and reception would not be affected.
I at least maintain administrative control while using free resources. I looked at the completely self hosted route a long time ago and didn’t want to deal with that. Fast forward to today and I’m dealing with a friend who has the polar opposite. Now their mental faculties are reduced and they lost devices. Couple this with not paying previous management entities and it’s a digital lock out. Recovering from this is a nightmare and I’m operating blind. For a normie this would be impossible.
I personally believe it is worth exploring the idea of a different email realm for communities. The concept is pretty simple. Don't accept email from gmail, microsoft, hotmail or any other non-community member. Community members don't spam, don't send email in bulk and have reputation.
It is funded by pay-per-transgression. If you are a community member and someone receives unwanted email your reputation suffers. If you are gmail, et al you have to pay for each email sent & received.
Someone once wrote (let me know if you know the source) that users are not the customer, because they don't pay. It is advertisers who are the real email customers. This has resulted in a business model where users are prey animals. This is upside down and probably cannot be fixed without a hard fork.
I don't mean this is a good idea, or implementation. But I think it is a good direction.
What about mail servers generally rejecting email (or marking as spam) from residential IP ranges? Decades of malware sending spam has spoiled self hosting emails.
I needed some minimal mail delivery for user registration confirmation and password recovery, and I finally caved and just use some free service. It's okay since those emails are really, really, sparse in my case. But it sucks that email, this one old and open technology, is not realistically self-hostable.
Yeah, hosting on or at least tunneling through a commercial IP address is definitely required in order not to be flagged as spam. Personally, I chose the latter option of hosting my MTA at home but tunneling its traffic through a VPS in a datacenter. It's been working pretty well ever since, although I'm not sure it's worth the effort versus just using a cheap hosted provider.
There was a blog posted to HN years ago describing a self hosted email setup in detail, and this was indeed the main issue. Everyone he emails is on a small number of big companies, and most of them don't like his server.
"After self-hosting my email for twenty-three years I have thrown in the towel"
Once hosting email for yourself, you may want to add new project-specific domains, or host email for friends and family. The database user accounts actually make it easier to add and remove users after the system is up and running.
This Purplehat guide provides a step by step procedure that's allowed many people and orgs to bring self-hosted email online...
IMHO, there are two components to "email" that do not necessarily need to be connected
1. receving mail
2. sending mail
Only #2 became difficult
Internet subscribers receive lots of email to which they never reply
Sometimes "throwaway" disposable email addresses are useful^1
Various third parties offer this as a "service", i.e., #1 is disconnected from #2
Self-hosting #1 can provide an alternative to using third parties
Generally, the only cost is a domain name registration
1. Also HN commenters have complained in the past that email sent via self-hosted SMTP to certain recipients, e.g., Gmail recpients, may end up stored on certain undesirable third party servers. This is because the recpient uses a third party for both #1 and #2, a so-called "email provider"
I don't think a PTR record is strictly necessary for good deliverability. In fact, I regularly receive emails from IPs with nonexistent or mistmatched PTR records.
Not sure why someone would go through the pain of cobbling up a self hosted solution based on Postfix when you have fully integrated solutions like https://stalw.art/, which are a breeze to setup.
Postfix has been around for decades and respects the Unix philosophy of doing one thing and doing it well. It's perhaps the most widely deployed MTA, and as such it has been thoroughly field tested.
Also, people in the FOSS community tend to be wary of "open source" projects primarily developed by a commercial company under dual licensing.
I am basically all in on Stalwart right now, but do not have the time to deal with email deliverability issues or asking my VPS to open a port, so I have been using AWS SES as the SMTP relay for awhile. No problems so far, hosting about a dozen mailboxes for friend's and family's personal use. The whole stack is so lightweight, I have it on the cheapest Arm-based VPS at Hetzner.
My hope is that Stalwart will get to their webmail project over the next couple of years. I am on SnappyMail, but basically need to use desktop and mobile clients to get the full mail/CardDAV/CalDAV experience, which admittedly is already pretty awesome.
Currently, I have Stalwart, PostgreSQL, SnappyMail, and Caddy, all inside a docker-compose file, and I have migrated, moved servers, and all that no problem.
I came to this thread purely to see if I was the only enlightened one.
Stalwart is perfect for small self-hosters: a single binary, a single-directory resilient datastore (by default), a UI for every setting, and defaults that guide you to a DNS config which maximizes your sender score. Plus support for all of the "power user" features such as ManageSieve and shared CalDav folders.
Honestly, I love hosting my email now. And the last remaining battery which could possibly be included is now WIP: webmail!
Unix philosophy need not apply when there is exactly one use case for integrating these tools. (Or at least, one case which covers 99% of users. The remainder can keep their managerie of arcane config formats and susceptibility to unsafe language CVEs.)
> "If something isn't working for you, please double-check your DNS records, and triple-check that TLS certificates are readable by the Postfix user, and that DKIM keys are readable by the OpenDKIM user. Postfix and OpenDKIM logs will also be useful. The OpenDKIM config file is especially unforgiving of typos, so watch out for small mistakes!"
I tried this over a period of years, aggressively changing my email server configuration as new challenges appeared, before realizing the basic problems were (a) a server's configuration is a moving target that requires constant revision, and (b) if your ISP has ever hosted a spammer, even briefly and inadvertently, then its entire address block may be universally blacklisted and you have to change ISPs, possibly several times.
So ... I gave up. If I had nothing better to do, if I just wanted to play email server whack-a-mole, that would be different, but I have a life apart from pleading with giant email recipients to trust my little server.
It's not as though Google, Microsoft, et al. have an incentive to trust small email servers -- quite the opposite. They can -- and do -- make the argument that they shouldn't trust anything but another big player like themselves.
Almost everything described in the article didn't exist in 1984. Postfix, OpenDKIM, TLS, SPF, DKIM, DMARC. Only very basic SMTP and DNS, but even MX records didn't exist.
in terms of a good self hosted email client, in this day and age, I'm looking for great AI integration. I.e. are there good open source projects that come packaged with a locally hosted LLM integration?
(had to dug my comment from under a flagged parent)
I self-hosted for well over 20 years, I did not throw the towel and I do not plan to. Self-hosting is a sign of pride. Neither my government nor my Prime Minister nor even my Ministry of Interior or Foreign Ministry can host their own email.
Last time I checked, only State Security self-hosted.
I was probably lucky, but I rarely had delivery problems. The last one was a couple years ago with Microsoft swallowing my emails and it was due to the combination of a fairly old exim and a TLS certificate verification quirk at *.protection.outlook.com. I found a fix in the form of a configuration option somewhere on SO.
In all fairness, there is very little maintenance involved, and whenever I have to do maintenance work, I take the opportunity to learn something new. Like this year, I decided to finally replace my aging Debian jessie setup by Arch Linux, and I rewrote all cron jobs as systemd timers.
I must admit that when I send a really important email, I check the mail server log if it went off without errors, but this does not bother me as checking logs manually once in a while is a good thing anyway.
Lastly, a piece of advice: treat self-hosting like a hobby and learn to enjoy it.
Oh and the very last thing: the person who designed Exim configuration for Debian deserves a special place in hell for all the hours wasted. If you set up Exim on Debian, just figure out how to use the upstream exim config and adapt it to your needs.
My first email usage was at University, pre-WWW. After that I briefly used some ISP email service, but that was on a time of very limited storage and POP only accounts, so I started hosting my own email even before having an always-on internet connection, using a relay and dynamic DNS to receive email when online. Now a days, I use a small VPS to route and receive email, but final destination and storage is on my home server. Over the years, I had, like others here, to ask Outlook and other providers to unblock my IP or domain, but it has been rare.
I really don’t want to live in a world where only two or three companies run email for the entire world, and this is my little act of resistance.
outlook.com keeps sending me dmarc reports with failed dkim... while every single other provider gives pass to all domains. at this point I don't even care anymore.
why Microsoft is so crappy?
They have a crappy internal DNS caching server in the email infra that times out early and returns NXDOMAIN for timeouted requests, causing permfail for DKIM instead of tempfail as RFC suggests in case of DNS timeouts. This crap has been going on for years.
They want you to use outlook.
> I decided to finally replace my aging Debian jessie setup by Arch Linux, and I rewrote all cron jobs as systemd timers.
Man, I wish I had 1% of the motivation I had 20 years ago to do something like this, before all the full time job, wife and child.
Don’t hurt me: Agentic coding tools like Claude Code or opencode helped me a lot to convert things to systemd units.
Stuff to keep you busy is always there, you can control what you spend the rest of the time on.
> treat self-hosting like a hobby and learn to enjoy it.
This is why I have stepped away from a lot of my self hosting. I have turned my attention/time elsewhere. Apparently though the time/money balance is shifting a bit again, so it may be worth it to go back.
My biggest hesitance to self hosting email specifically is dealing with spam. What does that look like these days and do you have any pointers to share?
> My biggest hesitance to self hosting email specifically is dealing with spam. What does that look like these days and do you have any pointers to share?
Postfix can easily be configured to reject incoming emails from senders without a reverse DNS mapping for their IP address, which makes it reject a lot of spam.
For spammers with reverse mapping greylisting still works fine, they almost never retry.
Certain commercial spammers (hello China :-0) use software which can be filtered with a just one rule matching their sending software, which is "nice" enough to display its name in their mail headers.
And last but not least spamassassin / rspamd work fine to filter whatever comes through.
In the end I get less than 10 spam emails per week. And these go into a separate mailbox filtered by good old procmail, based on spamassassin's ratings. I check the spam inbox maybe once a week for false positives and more often than not the box is empty.
I use a combination of DNSBL and SpamAssassin. Nowadays Rspamd is supposed to be better than SpamAssassin, but SpamAssassin has served me well enough so far, and I haven't gotten around to trying out Rspamd. When a spam email gets past SpamAssassin, I copy it to a special folder, which gets processed by a cron job to train SpamAssassin on it (sa-learn).
Overall the mail server is very low maintenance. I had to add SPF and DMARC a couple years ago (DKIM isn't necessary) and integrate TLS with letsencrypt (just a few lines in a config file), and sometimes a Debian upgrade requires reviewing the configuration (several years apart as well). There's really not that much to do.
rspamd is my go to solution. Out of the box you get a lot of protection. I use Exim as my MTA but I suggest you use Postfix if you are starting from scratch, only because you will find a lot more write ups on it.
The biggest issue is getting an IP address which is not in the banned lists. IP reputation is key along with SPF and do not send spam!
In the UK a "business" static IP address is sometimes/usually/probably/might be OK. If you are unfortunate then it is already in the lists and you can check that out at point of sign up.
You might look into IPv6 too. I managed to do the Hurricane Electric IPv6 email thing on my home connection for a laugh. That was a few years ago. It seems I need to do something more to get to Guru status.
I’m not sure that there is any pre made product for this, but I’ve been playing around with LLMs to identify spam, or just generally sorting emails for you. And even the self hosted models seem to be pretty good at classifying emails even without external information like spam blacklists or IP reputation.
Naive Bayes classifiers have been working fine for decades.
I think LLMs, even local ones, are probably way overkill for identifying spam or sorting/classifying emails.
The biggest issue isn't necessarily spam, it's proving you aren't spam.
If only we treat ads like we treat emails! Our world could probably be a bit better place to live in.
Email for me is a critical service, and the reasons I stopped self hosting after about 15 years is:
1. Because I couldn't ensure consistent backup and restore with regular monitoring,
2. no disaster recovery plan and in doing so it'd be more expensive than going through another email provider,
3. not always on top of security (my friend that I colo'd with also ran an email server and his system was struck with ransomware (with no backup [except a copy of email via thick client] or DR); I seemed to get away unscathed because I was using FreeBSD which generally less of a target).
I agree that it is little maintenance, but once you're off the happy path, it can be a huge pain in the arse and devastating.
DR: MX and retry
email has easily one of the best responses to failure modes ever and its ancient!
Most smtp daemons will put outbound emails in a queue and run the queue. If the other end is unavailable then it will generally retry on a schedule with some sort of increasing period and then give up after a week or so.
You can easily define multiple inbound relays via your MX records which predate SRV and generic TXT and are supported everywhere.
I've run a lot of other people's email, including my own vanity domains for decades. It really isn't rocket science.
Google and MS and Co really don't screw you around if you follow the rules and that largely involves only SPF being compulsory and the rest (DKIM n that) are nice to have. If you do send spam then you will be crucified and rightly so.
Email is not a critical (its important) service because of course you have several other means of communication starting off with the SIP n RTP server you also run ... 8)
I had a client domain banned by Gmail due to a missing DKIM, even though they had fewer than 1000 emails per month and SPF was correctly set up a decade ago. The bounce message explicitly said they are bouncing because DKIM is missing.
I agree with that aspect of DR; I guess I was more thinking of availability, in that I can probably handle a few hours of not receiving emails, but if it goes longer than a day or so then I'd be pretty miffed. Like I said it's all doable, but it requires a lot of effort, and is probably best not left to someone running a one man show, and once you have more than one person you likely now have to deal with trust and expenses.
I've run a lot of other people's email, including my own vanity domains for decades. It really isn't rocket science.
Again, so have I, and as I said the happy path is always easy, it's when things go wrong, and I'm not even talking about IP reputation or any of the usual issues that people bring up running email.
Email is not a critical (its important) service
Really depends; I still have many services such as banking where I need to use auth codes, also a lot of security is tied to my email in terms of private comms and recovering services.
Suppose your email service went down and the people you run email for complain, do you tell them "oh don't worry it's not a critical service, you can still communicate over other mediums"? Would that work for say gmail?
I have been self-hosting for about 25 years. I remember the protection.outlook.com issue. Once there was an issue with a bank that tried to do encryption, but used an expired certificate. But once I told them what the problem was, and that it was a problem for paying customers, they actually fixed it.
Being able to check the server log can be very useful. E.g. to tell someone that their mail was delivered to a served using their domain name, with that IP-address at that time.
Configure the dmarc reports, they tell you a lot and automatically why someone swallowed your mail.
I'm thinking of self-hosting email sending for my applications. Does anyone know if, with DMARC/DKIM, email reputation moved from the IP to the domain? If I can make sure only my server can send mail from my domain, shouldn't the sending IP then be irrelevant?
The sending IP remains very relevant; it may be in a third-party blacklist (RBL) or site local blacklist due to prior spam from said IP or even nearby IP(s). Let's have a look through /var/log/maillog... okay that didn't take long.
Spammers can setup DMARC, and have too many domains, so blocking by IP or ASN remains relevant (no legit email from that spammy country? Ban the country!). Reverse DNS is also important, as spammers have sent too much spam (shocking, I know) that some users complain about, a lot, so: no valid reverse DNS, no service. IP addresses or domains that are "too new" may also be a problem, or some sites will want you to fill out random webforms or talk to their support idiots (Hi, Microsoft! No, me logging into some cloud thing of yours was utterly irrelevant to the problem), and all this and more amounts to a lot of rakes you need to not step on to get email setup right.Yes, I self-host email. Gmail was routing OpenBSD mailing list traffic to the spam "folder", and self-hosting that email was easier than fighting with some rink-a-dink web UI.
Oh, one time about half the customers were in Google and the other half in Microsoft and Google and Microsoft were having some mail snit so yeah good luck getting some of those mails through. That took a while to clear up, and what can you do?
Correct I often setup SPF/etc with the domain, no IP
I used to do this. What finally killed it wasn't reputation, it was the fact that I needed 100% uptime or risk losing messages, getting my address blacklisted, etc. Email is supposed to be resilient to down time (retries, trying each MX record, etc.) but I found that large mail providers tend to just bounce and walk away.
Worse, GitHub (back in 2016 and 2018) would mark a recipient as "unavailable" after a single bounce, refusing to send any more notifications to that address. They since improved the situation and their support was actually very helpful and responsive here, but it's pretty clear that modern SMTP senders have an expectation that recipients will be "always online" that didn't exist when the protocol was invented.
I have a feature (called greylisting) whereby my server intentionally rejects the first mail it receives from a domain.
I have never had anyone claim that their mail has not been delivered to me, and I get a lot of mail.
Retry is built in to the spec, and if you’re really worried you can put a second “receive” SMTP server on the internet with a lower priority, and have it backhaul with LMTP.
———
Email was designed in a time where hosts were not perpetually connected to each other.
This is fearmongering. My mails always got resent after some hours or a day. It's absolutely NOT possible to tell if the problem is on your side, senders side or somewhere in between why a mail is not delivered once and no standard server config would simply toss it.
Host your own mail. I get 99% deliverability with 0 repuation since i do dkim and spf correct.
Don't be distracted by the "complexity" - if you config right it's totally doable.
Gives you actual private caldav too btw
>I get 99% deliverability with 0 repuation since i do dkim and spf correct.
Your anecdote of success doesn't matter to the others that correctly configured DKIM/SPF and still don't get their emails delivered to Gmail/Outlook/Yahoo/etc. E.g. : https://news.ycombinator.com/item?id=32715437
One of the reasons for hard-to-diagnose sending failures is that Gmail/Outlook have "extra invisible rules" that override correct DKIM/SPF settings because spammers and phishers also have correct DKIM/SPF. So they use extra heuristics such as "ip reputation" etc.
And even after one gets it working, e.g. "submit some form" to Microsoft and wait a few days to get things unblocked... the deliverability may break again because of another "invisible heuristic".
EDIT to reply: >No, that's because your relay overwrites part of the header which makes dkim strict break. Change to relaxed or don't modify the header on your relay.
Delivery reliability can still break without using a relay.
In fact, this unreliability of 100% self-hosting at home is why some self-hosters split it into a hybrid setup and add an external relay for outgoing SMTP and only keep self-hosting for receiving email.
No, that's because your relay overwrites part of the header which makes dkim strict break. Change to relaxed or don't modify the header on your relay.
Outlook business will accept your mail, Outlook private may filter, but the rates fluctuate so heavy i suspect its rules based on user behaiviour/interests. I dono, cant have both spamfree inbox and 0 false positives.
I know right. It’s like, “what did they do to my boy?” as to huddle over the bullet ridden corpse of your son.
> it was the fact that I needed 100% uptime or risk losing messages
Q: If your server(s) is/are offline for a few hours, why would you "lose messages"?
I've just checked my own email server -> "up 219 days"
Honestly, compared with the stuff we do all day, this is not hard...
> Q: If your server(s) is/are offline for a few hours, why would you "lose messages"?
They said...
>> Email is supposed to be resilient to down time (retries, trying each MX record, etc.) but I found that large mail providers tend to just bounce and walk away.
I take that to mean that if your server isn't availble to receive the mail at the time it is first offered, it won't be retried later. That wasn't the case (for most mail) when I gave up on self hosting 10 years ago, but it's plausible.
It's not reasonable. Mail not deliverable is not the same as house burned down, recipient moved unknown or sth, it simply means the letter was not received. Who and why messed up is unknown, thus NO mail server will mark you down after a single attempt.
Host your own!!
Reasonable and plausible are different things. I wouldn't be surprised if some outgoing servers just never get around to sending retries.
Here is my advice to anyone wanting to test out self-hosting email. Start by using your self-hosted email to sign-up for accounts. You don't have to use the email address for your personal correspondence
Use Mail-in-a-box to get started [1]. You can literally set it up in a couple of hours by following the instructions and everything should just work.
After a few years, you can think about switching your personal correspondence to your new email.
[1] https://mailinabox.email./
I can recommend Stalwart [1] which is a complete mail service contained in a single binary, that doesn't really have any external dependencies, and is really easy to install and update.
I've looked (and tried) a few other projects in the past, but Stalwart was the easiest to setup, and I haven't had any issues with it so far.
[1] https://github.com/stalwartlabs/stalwart
Wow! I was just about to comment how email is the one thing where I wish something that didn't follow the unix philosophy existed. Exactly due to this, it is easy to set up a mail server but it is hard to think of all the things around it: spam, fishing, dmarc, dkim, spf, etc.
This looks really nice, especially also for saas projects.
It’s also what Thunderbird is using to build their paid email hosting. Seems like a very ambitious project mostly done by a single person – impressive!
Has anyone compared Stalwart with say Mox or Maddy, in practice?
They all look about the same from a newb's perspective.
I'm not looking to self-host my email, but this looks fantastic. It's making me reconsider the decision, hm. Thank you for this.
I've been running MIAB for a few years now with generally good success as an outgoing sender using a rented cloud machine and a "clean" reputation IP. I've had to email the Microsoft postmaster on one occasion when my emails weren't reaching Outlook users, but they were surprisingly helpful and it's been working fine for years now. It's a good learning exercise in setting up stuff like DKIM/SPF/DMARC.
That said - receiving account sign-up emails is the absolute biggest pain in the backside with Mailinabox! The greylisting anti-spam feature relies on bouncing unknown senders and waiting for a retry. The trouble is, many legit sites just don't bother retrying. So email verification for new accounts and 2FA-type stuff often takes ages to come through, if at all. MIAB stubbornly has no easy, mail user-facing way to temporarily disable spam filtering and it's a real PITA at times.
Oh! That's what it is. I just thought some websites just took longer to send an email to my unknown domain.
I see that the only way to disable greylisting is to configure the underlying tool [1]. But it also means that SPAM will increase a lot.
[1] https://discourse.mailinabox.email/t/how-to-turn-off-edit-gr...
It's better to whitelist the domains you'll be getting mfa from.
Modern email providers, especially ones offered by ISPs often have the same problems that people criticize self-hosted providers for. Even Google has problems. For example, I regularly order via companies that use Shopify. Now, all of the shopify emails are going straight to spam in Gmail, despite constantly marking them as not spam. (These even pass dmarc/spf/dkim etc, so who knows what's going on here.)
Email delivery and receiving is not hard, but it's inevitably going to be imperfect, no matter the provider you use. There are so many bad actors out there, it's surprising that it works as well as it does.
> These even pass dmarc/spf/dkim etc, so who knows what's going on here.
Those have nothing to do with being spam, right? Spam is about content, those are about authenticity. Anybody can send authentic trash, or unauthenticated gold.
> For example, I regularly order via companies that use Shopify. Now, all of the shopify emails are going straight to spam in Gmail, despite constantly marking them as not spam. (These even pass dmarc/spf/dkim etc, so who knows what's going on here.)
There's a pretty good chance this is because Shopify is sending a lot of email users mark as spam, or is using the same mail server as someone who does. Then you marking them as not spam gives them a better score but the sender's reputation is still so bad that it can't break the threshold to stay out of the spam folder.
I mark them as spam. I only want the real notifications and not the free goodies and recap and others are interested in mails.
I have self hosted my email for about twenty years; fr about ten or fifteen I just forwarded everything to Gmail but had to revert to local ( started with local mail in emacs, but switched to imapd to solve the airplane ticket in the airport issue) because so much important stuff was marked as spam. Like in the middle of a conversation between me and on other person their reply to my email (which I always bcc:ed ack to myself) would disappear. Self hosted is much better. It took few iteration to get spf etc working.
That behaviour is the whole problem. If you use a self hosted or small time email provider you're much less likely to have email blocked or filtered by aggressive anti-not-gmail filters.
Hilarious Gmail addresses send tonnes of spam so filtering by provider doesn't do much there days anyway. But Google insists to continue
Bizarrely, I also find Gmail's spam algo is actually oversensitive to marketing emails from companies these days, which I never thought was something I would complain about. But like you said its super annoying when I actually want the emails.
Seems like we had the opposite problem 10ish years ago. But now the pendulum has swung a bit too far in the other direction.
Ultimately most of the spam I get these days is actually from individuals doing low volume cold outreach from personal email addresses...not companies sending bulk. The new gmail unsubscribe feature works great for marketing emails but is worthless against cold email spam -- which somehow rarely ever lands in spam.
Microsoft Outlook has been flagging their own marketing emails as spam for me lately. I'm not sure if I ought to be impressed or disappointed.
Maybe because I and others mark them as spam?
Where do people self-host these emails? When email self-hosting is talked about, my thoughts wander to Fastmail, Migadu, etcetera (I use one like these), but I quickly realise that's not it. On those lines, I do not believe these mail self-hosting folks are talking about some VPS, or server from some provider, or even AWS, et al., either — not self-hosting enough. It must be a computer/server always running at their home/basement/or so (with whatever power/Internet backup setup they have—or maybe not, as they might find it acceptable if something was missed/dropped). So is it that? And if that's what it is, then what is that mail self-hosting home setup of yours? What all have you got there? Just curious, I doubt I can go through that, as my patience gives in even trying to set up a VPS for a seedbox when it is time for the first maintenance/tweak.
I have a VPS at Hetzner. I pay 4 euro per month. It's inside a FreeBSD jail for separation.
Keeping FreeBSD up to date is extremely simple. run pkg update && pkg upgrade. It rarely breaks. Can't remember it ever breaking.
The main reason I prefer FreeBSD over Linux distros are the far superior package managers(pkg for binaries and ports for source code).
I also host my own web server using nginx, and sometimes other stuff. All in separate jails.
Back when I was a kid, I used to have my own servers.
I host it at home.
FWIW, some of the things I configure differently:
- More of anti-UCE, with postscreen (greylisting, DNSBL and DNSWL checks), policyd-spf, body_checks, check_sender_access, check_client_access, postscreen_access_list.
- Setting "home_mailbox = Maildir/", to keep mail in user directories and in the Maildir format (which seems to be less prone to corruption than mbox is, and well-supported by MUAs).
- Leaving TLS defaults, except for the paths. I used to set mandatory TLS, but then ran into some servers not using it, and figured that I do not trust the involved servers more than channels between them anyway (especially the servers that do not support TLS). Being overly strict with allowed protocol versions (or even ciphers) also reduces compatibility, while for encryption it is better to rely on OpenPGP.
- I do set Dovecot (for both IMAP and SMTP submission); the recent configuration change did not seem like a big deal to me, and it was documented, so I found it easy to update. It is nice to be able to use email from a server (and that ability does not go away with Dovecot), but a local MUA also has its advantages.
- Registered at dnswl.org, to improve deliverability in some cases.
I've been selfhosting for like dunno 10-15 years. Cheap kimsufi box, opensmtp, dovecot, later then rspamd, done. Never really had a problem. At one point telekom.de blocked my mailserver. I contacted them via postmaster@telekom.de (or something) explaining that while kimsufix boxes are notorious for shady stuff, this is actually a legit mailserver and they whitelisted me shortly after (yeah I was surprised too how smooth that went). So, yeah, can't confirm all the troubles everyone seems to get on about. However I do own the kimsufi box (and the corresponding IP) for a long time now, so maybe I'm just lucky.
I've been self-hosting my email for over 10 years now (I'm going to link a bunch of my old comments on old email HN threads). I have fallen back to using Amazon's SES to send because all of Digital Ocean's IP blocks suddenly got marked as bad and I don't have enough volume to improve a new IP reputation - https://news.ycombinator.com/item?id=39891262, https://news.ycombinator.com/item?id=38471262
I use Gmail as a free spam harvester to train my own spam filter - https://news.ycombinator.com/item?id=38843288.
But as others here have suggested greylisting is extremely helpful in this space as legitimate servers should always retry. Well only my power company is the exception and they will fall back to sending paper bills, but even Gmail falls foul for them. It's also one big reason I'm not worried about up to a week downtime. But I have two email servers, a receiving and a storage server, the receiving is cattle and I car re-deploy in minutes if needed. - https://news.ycombinator.com/item?id=38512732
On greylisting I would say using https://github.com/stevejenkins/postwhite (even if it's very old and not actively maintained) has proven very important for the annoying 2FA emails, I strongly contend that email isn't suitable for this use case but that's another conversation)
I missed an incoming message (fortunately an unimportant one) from Amazon SES recently, since its 54.240.27.30 address was listed by bl.spamcop.net: Amazon kept trying different addresses while running into greylisting, until it tried that address and was rejected. Possibly it is less of an issue when sending between large providers (e.g., Amazon to Gmail), but apparently still not a perfect solution to ensure message delivery.
Long winded admission that you might as well pay for google's email service.
Not really, I don't send many emails these days but find the ability to filter and run automations on emails coming into my server very useful.
Not to mention I use a separate email per service so I can tell if there's a data breach (usually end up seeing an uptick in spam).
*Edit: Not to mention I can pay for a relay service other than Amazon SES without using Google's services, and for significantly less.
And use it only for important things that won't be delivered if you don't send them from Google.
Receive only (e.g. account signup)? Use your own account. Not important anyway? Use your own account. Your recipient needs the email more than you? Use your own account. None of the above? Use a mainstream provider for that email only.
But this is why I'm using Amazon's SES for sending my very low volume emails now. It's never been blocked, if they ever start causing me grief I can switch to a new email relay service in a matter of minutes if necessary.
I think the following is a better guide for someone looking for a complete setup that includes an IMAP server and that can be used with regular email clients like Thunderbird:
https://workaround.org/ispmail-bookworm/
I set up my own server more or less following the above guide, but eschewed the database in favor of plain text files. I wanted to keep things simple since I am the only user, but the above guide should scale to big enterprise setups.
I also use this guide, but I switched it to PostgreSQL instead. The recent upgrade to Trixie brought a new Dovecot with breaking changes to its configuration. That was a bit of a pain to resolve, but everything is working fine now.
Self Plug-in: We are currently beta testing Hyvor Relay [0], a self-hosted alternative for sending emails. We are focusing more on observability (monitoring DKIM/SPF, periodically querying DNSBLs) and DNS automation.
A simple docker compose up can get a reasonably working setup [1]
[0]https://github.com/hyvor/relay [1]https://relay.hyvor.com/hosting/deploy-easy
I have a writeup in german about self-hosting current and with debian trixie on https://krei.se/Doc
If you do it yourself and do it correct it's a pleasure. I have automatic updates with automatic reboot, tailored systemd to make sure all is well and status reports per mail - total bliss, easy 2-3 years, with trixie now even 5 until you have to touch it again.
It's mature software.
Host yourself! The peace of mind and control is totally worth it.
Where is UUCP? Why are addresses not bang paths? Where is sendmail.cf?
Right. You better not self-host like it's 1984 because that would also mean you're an open relay. And vulnerable for pretty much anything you can think of.
This config doesn’t make an open relay.
This config wasn't available in 1984.
A typical config from 1984 is an open relay and vulnerable to the Morris Worm.
Those wore the days :-) I remember playing on a University lab with half a dozen Unix workstations, sending an email with the path of server1!server2!server3 etc and hearing the email flowing from server to server by the noise of the disks!
Why are addresses not bang paths?
That's what I thought of when I saw the title, too.
Where are my ...killer!jolet! people at?
Ditto. I was sorely disappointed to click through "1984" to find a subheading on "setting up postfix".
Assuming this is not hosted on your home system, since ISPs may block the ports and also most of the dynamic ips allocated are blacklisted, the issue with postfix is that its difficult to have a single set and forget config if you intend to use it on multiple internal machines, like for getting your root email on each system to one mailbox. Ideally you want a single main.cf for all your internal machines and for the outgoing/incoming mailhost to be determined solely by your mx or internal dns alias, but this is next to impossible with a single postfix config without getting mail loops on the system that is the mailhost. Exim and sendmail at least separate out the submit config from the rest of the configuration. Also you would be insane to try this without fail2ban or something similar. Postfix does a reasonable job of handling attackers but it does so quietly -- so you may not see the activity.
Say I want to test the waters for selfhosting email, and I already have my how domains setup with SaaS like Google workspace and equivalent. Is there a way to setup mx records so that both google and my own server gets email for a while? This would be useful to test the waters over a few months before fully migrating
Not with MX but, look at google's split domain documentation. You can either have them handle the domain and forward you a copy, or you can have your own domain be the primary and forward to google. I have been using the latter for a few years now since not all of the users in the domain are using Google Workspace. They have a special address for forwarding to so you don't get into a loop. It has been working flawlessly for us.
You can set up a lower-priority MX to point to Google, so if your server fails, then email is delivered to Google. But if your server is misconfigured and returns permanent 5xx errors for legitimate emails, then it won't work, and the emails won't be delivered to Google.
Configure google to forward mails to your self hosted server.
When replying reply from your self hosted server.
That way you can gradually shift over.
I had been self hosting like this for years.
No easy answer here. Individual MTAs or a cluster of them typically live under one unique domain. In your scenario, you'd have to point your existing records (or just MX) to your self-hosted instance, and have your self-hosted instance relay/autoforward to Gmail under a different domain. This might entail simply setting your Gmail back to @gmail.com.
Not really, SMTP relays will only send messages once, to one server.
But it’s not receiving that is the problem, that is generally fine, if ports are open at ISP / network level. It is the sending that is often tricky. Sending email on the other hand can be done from multiple servers (if SPF correctly configured) And nothing prevents you from sending email directly from your own relay. You could try that, and reception would not be affected.
I at least maintain administrative control while using free resources. I looked at the completely self hosted route a long time ago and didn’t want to deal with that. Fast forward to today and I’m dealing with a friend who has the polar opposite. Now their mental faculties are reduced and they lost devices. Couple this with not paying previous management entities and it’s a digital lock out. Recovering from this is a nightmare and I’m operating blind. For a normie this would be impossible.
I personally believe it is worth exploring the idea of a different email realm for communities. The concept is pretty simple. Don't accept email from gmail, microsoft, hotmail or any other non-community member. Community members don't spam, don't send email in bulk and have reputation.
It is funded by pay-per-transgression. If you are a community member and someone receives unwanted email your reputation suffers. If you are gmail, et al you have to pay for each email sent & received.
Someone once wrote (let me know if you know the source) that users are not the customer, because they don't pay. It is advertisers who are the real email customers. This has resulted in a business model where users are prey animals. This is upside down and probably cannot be fixed without a hard fork.
I don't mean this is a good idea, or implementation. But I think it is a good direction.
https://www.postfix-tutorial.com/
How big a deal is reverse DNS for this and anything else in 2025? I've not created any PTR records for anything for about 20 years now...
Ironically, self-hosted mail servers are more likely to bounce incoming mail from someone without a proper PTR than the big mail service providers.
No deal, even possible for ipv6 easily with eg Hetzner, Vultr
More like 1994 thereabouts, in 1984 most of us would be very lucky to have a dial up connection to the local BBS, under local phone call price rates.
I first started maintaining a nail server in 1997. Most of the stuff in the article is newer than that.
For 1984, I'd have expected UUCP and bang paths to peer mail hosts. Instead the article starts off by setting up DKIM, from over 2 decades later!
Not even that, Postfix didn't exist in 1994. This is a 2025 mail server setup and about as vanilla as it gets.
What about mail servers generally rejecting email (or marking as spam) from residential IP ranges? Decades of malware sending spam has spoiled self hosting emails.
I needed some minimal mail delivery for user registration confirmation and password recovery, and I finally caved and just use some free service. It's okay since those emails are really, really, sparse in my case. But it sucks that email, this one old and open technology, is not realistically self-hostable.
Yeah, hosting on or at least tunneling through a commercial IP address is definitely required in order not to be flagged as spam. Personally, I chose the latter option of hosting my MTA at home but tunneling its traffic through a VPS in a datacenter. It's been working pretty well ever since, although I'm not sure it's worth the effort versus just using a cheap hosted provider.
Is it some kind of free VPS?
VPS are generally paid, but it's only $5 or less a month.
For anyone interested in getting a mail server, I can really recommend Michael W. Lucas' Run Your Own Mail Server
There was a blog posted to HN years ago describing a self hosted email setup in detail, and this was indeed the main issue. Everyone he emails is on a small number of big companies, and most of them don't like his server.
"After self-hosting my email for twenty-three years I have thrown in the towel"
https://news.ycombinator.com/item?id=32715437
https://cfenollosa.com/blog/after-self-hosting-my-email-for-...
I'm interested in doing something like this and connecting it to an AI agent. My autoreply to spam could either an unsubscribe or ignore.
Don‘t. Learn about backscatter
Actually, full strength virtual (multi-domain) email hosting is also quite doable.
This is a great guide that's been used and updated for many years:
https://www.purplehat.org/?page_id=1450
Once hosting email for yourself, you may want to add new project-specific domains, or host email for friends and family. The database user accounts actually make it easier to add and remove users after the system is up and running.
This Purplehat guide provides a step by step procedure that's allowed many people and orgs to bring self-hosted email online...
What's the "like it's 1984" part?
I was hoping he’d set up Citadel! Email system and an 80s bbs platform in one. citadel.org
What's the "like it's 1984" part?
Maybe there's a sleep() command in there so that it takes six days to send an e-mail from upstate New York to Sweden?
Because I can tell you that's how long it took in 1984.
IMHO, there are two components to "email" that do not necessarily need to be connected
1. receving mail
2. sending mail
Only #2 became difficult
Internet subscribers receive lots of email to which they never reply
Sometimes "throwaway" disposable email addresses are useful^1
Various third parties offer this as a "service", i.e., #1 is disconnected from #2
Self-hosting #1 can provide an alternative to using third parties
Generally, the only cost is a domain name registration
1. Also HN commenters have complained in the past that email sent via self-hosted SMTP to certain recipients, e.g., Gmail recpients, may end up stored on certain undesirable third party servers. This is because the recpient uses a third party for both #1 and #2, a so-called "email provider"
What do people do about PTR records on residential addresses?
I don't think a PTR record is strictly necessary for good deliverability. In fact, I regularly receive emails from IPs with nonexistent or mistmatched PTR records.
Ars wrote a pretty good series about self-hosting emails back in (gasp!) 2014: https://arstechnica.com/information-technology/2014/02/how-t...
Not sure why someone would go through the pain of cobbling up a self hosted solution based on Postfix when you have fully integrated solutions like https://stalw.art/, which are a breeze to setup.
Postfix has been around for decades and respects the Unix philosophy of doing one thing and doing it well. It's perhaps the most widely deployed MTA, and as such it has been thoroughly field tested.
Also, people in the FOSS community tend to be wary of "open source" projects primarily developed by a commercial company under dual licensing.
I am basically all in on Stalwart right now, but do not have the time to deal with email deliverability issues or asking my VPS to open a port, so I have been using AWS SES as the SMTP relay for awhile. No problems so far, hosting about a dozen mailboxes for friend's and family's personal use. The whole stack is so lightweight, I have it on the cheapest Arm-based VPS at Hetzner.
My hope is that Stalwart will get to their webmail project over the next couple of years. I am on SnappyMail, but basically need to use desktop and mobile clients to get the full mail/CardDAV/CalDAV experience, which admittedly is already pretty awesome.
Currently, I have Stalwart, PostgreSQL, SnappyMail, and Caddy, all inside a docker-compose file, and I have migrated, moved servers, and all that no problem.
I came to this thread purely to see if I was the only enlightened one.
Stalwart is perfect for small self-hosters: a single binary, a single-directory resilient datastore (by default), a UI for every setting, and defaults that guide you to a DNS config which maximizes your sender score. Plus support for all of the "power user" features such as ManageSieve and shared CalDav folders.
Honestly, I love hosting my email now. And the last remaining battery which could possibly be included is now WIP: webmail!
Unix philosophy need not apply when there is exactly one use case for integrating these tools. (Or at least, one case which covers 99% of users. The remainder can keep their managerie of arcane config formats and susceptibility to unsafe language CVEs.)
Because postfix is foss, will work with everything and for all time and if there's a problem with it you'll actually be able to fix it.
I thought Stalwart’s license, AGPL is foss.
> "If something isn't working for you, please double-check your DNS records, and triple-check that TLS certificates are readable by the Postfix user, and that DKIM keys are readable by the OpenDKIM user. Postfix and OpenDKIM logs will also be useful. The OpenDKIM config file is especially unforgiving of typos, so watch out for small mistakes!"
I tried this over a period of years, aggressively changing my email server configuration as new challenges appeared, before realizing the basic problems were (a) a server's configuration is a moving target that requires constant revision, and (b) if your ISP has ever hosted a spammer, even briefly and inadvertently, then its entire address block may be universally blacklisted and you have to change ISPs, possibly several times.
So ... I gave up. If I had nothing better to do, if I just wanted to play email server whack-a-mole, that would be different, but I have a life apart from pleading with giant email recipients to trust my little server.
It's not as though Google, Microsoft, et al. have an incentive to trust small email servers -- quite the opposite. They can -- and do -- make the argument that they shouldn't trust anything but another big player like themselves.
I haven’t read the article and I am to afraid to open the link in case they are using sendmail.
How long are you going to keep the cat in the box?
Spoiler alert, it’s Postfix. So not really 1984 software. But then again, neither is Linux…
Almost everything described in the article didn't exist in 1984. Postfix, OpenDKIM, TLS, SPF, DKIM, DMARC. Only very basic SMTP and DNS, but even MX records didn't exist.
OpenDKIM seems dead:
https://github.com/trusteddomainproject/OpenDKIM/issues/236
But the experience of using mailx is close to that time, hence the title. Even though I’m too young to know for sure :)
in terms of a good self hosted email client, in this day and age, I'm looking for great AI integration. I.e. are there good open source projects that come packaged with a locally hosted LLM integration?