Writing an IMAP client for the Mac menu bar
I wanted to read and answer mail without switching to a mail app. Not a full client in a big window, just a popover from the menu bar for the ten-second stuff: anything new? Read it, reply, archive, back to work. That idea escalated into a whole menu bar app (Barista), but the mail client is still the part that took the most work. So this post is about that.
It's not my first go at IMAP. Years ago I started (and never finished) a Geary-style mail client for Linux, so the protocol itself wasn't new to me. What was new: the Mac App Store sandbox, a popover that opens and closes fifty times a day, and Apple's networking stack.
Network.framework, then SwiftNIO
The very first version talked to mail servers through Apple's NWConnection, with TLS from the first byte (port 993 for IMAP, 465 for SMTP). That covers Gmail and iCloud, and it's honestly the nicer way to do it.
But plenty of servers, especially company and self-hosted ones, only offer the older way: STARTTLS on ports 143 and 587. You connect unencrypted, ask the server to switch to TLS, and then do the TLS handshake on the same connection. NWConnection wants TLS decided when you create the connection, and there's no clean way to flip an open one over.
So the networking moved to SwiftNIO. With NIO you can drop a TLS handler into a live connection right after the STARTTLS exchange, which is exactly what the protocol wants. The nice surprise was how little the IMAP and SMTP code above it had to change. The new transport kept the same tiny contract the old one had: send a line, read a line, read exactly N bytes, start TLS.
Two small things I made sure of:
- After switching to TLS, anything the server sent before the handshake gets thrown away. Otherwise someone in the middle could slip fake responses in while the connection is still unencrypted, and the client would happily trust them.
- Before trying STARTTLS, the client checks the server actually offers it. If it doesn't, you get a clear message instead of the client quietly carrying on unencrypted.
Don't trust the server's numbers
IMAP tells you up front how big a chunk of data is going to be, like {12345} followed by that many bytes. A broken (or hostile) server could announce {999999999}, or send a line that just never ends. So there are hard limits: 4 MB for a normal line and 64 MB for one of those chunks. Gmail caps attachments at 25 MB, about 34 MB once encoded, so real mail never gets close.
Peeking shouldn't change anything
The message list only asks for the envelope (sender, subject, date) plus a few headers, using BODY.PEEK. The "peek" part matters. A normal fetch marks the message as read, which would mean that just glancing at your inbox marks everything read. When you open a message, it first asks how the message is put together and then downloads only the part it needs, not every attachment.
The message that kept coming back unread
This bug was fixed in 1.1.4. You open a message, Barista marks it read and tells the server. But a refresh might already be on its way, and that refresh asked the server before your change landed. So it comes back saying "still unread", the list gets replaced with the server's version, and a couple of minutes later the message you just read is bold again.
The fix is a small list of "changes the server hasn't confirmed yet". They get re-applied on top of every refresh until the server's own copy agrees, which is the only reliable sign the change stuck. If the server never agrees, the change expires after ten minutes, so a genuinely failed update doesn't keep lying to you forever. It's plain logic with no networking in it, so it's easy to unit-test without a server.
Grouping conversations
Threads are worked out on the Mac. Every message carries its own ID and the IDs of the messages it replies to, and those get linked together, so a chain of replies ends up as one conversation even if some messages in the middle are missing.
Then there's a fallback: messages with the same subject (ignoring the "Re:" and "Fwd:" bits) get grouped too, because lots of senders don't set those reply IDs at all. That's roughly what Gmail does as well. The known downside: a newsletter with the same subject every week turns into one very long thread.
Keeping HTML mail safe
HTML mail is shown with JavaScript always off. Remote content is blocked by default, so tracking pixels don't fire every time you open the popover. You can switch remote content on if you'd rather see the images.
On the sending side, line breaks are stripped out of things like the subject before they're written into the email. Without that, a subject like Hello\r\nBcc: someone@example.com would sneak in an extra header. Outgoing messages also get an ID based on your own email domain. An early version used the app's bundle ID there, and mail providers took that as a spam signal.
Drafts, scheduled sends and snoozed messages are stored encrypted on disk, with the key in the Keychain. Each one gets its own file, so a corrupted snooze list can't take your drafts down with it.
What it doesn't do (yet)
- No push. IMAP has a way to keep a connection open and get told about new mail right away (IDLE). Barista doesn't use it. It checks every two minutes, and after a failure it retries sooner, backing off from 20 seconds up to 5 minutes. For something living in the menu bar I'm still not sure keeping a connection open per account is worth it. If you have opinions on that, I'd love to hear them.
- No OAuth sign-in. Gmail and iCloud work with app-specific passwords. Accounts that only allow OAuth, like personal Outlook and most Microsoft 365 accounts, don't work yet.
If you want to build your own
There's plenty of good free material out there. A few that are worth having open:
- RFC 9051, the current IMAP spec. Most servers still speak the older RFC 3501, so keep that one handy too.
- RFC 2683, practical advice for people writing IMAP clients, like how not to hammer the server. If you only read one, read this one.
- RFC 2177 for IDLE (the push thing Barista doesn't do yet) and RFC 7162 for quickly catching up on what changed since you last looked.
- RFC 5322, RFC 2045 and RFC 2047 for the messages themselves: headers, MIME parts and those
=?UTF-8?...?=encoded subjects. - MailKit's FAQ. It's for a C# library, but it explains a lot of real-world IMAP and MIME weirdness, whatever language you're using.
- Gmail's IMAP extensions, if you want Gmail's labels and threads to make sense.
- Geary, a whole open source mail client to read through, and the one my old Linux attempt was modelled on.
- Dovecot, a real IMAP server you can run locally and break without touching your own mail.