On this page On this page
episode 26: Email protocols with Mauro De Gennaro from Stalwart Labs.
This episode features an in depth conversation with Mauro De Gennaro of Stalwart Labs that demystifies how email actually works in practice, from SMTP delivery and DNS based routing to IMAP, JMAP, spam filtering, encryption, and authentication standards like DKIM, SPF, and DMARC, all through the lens of building and operating a modern mail and collaboration server in Rust; Mauro shares his long personal journey from early BBS systems to founding Stalwart, explains why email protocols have evolved the way they have, how self hosting email can be practical and secure today, and why open standards, automation, and memory safety are key to reclaiming control over critical communication infrastructure.
If you like this podcast you might also like our modular network framework in Rust: https://ramaproxy.org
00:00 Intro00:50 Get to know Mauro De Gennaro08:44 Origins of Stalwart12:24 The Evolution of Stalwart and Its Features14:59 Life of a mail sent to a Stalwart server18:14 Understanding Email Transmission and Protocols22:54 Combination of Email with Calendar and Contacts26:37 Email File Attachments31:56 Stalwart and ACME39:44 Email Address validation44:47 Exploring the Sieve Protocol for Mail Filtering47:54 Email RFCs and standarization54:39 Experiences at FOSDEM and the Future of Email56:50 The Case for Self-Hosting Email Solutions01:00:27 Backup Strategies for Self-Hosted Email Servers01:04:35 Ensuring Redundancy and Reliability in Email Hosting01:12:38 Comparing Email Protocols: SMTP, POP3, and IMAP01:16:25 Shared Inboxes and Automated Email01:24:26 Understanding Email Authentication Standards01:27:54 ARC, SPF, DKIM and DMARC01:33:24 Integrating AI with Email Systems01:36:03 Self-Hosting Considerations01:39:31 Getting Started with Stalwart01:41:54 Outro
Music for this episode was composed by Dj Mailbox. Listen to his music at https://on.soundcloud.com/4MRyPSNj8FZoVGpytj
Elizabeth (Plabayo BV)
0:13 | 🔗
This is netstack.fm, your weekly podcast about networking, Rust, and everything in between. You are listening to episode 26, recorded on February 4th, 2026. In this episode, we talk with Mauro De Gennaro from Stalwart Labs about how email works in practice, the main protocols involved, how they fit together, and what it takes to build and operate an email server in Rust. Hello everybody for another week of netstack.fm, we are episode 26 and with me is Mauro from Stalwart Labs and I'm super excited for this one because it is the first time we can touch on mail protocols and how mail work and really start to understand it. So very welcome Mauro as you are the perfect guest for this topic. despite all the amazing work that you and your company have been doing and mostly in the open even and despite the fact that you have given several talks about it, I think it's a topic that's not being seen by the mass, not even within the Rust community. think many people know about what you're doing and so we would like to get to know you a bit better. Can you maybe introduce yourself a bit? yeah, sure. ⁓ My name is Mauro De Gennaro and I'm the lead developer at Stalwart Labs, ⁓ which is a company that develops Stalwart an email and collaboration mail server written in Rust. ⁓ And ⁓ in addition to the server, there are a few libraries, crates available ⁓ on our repository for different mail-related ⁓ functions, for example, email parsing, authentication, ⁓ also ⁓ email building and so on. very cool and and like how did you find yourself in this world of emails like for example like how did you get even into computer science or computer or hacking or networking or however you got into this Yeah, well, I got my first computer when I was in 91, I think. And it was a Commodore 64. at the beginning, I started playing games and then I wanted to see how those games were made. yeah, I started getting into programming. I got a few books and I started learning basic and so on. Then in 93, I started my own BBS, a bulletin board system, for those that don't know what it is. It's a place where you use to connect via modem to access files, chats from other computers. Most BBS were single node, unless you had money for a lot of phone lines. With the BBS, allowed me to have my first interaction with the internet. because I wanted to offer email to my BBS users. And back then, the only possibility was the UUCP protocol, which is like an offline protocol that fetches your emails in batches. So yeah, that was the first time I used internet, was through UUCP, reading newsgroups, and so on. And yeah, during this time I was also programming, then I got into C programming, which is my favorite language. Well, now it's Rust, but C has a special place in my heart. So around the late 90s, I realized that the only way to see your email online was to get Yahoo or AOL and so on and if not get the webmail from Exchange. So I wrote a webmail in C ⁓ using CGI 1.0 that ⁓ allowed you to access your ⁓ mail server, your emails that were hosted either using the mailbox format and then MailDeer and then also IMAP. And ⁓ after that, I wrote a mail server in C in the early 2000s, something like that. this was my first email related company. after that, that company was acquired. And then I started a new company that had its own ⁓ like a framework for creating business applications. And I also wrote once again another email server this time in Java. I don't really like Java anymore, but that was a dark episode in my past. yeah, so then after this company, yeah, I took some... time off to study, some personal stuff, travel and so on. And after a while, I started feeling the need to go back to programming again. ⁓ I saw, let's see how the, after so many years, how the ⁓ email landscape looked like. And I realized that it still looked like 20 years ago. You still have Dapco, Postfix, Exim, and the same... ⁓ Yeah, the same software as always and nothing had changed. The configurations were really complicated. They were from different people and they were not really scalable or memory-safe. So ⁓ yeah, and also I decided to change that and start writing ⁓ an email server ⁓ in Rust, which by the way, I had no previous experience with. with Rust before that point. And I started learning the language by implementing the mail parser trait, which is an email parsing library. and after that, I started creating other traits. then always with the goal of writing the mail server in Rust. once the mail server was completed, I continue with the collaboration part. And then I implemented every single protocol, email protocol and collaboration protocol out there, including IMAP, POP3, SMTP, JMAP, the new one. If you're not familiar with JMAP, it's a new protocol that is HTTP based that aims to replace... Almost everything basically, every single email and collaboration protocol. JMAP aims ⁓ to replace ⁓ IMAP, SMTP submissions, ⁓ also WebDav for files, CalDav for calendars, and CardDav Also Sieve and so on. it's a new protocol that is really exciting and it ⁓ allows you also to clients, develop clients much faster because you don't have to... deal with weird protocol stuff in JMAP, in IMAP, sorry, and with JMAP is just a JSON ⁓ request. Yeah, so basically that's my personal story. yeah, I've been doing this for a while. And now, now Rust is my favorite programming language. I won't touch anything else. Yeah, perhaps a Python script to do something. automated tasks, but any production code, including web-based code, has to be in Rust. Yeah, I mean, you want me to that decision? Seems reasonable to me. ⁓ Personally, I also went through a lot of different languages and yeah, for the stuff I'm doing as well as like around networking, around web stuff, yeah, for me Rust makes a lot of sense there too. So I understand it. seems you were dealing with ⁓ mails like since long, like as you said, from C to Java and then the origins of stalwarts. You were already writing repos and then eventually like at what point did you feel you actually could form a company around that and then how did that go? Well, the idea, because I have an entrepreneur side, so the goal was always to start a business from it, because I have done that in the past. Also, I have email related businesses, including email hosting and web hosting companies, as well as software development companies. My goal was to first start to try because email, curtain and email software company is quite risky because you're basically reinventing the wheel. So finding investment for that is tricky. So that's why I started doing everything as a solo developer. So yeah, that was the goal from from day one to have a mail server and collaboration server and then ⁓ expand the company. now the ⁓ store is a mail server and collaboration server is ⁓ now is feature complete, but it's not yet version 1.0. The version 1.0 is once, because now we're in the refinement phase. Like after so many years of... almost five years of development. There's a lot of things that need to be, a lot of part of the code that need to be refactored. The configuration needs to be reorganized, the way you configure a stalwart. So yeah, that is what I'm at. So after the stalwart reaches version 1.0, the idea is to expand the team. and ⁓ hire developers to work on the UI, on the web mail. Because currently, Stowart only has a web admin, ⁓ but not a web mail. And yeah, for lessons, I'm not really into UI. The idea is to expand the team to ⁓ have ⁓ more developers. ⁓ Currently, it's a... a small company of five people and ⁓ most of the employees are doing sales or support and also emit some customization for paid clients. But the idea is to expand the team once the development for the webmail starts. Okay, very cool. And so given also the fact that you were always doing is your own personal and family email, is that also happening on your own servers? Yes, yes that is correct. And that's, I guess by now that means you are using a stalwart as well for your own personal stuff. Yes, yes, yes. Also for calendars and file storage. very cool. so, yeah, I mean, first of all, huge amount of respect. It's like been probably like a long journey. So I imagine it's pretty satisfying to now look back at all the different crates and things you create because it's coming all together nicely. And the fact that you can dog food it is probably very satisfying, I can imagine. Yeah, yeah, yeah, but it's always... I'm also getting real anxious because I'm close to version 1.0 and I'm pushing hard, so it is... it releases as soon as possible. The goal is to release it this year because there are a few companies waiting for version 1.0 because all these stories December and there are a few... The last few releases have ⁓ breaking changes, but this is necessary to improve the performance and also reorganize the product. it will be great to have version 1.0 as soon as possible so more companies and also families and small organizations can start using it without worrying about... ⁓ having to worry about migrating their data when a new version ⁓ comes out. So yeah, that's the current status. cool and I can also see that you are getting part of your funding via funds that are established by NLNetLabs which I guess Anelnet Labs which is also ⁓ like we had them well it's NLNet foundation but they also have NLNet Labs like I guess the company split off but we had some podcast episodes already with the folks of NLNet Labs so it's very cool that these funds are also helping projects like yours Yeah, that is correct. the people from NLNet invited me to apply some years ago. Now, Stalwart has received two grants. So it really helped with the development. Very cool. And so in order to start to get an understanding and start to get some road that we can grasp onto as we explore the different protocols and your repositories in more detail, I would like to propose that maybe we can start by, okay, let's say I am... from a maybe Gmail server or from I'm using a Gmail client and I'm sending an email to let's say you and you are using of course stalwart like how does it for example know to what server to connect to, what protocol to select, because like you said, there are like newer protocols as well as older ones. Can you help me like uncover this journey of my like connection and email and where it's going to and then all these decisions are being made? ⁓ Okay, sure. So ⁓ the first time when you, let's say you write an email and you enter the recipient address, for example, john@example.org, ⁓ and you click send, ⁓ your mail client will contact your SMTP server, which is the one you configure while you created your account. So... It will ⁓ establish a connection on port, ⁓ the submissions port, which could be 567, if I'm not mistaken. And ⁓ then it will do the handshake, ⁓ authenticate itself. Usually the submission port requires authentication to obviously prevent open relaying. ⁓ the SMTP server, the MTA, the mail transport agent will take your, will accept your message and place it in the queue, in the internal message queue. And then based on the server policy or how busy it is, at some point it will pick up that message from the queue and it will see, this domain, this email is going to johnatexample.org. So it will take example.org and it will, using DNS, will perform an MX ⁓ lookup. It will check the ⁓ MX record. So on the MX record ⁓ is located in the DNS ⁓ server, for example, or obviously any other domain. And the MX record contains a list of mail servers with priorities. So on the... ⁓ your MTA will start attempting to deliver that message ⁓ in order ⁓ based on the priority. if the first entry on the list is mail1.example.org, it will establish a TCP connection to port 25 on mail.example.org. ⁓ this is the simple. explanation without any security or authentication protocol, right? ⁓ So it will then contact mail1.example.org. If there is a problem, let's say the connection is closed, it will try the second one, mail2.example.org. This time, let's say it works. It will say, it will issue a hello command is called. It will say, hello, I'm this server. I have a mail from this address and I want to deliver it to ⁓ John at example.org. ⁓ If the address is accepted, then the data stage of the transaction begins and the message is placed in the queue of the remote server. And then this server, assuming that ⁓ John is a local user, he will take the message from the queue. and place it in John's mailbox. And then ⁓ John, using, let's say, IMAP or JMAP, will connect to mail.example.org and will fetch the available messages, the new messages, and will then see your message. So this is the... the basic part. So like in the old days without any authentication and security and so on. Then there are a few ⁓ protocols to prevent spam and abuse. These are MTA STS, DMARC, DKIM, SPF. So yeah, I don't know you want... me to go ⁓ explain each one of them or like... Okay, okay. Yeah, in a bit, because I can see there's like a separation there, because I know you have the mail auth crate and there you, I think, implement most of them and I'm not sure all of them, but I see a lot of them defined there, but I can see that for example in your stalwart repo itself, that's where it seems to me that you're implementing things like IMAP, JMAP, SMTP, CalDav etc. Is that correct? Yeah, they're implementing the store repository in the mail server. Okay, and so you were also mentioning, okay, ⁓ you have, like for example, JMAP, but you said it replaces all the other ones, right? I mean, or it aims to replace IMAP, SMTP, CalDav CardDav WebDav Yeah, it's an alternative. ⁓ so, so our offer them in parallel, so the client chooses which one to use. Since JMAP, sorry. Okay, because you were, yeah, yeah, it's because you were in your example, you were saying, okay, first we have some activity in the, let's say, origin servers and that was done over SMTP, you said, but that's only because I guess the user configures to use SMTP. If the user there would have configured JMAP, then I guess it would use JMAP. I guess that's the choice there, I suppose. That is correct, yeah. So, JMAP also can handle email submissions. But the connection, the transport between the two servers is done using SMTP. So, yeah, always, always. So, JMAP only replaces the client, ⁓ the protocol for the client talking to a server, not server to server. Always. Okay, is SMTP, is that always on top of TCP or is it on top of UDP? Okay, and and but it can it can go via TLS no or not Yes, that is part of many of the authentication and authorization protocols in the mail auth crate are about that, about making sure that the, they're called transport security protocols such as MTA STS. they validate the how, they make sure that TLS is using and then you're. when you connect to a remote server that you're getting the right certificates. Protocol like Vane and MTA STS do that. okay and so something that is that was never clear to me is like how come like things like calendar and and file stuff is mixed up with like mail protocols because i mean sure they're both useful but there are like so many different things that you could also like do in a a mail program if you really wanted to like I mean what stops you from from playing videos in in a a mail server like like what is the origins there like how come these things are so often being implemented together You mean emails and calendars and contacts? Yeah, yeah, contacts I guess I can still kind of understand because it's like your address book, I guess. ⁓ Even though I'm not sure how come there's like even a protocol for that. like, yeah, the calendar and stuff. I mean, it seems like almost like a complete separate thing to me. Yes, ⁓ but they separate, but they're integrated. For example, the calendar uses email ⁓ to send the invitation, for example. ⁓ Also, ⁓ they use ⁓ both iCalendar and JSCalendar use email addresses as the participant identities. ⁓ And also most users expect the mail servers and also administrators expect the mail servers to also handle their Another reason is that, for example, with contacts, you can integrate the spam filter with contacts. And if you receive an email that is classified as spam, sender appears in your address book. the spam filter can learn to retrain, to reclassify the message as ham and then train the classifier. So ⁓ it learns that that message is not spam. yeah, it's a, although they're different, calendar is not email. ⁓ It is... ⁓ normal that the server, for example, the exchange does the same. Then you have this Docker container ⁓ software, for example, Mail in a Box and so on that do the same. They take a lot of different products and also include a calendar. ⁓ Some other ⁓ users, some users also even want like matrix integration for chats, but that might be too much. for example, now with JMap and file storage, you could also use stalwart a place to store your spreadsheets, documents, slides. But of course, the spreadsheet and ⁓ slide functionality will be implemented on the client side, not on the server. With Stalwart now you have all the tools to have most of your company data. It's not all files, calendars. So it is a complete collaboration server. It's basically, ⁓ of course you need also like if your company, Stalwart will never implement a CRM and so on, but it's a core component in any organization for collaboration. So you don't need anything else. With Stalwart can handle your email, calendar, contacts, ⁓ and files. Hmm, very interesting. Does that also mean that, well, I guess if I want to send one of those files as an attachment via mail, then it will still just anyway have to be part of, like how does it even work? it like, if I add an attachment to my email, is it just part of the SMTP protocol or? ⁓ Well, no, the attachments are different things. So the SMTP is a transport protocol that will send any kind of email with or without attachment. ⁓ Attachments, if you have a file, let's say you want to attach a file that is stored in your server, ⁓ unless you send them a link, the client needs to ⁓ retrieve that file and code it. and place it in the message using something called MIME, which is ⁓ a message encoding format. Because back in the day, you could not send 8-bit over a link. It had to be 7-bit plain ASCII. still nowadays, we're still encoding messages in ⁓ ASCII. So message bodies. They're encoded either using base64 that we all know or QuotedPrintable, which is for text. a ⁓ MIME message is a structure, a nested structure containing the different parts in the email, of the email. For example, usually emails contain a ⁓ text. and an HTML version so the client can decide which one to display. ⁓ Then it can contain other parts for the attachments. yeah, you can split and also you can contain other messages. So using these nested MIME structures, you can ⁓ send almost anything you want. has a... So more overhead obviously due to all the encoding, but this is what we have now. And as far as I know, there are no plans to improve that encoding format. There are a few attempts, but so far ⁓ MIME is here to stay. Yeah, I mean, and I'm looking at an example from your mail parser repository and to me it looks a lot like HTTP, where the different messages are separated via this giddy up separator it seems. And then you have these headers which define the content type and the content transfer encoding, which is not exactly like HTTP because they have different values, at least the transfer encoding for example quoted printable or something but yeah it looks I guess it's not the most efficient but it does seem to work and I can see why ⁓ what's by the way your opinion around like having this HTML versus like plain text because I always found this the fact that you can even do HTML like I don't know I never understood the need for it except for like okay it looks fancier but to me it always included so much more security risk for so little advantage because anyway you're mostly just content should be king I would think and your message is anyway plain text ⁓ Yeah, that is ⁓ first going back to what you mentioned about the different headers. the email consists, an email message consists of headers, which are key values. And then what you were seeing with the boundaries, those are the mime boundaries. So the boundaries and arbitrary string that should be chosen so it's unique because that is how each part is detected. then the sample that you were looking at on the mail parser crate includes, I'm looking at it, yeah, it has a text HTML and a text plain part, has an attachment, which is embedded in the HTML part. Regarding the use of ⁓ HTML in messages, it's up to the client. You can always choose what to use. HTML is the only way. Well, there are other alternatives. It's not only HTML, but it's a common, it's the most widely used ⁓ format for... emails with formatting. So if you want to include images and have some styling, text HTML is the only option out there that is accepted by almost any client. Hmm. Yeah, I mean, I can imagine and ⁓ okay. And then of course, as we mentioned, TLS as one of the authentication protocols and also security, because you don't really want your mail ideally in plain text, even though it's still happening a lot. certificates are getting shorter and so things like ACME are more and more necessary. Now most mail servers do not really implement ACME and then they would do something like NGNIX but I saw that you do support ACME out of the box which would allow like a trivial use of TLS certificates via things like LetsEncrypt I suppose. Yes, that is correct. It supports ⁓ ACME to make obtaining a certificate ⁓ easier. ⁓ also ⁓ the next release of Stolework will allow administrators to automatically upload because currently Stolework ⁓ is able to modify your DNS server to ⁓ create the ACME DNS01 authentication records. but the new version would allow you also to update any kind of record related to email, like the MX, the ⁓ MTI STS, T-KIM, and so on. So, yeah. But do you really want that? mean, often for security, wouldn't it be nicer if I could just have like a single, I don't know, let's say, HTTP endpoint and do the HTTP challenge so that my server does not need authentication over my records. What is up to the administrator? If you don't want the mail server to modify your DNS records, can also use the TLSALPN01 or the HTTP 01 ⁓ challenge types. The main advantage of using the DNS for Acme is that you can get wildcard certificates, which are useful in a mail server because ⁓ You have other domains, for example, MTA, STS. There is also an auto config domain, which is how mail clients detect configurations for an account. all these protocols expect a certain subdomain. So by obtaining a wildcard certificate, make sure that you don't have to obtain ⁓ separate certificates for well it could also be in the same certificate but there are multiple requests ⁓ with a wildcard certificate you can have as many subdomains as you like with just one certificate Super, yeah, okay, that makes totally sense. I wasn't aware, like, what's the reason that these, can you explain a bit what these protocols are and why do they need subdomains? ⁓ Well, it is part of the specification, for example, ⁓ the way the protocol is designed. for example, auto-config, let's say you enter, John wants to set up a new mail client and the mail client will ask, ⁓ where's your email address? He enters john at example.org. And to make things easier for John, ⁓ the email client will first try using this Autoconfig ⁓ protocol. And what it does is it takes example.org, which is a John's domain, and it adds Autoconfig. And ⁓ then it tries to perform to send an HTTP request to Autoconfig.example.org slash well-known, well, it's a path to a... ⁓ an XML or JSON file containing the mail server details. So this way, John does not have to worry about what is in the IP address or the host name, what is the port, is encryption enable, is implicit, or is to start TLS, a lot of technical details. They are handled automatically. And for this, you need the ⁓ auto-config. ⁓ subdomain. And then you have MTA STS, which what it does is when you send, when the mail server sends an email to another mail server, for example example.org, it will send a similar request to MTA-STS.example.org and obtain the the MTSD settings for that domain. So this file tells that, okay, mail.txt, it will return a list of the authorized domains for ⁓ the mail server. So if there is a problem with your DNS server, someone hacks your DNS server and adds an entry to redirect your email, that will prevent them from doing that because ⁓ MTSD will see Okay, this is a list of mail servers authorized to receive emails for ⁓ example.org. But I see something else in the DNS server. it means something is wrong. The server might be compromised, so delivery is not done. And in addition to that, MTA SDS tells the remote server that... ⁓ your server supports TLS and has a valid certificate. So that's another guarantee. So yeah, going back to the subdomains, those are two examples of domains that you need in addition to the domain that you use to connect. Let's say if you have mail.example.org, you also need the domains for autoconfig and TISTS. And for that reason, it's better to use a wildcard certificate for simplicity. Yeah, that makes totally sense. yeah, I I use the different channels myself as well. Sometimes wild cards comes in handy and in this case here. But I mean, all in all, it's already very cool that if you're using Stalwart that you can use ACME to begin with because... I mean, first of all, I don't agree with the registry companies, but like for some reason, and it's beyond me, is that they are reducing the lifetime of the certificates more and more. Like, I don't think it really helps with security, but they seem to claim so. But that doesn't mean that you are coming to a point where it's going to be very painful to manually issue certificates or... the fact that you as an administrator that uses Stalwart do not have to worry about that. mean, that's a, I would think that's, that's like almost like a huge selling point on its own. Yeah, organizations use a proxy for certificates, but yeah, by allowing Stalwart to handle your certificate, it makes things easier. Yeah and then one like even if you don't know about mail protocols one thing that most developers do eventually learn is that the emails themselves so the recipients and the senders ⁓ that identifier is inherently complex like Like, I mean, some try to do it with Regex, which is of course not the right approach because what can be contained in a valid email and the meaning of how it's formatted is pretty complex. So how are you handling it in a stalwart? Like how are you parsing the email identifiers? the email itself. ⁓ Well, an email address, ⁓ basically the trick is by not implementing ⁓ historic functionality that is no longer used. For example, an email address can have comments inside of it, inside the actual email address. So that is something that is not used nowadays and so I will reject them. ⁓ So you cannot create an account with that. ⁓ In addition to that, are certain attacks you can do. For example, you can use confusables, which are a similar Unicode character, for example, an A from using another language, for example, in Greek that looks like the A in ASCII, but it's not exactly, it's not the same. So there is a security. ⁓ ⁓ measure which is removing confusables Is that similar to what IDNA does for domains? Is that the kind of technique you also use for the identifier part? Yeah, yeah, yeah. But yes, that is correct. That also is a problem. Like for example, there is online like an apple.com domain that is using an A from some other ⁓ alphabet. So it's an example to showcase how phishing attacks could be implemented by registering domains that look really like the real one, but they use a different... ⁓ They use character from other encodings to confuse people. And then the email address is, if you are strict about it, which is you're validating as a mail server what addresses you allow to create, it's not that complicated because you just make sure that the top level domain is valid. There is a Rust Crate called PSL. that contains a list of the valid TLDs. So you make sure that it has a valid TLD and that ⁓ the domain part is correct, that you don't have like a dot, multiple dots next to each other, or it starts with a dot. So you do that basic validation. And yeah, that's basically it. Also you need to validate the local part to make sure that the... that the address has not started with symbols and so on. And in the case of Stolework, it will reject ⁓ comments. So that is how the validation is done. But then of course some things I would think you cannot really avoid, like for example the fact that it can be case insensitive or that the dots have no meaning or the plus operator allows you to add extra data so you can have unique addresses at the same recipient. All these things, do you allow them or is it also something you just reject? Yes, the SMTP server allows all that. So you can use comments or you can use mix case in the local part, which for some temporary addresses and so on use mix case local parts. that's allowed. You're allowed to send an email to an address like that. But on the store side, All addresses that are lowercase for simplicity and also to avoid ⁓ security issues, like someone creating an account called admin with a capital A, for example. So for that reason, Instalware as a mail store only accepts the lowercase ⁓ email addresses. Okay, but you do allow unicode if I understand correctly. Okay, very cool. So that's already one less worry. And then there was a protocol you mentioned, which is a sieve, like the sieve filter, or how do I pronounce it? Like what is that about and why is it used? Yes. Yeah, Sieve is a scripting language, for mail filtering. It's a very old specification and it's a very limited scripting language because you don't want users to create a script that can attack your server or cause a denial of service, for example, an infinite loop. So C does not allow to create to have infinite loops. So the only loop that you can create is over the main parts, so over the message parts, which are fixed. You cannot do any other type of ⁓ iteration. And Sieve consists of a base protocol and ⁓ some extensions to the protocol, for example, for vacations, for... regular expressions and so on. And yeah, it's quite simple. So basically it says if this happens, if this is containing the header, and then you have some actions like reject, discard, or file into to move the message to a certain folder. ⁓ Yeah, and Sieve scripts are managed using a protocol called Manage Sieve or Jamf for Sieve. So that is how you ⁓ configure, how you upload these scripts to the server. say it's old and it has limitations, does that mean that in some cases not even used anymore or like what was the future of Sieve Well, no, it is old, like SMTP, but there are no alternatives. is old, but still used. Nowadays, it's the only option to do mail filtering. ⁓ Actually, some email clients, when you have a rule editor, a mail filter editor with a UI, for example, it says, if from matches that, then this, then. do this or do that. Many mail clients, well, the only way they have is if they want to implement server-side filtering is to convert those UI, those rules defining the UI into a Sieve script. So yeah, Sieve is still the only way to do server-side mail filtering. The script is, the language is not bad, it's not really nice to look, but it... to look at but it works. Howcom is the only choice if it's on your server and if you are writing the server like stalwart that's something you just have full control over. Yes, but the problem is that there are two sides on this, which is the mail server and the mail client. And the protocols, as well as other protocols such as HTTP, are standardized by the IETF, the Internet Engineering Task Force. So each one of these protocols that we use, they have a sign, and it's called a RFC number. requests for comments. The HTTP protocol has one, ⁓ SMTP, IMAP, JMAP, even SIV, all of them have ⁓ their own RFC numbers. And ⁓ then ⁓ both clients and servers implement this RFC so they can talk to each other. So I could create my own scripted language, but if it's not standardized, it's not really... It will be hard that clients will ⁓ implement it because there is something supported by just one server and it's a proprietary mean, it will be considered a proprietary protocol ⁓ or scripting language in this case, if it's not standardized. So standardization ⁓ helps adoption because then you have both servers and ⁓ clients implementing the same protocol. ⁓ In fact, is Stalwart my server that ⁓ uses a protocol called MTA hooks, which is assigned by Stalwart. It's currently a proprietary protocol. It was created ⁓ to replace Milter, which is a binary protocol created by Sendmail many, many years ago. And this protocol is a... Milter is a binary protocol. for filtering. So when you set up a spam filter or a virus scanner on your mail server and you want to talk to it, common, the most popular, well, probably the only protocol you have to do this is Milter, which what it does is says, I have this connection from this IP that wants to send this email. What should I do? And Milter, This message is sent over the Milter protocol to the scanner, for example, the spam or virus scanner, and the ⁓ scanner returns a list of actions. It tells unmodification. tells, this message, it is spam discarded or rejected, or if not, it can say, these following headers to the message. The problem with Milter is that it was never standardized, so it's not a standard, it was created by SendMail. It is informally implemented by ⁓ mail servers such as Stoleware, example, also Postfix and ExSim, but it's poorly documented. In fact, there are two versions, one of them, there's not even documentation for it. So if you want to use it, you need to reverse engineer the protocol to understand or code that implements it. And for that reason, MTA Hooks was created with this HTTP-based protocol. Well, that is much easier to implement. currently, ⁓ there is some discussions at the IETF to standardize the protocol. during the next IETF, we will start discussing whether, if it makes sense to standardize this. ⁓ So yeah, anyone listening that is interested in this, they can join the IETF mailing list, mailman, dispatch. to help ⁓ standardize this email protocol as well as others. Yeah, and I also noticed that you and Stalwart are making several proposals yourself to IETF and you even have like an RFC repository for that. Are these already proposed or submitted or is it for now just happening in the repository? Yes, currently I'm working on ⁓ four, five different proposals. Three of them are for the JMAP protocol. One of them is to add metadata support for JMAP. The other one is to extend the JMAP for mail to support sharing. And the third one, JMAP related, JMAP for enhanced references. to use JSON ⁓ path to extend ⁓ the JMAP protocol to do more expressive queries on the responses. in addition to that, is another proposal. Sorry, I forgot to say this. All these JMAP protocols, sorry, these JMAP drafts have been adopted by the JMAP working group, so that is good news. It means that now they are on track to become ⁓ standards. And then there is another standard called object ID, ⁓ object ID bis, which is, I'm working on this standard ⁓ with Fastmail. We're co-offering this extension to obsolete the previous IMAP object ID extension. So this is a draft to fix an old IMAP extension called Object ID. And finally, there is an MTI Hooks proposal, but this one has not been adopted by any working group because it needs its own working group. So in IETF 125 in Shenzhen, the idea is to start a new working group to standardize these MTI hooks. But there's already some interest. A lot of people showed interest, both at IETF 123 in Madrid and also during FOSDEM that was discussed. Yeah, indeed you were at FOSDEM, I was there myself to give a talk about Rama, like how was your experience there and how was your talk received? ⁓ Well, this is my second year at FOSDEM I have never attended FOSDEM before that. Yeah, it was a really nice and rewarding experience. I was able to meet all the clients and other server developers, all the people involved in email. ⁓ And also before that, the day before, we did an email ⁓ hackathon. So where we discuss new ideas for protocols, what can be fixed and so on. And yeah, and the talk itself was, ⁓ yeah, it went well. It's now available on the video recordings available in case ⁓ anyone wants to take a look at it. It's called Scaling Email. Can open source ⁓ scale to Gmail scale. So if you can use open source products run something like Gmail. ⁓ the presentation goes over the different, because a store is just a mail and collaboration server, you also need a database server and so on. So the talk goes over that to see how far we are from having something like... Gmail and what needs to be done to get to that level. That's indeed, I mean in a way it should be the future right because I mean email is very critical and yet most of the people run on a Gmail or some other servers that doesn't mean that if for one reason or another their account is suddenly closed or they are locked out I mean that's a huge problem Yeah, a lot of... There is something that I also wanted to talk about, which is self-hosting your email. A lot of people are scared to do that because there are some horror stories about how complicated it is, how your emails will end up in the spam folder of the recipient and so on. And yeah, I have to say to those that give Stowart a chance because it's now really automated, it's not like before. It will tell you exactly what you need to change, what you need to change in your DNS server to have a fully secure and correct mail server. It will tell you everything. And also it comes with safe defaults, so you don't have to worry. about how it ⁓ is safe setting and so on. ⁓ By default, if you just install it, create an account and update your DNS server with the records provided by Stallward, that's it. You will be running a email server. The main, of course, is not you need some Unix experience. You need to know how to ⁓ connect with how to SSH into a server, do backups ⁓ and all that. And there's also another aspect, which is the IP reputation. You need to make sure that you find a VPS provider, a server hosting company that gives you a clean IP that was not used by spammer default. So if you follow those steps, you should be able to regain control over ⁓ over your data and you will be sure that ⁓ big companies are not using your emails to train AI models or to sell your stuff. And ⁓ in addition to that, Stower supports encryption address. So if you have OpenPGP or S-Mine, you can upload your public key. And before Stalwart writes the email to your, is the email arising in plain text? ⁓ and you have provided a public key store that will encrypt your message using that public key. So even if the server provider wants to read your emails, they can't because they're already encrypted key. So yeah, and now in the future we plan to and you implement the new quantum resistance PGP version 6, the old version 6 of PGP, so you can use the quantum resistance or quantum tolerance, I forgot the exact name, ⁓ that will make it much harder to decode your keys in the case there is a, yeah, if we ever get quantum computer, that and forward secrecy. So once that is ready, will be available as an installer. But in the meantime, the standard OpenPGP or S-MIME is pretty safe. No one will read your emails. For the time being, you can encrypt them and only you will be able to see them. Glen (Plabayo BV)
1:00:27 | 🔗
Okay, and so yeah, I do want to dive a bit deeper into that is because ⁓ I understand the concern, but I also do understand that stalwart makes it lot easier and they're also pretty good guides these days like about what records you have set with like as even stalwart as you said, if it has access to your ⁓ if it has admin privileges for your DNS registry even that you can do for the users. ⁓ but even then I noticed that despite the fact that you have a good IP reputation your domain might be already quite old so it's not a new domain you have all the correct DNS records etc it seems that some big mail providers still give you a low reputation anyway and you end up in spam folders like let's say an outlook server or I don't know like I don't self host myself yet we are planning to and Stalwart looks very interested for that ⁓ but I know some other folks with lot of experience with it and they say despite doing everything right you do anyway end up sometimes in a spam folder ⁓ without any fault of your own and it might even be that one day it works fine and the next day suddenly you send it out to the newsletter and you end up for all Gmail news in the spam folder like what's your experience with that? Mauro De Gennaro
1:01:58 | 🔗
⁓ Yeah, the first time I configured the mail server used by the domain stalwart, our internal domain, ⁓ we had the same problem, but what we did is we tried with a different provider and that fixed it. For example, the first one we were using DigitalOcean. for the mail server and that IP had problems. It did not appear on any block lists. based on those tools that are available online, was not ⁓ bad. It did not have bad reputation, but still the messages were placed in the junk mail folder, the experience provided. And the solution was to try with different. VPS providers until we found one with Hezner. We have an IP address from Finland that not for any, that's no connection to Finland, but is the one that was available and had a clean reputation. So yeah, it is a bit annoying, but it's the only way. It's just trying with different providers. ⁓ have the server for one month, for example, and see how it goes. ⁓ And then ⁓ until you find a server, an IP address that is not blocked. And another option that some people use is to a relay host, like, I don't know, like Amazon SES or something like that. So you receive your emails without problems or, and... ⁓ delivery is done ⁓ using this ⁓ relay provider such as MX2GO, something like that. are many alternatives out there for email relaying. In software also, you can have custom rules. You can say, okay, deliver everything from using MyIP, but for example, for MSN.com that does not like my IP for that use Amazon SES. So you can create custom routing rules for those domains you're having problems with. Glen (Plabayo BV)
1:04:35 | 🔗
Okay. And then, okay, that's, so it seems there is a way and it's sometimes you have to look for it, like you have to do some trial and error, but it should be in general a smooth experience. So that's reassuring. My other concern is a bit, okay. you can, I understand you can take backups like for example that you don't lose your mails how you take those backups I guess in the end it's files so you can use anything including like things like rsync I mean you can really you can back up however you want to as many different points as you want but still let's say I am using Hetzner everything's fine but one day the server is dead and I need to I don't know, move it somehow. And in the meanwhile, I don't receive any emails. That also means if I need to, I don't know, somehow need to receive an email in order to do some administrative work, like for example to log in somewhere. I mean, I'm kind of screwed, no? Like how do you deal with that once you are self-hosting that you have redundancy that even though you, server might not be working, you still have anyway access to your mails while transitioning. Mauro De Gennaro
1:05:48 | 🔗
Well, the good thing is that the SMTP, well, not the protocol, but mail servers by default, if your server is unavailable or returns 400 code, it means that it's not ready to accept your email at that time. So the remote server will retry to deliver the message. Usually it's around three days. It depends obviously on the remote server. So if you have a downtime of for a few hours, it won't be a problem because you will receive the email at some point. However, if you're really concerned about that, what you can do is have a backup host. You can have another software instance as a backup server that will hold. will not. It cannot or cannot, it's up to you. be used as a mail store, it can be used as a backup destination. So what I was mentioning before about the MX entries on the DNS server. So you can have an MX priority one with mail.example.org and an MX priority five with or a hundred, I mean the next one on the list, pointing to mail backup, for example, .example.org. And this happens if, for example, you have a downtime, if your server crashes or you're migrating or doing backup, and mail.example.org is unavailable, the email will be delivered to the backup host, to mail.example.org. And then this server will hold the message forever until you connect ⁓ your main server. And when that happens, it will deliver. all the messages that were received on that host. So, ⁓ yeah, that's way to do it, having backup hosts. Glen (Plabayo BV)
1:07:50 | 🔗
Okay, that makes sense, very cool. And how easy it is as a stalwart user to take backups and do recovery, like can I for example take regular snapshots and just, yeah, pretty much like deploy a new instance, like almost instant, or how difficult would it be for me to... to start from one and move to another because somehow the server provider is that. Mauro De Gennaro
1:08:20 | 🔗
So, Stower supports multiple database backends. It could be like for small host or single server, can use RocksDB or SQLite. And it also supports Postgres, MySQL, and FoundationDB. So the backup procedure depends on ⁓ which backend you have. So in Postgres, would dump, you create a snapshot and you... dump all the tables and in RockCV you can just copy the entire directory and it's with, yeah, that's pretty, very simple. So the backup procedure depends on the store you use. Glen (Plabayo BV)
1:09:02 | 🔗
So I choose the backend and everything, store or stores in persistent memory, that's all in that database. Mauro De Gennaro
1:09:11 | 🔗
Yes, that is a simple use case because you can then use specialized stores. You can have a Blob store like an S3 compatible store. In that case, you are not backing up the Blobs because your S3 store handles usually. For example, if you use Amazon S3, they will handle the fault tolerance and recovery for you. But if you use a self-hosted solution like Garage, You need to the garage instructions for backing that up or might not even be necessary because the store is handling the replication. For example, if you use Seth from Proxmox for the blobs, it's part of the cluster. So if one node dies, the data is also available in other nodes. So it depends on the store. So if you only have, for example, Postgres for everything, ⁓ need to just backup Postgres. But if you use a specialized store, for example, for blobs, perhaps you don't even need to backup blobs if your blob store is already taking care of them. Glen (Plabayo BV)
1:10:39 | 🔗
Okay, very interesting. Okay, you have a way to reassure people that self-hosting is fine. And again, I think it's very important. It should really be, I think it's very dangerous how we're relying on these single point of failure, especially given how... crucial email is. It's kind of like as if nobody had their own mailbox in their house but instead they were all using this central building to receive all their mail. It wouldn't be something you would want, especially for important papers, so it's a bit crazy how we somehow did get into the situation for email. Mauro De Gennaro
1:11:22 | 🔗
Yes, yeah, and the idea is to ⁓ make Stalwart even ⁓ easier to deploy because once version 1.0 is available, the idea is to also include it in the Linux distribution so you can, with one command, can fetch it and also include it as an option for the different cloud providers, for example, an Amazon image, then also a... a heads-on image that you just deployed and that's it. Because also the good thing about Stoleware is ⁓ the person managing the instance is not that familiar with UNIX. They can manage the server entirely from the web using the web admin. So they don't even need to SSH into a server. Well, it is good that they SSH to check the hard drive, how they're doing and so on. But if they want... If they're other ⁓ alerts using a different platform, they don't even need to log in. So because everything that is related to the mail server can be managed from the web. Glen (Plabayo BV)
1:12:38 | 🔗
Okay, wow, amazing. And then something that's also was not always clear to me is okay, so far we talked about SMTP, but often when people, mean, even if they're not experts in mail, but still they might do something basic as choosing what server or maybe like in order to be able to... receive email from different domains in their Gmail, they will still have seen at some point or another that okay there is SMTP but you also have what is called POP3. Can you kind of like explain a bit what POP3 is and how it differs from SMTP? Mauro De Gennaro
1:13:21 | 🔗
⁓ The right protocol to compare it against is IMAP and JMAP because POP3 is a protocol to fetch messages. It's a very, very old protocol, older than IMAP, and it's very simple. It doesn't support folders and everything goes to a main place. So you connect and you see how many messages are available and you download them. It doesn't support synchronization like a changelog. or anything like that. So because of those limitations, IMAP was created that supports folders. So you can have, you can keep your image in the server and have different folders. And then there are a bunch of IMAP extensions to make using IMAP better. So they improve the protocol. There is one called quick resync and conditional store. When you connect to a server, you get a change log. I said, this this inbox, this folder has change ID 123. And you download all the messages. Then when you connect the next day, for example, you tell the servers, what are the changes since change log ID 123? and the server tells you what's new, what are the new emails, the deletions and so on. So you don't have to download everything again. yeah, so it's the... So I would say that POP3 is the first protocol for mail retrieval. Then IMAP came after that and... ⁓ a bunch of extensions were created to make IMAP better. then around 10 years ago, JMAP was created to make ⁓ email access even better by including everything we learned from IMAP and ⁓ simplifying it and making email access even more powerful. it does, JMAP... does everything that IMAP does and more and more efficiently. Glen (Plabayo BV)
1:15:51 | 🔗
Okay, very cool. Thank you for clarifying that. And then something that's okay, we already talked a bit about the fact, okay, why is calendars linked to mail? We talked a bit about how it kind of works. We talked about the base setup, like in a bit I want to get into the authentication part a bit more, but before that, next to sending emails, let's say if I'm self-hosting as a company, there are two other use cases that I typically would look for. The first use case would also be want to have like let's say I create like I have a maybe like a product the product of the websites and I might want to have some shared inboxes like for invoices or for support and in my experience like in most traditional mail software that's not very easy like let's say if we would use something like fastmail okay you can create shared emails but we don't really like we that would mean that we receive both the email of let's say a support ticket but we would not really see what the other maybe colleague might have sent because in the way they are still separate so not even sure how it works is there a way if you store that you can like really collaborate fully on a single inbox for certain email addresses. Mauro De Gennaro
1:17:18 | 🔗
⁓ Yes, Stalwart supports groups. ⁓ so when you're a member of a group, each member has access to the same folder. So they all see the same messages. And this folder also has ⁓ a group address, shared group address, email address. And this folder is then displayed Glen (Plabayo BV)
1:17:42 | 🔗
Okay. Mauro De Gennaro
1:17:47 | 🔗
when you connect using using IMAP or JMAP, this share folder will appear on your client. So that way you can collaborate with other users. Glen (Plabayo BV)
1:17:59 | 🔗
Okay and I see like in vocals Frembo right now we use we use fast meal but for a long time we really want to switch to something like self-hosted ⁓ And I see, okay, I see folders for example, also in fast mail. And indeed we all receive like we have our own company and all my employees and my co-founders, we receive those emails, but we don't see when someone sends like something. So for now we are just like using BCC to make sure the other person sees or what we reply maybe like to a support ticket to a partner or whatever. but because otherwise we don't see the sending. Can you explain that and why that is that something that you do have that I can also see the sending? Mauro De Gennaro
1:18:52 | 🔗
⁓ Well, there is something up to the mail client. It's not a server functionality. ⁓ What is happening here is that the client is placing the send message in your send folder instead of the group folder. So ⁓ in Stallwart you have, there is a send folder for groups. ⁓ The thing is that the client is the one placing the message there. So it's not... ⁓ is not added there through SMTP. So what I'm saying is that when you send an email over SMTP, the server will not place that message in the send folder. It is done by the mail client, for example, Apple Mail, Thunderbird, and so on. So when you send an email, ⁓ Thunderbird or any client will send a message using SMTP or JMap submissions and then place a copy of that sent message in the sent item folders. So what you need to make sure is that your client, when you're using the group identity, when you're sending the email as a group, or always, or when you use any of the group aliases also, that that sent item is placed in the sent item folder of the group and not your personal one. So it's more a client. configuration. Glen (Plabayo BV)
1:20:21 | 🔗
And when you see these things like email threads, so where you see the different emails within the same email thread, does that mean that the reason why you see your email is because you're visualizing it from the send folder? Mauro De Gennaro
1:20:39 | 🔗
Exactly. Glen (Plabayo BV)
1:20:41 | 🔗
And because I'm not using like, because it's, mean, I'm usually even just using their official clients or their web client. And so I wonder why they don't just do it. like, and I never saw even an option for it. ⁓ it seemed like a limitation of, just that, that thing. So, ⁓ yeah, I'm a bit confused because what you explain, it seems to be pretty trivial to do. So I'm pretty puzzled now. Mauro De Gennaro
1:21:07 | 🔗
yes, it's a, I guess it's a function ID you, you will have to request that a message is sent from a group. ⁓ The sent item is placed in the group and not in your folder. Glen (Plabayo BV)
1:21:24 | 🔗
Yeah, yeah, but I mean, much better would just be to switch to stalwart. Mauro De Gennaro
1:21:29 | 🔗
yeah, but then you depend on the client you use. If you use Thunderbird you need to make sure, for example, that Thunderbird is placing the, when you're sending an email from the group identity that is placed in the message inside the group folder, not your folder. Glen (Plabayo BV)
1:21:44 | 🔗
Hmm yeah, well at least it's in my control because now I don't really well, I guess I could already use ⁓ a custom client and also have it like that, but Yeah, okay, I suppose that makes no difference then. ⁓ Okay, cool. And then I said like there are two use case after, so one is these groups. So seems like you cover that quite well. The second use case I would usually have is like I might have some also web application or some other software running and it might sometimes need to send emails for example for two factor indication or registration or ⁓ I don't know, like a newsletter or whatever. And so is there also a way for me to make like, let's say API keys or like various scoped accounts like that can then send emails on behalf of some addresses. Mauro De Gennaro
1:22:39 | 🔗
⁓ Yes, Sober Support application passwords. So you can create ⁓ an account, ⁓ you can create a special password for another product or ⁓ client. ⁓ then the next version, 0.16, will allow you to different permissions or reduce permissions for that application password. So for example, if you have an email client, a tool that only sends emails, for example, like a third party service, that you want them to use your account to send emails as you. You can create an application password and ⁓ limit the permissions so that they cannot read your emails. They can only ⁓ send emails, so only submissions. Glen (Plabayo BV)
1:23:37 | 🔗
Okay, yeah very cool because for example we also use some invoicing software and there they also by default use their own like let's say address for sending invoice but obviously it's a bit unprofessional and so we would rather have it use ⁓ our own email address or some customer, at least at our own domain. But yeah, in many places that's not really something you can do unless you or pay for a separate user. And it seems like here as you sell off Hosted and sell off DoIt and the fact that you support it, that would mean we can actually do that kind of stuff. Mauro De Gennaro
1:24:15 | 🔗
Yeah. Glen (Plabayo BV)
1:24:17 | 🔗
Okay, very cool. And then the thing like that we didn't, I mean there many things we didn't touch on yet, but... ⁓ Yeah, I was looking to check on the authentication part, but then I also found that you have these other crates, which was called, like I think it was called MailSender, and it's an email delivery library. So I guess that's the one I would use if I wanna send emails, and it says email delivery library for Rust with DKIM support. Like, what is DKIM? Mauro De Gennaro
1:24:39 | 🔗
Medicine, So, DKIM is a standard to authenticate senders. It creates a cryptographic signature with the contents of your emails. So, you can look at them if you ⁓ check any email that you receive and you look at the headers, the raw headers, you will see a line called, a header called DKIM signature. So that DKIMP contains a cryptographic ⁓ signature of the contents of your email and headers. And ⁓ this signature, the public key is available on your DNS server. So for example, if John sends an email from example.org, ⁓ A signature will be added to the header saying, okay, this is the DKIM signature. It was signed by the domain example.org and this is a signature. So the mail server will then check the DNS records of ⁓ example.org and for that signature in particular, and then will validate the signature using the public key available on your DNS server. So ⁓ if the signature matches, it means that that message was ⁓ likely sent by you ⁓ from your domain, at least from your server, because your server is the only one with access to the private key to sign that message. ⁓ But there are a few issues with DKIM related to forwarding and replay. And now the ETF is working on DKIM version 2, which will fix all those issues. ⁓ So yeah, hopefully, I don't think that will happen soon, but hopefully in next few years we'll have DKIM 2. Glen (Plabayo BV)
1:27:06 | 🔗
Okay, because as far as I know, DKIM was already supported in the mailauth crate Does that mean that the mail sender makes use of the mail outcrate? Mauro De Gennaro
1:27:18 | 🔗
Yeah, the mail-off crate is a one you can use you can sign your messages using mail-send which links the mail-off crate or if you're interested in just signing and using something else for sending you can use directly the mail auth crate Glen (Plabayo BV)
1:27:33 | 🔗
Okay and then because the mail auth crate is not only focused on DKIM but also ARC SPF and DMARC and I guess there must be some relationship between DMARC and ARC even that they have the same ending. Can we get like a quick overview about these protocols? Mauro De Gennaro
1:27:54 | 🔗
Yeah, ⁓ ARC ⁓ is an experiment, is classified as an experiment. It's used to ⁓ when forwarding, when there are intermediary hosts forwarding an email. So it creates ⁓ a trust chain of all the servers that forward the message, keep adding their signatures and sealing the messages. The problem with ARC is that it has a few issues. is not very effective, the ARC experiment is ending. It's called ARC stands for Authenticated Receiver Chain. So DKIM2 is going to replace the ARC, but before that, ARC is going to end. And then SPF, ⁓ it means Sender Protection. ⁓ framework. It is also another standard that works with the DNS. So basically, what you do with SPF is you tell senders which are the ⁓ IP addresses that your domain sends email from. So it is quite not super complex, but it has some complexity because you can include other domains, ⁓ create rules. ⁓ and tell the other server what to do. Like if you have an SPF ⁓ record that says, only send emails from the IP address one, two, three, four, and you create a rule saying what happens if that is not the case. If the server should accept the message anyway or reject it. And then finally you have DMARC, which combines SPF and DKIM. to trust the domain, to find out whether the domain is trusted or the email you're receiving is trusted or not. So for an email to be authenticated, that's the right word, it has to pass either SPF or DKIM. ⁓ So DMARC also works using the DNS and you say, For example, example.org ⁓ has DMARC enabled and it has a reject policy. It means that if the message does not pass, the key or SPF then reject it because it was not sent, likely was not sent from my domain. ⁓ Then the mark has another extension for reporting that you can get ⁓ emails. with from other mail servers telling you ⁓ what happened with DMARC, how many messages paste past DMARC checks and how many didn't and from which IP addresses are sent. So you get ⁓ aggregate reports every once in a while from other servers with the DMARC authentication results for that period. Glen (Plabayo BV)
1:31:12 | 🔗
hmm okay very cool and then we also have the ⁓ is it like a crate like the spam filter repository I'm not sure it's a crate no it's more like a set of rules it seems ⁓ yeah Mauro De Gennaro
1:31:31 | 🔗
Yeah, that is used. Those are the rules used by Stowaware Mail Server, but they're not, it's not a Rust create. cannot be used. Glen (Plabayo BV)
1:31:39 | 🔗
but okay and and is there because these are or or tomal files it seems or no rule no the list is tomal I had even the rules are tomal files yeah Mauro De Gennaro
1:31:50 | 🔗
Yeah, those are converted. Those are stored expressions that are compiled and tell the spam filter what to do. Yeah. Glen (Plabayo BV)
1:32:00 | 🔗
and as then compiled into CIF or as still something else. Mauro De Gennaro
1:32:04 | 🔗
Not only an internal faster representation. Glen (Plabayo BV)
1:32:08 | 🔗
Okay, okay, okay. And then, okay, so that's cool and I can kind of see how it works. I mean, people can look at the rules themselves to understand, like it seems pretty easy to read. ⁓ Now, another thing that is even easier than checking emails about whether it's spam or not is things like IP reputation and whether the domain is new or not. these kind of reputation and DNS lookups is it also something that stalwart does on behalf of the users to maybe like reject an email from a domain if it's like I don't know less than a day old or something things like that Mauro De Gennaro
1:32:50 | 🔗
Yes, those tricks are done as part of the spam filter rules. there are a few server DNS blocklists that do that. Glen (Plabayo BV)
1:33:00 | 🔗
Okay, okay, because I didn't immediately see that. I also know you have some LLM rule, which is basically just a set of prompts. How is like Stalwart integrating this? Is it like running it locally via something like Rig or I don't know, like how are you integrating LLM into your products and how useful is it really? Mauro De Gennaro
1:33:24 | 🔗
Well, yeah, they're quite useful, but they're expensive to run. So if you use like an open AI, yeah, you will have to pay for the credits, obviously. And ⁓ that is kind of OK if you have a small server. But if you're running this at scale, it will get quite expensive. So at that point, it is recommended that you run the model locally, also for privacy reasons. Glen (Plabayo BV)
1:33:53 | 🔗
Exactly. Mauro De Gennaro
1:33:53 | 🔗
But yeah, you need a good hardware that can handle that. Glen (Plabayo BV)
1:33:57 | 🔗
Yeah, but I mean even most cloud providers provide ⁓ specific like VPS service for that. Yeah, but I guess it's still expensive I suppose. But even then, I think as a company, mean, because so far we were discussing, okay, you can sell a host, but you would still so far mentioning like I would use an digital ocean, something like Hetzner But I mean, if you have your own office, like why not just have your own server in your office and get like a static IP from your telecom or they like concerns regarding that because at that point you can just... Mauro De Gennaro
1:34:07 | 🔗
Yeah. Glen (Plabayo BV)
1:34:32 | 🔗
have whatever hardware you want and like to my experience if you just have a decent ⁓ individual computer I mean that could easily run a mail server which can handle a local LLM without too much cost and you would have you wouldn't have to pay any I mean it would be truly be self-hosted right Mauro De Gennaro
1:34:51 | 🔗
Yes, that is an option. ⁓ But yeah, to receive emails, so you have your data right next to you. However, many servers do not accept delivery, email delivery from ⁓ residential IPs. So you will need something like a relay host to send emails. Glen (Plabayo BV)
1:35:13 | 🔗
But if you request a static IP for a company from a telecom, don't... Is it still seen as essential, I guess so. Mauro De Gennaro
1:35:22 | 🔗
Well, no, if you have a commercial IP, no. But if what you mean is your own home IP address, then it's probably marked as a residential IP. So many servers won't accept it. Glen (Plabayo BV)
1:35:40 | 🔗
I mean that's a shame because I would think, I imagine like a very, in my pink colored glasses, I imagine a future where maybe you or some other company would just prepare like plug and play little books that you can just plug into your basement. And now you have as a family, your own secure local mailbox and... I don't know, delivery system. I mean, I would think that should be like almost like a basic right to protect your citizens. Mauro De Gennaro
1:36:10 | 🔗
Yes, but it's due to spam, many remote servers will not accept your emails. That can be solved by having the VPS ⁓ relay host also running stalwart that sends your emails. If you're having problems, I if your IP is not marked as residential, then you don't need it. But if it is, you will need a VPS host in the middle. Glen (Plabayo BV)
1:36:36 | 🔗
could you also just have instead of like a full flashed relay could you also just have a proxy or something like an IP proxy where you would just say I just proxy my or tunnel my raw TCP traffic and it just comes out of this ⁓ valid cloud IP but it can be pretty dumb it doesn't have to do much because it's just relaying basically TCP packets. Mauro De Gennaro
1:37:03 | 🔗
Yeah, could be, it's not supported currently, but it could be a SOCKS5 proxy. Glen (Plabayo BV)
1:37:09 | 🔗
Yeah, I mean, I have some ideas around that. And, okay. And then I know some, let's say, home lab people, they had, they might be already self hosting other stuff via, via their basement or, or, or they're like wherever in the house, really. But then there was the issue residential, they might not really have, ⁓ aesthetic IP in some countries or they might have other issues. And so they, they use something like wire guards to, to still expose themselves. and have their server being reachable? Is that something that would be easily possible ⁓ via stalwart or that's again like something like not really supported or difficult to set up? Mauro De Gennaro
1:37:52 | 🔗
⁓ No, that won't work something like WireGuard because the remote server needs to access directly your IP, your server over port 25. So your server needs to be reachable on port 25. Glen (Plabayo BV)
1:38:05 | 🔗
And those ports are always hard-coded, like it's not like in, because like whenever in HTTP, okay, by default it's port 80 / 443, but my host part or authority part, or whatever you want to call it, optionally contain a port. That's not the case for these mail protocols, you see. Mauro De Gennaro
1:38:27 | 🔗
And now what you can configure, ⁓ use your own protocol number, sorry, any port number for IMAP, JMAP, but for SMTP server to server is only port 25. Glen (Plabayo BV)
1:38:42 | 🔗
Okay, that's... Yeah, I mean, you would expect that you could just choose a custom port if you wanted to, but I... Because the first time I hear from a protocol where it's really enforced, I know that they have all their reserved ports and default ports, but it's usually just a default. But you were saying that for SMTP it's not a default, it's just the requirement. Mauro De Gennaro
1:39:06 | 🔗
That's correct, yeah. Glen (Plabayo BV)
1:39:08 | 🔗
Fair enough. And so we cover a lot already and I could talk with you for four hours about mail protocols. I mean I've been learning so much. Is there something that you think we should still cover or that you specifically had in mind that you wanted to talk about before we start to wrap up? Mauro De Gennaro
1:39:31 | 🔗
⁓ No, no, I think we've been really thorough. Of course, there are lot of details, but I think the listener has ⁓ a good overview of how it works, how email works. Glen (Plabayo BV)
1:39:49 | 🔗
Yes, and so let's say, okay, there is a listener and I would think there are some that hear this conversation and they are now fully converted into wanting to use stalwart. Like what would be for them the next recommended steps? Like how would they begin learning how to use ⁓ like stalwart, but also they might not really be that familiar with setting up a server or anything like that, but still they are committed they think is the right thing to do are there like resources you could recommend and then I guess maybe you can think of some later that we could also include in the show notes like maybe could be books or could be stuff like where you say like okay if you have this kind of knowledge then you are ready to start using Stalwart even if you are new to the game Mauro De Gennaro
1:40:42 | 🔗
Yes, I will include the link but it's a either the Stalwart repository stalwartlabs/stalwart or the website from there in the documentation. There's an installation getting started ⁓ section with all the all the steps what they need to do to download Store how they can do it and how to install it Glen (Plabayo BV)
1:41:11 | 🔗
Very cool. Well in that case Mauro, I thank you for your time and the fact that you wanted to come talk a bit about how email works, which is despite how important it is to us all, very much a black box to everybody and our goal at Netstack is to have education around protocols and unpack them and... and sure they're no longer a mystery because they are really foundational to us and we have to get a better understanding of them and know how to use them correctly so thank you for that and for all the work you've been doing all these years. Mauro De Gennaro
1:41:50 | 🔗
Sure, you're welcome. Thank you for having me. Elizabeth (Plabayo)
1:41:55 | 🔗
Netstack.fm is brought to you by Plabayo building secure, open, and resilient infrastructure with Rust protocols, and purpose. This show is also made possible by Rama, the open source networking framework. Plabayo offers service contracts and welcome sponsorships to keep building and supporting its ecosystem. The theme music of this podcast was composed by DJ Mailbox. If you enjoyed this episode, don't forget to subscribe on your favorite podcast and leave a five-star review. It really helps others discover the show. for tuning in. We'll see you next time for the next handshake.