On this page On this page
Episode 14 – Roto And Cascade with Terts and Arya from NLnet Labs.
In this episode we have a conversation with Terts and Arya from NLnet Labs. Together we explore their paths into systems programming, the mission of NLnet Labs, and the critical internet infrastructure the organization maintains. The discussion spans DNSSEC, large scale DNS operations, Rotonda, and the Roto scripting language, with deep dives into performance engineering, zero copy design, and building resilient open source networking software. It is a technical episode that highlights the people and ideas behind essential internet protocols.
Learn more :
If you like this podcast you might also like our modular network framework in Rust: https://ramaproxy.org
00:00 01:14 10:37 17:43 25:29 41:06 45:55 49:34 52:05 01:00:36 01:06:34 01:17:14 01:26:06
Music for this episode was composed by Dj Mailbox. Listen to his music at https://on.soundcloud.com/4MRyPSNj8FZoVGpytj
Elizabeth (Plabayo)
0:14 | 🔗
This is netstack.fm, your weekly podcast about networking, Rust, and everything in between. You are listening to episode 14, recorded on the 12th of November, 2025, where Glen has a conversation about BGP and DNSSEC with Arya and Terts from NLnet Labs. welcome in another week of Netstack.FM Today with me is Terts and Arya and I am very excited about this episode. It's also the very first one where we have two guests and they both work for NLnet Labs a very exciting company which I'm afraid not many of our listeners know about yet. So I am also very happy that today our guests are able to shed some light on the fantastic work that they are doing. Some of the projects that we will be discussing is Roto. as well as cascades. But first, welcome Terts and Arya as tradition goes with this podcast, I would like to know a bit about both of you first. So Arya, maybe you can introduce a bit where you come from and how you got into this journey. Yeah, hi. I've been programming all my life, I think, which most of us can say. ⁓ And I started out with Java and C++, that sort of stuff. ⁓ And I very quickly dive into the lower level things. I really enjoyed C, and that sort of dove into learning assembly and stuff. ⁓ And I've always really enjoyed understanding how things work at sort of that level. ⁓ And I think about four years ago I picked up Rust and I don't know, I haven't really looked back since. It's just been, it's sort of changed the way I think that I write code now and it's really hard to go and pick up something else again. But I sort of ended up here through a slightly weird route. I think I... met Terts and a few other friends ⁓ at university ⁓ where I sort of, I just looked for people who also really enjoyed Rust and suddenly I was part of this group and it turns out that they had begun volunteering for and organizing Rust conferences. So they've been doing Rust and L, now Rust Week for I guess three, four years now. And so I naturally became a volunteer as well. And then ⁓ I met ⁓ Alex, who is now my boss, ⁓ who was MC-ing ⁓ at the conference, I think, two years ago now. Yeah. Yeah. And so, ⁓ and I just sort of ended up here. ⁓ It was just, I really enjoyed the experience. ⁓ But yeah, I think we'll talk more about NLNut Labs in a sec. Yeah, very cool. And what about you, Terts? How did you get there? ⁓ so, ⁓ I started, ⁓ here in Amsterdam, ⁓ doing a like weird liberal arts and sciences degree that wasn't doing much programming. ⁓ but I, I found this side job where I could do some, some programming on the side. ⁓ and that was just a lot of like Python and stuff. And at some point I found rust and I tried to introduce it there, but didn't really have much luck. But then ⁓ I did my master's in Delft and there at that university indeed I found Arya and a bunch of other people who like, I think the friendship really grew around Rust and like now it's much more, right? But that's how it started. ⁓ And ⁓ then we found that there were these Rust conferences happening. was first Rust and now kind of started as a local meetup. And I just started going there. And Jana, one of our friends was already involved. And then I sort of naturally rolled into organizing that too. So we did a couple of those conferences now. So we have Rust and now, and that now moved into Rust week because we. Well, we outgrew our local, our local skills, so to speak. It's been fantastic seeing it scale up from a tiny conference in Amsterdam. ⁓ And now it's, it's huge. It's literally fun. Yeah, it's big. ⁓ And then I indeed found Alex MC-ing that conference when it was being held in Amsterdam. And that just stuck in my mind. ⁓ And in the meantime, I worked on the ⁓ Yuyutils CoreUtils project, ⁓ which has now recently been added to Ubuntu. I'm not really involved anymore, but I worked on that for a long time ⁓ and even did some funded work for that. And when that kind of dried up ⁓ is when I was like, ⁓ remember that guy from, you know, that one conference that sounded like a fun company. ⁓ So I sent them an email and they wanted to have me, which is how I ended up here. And that is now almost two years ago. ⁓ very cool and so now that we know a bit more about both of you which is like a very fun and exciting story i would like to know a bit more about the company as well because we kind of got like already some glimpse of it so i don't know who of the both of you or maybe both of you want to give me like an overview of the company and specifically also the mission I'm going to let Terrence handle some of this. So, Enelad Labs is a company that's actually, well, it's an organization, a nonprofit organization. It's been around for quite a long time. It started in 1999. We celebrated 25 years recently. And it's sort of a spin-off of this other organization called Enelnet, which is now very much focused on funding ⁓ other open source projects that somehow relate to the internet. ⁓ And Enelnet Labs was a spin-off from that to do actually more implementation work. ⁓ So that started with implementing ⁓ DNSSEC ⁓ and then ⁓ it kind of ⁓ changed into that we maintain NSD and Unbound, which are these ⁓ pieces of DNS software. Like ⁓ NSD is an authoritative resolver and ⁓ authoritative server. And ⁓ Unbound is a recursive resolver. And those are like, they really blew up. They're everywhere. And those are, of course, ⁓ they were started in like, 2001 and 2007, so of course they're in C. But some of our colleagues had discovered Rust in the meantime before we joined the company and decided that, well, at some point we should start doing our new projects in Rust only. So that's kind of what we're doing now. We have these pieces of software that are incredibly popular and that we really, really want to maintain. So we have a couple of people working on that. And then we have other people who ⁓ make new projects in Rust. And it's small. It's only 17 people. ⁓ And we rely on ⁓ funding from ⁓ several organizations who give us grants. And also we offer support contracts where, you know, ⁓ companies can just have sort of a direct access to ⁓ our development team ⁓ and get help that way. ⁓ However, we do have the rule that everything we do is open source, everything goes onto the main branch, we don't have specialized versions of our software for specific companies ⁓ because we're really in it to, we'll just make really good software and ⁓ help the internet forward. And we also participate in ⁓ shaping the internet standards. So we have a lot of, we attend a lot of conferences, that's also been really fun, but we interact a lot with things like the IETF trying to ⁓ shape how, especially proposals around ⁓ DNS and BGP, so routing software, which is the other thing that we specialize in. And so, yeah, we deal with the implementation side, we deal with the standards, ⁓ trying to bring everything together. Yeah, very cool. talking about IETF, like in episode 11, we talked with Max Inden from Mozilla, and he had also a lot of great things to say about these IETF meetings, which are a lot like conferences, more like around together writing or working in these working groups. So I suppose you're all attending these as well. I mean, so our boss Benno is the chair of one of the working groups. I think it's the DNSop or co-chair, I think, of the DNSop working group. But yeah, so there's a lot of presence of NLNut Labs people in a lot of these things. Personally, we haven't been, but many of our colleagues are very involved in these working groups. Yeah. Yeah, well Max had a lot of great things to say and he said it was like very open and like even if you're like, I mean, even if you know the share but never been involved before, like it seems to be a very open group. So definitely seems to be something that is recommended by him. Now we first introduced BGP in episode 11 when we talked with Ryan from Oxide. That was the first time our listeners learned about and they learned a bit how it worked. Later in this episode I want to definitely dig a bit deeper because a lot of your work is around BGP and it's a very important protocol still. But before that, there ever like... I mean as far as I can see these original C projects still have a lot of traction and they are actually like different from your current rust projects as far as I can see at least they seem to have a different scope I suppose that's on purpose but even so is there like ever a maybe like an idea of like okay once we maybe like have these other new projects at a certain I don't know stable point we also want to port these or is there never like a question around like maybe they will just keep living forever I think there are, we do sometimes think about that, but we also just have people here who are just really, like who know that software really, really well. It's really, really stable and really, really important to people's operations. So ⁓ what we try to do is we try to provide like sort of more tools in addition to those. And maybe a new tool we introduce. can do part of the work of what an NSD or an Unbound would be doing. But in terms of really replacing them, there's no... That's not on the cards. It's not on the cards right now. Especially with... Yeah, mean, between NSD and Unbound. Especially with Unbound, it's something that's being used by a lot of people. you know, we're like... we have to maintain that software. our mission is to continue providing for the internet ⁓ and through internet infrastructure and Unbound and NSD are really, really crucial to that. So they're not sort of going anywhere. We're also not, like the organization also doesn't really believe in... the fully rewriting everything strategy. We're not in the, we're not there to rewrite everything in Rust. Yeah. So the, that software has, you know, many years of tests and bug fixes and running and being used in operation. If you're going to try to replace that with a new implementation, like you might do it in Rust, which helps alleviate some bugs, but you're going to add a whole bunch of new ones. in the process which you'll have to fix and it will be a very long and tedious effort probably. Yeah, I understand. And by the way, I love C. I like I also know why Rust is there. I mean, but in general for me, these are tools. So I understand your reasoning. And then in episode six, we talked with Daniel from Curl and that's a very stable software in C. And I think one way he achieves that is because he is still using like original, like even pre-NCC. So he doesn't like keep with the new standards. he like, because a of times in these like memory and safe quote-unquote languages it's often introducing bugs because they keep touching it they keep changing it they keep updating it so it's your strategy there also like we just keep with the original like version at that C was at that time and we don't keep evolving but we only like fix like issues when they are there but we don't keep like touching it and introducing new bugs or how how is your vision there on that on these projects and. I it's sort of, it's not for us to say we're not the ones maintaining that C code. But from what I understand, they're not ancient C versions. No, I think, I think they do. So the people at the organization who work on that, they do try to introduce new things all the time. It's not make it, yeah. They introduced. ⁓ Lots of like actually right now there's like a big refactor happening that's being reviewed by one of our colleagues that ⁓ you know overhauls some internals and tries to really make them better. So it's ⁓ that's not really the philosophy I'd say. Yeah, well. Well, Refactors in C that does scare me. In Rust, they don't scare me, but in C like, but I guess you have good reasons for it and I'm sure you have very good people. And then the other reason why sometimes people rewrite in Rust is not so much because they want to rewrite everything in Rust because that's so fun, but more because, I mean, similar to some other like quote unquote ancient languages, like at some point it might be very difficult to find very skilled C developers. While it might be, well, not that it's super easy to Rust developers but still it might become a bit easier in the future than then see so that that's another reason but I guess that that's not really a concern for now and maybe I mean are these projects ever also like finished is or but I suppose not because these specs also keep evolving Yeah, that's the main thing. There's always things that are changing. There's always, ⁓ you know, especially with ⁓ NSD and Unbound, there can be security concerns. And there's always, like, that landscape is always evolving. So I don't think where these pieces of software will ever be finished in any sense. Some smaller pieces might be like, it might be on like maintenance mode. But NSD and Unbound, they're They're big and being actively developed. coming back to your point about ⁓ the different reasons for rewriting things in Rust. So one of the things we'll talk about is Cascade, which is our new DNSX liner. And the history there is that we did have a DNSX liner in C before this, OpenDNSSEC, ⁓ which was introduced in about 2015. ⁓ so in that case, we are now in some sense rewriting it in Rust. But the underlying reason for this is more so that DNSSEC has changed. All of these specs have evolved a lot. Best practices have changed a lot. so OpenDNSSEC architecturally is no longer a good place where we're like, we can adapt this to do whatever we want. And so it just seemed like the right time to start from scratch. Yeah. make sense and I suppose also a lot of lessons have been learned and like usually with these projects like once you implement it fully and worked long enough you gain so much domain knowledge that if you would do it from scratch you would anyway do it better and yeah you could kind of repeat it forever if there was infinite money and time but sadly not so yeah actually that's a good segue or not segue like that's a good ⁓ continuation of our conversation let's continue over on Cascades and before that you it's all about DNSSEC but I we never actually mentioned DNSSEC in this podcast so maybe one of you could introduce our listeners to like what is DNSSEC and help them understand where it fits in the picture Yeah. So ⁓ DNS is in some sense a very naive protocol. It was introduced decades ago and it was never designed for an internet where you don't have trust. And so things like communicating in plain text and ⁓ any sort of lack of authentication or verification means that we've had to adapt it over the years to try to account for the way the internet works now. And DNSSEC is a really crucial part of that. DNSSEC is essentially, ⁓ actually I'll start over. So with DNS, you break down information into records. So you might say the address of google.com is this IP address and that forms a single A record or a quad A record. And so you can consider sort of. all of your information in DNS as broken down into these very small individual records. And when you want to query the DNS, you're asking for one or two particular records. DNSSEC tries to, yeah, the goal of DNSSEC is to assure clients that they're getting authentic information, that there isn't an attacker on the way who's changing the IP address that they should be talking to or telling them. this thing doesn't exist or any number of other ways of manipulating this information. ⁓ And the way this works is cryptographic signatures. So for every single ⁓ DNS record you have, you associate it with ⁓ an RR SIG, a signature record, ⁓ which will contain just a cryptographic signature over that material. ⁓ And there's all sorts of additional considerations. It gets incredibly complicated. There's a... As a simple example, you need to also keep track of non-existence. So for example, if I just gave you a signature over a record, an attacker could still say, well, this thing doesn't exist. And there's no way you also have to authenticate the non-existence of things. If you as a user are trying to access, for example, google.com, and then an attacker says, well, that doesn't exist, they can block off that access. ⁓ in a way that you can't tell that something went wrong. From your computer thinks that, ⁓ google.com just isn't a thing. ⁓ But DNSSEC also accounts for authenticating non-existence by introducing these additional NSSEC records ⁓ and then signing things over there. My point is there's a lot of weird edge cases, ⁓ both due to just the history and the evolution of DNS and just because security is really complicated. ⁓ And so that's sort of what DNSSEC as a whole is trying to do. Okay, I understand. So it seems all about like proving like that something is like valid is authorized exists, etc, which traditionally didn't exist. and originally like DNS, and I become perhaps more in a different realm, like in privacy, rather than like security, but like traditionally DNS was served over just UDP. And now they're also moving it to HTTPS. ⁓ this kind of DNS secting is it ever also done in plain text because I guess it could because in the end you're not really leaking anything private or does it not matter Yeah. So, so the original DNSSEC RFCs talk about, you know, what their security goals are. Because you can, the DNSSEC is here to sort of assure ⁓ the authenticity of your data, right? But it cannot assure privacy ⁓ and it cannot assure availability. An attacker can simply not talk to you, right? They can block off your communication. There's no way to work around that. DNSSEC inherently does not specify any means of encryption or sort of hiding your requests. It's just adding more information to authenticate that the responses are valid. So traditionally, DNSSEC is also done in plain text. ⁓ And yes, there are now a lot of attempts with DOH or DOQ to try to address the privacy implications, but fundamentally, it's a really hard problem. and why is it specifically a very hard problem? Well, if a user wants to ⁓ request certain information, right, they now need to ask somebody, some DNS server for that information. That DNS server inherently needs to know what the user is asking. ⁓ And with IP, can always, like, there's a million different ways to trace people between fingerprinting of DNS software, just tracking IP addresses. looking at things like timing patterns, ⁓ things like that, ⁓ it tends to be really straightforward ⁓ if a user's talking to a single server to figure out who they are ⁓ and where they're asking from, things like that. ⁓ And that's... You can try all sorts of interesting schemes like saying, I'm going to ask from different places or I'm going to ask different servers. But ⁓ I think fundamentally there just isn't a good answer. And the solutions that are being proposed are kind of like ⁓ the kind of proxy things, I think. you send a request to some other DNS server that can then send the request for you. that now... means that you have to, it's kind of like a VPN, like you have to trust the VPN provider now or like that other resolver. So ultimately you have to trust somebody who's going to read like your actual request and do another request for you. ⁓ You have to put your trust somewhere. can scatter and you can try to hide and do dodgy things, but ⁓ it's a hard problem. I think another way of looking at it is as a trade-off between privacy and efficiency. So if you were to use something like Tor, right, that could get you that degree of privacy, but it comes at an efficiency cost. And when DNS is being used by literally every single internet application, right, that's not a cost you can pay. There's one more thing that I think we should address here, which is that... ⁓ what these schemes of DNS over HTTPS do provide is that whenever your request is in transit, at that point it's not in plain text. So you always have to trust somebody, but at least with these schemes, it's not unencrypted on the wire. Which does help. Yeah. There are also, I just wanted to quickly mention, are also ⁓ additional, this is something that everybody's thinking about. ⁓ ⁓ Yeah, I mean that's fresh. We have ideas like ⁓ opportunistic encryption. There's the whole Deleg working group that are trying to address that, which I don't want to get into. ⁓ And ⁓ yeah, basically this is something that's on everybody's minds, but it's sort of just a hard problem even on paper. Yeah, yeah, no, that's fair. And I thank you very much, even though I put you bit on the spot, I really like your answer there. Now, as a, let's say, website administrator, it's still up to me to enable like a checkbox in my DNS registry to enable DNSSEC. Like, is there a reason why this is not just the default and you have to kind of opt out of it? And why would you ever not want it? Yeah, that's a good question. ⁓ We're not domain registrars. ⁓ We can't really speak to what that experience is like. I should say that there is a history of... DNSSEC is something that, from our perspectives as implementers, feels like something very obvious. ⁓ But there has been a history where people feel... ⁓ where DNSSEC has hurt people or where it's caused its own to not work properly and things like that. And while the entire community has tried to address that history and resolve those issues, ⁓ some of that, like those emotions don't leave. And working around that is really hard. So there's still a lot of people who might feel wary of DNSSEC, right? And that's understandable just based on the history. ⁓ And so, yeah, I understand that it doesn't feel like a hundred percent thing. also makes things slightly more complex, Like it some operational overhead. You have to now think about, I don't know, keys and things and... there's a lot of operational effort that goes into that. it's not a... know, like OpenDNSSEC, ⁓ when we collaborated to build that in 2015, it was sort of designed as this one button. just use this and suddenly you have DNSSEC and you don't need to think about it. ⁓ And while we still think that's possible and we're still chasing that, there is a lot of inherent complexity that you have to take on when you say, I'm going to use DNSSEC now. And that doesn't really go away. Okay, cool. And so I feel DNS is one of those protocols that carry a lot of weight. It's kind of like saying, implement HTTP. Like you're talking basically about like a million RFCs and I feel with DNS, it's pretty much the same. then, so you have like in Rust, I think the biggest open source project around DNS. is Hickory DNS and they provide a lot around DNS. Is that something you then also work together with? Because I imagine you have the DNS sex signer. Like it's still hard for me to figure out where in such a stack the DNS sex signer fits in. Like is it something that you plug in there? Is it something that you work with such providers? Is it something that you just run separately somehow? And then if so, how is the flow like? Yeah, I try to understand a bit like where it fits in there. ⁓ So Cascade and OpenDNSSEC are called bump in the wire DNSX liners. What that means is you have some DNS operator, right? For example, ⁓ .com or .fr, right? ⁓ And they have a DNS zone, which is essentially just a massive list of here's every single domain name under .fr. And in order to... ⁓ resolve the actual records within that domain name, you should go talk to somebody else. So these are essentially a long set of delegations. ⁓ And that comes down to the way that DNS is structured as this sort of distributed protocol. yeah, ⁓ so within the DNS operators ⁓ sort of internal pipeline, right, they will... collect all this DNS information, for example, through different registrars. ⁓ And then they have a DNS zone, which is a collection of all the records under, let's say, .fr. That can be served by a DNS server internally, right? Just like they might have an internal hidden DNS server, which is like, here's all of our DNS records, right? And normally you would then forward that into a bunch of public instances. ⁓ So for example, there might, you might have 20 different servers spanning the globe, which are the authoritative servers answering people answering queries for what is food or fr. ⁓ Now open DNS stack and cascade as these bump in the wired DNS exiners sit in between those two things. So you might have, you'll have your, your internal primary source of DNS data. And that does not include any signatures. then OpenDNS slash Cascade would consume that data and expose a new DNS server serving the same DNS zone, but now signed with all the additional records you need. And then that would now get picked up by all of your public ⁓ servers. And now suddenly you're serving DNSX signed data. So it's a sort of. But am I then correctly to assume that this kind of like a man the middle proxy or am I seeing it wrong? Kind of, yeah. So it's a daemon that they would be running internally and they just need to point all of these public instances, however they distribute that DNS data at this thing instead of their internal DNS source. Okay, so that also means in case they support the OHH, then they would be the one that terminating also at TLS and doing that part and maybe internally they do it much simpler or something. So they would, so wherever, I'm just going to say Cascade now, but so wherever Cascade sits, they're downstream those public servers, right? Cascade is supposed to be hidden. It's not going to be exposed to the public, but those public servers would then fetch the entire zone contents. And there are efficient ways to do this. There's a diff based mechanisms. And so each one of those servers would probably hold an entire copy. off the zone data and they would be serving that themselves using whatever transport mechanisms they need to. So individual queries never go back to the signer? So the signer is just sitting in one place and it doesn't actually experience that much load. It exports the entire zone to the other servers which then resolve the individual queries and do all sorts of things. Okay. and Okay, okay. And then what I also trying to understand is like, is there, okay, I suppose it makes it easy to adopt or to provide adoption by providing a separate daemon because you can kind of like almost like plug it in kind of like a reverse proxy. But let's say once it's more established, is there also like ⁓ not good reasons to just have it like included in your regular DNS servers? So ⁓ one of the things, and this also comes back to the sort of history and trust for DNSSEC, which is that ⁓ we want to build something, like DNSSEC is really complicated and we wanted to provide a product where DNSSEC is its one and only purpose, right? ⁓ And so there's less of a concern that there's gonna be different interacting parts and like DNSSEC gets lost somewhere along the way. the entire focus of this thing is to just do DNSSEC. ⁓ The other reason to have it as a separate daemon is that it will often need to do work itself in the sense of, for example, ⁓ DNS signatures can expire. So occasionally, even if your actual, the contents of your zone haven't changed, sometimes Cascade will be like, hey, I have a slightly different zone now because it had to refresh some signatures. ⁓ Concerns like that, ⁓ especially I mean, we're not touching on the elephant in the room, is DNSSEC keys. Key rollovers are perhaps the worst, the most complicated aspect of DNSSEC. ⁓ And so having a dedicated piece of software that's just like, I'm just gonna do DNSSEC now. ⁓ And the operator doesn't have to worry about what else it's trying to do, for example. We find that that's been really valuable. And that also provides... ⁓ of more transparency about what it's doing. can view its logs to just see the DNSSEC things. And another thing that we provide is these review hooks where can, where Cascade will expose the zone before it will do its signing and after, and then ⁓ you can inspect those before you actually publish it. ⁓ So because it's this extra step in your pipeline, you can inspect the before and after and then like hit a button to say, now I actually want to this looks good, I verified this with various pieces of software, now we can go to the public. So yeah, so in that way it's really meant for, you ⁓ know, important zones that really need to ensure that their data is correct. Okay, that's very cool. That makes totally sense. And then, like, we originally wanted to do the recording with you folks, like a bit earlier, but you were working very hard on, like, I believe it was the Cascade 0.10 alpha release. ⁓ So congratulations for that seems to have turned out very well. And that celebrates also 15 years of OpenDNSSEC. Now you do state like, do not use them production, but at the same time you offer like paid support contracts. So does that mean that like, imagine if someone works with you commercially, then maybe it does make sense to already use it because you can offer like support and you can integrate it correctly. Or is it like for now actually not at all used in production yet? Thank very much. So it should not be used in production right now. ⁓ It's ⁓ still alpha software. still ⁓ working out. Our sort of goal with the alpha was to at least have, to be able to show people what sort of user experience we're aiming for. And so the internals have, like we're still streamlining everything internally. And so, yeah, we are talking to ⁓ some operators, but they're only trying it out in. testing environments, there's no production usage of it yet. Yeah, so the support contract is ⁓ for us to have the money to develop this further and then we can help them ⁓ with any issues that they find in their testing environments and ensure that this will become production ready. Yeah. so intuitively I would think given how like valuable the software that you make is and how important these protocols are, it should be possible to get from important enough players support contracts. At the same time, I know companies also do not like to pay these. I mean, I don't know, like at least personally, I find them. that they much easier pay like a big enterprise for some bloat software than they would pay for something like this which might give them a lot more value but I'm not sure how your experiences or maybe that's far away from your area ⁓ of work. It is not our area of work. We do talk to Alex a lot, who is the main guy doing that kind of work. I'd say it varies a lot between different companies. Some companies say, like, we want to have support contracts for all our critical pieces of software because that's our policy and we want to make sure that we can talk to developers and get bugs fixed whenever we need. other companies are way more, ⁓ relaxed about it, say, where they say, well, well, as long as we're not experiencing any problems with it, we can just take it and that's fair, right? Like we, we distribute this as open source software and you're free to use it however you want. ⁓ do rely on crowns and funds. We do. Yes. ⁓ but yeah, it's, it's, it's just double, it's just like weird. double thing, whereas like, you can't use it. And we also wanted to be available to, you know, companies with a little bit of last cash. They don't, you know, we're happy to help out and happy to give them software. ⁓ yeah. I think I should also give a shout out to the sovereign tech agency, which has funded a lot of our work. ⁓ In particular, they helped us build up our DNS library domain. ⁓ And now we ⁓ have just started. ⁓ working on another grant from them for certain features within Cascade ⁓ for the production release. So there are these organizations which ⁓ we find, which we're incredibly thankful for, which are able to give us money. ⁓ like, NLnet Labs is reliant on ⁓ this sort of revenue. cool fair enough and then so I want to switch a bit also to Rotinator and like BGP and I think later we will go a bit also back onto Cascade but more in the light of another topic but I first want to do talk a bit about Rotinator before I forget because actually that's originally how I discovered you guys around the language like Roto which sadly I don't find much information about on your website like I find a lot about Rotinator, but Roto is like a bit in the shadows, despite I find it very exciting project, but I suppose it's more a means to an end, so I do understand. In episode 12, we talked with Ryan Goodfellow from Oxide, and he gave a nice introduction to BGP, so I don't think we have to do that again. But we will talk, of course, also about the BGP protocol. Now, before we begin, like... What was the reason that you actually created Roto and that you felt like a need for having to invent a language there because it's not like there is not a sufficient amount of scripting languages already, mean, both in the software world, but also specifically to Rust. So I imagine you had some good reasons there. Maybe start with the Tappanator. ⁓ So first, to clear up a little thing, is that we have some similarly named projects. There is Routinator, Rotonda, and Roto. ⁓ So ⁓ Routinator has to do with ⁓ RPKI. ⁓ That's also providing authentication and stuff to ⁓ the BGP protocol. ⁓ And then there's ⁓ Rotonda, which is our BGP collector. It's kind of this thing that you can use to monitor what happens with your various BGP routers over time. And it kind of can collect that ⁓ data and then allow you to access it, query it, look at it. ⁓ And then Roto is the scripting language that we have for Rotonda. ⁓ Yes, thank you for the correction. I in my notes I wrote Rotunda correctly, but somehow I stopped reading my notes. It is kind of confusing, do agree. ⁓ So the need for ⁓ Roto came from Rotonda. So Rotonda, like I said, is kind of this thing that ingests a lot of information. ⁓ You can kind of just see it as a ⁓ big database that just does BGP and... ⁓ BMP, which is like a monitoring thing over BGP, ⁓ gets put into. ⁓ And ⁓ a lot of data gets transmitted over BGP. ⁓ Like there's these big waves that happen in your network. Whenever something changes, all of these routers start talking to each other and correcting the information across the whole network. And that just kind of is a continuous process. So a lot of data comes in. So first of all, we need a database that can handle that. need a database that can store that information efficiently, query it efficiently. And that's all the responsibility of our colleague, Josper. He's made a great database specifically for that purpose. Now, the second problem we have is that there's a lot of data coming in, right? There's just a huge stream of data and ⁓ you usually want to filter that somehow. So most routing daemons have some way to specify ⁓ filters over data. ⁓ And you can use that, for example, in a BGP router to ⁓ make decisions about what ⁓ paths you actually want to distribute or to accept, or, you know, there's various kinds of things that these operators want to filter on. Maybe they want to change some data. They maybe don't want to fiddle with it. Like you have to understand that this is all all these operators, they're dealing with like physical connections to their peers and like several agreements that they made with other people. So they need a lot of control there. ⁓ So all these various routing demons that are that exist have ways to filter this data. And ⁓ What we wanted to provide is ⁓ also that, but we didn't really think that ⁓ all of these solutions that were out there ⁓ were good enough for our purpose because we need this ⁓ high speed and we also, ⁓ just as important is reliability. We kind of want, when you write a filter, that should just work. It shouldn't just crash. because, I don't know, you're trying to assign a number to a string, right? We want to ensure upfront that there's some correctness to this thing, at least from a type perspective. ⁓ So those two ⁓ properties combined, we want the speed, we want the reliability, ⁓ that ⁓ made us decide that we wanted to go for something that is statically typed. and also compiled. ⁓ And even before I joined the company, this was like an idea that had floated around. There was some prototype of Roto that was very different from what it is now. It also wasn't compiled yet and like, but it was like this filter thingy. And then when I joined the company, ⁓ think they, the ⁓ people here looked at what I had been doing at uni, which was a lot of programming language ⁓ stuff. And they were like, okay, well, you go do that. And I was like, that sounds great. So ⁓ what we did is we looked at this thing called Cranelift, which is a compiler backend written in Rust. It's kind of like an LVM alternative ⁓ with a focus on faster compilation, but less optimization of the final code. And The idea is we make a language that targets that. And then Cranelift has this beautiful thing ⁓ to ⁓ create, ⁓ to just compile that in memory. So it will create this, it will compile it to machine code and that machine code is just available to you and you can call it basically. ⁓ So that's what we're using there. ⁓ And then, so. And over time, this grew into more and more of a sort of serious scripting language because we figured that ⁓ if we want to provide people something that is kind of familiar, we actually have to provide something that is close to what they would expect, like with a sort of more familiar syntax. ⁓ so instead of going for sort of specialized syntax towards BGP, we're trying more to work going towards sort of general scripting language that we just provide a lot of, you know, functions and things like you would do in any scripted language to handle this BGP data. What magic? integration. you're right. the final part here is that ⁓ there are some scripting languages out there, ⁓ but they usually have some like C, FFI. So you, talk to them via, via C or they have to do some deserialization step or serialization and deserialization into the values of the language. Right, so for example, ⁓ most Python values are like dictionaries of various fields and values. So whenever you want to take a Rust value and you want to make that accessible to Python, you turn it into that dictionary and then when you get the result back, you convert it back. ⁓ Because we want... that speed not only within the execution of roto but also passing things to and from roto, ⁓ we have focused on ⁓ making the integration with Rust really tight. So because roto is entirely written in Rust, we can really efficiently map these Rust types to roto and back. ⁓ And you can, so you can register your types, ⁓ your Rust types. in roto, ⁓ which means that that type will now become available in roto if you write in your scripts. And then you can also provide functions and methods and constants and all those things ⁓ to be able to work with those types from roto. So ⁓ from Rust, you can call roto and then roto can call Rust functions again. And that means you can also like pass data including even like streaming bytes or something or that's outside the scope. Sure, yeah. So the one limitation is that whatever data you have, needs to be cloned. So ⁓ Roto will do lot of additional clones of the data just to be able to provide more of that scripting feel because we don't want references and those kinds of things. ⁓ But usually you just wrap your data in an ARC and call it a day. and it'll fit Roto. I think the only actual limitation is that generic types defined in Rust cannot be exported as generics into Roto. But other than that, it's kind of magic. can sort of, like, you've had some demos which you presented ⁓ where you could interact with Bevy, which is this Rust game engine, and control that from Roto because you can just directly expose functions which will talk to Bevy. to the roto code and it's incredibly efficient. No. Very cool. Yeah, because like that's how it is covered too, because we maintain a framework called Rama all about networking. And we were thinking of it as the scripting language besides Wasm to really control like middleware and even leave services. Now those are for protocols like HTTP and similar things, including mail traffic and whatever. But I suppose it should work. The only thing it seems I need to be like loanable or do you see some issues there or things where we might be stretching the scope of what Roto might be about? No, think that's a good ⁓ use case for it. It really shines in these cases where ⁓ you want to provide some scriptability, ⁓ but that script will be called many, many times. Which indeed for the same proxy kind of application is really the case. You write once and then for every request or something you want to execute this code. And that's kind of where it shines because we do this compilation step and this type checking step and then we can provide this efficient execution. ⁓ Funnily enough, there is somebody ⁓ outside of ⁓ Nelnet Labs who's picked up Roto in their own project. They go by ⁓ Algernon ⁓ online and they have this thing. which is also kind of a proxy that you put in front of your web server, and then ⁓ it will try to detect ⁓ AI crawlers. And if an AI crawler is detected, then it will just generate garbage for it, ⁓ because that is actually cheaper than serving the actual content. And so it's kind of to protect ⁓ all these web servers that are now being ⁓ well, that now have high costs because of these AI crawlers. ⁓ And ⁓ so that's actually kind of a similar case where they've integrated Roto and they, ⁓ so the rules for detecting these AI crawlers, that's what they have ⁓ programmed in Roto. And they have both a Roto version and the Lua version. And it does look like Roto is consistently. a little bit faster. Very cool. Yeah, definitely link it to me. I will also put in the show notes. That's a very interesting use case. so another thing I like usually about this language, and I hope it also works for Roto, is the fact that you might have like a proxy running on an end user device or a router or somewhere in the cloud, and you need to hot patch it. So you say like, want like a new version of these scripts. but you say okay we have to compile in Korean list I mean does it mean I can just I don't know embed this compiler inside my process or something and like I receive a new version I compile it if it compiles I can assume because it's compiling and also the fact that it comes from a source I trust that it's fine to replace it I mean is that more or less what I can do with Roto or am I seeing it wrong? Yeah, exactly. Exactly. So the way that you use this thing is that the host application compiles the script while it's running. ⁓ So you can do that. And how you do that is up to you. So you can either decide to just load the scripts at startup ⁓ or, and I did this in the demo that I did, this was for the ⁓ Eurorust presentation that I did, which has now recently also been published. So you can look at that. ⁓ ⁓ What I did there is I on every frame, I did a quick check on the script and what the modification time was of the file. And if that was newer than the last one I had seen, I would recompile it while the game was running or well, sort of visualization. wasn't really a game, but it was a visual thing while that was running. I would recompile it and then just pass that new function to the component that would actually call it. ⁓ And that would actually allow for sort of seamless ⁓ updates to the script. So I could just save a new version of the script and instantly it would update on the visualization. Well, I mean, that's cool. Yeah, I'm gonna link to that talk as well. mean, and originally I come from the game industry and yeah, we would indeed use scripting language like that in the past because yeah, if you had to compile all the time C++ and your entire game engine, would take forever. So yeah, that's very fascinating. And, even doing it every frame, I mean, that's crazy, but could result in some very interesting properties. So that's very cool. But... I was just checking the modification time every frame. So only when that was necessary, I could do a full compilation. But still, would barely notice the... I had a little graph in the corner to show the frame count. And it would dip slightly when a compilation was happening, but you wouldn't notice too much. Yeah, yeah, yeah, of course. Yeah, but I imagine you could even just do it, like you could check it, but then even if it's different, even that you could do, guess in the background and do something like an ARC swap or I don't know how you like replace these things. So yeah, but that's very cool. I like it. And, but what I didn't get ⁓ an answer on is like, does that mean that you need the Crane Lift compiler inside? I mean, is it like something you can use as like a library or how that works? Like, because something has to compile it. Yeah, for sure. ⁓ So the CraneLift is actually just a Rust ⁓ crate. it's just in your binary. If you include roto, CraneLift comes with it. You don't need an additional library for it installed. ⁓ It will just be compiled in. Very cool. And does that mean that you can even compile Rust on the fly and have that as a plugin? Or is that something? Am I going too crazy here? Not really. So Cranelift is only a compiler backend. So Rust can be compiled with Cranelift, but you need the Rust compiler frontend for it and the rest of the Rust compiler. ⁓ Cranelift is only a part. It's like the last part that does some internal representation that they have, some special simplified language. and they turn that into machine code and the Rust compiler can also output that ⁓ but you need the whole rest of the compiler toolchain to do that. Okay, and as a Roto user do I have to even care about the fact that it's using Crane Lift? Do I just import the Roto crate or do I also import something like Crane Lift myself? It's just Roto. And Roto will do all the compilation for you. Well, that's very cool. yeah, I mean, it sounds super exciting. You can definitely shout more from the roofs from it because among all the Rust scripting language, I think it's very exciting and now how you're describing it, only make me more excited. yeah, I mean, I honestly don't think many people know about it. Like, I guess your EuroRust talk helped a bit there, but yeah, definitely keep shouting. Are you good to? Yeah, yeah, a little bit. I think it is ⁓ picking up some traction, but ⁓ it's also new, right? ⁓ It's new and it's still lacking some features. For example, right now I'm really working on implementing lists, ⁓ which ⁓ I guess people sometimes look at a language like this and say, but it's lacking this one thing, so I'll just... shelve it. And ⁓ so part of what I'm really doing right now is trying to see what features people really need from this to have it be useful for them. But it is a new project. And also ⁓ what is interesting is that because it's this compiled thing, there are some things that I think the sort of more interpreted languages, dynamically typed languages get for free that we don't. So for example, with, with, these interpreted languages, you can, you can do lists quite easily because while everything is just a value in your language, doesn't really matter what kind of value it is. If you have a list, you can add anything to that list. It's fine. Right. You could want to add the string. Fine. Add a bull. Fine. It's still fine. It's now a list, a list of strings and bulls. but What I have to do is I have to ⁓ look at, I first have to implement like some kind of generics, right? I to, it's a list over a certain data type. And then I have to compile that properly and make sure that I get all the things right there. ⁓ I need to make sure that this list works for any data type that you pass into it. It allocates the right amounts of memory. ⁓ That just makes it. makes some features more difficult and slow to implement than other languages. I've been helping you out with some of the... Implementing generics in a language is just so complicated. And I've been helping Terts out or we've been talking through a lot of these features. It's just like, this is so much work. Especially with the... mean, yeah, as you're saying, lists... in terms of a runtime are one thing, but when you need to now start type checking them, right? doing code-chen, it gets really complicated. Yeah, for sure, I understand. As a user you don't think about it, but when you implement these languages it very much becomes complicated. So props for you on that. I would hope that you also get some external contributors and maybe some people funding, if they actually commercially use it because it's similar to Rotonda or Cascade. It seems like something worthwhile to fund if you rely on it, so why not? That would be lovely. What we do also see is that we've been showing off Roto whenever we've been showing off Rotunda. And that is also kind of interesting to see is that it provides us, by being the sort of general scripting language, it provides us so much flexibility ⁓ that we can say like, you can also do this quite easily because, you have a scripting language right here, so just do it. ⁓ And so it's been kind of a nice selling point there. ⁓ And so it does also feed into like the funding for Rotondi. It's very much part of that project. Okay, and then like BGP collects are quite interesting and also, or yeah, still like I've not been in contact with them a lot, like as I'm using them myself. I know in general, like how BGP works, I also know where it fits. But then I suppose it also means that you collect historic data and make it available or not. ⁓ Well, depends on what you mean by make it available. ⁓ Well, not you, but the one that you would use Rotonda, they would have also historic data, I suppose. They would, yes. So Rotonda collects this data ⁓ and it does that by first keeping it a memory and then whenever something gets overwritten, it writes it to disk. And that's how you can sort of have this big database of what happens over time and you can query that, export it however you want. Okay, and I think that's the kind of stuff that's used by researchers and ISPs. Is there like also an actual real-time operational use case for that? Yeah, I think so. indeed researchers and ISPs, but also, you know, it's nice for people to see what's going on in their network and how things have changed. like it might show you, okay, let's say something goes wrong in your network. You're an operator and something goes wrong in your network. Then you might want to see, where did this thing originate? How did the other routers react or How did they try to mitigate that problem? How did that fail? What exactly happened between all these various stations that you have? And if you have them all hooked up to Rotunda and sending their ⁓ data, then you can actually see that. Okay. Yeah, mean, and sometimes these things do happen. Like you sometimes hear about BGP fallout. I think the biggest one from recent history that I remember is the one from Facebook. Is that something where a BGP collector was probably also used to help them diagnose it and figure out what went wrong or how would that work? Probably, yeah. I think to diagnose sort of retroactively at least, ⁓ think there's ⁓ also things like what they call BGP looking glasses, which is sort of a collector without history, just to see what's currently going on in your network. ⁓ Those are also very popular to just have a sense of what's going on. ⁓ But then definitely, they have some collector thingies going on to diagnose the issues when they go wrong. Okay, and then I mean, BGP is so, I mean, despite most people don't know about it, we try to, of course, shed a bit of light in this podcast on these kind of protocols as they very fundamental to our world. Like, I imagine there must be already plenty of other BGP collectors. Like, what was then, for example, the reason that you felt it was important to start Rotunda? So there are some other pieces of software that do things like this, ⁓ but I think what happens usually is like you have these sort BGP demons. There's various things like that that just ⁓ they do. So a BGP collector and those kinds of things, ⁓ what they kind of do is they pretend to be a router usually, but they don't send anything. they don't send any updates. they just ingest, they start BGP sessions with their peers and then just ingest that data. And that certainly happens. And then people put that in ⁓ other databases and stuff. ⁓ So I think there's like GoBGP and some other pieces of software. ⁓ I think also some Python stuff ⁓ that people just kind of modify or use like plugins or things to then essentially pipe that information into databases and so that they can they can query that. Rotunda is meant to be sort of a more of a one-stop shop where we do all the BGP sessions for you, we keep track of the BGP state, like what the current routes for everything are and that kind of stuff, and we keep track of history for you. And by doing all of those things together, we can be very efficient ⁓ and ⁓ just sort of provide a more comprehensive package in that way. I think that's the main ⁓ difference from the existing software. Okay, that sounds like very good reasons to me and very exciting. I also wanted to talk a bit more about like all like, I mean, it seems that all your projects operate at an enormous gigantic scale or at least are meant to be like you're like piping a lot of traffic through these kind of projects. ⁓ They all you also don't want to spend too much resource on these. So that brings us automatically in a territory of like zero copy handling, SMD, all those kinds of things. Like, I mean, I imagine in both projects, so I leave it up to you. how, like I want to first start off in like, what kind of special considerations do you take into this? Like making sure it's like performing enough that you are not allocating too much besides the fact, okay, you're already using rust. It's already like not garbage collectible beyond that. Like how do you go the extra mile there? Did you want me to? I'll leave it up to Arya. This is Arya's specialty. Optimization is my favorite thing. yeah, like understanding how do you build something that can scale in this way is really difficult. ⁓ And you can look at, there's so many different levels that you can optimize at. You can optimize your network bandwidth. You can optimize the... even the asymptotic complexity of the algorithms you're using, you can optimize ⁓ the system calls you're performing. You can optimize your data structures, how much memory you're using. You can optimize how many allocations you're performing. ⁓ And ⁓ at the lowest and the best level, you can literally optimize individual instructions. And it's all about what does your application look like? Where are the hot pots in your application? What do you really have to think about? ⁓ with, I'll quickly touch a little bit on the Rotonda case, which I haven't really worked on, but ⁓ the person who mostly spends their time on that, which is Jasper, ⁓ has talked to me a bit about the performance considerations for Rotonda and especially their internal database, the Rotonda store. And ⁓ yeah, the sort of biggest driver there is not the network bandwidth, it's not really system calls, it's about the data structures. Right. And it's about, well, how do you take, just how do you, how do you store less data? So for one, like one really small, but interesting example that we, we saw was, ⁓ within, within the toronto store, you have to keep track of, I think it's for every single router peer you have, you might store some auxiliary data, which is essentially just a byte buffer. And previously inside the toronto store, it was being held as a VEC of U8. Right. Now I'm not going to like, now, so just thinking about, have like a hundred thousand or a few million VEC of U8s, right? What does that translate into? A VEC of U8 in place is 24 bytes on a 64 bit computer. and that's just overhead. That's not the actual data that you're thinking about. And so now automatically that's, know, 24 megabytes or however much, ⁓ however much data that you like, that is not inherently. part of the data you needed. It's just a bunch of pointers and capacities and lengths and stuff. And so how do you optimize that? So one of the things we notice is, well, we can actually cut off, we can merge the capacity and the length. This is not something that needs to grow dynamically. So you essentially replace it with a boxed slice, right? So a box of slice of U8 is 16 bytes while a vec of U8 is 24 bytes. And that's because a vec will hold some additional capacity, which you can now add into. or removed from, and we didn't have that need. So suddenly we dropped from 24 bytes to 16 bytes. And then we noticed, well, actually we have this hard limit that all of our data will fit within 64K. And so we can actually go further and say, I don't need a 64 bit pointer and a 64 bit length. I need a 64 bit pointer and a 16 bit length. And now we've dropped to, so now we have this. a specialized type which handles these byte buffers and it's exactly 10 bytes in size. And just to quickly iterate why this is like why these few bytes improvements are really important is what this ⁓ store is doing. This is keeping a full table of like which IP address can like what path do we need to take there? ⁓ And I think it also does that. her peer it talks to. So if you talk to three routers, it stores three versions of each of the paths. Now you can imagine ⁓ if you need to do that for every, even just IPv4 address, right? That is already a huge number of entries in your store. And then we also have IPv6, which is even way more possibilities. So like in terms of what the retina store needs to think about, right? my understanding is that you might have gigabytes or tens of gigabytes of data in memory there. And so when we're making such a small change of saying, okay, I took my back of UA and now it's 10 bytes, that's like a two-fold reduction of this one component. But that can mean a lot in terms of reducing the overall data structure sizes. And then you can go further and you can say, well, I'm gonna actually coalesce all the data between all of these. I can try to reduce the number of allocations I was performing. can try to group these in different ways. ⁓ Like those sorts of data structure optimizations are often the biggest and most ⁓ important things that you can optimize in most applications. If you're ever in case where you're like, I have a gigabyte of data, or I have 10 gigabytes or so, and sometimes you'll run into those cases with DNS software especially. ⁓ And then you're like, well, how do I really tackle this head on and try to find ways that I can make all sorts of incremental changes? Because even something as small as saying, my 24 bytes has become 16 bytes, right? Can save you gigabytes, just depending on what your application is doing. Depending, of course, on which part of the software you're looking at. We only do this kind of optimization for, well, we're talking specifically about the Rotunda store because that is... that has those kinds of characteristics where that is really important. ⁓ But then you also, I think ⁓ we, there have been some attempts to look at zero copy for your Rotondastore but I'll mainly talk about that in the DNS context. ⁓ So within DNS, ⁓ you run into all these really interesting use cases. For example, ⁓ so .com, right? The .com zone is the biggest zone. It's sort of the worst case scenario for any DNS software. right? In fact, to the extent that most open source public DNS software doesn't try to address the dot com case, like dot com will contain, I think it's around 300 million records, something like that. Uh, like 300 million. Um, well within there is all of the domain names and there's also, um, like the auxiliary data that you need to actually use those domain names. How big was that? The zone file you got, how many? ⁓ so for research, they actually, so normally.com is not like a publicly available file containing every single record. ⁓ but for research, they do, there are different ways to access it. So for example, we have a text file here, which is just the.com text file. It's I think 24 gigabytes. And, ⁓ and then you're like, well, if I want to, if I want a DNS server to serve.com, right. I need like 128 gigs of framer or something, you know, like even like. in some ways that that textual zone file format is more efficient than what you'll be storing in memory to serve this, right? You need to be able to access that information efficiently. And then suddenly everything gets way bigger. And so one of the things that I've been slowly approaching is how do you optimize these data structures? how do you not only improve memory usage, but also improve performance? ⁓ I think one of the things that most people don't realize or that people ⁓ misunderstand for whatever reason is that optimizing your data structures, reducing memory usage will almost always also improve your performance. Like these days, if you have gigabytes of stuff, your biggest bottleneck is memory access, almost always. And so just storing less data means that things stay in cache better. ⁓ Like your access will get more predictable. And in the end, you'll just also have better performance. ⁓ In fact, it's really hard to optimize the data structure so much that now you're like any further optimizations to make will either improve memory usage or performance. It's actually really hard to get to that level. ⁓ for, so I'll quickly touch on just like some of the vague optimization techniques that you can sort of strategies that I really like applying in these cases. ⁓ One of them is to just try to specialize and segment your data depending on what it is. With DNS records, especially for the big zones, you'll maybe have four or five different kinds of DNS records, right? It's not actually a wide variety. It's just like, I have these four or five things. I can actually now build specialized structures to manage each one of these. And then within that, I can now try to optimize like individual bytes away, right? Optimizing individual bytes, for a certain kind of DNS record can be a massive improvement because you might have a hundred million of those records. ⁓ And so as a simple example, ⁓ within DNS signature records, which do contribute to that 300 million, ⁓ all of those, the sort of wire format representation of ⁓ those DNS signature records include some redundant information. it will tell you, for example, ⁓ key, which cryptographic key was used to sign this record, which is probably the same across all of your records, right? You usually only are signing with a single key. And so moving that data out means that you save, you know, I think five or six bytes ⁓ over a few million records. That's a massive improvement. Those sorts of, yeah, so essentially specializing, understanding what your... data looks like and then specializing to store that in, yeah, just in better ways. That's often like the biggest, most important thing you can do. ⁓ But that's all from a DNS storage perspective. ⁓ So I did, actually listened to the episode 10, I think it was with Joshua Lieberfieser about Zetacopy and I found that really interesting. So. Everything I've talked about right now was from the perspective of a DNS exsigner or a DNS server, right? Where you just have this massive DNS zone and you're just working with it. But that's not actually the case for most DNS software. Usually you're talking about, I am a DNS client or I'm a DNS resolver. I'm just communicating over DNS, right? That's your main use case. ⁓ So, NLT Labs has their own DNS ⁓ library, right? It's called Domain. And it's sort of a general toolbox almost where it will provide facilities for handling zone files, for representing all your different kinds of records, ⁓ for ⁓ performing DNS queries and for serving DNS. Like it has different facilities for all of these things. And when I joined the company about a year ago, one of the things I was most interested in was ⁓ the in, it was in domain and how Like, are there ways that we can make this more efficient? Right? Because the core facility that this thing needs to do is network communication and transferring data over the DNS wire format. So ⁓ I made a big push ⁓ for changing how the domain API looks so that it can apply zero copy in a lot of cases. ⁓ Along that way, along that journey, I also spent a lot of time looking at the zero copy crate. ⁓ And I think in that episode, Joshua had mentioned that ⁓ zero copy can't handle variable sized data or variable sized fields. ⁓ Yeah, correctly they do that in a separate crate which is not on Crate.io. Exactly. Yeah. And our problem is that DNS is full of variable size data. almost every single thing in there is variable size. it's so while I initially did try to use zero copy, it just was not feasible. ⁓ Like I could not represent enough things using that. And so what ended up happening was ⁓ this in-house zero copy thing has grown as a small little module. now within the domain code. And it's more flexible than zero copy. It doesn't try to address exactly the same use cases. Like zero copy is formally verified, right? My code uses a lot of unsafe and it is not formally verified. And that's just a trade off. ⁓ But ⁓ I've tried to design something which is at least partially zero copy, but it will also be able to handle all the variable size data that DNS will throw at us. And so in a lot of cases, it can represent things just entirely as references into raw bytes, right? Which is the zero cost, that zero copy case. And in a few cases, it'll be like, well, I need to copy some things out. And that's sort of where the, that was what our trade-off ended up being. And so, I mean, we've heard from some people that they really like to see this ⁓ module become a separate crate. And, you know, that's something that we'll try to explore at some point. but ⁓ it's really interesting to compare and contrast zero copy with this internal module. ⁓ yeah, the other sort of important departure from zero copy is that ⁓ zero copy, all of the traits that zero copy gives you to implement, ⁓ you can't really implement yourself. Or like if you implement them, then I think it's just a. Yeah. If you, if you try to, well, first you can only implement them through the drive macros. And if you try to do things yourself, you'll discover that there's some private types and stuff. You can't actually do any of that. For good reason. Yeah. For good reason, which is that like zero copy will assure you that it will perform the deserialization and serialization correctly. Right. ⁓ And in, in the module that I ended up coming up, coming up with, ⁓ each of those traits can be implemented manually where you need to. but there's also derived macros which will usually do the right thing. ⁓ And so our goal was sort of, we just needed to make serialization and deserialization as efficient as reasonably possible while remaining really convenient to implement. And so, you know, for ⁓ one of the really cursed elements of the DNS wire format is domain names. You can have domain name compression, which is this whole thing. ⁓ And yeah, don't want to get into it. I've had enough nightmares there. ⁓ So we have specialized ⁓ trait implementations just for domain names so that we can serialize and deserialize them correctly. Meanwhile, any of your high-level record types will just use the derived macros. And so we've ended up saving on a lot of boilerplate ⁓ while still maintaining a lot of zero-copy stuff. So that was... really interesting. ⁓ We don't have proper benchmarks yet, like that entire API is still sort of coming together and being fleshed out. We're making a lot of changes along the way, trying to make things more flexible or taking advantage of new Rust features when we're in the opportunity, when we're, yeah, right now domains in this place where we're expecting to make a major breaking release soon. And so we're like, okay, well we can redo our APIs now, you know, we can now take advantage of ⁓ async FNs in traits, things like that. That's all of that. yeah, like that whole architecture is evolving, but at the core, it's able to do a lot of Zeta copy stuff. And so. Yeah, very cool. anyway, like I think Joshua would be very proud of you because more than his promotion of Zero Copy, like great, he wants to mostly promote the spirit of it and how you can do it. And that's also why he's in this working group on being some of it in the REST compiler. So ⁓ please do share the link with me to the module because I think he will probably be looking at it. Like I'm sure you will find it very interesting. Yeah. Yeah. That'd be really cool. I'd love to, yeah, well, if he's listening then please get in touch. It'd be a fun conversation. I will share it with him, he will see it. So do share. Sadly, have a feeling we have to round off because as much as I could talk with you guys the entire day, we do have to somehow keep it into a limit. In the future, probably in season two, I might do some roundtables around specific topics and then... I feel like I'm definitely able to invite ⁓ one or both of you at that point as well if you have me. But before we sign, yeah, yeah. Okay. And then before we sign off, I do want to give you at least the opportunity to bring something up that you think we didn't discuss yet. It could be a topic. It could be like something that you want to shout out, something that you want to promote around, maybe ask more like funding around, anything you want really. Yeah, take a break. Well, funding is always welcome. I think I just want to shout out the organization here. NLnet Labs has been a fantastic employer and they let us do a lot of cool stuff, ⁓ work on lot of these things. ⁓ They let us as relatively new hires take on these big cool projects ⁓ and I'm very grateful for them. ⁓ I'll also give a shout out to Rust Week, which is May 18th to 23rd, I think. ⁓ Will be volunteering, well, Terts is organizing, I'll be volunteering, please come. It'll be so much fun. ⁓ And I know I have a lot of side projects, but I don't feel like, I think they're all due course to share publicly. ⁓ There's some hash tables and things going on. But anyways, yeah. We'll share your zero copy stuff and then we'll call it a day. I do a lot of optimization stuff and there's always more to talk about. But anyways, yeah, thank you so much. We will start with that. Maybe if like a blog around that that that we can also share or is that not something you share? Yeah So I'll definitely put that in the yeah Share with me share anything of these things like where people can find you I will definitely also share that and then they can become like a scriber and follow you and see all the curse projects I do. That's fair. do. ⁓ yeah, sure. I'm happy to plug the blog. Very cool, well thank you very much both for your time. I know you're extremely busy with all the wonderful work you've been doing and I've been glad that we finally found the slot. Thank you so much. This was really fun. Elizabeth (Plabayo)
1:26:06 | 🔗
Netstack.fm is brought to you by Plabayo building secure, open, and resilient infrastructure with Rust protocols, and purpose. This show is also made possible by Rama, the open source networking framework. Plabayo offers service contracts and welcome sponsorships to keep building and supporting its ecosystem. The theme music of this podcast was composed by DJ Mailbox. If you enjoyed this episode, don't forget to subscribe on your favorite podcast platform and leave a five-star review. It really helps others discover the show. Thanks for tuning in. We'll see you next time for the next handshake.