On this page On this page
Episode 5 – Tokio with Carl Lerche.
In this episode of Netstack.fm , Glen speaks with Carl Lerche, the creator and maintainer of the Tokio Runtime, about his journey into technology, the evolution of programming languages, and the impact of Rust on the software development landscape. They discuss the rise of async programming, the development of networking libraries, and the future of Rust in infrastructure. Carl shares insights on the creation of the Bytes crate, the implications of io_uring, and his role at Amazon. The conversation also touches on the upcoming Tokio conference and the introduction of Toasty, a new query engine for Rust.
If you like this podcast you might also like our modular network framework in Rust: https://ramaproxy.org
00:00 00:45 04:01 05:52 09:12 13:40 17:39 19:15 25:57 26:12 29:40 30:51 32:30 38:58 49:20 53:10 55:17
Music for this episode was composed by Dj Mailbox. Listen to his music at https://on.soundcloud.com/4MRyPSNj8FZoVGpytj
Elizabeth (Plabayo)
0:18 | 🔗
This is netstack.fm, your weekly podcast about networking, Rust and everything in between. You are listening to episode five, recorded on the 27th of August, 2025, where Glenn has a conversation with Carl creator and maintainer of the Tokio Runtime, the async foundation powering much of the Rust ecosystem. It's another week for netstack.fm and today I welcome to our virtual studio Carl. Welcome! Thank you, super excited to be here. So today we will talk a bit about Async in Rust, a bit about Tokio's history and of course Carl his own history. He will also talk about the upcoming Tokio conference, as well as the future of Rust Async. Now Carl, despite all the impact you've had on the Rust Async ecosystem, or actually in fact that you have 186 crates on crates.io which is quite a lot, Some listeners may not know you yet. Therefore, let us await a bit, pardon the pun, and please start by sharing a bit about your journey and how you first got into technology. Okay, yeah, I'm Carl. Thanks again for having me. I've been programming since I was a kid, basically, in the 90s. You know, my family always had a computer. I remember my first computer was an old Zenith machine and it had, one of the versions of BASIC. I'm pretty sure it was GW BASIC. Back then you had to number your lines and I remember that you had to do like number all your lines 10, 20, 30 because you wanted to skip numbers so you could add lines later without having to update all of your code to reference different line numbers. But it always interested me, know, computers. always loved tinkering and programming and that kind of got me into it. probably around high school, the whole internet thing started happening and... I picked up a PHP book at that point and just kind of started hacking and kept doing it since. didn't think, interestingly enough, I was gonna go do computers or programming professionally. Not entirely sure why, but it took me a little bit to actually land into software as a profession, but. But did you know it was a profession? because personally, like I also got started with programming quite early, but like, like there were weeks where I was doing it a lot and then other weeks where I wasn't or even a month I wasn't at all. And then I didn't know it was a profession. I just thought it was something you do, I don't know, to fill time or because you enjoy it. Yeah, I mean, totally. think I actually grew up in Belgium and I was in Belgium at that time when I was kind of exploring computers and I would say kind of the 90s, there was not really at all a career track in programming. think, you know, going through high school and that early kind of education, I guess you my career track at that point was like going into some sort of civil engineering or that was what you did if you are good with Math. Also around that time, I ended up moving to the United States and to Portland actually, and went to Portland State University, which had a good software kind of programming I kind of dabbled in some of those courses, but for some reason I still wasn't like, I'm going to go and become a software engineer. still, again, while the dot com thing happened, it also kind of just crashed. So right when I was going through college, there was a gut of software jobs as my recollection was. I kind of left, Interesting and before we continue and I don't want to go too deep into it but It reminds me that maybe the the people are currently at the same stage in their life As you were back then and given there is now this entire LLM and AI fear for it and again, I don't want to go too deep into it right now But do you think that how those people also felt like am I? still in the right career, I mean, like all things, was a gut then and then people figure it out. Okay, how do we actually use the internet kind of thing? So, and you mentioned AI and I actually think there's a... a lot of what we're seeing right now with AI is reminiscent of that whole introduction of the internet where industry had a way of doing things, which was not with the internet. And when the internet was introduced and things were moving online, there was kind of this big fear that a lot of the jobs, especially retail and all that are going to go away. And, you know, there's going to be far fewer jobs for people. it is true that the internet and technology made a bunch of those kinds of jobs obsolete, but that's not really like, it's not like there's no jobs ever. There's new jobs kind of thing. Like the number of software programming jobs kind of exploded. And I kind of see you alluded to AI. There's a, there's kind of this fear that AI is going to take our jobs, especially now that the newest AI codegen tools are surprisingly good. It never is they take our jobs and no one can find anything to do. It's the jobs change and you evolve with them. And, you know, there's always a drive to keep building more and to, you know, keep advancing the state of technology and the state of our products and everything we use and make our lives better. And there will be new things and new ways to approach. the same kind of problems. I think AI is definitely gonna change our jobs. They're not gonna go away, they're just gonna be different. Now, you said, okay, in Belgium, high school, you were more like civil engineer than in United States. You were finally into college. majoring in math. How did you end up then in programming? Well, it turns out that's where the jobs were. Because right then, like right when I started entering the workforce, the recovery, like the post DotCom crash recovery was happening. And... the number of software jobs was kind of exploding again. Everyone figured out how to use the internet And there was a massive kind of expansion of things to build, mostly internet software and networking software. And my experience kind of like hacking around with PHP and all that was very applicable. And I got a job working PHP at a PHP shop. Interestingly enough, also around that time, I think about maybe two years or something into my time just building web apps professionally using PHP. That's when the whole Ruby on Rails thing started catching on. And I do also remember that, so I don't know if anyone remembers what it was like building web apps like. mid 2000s or something like that, but it was painful. compared to what we're used to now. Well, in a way it was also simpler, And by the way, you said, okay, PHP, but does it mean just PHP? Because I mean, there were also plenty of people, I forgot how it's called, but that you could like mix in C or whatever, native languages on your computers. Yeah, There's definitely some of that. I remember what I did was just pure native PHP, like the lamp stack. it Linux, Apache, MySQL, PHP? That's what it was all about back then. it was very exciting. whole Web 2.0 thing wasn't coined at that point at that point I would say the phase of... internet was just introduced and let's just put everything literally we can think of on the internet without really knowing how to do it past. So now things that were being built were more kind of, wasn't so much take existing stuff and shoehorn internet into it. was more now we're re-imagining what we're gonna build but on the internet and Software as a service, that concept was also just kind of starting, which was interesting. But I also do remember again, the Ruby on Rails thing, that first experience I saw when you saw the screencast, let's build a blog post in 15 minutes. That was again, nowadays, I feel like you look at something like that and you're like, yeah, it's nothing special. But... when you were coming from the way people did building software and specifically web apps before you saw that screencast, it was a game changer. And that concept of focusing on just productivity, specifically convention over configuration, these are all things that seem obvious now and are kind of ingrained, but... It wasn't back then. Like that's the thing that's funny to think about, like that idea of convention over configuration and just kind of. streamlining the happy path was not really a thing. it changed, I think how a lot of people thought about building software and designing libraries for kind of productivity and that kind of stuff. So it was, and it influenced a lot. It heavily influenced the way I kind of think about Yeah and of course it goes without mention but a lot of influential people from the Rust community they come from Ruby on Rails similar like you so is that for the same reasons where performance was always a mention? Yeah, interesting. Ruby was big back then and when it was taking off. It was also, again, it was taking off really in the early days of the internet where scale was a lot smaller. mean, obviously Google was already big and whatnot, but besides those huge kind of search engines, the scale of building a web app was much, much, smaller. And Twitter like wasn't even around yet, for example. I think Facebook was just starting, but... As the internet kind of grew and grew and grew and more and more people got online and the scale, like scale requirements and performance requirements kind of just increase at exponential rates. And I think there was, I mean, Ruby's not known for its performance and there were a lot of attempts and a lot of engineering time put into trying to speed up Ruby, but fundamentally the language itself. was not suited for really optimizing. And there was kind of like a diaspora, I remember, of just like people doing Ruby that spread out to other languages. A lot went to Go as well. Go was kind of like the early one. But I think the Rust. developers that came from Ruby didn't go to Go and partly, mean, probably a few reasons. One, just didn't like the language that much. Two, while Go was faster, it wasn't kind of like a revolution, I think, because the JVM was also there as well. The JVM was competitive. I think a lot of the Rubyists that went to Go just also just had a stronger version to the JVM, which understandable. Rust, when it came about, really was brand new thing where you got that of a native language without the risks of, mean, everyone knows, I read this Rust podcast, you know what the selling point is, but that was something very appealing to those of us who never wants to native, like C or C++ because we know we're not good at it. So it was an opportunity to kind of jump into that. And the way I got involved, I was working, at a startup called Tilde Katz who did a bunch of stuff like JavaScript and also was involved early on with Rust. But we were building kind of monitoring tool for Ruby specifically. but we're pushing a lot of traces essentially and we wanted it to be fast. So a native extension there was the only option, I remember I was like, let's just do it in C++, YOLO I guess. I'm not very good at it, but I, many people, I tend to vastly overestimate my capabilities. But Yehuda like, He knew Dave Herman through his work in JavaScript. Dave told Yehuda, I'm working on this language at Mozilla that Gives you the performance of C++ without the dangers. You should give it a shot. And so Yehuda told me, hey, you should try Rust. I pushed back initially pretty hard because Rust was very new, and I mean, if you use Rust back then, you would just be hitting ices constantly. this was at the time know if anyone listening remembers the very early days of Rust where it looked a lot like Go. It had a garbage collector. It had green threads, et cetera. So this was the point I got kind of got introduced to Rust was when they were doing that work to remove all of that, to turn into the Rust that you know now. so that was just landing and I almost, I feel like I remember like when we got introduced, it was literally like, hold up another two weeks, we'll land the last few PRs so that. you actually can run it without the garbage collector. Anyway, I was telling Yehuda hey, this is massively untested, completely experimental stuff. We probably should not use it in our product. was my argument and to just use C++. I side was, you can't write C++, you're no good at it. So eventually he won that argument. And I was pretty skeptical at first, but after actually using it, getting something working, It sells itself, I guess. And the rest is history. That was my introduction. I wanted to use Rust more. After that, I like building networking stuff here. And obviously, it was so early, there was nothing there. The only networking APIs what the standard library offers now, simple blocking sockets. So I started building MIO, M-I-O, just kind of like those epoll bindings went from there. that story that early? don't know. What was your original goal for mio because for sure it wasn't to build Tokio. Well, actual original goal was let's start a side project. My initial idea was to build, you know, kind of hack on a toy distributed database. Why not? Because that's an easy side project. But... was no networking libraries, right? So that is what got me started. like, all right, let me just quickly hack together, epoll bindings and go from there. Mio and Tokio and all that it was like this big yak shave that I still am not out of, because I never wrote that distributed database. But the idea was not Tokio, you're correct. so Mio was first and then There were some early experiments with futures that I made that didn't happen well before Async Await. But the Async Await stuff, I think that was designed by the Rust core team. And what they landed on... was quite nice. So, before even Async Await, there was the future traits and that was built as a external library And it kind of also tied back to how Mio was designed. A lot of async libraries for other languages are built around closures and callbacks, right? And if you're registering a callback, that has to be allocated somehow and moved off the stack because you're saving it all and storing it to be called that's how it works in JavaScript. Pretty much every async library at a high level worked roughly like that. And early attempts to design a async I/O system with Rust kind of followed that pattern closures and box them and whatnot. it was obvious that it was really painful. you're allocing closures everywhere, you're boxing everything, you're making dynamic trade objects everywhere. Everything has to be, know, send, sync, static, all of that stuff to make a compile. was pretty obvious early on that didn't work. so when I built mio the first thing I was looking at was libUV in JavaScript essentially the library backing Node.js. And they also used, function callbacks and whatnot. And I threw that out the window with mio The idea being, let's just model it after epoll and be as close to the metal as possible. what I didn't realize after that was that was the path to keep going. And was still playing with closures and whatnot. And I think, the Rust core team that were focusing on what futures would look like kind of came to the realization that you continue that kind of polling model and abstract it away. and were you involved? even if just as feedback? I was involved like watching in the site involved at a like superficial level, I guess. that, it was like, all what does Tokio look like after that? And they were also experimenting with how to bring that to the networking layer. So their early design for what and networking kind of abstraction built on top of futures and the future API at that point is pretty similar conceptually to what is today. But you build a stream of network events. essentially, and then you still have your reactor. So the main abstraction there was a future of the IO events. So you would still have your IO loop where you receive those events and then call like non-blocking functions. the Waker system was already... if you're familiar with internals of how futures work but the waker system was there there was like a dispatching system where you had the root feature that was quote unquote spawned. And then when the event came, the runtime called that root future, which is the same as a spawn task. And I believe that their original design for layering IO on top of that was inside of your spawn root future, you then had a stream of IO events that said readable writable, and you are almost building mini reactors within each spawn task yourself that were scoped to one socket, if that makes sense. And I guess the main insight I provided at that point with Tokio was that you could take that same Waker system and apply it directly on IO actions. So you didn't have to have future and a stream, because those were the two primitives they were building on. You could just take that entire abstraction and apply it to everything. So my initial version of Tokio was with poll read. I guess introducing poll read for better or worse, if you like it or not, was the Tokio contribution where you called poll read and pass in your waker and that then registered that point, That waker for that read was registered and then the spawn future would get called again and you ran your entire state machine. So that was the, contribution of Tokio to what was non-blocking Rust futures at that point. And then... Further down the line, I don't know if you yourself wrote a lot of code by manually implementing Futures for everything. But now imagine implementing an entire complex networking system on entirely manually implemented futures. It was doable. You manage your state machine, but it was painful and tedious and error prone. And the... Key thing at that point that the Rust core team did, you know, there was that pin was introduced and then how to take async awaits syntax and translate it to a state machine. That was all, as far as I know, novel work that's... significantly made writing async rust, like it significantly improved the ergonomics of async rust. So do think... the future for async Rust is bright where like, it's been interesting kind of seeing that evolution. Cause initially when Rust was introduced, it was pitched as a systems programming language, right? It was like, this is an alternative for C++, C like you use it to build that infrastructure level. But as you get to... use Rust and as the language has evolved to be, it really is a highly expressive language. And when you know it, you can be as productive with Rust as like any other language, I originally came to it while I was still doing mostly C++. And for me, I also saw it like as a new system language which solved every problem I had to with C++. things that I had to worry about and things that I had to be very careful about and there it was just solved for me. And then at the same time, I discovered Go. I was doing web services mostly in Go because... I really saw them as two different things, then more and more I noticed that Rust can handle this also very nicely. And the thing I sometimes say to folks in the Foundation or other people in the Rust community is the thing that they don't mention enough is how you can really fearlessly refactor your code bases For example, I maintain a big network framework called Rama, which is also built on top of Tokio. The amount of times I did some refactor, and it just works out fine. I don't have to worry about it. And that's definitely a power of what Rust and its type system gives to you. Yeah, for sure. that is the equation that Rust gives you. I think it'd be hard for Rust like to be the most productive language for early development and prototyping, but. On a whole, when you factor in long-term maintenance and like the growth and the longevity of software, Rust, I believe is highly competitive from a productive point of view And the way I think about, software engineers, programmers, they just want to get their job done as fast as possible, as well as possible. And whatever tool for the job that enables them to do a good job fast is going to be the tool they pick, right? it sounds simple, but that's incredibly complex calculus to pick that tool. and there's obviously going to be many biases when picking that tool, at it that way, that also kind of explains why Rust exploded for lower level software is because when you're building infrastructure level software, you have fundamental requirements that are like, this has to be fast and this has to be correct and this has to have predictable, reliable tail latencies. And when those are requirements for your software and you look at anything out there, including Go, including the JVM. Rust is the most productive tool for that job. Maybe not because of that initial implementation, but you're gonna get to software that meets those stringent reliability, consistency, predictability, performance goals faster with Rust. And... especially once you start learning it, right? Like what is like the hard part, what's crazy is that you can meet, like it's true for Rust, that calculus, that productivity is true for Rust, even if you have a team that doesn't know Rust going in, right? If you have one person that kind of knows Rust and a team of engineers that don't really know Rust, more often than not, obviously it's not a guarantee, but more often than not, if they're building infrastructure software, they're going to be able to learn Rust and build their software and meet their goals faster, you know, with Rust than building with the JVM. Fundamentally because it is really hard to get tail end sees down with a JVM. And I know I'm gonna probably make people upset, like JVM people upset. I don't care, but I know they... JVM engineers keep saying we're going to finally get tail encees down with the garbage collector with just one more revision of the garbage collector like there they've been It's I've been hearing that for 20 years. I still remember the pitch in the 90s Kind of late 90s. I was a kid Talking with some older, you know software engineers like I mean older they're probably there late 20s or whatever But I was a kid so whatever and they're like and I was like, hey, what language I pick I wanted to you know I think games is what I said. I want to build games because as a kid, and that seemed fun. ⁓ And I was like, should I learn C, C++? And they're like, no, you should pick up Java because soon Java's gonna be faster than C or C++ because the JVM is able to see more about your code and optimize it beyond what a static ahead of time compiler can do. I'm still waiting for that to be true. I think you are the one that created the Bytes crate, right? So how did that get started and what was the idea behind it? That got started early on in my involvement in Rust. when I was in that phase of there's nothing in the Rust ecosystem, let's build it out. And there were two things that motivated the bytes design. I would say one major thing that motivated the bytes design besides just having that crate, right? So it is inspired by libraries that other languages have, bytes management, just byte buffers that are reusable and shared. the reason why bytes went with a reference counting approach and sharing is very much because of what early non blocking networking was in Rust. without async await with manual futures where everything had to be send and sync and you could not. Easily share references across await points, right? That was a big source of friction with designing libraries or using buffers or anything. just having something reference counted that you could move around, clone cheaply and move into different things, like move into your futures made life a lot easier. Now it still is true that it's still applicable, but one of the main reasons for that reference counting thing is not really true anymore with async await. So oftentimes like maybe you don't need bytes. the main reason I have bytes again is if you want to have one buffer with different windows, like different views shared from multiple places in your code That's the history. Because I do believe you and I've seen it happen but at the same time, like most people that example use Tokio, they do it in a work stealing mode, And in that case, you still cannot really deal that good with references, right? Yes. So within one task, that's going to be pinned And I do think the task is a abstraction points and you should be very careful in general about sharing any sort of state between tasks. if you can avoid it. Don't, just like you don't wanna share state between threads if you can avoid it. And you wanna really think carefully about how two tasks interact and how you're gonna manage that synchronization. there's a bunch of ways to do it. Channels is probably the best one to reach for. Yeah, so to move a bit our conversation to a slightly different io_uring it's still a bit experimental, but the experimental phase within Tokio has been going on for a while. And I'm not sure what the story there is as an outsider, at least. Yeah, as an outsider, yeah, it's definitely probably confusing. What I would say is there was a lot of excitement about io_uring and there probably are use cases for it. But based off what the evolution and how I've seen experiments go and our experiments is that It was hyped a bit early on and the wins in practice for your average Tokio application are not. would say, in a way that we had hoped. So add that to the fact that the interface itself has been evolving a lot. Plus there have been security issues with the implementation, CVEs and whatnot. And it is a much more complicated interface. on Linux, it's almost definitely the way to go for files, but because of all those compounding factors and the lack of of real data showing that if we used it, we would get a material win. For most people, it does not make sense to use it. that also could, yeah. you mentioned Linux, but I know there's a similar system on Windows. Is that also being considered If you look at the implementation on Windows for Tokio, you that we have gone through great pains to avoid using IOCP. the point where we've used, and this isn't just Tokio, this is also Node.js and the whole bunch of other async abstractions. And you will notice that we avoid using it as much as possible in favor of biasing for internal APIs. And there is a reason for that. Is that similar for Windows support in general? Windows support, we try to make it as supportive as possible. Epoll is a really great battle tested, simple, efficient API and doing better than epoll is hard. It is true that for mixing file access with networking, you probably do want to use io_Uring in some cases. the Tokio U-Ring crate shows that you can mix the two and that is probably sufficient for file access. Again, unless you're doing something incredibly focused and performance sensitive, in which case you may want to consider not using Tokio as your main abstraction. Like again, if you are at that point where really tuning it matters, it may be that that tuning is a core part of your product where it makes sense. For Tokio, we want to do the best we possibly can for the majority of people Where it's as good as it can be out of the we try to prioritize correctness, ease of use. and safety and security, right? And that's obviously hard for a library to do for everybody. Everyone has different requirements. So I'm gonna be the first person to say there may be times where you don't need to use Tokio. I'll also say most engineers and I am a hundred percent in this bucket as well, tend to... underestimate the amount of work it takes to implement yourself and overestimate the benefits of doing so. there's some work to bring io_u ring into Tokio for the file system stuff. Again, tying networking and file system, like doing file system access on the same thread as networking in async context is either your file system access is not a core part of that loop, like you're reading config files or you're just reading, like you need to do some file system access like casually, in which case you probably, are fine moving it to another thread, or you need to tightly control how that happens, how the interaction between your file system access and your networking happens, in which case, You want to use the full capabilities of io_uring, In which probably means hand rolling a bunch of stuff, And so... to move the conversation in a slightly different direction is the fact that right now you work at Amazon, I believe. do they do it because they want you to work on Tokio and that's what you do there? is Tokio now more something you do outside your work? working on Tokio and supporting Amazon's usage of Rust networking libraries is my entire job there. So I think the adoption story of Rust at Amazon, especially AWS has been like pretty crazy to watch. Firecracker was the very first introduction to Rust, And since then, the adoption has been going like crazy at that infrastructure level kind of thing and rust is used extensively like EC2, know, DynamoDB, S3, like all those big services everywhere. Like it is growing really fast at Amazon and when they're building infrastructure level services with rust, it's going to be with Tokio usually not always because sometimes there are cases where it makes sense to hand roll, So my job is to support Amazon's usage of it, whether that is answer questions or helping debug issues, but also adding features and supporting the ecosystem continuing development as well. Okay, the Tokio project has a long history, you were there since the start because you created it initially. How is maintainership going and the delegation towards a new generation For Tokio, I have mostly taken a backseat for the past few years and Alice has been doing a great job with the day-to-day maintenance. And we also have a team of contributors as well, besides just Alice, but Alice is doing great job leading most of it. And we talk, like we still talk, I still get involved but for the most part there, I have taken to step back and my kind of mission, I guess, at this point is to really look at how to make Rust appealing for the general developer population building web apps or just generally networking applications that don't necessarily need stringent performance requirements, because what I see at Amazon, but also generally across the Rust ecosystem is, you once Rust is introduced within an organization, it tends to grow out from there and even to places where performance isn't a big consideration because of that reliability, and because Rust is productive, especially once you build organizational knowledge and experience and you start building internal libraries for it, like being able to share code that you build for one service across many without having to write it for many languages is a big productivity multiplier. So if you already have to use Rust for key components, using Rust elsewhere also makes sense. But that's, kind of growth story of using Rust elsewhere is hampered by a lack of higher level. libraries. I don't think there is a great ORM or database library for us. That's a really big project and it's a side project for me, but I'm working on hacking on Toasty on the side. Though I do hope to get it to the point of it being ready for introduction in real apps by the end of the year. It's somewhat an ambitious goal, but I'm hoping to do that. that full kind of how do I build a web app or how do I build a backend service that is talking to a JavaScript front end or a mobile front end. That story is not as great with Rust and I'd like to see it better because I do think that there's potential for us to be great in that space. Maybe not as productive again, initially for the prototyping as other languages, but over the long run when you're acturing in the entire needs of an organization, it starts making sense there. Yeah, true. And so you mentioned Toasty. maybe you can introduce the listeners a bit about Toasty Yeah. So, Toasty it's not really an ORM, but it kind of fills that need. I think of it more as a application level query engine, because I'm taking a lot of the same design principles of a query engine. And kind of the idea is you define your model level schema, like you were working at that application model. and the database is somewhat abstracted, though the goal is to not make it invisible, but not make it in your face when it doesn't matter, So you can always reach for database specifics, but for the most part, when you're building an application, you don't care. You just wanna like move forward, be productive and worry about later kind of thing. So. So for example, here at we also working on a couple of new commercial projects and for now I was thinking to just go for SQL and Postgres because that's what I know about. But would choosing Toasty at this point be an option? that entirely depends on how much you're willing to contribute. think it's very close to being at that point. if you try to use it, you're going to hit definitely issues and bugs and panics for cases that are not implemented yet that I meant to implement but haven't. And if you are willing to jump in and help fix those, believe it definitely, it won't be as fast as building with raw SQL right now because you're going to have to collaborate. So one goal of Toasty is also kind of this theory that you can make Rust libraries much easier to use if you don't focus so much on zero cost abstractions. Yes, Rust can do them. but they come with a cognitive cost at times and most of the time at that higher level you don't need them. So Toasty is trying to prioritize developer experience first with the theory, the hypothesis that... even with those productivity focused APIs, we can make it fast enough for most users with escape hatches when you need to, right? And just to be just a front-end, right? In the back-end, you still use something like Postgres? going to be able to target SQL. I'm also aiming to target other databases than just SQL. Like think DynamoDB is one early one because I'm at Amazon and also it's a good one to target early because of how it's modeled. Most key value kind of like NoSQL databases will also be able to be supported if DynamoDB is. the idea, and this is part of why I mentioned it's an application level query engine. And the idea is not to abstract DynamoDB from your model. So it's not that you can do any query that you can with SQL that will just run on DynamoDB. It's almost like the opposite. But I've used ORMs and I've used NoSQL libraries and I've used all these things throughout my career. And I'm always struck by how how 80 % is exactly the same and it's always that last 20 % diverges. part of the hypothesis and it's still an experiment is can I base the API around that 80 % and then make that last 20 % like opt in via features or just generally dependent on how you use the library. And interestingly enough, finding that that last 20 % is not actually that different because what you need to really power a good data model with DynamoDB is also useful for more advanced use cases with a SQL database where maybe you're building application level schema or your application level schema doesn't directly translate to the database schema. For one thing in DynamoDB is you denormalize, right? You have your core models and then for every query, if that doesn't fit nicely with your primary key that you have, you need to denormalize and access it via that. But that kind of translates to materialized views or other concepts as well with SQL, with more advanced SQL. what I initially thought would be completely diversion is turning out to interestingly be less divergent than I as well, if you kind of approach it as like this query engine, this application level query engine that knows about your underlying database idea is less to translate the query, but when you issue queries that don't make sense, it flags it. So usually when you're working with databases, you do a whole bunch of work. You target your driver, right? So what will probably happen is when you run your tests or you, you implementing a query that doesn't work, it's going to panic at And that's okay. Then what you don't want to get back to get to production. Then there's probably going to be an additional step when you commit your migrations and you're working like, okay, I'm done with my iteration. I'm going to generate migrations. That's going to be another step. That's more like a compilation time step, but you're committing it and it analyzes your queries and tells you, okay, this looks good. Or, this is. a query we can't officially run on DynamoDB. anyway, these are kind of crazy ideas I want to explore later. it's just an interesting observation I'm finding that supporting DynamoDB well and supporting SQL advanced cases with SQL is not that different. interesting well sign me up you also yeah go for it So yeah, I was gonna say the flip side is that makes the scope of Toasty very big and part of why it's taking so long. Yeah, fair enough, mean, projects can take a while, I don't mind. my superpower is drastically underestimating and then just jumping ahead first and powering through. Which is kind of like how we ended up building up all this kind of like rust ecosystem as well. You also mentioned chat that you were dabbing into AI and not so much that you were on the hype, but you want to keep evolving For sure, and I know AI is controversial and there's lots of reasons why it's controversial and there's a lot of legitimate concerns and I am not going to touch on that. I'd rather focus on engineering questions at the tool level. is AI a good tool, effective tool to improve my productivity as a software engineer. Because I do think like while there are a lot of issues, I hope that they will be solved as things get better, right? as software engineers, we're... responsible and like part of our job is to evaluate tools as they come from engineering mindsets and try to be as objective as possible. And every tool that's introduced has pros and cons. But we have to be somewhat objective as well. It's like, is this going to improve productivity where productivity is kind of like, can I get my job done faster within the constraints? Can I meet the requirements and the constraints faster? And the project is responsible for defining those constraints. fundamentally, not every software project has the same amount of stringent correctness requirements. And that's just a fact of life. I know we all Software engineers enjoying taking pride in their work and doing a good job and that is good. Reality is like if you're building a tool. like a side testing tool or even like a backend for a hobby app or some small kind of thing that doesn't really, doesn't have a stringent requirements. If never shipping a bug, was always a requirement, we would be using formally formal verification for everything. Right? We don't. Why? Because there's a productivity cost to using that. And there's a real trade off between reliability and productivity. So. If you assume that there's a real trade off, like you can evaluate AI, how much faster can I ship stuff? What is the real correctness trade off or the real trade off for that? And what I'm finding in my experiments of it is I am able to ship code faster within like reliability tolerances I'm okay with. And the reliability tolerance is going to depend on the project in question. And it's like the amount of productivity I'm gaining is such that it's really hard to ignore. is not just a little bit. I can say like from experience, looking at other people trying to use it, looking at my own experiences using it. is a learning curve, like if you try to use it without really understanding where and how to use it, it's going to slow you down for sure. Right? So second, when you're introducing it to existing code bases, there is kind of like you have to almost teach the tool how to work in that code base and figuring out how to do that is not obvious. Right? Third, Every AI tool is not the same. I would say four months ago, I would not be able to be productive increasing my productivity with them. The most recent iteration of tools, I think are changing that equation. they're getting better at a rate that is... Surprising, it is not just the models, because I've used tools that use the same underlying models with vastly different results. it's gonna be the lowest common in denominator. it's whichever is the weakest link between the model and the tool, that is gonna be the quality of the output. There are gonna be cases where there's lots of situations where they're not the right place to do it. And it's better to do it right now by yourself, like as a human. But when you know when to apply it, how to apply it, which tools to apply, It is a vastly different experience than using the wrong tools initially without that info. kind of my experience has been, I had that initial, like my first like two weeks using it, I would say. it was not proving successful at all. But there was almost kind of like this aha moment once I got my flow down where it unlocked a lot. And a big part of that I do think is knowing when to apply it quickly, knowing how to, and knowing when not to apply it. Can you share some lessons and what people can take away from So assert struct, which was a development kind of like a testing tool, pattern matching, an assertion for... complex structures, of like imagine like if you want to do really fancy match statements pass of what Rust lets you do for tests. That's what assert struct is for. That was my kind of initial, let me try to build this from scratch. And my experience there is for the most part, if you have the well-defined feature set and you're able to express that clearly in a new code base, it works really well. I started having some problems kind of when I want to really nail down the error messages and that's pretty fuzzy and that kind of like I had more friction there. It was impossible, it's, it was definitely, it's possible that that would have been something that would have been better done by hand because it's a lot more fuzzy what you're actually supposed to Secondly, while it was really building a cert struct quickly, you obviously have to review the code and it does weird stuff sometimes though. But I do remember one time, a search struct was kind of my experiment of look at the code and kind of touch it as little as possible and see what happens. I noticed as was building, it kind of just. was hitting a wall and just hitting bugs and like not able to not generalizing. I was like, what's going on? I finally looked at the code and I saw the, it took the wrong algorithm algorithmic path and it was actually part of the error messages. So I think a combination of not explaining, not being able to clearly define the specification for how error messages should look plus the tool went down the wrong algorithmic path. And I will not say that this is a like it's because it was AI. I kind of treat it as like a very eager, fast junior developer. Like this was a mistake human could like a less experienced human could make. Once I identified that and kind of was like, okay, step back and this is the algorithm you should take. I described it at a high level. It like we wrote everything. It was a lot smoother from there. I would say as a rule of thumb for new code bases, it tends to work a lot better because there's a question of context. Smaller code bases obviously is gonna work a lot better. So if you're able to structure clean abstraction points, it makes it easier. I have dabbled with it for Toasty. It is harder to get it to use for Toasty because Toasty is such a big project with lots of moving pieces like in the engine that doesn't make it impossible. At a general, if I were to use an AI tool 100 % for just Toasty, I don't think I would be more productive. But at least up until my experiences now, I think that it is still an open question and open problem of how do you get these AI tools to onboard with existing complex code bases? my experiences with Toasty right now might be completely different in six months as a disclaimer, because I do think there's going to be ways of introducing your AI tools to a code base. Don't want to dig into what I've tried so far too much, but suffice to say for some things in Toasty that were more isolated, generating tests, more like changes that are more isolated, any mechanical refactor, like I basically was able to churn through all of the larger refactors that I've been putting off super fast and that felt great. So those kinds of problems it's really good at solving. Anyway, I'm still like, I'm only like four weeks into it. So I don't have any groundbreaking. I'm not like the expert yet, but I am very bullish, I will say. think it teaches at the very least that we should all be open-minded and try to use them and go with the flow and see which tools come out, which integrations, models, etc. But... Yes. What I would say is it would be foolish to ignore this tool. I understand why and I understand how. engineers, always think we do a better job than we can't. Like it's the urge and I'm saying this also, like I'm not, I know, I recognize this in me, right? Like every single problem, I think I can do better myself. And I don't think I'm the only one, I think it's traits. And taking your hands off is hard. in relying on reviews, and it also does take a willingness to not nitpick everything. but like the skills I built up, like work on open source and PRs and like reviewing PRs and just generally code reviewing. I've built up a willingness to pick my battles and some things I'm not 100 % happy with, but I will just accept because it's not worth nitpicking. if you get a PR from a person that doesn't know the code base super well, they're trying to do a good job. You can't... always get everything you want out of that PR, like going back and nitpicking every single thing is going to be discouraging the person, it's going to kind of put them off the project. So I have over the years built up like a tolerance for, well, this isn't perfect, let's just move forward with it because it's acceptable. And that's... skill set, I guess, and acceptance also applies to working with AI tools, because you're not going to get exactly what you want. And it is almost like that 80-20, like you can get 80 % of the way there. Is that 20 % worth nitpicking or not? Oftentimes in general, it's not, or it's something you can come back to later. And it just lets you move faster. And as... Software as a craft, like I get that feels bad. Like it's going to be hard to have a tool and put your name almost on something that feel that may not be 100 % what 100 % artisanally crafted software. And a lot of code that I enjoy writing myself is I spend time making it look pretty in a way that I think is pretty, right? Maybe I'm the only one, I'm sure I'm not, but. there's like that pride of writing something you are proud of yourself that has more than just, does it work and is it good enough to work, but also aesthetics and other such aspects and nice architecture patterns and everything's clean and I for one like doing that and you're not going to always get that with output from gen AI. And the question is, is that okay or not? from a craftsman's point of view, it doesn't, it's not, I guess, like it feels bad from a reality of shipping software and meeting deliverables and business requirements. It probably is and knowing how to move forward with it in a way that works, doesn't doesn't sacrifice quality to a point that it doesn't meet requirements, right? Is going to be the art of using this tool. So that being said, when you are not toasting or vibing, you also found time apparently to organize and start the Tokio conference? Like what's the history there? Yes. So yeah, Tokio conference is going to be the very inaugural one in Portland, Oregon next year, April. I honestly don't remember the dates off my head. It's end of April. The reason for organizing it I wanted there to be a Rust conference one that's focusing on building network software. So while it's called TokioConf, it's not necessarily uniquely for TokioConf. It's more like async RustConf, but that doesn't have as much of a good ring to it. But secondly, I wanted a conference that really focused on the professional software developer or like serious hobbyists, but really on topics to help professional software developers at their job, help them do better, help them grow their abilities, help them, think about how to solve their problems, where they can do more, but generally. I want software developers that come to this conference and leave feeling like they have a lot to bring back to their job. I've not really felt like there was a conference in the Rust space that really focused on that combo of targeting the professional software developer in the networking space, and it's a big space, part in part because a lot of the rest conference are like community focus conferences, which are also great. Those are great conferences. There's nothing, wrong with them. Those are needed and they're high value and I like going to them, but they're less about really providing value to professional software developers and more about community and like open source contributions and advancing the state of the Rust project. So the goal of TokioConf was to build that space for bringing value and talking about how to do our jobs as professional software developers better. and less, it's less going to be about the future of Tokio. It's about, how can we bring value to professional software developers that might also be specifics around how to improve Tokio or the Tokio ecosystem in general. So the CFP for Tokio Conf will be opening very soon in September so I think it might actually be open when this airs I don't know putting any dates is tough for things that air in the future But that is going to be the goal and the topic and the kind of talks we hope to bring out. And so if you want to talk, share how you have been effective and what you've learned about using Tokio or Async Rust in your company, you've introduced it is gonna be a big thing. There's also like, where does it make sense? Where does it not? especially what I want to hear as well is if you have found that story of introducing for performance sensitive aspects and trying to introduce it to other aspects, what didn't go well there? Like what, maybe you abandoned that, maybe you didn't abandon it, but things didn't go as well, but you've found ways to make it work. Anyway, those kinds of stories are going to be, the kind of talks that we want. And so, yeah, given this day and age, it's not easy always for everybody to reach Portland. Is it also remote accessible and then both in terms of speakers as well as visitors? You Yeah. So for. this conference we're going to make it in person only though recorded talks will be available after. Part of that is I know like Portland is kind of off on the west coast. The main reason it's here is it makes it easier to organize and I am available to do it. I will say that if you're able to attend that would be great if you don't totally understand it and hopefully if this is pulled off successfully we will have future editions of Tokio Conf globally and for other demographics. and so as we are wrapping up the episode is it anything more you wish to plug or mention? you I think I mentioned everything so far, or have we gone on for a bit? I tend to ramble, it seems like. Hopefully you all found it interesting, got value out of it. Don't flame me too hard for my thoughts on AI. Keep an open mind. And I think it'd be worth just having that conversation, right? Because I will say I want to have the conversation of AI and software in an objective fashion, in a way that is open because I see a lot of parallels. with different inflection points in the software industry. to internet software, there was a big pushback against writing internet software because there was a feeling that software had to be run locally for it to work well. Then even before that, like you can look, I mean, this is before my time. but even when compilers were introduced right like there was a well compilers can't do as good of a job as a human that is very versed in this in like how to build software for platforms and that is true like I bet although I don't know that's true today I shouldn't say but initially it was true compilers were not as good as humans for building machine code, but they were far, more productive, opened up software programming for a much wider audience and resulted in building more and solving more problems. And I think... The exciting thing to me about this new tool, and again, there's a hype cycle. is still too early to know where it's going to land, but I'm excited about what it will unlock people to build. It's going to change the calculus and the economics of software. it means what in practice happens is lean into building more even more faster like solving more problems it's gonna be an exciting time I think Alright, with that positive note I would like it to end. I love exciting And I want to thank... yeah... Interesting times no matter what. interesting and exciting. thank you for showing up and having this nice conversation. No, thanks for having me again. Alright, have a good one. Elizabeth (Plabayo)
55:17 | 🔗
Netstack.fm is brought to you by Plabayo building secure, open and resilient infrastructure with rust, protocols and purpose. This show is also made possible by Rama, the open source networking framework. Plabayo offers service contracts and welcome sponsorships to keep building and supporting its ecosystem. The theme music of this podcast was composed by DJ Mailbox. If you enjoyed this episode, don't forget to subscribe on your favorite podcast platform and leave a five star review. It really helps others discover the show. Thanks for tuning in. We'll see you next time for the next handshake.