On this page On this page
In this remastered episode, Glen interviews Sean McArthur, the creator and maintainer of the Hyper ecosystem. They discuss Sean's journey in software engineering, the evolution of Rust and asynchronous programming, and the growth of Hyper from its inception at Mozilla to its current status in the web development landscape. Sean shares insights on the creation of hyper, hyper-util, http, headers, the Warp framework, and the challenges of integrating HTTP/3 and QUIC. The conversation also touches on collaboration with cURL, the FFI layer, and Sean's aspirations for the future of Hyper and the broader ecosystem.
Learn more about Sean McArthur, Hyper and Warp:
If you like this podcast you might also like our modular network framework in Rust: https://ramaproxy.org
00:00 Episode remaster notes00:37 Intro01:21 Get to know Sean McArthur and his origins in Rust08:32 The Impact of Mozilla on Sean's Career10:47 A short history of Hyper21:52 hyper-util, http and headers crates33:46 Warp36:14 Hyper's Core Focus and Future Directions38:21 Integrating HTTP/2 and HTTP/3 Support40:19 The reqwest crate45:04 The Complexities of HTTP/3 Integration48:40 Reflections on the cURL and Hyper Collaboration55:51 Future Aspirations for Hyper59:21 Encouraging Community Engagement in Open Source01:02:40 Outro
Music for this episode was composed by Dj Mailbox. Listen to his music at https://on.soundcloud.com/4MRyPSNj8FZoVGpytj .
Quick note before we begin, you are listening to a remastered version of episode 2, first published on the 26th of August 2025. It was one of our earliest episodes and it was created while we were still learning the ropes. Now we have refined it to deliver a more polished listing experience. We hope you enjoy it. As always, feel free to provide us feedback at our email or Discord. You can find those links in the show notes attached to this episode. Elizabeth (Plabayo)
0:55 | 🔗
This is netstack.fm, your weekly podcast about networking, Rust, and everything in between. You are listening to episode two, recorded on the 22nd of August, 2025, where Glen has a conversation with Sean McArthur, creator and maintainer of the foundational hyper ecosystem with a rich history. Welcome to the podcast. Today with me in our virtual studio is Sean. Not everybody knows him yet, but today we're going to introduce him. We will also talk a bit about the hyper ecosystem, about the past, the present and so on. yeah, so welcome Sean. Yeah, it's a pleasure and thank you for accepting it. So before we're to dive into the technicalities, maybe it would be nice if you could introduce yourself a bit, like what's your background? Sure. I've been in software engineering for, ⁓ I don't know, 18 years, something like that. I like that. I actually started with mobile games, not like iOS, iPhone games, ⁓ small games you'd play on your flip phones. And very quickly transitioned into web development, did web development for a little while. got into Rust and I've been doing Rust for 11 years I guess just mostly around the HTTP and web servers ecosystem, Yeah, cool. So that meant that you were in the pre-1.0 days and I think even in those days, like Tokio was, I think, pretty much still starting. Like, I think it was there, but it was like very, very young. And even the Async story in Rust was very young. Like, how did you experience those days? Yeah, I first, yes, I did start pre 1.0 back when there was pointer sigils, the add symbol and the tilde and things. ⁓ But those went away very quickly when I started getting into it. When I got into HTTP, ⁓ into HTTP at first it was just all synchronous. Well, no, it wasn't synchronous I.O. Because pre 1.0, there was lib green and ⁓ what was the other one? ⁓ Lib metal? Maybe there was like a green threaded IO and then regular IO. So I didn't worry about async IO at first. And then when 1.0 came out, got rid of the lib green idea and everything was synchronous, ⁓ which was fine. But eventually you want async IO to scale. yeah, Tokio wasn't there yet. I think the first thing I used was a thing called rotor. It's like this. ⁓ state machine thing. Like you'd basically define callbacks on a trait and return some enums ⁓ It was fine, but very quickly ⁓ Carl and Alex reached out. They were working on Tokio and futures and and thought that Hyper could make use of it. Yeah. And can you explain a bit like how come they reach out to you? Was it like specifically to you or there was like a public call? No, they reached out specifically to me. I was working on Hyper and I'd bump into Alex. were both at Mozilla. ⁓ And I'd gotten Hyper to work in Servo. And it seemed like it was the major HTTP implementation at the time. So it was one of the driving reasons to have async I/O. Yeah, there's other reasons, of course, but that's like the... The shiniest one, I think. Yeah, Very interesting. Yeah, I wasn't aware that Hyper started as Mozilla. Was it like one of those official Mozilla projects or was it one of those things like you can do a bit of it on your own time and a bit on the company time? Yeah, it was a scratch your own itch thing. ⁓ It wasn't ever my official job to work on. Okay, very interesting. And also on your About page you say that Rust is the least bad option. Can you elaborate a bit on that? Yeah. Yeah, that's me kind of, ⁓ I love Rust. I think Rust solves a ton of problems, but I also don't want to be stuck when something better comes along, which it inevitably will. ⁓ Then yeah, I'll move to that. To me, like all these tools are, they're just a means to an end, which is to make better software. ⁓ And I think Rust does a great job at that. But yeah, I look forward to when things are even better, certainly. Yeah, sounds very pragmatic. I mean, I'm kind of in same boat. So very cool. So that being said, like some folks, they think there's like a hidden language within Rust and others want more and more of Rust. Like they wanted the scope to grow. Where are you in that camp? Wow, I hadn't heard that a hidden language in Rust. I I think Rust Rust has a lot of complexity to it. There was There was this idea that, ⁓ I think it was Steve Klawnick that mentioned it. It was the, what is it, the weirdness budget, strangeness budget. I think Rust has used most of Doing even more weirder things just alienates people. ⁓ So I also think that the design that we have so far allows for a lot ⁓ of amazing things. I don't think we need to make things more complicated in order to make software better. And then given your background and given your main focus on what you work on with Rust, like how do you see the asynchronous story evolving and are you happy of it so far? Let's see. I'm I'm very happy where it is now versus where it was. Before, you had a lot of callbacks. You didn't have async await. ⁓ So you had everything was and then, and map, and combinators. Also, JavaScript had this for a while. had what they call it, callback Things are better now. ⁓ There's still things about it that are not, like there's parts of the language you can't use if you're using async, which is unfortunate. We recently got async, fn, and trait, but it has caveats that it can't ⁓ send or sync traits just right. So if you want those and you need to use, ⁓ you can't use async, fn, and trait, you need to return an impl trait instead. ⁓ But you can't, if you have associated types, then you can't use impl trait for instance. like if you were to take the service trait, for instance, and you wanted to say, hey, want to return an async block instead. I want to return an input trait. You can't because you can't put that in the associated type slot. ⁓ Yeah, so that's something you're waiting for, I suppose, or expecting to be there. Yeah, I mean, that's something where I don't think that it necessarily makes things more complicated. I think that just smooths out the, hey, if you already have async in the language, then it should work where you expect it, I guess. Okay, and so if we step a bit back of what you were earlier saying in our conversation, is that you were saying you got introduced, no, you were starting Hyper at Mozilla. Is it also because of that effect that you were at Mozilla that you got into Rust? Did it happen at the same time or were you first in Rust and then later Mozilla? I- Well, I was already at Mozilla, but I was working on something unrelated to Rust. And the team that I was working on didn't use Rust at all. I was working on Firefox accounts where just everything was in I was just slowly getting tired of the dynamic nature of of Javascript and especially writing 50,000, 100,000 lines of code in JavaScript and trying trying to keep that straight. Flow and TypeScript were just coming out, but not enough that we had embraced it. So I was just looking into what new languages were coming out and I was looking at Rust and Go. I tried both out and found that I preferred Rust a lot. And it just so happened that if I had questions, I could go and my bug my colleagues the same company and talk to them at at Mozilla All Hands. So that was nice, but I don't think it was what moved me. you like in general experimenting with languages throughout your career? Uh, yeah, yeah, I've tried quite a few. yeah, I've done Java originally when I was doing games and then kind PHP and Python and JavaScript. And, and I felt like it was time to try a different one. Um, I guess I did like that Mozilla backed it. Not, so much because I was at Mozilla myself, but more because I felt like they were at the time they were, you know, the reputation was stellar. The, were a fantastic steward of cool internet things. So that was one additional like, oh cool Mozilla is taking care of this site, I can probably trust it. Yeah, for sure. so Hyper got started and in the meanwhile, we are like many years further. That means there's like a lot of baggage, a lot of history. Like, can you maybe go like a bit through the history, like how Hyper evolved from then and how it is now? Sure. mean, it started because when I got into Rust, the first thing I did was I think I made a simple parser for localization or something just to learn some of the language. But then I was like, well, I mean, I make web servers. I want to make a web server. Where's the HTTP stuff? and there wasn't anything. So I started working on it. And then. And that was the first time that you did something like that? I mean, Did you implement those protocols before? No, I've never implemented HTTP before. I used it plenty, but never just as someone writing web servers, not as someone who cares about the finer details of HTTP 1.1 and 2 and so forth. So that was an additional fun thing to learn, also reminded that many people say HTTP is simple and most implementers of HTTP realize it's certainly not. Yeah, there is a lot of like ambiguity in these protocols and these RFCs and lot of details and how the wild is implementing things slightly different, like you could say buggy or even wrong, but it could like accept as the standard, et cetera. Yeah, yeah, have a lot of, oh, this is what the spec says, but then the real world, this is what actually happens. And so you have, you know, have de facto rules that you need to support in order to stay safe. Or, you know, for people to use your implementation, it needs to work with most of the internet. So. Yeah, yeah, for sure. Especially if you have a client and a server. So, okay, so you were started with like, I want to like have a server and there was nothing HTTP. And so you started writing that. I suppose that was still just HTTP 1 at that point. Yeah, and very basic HTTP 1 There was the Rust team at Mozilla was also working on a browser, experimental browser called Servo. It got picked back up again not too long ago. That was the next step was to meet with them and say, hey, I've been working on an HTTP library. ⁓ How do I make it so that Servo can download web pages? ⁓ And so that pushed the client implementation ⁓ and I worked with them and that was pretty exciting to do. ⁓ It also helped me to realize that was kind of the first time I was presented with, okay, they should be specs say this, but browsers have to do this. And how do you balance which one is right? Yeah, I suppose it's a bit of an impossible balance. And okay, and so then how did it move from there? My memory's a little fuzzy there. think ⁓ we had Rust 1.0. I don't remember if that happened before fitting in servo or not. then we started working on async stuff. I feel like shortly thereafter. And futures and trying to fit futures into hyper, there was a, I don't remember when that was, hyper zero point eight or nine or so is when we added async support ⁓ with Tokio. And I was using future 0.1 that was when futures were, I mean, futures were always a trait. Well, no, I take that back. So the original iteration, futures were kind of what a promise is in JavaScript. It was like ⁓ a channel, a little slot of memory, you'd synchronize on sending things to it. But that meant that we allocated with every single future and every combinator also had it to allocate to. ⁓ I don't remember who it was. It was Alex or Aaron or Carl or all together. They were able to get it to be a trait. And then combinators were basically free. And the expensive part was if you needed to allocate because you needed to share a slot of memory, But you had to pay for that anyways. It wasn't a cost of doing futures. And so they were working on that and reached out. We made a hyper start working with future 0.1 That's very interesting. then despite them reaching out quite early to you and you seem to be tied quite closely to Tokio, even though I think Hyper can also be run without Tokio, if I'm not wrong, it is still in its own organization, right? Like hyper-rs I think. Was it like always a deliberate choice or just like an historic artifact? Yeah, it's always been a deliberate choice. ⁓ I just went back and checked and found, yeah, I have some blog posts about async-hyper from 2016. So that's when that started happening. ⁓ But yeah, there was a moment where hyper was more closely linked with Tokio because that was kind of the only runtime that ⁓ was ⁓ and active development. And there were some others, know, they were less active and less of an ecosystem. And it just felt right to allow people to easily use it together. As part of Rust Hyper 1.0, which we released two years ago, ⁓ I put forth a particular effort to make it so you... Well, I mean, I... I put forth effort to remove the parts of Tokio that were still in Hyper. But even before then, Hyper was always generic over anything that implemented the Tokio I/O trait which did mean that people were able to hook in other runtimes just fine. They just had to implement those traits, which kind of felt a little weird. So I, I cleaned up, but was always an effort to know Hyper is not Tokio. Tokio is not Hyper. They're not the same thing. ⁓ You can use hyper with anything and it has been used with all sorts of runtimes Okay, very cool. Does that mean that some people are also using it with some of these io_uring runtimes? Yeah, I don't know. ⁓ I haven't, I've seen a couple of blog posts. There was one, what, last night or something. ⁓ Trying one out. I know ⁓ the I-O, the way I-O works is just a little different. And that was a consideration that we made in Hyper 1.0 to the traits since they're owned by Hyper. The idea is that we could add new functions that deal with like owned buffers instead of borrow buffers. to be able to integrate with io_uring when that makes sense. you can integrate right now just by copying the buffers, but then you kind of defeat the whole purpose of using io_uring in the first place. Yeah, it seems all a bit still in its child's Not sure how you look at it. Yeah. Yeah, I haven't seen, of the customers and stuff that I talk to, I haven't seen many people asking me like, hey, when is this going to happen? ⁓ There may be people that are pushing on it, but I just don't hear from them that often. Yeah, noted. By the way, thanks for sharing that it's a pretty amazing journey so far already. At what point of this journey did you realize that Hyper wasn't just a project, but was really something that you might be carrying forward long term? I don't know if there was a light bulb moment. ⁓ I think it just kind of started to happen, especially right around the beginning of the pandemic. ⁓ I changed jobs from a startup to working at Amazon. And the reason that I was hired, like what I was asked to continue to work on was hyper. ⁓ not some internal Amazon software. And I think that's what made me see, this is something ⁓ that can live outside of any particular company and... ⁓ and certainly with a bunch of people contributing. ⁓ Yeah, I mean that must be pretty fulfilling like because by that time you already were working on it quite a long time probably also plenty of your free time so how did you feel about that when you got hired there and did it meet the expectation? Yeah. Yeah, it was very useful. It was exceptionally useful to be there for some time because I was able to meet with companies that are not companies, with teams inside Amazon that were building really, really cool things. ⁓ And they, for the longest while, have been doing things in Java. And Rust was starting to become more more attractive, and they were ⁓ building new versions of things, new products entirely. and making use of Rust and they're like, hey, so yeah, we're using Tokio, we're using Hyper, we're Tonic or whatever. ⁓ you know, this is the scale that we're running at and we realized that ⁓ we wish that it were a little faster in this way, or we wish that it, you know, could handle this less common thing. ⁓ It was phenomenal to be able to meet people that were using it at such scale. Yeah, must be. And were you aware before already that they were using Hyper? Not, I wasn't aware to the extent of how it was used. I just, you know, had heard that, like, you you go to RustConf or something and, yeah, you know, I'm at this company and we've been using it for this thing. And like, that's kind of all you hear. Or maybe you get an issue filed on the issue tracker and you're like, ⁓ you check their profile. Well, it looks like they're at that company. ⁓ Maybe they're using it. Maybe it's just a hobby project, who knows. Yeah, so you kind of guessed, but you don't really know. Yeah, yeah, for sure. Very nice. And then at some point you also created the hyper-util crate At what point did that split off from Hyper itself and why? Yeah, that happened as part of Hyper 1.0. So I wanted to stabilize Hyper 1.0. There was a push. Various people had told me, know, like, it being under 1.0 can give a little bit of, oh, I don't know if I should use this in production. I don't know how stable the ecosystem is. Hyper could be an example to others of like, oh, well, if it can stay unstable forever, so can I. So I had quite a bit of push to make things stable. I mean, it makes sense. ⁓ All the reasons make sense. The reason it hadn't is because there were some parts that just didn't feel right. Some of the designs felt unstable. They felt like it wasn't sure it was the right way to do it. Now, that was some of the things around the client, ⁓ like the way the client, the higher level client would work. and also the way some of the server listening traits worked. So the way I solved that was I settled on, the core pieces of Hyper, the way it manages just a single connection, that's pretty solid. That doesn't really need to change. That can be Hyper And the higher level stuff like the pool and DNS and all that stuff that we're not sure it's stable yet, that can go into Hyper-Util. We did something similar with Tokio. ⁓ There is Tokio 1.0 and then there was a bunch of extra things like codec and the encoder decoder thing. And ⁓ then it was like, well, if that exactly the way it should be, I don't know. Let's just put it in Tokio Util and let people use it. Yeah, indeed. I mean, I suppose it's also because the entire streaming and async iterator and all those things are still a bit immature and in development and those things are very much related to it, Yeah. Mm-hmm. Yeah. So that stuff was pulled out into HyperUtil as a kind of exploration grounds. And as things become stable, well, there's no reason it can't come back into Hyper 1.0. Like, once we've figured out, this is the way to do a ⁓ connection pool in Client, and we're never going to change it, then it can just appear in Hyper directly. that makes sense. And then you also have other crates like HTTP for most of your basic primitive types. then you have also headers, which is like a typed implementation of some headers. How did those came to be? Did they also just spawn out of Hyper or was there a bigger story to it? Yeah, both of those ⁓ originally were in hyper itself. And, and as we were working on hyper, I was working with Carl and Alex and we were, we're making things async and, and we were trying to build out this ecosystem of futures and, services and HTTP and TLS and things. And we wanted people to be able to say, Like like what web server frameworks to be able to say hey my application works with a service of HTTP. ⁓ And I don't care the implementation. It could be hyper or it could be some other implementation. It doesn't matter. ⁓ Kind of like how how with Java you have ⁓ Circlets or. What is it in Python you have whiskey? was kind of the idea behind it. So we took some of these primitives out of hyper and turned them into the HTTP crate So that way you could set, could just depend on the HTTP types and it wasn't you depending on hyper. That was the idea behind it. yeah, I mean it's quite amazing. It's been used by a lot of different crates, as you said, not just by crates that use Hyper, but also many other stuff. I don't know the amount of downloads, but I'm sure it's like in the higher millions, especially given it's used by reqwest which is like, I mean, I think you recently had like a blog post where it was mentioning 10 million downloads per month. And as it's used by a lot of more crates, I mean, I'm sure it's used a lot. Now... How do you balance there between like maintainability and moving forward? Because I feel like those crates are like a bit, I wouldn't say like left behind, but they're definitely getting maintained less than others. Is there like a reason for it? Is because you wanted to go in a certain direction, but the timing is not right? Well, some of them, ⁓ from what I can see, just don't need as much maintenance. Like for instance, the HTTP crate itself. There's not too much that I feel that needs to be added. Maybe like a request method gets registered like query. We can add that. But to those types, there's not much to change. There is the headers create, which so. That originally was something that was part of hyper-directly. We had typed headers, and eventually that got moved out. We put just a regular kind of string-based header map into HTTP, which was more flexible. It meant less breaking changes as types would need to change. And headers became its own thing. I don't know if headers ever really became like an official This is the way people should do it. was it. The motivation I had behind it when I moved that stuff out of hyper was, ⁓ well, first of all, let me put it somewhere because we're doing just regular header map and HTTP and, ⁓ maybe come up with a way to allow new types to be added without breaking people. That was some, that was a problem with the type headers before is that we had added an implementation of something complicated. ⁓ say content location or I don't remember some of the other ones that, know, they'd have a bunch of types of them, web socket extensions, for instance. And, you know, we'd put up something basic and then someone would show up and say, yeah, but it doesn't allow us to do this, this other part of the spec that, you know, you didn't notice. Well, okay, in order to change that, you know, it'd be a breaking change and We didn't want those breaking changes to affect all of hyper. So that was another reason it got moved into another crate. And then I added in basic functionality for the simplest of types. ⁓ And then it struggled to find time in my priority to figure out a way to add in much more complicated types in a way that we don't have those breaking changes like we did before. Hmm. Yeah, I mean, I feel that, I mean, you're definitely right because despite how useful they are, as you grow that library with more and more headers, where each header can have quite a lot of complexity. mean, despite the fact that a lot of different RFCs do share like the same kind of way how you encode and decode certain things or put data together, still there's like a lot of nuance to it. And I suppose given the wild, there will also be always things where you need to consider like you want to support those kind of weird like maybe incorrect ways but still widely used ways. So I suppose it's a kind of crate which will be like, yeah, not something you can stabilize very soon. But then again, I don't know what the other option is because right now it's one of those crates which I do think has a lot of use. I mean, I'm definitely like a happy user of it. Like it's, I find a pretty great way to... to deal with HTTP data. I don't think it belongs in the HTTP crate So I think that was definitely a right call, but it's a very neat way where you in a very type safe way can access that rich data, which is like a lot nicer than, I mean, imagine if it's not there, then you as a user, need to manually parse. Like let's say you're an Axum user, you would need to manually parse those data types yourself, which can be pretty like error prone. ⁓ But then, like, I feel it's bit stale now. So yeah, that's why I was asking, like, okay, how do you see that evolving? Like, how do you deal with that kind of like, yeah, stalemate almost. Right. Yes, some of it is the. The. ⁓ Some of it was making it so that it was easier for other people to implement the header itself. It used to be like a weirder header when it was in Hyper. ⁓ So now sometimes when people ⁓ submit a pull request and I really want this complicated header, ⁓ I think the first step is, OK, ⁓ implement it in your own library or in your own application and you can immediately use it. that'll just get usage. And then hopefully, if you use it in more complicated ways, you might refine it yourself. And then eventually, OK, I've been using this for six months or a year or something, and I think it handles all the use cases. That's one of the easiest ways forward is you're not blocked, but it also allows it to be used. And then the other way is to. Yeah. Yeah, that's correct. Except of course you also have like utilities in those libraries which you don't expose. I mean, I get it why, because the more you expose, that's again something you have to worry about in your API service, but it does mean that quite often if someone wants to implement their own headers, they might need to like either copy out those utilities or like re-event the wheel there again. Yeah, yeah, it's chicken and egg. It gets hard. Another thing is... Does it have a lot of requests from your customers? ⁓ not, no, not too many. ⁓ not, about, not about headers. I think another step there to allowing it to continue to grow is probably to come up with I guess how to make these decisions, like how to agree that, this is the right kind of type to expose. This is the amount of the type to expose. Because another thing is that the more ⁓ references that you expose on a type, like if you return a function that's as ref and it can get a a reference to a slice, then you basically kind of locked in the internal data structure. needs to be some sort of vector or something, for instance. Like you can't just transform it into a vector and then hand out a slice just when someone borrows it, because then you'd be referencing something just as ⁓ a temporary in the stack. So the more references that you expose, the more of the internal structure you're kind of locking in place. So I think coming up with how do we decide like this is stable enough would probably go a ways towards making so we could merge more things into the tree. Yeah, yeah, fair enough. By the way, you, right at the beginning of our conversation, you were saying that you were interested in this because you want to like, okay, how do I serve some like website or some, I want to have a service. And I also know that you are the author of your own like framework called Warp. Can you tell a bit about that history quickly? what's the take, like what's your philosophy behind Warp, for example? Yeah, so one of the important pieces ⁓ when I was working on it was I didn't want to make something that was like everything else but just mine, ⁓ if that makes sense. I didn't want to say, ⁓ this is Axum, but Sean's Axum or something. ⁓ But I mean, did Did Axum already exist when you started it? No, it didn't. There was some others. There was a rocket and iron and ⁓ I think Actix and a few others. another reason why I really want to get into it was I'm very interested in making the type system do as much work for us as possible, kind of like the type headers. But even more so, using the type system for routing. ⁓ I also didn't like how many times in many web frameworks, not just on Rust, in most web frameworks I've used, there's a special thing around a method or the path. The path calls a method, and then everything else is different. And so with warp, everything is a filter, and everything always just extracts a thing. And it doesn't matter if it's a path parameter or a header or the body. They all behave the same. And the type system also allows you to make sure you didn't accidentally hook up the wrong kind of handler because it's enforced through arguments. There was a lot of influence from Scala frameworks, ACA, HTTP, and. What was the other one? There was Finch that I was taking a lot of inspiration from. so I wanted something that felt more like that, that used the push as much in the dive system as possible. Okay, and given there are all these frameworks on top of Hyper, will that mean that Hyper will always stay focused and slim at its core? Or do you see that maybe one day you also want some kind of richer, let's say almost framework or easier way to make a server directly with Hyper? ⁓ I I wouldn't want to add anything like routing or any of that, don't think. ⁓ Maybe improving, like adding some sort of accept layer, like there used to be. There used to be a way, like an accept trait, where you could ⁓ accept connections and turn them into HTTP connections. That moved out to hyper-util because the trait was lacking. I could see something like that coming back. But I prefer allowing people to develop different web server frameworks on top of it instead of me trying to pull something in. And warp, is that something that is like for you like a hobby project, like something you relax on or there's like also a professional purpose there? I was that was entirely just for my my interest. ⁓ No one was asking me to make it. It was some of it was to push ⁓ how we handled. H lists and parsing things of different types and pushing them into functions. And it did help. ⁓ Some of that logic got pulled into Actix and pulled into Axum. And it's now how they have different extractors. A lot of that was explored in warp originally. So I'm glad that it was able to develop some ideas. I still would love to explore more. I think that we're underutilizing. trait and impl trait in particular. ⁓ It's kind of annoying to use sometimes. So people prefer specific structs. But I think there's advantages that are underexplored. And I use things like warp and a couple of other crates that I don't really talk about as areas to test out these ideas before suggesting that other people should use them. Yeah, yeah, gotcha. And so while even HTTP1 wasn't as simple, like one should implement this kind of spec and well, collection of specifications, you realize even HTTP1 is not simple. Still in those days, it was still like at least contained within the HTTP realm. But I feel that since we moved to HTTP2 and also HTTP3, you also have quite a lot of... inner connections with other layers of the network stack such as DNS and especially TLS. But as far as I know, things like warp or Axum they don't really deal with those layers or don't really have direct support for them. Despite the fact that for example in TLS you can negotiate protocols with ALPN or in DNS you can have SVCB records or HTTPS records. Like how do you deal with that? Besides just saying to folks, yeah, you have to do it yourself. Like, is there like maybe a path forward to like where this kind of support can be more easily facilitated or how do you look at that? I think those sorts of things are rightly not in Hyper because they're HTTP-adjacent, but they're not HTTP itself. And I like that different implementations can exist. There is native TLS, and there's Rustls and there's KTLS. I like that there's that choice. And by me not picking one for Hyper, you can use whichever one you want. That said, I know that many people, show up and they don't want the choice. They don't want to think about it. They're like, can't you just pick a good option for me? And I think that's where reqwest comes in or other libraries at a similar level, where it says, yes, I behave kind of like hyper, but I picked a bunch of things for you that I think are good for 80 % of the people. And if it's not good for your 20 % use case, that's OK. You can go use the building blocks and make your own thing. And so like in those building blocks, let's then now move our scope towards requests, which like I think in a recent blog you mentioned had like 10 million downloads per month. And I know that you also worked a lot on it recently. Is that by the way, because as you were mentioning, like, if we go back to earlier in conversation, you were saying, okay, hyper grew a bit. also more and more because I was involved with Servo and like seeing how it can download web requests for Servo and as Servo, I think a year ago or a bit more got picked up again, are they using still Hyper or what is the story there? ⁓ I think there's two parts to that question. The latter one, if Servo still using it, I think so. I have not checked in on how, on what Servo is doing at all. I just saw the news that it started, they started working on it again. or something like that. Yeah, yeah. Like because there was like a lot of activity and I feel even like recently, yeah, there was like a lot of activity around request and You made some blog posts around it, how you want to make it more modular. And it's also because you were saying like, okay, sometimes people want choice and request is like a client library and there you give those like, like that you give like European native you, but at same time people can develop their own client with the building blocks. But where do you draw the line? Because At some point, even the building blocks are going to be opinionated. So is there like some way you guide yourself in those decisions? Yeah, the line still kind of stays. think reqwest becomes more and more. ⁓ Yeah. I mean reqwest, it's becoming more and more the thing that people use as a client, less so hyper, unless they're doing something where they want very fine grain control. So it does need to support more possibilities. ⁓ But I also feel like it's reqwest's job to make sure that those possibilities are still among the best options to choose from. ⁓ A lot of my recent work has been when people want most things in requests, but then they want something quite different that most other people wouldn't want. ⁓ Those people I still want to serve by doing the kind of work I'm doing right now, which is to take a bunch of the pieces that are in request and try and turn them into things that go into like hyper-util or a tower-http You're right that there is some opinions built into the building blocks, but I think that's where people can express their different opinions instead much more easily. Very cool. is like as you're also trying to ⁓ find sponsors and people like commercial folks that use your project and also pay you perhaps also for service contracts, is that more for like hyper or is there also like reqwest work in there? it's all All the above. I accept sponsors, but I also look for customers that ⁓ are using various repos that I maintain, even ones that I don't. ⁓ And then I provide advice for how they could do something different. ⁓ It also allows me to see, with some of the customers that I have that they're using requests in a certain way. And that, yeah, that helps me to prioritize. Okay, well, this is something that, you know, I have hundreds of feature requests. How do I prioritize which ones are not, which ones are important, but which ones can I definitely say are needed? And when I see a company with a specific customer need, you know, they're, dealing with users on their, in their product. it's easier to see, okay, well, this is something that I can prioritize as opposed to an issue from someone with a username maybe I don't recognize or don't even know how it's going to be used. It's just a lot harder to form a priority around it. Hmm, I understand. That's very reasonable. And then I know you have like, I suppose just like everybody else, experimental support for H3 and QUIC. Like how do you see that getting integrated into something like reqwest or even later in Hyper? Because with HTTP1 and HTTP2, those were still kind of like easily combinable. They both live in the TCP layer. You can even have like an "auto" one and once you should combine them with TLS et cetera. Okay. people can figure out how do you see it happening with like HTTP 3 added to that as it's like living on entirely different sockets. Yeah, that's what's been so complicated about it is that hyper's philosophy so far has been it doesn't want to care where the IO comes from. So we define these read and write traits, which work for HTTP 1 and 2. And you can provide a TCP stream or a Unix domain socket or who knows what. Sure, yeah, whatever you want. can be a vector even. Hmm or Bluetooth or whatever you you know, the Tokio's duplex stream. But with HTTP 3, quick is different. Like, yes, it is kind of reading and writing bytes, but there's also control streams that you have to deal with and additional error codes that mean things. So ⁓ it's been more complicated to fit it, to allow Hyper to say, you can provide me a QUIC implementation. I don't care where you got it. ⁓ In reqwest we do have experimental support because reqwest doesn't have an idea of being generic over the IO. reqwest is already chosen. It's going to use TCP streams. and not have to run your body. And how would a client know like when to make HTTP one, two or three requests? Is that purely based on the version or is there also something where you say we're going to check for a DNS record or something like that? Yeah, it doesn't yet use DNS records. It'd be nice to. Right now, the experimental support, have to tell it that there's what we call HTTP3 prior knowledge. You're basically saying, I know for sure this other end uses HTTP3, so use that instead of H1 or 2. And that's because of the version you were specifying your request? No it's on the client and the client builder, you just specify like http3 prior knowledge is true. and then all your requests will be HTTP 3 basically. I believe that's how it works. I don't recall if we changed that recently. that's more possible in reqwest because we take full control over most of the implementation. With hyper, there is, I put up a two pull requests. They're not ready by any means, but they provide a new set of traits that allow someone else to say to describe their quic implementation. They can say, this is how a quick connection works, and these are the receive and send streams. ⁓ And then there's a second pull request that is very much work in progress that would take those and then map them to the H3 create, kind of like how we are able to do H1 and H2. ⁓ Yeah, it's quite a hairy problem. But those definitely could benefit Yeah those could benefit from others taking a look. It's a lot of work. Yeah, for sure. And talking about a lot of work, like there was like very exciting news in the past years. I mean, there's always plenty of exciting news around Hyper, don't get me wrong. But when it was announced that Curl wanted to adopt as I think a second or third project, a Rust project to integrate as an alternative stack into it, a lot of people are excited and there was like an experimental implementation being prepared using Hyper and... I think both projects like learned a lot from it because maybe hyper at some things correctly or different and those got cracked into curl. And like, I'm sure also this implementation made you work a lot on the FFI layer, et cetera, and improve certain things to pass their test suites. But then after even your call for a champion, it kind of like all faded out. And I think now they're also like removing that kind of stack. Like, how do you look back at it? and. And yeah, tell a bit about that and also maybe tell a bit how we go start in the first place. Yeah, Yeah, It got started in the first place. I think it was 2020. I think Internet Security Research Group had started this initiative to have a lot of current software rewrite portions of it in Rust. And one of the places that they reached out to was Curl and wondering if they could replace their HTTP backend with something written Rust. And the idea was to use Hyper. We worked on that for a few years. ⁓ Initially, it was a large burst towards the beginning. And then there was this long tail of, there was like 800 tests or so that Curl has. with the Hyper back end, some 95 % of them were passing, but there was 5 % that, you know, they just do exercise weird parts of ⁓ HTTP that Hyper either didn't support or ⁓ didn't expose, ⁓ so that cruel can make use of and, but neither of us were really experts in debugging the other person's project. So it kind of languished there. And then additionally, there, there wasn't a, I want to say a customer that was asking for this at the end of the, at the end of the road, like a distribution that was like, Hey, once, once this is working, we would like to ship this. But all that work that was done so far, was it like paid work or that was more like passion work? It was more kind of Yeah, so I think the Internet Security Research Group paid to have some of the work done in CURL. And then I used some of my paid time at Amazon because it seemed like, well, that'd be cool once it works, we could turn it on at Amazon too. there wasn't a huge, like, OK, but you guys are stuck, and we really want this. How can we invest some more resources? That was kind of the part that was missing. And then the maintenance burden just kind of just start showing up over and over. CI would break, or it was annoying to have to refactor this and still be able to support the FFI, or something like that. So yeah, at the end of 2024, did get it could get removed again from curl. But we learned a lot from the process. ⁓ I learned a lot from working with Daniel. He's a very excellent maintainer. And curl also was improved as it found various things that, ⁓ it supported this, but hyper doesn't. And it turns out that that's actually better that it shouldn't support. So both things got improved. And are you still in contact with Daniel? Yeah, I mean, we don't talk all the time, sure. We're both in the pay attention to the HP working group from the ITF and stuff. Very nice. Yeah, I suppose you could also meet him in Basel. Like ⁓ I think next year, that's where the HTTP working group will like ⁓ meet or at least one of those groups. That said, ⁓ You were mentioning the FFI layer of Hyper. Was it mostly for Curl or was there already one before and if it was for Curl, like how did that dropping of that projects from Curl impact the FFI layer of Hyper? So it didn't exist before. It was made entirely to support cURL. ⁓ Since cURL no longer uses it, ⁓ there hasn't been any more explicit work on it. I have heard of a couple of people that that have been using it. I know that there is one pull request someone made to try and do the same thing to the server, which I assume is because they wanted to use FFI but the server API. But I haven't. I don't know much more than that other than what's in the pull request. But it continues to stay unstable. And we could remove it, I suppose, ⁓ unless there was some customer that said, no, no, no, we really need this. And in fact, we'd love to make it stable. But I haven't heard of that. So I don't know. And I believe in 2024 you announced that you were gonna work full-time as a maintainer or at least around things like Hyper, which of course was also a bit true already at Amazon, but that you wanted to be more independent, so I suppose you left Amazon. How did that turn out so far? Are you still walking that path and how do you balance that with the commercial work as well as Hyper? Yeah, that was in 2023. It's now been two years. It was May 2023. And it It continues to work out really well. I continue to be able to work largely on all of the open source things. And most of the paid work that I do is advising. And when meeting is meeting with a customer and learning that they wish Hyper or reqwest or Tower or something did something a little differently, then I go and spend some time doing that. And the key part there is that it's benefiting the open source project. So ⁓ it continues to be a very kind of similar situation to when I was at Amazon. Instead of meeting with internal Amazon teams, I'm meeting with people at all sorts of different companies. ⁓ It means that I have to do more sales, which is less fun. I know some people love sales. ⁓ That's not me. But it's also given me a ton of flexibility. So I appreciate that. Yeah, for sure. And I'm also very glad it's working out for you definitely. Very well deserved given how much value you're creating in the ecosystem. Now, how do you imagine Hyper to look five years from now? What do you hope it looks like? And in that kind of setting and environment, what role do you see yourself playing in that future, if it existed? Ooh, Oooh, in five years from now. Well, hopefully, HTTP3 is finally fully working. And if there's an HTTP4, I'd certainly hope there is not. I not. And and I hope we've made it a lot easier for people to build their opinionated stacks of servers and clients, that we've made it easier for people to reach for the the safe option ⁓ that we've advocated for ⁓ using the safer choice instead of the common choice, like using the retry budget instead of retry limits or something like that. ⁓ I hope that we've been able to continue to maintain the security standards that we set. We get several reports, ⁓ a few reports a month. and some of them turn into something bigger. I hope that we continue to have ⁓ collaborators that are willing to spend some time on those kind of stressful things. And I hope, I kind of hope that, not that I want to be less involved, but that if I were hit by a bus that it could continue on. So, yeah. How is that story today? Like, I know you're accepting more, let's say, co-maintainers or contributors. Is that turning out that, Is it growing your team? Well, I'm not sure if I can call it team, but still. Tom, I think it's, If you look at the list, we have, I don't know, 10 or 12 collaborators. We've We've slowly added a few each year year. And what also is true is that oftentimes many who were more involved become less involved. Maybe they change jobs, they're no longer even working on HTTP, or ⁓ they're no longer allowed to really work on open source anymore or something like that. So ⁓ it probably makes sense to go through and check the list and see if they're still interested in being active or if we can put them in the, what do call it, emeritus. ⁓ It's It's going okay. I'm sure it could always be better, but I also know that there are a few core collaborators that if the city I live in fell off face the planet, they could pick up and take over. That's very reassuring. Well, hopefully it doesn't happen. We wish you all the best either way. Like I know you were also like pondering in like an old blog post, like how you can encourage more contributors or collaborators. Did you already have some answers on that? Yeah. ⁓ ⁓ It's an ongoing thing that I discuss with people and myself and in a journal and existing collaborators. And so within this context, is there something specifically that you would like to see happening from companies or contributors to make the Rust network ecosystem healthier? ⁓ well, sure. mean, an increased sense of ownership and contributing is, would be healthy for the ecosystem. Certainly. ⁓ and what I mean by that is taking an interest in, ⁓ we depend on these things. Maybe we should care about their general health and not just, I have one specific bug that I want fixed. ⁓ That would be very healthy for the ecosystem in general. ⁓ Not just Hyper, but with Tokio and futures and all sorts of things. Are you still very heavily involved in Tokio by the way? ⁓ no, I do less. I do less. Okay, and then as we are like near near the end of our conversation for which I'm very grateful, like, is there anything you wish to say to your sponsor, future sponsors, maybe your Patreons, maybe people that sponsor you or on retainers? ⁓ Well, I appreciate it. Thank you for having me. I appreciate all those that sponsor and those that get retainers. ⁓ Allows me to keep working on it. ⁓ And I love what I do. And we love what you do Sean. So thank you for coming on this podcast. You were our first guest in our virtual studio and I've been wanting to talk to you since a while already so I thought this was a great opportunity. I learned a lot about the history of Hyper which is like going a lot deeper than I had imagined and I learned a lot about the human behind Hyper and the Hyper-Eco system for which I'm very grateful and I appreciate it very much. because behind every implementation and every protocol there is a hard-working human and even more an entire team around it of contributors, some long-term, some short-term. So very cool to see that happening. I wish you all the best, Sean Yeah, yeah, thank you very much. And you did remind me of, I guess, one last thing to say, which is what I kind of said towards the beginning is if there's anything that could do is to just make better software software. It always pains me when I see friends that are less technical using software that kind of fights with them or or hurts them in some way because of using insecure patterns or just not as much care. If there's anything we could do, if we could make better software, that'd mean that all the humans in our lives would benefit. I couldn't agree more. Thanks very much Sean, take care. Elizabeth (Plabayo)
1:02:40 | 🔗
Netstack.fm is brought to you by Plabayo building secure, open, and resilient infrastructure with Rust protocols, and purpose. This show is also made possible by Rama, the open source networking framework. Plabayo offers service contracts and welcome sponsorships to keep building and supporting its ecosystem. The theme music of this podcast was composed by DJ Mailbox. If you enjoyed this episode, don't forget to subscribe on your favorite podcast platform and leave a five-star review. It really helps others discover the show. Thanks for tuning in. We'll see you next time for the next handshake.