On this page On this page
Episode 6 – Curl with Daniel Stenberg.
In this episode of netstack.fm , Glen speaks with Daniel Stenberg, the creator and maintainer of Curl, one of the most widely used networking tools on the internet. They discuss Daniel's journey into programming and networking, the evolution of Curl from a simple tool to a comprehensive solution supporting multiple protocols, and the challenges of maintaining such a large open-source project. Daniel shares insights on the importance of community involvement, the complexities of debugging across various platforms, and his reflections on a 30-year journey with Curl. The conversation highlights the significance of open-source contributions and the future of Curl as a project.
If you like this podcast you might also like our modular network framework in Rust: https://ramaproxy.org
00:00 00:36 05:20 08:10 12:46 15:58 17:40 22:17 25:33 28:33 30:16 36:37 41:38 46:44 49:53 51:58 57:05 01:00:07
Music for this episode was composed by Dj Mailbox. Listen to his music at https://on.soundcloud.com/4MRyPSNj8FZoVGpytj
Elizabdeth (Plabayo)
0:13 | 🔗
This is netstack.fm, your weekly podcast about networking, Rust, and everything in between. You are listening to episode six, recorded on the 9th of September, 2025, where Glen has a conversation with Daniel Stenberg creator and maintainer of Curl one of the most widely used networking tools on the internet. So welcome everybody, it's another week for netstack.fm. Today with me is Daniel Stenberg. He is the creator and maintainer of Curl. It is one of the most widely used networking tools on the internet. So welcome Daniel. Thank you. Good to be here. Yeah, very nice to hear you. So today we want to explore a bit with you, your journey through decades of protocol work, as well as the story of Curl. I know you have like an amazing blog and other resources where you explain a lot of this already. So sadly, we won't be able to go as deep in the weeds as you do in your blog post through all these years, but I do want to at least scratch the surface for some of the people that might not know you yet. So... Could you maybe quickly introduce a bit like how you got started into like programming and networking, et cetera. yes, well it's a kind of a long story. Okay, so start out. So I'm Swedish, so I'm based in Stockholm and I started everything or my journey into computers started in the mid 80s actually where I got my Commodore 64. I actually found friends who had a Commodore 64 and got into that and then eventually bought my own together with my little brother in 1985. And then that was sort of kickstarted me, my interest in computers, programming and everything. ⁓ I got hooked and I'm still sort of fascinated by it. I still enjoy it. So that sort of changed my life in some ways. as a teenager then, I started programming. And I've been programming ever since in In early 1990s or mid 1990s I got into, I actually changed jobs and I got into a programming work where we had direct internet access. I learned about IRC and started to use, well internet directly, go for an Archie in those days and then of course the web. as it started to grow. And while working, when I discovered IRC really, that's how I got into network programming because I started to work on IRC bot together with a friend. That was really my first ⁓ steps into TCP programming, ⁓ client programming, open source, NC and everything. I have sort of. of course grown into that but that's how I got into programming networking stuff really and it was fascinating and I enjoyed it pretty much from the beginning. It was fun to make it sort of you know the networking part of it all. Okay, and so you mentioned there TCP because like I've used IRC a lot as well in my early days, but I never programmed a bot. So was it like some kind of protocol on top of TCP that you had to use? Oh yeah, so IRC is a protocol. This is an RFC. I think it's 1459 or something. So it's an old RFC from the, I think early 1990s or something. And it pretty much describes how to talk to a IRC server. So yes, but of course, when I started that, I didn't really know about what's a protocol and how do you do this? How hard can it be? I want to do this. And then I pretty much learned along the way. figured out about the IRC, checked other source codes for other clients and other bots. How do they do it? ⁓ So yeah, was a learning curve really into all of that. How to do TCP programming, how to speak protocols, how does it actually work? And IRC, ⁓ as many protocols, the RFC, it describes a protocol, but only so far. rest of the world has moved on a little bit since the RFC. So most of the clients actually didn't follow it completely and there were huge chunks of the IRC client things that weren't covered by the RFC. So there was totally outside of it. So it was good lesson in how network programming and different clients, different implementations still cooperate online. Yeah, and I guess that's a bit true today where there might be specifications, but they are written after the facts or like some more like a bit ambiguous and clients can choose how to use it. Absolutely. And as always, because a spec is language primarily, right? Or someone's vision of how things should work and then reality sets in and things change and time passes and everything. So over time, maybe everything in the spec is not exactly the way, know, implementations need to do or something. we, over time, we deviate a little bit and people interpret the language in the spec different ways. yeah. It is certainly exactly that way still. I mean, nowadays in real life, course, we over time, we try to make up for that. Hey, we make a new draft of the spec. We make a new version to accommodate everyone's new understandings and ideas how it actually, and since I've been working with HTTP so much, that's certainly how the HTTP protocol has evolved over time that. every decade or so we have to refresh it because then we have gathered a lot of new experiences more people and more implementations and the wording we did back then maybe it's not as clear maybe it was a little bit wrong and now we clarify and write it again and elaborate a little bit more to make sure that future implementers get a better understanding for how things should be. Yeah, yeah, correct. Even though I have feeling that those specifications, so there are a lot of them, a lot of RFCs, and I feel a lot of them, even though they are widely used standards, or never really going into like, like the status of the specifications, a lot of them are like standard or standard or draft, but not like, I feel a lot of them are stuck in draft while not being like accepted, even though they are being used. Yes, mean, there's a wide range of how standard and how experimental they are. there's certainly the entire spectrum of different specs. mean, a lot of specs have been written that were never implemented, never, they were sort of failed specs and shouldn't probably never have even been published because no one really cares about them. So there's certainly... And then we have the other end of the specs that we all implement them and we follow them and we try to make sure that everyone else does too. So RFCs as a concept is certainly a wide set of different specs really in that regard. And so if we go back to earlier in our conversation, you were telling where you came from and we also know now where Curl is. But I want to know how did it get started? I suppose it's hard to imagine that you had the vision of what it is now that it would become that. So my question is, like, what did you initially want to do with Curl and why did you build it? Yeah, exactly. As you say, when you look at what it is today, it's easy to think that we had some kind of vision that led to this point in time, but it certainly was not like that. So it just started as a little toy in because as I was working on that IRC bot that I was mentioning, it was called Dancer, by the way, the software project, and it was open source. We had contributors. ⁓ made it online of course and sort of helped out as an early open source project. ⁓ And while working on that RZBot, I figured that, hey, we should add a currency translation service to the bot. So you could ask the bot, hey, how much is 100 US dollars in Swedish currency today? I mean, how hard can that be? We just need to download the currency rates every now and then from, and I found some sites that I could download currency rates from. Okay, how do you download currency rates? Yeah, you just need a little tool to download HTTP. So that's how I started. And I found a little tool called HTTP get in late 1996 that I started this journey with. That's what's basically a hundred lines of code. It wasn't written by me then, I found, and it was also before Google, by the way, so couldn't Google for it. ⁓ So, but. ⁓ It was written by a Brazilian developer, Rafael, and it worked almost the way I wanted it to. So I patched it and I sent him some patches and he quite soon got bored by my patches because I think he just published it online. I don't think he wanted to maintain it or work on it anymore anyway. So he basically told me to... take over and do whatever you want with this and I did. So I became the maintainer of that HTTP GET project pretty quickly. And then it went on from there. So we started to download HTTP. I found the site on Gofer, added Gofer support. I added FTP support and I added upload support for FTP. So I changed the name once because it wasn't only HTTP anymore. And then I changed it again because it wasn't only GET anymore. So I changed the name twice. And the second time I changed it to curl in March 1998. C for client and the URL for URL then because it would specify the other endpoint as a URL. That means all the protocols you support are like supporting URLs in one way or another. Yes, exactly. that has been a little bit what I tried to, you know, and one of the ways in an open source project, of course, and any software project, it's easy to get a, you know, a creeping featureism that you just add and add and add stuff. it is somewhat important to know exactly what is in the project and what is not in the project, you know, where to draw the line in the sand and how to say if someone says, hey, we should add this feature. Is it in or is it out? ⁓ And I think sometimes saying no is sometimes more important than saying yes. So let me make sure that we stick true to some kind of core idea. What is curl really and what isn't it? So yeah, one of the core things is then that you should be able to specify the endpoint with the URL. So there should be a URL syntax standard for the endpoint. And in some cases we're maybe not staying true to that requirement but we try to. And also another thing is that curl should only do what I call transfers like uploads or downloads and not anything else. So this should be URL, it should be uploads and or downloads. We're not entirely true there either but that's sort of that's what I try to. focus on that's what curl should do. If it's not that then it's probably not for curl. understand. So I understood as well why you built it initially while you forked it from HTTP GET, well more like you patched it and then it became HTTP support obviously because that's how we started. But then you mentioned you also added Gopher support and FTP support and it wasn't entirely clear to me why you did that. Was I just curiousity? No, yeah, no, because I wanted more currencies for my currency translation service and I found other sites on those hosting data over those protocols. So I added that to the tool to just to get more protocols from, sorry, more currencies for my currency translation thing in my bot. So I was still sort of working on that bot and I just wanted to improve my currency translate. I mean, probably just completely pointless because I got a lot of currencies but currencies no one actually cared about but it was more of a sure why not I can do this how hard can it be I want to have more currencies and of course then over time I kind of grew bored with this currency part of that and I stopped doing that but the tool stuck and by that time I also got more contributors right and I had a few users people appreciate it so we fixed bugs we added features and you know hosted it online and then took off from there. In 2000, which is done about two years after I renamed it to curl, I also rewrote the internals a bit or sort of massage the internals a little bit to create the libcurl library so that you could then build curl as both the library as well as the command line tool. And that also made difference because as a library it's easy for others to well use curl as a component into other products right so that that also helped deployment usage more because other tools and devices and products also of course wanted that internet ability into their things so projects and people quite soon adopted libcurl into their projects to use for doing internet transfers which then helped curl getting more widespread. PHP for example was one of the first early adopters that they pretty much immediately once I had done libcurl they adopted that into and made it the default HTTP URL transfer thing in PHP. At PHP you know it has really well, taking over the internet and the web is used literally everywhere. by extension, they took Curl with them and made Curl also pretty widely used just because of that. and they still use it as the default backend for the client. Okay, and so you see me like the kind of person which likes to dog feed on its own products and scratch their own itch as you already mentioned, but you also do plenty of web server stuff. So I wonder how come it never, how come you never expand your scope to also include like server capabilities? I think because it's a completely different set of challenges. ⁓ I think ⁓ originally we were open to the idea of making it maybe something for the server side as well, but it's different. So doing servers is so much different than doing clients. So I think we're actually... ⁓ Early enough, we never got enough interest and traction in doing server side stuff. And I think it was a good move. So looking back, I think that was a wise decision to avoid server supporting the server side because it's so much different and scaling and doing server web servers and all these other protocols servers is a completely different thing than doing it clients. So I think we kept things simple and easy, the API easy and straightforward by by not doing the server side. So we're strictly client side. Yeah and I do think you're right in that because once you go into clients and servers I mean the combination is quite crazy sometimes. Now I do wonder talking about crazy is if we go back to the starting point and we go all the way to now where you went from HTTP go for FTP do you know by now how many protocols and total you support? Yes, we say that it's 28 different URL. It also depends on what you mean with a protocol, but URL schemes we support at least 28. So, know, HTTP, HTTPS, FTP, FTP, SSFTP, SEP, IMAP, SMTP, POP3, you know, there's a wide range of protocols. So yes, quite a lot of them. ⁓ I think the most recent ones are the WebSocket ones, ws and wss colon slash slash. Yeah, and that's still HTTP based, but then what non-HTP... Well, yeah, yes. Yeah, they start off over HTTP, but they upgrade into WebSocket. Yeah, yeah, correct. But then recently you also have like quic support, even though for that you started using other kind of backends, right? Well, yeah, so QUIC support, yes, we have that because we want to support HTTP 3. So we don't actually particularly highlight QUIC because QUIC is a mandatory thing to support HTTP 3 and we want to, so we support, you know, HTTPS colon slash slash and what HTTPS colon, what does it mean? We have a way to then ask for different versions or negotiate which version we speak over the wire. So we speak basically all versions of HTTP that ever has been. used you know 0.9 the first one that was actually made before the first rfc was made and then we then was 1.0 1.1 HTTP 2 and HTTP 3 and we support all of them and HTTP 3 of course is done over QUIC which is done over UDP instead of TLS and TCP Yes, but then for example based on QUIC you also nowadays have things like WebTransport. Is that something you support already or plan to support? That is sometimes brought up as an idea. I have my doubts about Web Transport as supported in curl. So I don't think so. I wouldn't say never really, but I don't think we're going to support Web Transport now. Okay, I mean I understand why because it seems out of scope but I ask because you also mentioned that you support WebSocket which also seems a bit out of scope for curl. Yeah exactly. I agree but WebSocket is also often used just for transfers and yeah so I agree WebSockets is a little bit in the grain zone. Is it really a protocol for curl? Yeah but now we have it. Is it only in Lipco or also in the CLI? Because if it is in the CLI, then I wonder how do you use it? Well you use it pretty much like you would use telnet. basically a session and it is mostly so it basically just echoes what you get and because in a lot of cases people actually use your ⁓ web socket just to stream data so you can just ask for a web socket stream and you can output it somewhere and it's sometimes kind of useful to do that as a command line tool. Yes, yes, even though I always imagine it mostly useful for breaking out of the limitations of a browser, but once you go out of a browser, I would think there are better options than WebSockets, or am I wrong? I think you're right but I think there's also a lot of existing infrastructure already using WebSockets and then you're sort of limited to sure this is how the other end works right so and if you if the other end works with WebSockets then you need to play with WebSockets so I think that's the reason why curl supports it because yes if you end up in a situation where the other end actually says sure this is WebSockets then we can speak it and as you say it's probably more of a lib curl thing because you probably want more interaction back and forth and you probably in most cases you have a particular since WebSockets is more like sockets so it could be pretty much anything so you probably have a when you do speak WebSockets you know something about the other end you want to do some particular communication with a particular implementation in the other end so you probably write something special in your app against something special. Okay. And then I wonder as you are, as far as I know, you are the only full-time employee on curl. I mean, not sure if you can call it employee, but the full-time developer since all these years. And so I wonder how, how do you get your head straight across all these different protocols and, and all these with, what is it like 200,000 lines of C code? I mean, it be something like that. Yes, it's about 180 000 lines of code. Well, how I do it. Yeah, so I've been doing this for a long time. All the protocols have been added gradually over time. I like network protocols. I think it's fun. enjoy it. So yeah, I read the specs. talk to people. I ⁓ try things out. So I don't know how do you learn things. I just go about and try it and work on it. write a lot of tests and fail a lot of times and correct the things over time. There's really no silver bullet or magic trick to use other than just set your mind to do it and do it. And of course, ⁓ just accept that of course I forget a lot of things. So I often have to go back and read the specs again or just read the code again because I've forgotten about the details that I wrote. Only last year, I don't remember. But I can always go back and refresh my mind and read the specs again. ⁓ right. This is how it looked like. Okay, and so as Curl evolved and as YouGrew as a person, a professional, in the meanwhile also C evolved, not as fast as some other languages, but it did change a lot since you used it for the first time. Do you as a Curl maintainer also keep up to date with the latest C developments and try to use them, or do you stick with one of the original versions? Yeah, so I'm more of, I don't follow the development and I don't follow along much. I'm really stuck in the old style of C. So we're using the C89, sometimes called C90 version, which is really the, yeah, really legacy version. And for reasons, right, because for a long time, it wasn't really possible to use anything more modern. And Possibly we can do it nowadays, but nowadays we're sort of a little bit stuck in old habits and we've felt like we don't feel any strong reasons to bump the requirements. So by sticking to this old legacy version of C, we also maintain this ease with how you can port it to all these different operating systems. I mean, we have this, I have this, I collect a list of all the operating systems curl has ever run on. and it's over a hundred different operating systems. And that's amazing in itself. But one of the reasons we can do that is because we stick to this old C version. So compilers are easy to get to there and compilers typically start out as C89 support and then possibly add more over time. But for some platforms it has been a real benefit to just stick to the old version. Yeah, for sure. And I want to also ask, and without any more intent, because I do like C myself, but the landscape of programming languages changed a lot today. And so I wonder, let's say the programming landscape of today would be the one in the nineties. I mean, it's a bit weird to imagine, but let's say the language of today would be then available. Would you still have picked C or would you have chosen something different? Yes. I actually yeah so since curl as a command line tool is separate from if I sort of separate them from the lib curl the library so like doing a library as in a system level in an operating system even today it's actually quite hard to not use C I mean there are or put the other way around is that there are actually many benefits of writing them in C because of limitation and a little bit how the different languages today are made. I I do shared libraries and I think shared libraries is a necessity for operating system level libraries. And you don't do that with Rust. You don't do that with Go. You don't do that with a lot of other modern things. So while C is not actually the only option available, it's really the most mature and the most solid solution to do those things. Sure, there are downsides to C, right? It's not memory safe and some things are ⁓ a little bit awkward to do or uncomfortable to write code. But that's also one of those things I'm so accustomed. I'm so, you know. used to doing it this way. it's also one of the things. ⁓ I'm certainly most productive in C, so changing language would be a huge undertaking. So huge. I won't change language. But I also tend to sort of emphasize that I don't rewrite curl in another language, but I don't rewrite it in C either. I don't rewrite curl at all. I keep polishing. That's what I do. I keep tweaking things. We keep adding. tiny things, we don't rewrite it because that's the way I think we keep it solid and safe and secure that we don't rewrite it even though there might be fun new languages coming. I understand. And it's pretty amazing to see your continuous development on Curl. And I've been following your blog since a long time and following along your progress. And even today you still have these kind of patches and updates of things which seem like minor and you would think maybe it was solved long ago, but you still managed to find new improvements, which is pretty amazing. But something I do get lost in is after all these years. I really don't know all the command line arguments and the flags and I wonder if you do. I think very few people actually know most of them. I think I know, well, I'll say it like this. I would have a hard time to list them all out of my head because, so we're doing a release tomorrow and we have now we have 272 command line options. And of course it's impossible to remember all of them. I can't but. if I would given a command line option, if you say, hey, what does this option do? I think I actually know what most of them do, if not all. So yeah, since I haven't implemented all of them myself, but I have certainly been involved in pretty much all of them. yes. And it is fascinating in the regard, as you mentioned that we keep fixing things that you would imagine have been fixed already because we've been doing this for, yeah. closing into 30 years, but we still have bugs and we still add things, right? We still change things. So we still introduce new bugs and the world is changing. So servers are changing and users are finding new ways to use things. so yes, it's just this continuous journey of things changing everywhere and moving. So all those moving pieces, yeah, they seem to be, that seems to then just make it that we will never be done. We will always keep finding things to change, improve, fix and tweak. So yeah. And then. I mean, that's nice, right? Because you enjoy the work. so like three years ago, I started my own network framework and I like it a lot and I like working on it. And yes, I did make the choice to do like I aim for proxy. So I do both clients and service, which makes it like the scope maybe like a bit big. And for me alone to do it, I also have three small kids and Sometimes you are lucky that you find companies that want to use it and want to hire you for a short time to add some feature or make a custom integration with it. But these streams don't seem to last long. And so I find it quite challenging to combine it with my family life, with getting income and doing the work I love because I mean... you're doing it close to 30 years. I wouldn't mind working on this also for 30 years. Like it's very pleasurable and each year you do it, it improves, you see it growing and these kinds of improvements accumulate. You can like each year you work on it, like it's very fun to work on and you can reap the benefits. But like how was it for you to combine this? Because I also know you have kids, you also had to have income because in the end you have to eat, but you also like to work on it. So. Through all these years, how did you manage to do that? Well, all the ⁓ first few decades, was just my spare time project. then, sure, I had a full time job. I did my curl stuff on my spare time. So late evenings, weekends, whenever I got some time off, I worked on curl. And then I had my full time job. And as you said, I have two kids. They're not small anymore. So I don't have to take care of them much now. Still, sure, life happens and you have a lot of commitments to do. Take care of the house, move the lawn, laundry, whatever. So yes, it's been challenging at times and I've lost a lot of sleep in the late nights when I worked on Curl instead of sleeping. ⁓ Again, that's hard, but one thing I've sort of tried to just lean back on is that time is also one of these things that you can just, you know, if you really make an effort, can, and you make, I've always, one of the skills I have used, I think, is that I'm pretty good at starting where I left the last time. pretty good at taking advantage of the time I have. So if I had 15 minutes in the evening, I could use those 15 minutes. And you know, then you get another hour tomorrow and two hours the day after that. And if you're able to actually use that time, the time adds up and then it'll become something over time because it adds up. And that's of course a difficult thing and a challenge. And as I sometimes say, it's also luxury because in a life you have to, there are a lot of things that need to work. of course, having a family, sometimes meant that sure, my wife took care of the kids while I sat and poked at the curl instead. And of course that has to work, right? It's a luxury for me to have that set up. But sure, just me wanting to do it, having a lot of fun and spending a lot of time at it. And then over time it started to become something. And of course, maybe it's open source. So I got a lot of people helping out, know, sending bug reports and offering patches to do things. So I was never completely alone. There was always a community out there working with me on these things. I understand. like I'm also doing open source because I've always done open source and it's only, this is my first big project I tried to my own in the past. was usually jumping on existing project and helping them out and learning a lot from that. But so, so I get it and I get what you're trying to say, but, maybe it's just me, but so let's say, okay. I did my work in the day, I also spent some time with my kids and now I find 50 minutes to do it, which does happen. But then those 50 minutes often turn out in one hour or two hours because I kind of like lose sense of time and then you lose sleep, which is not very sustainable. And next to that, I'm doing my day job. And while I like to work on many things, like I wonder and often think about the other project and maybe I even like sometimes... do some quick work of my project in what I should do, work done and you kind of like lose motivation. I wonder like, okay, I don't know. I find it very hard to combine and just think like, okay, I just keep 30 minutes to that or one hour to it because you think if I just give it two hours here, then I make even more progress and it's maybe even also feels more satisfying or fulfilling or I don't know. Like it's quite respectable that you've been able to... compartenise as well as you did for so long. But of course it's hard sometimes. Sometimes as you say, you get just sucked into something and then you spend far too much time on it. And sure, sometimes it would become, it was a challenge sort of, sure, I have my full-time job and I have my current stuff and I want stuff done in both, right? And there's a life. So how do we actually get time to do all of that and keep everything working? sometimes of course, one piece in that would break somehow or you would sort of not give it enough attention. And yeah, it was hard at times, but yeah, it worked out in the long term. And is it because the world was different? Like, did you never think much earlier, okay, maybe I can find a company that's willing to, that is building on Curl and maybe they can, maybe they won't sponsor me, but maybe they hire me to work full-time, which is something that happens nowadays sometimes in open source. Well, I never had any such offers, so that was never an option for me. I think I pretty early sort of wanted to work more with something like Carl, but I never had any real opportunity to do that. So ⁓ I worked in embedded systems primarily. did contracting as a sort of a consultant within different embedded, mostly different devices and things here in Sweden for many years. then at some point when I sort of finished up a contract and I started to look for another sort of mission to do, what should I do next? I was contacted by a friend who worked at Mozilla at the time. And he asked me if I was interested to maybe work on Firefox next. That was the first time, that was late 2013. And that was the first time I actually got an offer to do something that was related to protocols, client networking and all of that. of course that was a little bit of a dream come true already then. Because sure I wanted to work on HTTP full-time, because it sort of a little bit would combine my words a little bit. my spare time project curl and doing HTTP work hours. That would be great. So that's, course, it was a perfect match for me. So I ended up working for Mozilla with Firefox for five years then. But some. But that was still work that's not on-curve, it's work you like. Exactly, right. It was still just similar technology, not curl. But I allowed to spend some work hours on curl then, whilst I was employed by Mozilla. So then I could actually do a little bit more curl than just spare time. So spare time and a little bit of work time. Yeah. Well that does help a lot, at least then you might hopefully get some more sleep. But of course by then your kids were also starting to, I imagine, close to university or if not already. Yeah, they are by now, yeah. But by the time you were at Mozilla... No. and then, but by now, like finally, I think now you're at a point where you can work full-time on curl and have your open source donations to pay for you and maybe even some other contributors, right? Yeah, actually what pays most of my bill is are big companies that pay me for support of different kinds. So yeah, I offer curl support for commercial companies, which mostly means that I guarantee response time and help them out with their problems swiftly. So companies that Does it eat a lot of your time? Because that support might mean that you either troubleshoot with them or being in meetings with them or diagnose issues. Yes, it depends on, you know, it goes up and down a little bit depends on their problems. Usually it's more like an insurance for them that if something goes really bad, they know how to contact and I have a guarantee to help them with their problems ASAP. Because they are companies for which the curl component is crucial or extremely important to their... business, product, service, whatever. So they understand and appreciate the sort of getting things fixed in case they run into problems. But usually they don't run into problems. So they usually don't need to engage with me on a day to day basis. So in a normal day, I don't interact with any of them in any particular degree, but sure, sometimes I need to do a lot of that. So it really depends on what happens. Yeah, and so yeah, usually I'm pretty positive, but then sometimes I get into despair and something like Curl is quite huge. mean, among the open source projects is probably something that most projects will never reach to the use of like the extent of how much it's used and how big it is and how established it is. But then if I see how much donations you get, it still seems pretty minuscule to all the revenue. that you probably generate among these companies. And recently you also gave like a talk where you like mentioned, all these car companies use my projects, but then none of them contribute anything or even pay. And I would think if all of them just pay a little bit, your revenue would probably like skyrocket. And so, yeah, how do you feel about that? ⁓ Yeah, that's sort of my dream outcome. yeah, so you're right. I often say that and I feel that sometimes I have these weird incentives in the project that the better product we make, the less people contribute. Because the better it just works, so no one feels the need to actually help out. well, everything just works. We don't need to pay. most of my customers, were paying customers that pay for support. They sort of they become customers because they run into a problem or they want a feature either or. But that gives me this weird incentive that it's good for me to not ship a perfect product. I need to have some sand in the machine, right? So they every once in a while I need companies to run into problems so that they come and understand that they need to pay someone for support, which is weird. But yes, otherwise, I of course would like more companies to realize that it's in their interest to make sure that we survive and that we can ship a product even next year and the next year and the next year if they rely on it as a component in their products, right? I mean, in many cases, these companies, mean, those 47 car companies, they were just fun example because they're known brands and they were so clearly visible and I could do this visible slide. in some cases they might not even use curl a lot, right? So maybe it's not even worth it for them to pay anyone for curl because it's just a silly thing they just use once per month. But some of them use curl all the time and to them maybe it would be a big bloat to their business if curl wouldn't exist next year. So for them it would be a business interest to at least make sure that we survive and that we manage to keep doing things correctly and safely and securely and you blah, blah. So, it seems that that's not the world. That's not the way the world works, right? It's very market economy. They don't have to pay, so they don't pay. We give it away for free. The licenses, they can do this. It's totally legal and fine. So they do that. They don't have to pay. So they mostly seem to think that someone else will pay and sometimes someone else pays and sometimes someone else doesn't pay. Yeah, I mean it just makes me wonder what all the much much smaller projects in the world have any kind of hope of ever getting like sustainable and being able to work on it full-time. Yes, yeah I agree but then also we're all in different boats there so for example again then if you have more problems if your project is harder to use but feel a need then maybe that's actually a better position to be because then you might actually have a better position to sell your services around it and looking at other similar or projects in sort of in my position there. There seems to be a lot of different solutions and a lot of different situations where people are financially. So some projects seem to manage really well depending on factors in their area, right? So I don't think you can tell. You can't really judge the situation for the entire open source ecosystem just based on my situation because there's a lot of different situations. I understand. And so we talked a bit of, okay, you can get sponsorship, you can get hired by a company to work on it, but maybe another option would have been that you try to build yourself something commercially around curl. So you have curl, but then maybe you build some kind of commercial product or service, which heavily relies on having this as your core component. Is that something you ever tried or thought about? We have thought about it, we have discussed it. We have not done it, but we have not ruled it out either. So maybe at some point. ⁓ the difficult thing, well, first you have to figure out what that extra commercial thing should be. What kind of product would that be that would also be attractive and get customers. But then you also need to spend a lot of time on that and you have to develop that and sell that and market that. so it's a challenge in itself, right? Because then back again to what do you spend your time on? ⁓ Right now I spend an enormous amount of time on curl. So if I would say, sure, I want to do a commercial other product using curl and spend, I don't know how much time on that. I don't know, I would have a challenge to spend a lot of time on something else than curl as things are right now. Yeah, I understand. But I'm thinking more like, okay, now you were in this position where you can do this. But let's say we are going back to maybe 2007, 2006, where you were in a very different position. You were spending anyway your daytime on a day job. I would think maybe it's possible that, and so I wasn't meaning some kind of commercial version of curl, but more something where you yourself have also a company that heavily relies on curl. Right. And so, yeah. Yeah, but that's what I mean, but that still needs a lot of development and a good idea and a good product. ⁓ Yeah, yeah, correct. But yeah, maybe I'm a bit dreamy, but I would hope you might be able to find something like that and where you... If you accept the fact that you have eight hours of day job time or something, that instead of working for that company, you would work for your own company, work on this product and given that you would be maybe as a single developer, also the amount of revenue you have to make would be like less. I would think something like that is possible, especially given how early days you were in the web. Yeah, again, maybe I could have, but that's sort of a totally hypothetical scenario. I did not have any such ideas, so I didn't do that because I had nothing to do other than curl. And for curl, I never wanted it to be anything else than open source. I never considered anything else than open source. And I don't think it had been this success. unless it had been open source either. and I actually, when I talk about open source and I talk about curl and all these 20 47 current companies not paying anything for curl, then so many people are just saying, so why don't you just relicense curl under another license that makes them, I don't know, less likely to be able to do it like this. But then it just goes back to sort of as sort of a circle here that they wouldn't use curl if it wasn't this sort of liberally licensed and open for them to use. Because if I would say hey tomorrow it's gpl they wouldn't use it they would just use the previous version that's not gpl and they would fork it and go somewhere else because that's how they work right. So it's a little bit like that too. It's not that easy to just get someone to pay for it either and just changing the license even though people keep saying that I could do that just to get them to pay. It's not that easy either. No, correct. And so for example, I also have a very permissive license and I think it's the right thing to do. Over time also plenty of people contribute and I would feel it's kind of like a betrayal to them as well. And as you say, a project usually will not have that much success anyway or chance of surviving if it isn't as permissive as that because that's just the way it works. So it's a pretty challenging equation. ⁓ But yeah, you managed to pull it off, so that's very cool for you. Now, as the project has been going for so long, as you said, close to three decades, I wonder, are you ever thinking about some kind of succession plan or how this will like live beyond you? Yes, to the extent that I'm making sure that everything is laid out to successfully work even if I'm not around. There's no appointed heir, there's no prince to take over after me. We actually work really hard at making sure that everything is documented. expressed and planned for. And that's not only to just make sure that things work without me. It's just to make sure things are documented and properly structured and everything is known and in the clear. Everyone can figure out exactly how curl the project works, how we do releases, how we do blah blah. Everything in curl is completely documented. So if I go away tomorrow, everything should be known and established and we already know exactly how everything. anyone in theory than anyone can take over the project starting whenever. And really we are a whole bunch of maintainers in the project that have been around the project for many years. So there are certainly a bunch of people that are very knowledgeable about curl in general, the way we do things, protocols and the test suites and all of that. So I'm not worried. And I don't have any plans to ever, you know, just drop everything accidentally one day and just vanish. So I imagine that over time, it'll just be more other people getting involved and maybe over time, less me. then at some point just less, less me. And then at some point it doesn't really matter if I'm there or not. I understand, yeah. Now, it's one thing to develop this software and it on your machine and you can test it, but sometimes you run into these kind of weird issues that only affect a specific CPU architecture or OS platform. And while they are probably like... these days cloud versions where you can have your CI running on all these different architecture and platforms. It's a pretty slow feedback loop as well. So I wonder how do you tackle those kinds of issues if they occur like maybe very rare issues where it only happens sometimes on this specific client or whatever. Yeah, we have all of that. mean, first we have those hundred something operating systems. We have 28 different CPU architectures that you can run curl on. And then in addition to that, you can build curl on and off with, can toggle features on and off and you can select different backends to do TLS or SSH and IDN and stuff like that. So there are literally billions of build combinations. It's quite impossible to test everything and it's actually impossible for us to even reproduce a lot of problematic cases that people report. So we just have to do whatever is sensible as an engineering thing, right? So write as many tests as we can, run tests on a wide number of different platforms to try to cover it, try to build and test all the sort of the popular common build combinations, the features sets that we can. ⁓ But it still leaves out a lot of combinations and architectures and everything that we never test on when we can't do that. And then we just have to make sure that we write the code as sensible as possible to make sure that that possibly works. And then we rely on users, of course, to test that and report whatever doesn't work or even fix what doesn't work because we don't have access to those. fringe operating systems maybe. So there's really no secret to that either. It's just a lot of best practices. Just keep the engineering, write good code, clear, document it and try to not touch code that you can't verify. ⁓ But it's hard. It's course really hard. And since then we make a library that is used as a component within a lot of other things. It often ends up like where user runs. ⁓ A user runs curl in some kind of product. Hey, we did 200,000 transfers and then suddenly we got an error. Why? And you know, of course it's impossible for us to tell that, right? Because it could be any one of a bazillion different reasons. I mean, users still do that and they still have the problem and they will still report it. So we then, yeah, then we have to start the discussion, right? What can we remove from this equation? How can we reproduce it without involving maybe 200k transfers first? Maybe we can do 20. Does it still reproduce? Does it reproduce against all servers? Does it reproduce if you build it on a different platform? Does it reproduce if you change blah, blah? And you know, just the regular old engineering debugging things. Okay, but if I understand correctly, when this happens, you do also rely on them to get as close as possible to where the problem lies and maybe even them fixing it because you yourself might not have access to the exact conditions to be able to reproduce it. Right, and when it comes to them reporting bugs into an open source project, I actually view it more like me helping them to fix the problem rather than me fixing the problem for them. Because in many cases, it's really not possible for me to debug it even because sure, I can help them sort of guide the debugging and offer advice on how to what maybe look at next and so on. But in many cases, I really can't. Well, in many cases I can't debug it. In some cases, of course, go, that's something wrong, right? It shouldn't do that. And then I can take a look at the code and maybe see, well, maybe in this case, maybe we misbehave or something. So it depends on the cases. when it comes to submitting a ⁓ bug to an open source project, it really has to be the submitter's job to drive it forward. And I offer advice and insights and... Otherwise, it doesn't really work. It's a different thing if someone pays me to help them with the debugging, right? Then I go further an extra mile to actually, because then someone actually relies on me to and pays me to fix the issue for them. even paying customers, the challenge can be the same, right? Because they run it in some particular platform in some surrounding that might be really hard for me to reproduce, even if they are paying. Okay. ⁓ Yeah, correct. And so I wonder if you look back at those 30 years, do you feel like it was all worth it? Like would you have done it all over again? Yes, I think so, because I think in many ways I'm living the dream here. I'm working with my hobby full time and I get to do it all days every day as much as I want. I think I'm in a very luxury position. So yes, I think I would do it all again. mean, sure, there might be some details around the road that could have been done differently, but I think those are just details. think we've managed. pretty good in the great scheme of things. And how did your life partner look at all this through these years? I mean, I'm sure she supported you with how does she view the project? Yeah, that's a good question. I think she just views it like that. of that's my lifelong mission to work on. And I never get bored and I seem to never stop talking about it either. And, you know, well, years ago, she occasionally asked me if it was ever done, but she stopped doing that. And I think she's just realized that this is what I'm going to do. And for her, think it's more of a, you know, we all work on something, right? We all have jobs and now it's my job too. So it's like, maybe it's easier to view. That's what I do now. I work on this. This is my job. Even if it's, I view it as something slightly more than just a job, right? Because even if I wouldn't get paid, I could still probably couldn't drop it because this is now my life and my life is this kind of So yeah, I'm enjoying it still. And yeah, after soon 30 years, it's still fun. Very good, I hope that keeps like that. Maybe it could be like really a lifelong thing if that's what you want. We've talked a long time already and so I don't want to take too much of your time anymore, but I wonder if there's anything more you would like to discuss or plug into our podcast. I think we've covered a lot of things, so... Not that I can think of, no. Okay, in that case, I thank you very much for your time. I will also include your information and links in the show notes so people can find you. Well, I imagine they will anyway find you. If it's not hard to find. But yeah, yeah, it would be a bit funny for someone doing networking that you will be hard to find. But anyway, I do thank you and I wish you all the best and I hope you can keep finding enjoyment. in the beautiful craft work that you are doing. Elizabeth (Plabayo)
1:00:10 | 🔗
Netstack.fm is brought to you by Plabayo building secure, open, and resilient infrastructure with Rust protocols, and purpose. This show is also made possible by Rama, the open source networking framework. Plabayo offers service contracts and welcome sponsorships to keep building and supporting its ecosystem. The theme music of this podcast was composed by DJ Mailbox. If you enjoyed this episode, don't forget to subscribe on your favorite podcast platform and leave a five-star review. It really helps others discover the show. Thanks for tuning in. We'll see you next time for the next handshake.