On this page On this page
You can find a remastered version of this episode as episode 32 (https://netstack.fm/#episode-32 ) for an improved listening experience with better audio quality. Released on the 24th of March 2026. Please listen to that episode instead of this one.
In this episode, Glen interviews Delaney, the creator of DataStar, a lightweight framework designed for building real-time collaborative web applications. Delaney shares his journey from being a 3D artist to a developer, emphasizing the importance of hypermedia and real-time visualization. The conversation delves into the efficiency of DataStar, its use of Server-Sent Events (SSE), and the framework's potential for collaborative web experiences. Delaney also discusses the challenges of WebSockets and introduces future projects like DarkStar, aimed at enhancing networking capabilities. Overall, the episode highlights the transformative potential of DataStar in modern web development. In this conversation, Delaney discusses the intricacies of DataStar, a real-time system for handling large volumes of messages. He emphasizes the importance of simplicity in programming, the significance of measuring performance, and the role of abstraction in software development. Delaney also explains the core functions of DataStar, including patch elements and signals, and how they facilitate real-time interactivity. The discussion touches on offline support, the growth of the DataStar community, and the non-profit model that supports its development. Delaney encourages developers to engage with the community and emphasizes the importance of building solutions to real problems.
Learn more about Datastar and Hypermedia:
If you like this podcast you might also like our modular network framework in Rust: https://ramaproxy.org
00:00 00:42 02:39 06:27 15:33 23:01 29:42 33:55 46:04 50:19 53:27 57:25 59:27 01:02:56 01:07:09 01:09:26 01:13:22 01:16:33 01:18:35 01:31:02
Music for this episode was composed by Dj Mailbox. Listen to his music at https://on.soundcloud.com/4MRyPSNj8FZoVGpytj .
Elizabeth (Plabayo)
0:15 | 🔗
This is netstack.fm, your weekly podcast about networking, Rust, and everything in between. You are listening to episode four, recorded on the 1st of September, 2025, where Glen has a conversation with Delaney, creator of DataStar, a lightweight framework for building everything from simple sites to real-time collaborative web apps. I'm here with Delaney. He is the creator of Datastar And we're to talk a bit about hypermedia, about his own background, about how Datastar was created and some of his upcoming projects. So welcome Delaney. Hey, how you doing, guys? Yeah, nice to finally hear you in our podcasts. So it is a bit of a tradition by now where I ask as my first question a bit, what is your background? Like how did you arrive at this point? But still before Datastar let's say. Sure, so from a technical standpoint, I actually started as a 3D artist. So I was actually doing 3D art and doing rigging and all that stuff. And I kept asking developers, hey, I want this kind of tool or like the quaternion stuff doesn't work exactly like how I want. How do I do this? And long story short, got into coding because I couldn't get the answers back I wanted. So I kind of went from a 3D artist to deep in the weeds of like hard math and kind of came out going, hey, engineering, I actually. it's just solving a different puzzle. And so I got into doing more development and then started doing visualizations. And I kind of, the thing that I'm probably best known for is I do a lot of real time visualization and system design stuff. So I like things that you can visualize in real time. And it kind of is what led to DataStar. Cause I love the web as a platform, but the way you interact with it isn't the best, especially if you're back in focused. If you care about the network and all that stuff. What's considered the industry standard right now is complete garbage in my opinion. So I went to try to be the change I wanted to see and make something that's better, that's built for real professionals doing real interesting things. So that's how Distar got started. Okay, very interesting. So Datastar, we're going to talk about it in a bit, but like how was your introduction to Hypermedia? Yeah, so I've been doing web stuff off and on since before CSS was a thing, right? So I was part of the original .com, like Gold Rush time. That's kind when I started doing a lot of development stuff. And I always thought the web is an interesting platform in that it's a complete, almost its own OS, it's its own sandbox that you can deliver things in. And that's really interesting because like if you've done game development, like you create a binary, you do patches, it's kind of a hard thing to do where it's with hypermedia. every time you go to a site, it can be if you get a fresh version of that, which means that you don't have versioning and you have the version controls, and if you keep most of your state in the backend where it belongs, then what happens is the UI, you're not tied to an API like with protobuf and all these other kinds of things, you give a piece of content, and then in that content is also the links of what you get to do next. So it's like the original hypercard, I don't know, I'm dating myself talking about hypercard, but. Hypercard was like literally the first click your own adventure thing on the Mac like in the Apple II days and with that's really what it is. It's a hyper immediate thing where your API and your UI is really what you're allowed to do next. So for example if you're on a bank account website you don't get to see what your deposits are until you've already logged in. Well once you've logged in now you can see these things. Well you can't deposit money unless you do this like what you're allowed to do next is determined by what the backend state is at that moment in time. And I think that's really, really powerful because it allows you to really control things in a, you don't have to trust the end user, because what's done nowadays most of the time is a spa, which is a single page application like React or Vue or Svelte. All of them are relying on sending the entire application's state down to the front end and then to some degree having to do validation and all this stuff on their routes and all that. and then you send it back, well you still can't trust it because it could have come from a bad actor. So you still have to end up validating everything on the back end anyway. So now you're validating in two places and that just becomes a nightmare. So my thing with hypermedia is that it allows you to do things and only have to do it once. And you still get, you can get really fast and interactive applications and I feel like that was lost for a while. There's a project called HTMX where Kirsten Gross was basically saying, hey, we can do hypermedia with just a few extra tweaks. And I thought that was a really smart thing. The only thing is it didn't go far enough. And I wanted to do stuff where I could do real-time things in microseconds, not in upholding every half a minute or half a second, even minutes every second. So there's things that were done that aren't very efficient, and I wanted to bring the efficiency from game dev and real-time systems to hypermedia. At that point you are talking about HTMX, right? Yeah, so I started using HTMX and it just wasn't good enough for what I wanted to do. And that led me to the thing of basically what their whole idea was is that, hey, if you just do a little polling on here and grab datastar, kind of like most React apps also do this too. If you want datastar from the backend and you want a real time dashboard, you're polling basically every second to every five seconds, right? And the problem is that's super inefficient because not only are you polling, but you're pulling JSON datastar, then converting it using JavaScript into HTML. Well, it's a lot faster to just send down HTML directly. And also, if you do something like SSE, which DataStars kind of got to be known for, you're cutting that real time, that RTT in half, because you're only sending down datastar when the server chooses. So it decides when to send the datastar. So now you've cut your RTT in half, because you're not constantly asking for it. And you can also have a lot more control over back pressure and all these other ideas. So you have a lot more control over the system by it being back end focused. Yes, and SSE standing for server side events, which is like a way to stream your body of an HTTP response. Now, so that means that like we're going to talk about all that in a second, but like for me personally, like I was aware quite deep into like HTTP and I was building clients, proxy service, et cetera. What I wasn't aware of was service and events like datastar was for me the first time. I even heard of that and then since I discovered that, I also discovered like plenty of what is now AI chatbots make use of it to stream their responses, which makes sense. But like how did you got to know SSE or is it just like a consequence because you're so long in the weeds already? That's a good question. So I actually had built quite a few real time systems for military applications doing like real time tracking of stuff. I guess that's what I can say. And the thing that I found is web sockets are just a nightmare in practice to actually keep open. Also things like compression, like security, all these things are really hard to do. And I had completely, I looked at SSE like 10 years ago. and I didn't get it because it was just a text-based interface. I didn't really see the value in it at the time because I was doing like little level binary stuff. But that was myopic in my point of view because once you have an SSE, SSE is just a response type. People think that it's some kind of thing like WebSocket or something. It is literally just a normal fetch. But instead of sending back text slash HTML, you're sending back text slash event stream. And then you're saying when there's more than X amount of new lines, send down and flush the response. But it's still only a single response. So it's only one request and one response using normal fetch. It's not some extra special thing. It's just a normal response. So that means it can work with PHP or anything that just takes a normal HTTP response. It's just a matter of you can keep it open longer. The one thing that's really kind of an eye-opener is that because it's an open connection, you can actually use things like Brotli or Z-standard compression within a window. So now, because most of the time your HTML is very consistent, it actually compresses amazingly well. In fact, we found through testing that we can have sit down at 2,500 divs every 200 milliseconds, and with minimal changes, you're looking at like 10 to 20 bytes going across the wire of actual datastar. So the irony is that you can get between 30x and 6,000x. So you're talking about a like... like 60,000 % difference in size because instead of sending down individual binary things, you're keeping that window open. And that's the kind of thing you just can't do with web sockets. It's just not built with that in mind. You'd have to build an entire platform to do that. the nice part is we're just relying on what's already built into the browser. So you can get huge benefits of network transmission and actually ironically, CPU usage by sending down the whole thing every frame because it's actually will do the... compression windows stuff for you automatically. So you have a simpler interface. You mean you're talking about accept encoding here, right? okay, so we have to dive into that a bit. So for example, you were talking about WebSockets and there you have WebSocket extensions where you can say per message deflates and then they do the compression per message. And the reason for that is because they say, okay, you don't want to get stuck. where you're waiting for a buffer of your compression algorithm to prevent it from streaming. Otherwise you might like send a message on your WebSocket connection, but it might not be sending it yet because it's waiting to be compressed. And that's why they're recommending the per message deflate. But you are saying you're making use in some of your like datastar SSE responses, just the regular accept encoding. the regular compression as supported by your server and client, which like the client usually is the browser in this case, and it just works out, you say Yeah, and the thing that's really valuable about that is that you're relying on the browser to do what it's trying to do. A lot of times you'll see a lot of web frameworks are huge, right? Even just to get React to render something on screen is like 30K gzipped and all that stuff. The entire Data Star platform with everything possible in it is 12K and for the pro version I think it's 14 kilobytes for everything possible. So, and the reason why it's so small is because we rely on the browser to do its job. Right? So things like these encoding things are already built into the browser. Why would we add more things on top of it? It's already built there. Like nothing that we do in JavaScript would actually be any faster than what's done in the C++ or Rust level beneath it. Right? So we just rely on the browser to do its job. And by keeping that window open on a single request, because technically the SC stuff is just a single request with the multiple responses are just chunks of that same response that's coming down. So. We're relying on the browser to do its job and keep the window open and it does amazingly well. So we just allow the browser to do what it's good at. That's pretty much what we do is we get out of the way as much as possible. you never noticed that it was waiting for a buffer of your So the nice part is that when you're doing the whole point of the, like, this is not an SSE thing, because basically all HTTP is now done via streaming. Like if you look at HTTP streaming, that's kind of how all responses are sent back. But by you sending that flush buffer, the flusher event thing, and I can go, I know it's flusher, and a couple other ones, it's named slightly differently. But you're basically telling the NIC card that, if you can send this buffer, do so. Just like when you're doing a console log, right? Like you can console log, until you call flush, you're not guaranteed that it's gonna flush during that time. But if you call flush to your standard IO, it's gonna do its best to try to flush the buffer. It's the same thing, but instead of going to standard out, it's going to your NIC card or to your network. So you can set it up, depending on how you set up your Windows, you can set the initial buffer to be like 10 kilobytes or something, right, or even smaller. So it just depends on how you set up your buffer. Again, this is how you set up your compression thing. Compression middleware has nothing to do with the individual messages you're sending, if that makes sense. Okay, okay. So it's just a matter of like configuring your compression middleware on the server properly. Yeah, 100%. So it's one of those things where if you had like, since one tiny message that was a few bytes, you may not see it unless you set your windows correctly. But if you're sending a real-time streaming dashboard, this is a non-issue. So it just kind of depends on how much datastar you're actually trying to send over the wire. But I guess you might see with trailer datastar, like maybe your window is big enough, but then the final, maybe messages might get delayed, no? Well, and that's the question, what's final? As long as the page is open. So there is no final message in the way I build things. So as long as you keep the page open, there is no final. So what I tend to do is I do a very seek your rest style, which is command query responsibility segregation, which means anything that you interact with on the page is sending up a command to an endpoint saying, you know, I want to click on this button, I want to change this value, I want to change my name, that kind of thing. And then I have one per page, basically a slash update. that is just streaming down datastar, which could get stuff from a database, it could get stuff from another user. It all comes down to that update page, and I do these big, fat morphs on the page because the compression's so good, I'll just send down a whole new version of that page. And now all of sudden, I don't care where the update came from in the backend. It doesn't matter who did it, I could have done it, someone else could have done it, it could have been any kind of, but I get one update that completely changes my page, and Data Star handles all the stuff of making that efficient for you. Your idea is ironically a lot like what the original React library back 10 years ago was, was just, you just basically render out what you want on the page and let it figure out all the VDOM stuff and all that. Well we can do that without the VDOM and from any back end in any language, you can basically say, here's what I want you to render, let Datastar and God figure out how to get that to work. You just send a big fat update, send it down and we take care of it from there. And it's very, very, very efficient. And the big fat update is the entire page usually or is like snippets. Yeah, yeah, you can send an entire page and we will make sure that all your focus is staying intact and everything is the same. So, you can't do this with React, for example. Like, let's say you had an input box and you did a big update. It has to do all this VDOM work to try to keep that focus in place, right? We do a offshoot of ideomorph and all these other things where nothing in your page changes unless it has to, kind of thing, and we update the things that are in place, but we also can create new DOM objects and all this. The details don't matter, but the point is, you can send a whole new fat update, and as long as IDs exist on the things that can have changes to them, we'll just take care of it for you. All your focus stays the same, all your scroll stays the same, all that stuff's handled automatically, and you can just, from your back end, send out what you want the UI to look like, and we take care of it from there. Very interesting. And then at what point do you, for example, update the history of your browser? that, because what I really dislike in spa applications, for example, is where you're doing a lot of complicated actions could be like, let's say you're trying to book a holiday with your partner and you're configuring a whole lot of stuff. And then you want to send what you configure to your partner, but you simply cannot because it's like some kind of very opaque URL with no state in it. How do you typically handle it or how do you recommend that users of Datastar handle that? So when it comes to something where you're sharing state, either it's a URL that's a guarantee, like you're basically saying, we created a new booking UUID, right? Like this is our booking, if you copy paste that booking to someone else, you can see live as things are being updated or changed or affected by that. So like. Well, could be that or also just could be like in a public search where you're looking for a hotel and you have all your filters applied and those filters are your credit parameters. Because you were saying that as soon as you... because I'm just building a bit on what you were telling, where you were saying, okay, every time I click on a button or do some kind of interaction, I just send this on my open persistent connection back to the server. The server streams back the new state in one fat blob. But then I wonder at what point do you update the URL or do you it also just each time? So I believe that people mess around with the URL way too much because it messes around with history, it messes around with the state of the user, it causes so many issues. So if I was gonna do something where it's a collaborative thing where I want people to be able to share that, I would basically have a search, like you have a search session, right? You're searching and you want to share that. So that would be a link to another search slash and then a UUID. And then if you copy paste that to anybody else, whatever the current state of that search is, both of you will able to look at it live. and be able to interact with it at the same time. So, I would make, to me, that's a new back end state. That is information that you don't want to lose. Because you want to see, how many times did someone create something? How many times did they copy it? Which browsers were you? Like, all that information that you don't want to throw on the ground if you're running a business, right? So, like, I would make that a new entity that then has a bunch of datastar built around it. So, search to me is just another object in my system. Hmm. Yeah, that's true. It's collaborative. Sometimes you just want to share like maybe or let's say it's like a website where you can watch all kind of food like you're looking within your local region for the best pasta or whatever. You might want to like send on a Facebook page or some kind of community group page. Here is a link where you see all the best pastas in my city. At that point, you don't want to it collaborative, right? You do want this kind of like filter thing. Does that mean that you, yeah? Well, the thing is, to me that is collaborative, you're just changing who's allowed to change what. So for example, you can have a collaborative Google Doc where people are allowed to view or they're allowed to edit or like that, but it's still collaborative because as I'm making changes, you're actively able to see what I'm doing. And my thing, especially with Data Star, is that I think the future Web 4.0 is gonna be all about collaboration first. So the idea that you should be able to share any part of any page with anyone and then you decide how much to lock that down. that's totally your prerogative. But the idea is that every page should be basically the equivalent of a real time system. So that if you're sharing my favorite pastas and guess what, that's the equivalent of a playlist in a Netflix thing or a watch list or something, right? Like you're saying these are the top places. As you're editing that or as you're filling that out, you wanna be able to have someone look at that page and be able to see, I'm doing a live stream of this, what I'm working on. my point of view is there should be no difference between the live stream other than you tweeting out, hey, I'm going to be editing this at this time. But anyone should be able to look at anything that anyone else is doing if they chose to and be able to interact with them directly. So I think that everything should be the equivalent of a Google Doc everywhere. So every Excel spreadsheet, every page that your favorite post does, every blog, if you want to watch someone type away, figure out their blog and they want to make that part of the experience, you should be able to see that live. So I think everyone being in a shared state with their datastar is where we're going to be going. That's going to become table stakes at some point in the future. Really? Because I think it's kind of, like, the thing is that the original web, like, at least when in the late 90s, was very much about you got to have at least a static page that was someone's intent. Like, hey, I'm interested in this stuff, here's links to what everybody's doing. Well, all I'm doing is saying, hey, I want that same idea, but I want it to be the second that I know it, everyone else knows about it. So for a perfect example of this is there's a game of life and there's also a checkbox thing that Anders had done, Anders is in the community, and it's a billion checkboxes. And it's not only a billion, it's actually not billion checkboxes, it's a billion-celled spreadsheet. And it's a shared spreadsheet for the entire world. And everyone can edit on any cell and add any comment they want. And that's running on a $5 server. Like, that is a real-time collaborative thing with a billion entries of whatever you want for datastar and it's running on a $5 server. That's the world I want to see for the future because once you have the ability to do that for dirt cheap and it's fun and easy to make, I think the entire thing he wrote in that was like 500 lines of closure script, right, or closure. So once you have this ability, I think that it opens up a lot of the opportunities for things that you can do collaborative. Like for example, one of my pet projects I want to get to at some point is I'd love to build a real tabletop simulator that was built purely in the web, purely with the standards that are there instead of tabletop simulator. There's a lot of stuff that kind of does it halfway, but a lot of the collaboration stuff is not, I don't think, good enough. Now all of a sudden, if everyone's on the same page, if you can do a billion, you can do one game session, right? So my thing is that once you can have these technical abilities, it's all about spending the time on the individual game and the mechanics and all that kind of stuff, because now... are guaranteed that you can pull it off. So I think it'll open up the door to a lot more interesting stuff. It is funny that in the Datastar world, a hello world is like a fully reactive live chat with people. Like that's the first thing almost everyone makes. It's just like, want to make, like I basically want to write my own little Discord is how people start with Datastar. Which is not the way that you'd think of a normal web application. Like you'd make a static little hello world thing, right? But. That's so easy in Data Star that you're like, okay, well now I can do real-time chat and real-time things, why would I not just do that? Because once you have that ability, you start seeing value in it right away. Yeah. Yeah. I I already got away with so much new knowledge. Like I honestly didn't consider it like this, but it makes a lot of sense. Now I do want to go like recap a bit because this is the first podcast episode where we work so high on the like ISO layer, the application layer. I mean, because it's not just HTTP, like it's something very specific within HTTP. So let's say someone comes to you and says, Hey, Delaney I am I have experience with writing an HTTP server like I can add routes I have my favorite backends like what is datastar and why would I use it like what would you tell them like how would you introduce datastore to them Sure, so let's say, like, because this is, you know, like, has a lot of Rust background, like, let's say I like writing Rust and I like using Tokyo and I know that whole world and I just want to be able to hook into what I already know. The nice part is, is that DataStar works directly out of the box with HTTP 1.1, 2, and 3 out of the box. So, you don't have to, like, for example, like, WebSockets do not work in Web 2.0. Latest don't. Or HTTP 2.0. and that's a problem. Well, some browsers do support it, but most not. No, the HTTP2 itself does not support web sockets. It doesn't support it. Like they have to actually downgrade to 1.1 and then do an up web socket upgrade and stuff. Like it's not part of HTTP2. No, no, you can do HTTP to WebSockets like it does exist. I would love to see that because that doesn't, it's not part of the spec at all. It's literally not part of the spec, so I don't know how you're pulling that off then. I'm pretty certain I have an example on my network framework where I do it. You are correct though, like it's not fully standard compliant, but it seems to work. I want to test it again. I'm going to it against a wireshark and I will let you know after this podcast, but I'm pretty certain it works. Yeah, want to, yeah, WebSocket's into, WebSocket's in HTTP2, now what you can do is you can actually be HTTP2 connection and you downgrade it to 1.1 and then you have to do the WebSocket upgrade and stuff, which I bet you is what's happening. Yeah, I do understand, but no, because it like this example is just running an HTTP 2 server, so it could not possible downgrade. OK. Yeah, I will bet $1 on that one. How about that? We'll bet $1 on that. It's gonna be very painful to wire a dollar to Europe. Yeah, it's going to cost like $20 to do that. yeah, I remember, but I, that'd be interesting to find out because the, yeah, from everything I've done in deployed HTTP two and WebSockets don't work together. So I would say no matter what, like it just uses regular web stuff that you've learned. The other thing that's of value with datastar start is that it is tiny. It's like one third the size of most frameworks and it actually does more out of the box. So there's that and What's also nice is we actually have a ADR, a set of specs that means that if you use one of our SDKs, you now can talk to people that use PHP or Go or TypeScript or any of these other things, and the DataStar part, it's gonna be the same for all those different languages. So everyone's on the same page, anyone can help you out with the specifics that are DataStar related. And the other thing that's nice is that everything about DataStar is completely HTML spec compliant. It's not like we went off and kind of made our own template language and our own set of tools. It's all using standard HTTP. In fact, we're one of the only ones that actually uses the specs completely. So your spec compliant, it's tiny, and you really only, if you're doing stuff like Rust, you'd only have to learn three functions to be able to actually build almost any application in the world, which is pretty crazy. So it's fast and efficient, and if you care about this stuff, it's weirdly efficient in that I'm very much a protobuf or a Captain Proto or like a lower level guy. And when I think about network stuff. But the irony is that big fat updates over HTTP is so much more efficient. And it's such a weird place to be to think that that's actually the efficient way to go. So if you're new to this stuff, I highly recommend joining the Discord and joining like the Rust channel and start asking questions. until you actually test it yourself, you will not believe us with how fast it is. Yeah, and even on your website, right? Like the first thing people see is this banner where you have real-time updates. Yeah. So, and even in that, so the thing that's interesting about the DataStar approach is that we don't try to re-implement the web. There's things like, for example, web components that are already built into the browser that's a way to create new and extend new elements on the page. And that banner that you see, the real-time banner that's doing all the crazy stuff, that's just a web component. It's like one kilobyte of a web component and it's hooking into DataStar and all that stuff. So the key I think with DataStar is that we're not trying to replace what the browser's good at, we're just extending it to make it easier to make it declarative. So the irony with a lot of the parts of the browser nowadays, there's really powerful parts, but you have to do everything in a JavaScript interface. And all datastar storage is doing is making that a more declarative interface. That's it. Like that's really what it does. Yeah. And so at what points do you still grab to JavaScript, for example? I don't ever, like if I need to write some local thing, maybe for like a canvas or for a 3D engine or something like that, I would wrap it and build it as a web component, but when it comes to just JavaScript on the page, zero. For almost every website that you need ever, you don't need any JavaScript on the page anymore if you use DataStar. So it simplifies your stuff. If you like being in your backend language, you're gonna love it because even if, the irony is we have a lot of people that use TypeScript, right? But they're using Bun or... Dino or some of these other things and they're thinking about it at a deeper level and this completely makes it so that the front end is its own process. They don't have to worry about it at all because it's just templates in HTML. So having a declarative front end does make things a lot faster to write, even if you're in tech. Okay, and so typically when people, for example, add some features such as chat support to the website, they do use WebSockets. You would also use DataStar for that, I suppose. Yeah, I 100 % think you should not be using WebSockets at all. The problem is that you get into issues with sticky sessions, you get into issues of ReConnects, like all these things that become a big deal in production just go away doing it the day to start way. They just don't exist. Like HTTP ReConnects are built in. We already have all the compression stuff built in. We actually have exponential back off and jitter and all the things that you would need in a production system that you'd have to write yourself if you're doing WebSockets and most people don't do it correctly. That's all this built in in native. there's just a lot of things that are easy, like getting this up and running with a web socket, you know, it's 10, 15 lines, but actually making a production level system takes so much work and so much effort. And I just would, I would recommend no one do it for anything ever unless they actually have to. Because the thing is, my thing is when you get to the level where that matters, then you're better off making a native app. Like there's nothing wrong with native apps. There's nothing wrong with sending. like direct protobufs or even going lower with the SIRDE stuff and getting to a low level. Bebop is one that's available in Rust and it's a great format. But my thing is that once you're at that level, you want to run a native app. Like you don't want to run a web app. So my thing is if you're gonna be on a browser, embrace the web and do things the web way and it's really, really fast if you do that. If you're gonna pay that cost already of being hypermedia. Okay, so I'm almost afraid to ask, but then what do you think about web transport? I love Web Transport. See, the irony is that Web Transport is trying to fix all the things wrong with WebRTC and WebSockets, and it's doing it over UDP, reliable UDP, without the head of line blocking. So I actually have a side project that's been sitting kinda in the wings until Safari gets off their butt and actually supports it. But I want to do some stuff with Web Transport called DarkStar, because we already have DataStar. And the DarkStar stuff, the interesting thing about that is it's for everything that DataStar is not good at from a network. point of view. what I want to do is actually have a small interface that allows you to write, if you write deterministic code that basically says, you know, on this frame this happens no matter what, like with these inputs, if I have a pure function, here's what the results are. If I can make it so that people have that interface, then you can actually do net rollback or net code with rollback, which means that you could do fighting games, first person shooters, all that kind of stuff. that the web is not good at and you could actually do it directly over web transport. So the neat part is what happens with, like for example, when you've ever seen fighting games in last 10, 15 years, you play fighting games over the internet, which seems crazy, right? Because you're battling directly there. How do you make that work? So there's this idea of net rollback code, which means that you simulate the game locally using the last known inputs for the person and you make predictions on what their next input's gonna be. So let's say they hit the punch button. Well, the most likely, their next couple of frames, because that's all you're caring about, it's like 10 frames into the future, is most likely gonna still be the punch button, right? So you predict that they hit the punch button, but all of sudden they hit the back button. So what happens is it comes in, and then you have to basically roll back to that original state, reapply it, and then now get to the state where you're both in sync now. So if you build a deterministic pure function basically for your local state, then I can do all this re-predicting. sending things back and doing things correctly over web transport. And then you can do truly like first person shooters, real time games, fighting games, all that kind of stuff. And it also means that you could do local stuff and then it'll feel like you're directly connected with that person even though it's directly simulating on your machine. So that's something that I have planned for the future but that's a separate thing from DataStar. And is that something people can help you with? Let's say they want the same or is that something you want to first get to a certain point before you publicize it? No, I would love some help with it. The thing is it's more about the deciding what that API is gonna look like. And a lot of the ideas are coming from projects like GGPO, is a good game piece out, which is a net rollback code thing that's been around for anyone in this space kinda knows what I'm talking about. Yeah, anyone that's interested in Dark Star and basically pushing things forward, come talk to me 100%. The thing that's interesting is that I see it, like I'm talking about games because the thing is, Data Star handles, for what most people need to do with a website, it handles 99 % of it today. But it doesn't handle that last bit of like, hey, I want to have a real time, like, predictive, because most of time, you don't want your bank account to be predictive. Like, right, you don't want there to be like a, hey, I think that I'm gonna deposit $10,000 today. Like, you don't want that to be them telling you it's there, but it's not there, kind of thing. Whereas with a game, you don't care if you punched and like went back and changed the results and then. actually punch the gate, right? So most of the web is built where you want it be very deterministic, very server authoritative thing, but there are some things that are okay to be local kind of things. So I think 99 % of people should be doing things the DataStar way anyway, so DarkStar is very much of a, the only places where can see this being valuable is where the datastar doesn't really matter, and that's something like a fighting game or like a first person shooter, that kind of thing. I just want to make people clear is that I still think you should build your actual website and things that matter in DataStars. Okay, so I first want to try to summarize in my own words what Datastar is and then you can correct me and then using that knowledge I want to go back to Darkstar if that's all right for you. Okay, so how I got to know Datastar and based on our current conversation, Datastar is basically, well first of all, it's a specification and it's also a a library, a framework that you download, the JavaScript blob. And this, what it does is it wraps around the SSE client, which is shipped with your web API for your browser, which does a lot of nice things to make a production ready, like get a reconnect, whatever. But I think it piggybacks for that on the SSE clients. And then what it also does, it has a lot of very good code to ensure that when you send HTML, that it will div. and that will ensure to only update what needs to be updated. And it's doing that all by building it's, well, the way it works is specification around what HTTP is. So basically it's, which also is what we already said, because it's built around the SSC client. I mean, did I get it correctly so far? Mostly, got it correct. I wish it was that case. The thing is, we don't wrap around the standard. There's a thing built into the browser called event source. So you can say new event source and you get the, the problem is that doesn't support things like post and patch and put and update and all these other things. So it doesn't support all the verbs and it also doesn't have things like exponential back off or like it's reconnect stuff as kind of, if you tab out, you can't tab back kind of thing. Like if you tab away from that. it automatically disconnects, but we want to make that an optional thing. So if you want to save battery, it disconnects, and then you click back on the tab, then it reconnects. But you also want to possibly keep it on even when you disconnect. So there's all these things that the built-in client doesn't do for you. So part of that 10 kilobytes is writing a whole new fresh version of it that's built on top of Fetch directly, and we do that as well in that space. So I wish that the built-in one worked well enough, but it does not. if I can get any of the browser vendors to just take on what DataStar has done and just build it into the actual event source client, I would be a happy camper. Because I wish we didn't have to do that. Yeah, maybe you can start with the Ladybird guys. They seem to be open to do it better if possible. So maybe they are. And I imagine they might not even have support yet for SSE. I'm not sure. But they might be open to it. Yeah, if they want to get me in contact with the right people, I'm happy to yell into the clouds of like, why aren't you guys doing this? We're doing it as a JavaScript library. This should be built into the C++ side. And do you notice not any, because event source, I imagine it boils down to C++. Does it, or is it not true? So do you notice some like performance loss there or something? No, the thing is that we're already relying mostly on the fetch that is already built into the C++ side, so it's not that bad. I just think that there's a lot there that could be a lot better. The interface that we give people for the SSE stuff, it's better than the actual SSE interface that's been agreed to by all the web standards people. So I would just like to see more options there that are correct. Like the actual back office. And do people ever play? Yeah, do people ever touch the client that you provide or is that necessary? Generally not, like the thing, that's the nice part that is kind of a hidden superpower of DataStar is because everything is declarative. When you're writing code against it, you're just putting in some attributes and templates. Like when you want to click on something you're not setting up an event and disconnecting the event into one last, you're just doing datastar on click, datastar on scroll, datastar on mouse move, right? So all that stuff is handled for you in the background. It's kind of like when you do SQL, for example. if you're doing Postgres or SQLite or DuckDB or any of these things, you say, hey, I wanna create an index. You don't have to know if it's a B-tree or an LSM or all these things, like how it actually is gonna do it, which pages it has to load, how many cover pages it needs to allocate for this thing. You don't have to know any of that. You just say, create index on this and this, like on A and B columns. So the declarative nature means that... person that's actually writing the implementation can do all kinds of crazy, interesting things and you don't have to know or care because you're just like, oh, I just created an index. So, DataStar kind of leans into that so that you don't have to be a HTTP master or have to know all these little intricacies about things to be useful in DataStar. In fact, most of the time, if you go through the guide on DataStar and you get done and you start using it in your whatever language you want, you're gonna have a moment where you go, oh, that's it? Like you have an up and working HTTP solution and it'll take you like an hour to be fluent in DataStar. So that's the kind of the irony is that we talk a lot about it but just read the guide and go through it and you'll be done in an hour and be a DataStar master. I'll give you a certificate and everything so. How do I get my certificate? I need, we're actually, it's a little bit of a sneak peek, but the shirt, we've got some shirts for some conferences, and we are gonna make a little store. So we'll make sure to make a sticker that you can buy that is like a certified DataStar spacecraft mechanic or something like that, something like that. So you'll be able to get your own certified DataStar mechanic for the low cost of like a dollar or whatever we can get the stickers for. We'll figure out a way to make sure that you can get your certification and it's a super expensive process to go through, as in like a dollar or two. Look at that. Maybe if I prove you that WebSoc is over HTTP 2 is possible, can send me the sticker. I would, I will happily send you a sticker if I am wrong about that. will happily, I mean have to make the stickers first, but I will happily send you a sticker if I am wrong. And yeah, I don't know, I don't need anything from you. So like, it's a win-win for you. Look at that, I cannot lose. Now, the next thing is, and maybe it's because, so even though I dap into the web world over more than like, you know, close to two decades, like I was never a web guy, like the web is part of our culture and the web is where I also grew up in. And so it started with like the typical lamp stack in PHP and all that. And then it went to, I went through a lot of things. I tried to learn all these. frameworks and I just I always find way too complex like sometimes it's funny because people say you can write a kernel driver you can write like a driver you can reverse near a binary wow it's amazing but I think like well you're writing react.js like you are amazing because grasping all that complexity I find it like a lot more complex than anything I ever did yet when I so I so I feel like I know datastore and I can write my server and even like maintain two SDKs, it does take me a bit to go from like the simple initial examples. Okay, I can stream some little lists or I can write a chat up, sure, but then when I go through the examples, I do sometimes still feel lost. And I imagine it might just be because I'm missing some knowledge from the web world or some concepts. I don't know. Like I feel there's like a gap between like going from like my hello world to then something that I master in production. So my day job, I work at Synadia, we are the people behind NATS, which is a real time system for handling billions of messages an hour kind of thing. And so maybe it's just that I'm used to big datastar, is that the only way to handle big datastar is to use big dumb buffers, right? Like computers are really smart at big dumb buffers and arrays of things. And so the way I look at things is I'm too dumb to be smart and too smart to be dumb. So I'm somewhere in between of like, look, you probably are over-complicating it. if you've ever run our Discord channel, you probably see me pretending to people that you're probably over-complicating it. And it's not because I have some sage wisdom, it's more that things are faster when there's less code. And less code is just big arrays, like either maps or arrays. the irony is that when people talk about O, like big O notation, they forget what the N and the O, there's other parts of that. for example, like, people will talk about a lookup in a hash map is gonna be faster than iterating an array. Well, it depends. If it's less than thousand entries in the array, it's probably gonna be faster in the array, even sometimes up to a hundred thousand, which sounds crazy, but that can be the case depending on how things are structured, especially with sorted sets and all that. So my thing is always, I don't assume anything about anything, and I measure everything. And I basically say, I'm dumb about everything, because the problem is on this computer at this time, here's the numbers I'm getting and trying to get really good at looking at the signal to noise ratio of like how much of this is just because the computer's already overheated or because of some other thing that's running in the background. But I try to measure everything that I do and trust none of my own intuitions because I tend to be wrong, right? So being humble enough to know when you need to measure is I think the biggest value that you can have. And the other one for me at least is almost always, if you're dealing with actual datastar, because I've dealt with military stuff and gaming, isn't like slot gaming and accounting type stuff. And I really do think that the secure us and event sourcing are kind of the thing that makes life so much easier for me. So a lot of times what people do is they'll basically put together a update and then they'll say, oh, then I'm gonna send back the response to that, right? And I do think that the sending a command, that going into a command queue, and then you have something that's pulling that off and then creating new events. And then you can have multiple people looking at those events and creating their own projections. That seems more complicated, but it scales up all the time. the example that I use that I think is the example that kind of really did open up my eyes to the Seeker REST and the style of work that I do is, let's say you have a cherry tree in your front yard, right? That cherry tree is actually a tree, right? It is a tree with like properties. But, so when you send a command saying, hey, I want to chop down that tree, right? There's a physical action of chopping down that tree. But there's also the projection of what that meant to someone else. So did that mean something to your landscape guy? Like it's either one that's actually chopping down the tree. Does it matter to your HOA or to your local municipality? Like are you allowed to down your own tree? It matters to the... the person that's running your irrigation lines? Do we have to remove part of that water line? So for every single person that's interested in that tree, their idea of what the tree is, like the guy that's running the municipality, that's just one tree. It doesn't care if it's mahogany or any of these other things. So your idea of what that projection is, it's different between each of those people, which means each of those people probably have a different database that has the concept of your tree means something completely different to them. So the problem is I see a lot of people trying to make the tree fit the municipality and the gardener and trying to add all these things to the tree. But the tree is just the tree. doesn't matter what any of these other people think. So they have to project and create their own versions of that datastar. And so a lot of times when I see people making things complex it's because they're trying to lump it all into one thing as opposed to no there's a series of steps. There's the actual tree that I'm doing actual events to and then there's these other ideas of what the tree is. So that's the thing that kind of helped me really. And then. You may look at your municipality and they say, hey, this is going to cost you an extra $10 if you get rid of that tree. You're now also a projection looking at that datastar and then deciding what you want to try to do in the tree. So everyone's kind of projection of other people's datastar. And then you only are sending commands to things that you can control yourself. So who owns the tree and who controls the tree and who is just creating datastar from the tree? I know that I probably taking the analogy too far, like it really does. Having an idea of who owns what in your system. changes the way that you think about the overall system. Because you're not trying to combine it all into one hard piece, but you're also not trying to break it up into a million microservices. Like you're not trying to make a service per function, you're saying, oh I have a gardener, okay he has a different concept of what these things are. But so the tree, to him, is just like a work order, but his tools, his actual chainsaw, is to him a big deal. So he has to keep maintenance on it and do all these other things. So how much of... Object matters to you, it also depends on how much you have to control it, if that makes sense. Hmm, yeah, it does make sense and I agree. It's sometimes difficult to find where the abstraction layers are and or when you need to abstract, which is something you learn the more senior you get because when you just start to learn to program you think you want to abstract everything away and you want to like be super dry and and then you learn actually that also as it's cost and so you want less abstraction layer sometimes and sometimes duplication or like you say like it's one tree, but depending on the use case, it's a very different kind of way to deal with it or how to control it or to view it. here's a perfect example that I think this is something that we're starting to figure out in computer science, but computer science is not a very old profession, right? But always look to how other industries work. So if you look at legal or accounting or medical, for example, like when you go to the doctor's office, they have you have it fill out a form three times. And you're like, I want to fill this out three times. I like I can't just fill that one. Can you be more efficient? But because it's going to go to three different offices and it's the most efficient for you to do it while you're waiting for the doctor than it is for the person that's dealing with this stuff to try to find copies of this stuff and like do all this stuff. So they actually do a more event sourcing thing where they actually make copies of everything as independent things and then if they have to address each other they talk to you like that seems inefficient to you but these are industries that have been around for hundreds or thousands of years. We can learn from them instead of us trying to think that we have the magic solution that we can technically go our way out of it where a lot of times it's a people issue not a technical issue. And I see a lot of times where people try to automate something that is a personnel issue, not a technical issue. And I think that's some of the senior stuff of going, yeah, sometimes it's better to just copy the thing. And there is the other thing of people don't want to repeat themselves, but there's times when repeating yourself is actually simpler because you're doing locality behavior, keeping all of your stuff in one place and having an extra copy of it. Until you have two or three copies, it doesn't make sense to make a helper function or something. You have this and then this can adapt. This area can adapt and that might change over time. So people try to make it. Don't repeat yourself, I think is bad advice to give to people most of the time. Until you find the patterns. Once you find the patterns and once you know what you're building, yeah, then you can consolidate and do that stuff, but until you're doing that, it's just a waste of time. Hmm. Yeah, totally agree. So we already went a bit over what Datastar is. I now learned that you have your own event source like client. Now to talk a bit more about the specification or protocol or however you want to call it. So there is the concept of, what was it? Fragments, which is like your HTML stuff and also your JavaScript. So anything that you want to merge or remove or add to your DOM. And then you also have the I think there were three. Then you also have signals. I forgot what the third one was. Yeah, so there's basically there's only three, when you try to write a, learn DataStar, there's literally only three functions you have to learn. There's patch elements, which is just basically taking HTML and dumping it into the page. Then there's patch signals, which are the reactive real-time systems that allow you to do, basically hook up and do real-time connections that are efficient. And the way that works is basically for people that wanna nerd out, it's a dependency graph where basically you can say, I have A, B, and C, and B and C create D, and then I rely on A and D, and it basically works through the graph and only does updates as you need, as opposed to updating all of them all the time. So there's a dependency graph that signals build. And then the other thing you have to learn is read signals, which is on the, so those two things go down, and then there's a read signal, which basically, how do you go from the client, the browser, back up to your server? So there you go. That's all of DataStar for most backend developers is, patch elements, patch signals, and read signals. That's it, because you don't need to read back HTML. That's it. Yeah, so I think most people are very... So for most people it's going to be very easy to learn patch fragments. That's super obvious and very clear. I think what more people are maybe, and including me, or initially a bit afraid of or they don't know how it works is the signal concept. And also sometimes you read on the Datastar Discord, or maybe I misread it, is that someone might overuse signals and then there might be an advice like... like try to use as few signals as possible even some people might only have like patch fragments so when would I what would be a correct usage of signals according to you? No, it's a great question. I do think people go for these signals way too often, and the reason why is because basically any front-end development that you do in React or Svelte or Solid is all signal. It's basically all signals. It's all signals all the time. And then HTML is like the last thing you do. Whereas for us, it's like HTML is all you do, and if you're only when you're trying to do something reactive. So if you have something that is two-way bound, so like let's say an input, like literally an HTML input, or a calendar. or anything that the person can directly control. So anything that they can control and you say, want the server to know about this information and be able to send it back up. That's the only time you should be doing it. So if you have like a brochure, if you have a blog, there should be zero signals on that page, literally zero, right? There should be no, cause even the links and all the other things on the page are just normal HTML links. You don't have to make signals. But if you have a form that they can fill out, you can still do that without signals. So if it's just a standard form, But let's say you have something that's more reactive. like let's say you're doing a car dealership and they can pick the brand to look at and then that will change the model option. So if you picked a Toyota, now all of sudden you see the Tacomas, the Land Cruiser, sorry I don't know enough cars. But you'd see the next select thing will be filled in based off of the other option, right? So you might want to send that signal off to the back end and then. hook into another datastar bound signal so that when things change it automatically updates. So if you're doing something that's dynamic that the person can control, use signals. Otherwise, no signals. Because people want to use it for everything. Yeah, but I mean, what you were saying there is like, I'm selecting filters or selecting options and it sends some, you're talking there about the read signals, right? Because I'm sending something to the server. But then there is still this concept of the signals on the DOM side, because if you want to do that, you still don't necessarily need the signals on the client side, because you could just say, I'm sending some kind of datastar, they are the read signals to the server and the server could just send me back. your, your, what we were initially calling this fat blob of datastar, this, this just giant element, your, your page, basically, even if it's like looking interactive, like let's say they can go through my Toyota and rotate it 360 degrees and theory, you could just do that all in pure HTML and just send us your fragment. Right. So yeah. So at that point, I'm still wondering at what point would I use signals on the client side? Yeah, yeah. I think whenever you have an input or a text area, you want signals. Because you want to basically, as they make changes, have that datastar be ready so as soon as you send a post or a patch or a put, it's automatically hooked in and automatically sent for you. That's about the line where I'd say when you are sending datastar, when you're having someone fill in or interact with an element that's not a button or something, then they're having more of an impact. Whereas like a button, you can have a button that says on click, go do this specific thing. But when it comes to text input, you care about the details of what that text input is. It's not just a one-off click thing. True, but for that, because for that you have your regular form support, right? Where you have like your text fields and your part. We have form support, but you can also do DataStar without using forms at all, because I'm actually not a big fan of forms. They don't handle nested datastar very well. They have this weird encoding, like they're annoying to use in the backend, whereas the DataStar standard way of doing things is to literally just send back a nested JSON thing of all the signals on the page, which, depending on how complicated your thing is, it can be a lot easier to have a nested JSON object that you just deal with in your backend. Like in Rustland, you just have structs that you just decode. from SIRDE and you'd be done. So that's a lot easier than form interaction in my opinion. So we tend to do that. I don't like forms at all. Hmm, no, that's very interesting. No, in theory I also, I mean, I don't have a strong opinion about it. What I do like is the building accessibility support for your text fields and your buttons, but I suppose you are still advocating for using those, the standard elements, but... yeah, 100%. Yeah, you still ARIA controls, all that kind of stuff. You can do that without doing form. Yes. And there's no, like, there's no disadvantage. Like it's not, just, don't know enough. Is there no disadvantage of not using a form in terms of accessibility? No, in fact, it's in some ways easier because you can actually have multiple parts of that thing, not just in that form, but in other areas. In a form, you have to wrap it in that form object, whereas this way you can actually break it up on your page in different ways, like doing a more GUI, or a wizard style. I actually think it's more, the only thing that is lacking is that forms are the only way to really do file uploads correctly. So if you're doing a file upload, that's kind of the time when you want a form, but it's more of a limitation of the browser, nothing else. Okay, yeah, I didn't consider that. okay, so that would mean that your signals are kinda what your element values are otherwise. And do you have to do something special as a person developing an application? Like how do these signals get injected into your action or I don't know, however you wanna call it. Mm-hmm. Yep. Yeah, with the action, when you do a at post and you're sending off this thing, we automatically take what the current signal state is and send it up to the server. And the reason why is one of the hardest things that if you do live view or any of these diffing frameworks or CRDTs or any of that stuff, it's how do you keep things in sync? And my opinion is you should send all the datastar from the client to the because there shouldn't be that many signals on a page. But when you do have something, the server should know the entire state of what the person's allowed to control. So that way they decide what to update and what to change. So in general, you send all the datastar that the person can interact with, which again is the signals, up to the page. And most of the time that should be like five to 10 signals, so it's not even a big deal to send all of them. So you're basically sending up an adjacent nested object that is the current state of the front end. Okay, what if you want like some kind of live validation, how would you handle that? So I'm a very big believer in front-end validation is almost always a waste of time, unless you're doing some kind of web component thing that you're guaranteeing it for yourself. But I tend to do very minimal front-end validation. And the thing is with the fast enough turnaround times and stuff, it's better to come from the back end anyway. Because you still have to validate and all that stuff. datastar starts fast enough that you can be typing and automatically sending a response as you're typing and debouncing and actually sending results and then updating. the thing to actually put in an error underneath the input, for example. So the nice part is you don't have to like, in most frameworks you have to like make a hidden field that then gets made visible when there's an error or not an error. Whereas in the Data Star way, you just have that area, you just update the entire area and it will have the error underneath or will just allow or disable or do all these other kinds of things. Most of the frameworks like will just update that area with the error underneath, but you might. have other parts of the page that now are not available until that gets fixed, right? You might have to disable whole areas. So my opinion is that's why I do fat updates, because then I can update the whole thing with now here's the valid state of like with the errors in line, everything disabled or not allowed, that kind of thing. my thing is the more dynamic you get, the more you want to rerender the whole thing anyway, depending on the state of the application. Okay, very interesting. And do you have to any special work on your side to ensure that your entire like state of your, where your mouse is, your cursor is, like your tap grouping or your, I don't know. Like you have lot of things I imagine like implicit state as you're controlling your frontend as a user. Do you as the updater of this DOM have to take some special actions to preserve all that? Yeah, we do, but you don't. That's the beauty of it. It's all declarative. You just send, like, this is what want the state to be, and we handle all the scroll stuff and all the focuses and all that junk. It's just handled for you. You you update it saying, they made an error. I wanna make sure there's an error underneath. I wanna make sure to disable this section. I wanna make sure that the button doesn't even exist. So instead of just disabling the button, you can just make it so that the submit button just doesn't exist anymore. And that's fine. That's easier, because you can just render what you want on the page when you want it. Yeah, I got you. And then I wonder what you think about offline support in browsers and I guess as a datastar user, how would you support it then? My thing is that, I think there's been a push by VCs a lot to do the offline first thing, and I think they're just missing the point of what a browser is. a browser is a hypermedia client, and like all the stuff around PWA, around service workers and all that stuff, I think that there's caching things that can be done with the service workers to some degree, but it is a online first thing. There's nothing, if that's what you want, then you should just make a app. And there's nothing wrong with that. Make a native app. If you went offline first, use Flutter, use React Native, like whatever, or use Rust directly, use Go. I don't care, but the thing is you're making a native app at that point. So just embrace it and actually get better results. Immediate mode, like using ImageGooey or Bevy or any of these things, you can make a really nice native app. Just do that. There's nothing wrong with that. Just don't try to make a hyper-mediab- Like, you're getting the worst version of both of them, in my opinion, when you do offline first. It's just a dead end, in my opinion. No, I totally agree. I I wasn't thinking so much about like offline first, because in that case, I do agree you want native first. like a lot of times you anyway need the internet because you're looking up datastar, but I'm more thinking, let's say you have a web app and it works great with datastar. And most of the times you just connect and you're fine. But maybe sometimes you want to use it with all internet because I don't know, like it just because you have no internet or whatever. and you want to still have access to maybe things you made with your web app based on the real-time datastar. But at that point, you don't really need it because I don't know, like it's just a part of your app. I mean, I'm talking very abstract, but I'm just saying that there are valid use cases where it's not offline first, but still, it's nice if someone can at least still, I don't know, access the datastar that they already collected. yeah. Yeah. So there's caching that's already built into the browser. There's also these ideas of service workers, like Workbox, that allow you to basically proxy out those changes so that it uses the cache directly for things. So yeah, there's already options in that, like, PWA is already built into the browser. You can do this kind of stuff today with any page. It's not DataStar related, but 100 % you can make a PWA version of DataStar. My thing is that Like, having it offline is fine, but like most of the things that I'm interested in are collaboration first, which doesn't make sense in an offline context. It just doesn't make sense, in my opinion. And I think the thing is, the times when it doesn't make sense, it's becoming less and less. Like you have satellite internet, you have internet on planes. Like, we're getting more connected. I just think that focusing on that too much right now, like if that's the case, I still think that a native app is the way to go. Well not in any way, it's a nice reason to try to convince people to buy your native app which you can charge a bit of money for as like an extra feature. Okay cool, so I mean like Datastar, I like it, it's nice, but there are a lot of nice projects we never see the daylight. So did you ever think that Datastar would become this popular? I'm sure it's still niche in the grand scheme of things, but still... it did became a thing and it did reach for and out and it does have a community so did you ever expect it? I did think it was kind of weird because it was in the htmx discord as just an extra channel and when we split off I did not think we would have like 1500 users in the discord like we have now or whatever the numbers in that maybe 1400 now so there that's interesting but I did actually Ryan Carnito is the guy behind solidjs and he was on twitter the other day saying you know like we are technically better in every way but you see like the react numbers on a graph and you see the solid numbers basically look like a bottom line And I was like, gee, that's funny. And I look at the solid numbers and then you look at datastar starts numbers, right? So we're a niche within a niche within a niche. And so if I was trying to make a big VC push and try to make a bunch of money on it, this would be a failure. What I see is that I've gotten some really good engineers to get involved. And the conversations that we've had inside the Discord channels have been great. and I feel like it's been a big win for me and yet it's also like a superpower in some ways because anyone that really cares about efficiency and speed and wants to use the web, Star is the best game in town, bar none, and I will take anybody up on that. And that's not to say that we're that good, it's that everyone is so busy fixing the wrong problem in the web that we look good by comparison. So why are things so big, why are things so slow in the web? The thing is there shouldn't be, right? Like you have basically a scripting engine on top of a really fast rendering pipeline that is the browser. Like there's no reason why it should be slow. So our thing is more, I hope that we become kind of a baseline and everyone speeds up and gets faster. So even if you don't want to use us, know, people in the, in this industry can kind of see that like there are other capabilities. But I guess my thing is that, yeah, we're very unwell known. Like no one knows about us and we kind of go, In a weird way, we go against everyone in the spa world, because we're telling you guys that you're bad and slow. And even people that are in the MPA world, we're telling you, yeah, you're bad and slow too. So we're not really making friends out here, which I guess is okay, but I like the technical challenge that we're trying to face. what has been interesting though, I will say, I'm kind of shocked at, from game development, like I think we both can agree that like you care about, like here's my... my budget for this frame, right? Like, here's how many microseconds I have to do whatever I'm trying to do. And that's how you're thinking about things. So you don't care about opinions. It's like, this is my framework, this is my budget, this is how much time I have to do something. What I found in the web world is so much of it is based off of vibes and about feelings. And that is really weird to me coming from game dev world in that they're like, yeah, I just don't, I don't really like the interface or I don't like this, like, I don't like the vibe of this. I like doing things this way in TypeScript better. It's like, Okay, but mine's orders of magnitude faster. Well, yeah, but I don't know. Does that really matter? I'm like, yes, everything matters. It does matter, and yet that doesn't seem to be the norm of web development, and I don't know how to fix that. So it sounds to me like I'm just a bitter old man complaining about this stuff, and yet we are the fastest thing out there, and I will take anybody up on that, and yet you don't get takers, because they basically walk away once you start showing numbers. So that's been a weird thing for me. I don't know how to equate that. think anyone in the ruster networking world will not feel like that. I'll be like, that's weird. So it's been weird to me too. Yeah, I mean the web in general feels very slow and very bloated and like even just reading a stupid news article sometimes loads for everything like it's just a text and one image like but of course if you look on the hoots it's not always so because the framework is alone which it is also but also because they do so much analytics calls and this and I don't know like there is a lot of stuff it does that's not really related to your content anyway which I guess is part of it as well. Yeah, it's more the mentality of like, I will just de-skill and let someone else take care of this stuff for me. That's also one reason why I don't think DataStar will ever get super, super popular in that we are actively pushing you to like learn the specs and read the web and understand it. And we don't, we're not trying to get the new guy from a bootcamp thing. It's like, no, this is built for professionals. They're doing professional things that are actually dealing with real datastar and have to like be responsible for their work. So we're very not. AI vibe code, so we're gonna fail. If we were trying to VC it, we would not be in good place, because we're the anti-version of that. Maybe, despite the fact that, mean, for example, AIs are relatively okay in helping you out with writing HTMLs on CSS. And because CSS became so powerful, I mean, if you compare it to 10 years ago where you maybe used a lot of transpilers to CSS because you want to achieve certain things, I find that now with just pure HTML and CSS, you can build anything you want. I do think there is a use for LLMs. I mean, if you like it or not, I don't know. Like I don't really have a strong opinion about it. So that does make me think that writing or vibing or whatever, like a datastar web app might be relatively easy because the technology is pretty simple. Like giving it the context of what datastar is, you can probably do it with a one prompt. So I don't know. So yeah, I actually have friends that are in the AI space and it's, ironically they're using DataStar as a really good context tester in that none of them have been trained on it because it's new enough and small enough and none of them know about it. But our entire docs are literally a one page thing that gives you all the context you need and seeing which engines actually can handle that well. So far I've seen Cloud Code be the best at that. I'm not against AI, but I am against like, using it as a crutch. For me, it can type in auto complete faster than I can type. If I'm like, hey, I need to write this test and I need to write in this style with this thing, okay, go make a thousand fuzzy tests. It can do that faster than I can. But I think a lot of people are relying on it to give them advice, and that's where I think it's really dangerous. In that people are not doing the hard, low level stuff to understand when something's not gonna work and when something's actually gonna be a cost. So it's about understanding the cost, but when you have it, you're basically doing the equivalent of pair programming with a junior programmer that doesn't know enough, and you have to keep coming in context and context. it depends on the problem. It's not that I'm against it, it's that I think I'm against a lot of the ways that people use it right now. More so, they're not using it as a tool, they're using it as a crutch instead of getting stronger. So that's kind of in between with it. Hmm. No. Yeah, yeah. mean, it needs get over this hype cycle. probably, my bet is that it will go to something local only anyway, where like, and even if it's like a smaller model, doesn't really matter because they still run fast enough. And if you have a strong developer machine, it can do pretty well. And I don't see it ever scale economically to a grind. But I mean, that's like, that's, you know, predicting the future. I mean, I'm probably can't feel just as much in predicting it. I would feel much more confident if it was a local first thing. In fact, CLAUD just changed their EULA to be like, you're gonna give us all your datastar even if you're on the $200 plan unless you opt out of it. And it's like, wait, you guys just said you would never do that kind of thing, which of course you're gonna see kind of thing. So I think that's definitely on a hype cycle, but the problem is that there is some value to it. That's the problem right now, right? It's kind of like the idea, a lot of the crypto bros destroyed a lot of the interest. The cryptography space, especially cryptocurrency stuff, I was super interested in the BIP 39 and how you transfer passwords while also creating SHA-256. There's a lot of interesting ideas in that space, but the crypto bros completely destroyed the value of it to some degree. So it's all the speculation that caused the issues for me personally. Yeah, yeah. mean, I also worked in this space and I have developed some, I would say interesting crypto use cases, crypto, not talking crypto, but crypto blockchain. I still think there are use cases. What my feeling is, and actually there is an analogy you can make to the LLM spaces. feel because of all this hype, because of speculation, because everybody was just buying Bitcoin. What I feel is that people missed the point that this is actually still very early research like Imagine if you put all that money in research and keep thinking about this problem, which anyway, something like what blockchain wants to solve in a way always feels like an impossible problem because you always have this kind of yeah, contention between fully decentralized and still have some guarantees, whatever. So in a way it's impossible, but still I feel with a lot of more research around like how you can get to consensus algorithms, how you can find application. I feel you can do so much more. And I feel the same here like... Imagine all that money that's now being spent on silly little chatbots, which, okay, they have use case and they do give value, but how much value do they really give? Not that much compared to the money you put into it. But what if you put all that money into research and not just on AI, but like all kinds of research? mean, I 100 % agree, in fact, one of areas that I've thought, because obviously if I'm a big event sourcing guy, the idea of a blockchain makes a lot of sense to me, right? Because it's basically a personal blockchain is the stuff I'm into. And even, there's a great project, there's IPFS, which is the Interplanetary File System, but there's an underlying project called IPLD, which is really interesting for people that are thinking about networking, it's like how do you make a, basically a unified version of datastar that is shareable and also linkable and all these other things. And that was super valuable and they basically have no backing anywhere and that's by far the most interesting thing in that space and people barely know about it because they basically put their meme coins on the IPFS but they don't understand how any of this stuff underneath it works and that's the part that's really interesting because it could actually solve some lower level ideas of how to save your datastar and how do you share datastar and all and stuff. yeah, that's, I hope the hype cycle ends, the tools are here and they have value. But yeah, people should treat them like the equivalent of autocomplete or like a fast typer. So that's kind of where I draw the line. But everyone has their own line, man. That's what I found out too. Yeah, and they will shift and funny enough, I have not been able to do an episode without talking about AI at some point. But that being said, I do want to move forward a bit and I think now I have a much better feeling about DataStore. Now I want to go back a bit to Darkstar So DataStar relates to HTTP and that spec. So then how does DarkStar relate to WebTransport? Like what does DarkStar do on top of WebTransport? That's missing there. So the parts that I think are missing is, first of all, it has the ideas of datagrams for things like UDP datagrams, so you can send one-way lossy transport. That's a very low-level spec. It literally is like, send datagram, that's it. But if you wanna do video or any of these other ideas, it's basically YOLO, you go do it yourself. So I want to look at how Pion and some of the other people in the WebRTC world are doing their datagram stuff. Basically, I want to be able to recreate this Riverside thing, like a video chat, directly with it in a declarative way. So if you basically hand it a video tag, with datastar dash dark star, it just magically hooks up and does the webcam and all that stuff. So there's some of that stuff that I want to do. Basically, I want to take the ideas of Web Transport and make them declarative, just like we did with datastar star. So once you say, for example, you have a web component and you add datastar dash. net-rollback, that kind of thing. All of sudden it knows that hey, I'm gonna start sending this, in this kind of way and automatically fetch and recreate you and rerun this function for you automatically. So it doesn't matter if it was a WASM endpoint or it was some JavaScript code or TypeScript or ClosureScript, it doesn't matter. It basically is a wrapper on top of it that you're setting up declaratively and that it's doing all the imperative stuff under the hood. So again, it's more about taking the specs and making them declarative so that you can figure it out. any language, not just, the problem with a of these things is that people will try to make something work, but it'll only work in their back-end language. And I want to make it a unified ⁓ hypermedia format so that you can just send down datastar So it's actually in some way simpler than you probably think it is. Yeah, yeah. I mean, actually for now I have no idea. but I, what you just said does make me think like currently I'm building something with datastar. mean, I'm building a couple of things with datastar star and one of them, and then for what I'm using a datastar makes perfect sense. But actually at some point I do also want to support for real time audio and video editing. And so once I get to that point, I will definitely talk to you and maybe I could be like an early. like commercial user of it and like contribute where I need to, of course. Yeah, it's an interesting area where, yeah, if you're doing video audio stuff, like if you're sharing it, creating a DAW, right, that kind of thing, digital audio workstation, something like that, you 100 % want it, you can't wait for the commits from the backend, right, like you have to fill a buffer and you have to do all the web audio stuff to do that correctly. So, yeah, it's for exactly that kind of problem. Like, it's exactly, but how many people have that problem? Not that many, right, so. ⁓ Luckily, because I mean, it's not, I mean, you definitely don't want to it for, and then, and then actually, so there are still two things I want to discuss and then we can start to wrap up is you, at some point you mentioned, okay, you have your blog, like, and you, you, wouldn't use signals there. got you. But why would you use even datastar there? It's a hard problem. Yeah. So the thing about DataStar is that it basically automatically hooks up, if you're just doing a straight, like, static blog brochure-ware site, you shouldn't use DataStar. You just shouldn't. Like, you don't need to. Just create the HTML and send it down the page. But the second you have something that is dynamic, you want to do it. Like, there was a while back where I was talking to the Grafano people. I don't think they ended up implementing it, but the idea was they were gonna use DataStar on their documents page. And even though the documents are very static, there's 18,000 of them, right? There's 18,000 pages to their website. And what they were gonna do is basically make it so that they use DataStar on the page and they would only load the first part of the datastar and then when you got down, scroll down a little bit, it would use the datastar on it, intersect, to then load more of the datastar. So you would basically not be loading the entire page. So the idea there is with a 10 kilobyte one-time cost, they now have a interactive way to automatically update sections of their pages so that they're saving on network bandwidth, right? So instead of sending the entire page every time, they're now sending 1 20th of the page. Well that all of sudden now saves them 20x on their server costs for sending down datastar. Those are the kinds of things that if you're like, hey, even on a static site, Datastar is really valuable and if you pay a one time 10k cost, you probably will find a reason to use it on your page. As soon as you do anything dynamic around a CSS class or styles or... like a click event or any of these kinds of things, you're probably going to end up paying, if you're writing any JavaScript on your page at all, you can probably completely replace it with Datastar. Okay, very interesting. And then in this podcast, we also always look a bit at the human aspects and we already got to know you a bit better for which I'm grateful. But then also you put a lot of time and effort in Datastar and maybe soon hopefully also DarkStore and some other projects. Like how do you fund it? How do you find? Yeah, because I mean, in the end you have to buy bread, you have to feed your family. So how do you do that? How do you combine it with your professional work? So the good news is that working at Synadia, I use Datastar on a daily basis. Like I build the applications that we build in Datastar. So I'm dogfooding it. It's not just like, this is a cute little thing. Like I actively use this for work every day. So that helps a lot, right? And the thing is it's not like I built a framework and then I got the job using the framework. No, I built the framework so I could do my job, right? It's kind of the opposite of what a lot of these things are. There's a lot of library authors that build libraries, but they don't actually use it as their work. So I think that helps a lot because I'm dog fooding it all the time. And then I, I just have a normal job and I have kids and I have all the normal responsibilities. I just feel like this is a part of the web that I figured out how to solve. So it's kind of my responsibility to solve as best I can. So I built the whole open source thing. What we have done in this last little bit, The problem is with a lot of the front end frameworks is they're now getting bought out by VCs. So Vercell is like owning Remix and it's owning it solid and possibly there's a couple other even Vue I think is actually under the Vercell name now. And I didn't want that to happen to DataStar and I didn't want people to worry about that. So we actually went through all the effort of setting up a nonprofit 501C3. So we actually have a nonprofit that DataStar and all of its stuff is under and then we have a set of pro plugins and attributes and like we have a Data Star inspector that allows you to watch things in real time on your page. And we have a bunch of different projects under that and the thing is you pay a $300 one time license and that helps pay for things like we have a couple of people who pays for monster drinks and we'll have a little bit of a fund for being able to go and talk at conferences and stuff like that. But it's not enough to have a job at all. We're not popular enough for that to be the case but it's enough to pay for itself. So it used to be everything was coming out of my pocket. And now that's not the case. But the way I think of it, it truly is a non-profit thing. We're trying to make the world and the web a better place and actually save money and save networking and hopefully be a green initiative. Because when you can save 600X on your networking costs, there's actual real-world consequences to that. If you're 600X smaller, you can actually have less of an impact on the world around you. And I think that there's value to that. So we're trying to be small and fast and efficient. That's what we can do. I would love for us to be more popular, but what are you gonna do? You're also kind of off in the woods doing your own thing. A lot of the times you're popular because you're popular. React got, I remember back 10 years or 12 years ago when everyone was saying React is stupid, they're using this JSX thing, they're not doing standard web, no one's ever gonna use this. I remember those days. And all of sudden somewhere along the lines it flipped and now that's just the way you build the web. So there's a chance that that could happen to us in the future, but I'm not holding my breath. Luckily we have enough money in the bank that it'll be hosted for a long time. We'll have enough to make t-shirts and stuff like that, and hopefully stickers for people that are proven wrong about HTTP2 in websites. Yeah, for sure. So like, yeah, mean, for now I don't have a pro version yet because I didn't see the need for it yet. Well, you the thing is, yeah, you don't need it. I think people think that we're using it as some kind of like ploy to make bank or something like that. Like there's not enough to make bank. I'm sorry. Like if I was doing this, I'd be paid for like pennies, if not negative for what it costs. So it's definitely not for that. But I also think that people should not buy pro unless they actually have a need for it. Like at all. Like most of the things are done and should be done with the open source one. The other thing, to be clear, None of the things that are in Pro isn't like a special thing that we have a special version, like it's all doing the same plugin stuff that anybody else could do. So anybody could write the plugins or do the stuff that we're doing. It's just that we tend to be good at doing this kind of stuff. So we put stuff in there that we think is more valuable to enterprises, but also a lot of times we put stuff in there that we don't think you should be using. So if you're gonna use something that we don't think is actually like is an anti-pattern, you're gonna pay because you're a support burden for an open source project. That's part of why we do it. understand and so why would I need or why would I have a need for something in the pro package? So that's a great question. The things that I think have some value, like if you're coming from a spa framework and trying to, like a brownfield project and trying to bring it over to Datastar, there's things like, for example, datastar persist is saving stuff into local store. That can be useful if you already have stuff that's in local store and you're already doing it that way. I think it should be using cookies and all these things in the backend, but if you are doing stuff like that, that can be useful. There's also things that we have some plugins built around like manipulating the history and the query parameter of your URL, which I think is a terrible idea, but we have the ability to do it if you need to do it. But you could also write this plug-in yourself, but there's those other kinds of things. I'm like, I don't think you should do it, so I don't support doing that without at least having some skin in the game in some ways. What I do think you want Pro for is things like Digital Star Inspector, which basically you can pop on a page and you can see real-time versions of all the signals on your page, every single event that's happening, because a lot of the browsers don't handle SSE very well. like they have kind of bad support for it. So we built our own editor that you can actually see all the live patch elements and patch signals coming through and actually copy and log them and do all that kind of stuff. You get a real time view of all your signals. You can actually edit your signals in the editor, update on the page, and if you hover over part of the signals, it'll automatically tell you what part of the page that those signals are used in. So stuff like that. Like Inspector's a really valuable little tool. The other things that are coming in pro shortly. I'm working on a web component framework that's purely declarative, which is gonna be called Rocket. And that's really powerful because it makes web components declarative, which ironically are not normally. And then I have another thing coming called Stellar, which is my, I think is a CSS killer, or a Tailwind killer, in that it's a editor that basically generates really good design system that ends up being just CSS variables. So you're just relying on, you're just doing regular CSS, but it, builds the design system in such a cool and fast way that you can just go back to using CSS like we were talking about before. Tailwind was built for a time that doesn't exist anymore. The way you write Tailwind is actually really complicated and convoluted compared to just using CSS. So Stellar is about creating a good design system quickly and then creating the CSS variables that you can use and do interesting things with. So, but it's outputting CSUSA or... Yeah, still outputting CSS variables. like for example, you'd still say font size equals and you do the var dash dash size dash 10. But the things are, for example, that we're doing a ton of math in the background so that, for example, you don't have to set breakpoints between mobile and like all the different breakpoints, can do medium and large and all this stuff. It automatically does fluid binding for every variable. like whether you're on a mobile or on a large view, we automatically do perfect pixel perfect. scaling and exponentials and all these other things for you automatically. We also do things, for example, with colors where we guarantee that you'll have the correct contrast. You can't have a bad set of colors that are next to each other. Like it does all the compliments and all that stuff automatically for you. And it also follows the contrast accessibility guide rules. So you know how sometimes you'll get colors but then they don't match or they have a weird thing where like because the one thing's green, the other thing's blue, the blue seems darker even though that's right color. All that stuff we take care of for you automatically. So you can literally just point this to a background image and we'll design an entire theme for you in milliseconds. And so that also can happen live as well. And can you... And people can see a sneak view already? Yeah, sent it, did it on the Discord, I did a video of it a while ago. I'm actively working on it today, so I will have a better video probably out later today, hopefully, of all the new features that I'm doing with it, yeah. Okay. Yeah, do send me a link because any like of this kind of I can put in the show notes and actually so you're telling me that I get all of that if I buy the pro version Yeah, and the thing is it's a $300 one-time license and you're done. So it's a lifetime license for all the stuff that's added. No, for the company it's a thousand if you're under a certain size. But the thing is we're getting more and more things that are coming in. We'll also be working on a UI library that's using Stellar and Rocket as well. like that and datastar grids and all these other things that people will spend, like people will spend $1500 or maybe even $2000 now on AG grid and we'll just have one built in. But that's for a single developer, right? It for a company. right, for any language built in. So the thing is there's value there and it's gonna get, the price will go up over time as it gets more more features. So people that buy it now will get more out of it as it gets bigger and bigger, so. Yeah, for sure. I just so far I didn't buy that because none of these products that we were building do have any uses yet. But like a... Yeah, it just like it does seem very interesting. So then I'm thinking like, okay, but I'm a company so I do even though I'm pretty much a single developer, but still it's I think it's only fair to buy the company license, I guess. Yeah. Well. It's really up to you, like, for your, like, do solo until you don't need to kind of thing. And Ben Croker, of course, asked him about it, but my thing is more, like, if you don't see value in it, don't buy it. But I will say that it's gonna get more expensive over time, and we're gonna be adding a lot of stuff to it. So it's one of those things of, if you see value, great, get it. If you don't, don't get it. There's also the nice part about being a non-profit is people can donate directly and actually get a tax write-off if they want to, right? So. It really depends on what makes sense for people. And my thing is, I'm not gonna be going out, like if you don't see the value in it, don't buy it. But if you do see the value, like it's a couple of hours of contracting time for most developers to make it worthwhile. And so if it's worthwhile to you, great. If it's not, fine. you're not gonna, so it's, the problem is I think open source is completely broken, right? Like everyone wants everything for free, but they also want all the support for free. And. I just wanted people to have a little bit of skin in the game and the rest of it, there's no, like none of the profit, we're a non-profit with no stock. So it's not like I could even sell it or like we're stewards of the project. We don't own any of the non-profit, which is a weird thing about this stuff. yeah, it's, we're set up to be a long-term success and we're trying not to get burnt out either. Cause it would be nice to able to like grab a soda. and have someone buy a Coke, right? Something like that. So it's not a lot of money, but it's enough to keep things going. So that's good. And that's fine. Like we're all still here for the fun of it. And if it wasn't fun, we wouldn't have done it in the first place. Yeah, totally agree. Okay, with that said, think we discussed a lot already. Do you feel that we missed something around any of your projects or about you? No, mean, I'm not the, all it'll say is that I'm not that interesting and I'm not like a great developer. So a lot of the times people feel like they can't accomplish something because they're like, if all these people and all this money has gone into web development, how could it be like, how can I fix it? And I guarantee you, if you have an itch that's itchy enough, you can do better than what's out there. So it's more like don't have that imposter syndrome. Just get in there and start asking questions and start building stuff. Like stop. fighting yourself and just try to solve the problems that you need. And there's gonna be other people that have that problem too. So that's the thing that I found is that you will find a community if you care enough about something. So start caring, start building products and stop complaining. very nice and a good note to end on. So with that I want to thank you for your time, Delandee. Well, thank you. And thank you for all the time you spent in the Discord and helping out with the project. We really do appreciate it. We have a really good set of people that care a lot. And even if you're not a web person, come in and ask questions. We have a lot of talk about event sourcing and CQRS and networking and setting up distributed systems. It's interesting because it's mostly back end developers talking to other back end developers. There's a lot of interesting stuff that has nothing to do with DataStar. So I highly recommend checking out the Discord for sure. I do second that and it's all very nice people and they all seem very nice to help you out and yeah it's just a very friendly bunch of people so I do recommend it as well. take care and we're gonna invite you back once it is more about Darkstar or if there's anything else you want to share in the future. Absolutely, thanks so much for having me. Elizabeth (Plabayo)
1:31:03 | 🔗
Netstack.fm is brought to you by Plabayo building secure, open, and resilient infrastructure with Rust protocols, and purpose. This show is also made possible by Rama, the open source networking framework. Plabayo offers service contracts and welcome sponsorships to keep building and supporting its ecosystem. The theme music of this podcast was composed by DJ Mailbox. If you enjoyed this episode, don't forget to subscribe on your favorite podcast platform and leave a five-star review. It really helps others discover the show. Thanks for tuning in. We'll see you next time for the next handshake.