Is TypeScript the King of AI? Plus CopilotKit, TSAI Preview, and Mastra Agent Studio

Hello everyone and welcome to AI Agents Hour. What up? We are in a different location than normal today. So, we're in person clearly. We're hanging out and we

thought it'd be uh fitting because we had a whole bunch of the MRA team in town to have the uh have the live stream at a bar. So, we've joked about it, we talked about it, and now it's happening. This might be a one-time experiment. We may never do it again. Or maybe this is just the new vibe. We'll see.

Our whole a lot of people from our team are literally over there working right now. So, they may um they may drop in. The sound is probably going to be a little less quality than normal. Uh you might have some background noise. Let us know

in the chat how bad it is. We can try to do a few things on our ends to improve it, but we're not going to be able to completely fix it. But we have a pretty sweet show lined up today. We're going to be talking AI news like always. We're

going to talk about a recent Monster launch. We're going to have a tie from Copilot Kit coming on. And then we're going to talk about the TSAI conference which is this week in a few days. 100%.

It's going to be pretty sick. We are at a place called Victory Hall. If you're in SF, you should check it out. Usually

no one ever comes here which is why we come here but today of all days it is popping in here. So it's it's a busy day. Yeah. They knew we were coming I guess. Yeah. So if you uh you know if you're

interested in finding us now you know you know we just we doxed ourselves. We just doxed oursel. But hey come say hi have a drink. But we're going to start with talking about a yeah something that I saw which

I thought was kind of interesting. Before we dive in to everything else, saw this tweet come by from Theo. Okay, Typescript is officially the number one language on GitHub. And

that's cool. And we also, if you want to read the actual full post, you can see, yeah, AI leads TypeScript to number one. That's literally the the title of the blog post from Octiverse 2025 and it talks about if you go down in here, you know, you can read the whole post, but it it basically mentions that

now TypeScript is over. Look at that curve too. Yeah, it essentially talks about how Typescript has overt taken Python and JavaScript to become the most used language on GitHub, reflecting how developers are reshaping their toolkits.

This marks the most significant language shift in more than a decade. Damn. It's almost like we talked about this many times. Almost how we imagined it. Hey, Ryan Carson's here. Hey, Ryan. Hey, Ryan.

Tell us how the sound quality is because we're a little nervous about it, but you know, we're having fun. Having fun. We're in SF. We're at a bar. We're just live streaming from here. All right. So, yeah. So, that's pretty

cool. Typescript is uh clearly becoming more and more popular. Typescript ships.

This is kind of a lead into what we'll be talking about later when we uh yeah talk about the TSA icon typescript ships. But I wanted to talk through just quickly before we go into the news for today just a little bit about a recent launch we had last week. So, one of the things we've been hearing over and over, if you've used Mastra, you've probably seen we have this really nice playground

that actually funny story, the first version of Mastra, we were building this playground and we basically pulled it out. We're like, it's not good enough to ship with. It doesn't work. So, the first

the first version of MRA didn't have it, but we built it. We had we just pulled it out right at the last minute because it it just sucked. Yeah. But we knew it was important or we thought it we at least had the idea that

it would be important because getting into AI, you know, for ourselves, we've kind of come from the web world. We've had like web admins. We've we've had this idea of being able to wanting to visually see the the progress you're making and test and iterate. And so we added it back in. We've been iterating on it a little bit, but it was kind of

always something that we had, but we didn't we never really focused on. But what we kept hearing is over and over people just wanted to figure out like how do I deploy the playground? How do I use this playground in production? We

heard that over and over. And so we pretty quickly realized that playground was a little bit bigger than just a small testing environment. And so we've rebranded it, rebranded it to studio or agent studio. And now you can just

deploy it in mastercloud for free and collaborate with others on your team. ship your agents to the rest of your team to iterate on more quickly. And yeah, so that came out last week.

Dude, Studio is something that we thought was uh it was cool. We didn't realize how useful it was until you get very deep into your agent career, let's just say. And then people without the studio always tell us, I wish I had a studio in like where I was. So, um,

yeah, I'm glad. Oh, we have a bunch of comments. Yeah, Ruben, looking forward to the conference this week. I'll be watching online. We'll be watching you. Yeah, thanks for thanks for tuning in. We'll we'll be here.

Mauricio from Brazil. And then this one's a good one. Oh, ro this is exactly why we you know why we built it because we kept hearing oftentimes you'll have multiple technical people you know multiple engineers multiple developers and then you'll have someone who's kind of like a technical PM or engineering manager that wants to be able to collaborate and see the progress and so yeah you you

basically nailed the the use case there is is how do we get those slightly less technical people that that aren't touching code involved early in the project. Also, a fun story. Ro and I have been to uh the dog patch saloon together. So, thanks for calling in.

All right. Okay. So, it's time. We do this every

week. We dive into some of the news. Yep. And you know, today we have, you know,

we'll try to go through this kind of quickly just because we do have a guest coming up. There's also some drama, too. Okay, let's start with the drama. We always Yesterday's timeline was crazy. drama between Sam Alman and Elon Musk.

Yeah. Um, dude, it was crazy as Like, and we're talking about Open AI, of course, today. Yeah. This is This is it. This is on the list. Yeah. Let's Let's talk about the drama. Let's talk about the drama first. Okay.

So, this is how I I'm going to play I'm gonna play I'm going to do a playbyplay. Okay. And I'm gonna also say some funny Okay. So, Sam Almond uh posted this picture. It's like a tale in three

steps where he like bought a Tesla Roadster and then he asked to be cancelled or he asked to get refunded and then the third the third picture is like you know like post can't host not found in email which is hilarious. Okay, we have to also establish that that's pretty funny as a roast, right? Then Elon like he goes he gets all defensive and He's just like, "Hey, well, someone made a nonprofit

turn into a profit and all this like nonprofit about OpenAI." And then now you guys go watch it. Go look it on the timeline. It's hilarious, dude. But that was yesterday. Yeah. Well, I mean, and Elon was

recently on the All-In podcast talking about the lawsuit that he has against OpenAI and Well, Yep. And Ruben brings up Yeah. There's been some recent like OpenAI health or legal drama where I I've heard just as of today, I don't know. I haven't validated this, but maybe Chad GBT is not going to give you medical advice anymore. So, there's a whole bunch of uh

it's it's always a little bit of drama in They also released the court or like the the the manuscript of Ilia's uh board thing. I read it. It's hilarious, dude.

This is so funny. It's like Yeah, it feels like watching an episode of like Suits or something, you know? It's honestly like at some point there will be a, you know, like a Silicon Valley show or a social network movie made around this time. You just know, you just know it will be and it feels like we're kind of living it, which is is fun because you get that uh

entertainment while we're all shipping stuff. I watched the like the Uber movie with like Jose Joseph Gordon Levit as a whatever that guy is. Um, Travis, that movie is tight, but it's also like being in like startups and stuff, you know, it's kind of like it's not for us, it's for like the general masses and

This drama is hilarious, though. All right, so let's talk about some other news. Uh, I did see something that maybe teased OpenAI might be planning to IPO. They figured out how to go from a

nonprofit to a for-profit in some way. I don't know all the legal uhations, but they've they've figured it out. They found some legal maneuvering to make it happen. So that is, you know, likely

coming and estimated one trillion. That's a lot of money, you know, if you work at OpenAI. Yeah. Good. Good for you. Good for you. But they have a bunch of other things

they've released as well. So they're doing a collaboration with AWS and, you know, a lot of information around how they're just going to be using a lot of AWS uh resources. But they did in the the post they talk about not just buying like a lotment for GPUs but buying CPUs as well. So that's interesting like they want they want it all.

Open AAI wants it all. What are those CPUs for? I wonder.

Yeah, that is kind of interesting. Uh they've released something called Arvark. It's an agent that finds and fixes security bugs. Uses GPT5. So pretty sick. Yeah, it's it's nice that

they are like, you know, building some of this stuff. They did release something open source as well. So, last but not least, as far as OpenAI things that they've released in the last week, OSS safeguard, which is which is essentially like guardrails, right? And we've been talking about

guardrails for quite a bit, but it does have kind of a a slightly larger but then a pretty small model that you can kind of build write your own custom safety policies and then it'll try to be the guardrails to enforce those safety policies. Yeah, this is really cool. And I think this is telegraphing like what we're going to see in 2026 is SLM movement where you

have smaller models doing specific things that make sense. We had uh the homies on a couple weeks ago uh for uh the condom episode. Super agent. We had Super Agent episode that forever be known as the

condom episode, but this is kind of that same vibe. Yeah. And you know, and I think we talked with Osmosis, we talked with Andy about like reinforcement learning. And so I think this idea of smaller, more well-tuned models toward your uh

specific use case could be better in certain situations. It's not it's not going to replace these large models. The large models have a place, but I think we'll we'll continue to see that more and more. Yeah, model routing will be very

interesting in 2026. You know, uh Ruben has an idea. Maybe we should do that. We'll make a video about using open models and small language models with MRA. Yeah, we should do that.

That's a good idea. We still need to do Maybe we should do a workshop where you know how we always wanted to break down every model. Maybe we should do that in a workshop or something. Yeah, because the Chinese are coming as we

tell you all all the time. Yeah. Yeah, we we do we will at some point also just kind of go through various it's on the list. It's on the

short list of of topics where we just cover all the major, you know, Chinese model providers as well as like their list of models. All right. So, thank you, Nick. You support that. Continuing on here, we do have

uh stage hand has released kind of an update. They did a rewrite to run basically they dropped Playright and they're running directly on CDP. Obby, what's CDP, dude? Honestly, it's just

to control the browser. Um, yeah. And Playright, dude, like having been in webdev for so long, playright is actually, let's say, like the latest evolution of this right? Because we have been using different Whoever used Cypress, like let's pour one out for

Cypress real quick, you know, like things like that. And then he also had like Mocha, like ET and all this So now Playright's like the like the it was the coolest one in this evolution of freaking like um browser testing type And you know Paul fitting they don't need it. Sage hand is a genic version of

this. You could just you know you could do this stuff already. If you were going to use Chromium browser why do you need playright? I get it. It's tight. I mean, it made sense at the beginning because it was a way to get there faster, but

then I think they've realized they actually have more flexibility if they just get rid of that dependency. It's, you know, I think that's a pretty common trend. You kind of like move fast, you're dependent on certain things, and then over time maybe you realize the dependency wasn't, you know, wasn't actually needed or isn't needed in the long term. Hey, Tony's here. Get over here. All right.

So, we are taking a break here. Let's uh What's up, man? You guys know Tony? Hold on. We're gonna stop. You can't

see. There we go. There it is. Here. Hello. We're getting We got the whole uh a lot

of the team together. Not everybody, but it's an exciting uh exciting week here in San Francisco. You You just rolled in, Tony. Yeah. Just got here. Tony rolled in from another country.

Yeah. I I rolled in just, you know, 20 minutes ago from another state, but legit different country. Um, I rolled in here out of bed, so that's what I did. All right. Well, yeah, the audience

says, "What's up, Tony?" What's up, Tony? What are you excited about most for this week? Maybe that's a good segue.

The conference. Yeah, the conference. Anything specific about the conference? Any speakers you're excited to see?

all the speakers. There's a very good lineup. Yeah. Yeah. We're going to be talking a single one. So,

okay. We uh Yeah, we're going to be talking about the conference here in a little bit and we'll be previewing some of the speakers, but I agree. The lineup is sick. Super sick. And if you have if you're not already

registered, you can't buy tickets for in person unless maybe you DM us. We might have one spot we can find for for a dedicated audience member in SF if you were if you didn't get uh tickets. But otherwise, uh, you can attend virtually.

So, please, uh, please go sign up tsconf.ai. We'll talk more about it.

It's been a while since we all seen each other. We work remote. Mantra is like a remote company, but we're all homies from a long time. Like, we all came together and so like these things are special like to meet up and stuff. Yeah,

we'll we'll we'll try to bring on some more mastros at the end of the the episode when uh because I know a Tai is coming soon, which reminds me, I gotta send him some information so we can actually join this thing. All right. Did you get a beer yet? Not yet. It's time, dude. It's time. All right.

Good seeing everyone. I'm going to go and get something to drink. Yeah. I'm thirsty, dude. You get us something. I have a tab

open. So, put it on my tab. Uh give me some kind of cider. Yeah,

I'll take a second. That sounds good. All right, everybody. Yeah, we're just we're having a good time. So, you want

to uh talk about this next one and then I'm going to get our guest on. Okay. So, so as I was saying, and I keep saying every week, the Chinese are coming. Okay.

And there was actually a comment in the live stream that I'll I'll I'll do right now. Do you have a favorite model outside of the US? Yes, it's Kimmy aka the thing that we're talking about right now.

So, Kimmy linear is it just came out. We do model highlights all the time. Um, the cool thing is we're going to have this like hugging face adapter soon. So,

if you're like a JavaScript dev and you want to try out these like models and stuff, it becomes a lot easier. Um, cuz we obviously really like them ourselves. Uh, so just another Kimmy one. So,

that's cool. And then let's talk about another I told you, dudes, the Chinese are coming. Okay. Uh Quen, another one of our favorites uh internally here at

Monra, Max Thinking here. I actually tried this one. It's kind of slow, dude.

to be honest, it's kind of slow and I don't think Max thinking in general matters to me, but that's just my opinion, you know. Uh it's not like it pondered some that I would never think myself. Like it's kind of whack to even spend time on that, but you know. Yeah. But I will say though, it's not even released yet, but if it's true on

the benchmarks, there's it's actually pretty good. 100% on two different benchmarks, aime and hmmt, which that'd be the first one. I guess we can pull up, we do have those links here. We can pull up the benchmarks so you can kind of see.

And I think a lot of these are just like it's hard to know if is it just benchmark maxing where it's basically training on the benchmark or if it's actually like did it actually solve the real problem. That's why these benchmarks usually come and go because eventually you've set a benchmark, all the models train towards it, they they do a really good job and then they move on. They need a harder benchmark or a different benchmark. So you can see

there hasn't been any 100% yet on this AIM 2025. So that in its own right is a little bit interesting. And then you know this HMMT 2025 benchmark is also uh you know there's been a few the only one that's been 100% is GPT5 Pro. So as

you can see it's basically the statement is it's going to be up there with GPT5 Pro as far as reasoning is concerned. So it is interesting. It always makes you at least stop to think that that's a pretty pretty bold claim.

So we'll see when it actually comes out. But overall um it is looks promising from from a you know kind of open model. And we're going to pause uh talk. We

have some more AI news. We have some more models to talk about, some more releases, but we have some guests as well. And so these are some people we've also shared beers with at bars. And you know, we're just going to be uh sharing

a conversation. So let's bring them on. We have a Tai from Co-Pilot Kit.

Hey Shane. Hey. Yeah. And Julie. Ulie's here. So, I'm gonna bring him on, too. Why not?

Oh, wow. Hey, guys. Hey, guys. Surprise guest.

How's it going? It's going well. We are actually in a bar right now.

So, we got a whole bunch team. Yeah. And we are just co-working. You know, we

didn't get a co-working space for today because everyone's just getting in at different times. So, we are just co-working from a bar in San Francisco. We always meet in bars, whether in person or virtually. It's It's always about Yeah, it's like that's where we hung out

last time. Is that a good sign or a bad sign? I don't know. Seems pretty great to me. Yeah, it seems like a good sign to me.

But yeah, we're we are uh trying to make some moves and while we're here, we're also having some fun. So, yeah, but it we were excited to bring you on obviously to talk a little bit about co-pilot kit and of course say thanks for being part of the TSAI comp which we're going to talk about a while as well. Uh but maybe I know I'm assuming

most people know about Copilot yet if they're watching this but if not just in case they haven't they haven't heard they've been living under a rock tell us a little bit about Copilot I don't know give a quick pitch uh essentially Copek is the agentic applications company right we make the connective tissue uh

between aentic backends uh which is your your guys' world andic front ends right userfacing agentic applications uh and We're also the company behind the AGUI protocol, the agent user interaction protocol, which is a general purpose connective tissue between again an agenic front end and aenic back end. From a technical standpoint, you know, the last 30 years of the internet,

everything has been running on the request response paradigm, right? You make some client makes request to a server, get some data, renders that data, and interaction is over. Uh, but agents, you know, have a long list of qualities that make them incompatible with that. So that's a technical kind of source for all this. Uh and yeah, we at

this point power, you know, million of millions of agent user interactions every single week, growing exponentially, have over 180,000 installs to our packages every week, uh all the way from, you know, Fortune 100s, uh startup unicorns, just startups, everybody in between. And yeah, very excited to be working with you guys, uh on the master integrations.

Yeah. Yeah, definitely. We we know that's it's been a problem that a lot of our users have been trying to solve is what's you know how do we build rich user interfaces for these agents and so it's always it's cool to see all the progress you've been making but also to have tools that users can can kind of

pull off the shelf and and customize to make make it fit whatever they want to build right it's kind of gives them the flexibility but also like lets them move a lot faster yeah exactly I mean there's a whole new set of building blocks that you need when you build these agendic applications all the way from obviously the basics just chat uh to generative

UIs there's different flavors of those uh but then all the way to you know uh shared state uh where you can have state synchronization between agents and applications there's different flavors of those too there's read write and then there's read only in either direction you have frontend tool execution so anyways this list of building blocks

it's not infinitely long but it's not that short either uh and um We essentially, you know, want to take care of that so that both application developers don't have to worry about that and then also you guys can give all of your ecosystem these functionalities hopefully relatively easily at least more more easily than you're playing it

yourself uh by kind of integrating with with a a protocol. So, like if I'm a front-end developer and I want to get into generative UI, like what does that mean to a like a person not in AI yet? like what are like the front end like how does your life change like as a front-end dev getting into generative AI through the UI? Yeah. Yeah. And you know what? I could also potentially use this as a

springboard to share screen share a little bit and we do. We love that. We love seeing demos.

All right, let's do that. De see how that works. All right, can you guys see this? We can. All right. Great. But maybe I'll so I'll

start here. Uh but I'll I'll uh I can share some some CLI stuff too to show people how they can actually get this type of stuff working on their own computers in a few minutes really. But um this is the AGY dojo. It's just kind

of a showcase of a bunch of the building blocks uh that we have in uh and you know just to show that you know you have chat obviously so you can just say chat with the agent get something back with streaming and so on. This is also showcases support from for front-end tool calls with this silly example of

you know changing the background color right so uh you can do a lot of really interesting things by the agent being able to execute front-end tools but this is just a bunch of hello world snippets uh to to kind of get to the core uh there's backend tool rendering so that's one type of generative UI right when the when an agent calls some tool in the

background uh you want to be able to show both the inputs the arguments to those tools and the outputs what it comes up with. And uh so this is backend tool running is one of these examples. Uh and then of course you have other flavors too including um you know front end uh front-end tools. So these are

tools that live just on the front end. We have here a haiku generator. We see here it you know generate a nice haiku for us. uh and you know again these are kind of simple little examples but but showcase uh uh the ability to not just

have chat in the experience and then this is another kind of neat example which is shared state between the agentic uh front end and the agentic back end. So here we have a recipe builder and let's say we want to make a you know make a spicy pasta dish and we'll see here that as the agent is on it's going to update this shared state now that exists between the front end and the back end and let's hope the demo

gods are with us. Guess gods are not with us today. Try that one more time. I'll switch to one of the Hold on. Let

me see. Uh, make a spicy pasta dish. This is definitely Yeah. Okay, there we go. So, we have here

update it. Anyways, you need you need the demo gods to be on your side. That's always important. Okay, we got we got

this update here coming. Um, so this is some of the things that you can do with with share state. There's the the list actually goes on a lot more. Uh we we I realize we need to complete our dojo but but all these features are working on

integrations. We just don't have all them in the dojo. Uh but maybe to talk a little bit about generative UI. So

you know generative UI is about again things in the chat aentic chat that are not just text right they're actual components you can use and there's a really big spectrum even on this just one part of you know this aenic front end. There's this there's a big spectrum of possibilities for generative UIs.

There's what we call static generative UIs and that's where you have you know maybe a long but a finite list of things that the front end specifically supports. And the advantages there are you you know the designers can get these pixel perfect right for the five 10 things are showed most uh uh commonly in

your application. The disadvantage is that there's all they're pretty high coupling between the front and the back end like you're used to you know in the old world preai. Uh but there's other flavors of generative UI that solve diff have different trade-off make different trade-offs here. So there's for for example fully open-ended generative UIs

allow the agent to say to bring any UI it wants at all either through an iframe or through YH HTML. You have some examples on that uh in some generative UI formats that AGUI supports for example MCP UI. Um and then there's this middle of the road uh which I think is really interesting which we call declarative generative UI. Uh and that

allows um the front end the back end to agree on a spec essentially of semiopen UIs and then the the agent can specify these cards and these cards themselves are not fixed. They can be very different from one another but but they use the same building blocks. So let's say they use um and in terms of input

forms you can have address forms or uh date forms you have maybe who will autocomplete for the for the address and for the date whatever you want and then you have output uh fields you can have uh graphs and charts and so on. So, so there's a big spectrum of possibilities there. Uh, and uh, yeah, we we essentially try to be agnostic. We we

explain to people the different benefits of each ones. Uh, but we we make all of them available for for developers to use as they see fit. Yeah. Do I love that uh I love that recipe demo because it just shows like the beauty of like shared state. Like I

mean most things are schema driven, right? As we've known, we always are gravitated towards like the schemadriven development. The only problem then is like you have to make sure the agent actually does whatever the you want and that becomes the the next problem of that, right? Yeah. And I can show like so the you

know these are the building blocks uh that you can use but then what what applications you actually build with it and this is maybe the the bonus at the end is this is just the CLI command for getting started copit but you know you can build there's really two c two reasons why people building these genic applications one is just to you know

simplify complex SAS right so this is an example of a SAS copilot you can ask here everything for us we're doing is developer facing so these are like you know developer platform where you can manage developers and you have testers and lab admins and so on. So I can ask you know uh show me the breakdown of work by a platform

and you can see here okay here it is front end backend docs and so on. Uh how about the breakdown by contributor, right? So nobody nobody can hide. So we see here, you know,

John John is pretty active. Uh uh Cersei is not not so much, right? So you can have this interaction. Um and but we can also have like in app,

right? So I can ask it, you know, uh, show me all the PRs that you think are at high risk for emerging by the end of today. And now it can use actual judgment and show show them show me these PRs. All right. So it can use actual

judgment. Yeah, that is tight, dude. Yeah. No, it's nice. It's nice. And I

mean this is very soon going to become the norm in all the SAS that we use uh uh probably for the benefit of all us. And then the other reason people use these genic applications is to just accelerate core work right and obviously developers we're used to that a lot in our day-to-day engineering workflow now with cloud code and cursor and all these

applications but um the same is really coming to every vertical. So this is a really simple I think we we guys we built this together for the the first uh hack uh uh what was it like the event that we did the workshop yeah there we go um and this is kind of a mini linear right a project management software but now that has a co-pilot in it we can ask it you know uh create a project to redesign my

website from scratch make all the decisions yourself don't consult me and assign tickets to uh different team members based on their strengths, right? So, this is a bit of an artificial example to keep the demo short. Uh, and you know, this is something that we built during the workshop in in what in like a couple

hours maybe. So, so this is very easy to build this stuff. So, we got this all these to-dos here. Say like you know uh

break the tickets up a lot more. This is too high level. And we see here the example of you know junior view. You can see what's happening behind the scenes here. Again, this is kind of this uh uh workshopy

example. So, so pretty straightforward. But now we see it's it's it's done here, right? That's the power of these tools. You

know, to fully automate your project manager, we're probably not quite there yet, but to give your project manager this superpower uh like you know, cloud code for everything. Uh we're here. So, so that's uh kind of neat, dude. I love this man. So sick. I love it. Yeah. Yeah, I really think too there's

even like another level that we're starting to see more and more people experiment with which is not only is the you know is the co-pilot helping you collaborate on something but the co-pilot could also be you know you can take a step further can be educating you on how to use the product if it's a new product it could be like exposing different parts of the UI to you as you

kind of like get further along with un like onboarding of the product there's like there's like multiple levels of where like AI injected into an app into like the SAS application could really help not only the onboarding experience but also the collaboration experience get further a lot faster. Yeah, it's really taming complexity like

you now have this almost like a person that that ships with your app knows everything about it knows everything about the user can help the user use it. Uh so it's both in terms of you know managing the complexity and managing the actual work like we can hand off some work to this person uh and and uh you

know it it's not like you you're out of the loop. You're still very much you know there's this whole big discussion we can have on you know autonomy versus not autonomy. Maybe an interesting angle here actually is um when you flip this this script where now the agents start to learn from the user as opposed to uh the user kind of directing the agent and and I think

that's uh that's something we're also very excited about kind of the next slightly around the corner taking all these interactions between agents and users and and essentially building RL data sets from them. uh and then you can use those and that's just to for clarity for anybody watching it's not us stealing the data or taking the data

it's building infrastructure to allow any any user customer to to make this data useful actionable so that's something that uh I I think is is going to be you know RL is what took that's the main thing that works to to improve these agents you know at scale uh that's what you know took the original GPT uh autocomplete to chat GPT the um it's the models are really good

at coding because there's so much so many RL data sets and there's now you know the leaders here are kind of already collecting all these interactions I like to ask people who's who has ever like trained RL data sets and like nobody raises the hand says like who's professor Andy would yeah we it's funny that you bring this

up because we literally had conversation with our friend Andy from Osmosis who's doing the same thing it's it's I do believe that you're on to something and he's on to something and we've all been talking about it a little bit is that RL is is going to be huge in the next year and beyond. But I do think it has it's

going to have a place in 2026 where a lot people who maybe heard fine-tuning, didn't really care about it, thought it was too hard, and now it's like, okay, well RL is maybe a little bit more approachable, but you you kind of have to you have to collect the data. You have to be able to then curate that data and then know what you're doing to

actually do something with it once you have it. Yeah. Yeah. there's a lot of these bottlenecks to actually be able to use

these systems. uh like just the infrastructure is excuse me is a big one but that's that's getting a lot of progress really quickly both you know by the model companies open AAI anthropic have like better and better fine-tuning APIs and also you know uh thinking machine labs and all those um but but data is kind of still a really expensive bottleneck and have to pay you know scalei like I don't know many many many

thousands of dollars for for like a little bit of RL um And uh and it's also ultimately not that specific to your application, right? Because you can't hire people who can RL your sales co-pilot or your application, whatever that is. But if you flip the script and and now start having your users be able to be be the trainers. Um because every act of interaction with an

agent is is a nudge like did it work well or did not work well. Um and and that flips the script there. But uh you know that's the next uh a little bit around the corner. Uh I think it might be here dude honestly

like I think it's coming like not even around the corner like tomorrow. Tomorrow you know. Yeah. Well I mean yeah I don't know.

Is that is that Professor Andy at Osmosis? Well I mean I don't want to put words in his mouth but you know like uh writing is on the wall. Yeah people are working on it for sure. I mean yeah the writing is on the wall. Yeah 100%. He mentioned that they threw an RL

event and they had a capacity for 200 people but 800 people signed up. So there's a lot of interest. Seems like a hype thing too. Yeah. I mean there's some hype of course this I mean look it's we've seen it working

and and that's that's the tool that the model companies use like reach for when they want to enter the application world. It's okay let's let's RL it for this application and and it works. It works really well like really really well. It's how do you tie all these pieces so that you can bring it to any team at any company. Get like RL agents

are customized for you know per agent per customer. Uh I think that's coming you know sooner rather than later but you know like if you could self-s serve that then it's game over right that's the that's the dream. Yeah that's uh yeah you can self-s serve yeah because you typically using smaller models that are more like hyper tuned towards what you need. That's that's often what we've

been seeing a lot or people are talking about a lot more. There's there's a lot of like token efficiency gains you can get from it as well. I mean there and as well as like quality quality is always number one like people are always asking how do we increase the quality but there's other gains you can get from it too. That's true. Yeah. Cost cost is definitely part of the equation

especially when people actually roll through production because these are these things but um typically cost is like it's kind of one of those things where it's actually going to cost you a lot more in the beginning probably. So it's like it's actually more expensive in the beginning, but at a certain level of

scale, maybe it actually decreases cost. That's right. And there's also, by the way, there's what people call in RL. So

that's not even fine-tuning. It's essentially prompt optimization, but you encode in the prompt, you know, all the the things that worked well and did not work well, all the user journeys through an application like to get to some good end result. There's a lot of information there. you can use it on the model weights but uh just making that

accessible through the you know LLM at generation time is yeah in the right way the right format is is also really really powerful so yeah I mean that yeah that that's just like another tool for context engineering right it's like how do you get the right how do you improve the prompt in a way that you can get the best results you can get out of the existing model and then if you need to

go further you can actually pull pull something out that changes the model weights how how much time do we have We got like five more minutes or so. Five minutes. Do you want to do a quick uh showcase of the the master getting started CLI with copet?

Sure. Yeah. Yeah, let's see it. All right, let's let's do it. Um I think I'll just share my entire screen, which

is risky, but uh I'll risk it. You don't send me any funny. Um all right. So, so this is it's really it's

it's really cool. I really shout out to uh Tyler who built this CLI and also shout out for the integration building Obby. You you are honestly a big pusher behind that. You and Ward alongside

Marcus on our side. So good job guys. Thank you. Thank you.

Uh all right here. So NPX goit latest. We're going to use MRA. Let's call it uh

hello. Hello again MRA. That's that's really all there is to it.

Now, I cheated and cloned it in advance so that I don't have to install and also put my OpenAI key live here. Um, but it's it's that's all it is. It's this is the exact same folder and I just run npm rundev.

Um, and if I now go to the 3000, what what you get at the end of this is this hello world kind of environment that has everything already loaded in it for from many of these building blocks, right? So you have generative UI to get the weather in San Francisco. We have front end tool calls and these are just suggestions to kind of have a guided

experience. We have set the theme to green. So now we have a green theme. Human in the loop. Please go to the

moon. Do we want to launch or abort? Let's launch. And what if we aborted?

Would we aborted? Um there's some shared state here. So this is a bunch of proverbs. Copa kit may be new, but it's

the biggest last beard. Okay, that's nice. Let's write a proverb about AI.

Who comes up with AI is like a compass. It guides but does not decide. That's uh that's deep.

I live my life by that one. Yeah. And we can update this shared state, right? So remove one random proverb.

Let's see what it chooses to remove. Okay. AI is this compass remains here and we can read this shared state. So

what are the proverbs? Yeah, it can see it. So So that's nice. the agent. This

is a showcase of some of these things. Uh from a code standpoint, this is extremely stren straightforward. So if I look here, let me zoom in a bit. We have the MRA weather agent and we give it the

weather tool to go fetch the thing. There's memory uh using SQL store and that's really all there is to it. And then on the front end side, there's examples of this is a front-end action.

So we just use this use copied action find the name of this this tool what arguments we're expecting and then a handler. So this is now going to get passed to the agent. Uh and by the way if folks are concerned about uh what can the agent execute every tool? If you want you can whitelist or allow list um uh certain tools and you know deny list

some other ones. Um we have the shared state which is kind of neat. This is all it takes. You have now state and set states with use cogent. Just like in React you have use state now you have

use co agent and all the state synization with streaming with uh you know delta computation. So you you make it really efficiently which is really important really quickly all is taken care of. Um this is to render backend tool calls. So this is weather tool call

and human in the loop. Finally, we have this render and wait for response where you have this respond call back and you you just call it when when the user did something. Uh and uh that's that's really all it takes. So this is really really easy to build these the demos we showed before. It's really on the order

of a day of work to build these things uh pretty quickly. Dude, it's great, dude. A lot of improvements, too. I love it. Yeah, a lot of improvements and a lot more coming, too. Yeah, we we've seen

all the stuff, you know, at different points, but it is it's cool to kind of look back and say like six months ago, what was it what it looked like compared to what it does today because it's I know it's six months ago. I don't think we I don't think we had anything yet and and our uh AGUI I think was still a still a Yeah, we launched AUI with

I remember you and I talking about the name because we went back and forth about what you were deciding to name it. Yeah, you you hated it, right? And I will say like I still do I actually but I think it landed like I was the the other name that I thought was you know arguably better in my opinion might not have landed because this one definitely did. So I'm sure like if

you're listening to this if you're tuning in watching this live have you heard of Agui? I'm guessing a lot of you have and maybe you didn't know. The nice thing about is it's just a protocol right so it's like yes of course C-Pilot Kit supports it but it's meant to be an open protocol that Yeah. That's right. Yeah. Front end framework should be able to

support and use. And if we could agree on the right standards, it makes it easier for the back end and the front end to communicate. Yeah, that's right. And there's uh you know, we make React and Angular clients, but there's now community clients with

Cotlin and Rust and uhnet coming soon from interesting sources and and all of uh there's there's good work uh being done in the space. Um um in addition there's a slack client in the talks of that and messaging based client. So you can put these things you know on a WhatsApp or uh you know RCS that's another type of protocol. Anyways, yeah, there's there's

the cool thing is like Monster is integrated with all of them like just by being compatible with AGUI which is cool. Like you can have Monster agents running in your glasses as one guy in our community demos. That's cool. He built a HUI smart glass client which is kind kind of neat. Yeah,

that shit's tight. Yeah. Yeah. Um, well, all right, guys. So, what's the best way for people to follow what

you're doing? Yeah, that's you. Uh, copilot kit on Twitter has it all.

All right. So, go follow copilot kit. Uh, go to tinkers, right?

Tinkers. Yeah. Shout out. Go to tinkers. Dev tool tinkers. Yeah.

Or we'll obviously see you in a couple days at TSAI. Excited to hang out there. And yeah. Yeah. Well, thanks for coming on the show. You're always welcome back if you have cool new stuff to to showcase when you launch new things.

Awesome. Yeah, we got we have uh 1 150 really coming soon which has a few things maybe I'll mention briefly and we we should definitely do another one of those when that comes out. It's got all new internals uh uh but no breaking changes. So all the existing interfaces, developer interfaces continue to work. There's new features like a lot of stateful features with

full support for self-hosting on anybody's clouds or onrem. Um, and then there's all new developer interfaces, too. So, a lot of the what I showed here, these interfaces, there's, you know, shiny new variants. Uh, but, uh, yeah. What beer are you drinking, Shane?

I don't know. They just brought me I told them to bring me some kind of cider. You know, it's two o'clock, you know, trying to keep it take it take it easy right now. But this is my seventh beer. I'm just kidding. I believe it. So, it's your

seventh. We'd hear the real thoughts. If you think he has no filter now, wait till we get a couple more beers. Wait till Wait till the seventh. How long is the stream going? Is it uh

is going? No, we'll be closing. We got We got We're going to hang out with the team. So, we're we got like 10 15 more minutes to close

out and and uh wrap some things up. But we appreciate you both coming on and yeah, we'll we'll talk to you again in a couple days. Thanks for having us. See you guys.

Yeah. See you guys. All right, now it's just us. All right,

back to just the two of us. All right, well that was cool. That was fun. Always good to see I always always like

to see live demos. If you're tuning in now, this is AI Agents Hour. I'm Shane.

This is Obby. We do this every week. We're not normally in a bar, but today we are. Uh, normally we record this at

or do this live at about noon Pacific time. Today we started a little late because I was traveling, but we talked a little bit about Master Agent Studio. We did some AI news. We talked with Julie and Tai from Copilot Kit. We have a few

uh more news topics we want to cover and then we're going to talk about the TSA icon comp. Yeah, I found myself in this bar. There are some uh yeah, there are some interesting comments we can share. So,

Joseph says, "If I have built an agent using Purcell's AI SDK, can I bolt on Maestra to help my existing agent more resilient to errors, persistent and better evals, and tracking for the agent?" Yes. Yes. So, typically the way we see people

do that is where you normally are calling like generator stream calls in AIDS, you just swap out a master agent for that. Use the master class, get the get the agent, call our generator stream, and it should still should just work. You know, there's obviously there's obviously a few things you can piece together there, but you can get memory, observability, you can run

evals, you get all you get all the stuff and it it's pretty easy. I mean, MRA is kind of built on a lot of different pieces of AIDS as well. So, you kind of we often have that where you run both together. So, you can kind of you can kind of do it

you can kind of piece out different parts of it. Like maybe you don't want to replace your entire application. You just want to swap in a master agent for this thing. see if you like it and then eventually you just end up swapping everything uh all those generated calls.

All right. And I don't know how to pron Hey, thanks for the new awesome update that I can share the dev server with my team and convince or let them decide why master is killer and AI agent framework race. I appreciate you. I like you, bro. Yeah, thank you.

All right, let's talk some more AI news. So, we didn't get to everything. So, we talked about Quinn. Let's talk about

cognition. You pull that up and then I'll pull up and we'll talk about the next one. Last week or felt like last week was the week of coding agent releases. Dude, it was last week.

Literally, it feels like it was a month ago. Last week because I've been using Composer One. This is not even what we're talking about. I've been using Composer One and it's tight. And then you got Devon over

here or sorry, Cognition over here uh introducing their own agent as well. You can just build your own model. That's what that's what they're kind of saying, right? So they're basically built their own uh agent model specific

for you know probably I imagine it's kind of tuned for the types of tasks that they're seeing, right? Yeah. So they partnered with Cerebras. It's available in Windsurf which still exists. What the is windfur dude? I

remember when we used to be wind surfers. We used to be wind surfers, dude. Why? But like

Yeah, it's cool though. But like if it's available in Windsurf, I feel like no one's going to use it. Yeah. I wonder is it only available in Windsurf though? I bet it's available, you know, you I'm sure you can use it in

other places, but I doubt it's in cursor or anything yet. But it is uh yeah, that's not the only model that was released. Along the same lines we had cursor introducing cursor 2.0 our first coding model and best way to code with

agents. It is. So this is it's called composer, right? Yep. Composer one. Dude, I've been using

that nonstop. I love it because it's not that smart, but it's so fast. it could do like if you know what you're doing, you just tell it like it does it like so quickly. You don't have

to wait, but if you ask it to think for itself kind of, it's kind of sucks from that perspective. So you're it's still just an agent in the sidebar. Yeah, it's just another model. Um like

we've been doing a lot like we're going to be releasing a 1.0 soon. So I'm like deleting a bunch of which is something that I don't want to do by hand, but I also want to do it kind of agentically because I don't want to destroy some I didn't mean to. And I just don't want to spend, you don't, you shouldn't spend money on

tasks like that. I'm just using composer because it's so fast because I'm, you know, I know what I'm doing, right? Um, but then I had to like try to build some new It failed so hard, dude. Like it was not good. Okay. So, so anecdotal evidence says

composer great. If you want to if you just need it to do things for you and you want to like guide it and just have it make make moves really quickly. If you need it to solve your problems for you, you might want to stick with 4.5. Yeah, stick with 4.5 or or cloud coat. You know what? Whatever.

But that's cool. So I'm a cursor fan. Stan fan sand. All of

them. So this was uh so let's talk about we've been talking about it's the year of you know we've been mentioning 2026 is the year of small models. We've also been mentioning it's the year of ancient memory indeed. So we have

the homies once again. Yes. So Mezero that they raised a bunch of money and you know they've had 14 million downloads a ton of GitHub stars.

They talk a little bit about, you know, you can watch the video. Of course, we're not going to watch it here, but there's a lot of excitement around agent memory and what that means. So, congrats to me zero. You know, come on the show.

Where you at? Taran Jeet's a legend, dude. That guy in our YC batch, everyone was like, "Yo, Taran Jeet, you know Taran Jeet?" I'm like, "Not yet." He's like, "That guy's a legend." And I met him. He

is a legend. He is a legend. All right. Yeah. Taran Jeet, open invitation. come on the show.

We'll get him on the show. Uh, but congrats to Mezzero. And last bit of news, I thought this one was pretty cool, mainly because All right. I thought this was pretty cool. Mainly because uh it's I've used a

lot of voice related uh models. Specifically, I used a lot of 11 Labs and this demo was pretty sick. We're not going to watch the whole thing, but I want to play it just because I think it uh it was a pretty cool demo. So, Cartisia announced Sonic 3, and

basically, you know, it's just a good it's a good model from from the demos. Again, haven't actually tested it, but it appears really pretty awesome. I don't know why we can't hear this. They're so great. Even Elon loved our

last model. I'm excited to show you our latest Sonic 3. Now, can you tell which one's the real Elon? I I I don't really have a business plan.

What's the business plan? The answer is neither. Both are Sonic 3.

Here's the thing. Even my voice was AI generated this entire time. That's amazing.

Sonic 3 is the best model for real-time conversation. Let's see it in action. Hi, this is Sarah calling back from Saffron Kitchen. Another robot call.

I get that a lot. So, were you looking to make a reservation? Yeah, tonight for 2 at 8.

8:00. Got it. Oh, wait. Actually, honey, tonight

Oh, 8:30 then. Actually, wait. Did you take walkings?

Uhhuh. But it gets pretty packed. Better to have the res. Got it. Yeah, it really works.

Sounds great. So to confirm, your phone number is crazy. So that's a demo again. That's

dope. That's the cherrypicked demo. Speaking Hindi and out there. Yeah, it's like multi multi- language,

you know. So overall though, Sonic 3 very promising, especially if you like familiar with 11 Labs and other real-time voice. uh you know OpenAI is real time voice.

Yeah. Uh they talk you know they do talk a little bit about some of the details of it. So you can check out that thread, learn a little bit more about it if you're building real-time voice apps. It's it's always promising to see new breakthroughs. I don't know if it's considered a

breakthrough, but it does seem better than a lot of the other demos that I've seen. So seems very realistic. Seems like the latency is pretty good. So if

the demo holds up, it is uh it's pretty awesome. All right. So, last let's talk a little bit about the TSA icon. So,

indeed, we're going to talk through and we're going to highlight the agenda. So, if you are there, you know what you're in for. If if you're going to show up, you know, come say hi. We'll be there. But if you're not and you, you know, you can of

course register and watch it virtually. We'll talk about what you can expect to see. And, you know, we don't really know exactly what's going to happen, but we'll give you the the rundown. All right. So, tsconf.ai.

Go there, sign up if you have not already. You can still get a virtual ticket. So, first we get the opening keynote. Sam, me and Obby are all going to chat.

The one thing you will notice, see what that is about. Yeah, we still are planning this, you know, very much like we we handle the planning for this show. Sometimes we throw it together. But one thing you'll notice that I actually really like about

the way we're running this conference and the way we even do guests on this show is like we bring on people for a very short amount of time and it's like give us the best stuff. Yeah. You don't have 45 minutes. This is not a 45minute keynote. You get like 15 to 20

minutes. Like give us the best stuff you got. Yeah. And pack it into something that's actually like action-packed something

people can like take away something rather than having like these really long drawn out. So it's where you just shill your product. Yeah. So it should feel like you're getting like a lot of value in a very short amount of time. That's the whole goal. So you can see like literally the

opening keynote is 15 minutes. So for three people. So I'm going to talk for like six seconds.

Same 30 seconds maybe. And then you got Paul. He's talking about you know lessons learned building an AI first browser automation framework. Interesting. Yeah. Of course you know Typescript it's

15 minutes long. You got Swix. So it says his 15 minutes to talk about the impossible triangle of LLM infra. Then

we have a you know a short agent tooling panel with Sean Pool and Brian Holt. So that's cool. Then you know a little break talked with David Kramer. He's got his 15 minutes to talk about

what if bugs fix themselves. So for that one then we have Luis from Replet talking about Replet's quest for autonomy. Then we have Nico from Purcell talking about building AI applications with AI SDK. Sick. We got lunch and Sam's gonna sign some

books. We got, you know, maybe maybe a new book. We'll see. You know, like there's some Easter eggs out there

already. Show up and yeah, you maybe seen it. We got a new book. We'll be, you know,

you'll be able to grab a copy. Then we got a whole bunch of lightning demos, which you saw Ty Ty was just here. He's in that list. Dan from Boppy,

Sergio from Arcade, Dustin Goose is on the loose. Yeah, Goose is gonna be on the loose on Thursday, Dustin from Postman, Simon from Assistant UI, Zach from Circuit and Chisel. And then we have a future observability panel with Mark from Langfuse and Aparna from Arise Off for MCP and agents with Michael Grwich from work OS.

That's going to be dope. Writing TypeScript with Codeex, Neil from OpenAI. And then we have the closing remarks which we're just gonna kind of you get me, Sam, and Obby again just wrapping things up. And then probably say peace.

Peace. We will definitely say peace. And then of course happy hour networking. And if you haven't seen it, there is an

afterparty. There is an afterparty. So that's the conference. It's going to

be exciting. You should The whole day wasn't enough to hang out with us. You could have an afterparty as well. Yes. thrown by our homies at Neon and Code Rabbit and whoever the you

know. Yeah. Yeah. There's a The funny thing is so the afterparty has twice as many

registrants as the conference in person conference. People just want to come hang out and have drinks. Yeah. I think there's like

Southern Pacific. Yeah. Obviously with the conference is sold out, but I think you know we just over 300 in the conference. We we let a

few extra people in, but I think there's like 500 registered for the afterparty. So, it's going to be a uh it's going to be an event. Come to the afterparty. Should we get uh some of the Yeah.

Yeah. All right. Well, while Obby uh maybe goes and recruits some MRO members to come say hello and what they're looking at. Uh

yeah. Yeah. We're gonna bring on some people from the Monster Team because why not? You know, this is our live stream. We're

gonna have fun. Daniel's gonna come on. Why not? Come on over. Come on. Step into the

office. I'm coming right up. Coming right up. All right. We got Daniel. Daniel, tell

people what you work on or what some of the things you've been working on in Master. Oh, I mean recently bunch of processor related things. Uh did a workshop last week on that. Yeah, tell us about the guardrails workshop. Um, it was uh it was it was

pretty pretty fun. Um, I would definitely go check it out uh on on the YouTube page of the X channel. Yeah, definitely. Oh, you looking for a way to drop that

right now? Oh, nice. Go sign up. Go reg and subscribe while you're there. We're trying to get our

YouTube count up. If you're watching on YouTube, make sure he says dress like he plays golf. Get over here, Grayson. Yeah, we're gonna All right. So, for the ring, guys, do not wiggle. You kind of look like a like a high

school science teacher. Let's get the science teacher in here. Send it up to Grayson, y'all. All right, that's that's Grayson. Grayson, tell people who haven't met you

what you're doing at MRA. Um, I'm helping keep customers unblocked. Um, inform the framework, what customers need in the real world, and make sure it all works. And what you

know you you've built some agents before. What's something you were doing before this? Uh yeah, building building agents in like the finance manufacturing space. So they were internal agents that helped the business run more effectively. You

know, automated invoice ingestion where it gets the invoice and matches it to a vendor and helps accounting reconcile. Helping source products to manufacture them. So, you know, I want to make sunglasses uh made in India and it would pull back uh manufacturers that could do that. All kinds of things like that.

Cool. Well, yeah, we're excited to have you. Cool. Thanks, guys. All right, science teacher. He dude, you're looking good. You're

looking good. Nick. Nick, come up here. This is Nick. He's He's the guy. Uh we

call him the king of rag. There's another one. What else? No, there I'll think of it. We we had some

names. The emissary of evals. The emissary of eval. The king of rag. Um yeah, Nick is

obvious. But also tell us a little bit about what what you've been working on recently. Lately, it's been a lot of uh working on storage, working on workflows, making those better. Um getting everything

ready for our 1.0 release. So hopefully everyone has a has a good time with the 1.0. All right. Next. Next up, we got to bring Eric in here.

If you've been uh playing around with our playground and looking at our observability, you've definitely seen some of the improvements. So, Eric, tell us a little bit about yourself and what you've been working on at MRA. Uh I started in MRA about three months ago. I've been building up a entire new

tracing system. Um before this, I worked at a data stacks, worked out the graph system there. Yeah. And so, yeah, if you've used the new AI tracing, you've seen and probably

very much appreciated from what I've heard from people, uh, Eric's work. So, Eric's the Yeah. So, thanks Eric.

Marvin, you can't you can't not get in here, but you're you're a fan favorite. You've been here before. This is Marvin.

Marvin, what's some of the stuff you've been working on recently? Oh, wow. Oh, wow. A lot of things. I'm mostly working on the play on the

studio. Sorry. It's not playground studio now. working on the local studio and on the cloud studio and recently we

have released a new cloud studio service if you have tried this one give us feedbacks would be nice yeah we talked about that earlier so if you've tried uh the master agent studio in cloud please let us know and we can't we need we need to get the next person on you know we are we got at least one more Tony was already on all right we got the man himself Oh.

All right. We got Sam. So, the three the three co-founders in one frame. Boom. All right.

Yeah. So, tell us a little bit. So, we teased we teased the new book. So, tell us so tell

us a just a tiny sliver of what they can get if they come to the TSA comp and get your new book on uh on Thursday. So, at TSA I comp we're going to be um we're going to be sharing the new book with everybody there. So the book this is the this is the first book uh it's in frame uh principles of building AI agents um you probably read it um it

kind of goes through the basic primitives um agents workflows rag evals tracing but um what what the second book is called patterns for building AI agents and that goes through sort of the more deep stuff like how do you do context engineering how do you like architect um an an agent so like we've on Shane. How many whiteboarding sessions do you think that like the three of us have done with people? 50

60 dozens upon dozens. Like it might not be it might not be hundred yet, but it's got to be close. It's got to be close. I mean, we you know, so like a lot of the sort of like multi how do I do

multi- aent? Should I have a you know supervisor? Like all all like how you think about that? How you think about

grouping agent like functionality together? Um and then all the different pieces of of context engineering. So like memory and and and and sort of like tuning that so that your application improves performance. Um but we close it

out with sort of like eval um and there's just a lot of patterns that have emerged over the summer um around things like um how do you uh like how do you pick and I'm not even like yes there's like the actual like how you you write an eval but a lot of it is like thinking through what are the metrics for success

of your agent like is false positive more important? What about false negatives? And then, you know, what are the sort of the failure modes that are driving that metric kind of like up or down? And how do you just sort of like

think about that? You know, principles is like really a guide to getting started. But what what patterns is it's a guide to the like to starting to like improve. Yeah, it's a chamber of secrets. Chamber of Secrets. It's a sequel. It's like how you're

starting to get your application uh higher accuracy and and towards production. Oh, funny thing about eval from the book though, we Eug John took a eval course and he learned nothing, which is good. That means we already knew what the we were doing. Yeah. All right. And Eugion isn't here, but

he'll be here. He'll be here soon. So, yeah. And there's a, you know, for all of you other MRA mastros that are not here, we

miss you. Yeah. And for everyone tuning in, uh, this is actually like a a tough thing because like a lot of our team members could come, but a lot of them couldn't come because of like visa concerns or like family um, important family events.

Um, and so like we're happy to get together, but the hardest part of a remote team gathering is always the people who couldn't make it. Yeah, absolutely. That's a good way to that's a good way to end it. Uh, we

appreciate you all for tuning in. you know, make sure you're following me and Obby and the rest of us. You know, we did get a new uh exhandle at MRA. So, yeah, MRA now. So, follows to Shane Shane like he

thanked it. So, follow us there on YouTube. Make sure you're you know, we do like the uh the subscribe numbers to keep going up. So, please uh go follow like and subscribe. Like and subscribe. Like and subscribe.

Subscribe. We'll see you at the Jamber Secrets. and and we hope to see you all at TS comp uh TSAI comp on Thursday. If you can't attend, register anyways. You can you

can watch it remotely. We'll be here on the live stream. You'll see us and a lot of other really cool speakers. And with

that, let's sign off. Peace. Peace. It see