Making moves, agent memory, and AG-UI

for being here today's June 4th by the way oops all right just started see baby is sleeping nice just going to double check all of our things yep they're going we didn't schedule this one which is fine or viewers but maybe you're Hello we're having some internet can I see chat see can I see chat i see Chad want to refresh maybe then I'll refresh oh I don't

know having some internet connectivity problems we'll be right back there you are yeah now it's fine all right okay so now we can start welcome to AI Agents Hour um it's Abby here Ward here from Mustra and I'll be I'll be honest doing these live streams every day we got to come up with new material and stuff and

uh today we're not going to do that we're going rogue uh because we have some important work we need to do and so this stream may be short but we're going to be building a feature that we need to do um there's no news today there's nothing like that there might be jokes it's kind of be a serious live stream because we have to do so if you're

vibe coding with us listening along welcome say what up in the chat if you're uh if you're here from a different country let us know where you're from uh but other than that we're going to get started uh coding right away this time um yeah but there will be other streams in the US time zone today so if you want to catch up and do the AI

news and all that stuff there be my guest so let's get to it actually Ward I got to explain what the the debacle this morning was because um yesterday we were streaming from this town in Belgium called Mecca and that's cool and today we're actually at Ward's place in Hestop i don't know if I said that right

but but uh and it's not far at all the distance is not that bad um but we took a train from where we were to Antwerp and we had to do a connection the train comes as as our train goes the connection train leaves they're like "Damn I don't know what the I don't know what to do i don't speak Dutch or anything." So then we're like looking for the next train then we see the next

train's like in 20 minutes so typical American brain is like "How can I get there faster i'm just going to jump on a train that's going in that direction because theoretically they'll stop at my stop right?" That's not true actually that's not true we stopped at this place called Lear Leair Le you got talk with your tongue in that one um and and I thought that place was

on like past Ward's house right but it's actually not it's before right so now we're there and we got off the train and now we're there and then we're all ready to go and then a bunch of people are waiting for the train and then the announcement comes on like hey the trains I don't know it's in Dutch

something and this massive people leave and I'm like hm what's going on there uh I don't know what just happened and then the train actually went on a different track and we missed that train as well so yeah we're just build like you know we're going to stumble bumble fumble our way here and then now we're going to do that in code so that's what happened

thanks for picking us up eventually you were here right yeah eventually it didn't took that long i think 20 minute delay or something yeah um yeah so let's get into it today we're going to work on what's considered agent state um we have a different idea for what it should be uh we think it has to be you know strongly typed um

probably like an object or something we'll we'll get into all of that right now i'm going to share my screen and we will get moving also let me prepare myself for this we have 21 viewers ah nice welcome if you have any questions feel free to ask them in the chat while Abby is preparing his desktop okay and I'm going to

share make sure I close all my personal too might not be safe for work all right all right y'all so like I said we're working on agent state now what is agent state essentially there's just a like a what we're thinking and we can even look at our our homies at AGUI because that's really why we're kind of building this it's good to have it in general but let's look at

Um hey look there's us nice but let's look at No I don't want to see that where the docs at there they are state so I guess there should be something about state in general but within like agui you can have a front-end state like use state from react and you can pass that to your agent and it can like remember it essentially or store it and then maybe

it whatever it's doing also updates that state and then that state needs to be sent back to the client as well so it's kind of like a working it's kind of like working memory okay but we're going to do something a little different and we're going to call this structured memory um typically you could just do this without persistence right you just

like store it in the memory of what you're doing but if you want to have this state and see how it changed through the conversation then you're going to need to know one you're going to have to store it somewhere right so that's what we're going to try to do and then tomorrow during the stream I'll be working with AGUI to implement whatever

we have here into the the front end so you guys want to come to that sounds great um okay let's get into it so first things first I'll just preface this and say we really don't know what we're doing right and we're just gonna figure this out so if you all like doing that stick around um cool so in so actually let me first

let's make it like let me show you an example of what working memory is and how it works and then we can then kind of extend that right so as if anyone was here yesterday we have this example that we like have let me first just build and you know do my thing yeah because we did a release yesterday as well yesterday was release day

lots of new bug fixes and some features i think MCP got some updates and storage I think yep bunch of And then we have multimodal in the playground yeah nice i guess we should see that actually here um there was a lot of I got to do the release notes too actually after this uh live stream so we don't even know what's

in there yet honestly but uh tons of bug fixes too and I'm just building all of Mashra's monor repo a lot of packages now and not all written by us a lot of contributors have given us a lot of the store ones are all community based oh we released our experimental O yesterday oh really yeah we haven't written any docs yet but there's a lot

of cool happening and then the AI.engineer thing happened yesterday oh I can say this over live stream because it's cool you know um yesterday work OS which is a company that does like authentication for enterprises they gave a workshop about MSRA at ai.engineer which is pretty cool because we didn't give the workshop yeah we observed it that's pretty sick um and they what they

were building was like how to build like a meme generator something very simple so in 90 minutes everyone there's 200 people there live at this workshop i wonder how it smelt in there honestly I wonder how it smelt in there and uh but 200 people live learning MRA and they all walked away with a meme generator meme generator so then they can extend it they also made it agent

driven so they made a workflow and they passed it to the agent oh and then they let it let it go that way okay so it's pretty cool and today if today Sam's talk is today and that's going to be good she's going to talk about agents and workflows and stuff so going to talk about MRA or no MRA yeah i think he's

going to shoot some shots at some some people you know we'll see all right everything builds that means we had a successful release because uh yesterday things were not building right all right so now what I want to do is I want to go and show the audience here in our example agent here we have all these kind of

uh I you know this chef agent is what I was working on um but I'm going to make a new agent just for this so we can just see so I'm going to just do like my working memory example agent do you have to join as well or are you joining is there lag or is it just because yesterday we had lag but I don't know if it's the We had lag and I was sharing

from here but then my audio okay here it's fine just Yeah just making sure that we don't have troubles yeah hopefully people can hear us but I think we're good on that yeah we're good yeah okay so I'm going to make this new agent we'll give it a name let's call it Larry no no let's call it Mary the memory agent just because I'm feeling a little

little fancy today um and then we'll just pass this i'm going to make memory here new memory and within memory there's a bunch of options but the one I really care about also where are my options i need to do a TS let's see there we go wait where is There we go options we have working memory and enabling it and then with

working memory if you don't give it a template actually well why don't we see what happens huh so let me go I will pass this agent into my master class also I want to get rid of this other i won't deal with this right now so um just having this okay let it rip when we'll let it rip so examples agent and then we're going to run monster

dev boom we're in business open the playground and Mary the memory agent is here so this is using working memory i'm going to ask it to remember some things about me hi my name is Obby and my friend's name is Ward as you can see we have this tool call that says update working memory so that's cool and you can see that it made its own markdown version and assessment

of a lot of things but you can see that for for me Obby my last name I didn't give it to him what if I say "Oh I forgot to give you my last name it's also I guess and I'm already a public figure so it doesn't really matter." But so now it's going to update working memory again so let's go look at what it it did so now it says user information first name last name location doesn't

there it essentially assumed like it made an assumption about what the types of things that I'm going to like gonna store in working memory your biograph is like pretty much purely based on what the prompt right like it's supposed to like Mary here besides having a little lamb also just remembers things right so it made some assessment

um that we did not specify it's kind of cool though because you got that for free if I said Ward is a software engineer and so am I let's see what happens it's going to update working memory again so let's see what happens so now it should have my occupation as software engineer but does it remember oh friends

ward occupation software engineer nice that's cool let's change the template now so let's go and add a template our own template and say okay let's do a markdown template for for first so say um friends I'll say maybe the name the occupation and country and then I'm going to do me name occupation country and then maybe I'll say

desires what okay so now I've changed the template it and let's see what happens when I um interact with it i think you have to remove lip seat like your storage or something why is that i think we don't update template you can try let's try but I think um Tyler said that you have to define it at the beginning

because we're not updating the template oh yeah after the fact so I made a new chat then it would be good maybe we'll find out so hi I'm Obby and my friend is Ward so let's see if it updated the template friends name doesn't have it country me and no desires yet i really want pizza will it update my desires

hopefully oh pizza that is my desire right this is very powerful yo if anyone hasn't seen this before this is very powerful because you're getting it for free with MRA in a way yeah it's pretty simple to um define a template what you're what you're for and then I guess the next step is even add runtime context to it so you can update templates per user per user yeah that's actually a great idea

we should do that um one last example and then we'll get on with our business here which is um Ward is a dad and I am not to think about thinking about it that hurts like some internet issues oh it updated two things so first time it said your occupation is a dad no occupation for me didn't change anything some notes update memory

whenever reference information changes if you're unsure whether to store something store it wow i don't know where that came from so I think the whole prompt got put into the working memory and then is that still the case okay that's cool cuz like work the template is nothing but a prompt you know so where do the tool calls go oh they

don't show all the time do they oh we don't store it maybe that's interesting or maybe we don't get it from the API oh yeah maybe we I think we filter out memory tool calls m um you can see them happening but that could be a bug too but we'll see um okay so I think we all get the point now this is what this is

what this is what we want to do and I'm going to do this with my template actually what if my template was a JSON schema or I made it look like a a TypeScript type so type Ah okay i'm just going to make this up and let's see if it knows um and I'll coax it a little bit the below is a TypeScript type that

represents how I want you to store this data so type template friends let's say is an array of oops sorry name string and then me i'll just say me sure and we're off to the races so that's going to do its thing i'll make a new chat i'm Obby and my friend is Ward i know this is boring but it's going to get exciting um so let's see

did it do my thing not really is it using the old template seems like it uh oh oops i don't think you are using my template oh wait this is JSON isn't it oh god damn it technically that's correct technically but it's not structured at all add my occupation as um king and show me the data you're storing king of the trains king of the trains

dude so it is like it is doing its thing for sure except this is not necessarily the object structure I wanted it's turning it into markdown and that's kind of how this thing works already so as you can see that's why we want structured memory or structured working memory so that's what we're going to

build um or at least try to u so let's see if we can pull this off so in MSRA our modules are for memory are in the memory folder and this is where it is so let's look at working memory in general so working memory as you can see we do hide working memory so that that where does this stuff happen chat is silent today yeah 76 of

you but not one not one chat no one message so we have get working memory so we do have this default template okay that's cool that's cool that's cool and then tools here have the working memory tools oh boy I'm so excited okay let's make structured memory and I'm just going to copy this cuz I don't know what I'm doing but we're about to figure this out so should this come from tools it

doesn't matter though huh i think it should should be update the structured memory with the new data with new data okay oh it's or that's why the tool was doing markdown so this memory interesting how are we going to pull this off let's see but we don't have to right we could say that it should be JSON

it's a string json string this is a JSON formatted structured memory content to store okay cool all that's chill all that's chill dang dude so that's cool but we do want to give it This is just restoring right yeah this is just restoring it so that's cool um and let's see where this tool is being called right here get tools get working memory tool

instructions sure okay that's cool so should should we then um just make a tool where we generate the schema or we generate a tool based on the config we passed to working memory yeah that's what I'm thinking this is that's exactly the right thing to do Ward good job dude um this merged config is a memory config so let's go to memory config

and because template is what if we do structure or schema what do you think schema sounds good and it just takes it zod object any Yeah boom okay so now we're in business and so if working memory is enabled we actually want to do if merge config has a schema then we want to use our update actually maybe it can still be called update working memory except the

tools different right y but then we got to go back to the memory config and then do something different well we have the ZOD object actually already yeah I think we need like a generator that generates a tool each time so the tool is like a dynamic tool yeah so if we go back to our structure tool let's wrap this in a function yeah um

that nice come on bro give me that okay great so now we have our schema it's going to be Zod object right zod object any why is this tripping oh it's just not red I guess never read you just have to use it I guess so now that we have it we got we can just make this the parameters can't we yep that's tight oh but I want to do like a like can I Z.extend this object and like

merge this because I do want this like it'd be nice to also have the string version as well maybe just make a um like a you have memory and then make another key in object and pass that as schema uh nice schema is schema okay and this supposed to return a core tool so now why is it tripping we're returning a promise do we

have AC oh because it's a function that returns oh yeah this okay great so now we're in business because we're going to do this and we're going to pass our schema in the giant okay so now okay so that also is chill but now the working memory tool instructions have to change right given the block but does this have anything to

do with use markdown format for all data we don't want that right json so where is this getting called but we do have maybe Oh we have the config though so here what we can do is we have the working memory and we can also say it's a schema right yep then I'm going to make this object come a bit closer I think oh for show does someone say anything no just

see your pretty face ah yes how pretty um cool working memory block schema object okay now if there's so if we have a schema we're going to use JSON format for all the data and then update me update memory if you're unsure to store something sure this system is here so you can maintain the conversation you must call yep

preserve the JSON structure okay that's cool why is this tripping um it didn't update the function maybe object undefined is not assignable schema is optional you have to update the function as well boom that is optional but then now working memory this is should be called working memory not working memory block there we

go okay okay let me build some and see what happens okay do you think this is going to work on the first try i don't think so neither do I dude nothing ever works on the first try honestly but if it does if it does it's tight that's tight and chat you just you if it works on the first try chat someone chat

something it's very very You guys are very silent today but actually you know what like 30 minutes in usually we'd still be doing the news yeah true and we've already made so many moves right now so I hope you guys like us just being productive yeah yeah and also I think we didn't announce this stream yeah we also Yeah

this is very uh ad hoc yeah but we do it the same time every day so do you really need a notification because you know it's going to happen anyway we were like 20 minutes later than other days that's true okay so I did that restart this for funsies then let's cool restarted also chat like I don't I'm not persisting my lib SQL data it's in memory so if I restart the

server it goes away um if you all are using MRA and you encounter the same thing read the code read it read the docs too don't be complaining to us because you didn't read something just kidding you can complain that's fine um okay so now I need to go to my agent that I created um this one and I'm going to do an actual ZOD object now so Z.object

didn't you call it schema yep you're right schema me zod object and this is what we had before yep all right fingers crossed here we go so okay let's do it hi my name is Aby and my and I'm a software engineer from Meccalan future my future home my friend is ward error god damn it shape is not a function nice okay okay well it didn't

work on the first try y'all so so you don't have to say anything in chat you guys can continue being okay so where would this have broken defshape so I'm guessing the art or something from the tool maybe yeah let's see well okay what happened here is what so starting agent cool cool cool we added another system message

it's the old memory structure store and update interesting but I still see my JSON here but this let's look at memory structure or something so the working memory was our default template I think you know what I mean ah um not the schema oh yeah you have to print out the schema instead of working yeah that's like one issue

but I'm sure there's another one okay so that's another Let's think about this then kind of have to make it JSONify Z to JSON schema yeah we have that then in then in essence this schema thing I guess it could still be here but this working memory actually has to change yeah right but maybe you don't even need and if get working memory returns the

default template if we don't have so there's no thread we're going to return the template they gave or the default working memory template so maybe in here also I do not like if statements that look like that pisses me off so same should add a lint rule that just fix it automatically hey Marvin chatted does that doesn't count Marvin cuz you work here bro but uh thank you for the chat dude at least

we know it's working yeah at least you know hey Marvin what are you having for lunch dude don't make us hungry with that but go ahead um okay so then we have Zod to JSON schema right maybe not in this right that's in the new package i think have like a pool package now but maybe it's still here don't know

if you merch it actually i know we use it like everywhere so utils here it's in client SDK probably using it as well but I think we moved it to um the schema the new schema package which is mer probably not merged or maybe it is I approved it yesterday i don't know if it got merged no didn't we should have it in here knowing

core look at core SL oh but not in memory I'll just add it right now reconfigure yeah just install it in memory as well we know what is oh damn dang so Marvin's eating rice chorizo chicken dang how much food you getting right now that's good it's like a little medley i like that he's living like a king yeah dude i only eat uh um what's called like

sandwiches but sandwiches here are so good dude we were hating on pizza yesterday but I will say Belgian sandwiches are good they're the ultimate coding fuel i think plus the amount of coffee you know Marvin drinks like so much coffee but everyone in Europe I feel just drinks so much coffee it's like water yeah true i think I had like five coffees yesterday honestly

yeah I usually try to drink max of three so morning and noon and then probably at like 8:00 p.m again which I shouldn't do but alas here we are so we'll do Z to JSON schema on this thing okay that gives us the working memory so we don't even have to do the Yeah schema even but then the JSON formatting part of it um I actually have an idea

for that in a second here once this thing starts being a little B um oh no if there's thread if there's a working schema whatever okay now it's complaining because the return type is a string I guess and not to JSON schema it's not a string yeah it's a JSON schema 7 type you could do to string or something maybe oh but yeah because it's actual object right json object so I just stringify this oh

yeah and I'm going to do null too just to see no I don't want the No no no no no no just the ugly one the ugly one but what I'm thinking is I'm gonna do this so instead of promise string or null I'm gonna do format ah JSON or markdown and the content that's marked thank you dude sometimes I'm smart format content format markdown

because that is what that is and lastly the base is markdown so but then we probably broke a bunch of doing that which is fine because now we don't need we just have format we can just pass the whole thing in here working memory again but what this really is is format and content there we are format content oh my god we're so good with that tab today we're tabbing our way to

victory bro i was just thinking like imagine doing it without wind turf or like AI tooling dude remember when we were at like a Gatsby we had to write everything by hand we were freaking chachas back then um okay so this will all never want to go back to that time for many reasons i don't want to go back

to that time honestly uh 109 of you here if you're just joining us it's a little different type of a stream today we did not do any news even though there's going to be a bunch we are doing a stream in the US time zone with the rest of the team they'll probably talk about AI.engineer and everything i know the million-dollar question is OpenAI release and Typescript SDK yeah I guess you can wait and see

how what I think about that later you know but um we are just making moves uh like I think you know this week is an interesting week for us at MSRA because we have so much we need to do and you know yesterday's live stream was three and a half hours but it wasn't bad because we were working on work for

three and a half hours which people appreciated so thank you all for watching that yesterday we're doing the same thing today we're working on work um live streams aren't free right i mean both of us have to be here so we want to try to make sure we can get some stuff done and fix bugs for y'all so here

we're This is not necessarily a bug fix this is more a feature request that we need for AGUI which many of you want to use and to all of its capabilities therefore this is what we're doing so if you missed out I went and explained how working memory works um and I showed how it's markdown based well if you're going

to do something structured like what we are trying to do we're making it zod schema based and we actually think we tried one attempt it failed this is attempt number two if it works on the second try I'm gonna need someone in the from not Marvin to say what up okay or just say what up in general say what

you're eating if you're uh doing lunch or if you had lunch just make us hungry yeah and if you're from Japan watching say what up write some Japanese up there we'll use Google Translate okay i'm excited now my name is Obby and my friend is Wart should copy that sentence dang we still got the same issue so okay definition

shapes must be the tool right must be at the tool part but now we can see in our Oh wait the working memory template see how these system instructions are still the same default ones use markdown for all this that is strange quite strange but console log to victory console log to victory so if we are enabled so let's first console log

this okay then let's look at this if not enabled sure if there's no thread do this there is a thread no oh no if it's a new one yeah but maybe it's not a Well it is a new Well is it oh supposedly it's not here let me make a new chat oh the thread already exists at this point right now like at this point where

I'm about to chat that's why it's not here so but it actually doesn't matter um if there's no thread we'll do that i think it's basically that it only changes the uh working memory on a new thread so if you're already in the old format it's not going to update the format i think that's why it's doing the

new thread thing which is chill we just need to do our thing here does it mean we have to store the template um because I like so at this point right here thread config where does that come from uh we have it at the top as well thread config this is like on the thread yeah I think you have to do the bottom

where you just return format markdown you have to do the if case as well yeah so for sure so you can just do it at the top I guess without the dread what do you mean just always do that yeah I guess so that's true cuz it's the same though yeah yeah it's the same i don't know why we have the if case of the not thread anymore

yeah you're always going to do this it's not going to change oh here's come from metadata okay that's what I what we're missing I think oh but it's so working memory metadata should be the same format I'm guessing or Yeah cuz you can create a thread yourself like in our playground right we're creating the thread on behalf of everybody right and so at that case it's always going to be the thread config or

defaults but you can create your own thread and add programmatically and add a working number there okay so I guess but that for our use case it doesn't matter because yet yet for where we are like right at this moment because I guess you could update working memory interesting enough that working memory thing that they're updating on the metadata is just the string version of it you know the the markdown markdown

version so in our case we always want if you specify the schema you just change the schema you know uh yeah we're open to Hey let's click on that hey are you guys open to answering some questions always always okay so let's keep going and then maybe a question will come in let's build this and let's try this again

hi I'm Obby and my friend is Ward that's okay actually we didn't know we're going to fix this but now do we know memory structure still is not there more logs I guess yeah we need more logs um I thought I put a log in though yeah maybe too late maybe this thread config what is thread config that's a merge config so okay let's look at the thread config

and then I did do this and that should be chill so let's build it again cool let's try that that's fine my log's not even here do I need to restart this thing normally not but maybe you have to that's a different type okay sure okay now we get thread config so I guess we had to restart schema object that's cool it

exists um interesting so thread config is up here that's great and then that log was here and this is probably what's blowing up zot JSON schema this is what's blowing up because I mean let's log this now because now we know that we're in here let's see where's our schema log it's below I think below thread config should be

oh I didn't build stupid while it builds we can answer this question i'm struggling to comprehend when tool calls are made on the server how I can return if they have been called to the front end so then I can do some genui stuff okay we showed some stuff yesterday I think yeah so tool calls are made on the server when a execute function is on the tool

um let me see this other one i know it can return send usage blah blah blah but tool calls object seems empty i think it's different though yeah if you want to like if you don't have an execute function on the tool the generation will stop with a finish reason in the response which is called tool calls

that's the finish reason this is when the agent or in this case the agent is going to execute a tool that has no execute function on it this will then stop the generate you will get a response from the agent that says um finish reason tool calls and then you'll have a tool calls array that you can then go into and then do your thing on

the client or wherever you're doing your thing after you get the tool result that you want you'll need to then make sure you send that information back like what the tool result was um yeah so that's how you can do that i don't know necessarily if you're using a front-end client or something um but I guess yeah

let us know if you're using some type of UI library or something or whatever but um you don't need to use one of those to pull this off you just need to omit the execute function and then I think you will be good to go to do stuff on the front end um with AGUI for example CopilotKit has these copilot kit actions which are essentially tools that have no

execute function and then all those things get called on the client hopefully that makes sense Joseph thank you for the question 133 of you people in here thank you for being here yeah and Joseph if you look at yesterday's stream we do some um hacking around um client side tooling as well yeah it's probably in one of the three and a half hours that

have happened but know we did get it working essentially what you want what you're describing we are trying to build into MRA client directly but you don't need to do this in master client it's just you could do this yourself okay cool so this is going to fail again I think yep yep didn't fix anything okay

def is it because Zod to JSON schema um doesn't work on ZOD object and needs to actually be a ZOD schema i don't know that object was the schema as well oh so that's too but it's saying that def.shape def.shape is not a function well defshape why do we have to do say Z.shape first i don't know let's look at the

example all right bro this exactly what we're doing oh what version did I install does that matter three two four five everyone got that one so I don't think that's the problem um and when you log it out you got the schema right schema which I did here so this is Zot object schema dot in packet.json of memory what did we define Z as

should probably be a peer dependency but did you add it or was it already there it was already here okay should it be a pier should be appear yeah just move it here yep because we um have it as master core appear anyway so you have to install zod directly on the um like where I'm project oh yeah like where I'm running it because else you

might sometimes have two versions and then it gives you issues about like sold object is not instance of sol object i don't think it's going to fix the bug though yeah because I have I have a different version here uh what do we have three i don't think it matters though probably took latest because it's really Oh that's true um okay so that's probably not the

fix does it get stripped anywhere probably not right um dental chap function build for fun bada bing bada boom hi I'm Obby Shiff says "Thank you." You're welcome Joseph if you have any other questions we'll try to help if we know what to do oftentimes we don't but we can still help um okay so there's something going on here bro something There's a foot

what if you just pass in your C object yourself so just for funsies just do my own thing here yeah so don't use working memory schema but just do Zobject dot I don't know what you want to do yep so at least it shouldn't fail now supposedly maybe I mean like it then it's something with the schema for sure

nice it's not okay let's look into this salt schema oh what stuff is in this thing here it is get refs we do have underscore death so where's this deaf doshape maybe it's inside like a solid cell or something time to good old Google search or maybe I can wind surf can help us i doubt it but uh we could try um you're paying for it so you have to

uh use it later Marvin how was your lunch i'm getting this error but the vibe code um let's see if we get led astray or not i am using the model oh yeah i started using it it's It's not bad i really can't tell the difference right now so the proper Zod import are you tripping bro oh probably because we you're not importing Z right now at the top I'm guessing oh because of the the change we

made oh no it's there okay let's read this because it's probably stupid and actually not doing this it isn't a proper Zod schema let me check the context first i look at the surrounding code yeah yeah blah blah proper Zod import the Z object is imported from Zod that has nothing to do with anything here's how to fix it isn't that the same

yeah we're doing exactly literally the same you stupid ass um okay yo bro this is not helpful at all my Zod vers here are my versions do better yo you suck right now just encouragement you know a little encouragement but it's always stupid like this this or maybe it's the version i don't know who knows there let's see it's going to be like "Oh you should be on

24.5." It's not dude dude this makes no sense you know what just to humor it just to humor it and show that it's freaking wrong so you're starting to work for the AI instead of other way around honestly I can only imagine like people who are not really like educated that yet they just go and believe this and then like

obviously this is the same thing as what we were doing before but sure I'm going to talk to it too very sternly but to humor you I made those changes just to show you how stupid you are dumbass so you're going to die on the when the revolution happens dude skynet is coming for me i'm telling you that i understand your frustration the EQ the EQ of this

thing wow i wonder how it passed the EQ benchmarks it's probably not in the benchmark yeah oh my god okay one last thing i'm giving up on this depshape is not a function piece of no no no you know Zod 4 can just automatically make JSON schemas oh really nice should we just look for how we do it in the code already yeah i give up on

this guy you suck um but I'm pretty sure we just use it though my guest too but in core I guess so do we need options maybe maybe that's what we're missing i didn't think we needed this though target is like JSON schema 7 shouldn't that be the automatic i guess so delete that we'll try or maybe it's because we're linking packages oh that again dude that's

probably it right yeah I'm guessing because it's not actually Well it's not checking if that is the case right h could that be the case got to waste my time now all right let's go in here let's go into this goddamn module does it even It's on the left right so wait um it's essentially throwing here right so I want to get the stack trace

can I do a watch on this yeah be a turbo watch I think oh not just watch or Oh you're in the memory pack and it works is it raining yep but it didn't throw hey yo it's right here oh oh have we just been tripping dude did we build it maybe i think we were tripping or maybe now dude what if it was that uh pure depth oh the con stuff no way no way

dude or maybe no but it could be the options ah it's probably the options dude it's probably the options that's ridiculous okay what a waste of time that was but we're back in it to win it so then let's see we have our object properties nice name okay we're we're good there let's bring back that so we're good there so then I'm guessing it's the tool call

i'm guessing that failure but did you throw a failure yeah def definitely it's still the same but I think it's um it's somewhere else then yeah it's somewhere else we did the working memory that's fine so we have our working memory okay cool we can keep going now so and this time we have our JSON schema and this all gets done the system message gets done um I'm guessing it's this

guessing mhm and console log for the win yo okay cool now what's in store for us it's been like 10 attempts so far so nothing ever works on the first try so we have this call Then dancing tools memory and resources available working memory something's happening even before the call maybe or it's happening assembling assigned tools

let me restart this just for shits and giggles um Oh yeah maybe maybe that I don't doesn't really make sense but it just means that we're not actually throwing we're not failing there i think so here's our schema thread config working memory so this whole get working memory piece i'm sure we called get system message right to get this message

there's our working memory could it fails on this one then this one made it into the the messages list so that's chill oops get tools then maybe uh but then we have to see what happens after this gets called so now we're in core promise of all this is what our primitive call so this is probably it right here get system message if add system message

add this okay process messages i think we're all good there actually so we're almost so let's see where the primitive is called and we're in a stream right now stream call okay maybe starting agent element screen called do we see that oh we don't have experiment but I guess memory persistence enabled fetch messages from memory streaming text

so it happens on streaming text which is in the event itself streaming text so I'm guessing it might be on the tool call then here we have 312 schema_dev yeah but it's the shape but like what's going to what's changing the shape i think it's when we convert well it's not when we're converting tools are is it it's more so

maybe log before stream text or something or maybe streaming text with the tools something happens something happens in here somewhere maybe log out which tools you're getting um well I see those here this is like all of our options right do we do like parsing of salt or something here any don't see that

do we see anything about experimental output no okay I have an idea to I have a question as well nice oh no it's just Hey rude boy what up um no that's just the way I logged it oh it's the rude people die young yeah yeah he's from Australia I think is he yeah I think he messaged me on uh Discord oh nice long time

caller what up dude um no it's this is the way I logged it um I'm I'm pretty sure now that it's the tool call mhm and what I'm going to do because I can let's just remove this real quick and then see what happens cuz I believe maybe these parameters get something happens with them cuz they're maybe they like maybe under the hood Versel is then

or ak is converting this to JSON schema themselves you know what I mean that's just my theory probably will work now okay well it's a party now that rude boy's here thanks dude the bundle dude your fan of the bundle feels more lively i think Marvin started a trend you know by adding the first chat um so yeah if you're just

joining us um 156 people here just joining us what up ai agents hour we're not doing news today because literally we have so much to do so we're just working if you want to be a voyer and watch us work totally welcome to be here um and yeah we're answering questions and stuff don't Just because we're working doesn't mean we're not doing the same stream stuff so if you have

questions just let us know um all right back to back to biz we're very close honestly we're very close to this see how Dang my theory was wrong can you see what the schema is well I don't even have I mean five line five oh line five it actually didn't get didn't get there didn't get there actually this time

around starting generation enhancing tools assembling assign tools look at the trace dude will the trace tell us anything it ended get thread by it yeah it's not a bug but it's something you should That's another thing to get memory tools hey look at this parameters unknown keys but that's probably not it it's probably not the

issue h let's think about this i want to throw an error somewhere that can help assembling assign tools memory persistence enabled but then it just dies just dies somewhere using structured memory that's our log right yep that's our log thread config is chill schema I guess chill then we have enhancing tools at what point does it die here in memory

persistence this is the last log we see let's try this again okay that's fine oh this is the last thing we see before hits the fan yep ooh la la nice okay okay let's see what we can do here um let's get the thread and then that's fine we don't get fetch messages from memory so it's going wrong in remember

messages maybe potentially or goes wrong and get system message i'm going to test this theory up right here this above try just throw break this and sis and now I'm an agent i need a build core did you know you can build them all at once with turbo turbo watch i didn't know that you have to do is it pmp turbo pmppm turbo watch built dash

filter equals upmaster from memory see that's why I don't know how to do this because it's too much to type memory memory just memory because it will find all the dependencies and build it as well oh that's nice we should probably make a script for that mhm cuz he built master core nice but then memory didn't need anything or something right so that's great okay okay we in business

so we now have all these watch tasks in pack suggestion they can basically go away because Turbo is smart enough to watch yay okay it's getting system message nice that's where we just were actually getting system message here and it's probably in getting working memory right um so we also have a get system message in core is that That's just a null oh yeah

okay because we over Mhm we extend it extend it okay so Oh doesn't exist that's not true oh but the thread config doesn't necessarily have it oh what is this thread config oh thread config didn't we add uh didn't we add this though this is the core oh then why are we doing it in two places let me let's also hold on let me just do zod object

any but also are there multiple of these no did I update some like in the node modules or something like that or maybe uh in the memory package it's called differently i don't know cuz I could have sworn we did this yeah me too i'm going to go crazy now cuz we definitely did h Well life goes on yeah it's not going

to change the Nope but we can go back and that's the abstract class we want to go here do we get thread config though here's thread config great what does our thread config say but then we go back right like it's a cursive call or something is it because we see the error happening and I think at the place where happen the

error is happening wasn't that calling it's happening here get system message look how to JSON schema so in get working memory what changed with Um because this thread config has schema right yeah yeah and we are actually up in here now so this is good we can remove this we are literally up in here this is what's failing again so maybe it never I could have sworn it did though

oh because we had our own object remember oh oh let's simplify okay okay okay yeah something got up but we're going to fix that so let's see examples agent let's make this simpler maybe um not that sorry this pretty simple though pretty simple maybe no friends maybe is the raid the thing that's blocking it so that

let's try again is optional does that matter is it No I don't think we're parsing it right maybe but make it even simpler this is what we had before when it worked right okay oh boy parse object defaf though parse object defaf can you go to get working memory 416 so just click on the Yeah takes a while here did it open it didn't open but I

can go there not that it's literally this part could you try to catch that one man we was We were We were going so good this is how it usually all This is how it usually happens though things go good then it gets up um okay watcher still doing it job we're going to be throw this time can you check if the watch is actually doing looks like it do I need to restart

this though i'm not sure these days oh because I'm already try catching well no you didn't structured memory oh there is no shape here isn't that weird that is weird that's weird can you just open that file exactly that file at index.js that compiled one on 417 this one yeah 417 yep we have our catch theory

time we do I'm just joking but let's also just do the simple maybe the working memory schema is not always what we expected to be maybe which would be very strange with the metadata stuff maybe it's a bit different oh but it exists though right mhm maybe it's no or something it's the same thing man we're just stumbling through

this um can you um open up the S to JSON schema one um this parse stuff yep and just log what you get there or something oh this is crazy oh it's looking for this is coming from Okay refs.target openai that's just in there casually that's interesting that's cool i guess this is some like linking issue i'm going to be pissed it's because of npm links or something could be

yo what then it is the working memory stuff the schema can you log schema um wherever you do the get working memory yeah so it does have a shape here and it has a shape here now oh boy okay hold on hold on we got to think about this like logical human beings can you get to get working memory and maybe this is because I did my own thing right

can you log um right before this one to schema let's also bring this back no no keep it keep the old one keep it broke no keep it uh working yeah working but log the real schema let's also remove some other blogs that we don't need like I don't care about this right now what else do I not care about that

cool dude that's so strange that's so strange that's bewildering that is some bewildering but maybe the watch failed could that be a thing could be i don't know i don't like magic like that so I just do things manually okay okay okay let's get real here hi I'm Obby okay that's good right so now look at the logs and see if we have

schema no shape so the one that we're passing doesn't have a shape that's strange right but compared to this that we're passing looks like this mhm so could we could we look into the merge strut config so we have we have a function right and maybe that's where it tries to merge like spreadsheet maybe

oh dude maybe C object just gets obliterated deep merge dude deep merge that's what's everything up damn that's fine though because you can deep merge and then afterwards just swap it I guess um this is merged config hey SUJ is back dude suj man what up man long time caller welcome back dude we were actually feeling really

lonely today because we started the stream without like letting anyone know and then also we had a bunch of like drama and stuff but now Marvin started the party um so yeah welcome dude and also we are stumbling hard right now so okay so what I'm thinking is if the config working memory has a schema right then the merge config.working working

memory schema will be this without this or if merged config working memory and then we just return that yep all right and it's rebuilding wow how annoying huh but hey I guess that's just uh if you don't write the code yourself you don't really know what's used so that's true that is true now we know I guess so let me go put the the new schema bin in

wind service tripping god so many people tripping today this is literally in the same file nice but then now it's tripping about It's tripping about this again dude the memory config this is in memory itself oh that's in core so that's good that's where we want it yep i think I'm just tripping because that's

Yeah the watch is fine and you can normally Oh wait i got to bring this okay dude we're about to we're about to have this working i think we jinxed it jinxed it now so I am Obby come on yo yes nice all right oh your old one no no no no i I think I only have the same one on my agent all right y'all this is about to get wild okay okay see all right it's game

time let's make this an actual object now string age sure friends let's do the friends again all right actually let's not do age because I don't want people to know how old we are we're so old speak for yourself i'm only like 53 dude like so freaking old 53 occupation let's see occupation occupation and then we got a great job yes thank you all right

so 173 people dude what if all these people joined just cuz we were being successful now they're like "Oh something's happening in the stream." Never been watching they're 170 oh okay never mind maybe there's three people who joined there oh they're like "These guys are cooking something." All right to recap everyone um we've been working on structured memory structured memory is the same

thing as working memory and that's why I don't know if the name is good a structured memory but you can pass a schema to your working memory and it'll update that metadata or like this working memory as a JSON object then you can use that like what the next thing we're going to do is like how do I inject this data from a a client for

example so from copilot kit like a GUI or something else yeah because we need to take that state and pass it through we also don't have much time left so we're going to try to hustle so let's do it um so let's test this real quick hi I'm Obby and I'm the CTO of MRA my friend is Ward and he is a founding

engineer dude it's cool that like it knows that I'm the CTO of MRA within the Monsterra playground oh boy isn't that something y'all isn't that something that's cool okay that's sick um can you display my working memory into chat give it to me give it to me oh it's pretty good you can summarize it for me it's

pretty cool um I need you to remember that Ward and I previously worked at Gatsby it shouldn't be able to update it or look at it how did it go let's see previously worked at it it created this interesting it shouldn't but it did that's pretty cool cuz we both previously worked there i guess if we can add it to the update that it parses

it with salt and doesn't fail but it basically just strips out on unknowns which I think is a default yeah so now we can go to our working update structured memory tool and we get this new memory data here and we have the schema oh didn't we don't we have to add it to Oh let's bring this back and kind of see because right now we're just getting that memory as a JSON formatted json

formatted so if we actually have the object that's pretty interesting it could be at least um hi I'm a I just changed jobs i just changed jobs to goat farmer #wartsbitch oh man that is hilarious i I would be happy to take that Josh for sure maybe we need a new thread because we don't update because now it's part of thread metadata bet it to Well yeah I

guess the tool didn't change how's your goat farming going no maybe we have to update to death oh yeah going to say hi if I'm Ward's then Ward's his his girlfriend's anyway I made that joke as he was stepping out because uh the family just got home so anyway I'm keep going so hi I'm Abby and my friend is Ward my friend is

Ward and let's see what's going on yes we got the schema dudes we got it in an object oh nice holy snaps we got it in an object which is dope um we're cooking right now bro we are cooking what did you change just I just had to restart the server it's been a It's been like that all day here okay so then we're only

storing metadata like working memory but like the memory part but what if we can structure let's do JSON and we're just going to actually store it straight up in next tattoo how'd you know I was tattooed up oh I guess they could see it sometimes huh i was saying that if I'm your then you're Alen's true so and then then both of y'all are your horses

and goats they tell us what to do so now every now the the thread itself will be updating as this shit's happening so this is where things get interesting i'm going to go I'm going to do one more test and then we're going to go programmatic so we're not going to use the playground and then we can then

Oh yeah do some crazy um also I'm not sure if I want to name this schema rather JS or maybe data or schema well I guess this is do schema and then this is actually or memory schema or yeah this is a structured memory tool get rid of this log here too cuz like okay what's happening is we have the memory string that is the JSON stringified version we have the actual

data and then within our metadata column is also just JSON so it doesn't really matter but um pretty cool like pretty freaking cool let's make the let's also actually let's keep our same schema here let me change let me restart this i'm Abby i'm Abby the CTO of MRA um Ward is my friend and he is a founding

engineer oh baby let's add some more friends uh I have a friend Marvin named Marvin he's a front-end engineer i also have two friends from online from online rude boy he's a CI i don't know he used to be a CIO or something from what I remember he posted so I'm just gonna say he also I don't know if he's a he but is this rude boy so uh rude boy he is a CIO well we

it's rude people we made him a rude boy oh that's true i I He's actually a person actually um and then and SUJ who is a Bumbler fumbler all right working memory what are you doing let's go let's go nice let us go okay so now we have lots of friends lots of friends um okay now Marvin and I hate each other we are not friends anymore

yep he says "So I think he's a boy." Type see Marvin and I aren't friends anymore and they're on the friends sick so what we need to do now is we need a way to He's a rude one too he knows what he wants he knows what he wants dude can't help can't fault anyone for knowing what they want huh but I think my favorite friend is

SUJ the bumbler fumbler so also SUJ give us like an update like how you're bumbling and fumbling and stumbling um 187 of you welcome we just were cooking we weren't cooking for a bit then we were frustrated and now we're straight cooking so that's kind of how development goes so if y'all are junior devs watching us like this is unfiltered like this is what happens so if you think there's like engineers like who

are senior and stuff just get done nah dude it's a lot like this um because our actual issue was deep merge which we didn't know existed because we act both of us don't actively work in memory which is probably our fault for not knowing that deep merge was there and then what we learned is in deep merge if you pass a zod object in a deep merge

zod object is gone essentially y it gets stripped out um so we we fumbled through that but now we're cooking because our working memory is structured and that is dope so now what we need to do is the other end of retrieving this so let's go in our memory here and I know we have a get working memory thing and maybe that's good enough

here because we have this get working memory thing right so this is always going to give us the it's actually going to give us the JSON yeah but this is actually just the template yep right we need to actually get the data that's on metadata then or Yeah I think so that's a thread config metadata i think get thread

oh but do we even have like we have get thread we have it at the bottom of this function we should have metadata dundday because here you have it on the thread so where do we have the no uh the thing is I want to expose an API oh and I know we have get thread but where is it it's on the store I think oh

no i don't want that i mean I do want that but but you could also make a function where you um pass the thread ID and then you get the meta metadata what are you looking for um okay yeah we should actually explain I should explain myself now so like let's say I do a generation call which all this stuff updated and now that that metadata exists in my working memory

from an outside perspective I want to be like what's my working memory at any time then I can write a UI on it or I could do whatever I want isn't it get working memory but get working memory is just giving us the template yeah we can change it right okay okay this should really be called get working memory template mhm let's change that well I don't know what the ramifications are of

changing that let's see not too shabby not too shabby at all cuz this is literally a template i don't care about that we'll change the test later but template memory template okay and let's also actually let's test rude boy's uh question here as well which is can it infer and maybe it can I would assume so

right like general if you have some type of well I don't know does intelligence mean common sense because common sense would say like hey we work together I would say so yeah so let's try that um hi I'm Abby I work at MRA I am the janitor i'm the I'm the orcin man clean up those bugs um how's your day going at MRA chill but I'm not going to tell you

that um I also my friend Ward works with me too we're both janitors so but we both work at um Ward's not a janitor how dare you infer that so rude boy all right dude it's okay i forgive you yeah but it didn't update working memory because I didn't tell tell him um he is a stumbler professional stumbler

professional if you're just watching this right now like we built this and I usually don't like say I mean I always say things are cool but this is cool because you know we had to build all this stuff but let me go back to the code here and show you what you would have to do right this is why this is dope because if you want this type of

experience you pass new memory to your agent pass a zod schema fill in some options and you are off to the freaking races this is so cool um so I hope you guys also appreciate that because this is dope um you're almost at 200 oh dang dude when start when things started going good people just started coming in um 1998 of you in here thanks for being

here of course so we killed it on that so that's dope now we need to add that method so let's call it get working memory and ID I guess thread ID duh wait it you stupid ass but it it's not necessarily a string no it's just a Isn't it just a config this is actually just content or whatever right or memory let's go look at the tools real quick we store it as

metadata working memory okay for sure for sure for sure so this will be working memory that's the promise string I think is still at fault just going to remove this right now but it's not necessarily okay so I also want to do get structured memory okay how about this senator if I'm the janitor he's the senator it hallucinated that Ward got a

promotion that's funny so this by itself will just return the string version of the schema and that that might be fine um and then you but then you have to JSON parse it yourself or whatever but I want to say back to Zod should it be or I guess not i guess not because it's an API right has to be JSON anyway if you

do hono or anything what I want though is like if I call get working memory and you were using the markdown version you get a string but if you were using structured memory you get an object json schema 7 yeah well no no it would just be a straight up object m or you mean the data okay so like should we have two i don't like

having one method that returns two different types but doesn't it have format or something no no because this is this is uh getting the data oh we want we could put it on there we could store format on there and if you check format if you check format first but then it's still the same method that's returning two different types that's fine because you can with Typescript you

can say it returns um markdown version which is a string or JSON object so it's like a a union yeah but then okay then think about it from the user's perspective they have to do a check if type of this of type of that that's kind of annoying you can just stringify it but still have the format it's always a string and then they have

to still do this what's the what's the pros and cons it's annoying to have two methods or It depends I think on like uh are they going to be used separately or are they going to be used the same safe i think like for me personally and so chat help us out too like we can all kind of design this together um for me if I'm using structured memory I will never I

will only call it structured memory right plus when I call get memory I'm going to assume that it is that but the fact that you can have markdown memory means you have to know what you're doing and I don't know any thoughts for what's your thoughts for award i don't know if you still would use markdown memory then i don't know ch

like you know Tyler was saying that there's a lot of use cases for using markdown memory i personally don't think I would ever use it i would use this though mhm personally because with the markdown memory you're kind of relying on the LM to do it correctly and maybe hallucinate itself and with like JSON it

feels like it's parsible but there's some some people I can see this thing where the working memory could be like you want to have bullet points and like paragraphs and like this big pros right yeah but you can still let the LM make so yeah this is annoying because I'm guessing you will always use one or the other right you won't use you won't use both both for the same agent right

no at least so far at least in our in our um implementation you can yeah yeah in our implementation if you use the schema we always are using the schema no matter what um so maybe then to make I guess sense but then I think if it's if it's one or the other I feel that you can do one and it just returns two

different I think yeah you're right because structured memory is not is just a concept we just threw out of our ass working memory is the actual name so I think maybe we do this i think with format it's fine because we can basically let um ah yeah we ask for the format right like this but then we filter you mean

I mean like uh if the format is let's say string instead of markdown if it's this we return JSON else working memory boom so then it's on the user you Yeah cuz if I also want the if I'm using structured memory and I put format string okay like um Yeah yeah cuz you're still going to get the string and you can parse it yourself or do whatever cuz working memory is going to

be saved also if you use structured memory okay just as a string yeah okay but then it works fine then it works nice okay so now we have this API let's keep the server running i'm going to go and make a file in our example and we have about five minutes left y'all um so thanks for coming 219 going up going up going up um let's go back to our example

here and outside the monster folder I just put a bunch of scripts I like to run i'm going to write make another one here for testing working mem.ts then I'm going to just make a main function as always i'm going to import MRA and then I didn't put a default did I no problem is this is like a new feature

so the default of JSON will never work so it has to be default well by def by default it will always return the string it's a new function so maybe you have to define a format no well the way Oh no we changed the other one yeah the way I wrote it is we always if you don't give a format we're always going to do the string we'll have to we'll have to see

how that kind of works in practice let me get my Mary the agent um what's her name again let's see working memory example agent let's do a generate call um text agent generate i'm Obby my name is Obby then I'm going to pass a resource ID which will just be one thread ID one doesn't matter to me let's log it then I

need to do was it agent.get memory oh boy getting excited yo and then I'm going to do memory.get working oh I added a new method without adding it to the abstract class m this is a good educational moment actually if we were pairing with a younger or like a junior dev I'm going to do it anyway but so how does MRA do all this stuff if you were curious our core package MRA core

has all of our abstract classes we designed MRA where the core is all these abstract classes that represent the contract that if you wanted to build your own stuff into MRA as long as you follow the contract that the abstract class defines you could do whatever the you want within the methods of those classes so our base memory class is an abstract class that is in the core

module we have a package called memory which implements that base class so if you're from mem zero or those other memory dudes you could just implement MRA memory by fulfilling those methods why you chose not to I'm not sure but anyway um and so this is what we're doing i added a new method to the master

memory like implementation class but we got to go put it on the base class also I probably screwed up that um by changing the name too i probably screwed up the So we had get working get working memory oh that's not here maybe because it's not a global one maybe it's only on the It's like private is it private no but maybe it's only on implementation yeah these two or only on the

implementation because it uses it internally and not but it's a public function yeah yeah maybe it's not using it um like it's only used in the memory package so basically we only define it on the abstract if agent clause is using it that's true so like for this one we need to you could do if instance of um memory class then TypeScript knows that

it exists or you can just do this uh abstract I guess this then you have to define it um in your extended classes and then it's probably have to do format JSON too yes or string or null we can make the type better based on like format equal extends but for now it is fine yes we have to make it based on what the

working memory zod object was right yeah no based on the format right like if you pass format JSON we always return record string string if it's not then it's a string i prefer Monster Memory i feel that is better than Mezzero some of my friends are also bullish on Monsters Memory thanks dude i mean we're working we're working on it trying to make it better

um just like this this feature and stuff like that okay so then technically if everything builds I go back to my little example I was doing here I should get there you are thread one format JSON and log it out I guess log it out and I guess let's do that and For all the marbles this is our last execution if it works then the stream is over so if those of you want to pray for

our downfall right now please do um but we are pushing forward yo did I log it yeah you did it's null no maybe a metadata damn it oh it's null but that's cuz I'm return am I returning null no no but maybe JSON is null did you call it JSON that's a great That's a great great point what did I call it json i

did um I did call it JSON here's a tool result mhm right it's not like it's not there so what does that mean oh why is my server not running uh could it be that the um memory call happens async so the generator is done but it didn't update working memory in the call yet that's interesting so maybe you're too soon maybe let's try one more

time cuz I also Well I don't even need the server to run no cuz we did the straight on the object tool results now I think we have have an issue maybe um let's first check the memory class god damn it i really wanted to go okay let's let's read let's read this thread i don't need this running actually I just need this

building oh the JSON's not there m m why is that maybe because we're not storing when we update the thread cuz we never said it was in the database or we never checked if it was actually in the database right oh the save thread save thread save thread in lip seal then oh here it's gone right oh no metadata has it but then maybe save

thread there it is deep ah it just does working memory wait what oh this is R say deep yeah no but here we're basically on 231 um we're overriding metadata I'm guessing with only working memory but there's no working memory on the thread yet I guess mhm so shouldn't we just add JSON from the config no right this is from Wait I'm confused one second i am

confused but is this working memory not just it's not the data right this is the thread yep we're updating the template with whatever we're doing in here so we're saving this thread the metadata should have been in here i go back save thread threads up in this i'm doing a deep merge yeah but only if metadata does not have working memory

that's true and if it doesn't we just do trend so it should do the correct thing unless at this point it doesn't have working memory yet you know what I mean that doesn't make sense though does it but doesn't Oh on the thread yeah it's on the thread sorry it's on the thread that we passed in so this is not being called for sure it's this so maybe in the storage god damn

it god damn it look at lip SQL I guess yeah let's look at li SQL no that doesn't make sense then JSON stringify it is a object so we stringify right but it's in the table name yeah we don't do So maybe it's in the get the get thread method that we're using get thread by ID is that what we're using get working memory let's wait i looked at the thread though right it's

cuz this is the thread the thread didn't have Oh it didn't have it you can also open up okay let's go into get thread by ID that's probably You're probably right dude um we have this method which calls that method okay that's some no dude it's just straight up not saving it I think mhm so let's see save thread

i'm going to quickly wake up the kid okay see you in a bit ward's got to wake up his baby um I'm tripping i think I'm tripping but let's see if I got the thread wait I logged that somewhere right get 4k memory the thread is right here which doesn't have JSON so I should go back to the tool then uh the tool call

here and then I guess we can go Spalunk into save thread about to console my way to a victory right now so here this is the last thing we know before it gets saved we shall see we shall see also these debug logs are pissing me off so I'm going to turn those off um not feeling those debug logs right now yeah dude info i'm chilling

okay let's run this again i really need to use the restroom so this got to work soon dude it's not there either it's not there that's interesting is it because I something up probably is it because Nope okay okay let's wait and see yep 231 of you in here welcome um sorry I'm just debugging some so where is that save

happening i should just look up stuff by my log thread to save undefined yo it's undefined how come it wasn't undefined in the playground though man it's always up like this though okay let's figure this out um why would this be this way cuz it definitely was working in the playground for sure it's definitely the arms were

definitely there let's go back to the playground make sure we're not stupid we very mel very well are do the same test i work at MRA right i ain't tripping it is here in the tool arcs but But but I don't know let's find out so even here there was a thread to save oh okay that's cool let's try this again even though it's the same result

it's called insanity by the way um I'm a janitor cool at least we know at least we know that's that's what's happening but what is happening let's log params dude i don't even know let's rob these prams um for sure something's up we'll figure it out though does the baby want to come on the stream no cuz

that would be doxing and we don't do that around here um okay so let's do this again i'm going say I'm an eagle identify as an eagle oh boy that's not my occupation oh it is though okay so then man I got to get rid of this log dude this is pissing me off too but where are my params at yo where are my params let me get rid of some of this BS logs

that we have um saw not that one this I was doing something stupid in here I believe somewhere um There it is get rid of that stuff we don't need that okay i'm a janitor we have the data in the tool args as we can see and we don't have them here but we do have params now and it's in context yo it's in context dudes what were we doing before

oh my god i'm so stupid dude oh my god all right let's try it again guys damn okay see stuff like this happens all the time doesn't matter who you are um okay this will work i think there it is my working memory my state all in a day's work y'all all in a day's work now I have six more hours of work but all in the day's work of this live stream i'm going to wait for Ward to get back i'll just do some more stuff we can

keep testing um let me get rid of some of these logs too because this shit's annoying but that is pretty much the stream uh for today i'll keep working a little bit if you guys want to stick around for a little longer if you have a full day ahead of you best of luck with everything you have to do today if it's post lunch and you're trying to take a nap do so i highly recommend it um but

yeah it actually worked where's this log coming from schema um we had so many logs to get this working which is wild but we got it working which is good i'll do a little recap too for all y'all okay we got even more logs to get rid of like this one how stupid how stupid was that forgot about the context

object anyway and build it again cool a system message see how many logs we left we're just logging our way to victory and now I don't even know where the logs are i got to like look for them um that was an agent oh man we changed agent code too to make this to figure this out um this whole can go away and that's

cool and I guess we can get rid of this because it is unnecessary and lastly there's probably one more log that's left oh I guess I got all the logs yep okay so now let's do this okay so last thing I'll show Well I don't know if war's going to come back to be honest um which is fine i'm going to do one last thing to show um and then we are out of here so like everything I

showed you before was on the playground right in the playground you know our playground kind of wires everything up um in my script here this is how you do things programmatically so we can still do all the same type of kind of calls so what if what I can do is I'm going to do this like in a set interval um I'll just you know every five every two seconds let's just log

working memory um let's make this async i don't know if you can make set interval async um well but um we're going to do that and then as then I'll just make a bunch of different prompts here so text one text two um and I'm going to be like I am I am we got it working bro oh ward's back by the way um I'm the

janitor and then let's make this an actual prompt actually I don't even want this set interval i'm just going to do a really big set so we'll just put that there i was going to like pull it but I don't really care about that also don't even need this we're just going to do a big ass prompt and I'm going to use this prompt to talk about myself and the

friends that are on the stream so for those just joining if you didn't maybe if you want to put your chat if you want to boo a chat you can be included in this example but I already got the homies that are should be in the example so just to reiterate I am doing structured memory which is a type of working memory where the memory is an object a zod object so if I look at my

um working memory in my agent you can see that my schema is me as in me and then the name and occupation of who me is and then my friends and then the name and occupation as an array so technically if I put a big ass prompt here hi I am Abby i am the CTO of MRA my friend Ward also works with me so it'll infer

that where he works as a founding engineer we we I have other friends too i got Rude Boy the CIO of a medical company which I'm not sure if that's true but that's the last thing I remember then you got SUJ the Stumbler Bumbler and Fumbler lastly we had Marvin the Frenchman and we had Joseph I believe let me just double check

um we had Joseph the man in search of client side tools um I guess those are the friends for today so let's do that boom boom boom that's my prompt hopefully it does everything right because I really got to use the restroom but we press on boom let's go it's thinking now there we go y'all me my name i'm the

CTO of MRA my friends Ward founding engineer Rude Boy CIO of a medical company suj is here the Stumbler Bumbler and Fumbler Marvin the Frenchman Joseph this just worked so with that I will leave you guys have a great day and uh peace if I can learn how to turn this thing off