Jensen should have watched our show

Shane: And this person turned out allegedly is a Chinese spy.

Abhi: Allegedly.

Shane: Allegedly. Which is funny because if Jensen just watched the show 12 months ago, we told you about the spies in San Francisco. Welcome everybody, this is Agents Hour. We're gonna do the news. We do this every week, besides last week, but every week besides that we do the news and we're gonna do it today. Lot of drama, a lot of things to be talking about. Let's start with the apology letter. So Jensen wrote an internal apology letter that leaked, some quotes, I owe my family and all of you an apology. I was unfaithful to my wife, my actions hurt her and our children, there's no excuse for what I did, I am deeply sorry. Leading NVIDIA is a privilege in my lifetime. So a whole bunch of things. You can read the whole letter if you would like. It's been posted on X many times and reshared, but you might be wondering why. What did Jensen do that was worth kind of this, again, it wasn't public, but now it is, but this very like internal NVIDIA is a big company, apology. So this is a good summary by Matt Van Swol. I'll kind of just give the highlights of what we know. And you might be wondering why we care. Well, Jensen runs probably the largest, I think market cap-wise the largest or one of the largest companies in the world. And if you're in AI, NVIDIA really matters. So the stock prices kind of took a bit of a tumble and you know a lot of people are speculating is Jensen going to be let go? Like, does he... There's an internal investigation. But to sum it up, apparently Jensen had been having this is a alleged, I guess it's confirmed if he kind of publicly apologized. But was having an affair with someone and this person turned out allegedly is a Chinese spy.

Abhi: Allegedly.

Shane: Allegedly. Which is funny because if Jensen just watched the show 12 months ago, we told you about the spies in San Francisco. We talked about the Chinese spies. If you just watched the show, you would have been you would have been saying that.

Abhi: I mean our you know overseas.

Shane: I think the you know the thought was to get, you know, better deals on chips and then where do those chips end up? Do they end up you know making their way to China? I think that was the speculation. There's an investigation going on. We don't know anything other than what you've you know you've probably all seen.

Abhi: We know that he got got.

Shane: Yeah, but we do know it is not it's not a good look for him, of course. Not a great, you know, look for NVIDIA and that hurts NVIDIA stock price, of course, and maybe, you know, has some repercussions on just the AI industry in general, but we will see what those might end up being. And then I thought this was a funny post Jensen, that's not what we meant by Chinese models. But let's go ahead and talk about Fable. So Fable 5.2 is here. You probably saw it. This was from November ninth, so a few days ago. Both Fable 5.2 and Mythos 5.2 were launched. Have you had a chance to play around with it at all?

Abhi: Yeah. It's been pretty good, but I think Opus 5.5 is still my choice of model. I don't know, I don't feel like that it's any better, but.

Shane: I mean I'm still using Opus 5.5 for most things. I did send Fable on some long tasks and it I haven't been able to have the time to really verify how good it did, but it cooked for a long time. Spent a lot of tokens, so you know, I still have to go back and and see how well it did. But I do think from what I've heard and what I've, you know, just from people on the team, Opus 5.5 is seems almost as good. Fable 5.2 might be a little better, but it's a lot more expensive and you're gonna burn your especially if you're using the subscription, you'll burn your usage much faster. So we do have, you know just kind of compare Opus 5.5 on the benchmarks. So on FrontierCode it's a little better than Opus 5.5. You know you got all kinds of different, I guess, science benchmarks, coding benchmarks, knowledge work benchmarks, and you can see specifically kind of in the knowledge work area is where it starts to get quite a bit better. Multidisciplinary reasoning, visual chart recognition. Actually, it looks like it's better in every measure, which is interesting. Like and maybe they just picked these benchmarks because they showed the story they wanted to tell. But you can see it's quite a big jump from Fable 5.1 and a decent jump in most cases. From Opus 5.5. But again, I kinda wonder at the edges of these benchmarks, can the average person even feel those differences? Because there can be... Again, I've had one really long running task and then some other like short interactions. I couldn't notice significant things. And maybe it's we're at the edges of where like I just can't even comprehend the differences because the model's just getting that good. No, I'm sure there'll be another big jump at some point, but I I haven't noticed a major difference yet. Also related to Anthropic, they rolled out Claude Whiteboards, Claude Sheets, Claude Data. Any traces you send them are gonna end up as products in Claude, so we shouldn't be surprised. Alright, new models.

Abhi: Alright. So we have a bunch of new models this week, but at this point, I think most models I think are coalescing around one benchmark, which is like the agentic benchmark and you'll see I think most models are now trying to be more than just coding agents good at agentic tasks. Also Gemini 4 Carbon is out and it's doing phenomenally well across all the benchmarks, especially agentic benchmarks, which makes you wanna try it because if you're actually building a product agent that's not necessarily doing coding, you want it to be good at generally a lot of things. So this could be really interesting.

Shane: Yeah, I mean you see it like across knowledge work and and even Fable was promoting that more, right? So I do think it's it's good that we started to like emerge from this bubble of just coding agents. And because we've been talking about that for a long time and it feels like models are starting to be trained more for that. I feel like we need some good like writing benchmarks because I think most models are so terrible at writing, you know, in general.

Abhi: Next up we have Genie 4, which is a world model. So if you don't know what a world model is, it's a model that can make interactive environments and, you know, we've been saying this for a while in the industry that world models could be bigger than LLMs because you can create your own worlds and stories and a bunch of cool shit. And so Genie 4 is a model that'll help you do that. Kimi K3.1 Pro open weights. So Moonshot released the model weights. That's all cool. Here's a big one because I've been using ChatGPT a lot and image generation has been kind of hit or miss. But GPT-Image-2.6 is out. Pretty much it's 50% faster generation. You can do consistency across edits, which is super important because you do you want to have that consistency, especially if you're building like a diagram or something, it needs to be consistent, it needs to not take two steps back. When the model generates the new image. So definitely must try. I think it's good.

Shane: It's always a pain when you try to fix one thing and it wrecks something else that was already pretty good. You know, like make subtle changes somewhere else and you didn't ask it to do that. So I think that's huge.

Abhi: And then to round us out, Mistral, you know, Mistral's French. They have Le Chonk. It's agentic coding, cybersecurity. They're still in the coding game. And yeah. I don't really care about Mistral, so.

Shane: I don't know anyone that's really using it significantly. But I guess they're an open model and they at least are starting to benchmark well again after what seemed like being left for dead. And finally, friend of the show announced this, and many people from Google also announced it, but Google released a decisions model API. So everyone's you know trying to eat Jev's lunch.

Abhi: Leave in Google.

Shane: Alright, now let's talk about if models and you know tools, I guess, can improve themselves. So this was really interesting.

Abhi: Yeah, so I guess before we get into the research, I think there's a lot of definitions to explain. So I read this paper, it's really cool. And you know, funny enough, like we hear a lot of terms, but we don't necessarily stop to go and understand them when we're moving and building so quickly. I actually didn't really understand what a weight is in a model. And so I was like, what is a model weight? And that will help us kind of understand what this research paper is. Because when we're talking about a model that has a billion or seventy billion or a hundred billion trillion parameters. We're essentially talking about a bunch of numbers that determine how a like a neural network or decision tree like what it does to predict something and when you train a model, you're trying to figure out what those numbers should be. There's all these different numbers that you need to input into the model for it to predict things properly. And so that's where training models really comes in. And so there's like this technique that I read about, which is really cool, called gradient descent. And I think the best way I can help people understand it, if you already understand it Great for you. I didn't. Like imagine if you're sitting at a top of a hill and you can't really see the slope down the hill, but it's just covered in a bunch of trees. And so if you were to be tasked to go down the hill as you go down the hill, you're starting to see the direction of the hill and like where it can go and how you can go down it. So that's essentially what gradient descent is, as like the loss rate or error rate goes down as you continue forward and so that's how model weights, you know, get trained. So like every prediction in a model training uses something called a loss function, right? And that loss function allows it to contribute a weight for the error or the loss. So that's kind of the basics. You should read more about it. But then the question is what if an LLM can tune its own weights. And this is where things get interesting as fuck. Because if you can give it a copy of itself and then have it try to inspect the model, figure out what the neural network is supposed to do, can it change its own weights to predict better outcomes? And so these dudes at MIT, they trained a Qwen 3.5, I think it was a 0.8 billion parameter model. They gave it this like benchmark called MATH-500. Bunch of math problems. Before editing itself, it got like a 7.9% in accuracy score. And then after it's went on the loop to improve itself, it's 48.1%, which is quite big, you know. So I guess my last point here and this is why this is super interesting is like you know a lot of us are just using the models to build our agents. Maybe these practices get a lot easier if you're like a really good agent builder. Maybe it's and if the agents can then tune the models themselves or LLMs can, you can probably start learning how to do this yourself.

Shane: Yeah, I mean I think there was some questions on this report and how meaningful it actually was. Some people said, well, it just kind of like hacked around something in the weights that was actually some kind of error. But again, it did just make a change that it accomplished the task, which was probably to like improve on this thing. I think you should take it with a grain of salt. It's a really cool finding, but there are some like nuances to how good it is. But over time it might get better, right?

Abhi: Yeah, like a better model was it was tuning the weights of Qwen. It wasn't doing it itself. But I mean you could have a lot of things.

Shane: A really good model looked at a smaller model, saw that how well it was accomplishing a task, changed the weights, it accomplished the task better. After it actually directly modified the weights. And then, you know, reproduced the results. And then of course other people, you know, pointed out some of the failures of it, but I think it is an interesting discussion because even if it isn't fully capable today, as the models get smarter, maybe it will be. Maybe actually what is somewhat possible today becomes more easily possible in the future. So there's another post here which is not necessarily about models improving themselves. But can harnesses self-improve? This post says agent harnesses are usually hand-tuned or optimized offline against benchmarks. What if an agent could improve its own harness while doing the work? So essentially rewriting its own code and not blowing up while it's doing it, right? Not failing in a way that it but in a recoverable way where it can write its own code and then just like keep trying to accomplish the task. So it's eventually essentially a self-improving harness. So this is a cool idea. Funny enough you can kind of do this with Mastra Code today. It's not documented, but you know we have this concept of plugins so and plugins can be like hot reloaded. And so you can have the harness write its own plugins. So it technically can do this. And I think other harnesses can kind of do this in different ways. But it is kind of an interesting conversation because I think we'll see more of this as we go on.

Abhi: Yeah, I think, you know, I still think the harness and agent divide is like still tough for people to grok, but like if you own your agent's code, you can do loops like this, which I guess is the same thing as the harness. So I wouldn't say every harness is chasing this. I think the data that they're collecting from people using it is pushing that. Like, share, and subscribe.

Shane: And follow us on X.

Abhi: And tell your friends.

Shane: And their friends.

Abhi: I mean we're not begging.

Shane: Well, maybe a little bit.

Abhi: Subscribe to Agents Hour every Monday, noon Pacific.

Shane: Next let's talk about all the acquisitions. There's a ton of acquisitions going on. Seems like every week now. So a lot of like consolidation around in it. You see this with any kind of expansion cycle, right? You got all this new technology, all these companies starting up. Like some companies will be blowing up and people want to acquire them for that reason. Some companies will have really good technology and people want to acquire them for that reason or really good teams. And so it seems like there's really new acquisitions every single week. First, friend of the show, Charlie from Conductor. This says, I'm very excited to share that Conductor is joining Meta. We have some big crazy plans for the next interface. If you like Conductor, you're gonna love what's coming. Essentially Meta and you know kind of the superintelligence team over there has acquired Conductor. We've had Charlie on the show once or twice at this point.

Abhi: Yeah.

Shane: Talking about Conductor. I think there's still one or two people on the team that still really like use Conductor for different things.

Abhi: Sucks for them.

Shane: And so they were kind of like what do I do now? I really liked Conductor. What's next? But good for Charlie and the team. I think I'd be interested to see what they kind of come up with and what they work on at Meta. I'm assuming they're taking some of the things that they were working on and learning from Conductor and trying to apply them over there, you know, on that team so congrats.

Abhi: Congrats.

Shane: Next up, Dex. Another friend of the show. Super stoked to announce that HumanLayer is joining forces with Factory. So this is a Factory Droid. Incredibly grateful to all your customers, investors, friends, and family who cheered us on this crazy journey in the past three years. And then of course they have he has his in-depth take. There's announcements from you know the factory team as well. Super excited for Dex. You know, we had Dex on a couple months back talking about just software factories before we even built ours. So this is probably what, four months ago or so that I think Dex was last on. I think he's coming on again here in the next couple months once he gets settled in over there. So we'll get Dex back on the show. Congrats to Dex and the team.

Abhi: We were always competitors.

Shane: We were always competitors. That's okay. OpenAI on November 4th said It's funny, all these were on all three of these so far were on November 4th, so big acquisition day apparently. OpenAI says today we're announcing that OpenAI will acquire Pierre Computer. To accelerate our vision for making Codex a place where teams can build software together with AI, from idea to first version they ship. So Pierre's open source developer tools already power parts of Codex today.

Abhi: It's gonna be dope, dude. They're gonna build Git.

Shane: Every lab's trying to do it, it seems like. So this is another other friends of Mastra, so Oura to acquire Counsel Health. I think it was for like $635 million, $685 million. I don't know. It's in the post. A lot of money. Counsel Health's we've done quite a bit with them in the past. They're big fans of Mastra, so congrats to them. If you care about personal health and the Oura rings and all that, there you go. But now, a lot of quick hits. If you watched the earlier parts of the show, we had David Cramer on. We talked about this a little bit, but MCP or CLI? Abhi, give me your 30-second take. Should teams be building an MCP? Should they be building a CLI? Should they be building both? What do you recommend?

Abhi: Okay, so first I think you should have a really good API layer in your application that way you can expose it to anything. But then you should focus on MCP. You should also focus on MCP apps because then you can deliver UI and all your data to people wherever they are instead of having a UI. No, you should have a CLI, but that's just using your API in the CLI. And then finally, should you have a UI? If you come from the web days, maybe you would say yes. If you don't give a fuck, maybe you say no. That's my take.

Shane: I think my take is similar Build a good API first, build the MCP before the CLI, and then if you need features that only the CLI can provide and you have power users that can really take advantage of it, then the CLI can make sense. Notion said, give all your agents one shared brain in Notion. Now ChatGPT, Claude, Muse, and Bot can save what they learn to the same memory and use the same skills in Notion. Everyone's trying to build different versions of company brains. Notion's kind of reasonably well positioned to do that. So a big announcement from them. This is not necessarily AI related, but it probably wouldn't have happened without AI. Charlie Cheever shared a preview of something called Exact. It's an agent native app framework written in Rust, which ends up being so it's basically Next.js, right? Next.js, Vite, you know, trying to compete with them, but built in Rust, super fast, currently pretty flexible. I don't know. A lot of a lot of hype around this.

Abhi: It's because it's built in Rust.

Shane: Yeah, anytime you say the word Rust, you get extra distribution, it seems like. People are excited. No one can read it, but people like it. Content Studio came out from LangChain and more people building company brains. This has been posted by a bunch of people, but figured we'd share the post from David. GitHub has a new PR overview. You seen this?

Abhi: Yeah.

Shane: What do you think? Useful, not useful.

Abhi: I think I still don't care about it, but it's better than before.

Shane: Some people said they hate it. Get your AI slop out of my PRs. And then some people said, they like it. David seems to like it. I think it's good. You know, sometimes you we have to try to condense information down so I think it's overall it's good. I'm just happy GitHub is like shipping some changes.

Abhi: The number one best thing about it is the checks are... You don't have to scroll.

Shane: Yes. Yep, I agree. I think the checks are probably the best part because you don't have to like find us especially if you have like review bots that review each other and there's so many comments. So sometimes you're scrolling for like two minutes to find where the checks live. Now just put it at the top. That's probably the best part. The summaries I think are okay. I still think it's an AI summarization, right? So it's it's not gonna give you all the context.

Abhi: Yeah.

Shane: But you know, maybe somewhat useful. Introducing Grok Bot 1.0. This came out just today. I don't know the big difference. There's a few changes that that came out. I think teams like Grok Bots can more Interact with each other better, basically a bunch of improvements to Grok Bot. If you're already using Grok Bot, I don't think you're gonna notice there's probably a few features. I haven't noticed any major difference. Again, it was only announced a couple hours ago. Apparently just more collaboration. You can use your Grok Bots in Slack now, I guess, as well. So maybe that's the big difference. Have you used Grok Bot at all?

Abhi: Yep, I've used all of them now.

Shane: I've been using, you know, obviously Grok Bot quite a bit. Haven't really used Muse much, use Dots a little. Do you have a favorite?

Abhi: I don't like any of them.

Shane: I think Grok Bot is like reasonably good at a few things. I had it go find me some things on Amazon. It couldn't buy it, because the Amazon doesn't let you buy it, but it went and found it. I didn't have to go do it. I don't want to think about it. I just want you to do it for me. And it did it.

Abhi: I used a single ChatGPT thread. That's what I use.

Shane: Just been using the same thread for years.

Abhi: Yeah, and I have integrations, so you know I already have my Dot and Grok Bot in my thread.

Shane: Yeah. Yeah, it pretty much all works the same. I'll say the other thing that I have Grok Bot do which I had originally built an agent for, I just kind of had stopped improving it and updating it. But I have it manage my inbox. And so now my inbox has less unread emails, but it my Grok Bot just yells at me a couple times a day and said, Hey, it's been three days. You still haven't responded to this email. It's been sitting there for three days. So now my Grok Bot just like berates me and says I need to do things that I don't I still don't do. But I feel a little bit better about it because when I open my email, I have a lot less unread. And last but not least, so this was from November 4th. This is from Tyler from the Mastra team. So today we're shipping a knowledge graph primitive to Mastra OSS. Use it for agent communication, swarms and teams, run books, policies, structured memory, and even agent civilizations. I don't know if anyone's using it for that, but maybe someone will. It's like a CMS.

Abhi: For agents.

Shane: Yeah, it's like a CMS your agents can use for controlled self-organization. So tell us a little bit about knowledge and then just as a you know. Candidly we're gonna bring Tyler on to really give us a deep dive either next week or the week after. We'll we'll get him on here. But tell us a little bit about it.

Abhi: So knowledge is what we think is the next component of context engineering, while memory is for recalling within a certain thread, knowledge is more expansive to what we call a scope so knowledge is in thought of scopes, you have like different organizational scopes, or as a user you can organize your knowledge however the hell you want. Now within scopes you have these top level pages and pages have sub pages and now you just have structured or unstructured content in them and you know in open source we've launched the basic ability to do that. We're going to be following that up with a product called Knowledge. And it'll allow you to connect with Mastra Connect and other third party sources. You can start importing and exporting from your knowledge base. All you have to do once you have this stuff set up is you give it to a Mastra agent and it gets like a hundred times smarter. And we use it internally and it works very well.

Shane: Yeah. And the nice thing is it's not just limited, it will not be just limited to Mastra agents. Right now in the open source, you kind of is. It really isn't, because it's all available through the API, but we'll make it even easier. So if you want to use, you know, Mastra knowledge with Claude Code, or Claude Desktop, or your own Grok Bot, you could probably do that, right? Or any other agent you've built. Or even for humans, right? Like there might be you might build tools on top of it for humans to like get data out of it. So it's really meant to be it's not tied to just Mastra, but I think it's useful for anyone who really wants to build. And again we kinda talked about it a few times already today. It's just if you're trying to build a company brain, you're trying to build, you know, a way to get a bunch of information that your organization needs into one place. They so you can give it to agents or humans. This is a really good solution for that. And you know, we had David on the show from Sentry, if you watch if you watch this whole thing, this whole live stream, and he talked about how their internal harness was so much better because of the internal knowledge that they were able to give it. So.

Abhi: Yeah.

Shane: More to come. That is the show. If you have not already, which I'd be shocked if you've you know, if you haven't Go follow Mastra and me and Abhi on X. You know, we're good follows. You know, let us know what you think. Also, check us out on YouTube. We're on Spotify, Apple Podcasts. Go give us that rating. We appreciate that. And with that, any parting words?

Abhi: Have fun. Good luck. Good luck.

Shane: Have fun. We'll see ya.