AI++ // memory, search, evals and risk; what it takes to be an agent


Claude Mythos is here, except it’s called Fable 5 and comes with a few restrictions. It appears to be the largest model released and, according to the benchmarks, the most accomplished. Even more so than Opus 4.8 that was only released 2 weeks ago. It’s also the most expensive model, so you might want to think twice before swapping it into your RAG support chat bot.

While it’s impressive to see the frontier march forward, this week in AI++ we’ll take a look at some of the techniques people are using to build agents for production. We’ll also have a look at a new Langflow release and round up some looks at search and memory for agents.

Phil Nash

Developer relations engineer for IBM

🛠️ Building with AI, Agents & MCP

Agents in production

To understand whether your agent is doing its job you need to evaluate its performance. The guide at How to Evaluate AI Agents is a great start to understand what you’re targeting and the ways to go about it. The OpenAI team walk us through building self-improving tax agents with Codex and shows how evals help you build loops that hill-climb to the best results.

The Anthropic team wrote about containing Claude across the different products in which it exists, identifying risks, reducing blast radius and identifying what to trust. Along similar lines, Sean Goedecke compares the risks and benefits of agents over pipelines.

Finally, LangChain describe how Lyft built their own agent platform sharing how the agents are evaluated in production with tracing and monitoring.

Brand new Langflow

Great news in the world of Langflow with the release of version 1.10. This version upgrades the Langflow Assistant from building components to building whole flows with you. It also adds Memory Bases that persist conversation context across sessions in a flow, and configurable database connectors for Knowledge Bases.

Memory and Search

We’ll start with this in depth look into agentic search that was originally a talk from the AI Engineer Europe conference. Watch the talk or walk through the examples yourself. A recent study showed that grep is all you need, but was it right? Was the harness doing a lot of the work instead?

Have you considered what to do with images in RAG? The team at Kapa have, and they describe how they index images for RAG.

Finally, the team at mem0 do a rundown of how popular agent harnesses manage their memory. There is a lot of work to be done in the memory space, and this is a great overview of techniques and drawbacks.

🧠 New models

🗞️ Other news

🧑‍💻 Code & Libraries

🔦 Langflow Spotlight

Did you see that you can now apply policies to agent actions in Langflow? Policies turns natural language rules into guards for tools directly within the agent. Prompts can guide behavior, but Policies constrain execution. Learn about how Policies work in this blog post.

🗓️ Events

The AI Coding Summit will be in London and online on July 6th and 7th with talks and workshops on MCP, agentic systems, AI-driven testing & debugging, and real-world best practices.

Use the promo code AI++ for a 10% discount on tickets.

Enjoy this newsletter? Forward it to a friend.

New Orchard Road, Armonk, NY 10504
Unsubscribe · Preferences

AI++ newsletter

Subscribe for all the latest news for developers on AI, Agents and MCP curated by the Langflow team.

Read more from AI++ newsletter

There is a lot of breathless writing in this industry about how AI is a great leveller, a tool that erases the gap between the expert and the beginner. Sean Goedecke's post LLMs reward expertise — which was Hacker News's top story this week — argues the opposite, and I think he's right. The quality of what you get back from a model scales with the quality of what you bring to the conversation. The gap between an expert and a novice wielding the same model is wider now, not narrower. That's...

A study from Capraro, Marcoccia and Quattrociocchi, written up by The Next Web, gave people AI assistance on a reasoning task and watched their accuracy fall from 27% to 9%. Confidence went the other way, from 30% to 76%, and willingness to say "I don't know" collapsed from 44% to 3%. That gap between being wrong and feeling right is the same failure mode we keep building into our agents, and the mood is showing elsewhere too: Alec Scollon reckons he has LLM burnout, Yael Grauer would like...

The AI Engineer World’s Fair was in San Francisco last week. No, I wasn’t there, and yes, I was feeling all of the FOMO. If you’re in the same position, then you can catch up with what went on with Latent Space’s write-ups from the event and subscribe to the YouTube playlist as it starts to to publish the talks. You can also check out the talks from the online track while you wait. In this edition of AI++ we take a look at the skills it takes to build great Skills, get hands-on with some...