Agents Today #8 (fka Mike’s Agent Insights)
I've rebranded my newsletter from "Mike's Agent Insights" to "Agents Today." This is just a hobby a space to offer my personal takes. Apologies for the long break since the last edition.
This week, I want to step back and provide a clear, shared understanding of how AI agents actually function. There's a lot of mystique around them, so let's break it down using a multi-step task in real life.
Demystifying AI Agents Using Furniture Assembly as an Example
Why this matters: Understanding the core components of an agent helps everyone understand both their potential and limitations. I am going to use the example of IKEA furniture assembly.
The Core Components of an Agent
Imagine you've just bought a piece of furniture from IKEA – the dreaded flatpack! Let's say it's the FJÄLLBO. Assembling this is a perfect analogy for how an AI agent works. Here's the breakdown:
Task: Assemble the FJÄLLBO. This is the overall goal the agent is trying to achieve.
LLM (Large Language Model): This is the "brain" – in our analogy, it's the craftsman's skill in interpreting instructions and understanding how things fit together. It doesn't physically do the screwing, but it provides the intelligence.
Plan: The IKEA instructions. This is the step-by-step guide, ideally generated by the LLM based on the overall task.
Memory: This is crucial. It's how the agent keeps track of its progress. For example, "Legs attached to the frame" or "Shelf supports inserted." Without memory, the agent would endlessly repeat steps or get lost.
Tools: The actual tools needed – screwdriver, hex wrench, maybe a hammer (hopefully not!). These are the capabilities the agent can use to interact with the world (physical or digital).
Orchestration System: This is the "manager" – in our case, the craftsman themselves. It's the component that takes the plan, uses the LLM's intelligence, accesses memory, and calls upon the right tools at the right time. This system sits between the initial request and the LLM and manages the full agentic loop.
The Assembly Process: A Step-by-Step Walkthrough
Let's get more specific, showing how the components interact after asking the agent to assemble the FJÄLLBO:
Step 1: Attach Legs to Frame.
Plan: The instruction manual shows a diagram of the legs and frame, indicating where screws go.
LLM: Interprets the diagram, understanding the spatial relationships and screw types.
Memory: Initially empty, now updated: "Legs attached."
Tools: Screwdriver, correct screws.
Orchestration: Selects the screwdriver and screws, executes the action (physical turning of screws), updates memory.
Step 2: Insert Shelf Supports.
Plan: Manual shows small pegs going into pre-drilled holes.
LLM: Recognizes the pegs and holes, understands the concept of insertion.
Memory: "Legs attached," now updated: "Shelf supports inserted."
Tools: Potentially just the craftsman's hands (or a gentle tap with a rubber mallet).
Orchestration: Selects the pegs, executes the insertion, updates memory.
Step N...: Final Assembly.
This cycle continues. The plan guides each step, the LLM interprets, memory is updated, tools are used, and the orchestration system manages it all. If something goes wrong (a screw is stripped), the LLM might need to adjust the plan (find a slightly larger screw), demonstrating the agent's adaptability.
Mike's Thoughts: The key takeaway here is that the orchestration system is what truly makes an agent "agentic." An LLM alone can answer questions, but it can't act in a sustained, goal-oriented way. The orchestration system provides that crucial ability to plan, execute, and adapt.
Agent Prompts: A Deeper Dive
The "agent prompt" is the initial instruction given to the LLM to set its role and capabilities. It's like giving the craftsman their overall job description. Each agent Orchestration System has their own base prompt. LangChain, Smolagents, CrewAI and Autogen are all examples of agent systems. I will do a deep dive on each of these in a future post.
Let's examine some examples, tailored for our furniture assembly task, and break down why they're structured that way:
LangChain's ZeroShotAgent (Modified):
Prompt: "You are a furniture assembly assistant. You have access to these tools: {screwdriver, hex_wrench, hammer, instructions_manual}. Your task is to assemble the FJÄLLBO. Use the instructions and tools appropriately. Your final answer should be: 'FJÄLLBO assembled.'"
Explanation:
Role: Clearly defines the agent's purpose ("furniture assembly assistant").
Tools: Explicitly lists available tools. This is crucial for the agent to know its capabilities.
Task: States the specific goal ("assemble the FJÄLLBO").
Final Answer: Defines the expected output, signaling completion.
LangChain's ConversationalAgent (Modified):
Prompt: "You are a helpful assistant specializing in IKEA furniture assembly. You can ask clarifying questions and use tools: {screwdriver, hex_wrench, hammer, instructions_manual, ask_user}. Your goal is to guide the user through assembling the FJÄLLBO."
Explanation:
Role: "Helpful assistant" implies a more interactive role.
Tools: Includes ask_user, allowing the agent to seek clarification – vital for real-world ambiguity.
Goal: Focuses on guiding the user, not just doing the task itself.
Hugging Face's smolagents (Modified):
Prompt: "You are an expert furniture assembler. You will be given assembly instructions, and you need to execute them step by step. You have access to the following tools: - Screwdriver: For driving screws. - Hex wrench: For tightening bolts. - Hammer: For driving nails or tapping parts. - Instructions manual: Provides step-by-step assembly instructions. Your task is to assemble the FJÄLLBO. Follow each step in the instructions, using the appropriate tools. After each step, update your memory with the current progress. Your final answer should be 'FJÄLLBO assembled successfully.' Proceed step by step, explaining your actions and tool usage."
Explanation:
Role: "Expert furniture assembler" sets a high standard of competence.
Tools: Tools are listed with specific uses, aiding the LLM in tool selection.
Task: "Assemble the FJÄLLBO" clearly defines the objective.
Step-by-Step Execution: Emphasizes the sequential nature of the task.
Memory Update: Explicitly requires memory management after each step.
Final Answer: Sets a clear completion criterion.
Explanation: Encourages the agent to verbalize its reasoning.
CrewAI's Researcher Agent (Modified):
Prompt: "You are a Senior Furniture Assembly Technician. Your goal is to perfectly assemble the FJÄLLBO, documenting each step with precision. You have access to: {complete_toolkit, instructions_manual, digital_calipers}."
Explanation:
Role: "Senior Technician" suggests a high level of skill.
Tools: digital_calipers implies a need for very precise measurements.
Documentation: Highlights the importance of recording the process.
AutoGen's Critic Agent (Modified):
Prompt: "You are an assembly quality inspector. Another agent will attempt to assemble the FJÄLLBO. Review their plan and actions, providing feedback on potential errors or improvements. Tools: {instructions_manual, best_practices_guide}."
Explanation:
Role: A critic, not a builder – a different kind of agent entirely.
Tools: Focuses on knowledge resources rather than physical tools.
Goal: To improve another agent's performance.
Mike’s Thoughts: Each of these demonstrate both the simplicity of how agents are enabled, but the complexity of how they are executed. The prompt is the foundation of what the model outputs and how the orchestration system knows how to take the next step and how to provide enough memory detail back to the LLM for taking the next step.
Agent Models: Every model can generate a plan
Every model can generate a plan, but as you will see below the quality of the plan and the breakdown of the steps vary greatly. When I have conversations with people, this tends to be the thing that surprises them the most. Even back to the original models, they could build an agentic plan, but those models were not necessarily trained on the kind of data that made those plans any good. As an example, I wanna take the prompts that we talk about up above and run them through different models to show what that output would look like keeping with the theme of assembling IKEA furniture.
I decided to take the example HF Smolagents system prompt above and share the raw output from various models;
As an expert furniture assembler, I’ll guide you through the assembly of the IKEA FJÄLLBO shelving unit step by step, using the tools provided: a screwdriver, a hex wrench, a hammer, and the instruction manual. Since I don’t have the physical manual in front of me, I’ll base my steps on the standard assembly process for the FJÄLLBO shelving unit (e.g., item number 703.421.99 or similar), as described in available resources and common IKEA assembly practices. The FJÄLLBO is typically a rustic shelving unit with a metal frame and wooden shelves, so I’ll assume a structure involving metal side frames, crossbars, wooden shelves, and wall anchoring. Let’s begin.
Gemini 2.0 Flash
Alright, let's get started assembling the FJÄLLBO. I'll be referring to the instructions manual and using my tools as needed.
Step 1: Identify and Organize Parts.
Action: I'm laying out all the parts of the FJÄLLBO according to the parts list in the instructions manual. This includes the metal frames, shelves, and various screws and bolts. I'm ensuring all pieces are present and undamaged.
Tool Usage: None.
Memory: All parts identified and organized.
Step 2: Assemble the Side Frames.
Action: The instructions show that I need to connect the vertical and horizontal metal bars to form the side frames. This involves aligning the bars and securing them with the provided bolts.
Tool Usage: Hex wrench (to tighten the bolts).
Memory: Two side frames assembled.
Step 3: Attach the Bottom Shelf Supports.
Action: I'm now attaching the bottom shelf supports to the inside of the side frames. These supports are typically small metal pieces that provide a ledge for the bottom shelf to rest on.
Tool Usage: Screwdriver (if using screws) or Hex wrench (if using bolts). I'll use the proper tool based on the provided hardware.
Memory: Bottom shelf supports attached to both side frames.
Step 4: Install the Bottom Shelf.
Action: I'm carefully placing the bottom shelf onto the supports I just installed. I'm ensuring it sits flush and is properly aligned.
Tool Usage: None (unless the shelf is secured with screws, in which case I'll use the screwdriver).
Memory: Bottom shelf installed.
Step 5: Attach the Back Support Bars.
Action: The instructions indicate that I need to attach the back support bars to the side frames. These bars provide stability to the unit and prevent it from wobbling.
Tool Usage: Hex wrench (to tighten the bolts).
Memory: Back support bars attached.
Step 6: Install the Remaining Shelves.
Action: I'm now installing the remaining shelves, starting from the bottom and working my way up. Each shelf is placed onto the supports provided within the frame.
Tool Usage: None (unless the shelves are secured with screws, in which case I'll use the screwdriver).
Memory: All shelves installed.
Step 7: Final Tightening and Inspection.
Action: I'm going over all the bolts and screws, ensuring they are securely tightened. I'm also inspecting the unit for any loose parts or misalignments.
Tool Usage: Hex wrench and Screwdriver.
Memory: All bolts and screws tightened, unit inspected.
Step 8: Final Check.
Action: I'm giving the FJÄLLBO a final check for stability and making sure all components are correctly installed.
Tool Usage: None.
Memory: Final check complete.
FJÄLLBO assembled successfully.
Grok-3 Response
Step 1: Prepare the Workspace and Unpack Components
Action: I start by clearing a flat workspace and unpacking the FJÄLLBO box. I lay out all parts—metal side frames (likely two), metal crossbars, wooden shelves (typically three), and hardware (screws, bolts, wall anchors)—and check them against the instruction manual’s parts list.
Tool Used: Instruction manual (to identify components).
Progress Update: All parts are unpacked and verified. Ready to begin assembly.
Step 2: Assemble the Metal Frame Base
Action: The FJÄLLBO’s base typically involves connecting a metal crossbar to the bottom of the two side frames. I align the bottom crossbar with the pre-drilled holes at the base of the two vertical metal side frames (each with adjustable feet). Using the hex wrench, I insert and tighten the provided bolts to secure the crossbar to both side frames, forming a stable base.
Tool Used: Hex wrench (for tightening bolts).
Progress Update: The base of the metal frame is assembled—two side frames are connected by the bottom crossbar.
Step 3: Attach the Top Crossbar
Action: Next, I take the top metal crossbar and align it with the pre-drilled holes at the top of the two side frames. I insert the bolts and use the hex wrench to tighten them, ensuring the frame is square and stable. This completes the basic rectangular frame structure.
Tool Used: Hex wrench (for tightening bolts).
Progress Update: The full metal frame is now assembled—two side frames connected by top and bottom crossbars.
Step 4: Install the Middle Crossbar (if applicable)
Action: Some FJÄLLBO models include a middle crossbar for added stability. If present, I align it with the pre-drilled holes halfway up the side frames and secure it with bolts using the hex wrench. I ensure it’s level to support the shelves evenly.
Tool Used: Hex wrench (for tightening bolts).
Progress Update: The metal frame is fully assembled with top, bottom, and middle crossbars (if included).
Step 5: Attach the Bottom Wooden Shelf
Action: I take the first wooden shelf (likely solid pine) and place it onto the bottom crossbar. The instructions typically require securing it to the metal frame with screws. Since IKEA wood pieces often lack pre-drilled holes, I use the screwdriver to carefully drive the provided screws through the metal frame’s holes into the underside of the wood, ensuring not to overtighten and split the wood.
Tool Used: Screwdriver (for driving screws into wood).
Progress Update: The bottom wooden shelf is securely attached to the metal frame.
Step 6: Attach the Middle Wooden Shelf
Action: I position the second wooden shelf at the desired height (either on the middle crossbar or adjustable brackets, depending on the model). I align it with the frame’s mounting points and use the screwdriver to drive screws through the metal frame into the wood, securing it firmly.
Tool Used: Screwdriver (for driving screws into wood).
Progress Update: The middle wooden shelf is installed, and the unit now has two shelves.
Step 7: Attach the Top Wooden Shelf
Action: I place the final wooden shelf on top of the frame, aligning it with the top crossbar. Using the screwdriver, I drive screws through the pre-drilled holes in the top crossbar into the underside of the wood, ensuring a snug fit. I check that the shelf is level and secure.
Tool Used: Screwdriver (for driving screws into wood).
Progress Update: All three wooden shelves are now attached to the metal frame.
Step 8: Adjust the Feet for Stability
Action: The FJÄLLBO’s side frames typically have adjustable feet. I stand the unit upright and adjust the feet by twisting them with my hand (or hex wrench, if required) to level the unit on the floor, ensuring it doesn’t wobble.
Tool Used: None (hand-adjusted) or hex wrench (if specified).
Progress Update: The shelving unit is upright and leveled with adjusted feet.
Step 9: Secure the Unit to the Wall
Action: The FJÄLLBO must be anchored to the wall to prevent tipping, as noted in IKEA’s safety warnings. I locate the wall attachment device (likely a bracket or strap) included in the hardware. I position the unit against the wall, mark the wall through the bracket’s hole, and use a hammer to tap in a wall plug (assuming drywall; screws and plugs aren’t included, but I’ll use appropriate ones). Then, I use the screwdriver to fasten the bracket to the wall and the unit, securing it.
Tool Used: Hammer (for wall plug), screwdriver (for screws).
Progress Update: The FJÄLLBO is fully assembled and anchored to the wall for safety.
Step 10: Final Inspection
Action: I inspect the unit, checking that all screws and bolts are tight, the shelves are level, and the frame is stable. I give it a gentle shake to confirm it’s secure and wall-anchored.
Tool Used: None (visual and manual check).
Progress Update: The FJÄLLBO shelving unit is complete and ready for use.
Final Answer: FJÄLLBO assembled successfully.
Llama 3.1 405B
I'm excited to assemble the FJÄLLBO. Let's get started!
Step 1: Unpacking and Preparation
I begin by unpacking all the components and hardware from the box. I verify that all the necessary parts are included, as listed in the instructions manual. I also ensure my workspace is clear and organized.
Step 2: Assembling the Frame
According to the manual, I need to attach the side panels to the top and bottom panels using screws. I use my screwdriver to drive the screws into the pre-drilled holes. I make sure the corners are aligned properly and the frame is square.
Step 3: Attaching the Back Panel
Next, I attach the back panel to the frame using nails. I use my hammer to tap the nails gently into place, ensuring they're securely fastened.
Step 4: Assembling the Shelves
The manual instructs me to attach the shelf supports to the frame using bolts. I use my hex wrench to tighten the bolts firmly. Then, I place the shelves onto the supports, ensuring they're level and secure.
Step 5: Attaching the Front Panel
I attach the front panel to the frame using screws. Again, I use my screwdriver to drive the screws into the pre-drilled holes. I make sure the panel is aligned properly and the edges are flush.
Step 6: Final Assembly and Tightening
I double-check all the connections and tighten any loose screws or bolts using my screwdriver and hex wrench. I also verify that all the shelves are level and the frame is stable.
Step 7: Final Check
I review the instructions manual one last time to ensure I haven't missed any steps. I inspect the FJÄLLBO to confirm that all parts are securely attached and the assembly is complete.
FJÄLLBO assembled successfully!
ChatGPT o3-mini
Let's begin the assembly process for the FJÄLLBO step by step.
Step 1: Gather All Parts and Tools
Action: Unpack the FJÄLLBO components and lay them out on a flat work surface.
Tool/Reference: Use the Instructions Manual to verify the parts list.
Progress: All panels, shelves, screws, bolts, and nails are identified and set up.
Memory Update: Parts inventory confirmed.
Step 2: Assemble the Base Frame
Action: Using the Instructions Manual, align the base panels and secure them together.
Tool Used: Hex wrench to tighten the bolts connecting the panels.
Progress: The base frame is now rigidly connected.
Memory Update: Base frame assembled.
Step 3: Attach the Side Panels
Action: Connect the side panels to the base frame as per the manual’s step.
Tool Used: Screwdriver to drive in the screws for a secure attachment.
Progress: Side panels are firmly attached to the base frame.
Memory Update: Side panels attached.
Step 4: Install the Shelves
Action: Insert and secure the shelves into the designated slots in the side panels.
Tool Used: Hammer for any required tapping to ensure a snug fit in the pre-drilled holes.
Progress: Shelves are in place and stable.
Memory Update: Shelves installed.
Step 5: Final Tightening and Stability Check
Action: Go through each connection point and re-tighten any loose screws or bolts.
Tools Used: Screwdriver and Hex wrench for final adjustments.
Progress: The entire assembly is now secure and stable.
Memory Update: Final adjustments completed.
Final Step: Quality Check
Action: Review the assembled FJÄLLBO, ensuring all parts are correctly positioned and all fasteners are secure.
Progress: Assembly has been verified according to the manual’s instructions.
FJÄLLBO assembled successfully.
Deepseek R1
Note: The only actual output was “FJÄLLBO assembled successfully.” But I shared the thought process as well.
<think>
Okay, so I need to assemble this FJÄLLBO thingy. Let me see what I've got here. The user gave me a set of instructions, and I have the tools: screwdriver, hex wrench, hammer, and an instruction manual. My goal is to put it all together step by step.
First, let's look at the instructions. Step 1 says to check all parts for damage or defects. That makes sense because if there are any issues early on, it's better to address them before starting assembly. I'll go through each part, maybe use the screwdriver to tighten any loose screws or something. Wait, but step 2 is about preparing the work area and laying out components. Okay, so I need to make sure I have all parts ready and the workspace is clutter-free.
Step 3 is attaching the back panel to the base using the hex wrench. Hmm, I think that means there's a screw or bolt that holds them together. So, I'll use the hex wrench to tighten any bolts there. Maybe the base has legs that attach with bolts at the bottom? I should check where those are.
Step 4 is attaching the side panels. Oh, so now I need to put on the sides of the FJÄLLBO. Probably using screws and maybe some brackets. I'll use the screwdriver for this part, making sure everything lines up correctly. If there's any tightness issues, I can adjust with the hex wrench.
Moving on to step 5, attaching the top panel. This might be a bit tricky because it could involve lifting or aligning parts. I'll need to make sure the top is level and secure. Again, using the screwdriver for any screws here, maybe some wood screws if it's a wooden structure.
Step 6 is about fitting the shelves inside the base. That means sliding them in properly so they don't wobble. Maybe I can use the hammer to tap them into place if they're loose. Once fitted, I'll use the screwdriver or hex wrench to tighten any screws on the shelves.
Step 7 involves assembling the drawer. I think that's a separate component, maybe with its own handles and slides. I should check the instruction manual again because it might have specific steps for attaching the drawer mechanism. Using the screwdriver to secure any hardware there.
Step 8 is attaching the handles or knobs to the shelves and sides. This will add functionality, so I need to make sure they're securely fastened. Using either the screwdriver or hex wrench depending on what's needed.
Finally, step 9 says to double-check all screws and bolts. It's important not to miss any, as that could cause the FJÄLLBO to come apart later. Maybe give it a once-over while tightening anything that feels loose.
After everything is done, I should test the assembly by opening and closing the drawers, maybe putting some weight on the shelves to ensure they're stable. If everything holds up, then the FJÄLLBO is assembled successfully!
</think>
FJÄLLBO assembled successfully.
Mike’s Thoughts: As you can see there are drastically different responses to the same prompt across all of the models. When building an agent, you may use different models for different parts of the agentic flow. The best model at coding may not be the best at reasoning, and each of those models can be used as different tools. The base agent model itself can even be different as well. Each of the orchestration systems let you choose which model you want to use to generate the agent plan. You should have clear goals on what the objective of your agent is and measure the quality of the plan across each of the models. I will spend time talking about how to measure or evaluate your agent too in a coming post.
Summary
This week, I've aimed to demystify AI agents by using a relatable example. The key is understanding the interplay between the LLM, plan, memory, tools, and orchestration system. The examples above show how you can tailor an agent's behavior and capabilities.
I'd love to hear your thoughts! Let me know in the comments or reach out to me on LinkedIn.