Lesson 15: Agent Architecture | AI Engineering
Module 4: AI Agents & Tools

Agent Architecture

Master the ReAct (Reason + Act) pattern, understand the observation-thought-action loop, explore planning strategies, and build autonomous agents from scratch.

Lesson 15 ~25 min Advanced

What Is an AI Agent?

Agents vs. Chains

A chain is a fixed sequence of LLM calls — step A always leads to step B. An agent is an LLM that decides its own control flow. It observes, reasons about what to do next, acts, and loops until a goal is met. The LLM becomes the orchestrator, not just the executor.

The key insight: instead of hardcoding "call tool A then tool B", you give the LLM a set of tools and a goal, and let it figure out the sequence. This makes agents flexible but less predictable.

Property Chain Agent Workflow
Control Flow Fixed, linear Dynamic, LLM-decided Predefined graph with conditionals
Tool Use Predetermined steps LLM selects tools per iteration Tools at specific nodes
Looping No Yes — iterates until done Explicit cycle edges
Predictability High Low — may take different paths Medium
Error Recovery None (fails) Can retry, re-plan Explicit error edges
Best For Simple pipelines Open-ended tasks Complex but bounded processes

The ReAct Pattern (Reason + Act)

Core Idea

ReAct interleaves reasoning traces (thinking aloud) with actions (tool calls). The LLM generates a Thought explaining its reasoning, then an Action to execute, then receives an Observation (the result). This loop continues until the LLM produces a final answer.

The three components in each iteration:

  • Thought: The LLM reasons about what it knows and what to do next
  • Action: The LLM selects a tool and provides arguments
  • Observation: The system executes the action and returns results
Why the "Thought" step matters: Without explicit reasoning, LLMs jump to actions prematurely. The thought step forces the model to plan, consider what information it still needs, and avoid redundant tool calls. Research shows ReAct outperforms action-only agents by 20-30% on complex tasks.

Agent Decision Flow

Planning Strategies

  • No planning (greedy): Decide one step at a time — fast but myopic
  • Plan-then-execute: Generate full plan upfront, then execute — rigid but structured
  • Plan-and-adapt: Create initial plan, revise after each observation — best balance
  • Tree-of-thought: Explore multiple paths, backtrack on failure — expensive but thorough

Stopping Conditions

Agents need explicit stopping logic to prevent infinite loops:

  1. Final Answer: LLM declares it has sufficient information to respond
  2. Max Iterations: Hard cap (typically 5-15) prevents runaway agents
  3. Token Budget: Stop when approaching context window limits
  4. Confidence Threshold: Stop when the LLM's self-assessed confidence is high enough
  5. Timeout: Wall-clock time limit for real-time applications

Agent Memory Types

Memory Is What Makes Agents Useful

Without memory, an agent starts fresh every call. Memory lets agents learn from past interactions, maintain context across tool calls, and build up knowledge over time.

Memory Type Analogy Implementation Lifespan
Working Memory Scratch pad Current context window Single agent run
Short-term (Episodic) Conversation history Message buffer Session
Long-term (Semantic) Knowledge base Vector store Persistent
Procedural Learned skills Saved tool chains / prompts Persistent

Building a ReAct Agent From Scratch

No frameworks — just pure Python and an LLM API. This is the core pattern every agent framework wraps.

"""ReAct Agent — No frameworks, just the pattern."""
import json
import openai

client = openai.OpenAI()

# ─── Define Tools ────────────────────────────────────────────

def search_wikipedia(query: str) -> str:
    """Search Wikipedia and return a summary."""
    import urllib.request, urllib.parse
    url = f"https://en.wikipedia.org/api/rest_v1/page/summary/{urllib.parse.quote(query)}"
    try:
        with urllib.request.urlopen(url) as resp:
            data = json.loads(resp.read())
            return data.get("extract", "No article found.")
    except Exception as e:
        return f"Search failed: {e}"

def calculate(expression: str) -> str:
    """Evaluate a math expression safely."""
    allowed = set("0123456789+-*/.() ")
    if not all(c in allowed for c in expression):
        return "Error: Invalid characters in expression"
    try:
        result = eval(expression)  # Safe due to character whitelist
        return str(result)
    except Exception as e:
        return f"Calculation error: {e}"

# ─── Tool Registry ───────────────────────────────────────────

TOOLS = {
    "search_wikipedia": {
        "fn": search_wikipedia,
        "description": "Search Wikipedia for a topic. Input: search query string.",
    },
    "calculate": {
        "fn": calculate,
        "description": "Evaluate a math expression. Input: expression string like '2 + 2'.",
    },
}

# ─── System Prompt ───────────────────────────────────────────

SYSTEM_PROMPT = """You are a ReAct agent. For each step, output EXACTLY one of:

Thought: [your reasoning about what to do next]
Action: [tool_name]: [input]

OR when you have enough information:

Thought: [final reasoning]
Answer: [your final response to the user]

Available tools:
{tools}

Rules:
- Always start with a Thought
- Only use one Action per step
- Wait for Observation before your next Thought
- When you have enough info, give the final Answer
""".format(tools="\n".join(f"- {name}: {t['description']}" for name, t in TOOLS.items()))

# ─── Agent Loop ──────────────────────────────────────────────

def run_agent(user_query: str, max_iterations: int = 10) -> str:
    """Execute the ReAct loop until answer or max iterations."""
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": user_query},
    ]
    
    for i in range(max_iterations):
        # Get LLM response
        response = client.chat.completions.create(
            model="gpt-4o",
            messages=messages,
            temperature=0,
            max_tokens=500,
        )
        
        llm_output = response.choices[0].message.content.strip()
        messages.append({"role": "assistant", "content": llm_output})
        
        print(f"\n--- Step {i+1} ---")
        print(llm_output)
        
        # Check if agent produced a final answer
        if "Answer:" in llm_output:
            answer = llm_output.split("Answer:")[-1].strip()
            return answer
        
        # Parse and execute action
        if "Action:" in llm_output:
            action_line = llm_output.split("Action:")[-1].strip()
            tool_name, tool_input = action_line.split(":", 1)
            tool_name = tool_name.strip()
            tool_input = tool_input.strip()
            
            if tool_name in TOOLS:
                observation = TOOLS[tool_name]["fn"](tool_input)
            else:
                observation = f"Error: Unknown tool '{tool_name}'"
            
            print(f"Observation: {observation[:200]}")
            messages.append({"role": "user", "content": f"Observation: {observation}"})
        else:
            # No action and no answer — prompt for next step
            messages.append({"role": "user", "content": "Continue with your next Thought or Answer."})
    
    return "Agent reached maximum iterations without a final answer."

# ─── Run It ──────────────────────────────────────────────────

result = run_agent("What is the population of France divided by the area of Germany?")
print(f"\nFinal Answer: {result}")
Key implementation details: The agent loop is dead simple — call LLM, check for Answer/Action, execute tool, append observation, repeat. Everything complex (reasoning, tool selection, stopping) happens inside the LLM via the system prompt. This is why prompt engineering matters so much for agents.

Making Agents Robust

"""Robust agent with error handling, retries, and budget tracking."""

class AgentBudget:
    """Track and limit agent resource consumption."""
    def __init__(self, max_iterations=10, max_tokens=50000, max_time_seconds=60):
        self.max_iterations = max_iterations
        self.max_tokens = max_tokens
        self.max_time_seconds = max_time_seconds
        self.iterations_used = 0
        self.tokens_used = 0
        self.start_time = None
    
    def check(self):
        import time
        if self.iterations_used >= self.max_iterations:
            raise BudgetExceeded("Max iterations reached")
        if self.tokens_used >= self.max_tokens:
            raise BudgetExceeded("Token budget exhausted")
        if self.start_time and (time.time() - self.start_time) > self.max_time_seconds:
            raise BudgetExceeded("Time limit exceeded")

class BudgetExceeded(Exception):
    pass

def run_robust_agent(query: str, budget: AgentBudget = None) -> dict:
    """Agent with full error handling and observability."""
    import time
    
    budget = budget or AgentBudget()
    budget.start_time = time.time()
    trace = []  # Full execution trace for debugging
    
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": query},
    ]
    
    try:
        while True:
            budget.check()
            budget.iterations_used += 1
            
            response = client.chat.completions.create(
                model="gpt-4o",
                messages=messages,
                temperature=0,
            )
            
            llm_output = response.choices[0].message.content.strip()
            budget.tokens_used += response.usage.total_tokens
            
            trace.append({"step": budget.iterations_used, "output": llm_output})
            messages.append({"role": "assistant", "content": llm_output})
            
            if "Answer:" in llm_output:
                return {"answer": llm_output.split("Answer:")[-1].strip(),
                        "trace": trace, "budget": vars(budget)}
            
            if "Action:" in llm_output:
                action_line = llm_output.split("Action:")[-1].strip()
                try:
                    tool_name, tool_input = action_line.split(":", 1)
                    tool_name = tool_name.strip()
                    observation = TOOLS[tool_name]["fn"](tool_input.strip())
                except KeyError:
                    observation = f"Error: Tool '{tool_name}' not found. Available: {list(TOOLS.keys())}"
                except Exception as e:
                    observation = f"Error executing tool: {e}"
                
                trace.append({"step": budget.iterations_used, "observation": observation})
                messages.append({"role": "user", "content": f"Observation: {observation}"})
    
    except BudgetExceeded as e:
        return {"answer": None, "error": str(e), "trace": trace, "budget": vars(budget)}

Mini-Project: Wikipedia Research Agent

🛠️ Build a ReAct Agent with Wikipedia + Calculator

Build a complete ReAct agent that can answer complex questions requiring multiple Wikipedia lookups and calculations.

  1. Implement the tool registry with search_wikipedia and calculate
  2. Write the ReAct system prompt with clear formatting rules
  3. Build the agent loop with proper parsing of Thought/Action/Answer
  4. Add budget tracking (max 8 iterations, 30s timeout)
  5. Add execution tracing — log every thought, action, observation
  6. Test with multi-step questions:
    • "What year was the Eiffel Tower built, and how many years ago was that?"
    • "Compare the populations of Tokyo and London. Which is larger and by how much?"
    • "What is the GDP of Brazil divided by its population?"
  7. Add a third tool: get_current_date for time-aware calculations

Stretch Goals

  • Add a plan-and-adapt step: agent writes a plan first, then revises after each observation
  • Implement retry logic: if a tool fails, the agent tries a different approach
  • Add a "reflection" step after every 3 iterations to check if progress is being made

Key Takeaways

  • Agents differ from chains by having dynamic control flow — the LLM decides what to do next
  • The ReAct pattern interleaves Thought → Action → Observation in a loop until a final Answer
  • Always implement stopping conditions: max iterations, token budget, and timeouts
  • Agent memory spans working (context window), episodic (session), and semantic (persistent) types
  • Start simple — a basic ReAct loop with good tools beats a complex framework with bad prompts
  • Full execution traces are essential for debugging agent behavior