← Agent Architectures: Advanced Strategies for Intelligent LLM Systems

References and Supporting Papers

Collect supporting papers and references for the agent architecture concepts discussed throughout the book.

This section collects papers, essays, and project references related to the themes in the book. It should be treated as a starting point for further reading, not as a fully audited citation apparatus. Some entries need source verification before a formal publication pass.

Chapter 1: Introduction to LLM Agents

  • Vaswani et al. (2017). Attention is All You Need. Introduced the transformer architecture foundational to modern LLMs.
  • Brown et al. (2020). Language Models are Few-Shot Learners (GPT-3). Demonstrates general capabilities of LLMs as zero/few-shot learners.
  • OpenAI (2023). Introducing Function Calling. Relevant to tool-calling interfaces and structured model outputs.

Chapter 2: Methodologies and Core Patterns

  • Yao et al. (2022). ReAct: Synergizing Reasoning and Acting in Language Models. Relevant to reasoning-and-acting loops.
  • Jiang et al. (2023). Active-Prompt: Prompt Engineering with Chain-of-Thought Reasoning. Related to prompt refinement and idea iteration.
  • McLuhan, M. (1964). Understanding Media: The Extensions of Man. “The medium is the message” section reference.

Chapter 3: The Architecture of Agent Behavior

  • Shinn et al. (2023). Reflexion: Language Agents with Verbal Reinforcement Learning. Relevant to reflection and feedback loops.
  • Liu et al. (2023). ToolLLM: Facilitating Tool Learning with Language Models. Basis for tool-augmented agent capabilities.
  • Rajani et al. (2019). Explain Yourself! Leveraging Language Models for Commonsense Reasoning. Relevant background for explanation and reasoning traces.
  • Microsoft (2023). AutoGen: Enabling Next gen LLM Applications. Practical implementation of agent roles and multi-agent coordination.

Chapter 4: Designing Your First Agent (On Your Phone)

  • OpenAI Community & Prompt Engineering Guides (2022–2023). Prompt design as an accessible interface to agent behaviors.
  • Qin et al. (2023). ToolBench: Towards Empowering Large Language Models with In-Context Tool Learning. Basis for simulating tools with prompts.
  • Shinn et al. (2023). Reflexion. Related to critique and revision patterns.

Chapter 5: The Thinking Agent

  • Madaan et al. (2023). Self-Refine: Iterative Refinement with Self-Feedback. The model-as-critic structure.
  • Liu et al. (2023). Reviewer LLMs. Citation needs verification before publication.
  • Bai et al. (2022). Training a Helpful and Harmless Assistant with RLHF. Introduces reward feedback loops and output alignment strategies.

Chapter 6: Architecting Agent-Based Systems

  • Wu et al. (2023). AgentVerse: Facilitating Multi-Agent Collaboration. Details centralized vs decentralized agent architectures.
  • Zhang et al. (2023). CAMEL: Communicative Agents for Mind Exploration of Large Scale Language Model Society. Supports multi-agent dialogue frameworks.
  • Zeng et al. (2022). A Survey of Multi-Agent Systems. Gives academic grounding to MAS coordination techniques.
  • Patil et al. (2023). Gorilla: Large Language Model Connected with Massive APIs. Relevant to tool and API use.

Chapter 7: A New Way of Working With Technology

  • Schick et al. (2023). Toolformer: Language Models Can Teach Themselves to Use Tools. arXiv:2302.04761
    → Relevant to tool-use interfaces.
  • Paranjape et al. (2023). DSPy: Compiling Declarative Language Model Programs. arXiv:2310.01848
    → Relevant to declarative language-model programs.
  • Yao et al. (2022). ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629
    → Relevant to reasoning-and-acting loops.
  • Wu et al. (2023). AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation Frameworks. arXiv:2309.11455
    → Demonstrates modular, conversation-first interactions among agents.
  • Shinn et al. (2023). Reflexion: Language Agents with Verbal Reinforcement Learning. arXiv:2303.11366
    → Relevant to reflection and revision patterns.

Chapter 8: Companion Agents

  • Shinn et al. (2023). Reflexion: Language Agents with Verbal Reinforcement Learning.
    → Related to reflection loops; does not by itself establish companion-agent memory.
  • Liu et al. (2023). CAMEL: Communicative Agents for Mind Exploration of Large Scale Language Model Society.
    → Relevant to role assignment in multi-agent simulations.
  • Paranjape et al. (2023). DSPy.
    → Relevant to modular language-model program design.
  • Wu et al. (2023). AutoGen.
    → Foundation for prompt-based team construction and multi-role behavior.
  • Yao et al. (2024). MARS: A Multi-Agent Framework Incorporating Socratic Guidance for Automated Prompt Optimization.
    → Citation needs verification before publication.

Chapter 9: Designing Your Digital Lens

  • Yao et al. (2022). ReAct.
    → Relevant to task-contextual reasoning.
  • Paranjape et al. (2023). DSPy.
    → Relevant to declarative interfaces; filtering claim needs verification.
  • Wu et al. (2023). AutoGen.
    → Conversation-driven interface for lens behavior.
  • Schick et al. (2023). Toolformer.
    → Relevant to tool-use learning; content-filtering application is an extrapolation.
  • Shinn et al. (2023). Reflexion.
    → Relevant to feedback-driven revision loops.
  • Yao et al. (2024). MCTS-RAG: Enhance Retrieval-Augmented Generation with Monte Carlo Tree Search.
    → Retrieval/planning relevance needs verification before publication.

The following references are candidates for Chapter 10’s discussion of freestyle cognition, research automation, and AI-assisted prototyping. Verify each title, author list, date, and relevance before final publication.

Chapter 10: Freestyle Cognition

  • Co-Intelligence: A Unified Framework for Learning with Language Agents
    Shinn et al., 2023 – arXiv:2309.00615
    Needs verification.

  • Language Agents as Collaborators for Scientific Discovery
    Yao et al., 2023 – arXiv:2310.02634
    Needs verification.

  • Autoformalization with Large Language Models
    Lehner et al., 2023 – arXiv:2301.13867
    Relevant to translating natural language into formal representations.

  • DSPy: Deep Language Systems Made Easy
    Khattab et al., 2024 – arXiv:2402.19151
    Relevant to programmatic language-model workflows.

  • Open Reasoner Zero: Scaling Reasoning in Language Models via Thought Decoding
    Sun et al., 2024 – arXiv:2403.09353
    Needs verification.

  • MCTS-RAG: Enhance Retrieval-Augmented Generation with Monte Carlo Tree Search
    Xu et al., 2024 – arXiv:2403.05942
    Needs verification.

  • Auto-J: Judging Large Language Models Without Ground Truth
    Xu et al., 2024 – arXiv:2403.10001
    Needs verification.

  • ReaRec: Think Before Recommend – Unleashing Latent Reasoning in Recommenders
    Zhang et al., 2024 – arXiv:2503.22675
    Needs verification.

Chapter 11: The World You’re Building

  • Anthropic (2023). Constitutional AI.
    → Emphasizes building AI systems aligned with user-defined values and principles.
  • OpenAI (2023). GPTs Can Now Browse, Code, and Use Tools — System Card.
    → Demonstrates how agents extend human capability across domains.
  • Sutton, R. (2019). The Bitter Lesson.
    → Long-term AI advancement comes from letting systems learn from interaction at scale.
  • McLuhan, M. (1964). Understanding Media.
    → Returning to the theme of tools as extensions of human thought and agency.