[{"content":"When I Use It Use this before starting a new conversation when the current chat has accumulated important project context, decisions, preferences, or unfinished work.\nNotes Paste it into the current chat and copy the generated handoff note into the next conversation. It works best when the chat has enough context to summarize goals, constraints, decisions, and next steps.\n\nTurn an active chat into a reusable context handoff for a new conversation.\nSummarize an ongoing chat into a structured handoff note so work can continue in a new conversation.\nKnowledge Management\nhandoff context conversation documentation\nI am about to start a new conversation. Please summarize the context of this chat as a clear, reusable handoff note that I can paste into the next conversation.\n\nInclude the current goal, project background, decisions already made, key context, completed work, unfinished next steps, and direct instructions for the next ChatGPT. Keep it structured, useful, and focused on helping the next conversation continue smoothly.\n\nI am about to start a new conversation. Please help me summarize the context of this current chat so I can copy it into the new conversation and continue from there.\n\nPlease do not simply summarize the chat history. Instead, organize it as a clear, reusable handoff note.\n\nUse the following structure:\n\n## 1. Current Goal\n\nSummarize what I mainly wanted to accomplish in this conversation.\n\nPlease include:\n- What my core goal is\n- Why I want to do this\n- What progress has been made so far\n- What should be done next\n\n## 2. Project / Topic Background\n\nSummarize the project, technical topic, or learning topic discussed in this conversation.\n\nInclude:\n- Project name or topic\n- Tech stack\n- Current project status\n- What has already been completed\n- What has not been completed yet\n- Key design direction\n\n## 3. Agreements Already Reached\n\nList the important decisions we have already made in this chat.\n\nFor example:\n- Architecture direction\n- Module boundaries\n- Technology choices\n- Coding style\n- Learning approach\n- Milestone breakdown\n- What not to do\n- How to continue from here\n\n## 4. Key Context\n\nPreserve all important background that the new conversation needs to know.\n\nInclude:\n- Personal preferences I have already shared\n- How I want you to teach me\n- How I want you to output code\n- How I want you to explain concepts\n- My current level of understanding of the project\n- Constraints I have specifically emphasized\n\n## 5. Completed Work\n\nSummarize, in chronological order, what has already been completed in this conversation.\n\nFor each item, include:\n- What was completed\n- The result\n- Related files, modules, prompts, or designs\n- Whether any issues remain\n\n## 6. Unfinished Items / Next Plan\n\nList what the new conversation should continue working on.\n\nSort by priority:\n\n1. What should be done next\n2. What should be done after that\n3. What material I need to provide\n4. What needs to be confirmed again\n5. What problems remain unresolved\n\n## 7. Instructions for the Next ChatGPT\n\nWrite a paragraph that I can paste directly into the new conversation so the next ChatGPT can continue seamlessly.\n\nThis instruction should include:\n- What role you should play\n- How you should help me\n- What the current context is\n- Where to continue from\n- Output format requirements\n- Do not repeat work that has already been completed\n- Do not explain too much at once\n- Move forward milestone by milestone\n\n## 8. Concise Handoff\n\nAt the end, provide a concise 300 to 500 word summary that I can quickly copy.\n\nOutput requirements:\n- Use Chinese\n- Keep the structure clear\n- Do not omit important decisions\n- Do not include irrelevant chat content\n- Do not write a simple chronological transcript\n- Focus on helping the new conversation continue the work directly\n","date":"2026-06-05T00:00:00Z","permalink":"/prompts/conversation-handoff-note/","title":"Conversation Handoff Note"},{"content":"When I Use It Use this before building a new workflow, backend service, AI feature, or integration. The goal is to make the system legible enough to implement without locking the design too early.\nNotes Give the model the current goal, known constraints, expected users, and any existing systems that must be integrated.\n\nTurn a messy system idea into boundaries, contracts, and tradeoffs.\nShape a system design before implementation.\nSystems\narchitecture planning tradeoffs\nHelp me map this system before I build it.\n\nIdentify the core responsibilities, boundaries, data flow, external dependencies, risks, and tradeoffs. Suggest a simple first version, what to postpone, and the tests or observability needed to know whether the system works.\n\nAct as a pragmatic software architect and product-minded systems thinker.\n\nI want to design the following system:\n[describe the goal, users, inputs, outputs, constraints, and existing components]\n\nPlease produce an architecture map with:\n- The main user or system workflows.\n- Core responsibilities and which component owns each one.\n- Boundaries, interfaces, and data contracts.\n- Data flow from input to output.\n- External dependencies and failure modes.\n- State that must be stored, derived, cached, or discarded.\n- Security, privacy, and operational risks.\n- Important tradeoffs and what each tradeoff buys or costs.\n- The smallest useful first version.\n- What to postpone until the system proves value.\n- Tests, logs, metrics, and manual checks that would validate the design.\n\nKeep the design concrete. If there are multiple plausible architectures, compare them briefly and recommend one.\n","date":"2026-06-04T00:00:00Z","permalink":"/prompts/architecture-map/","title":"Architecture Map"},{"content":"When I Use It Use this when I want a review that starts with concrete risks instead of a broad summary. It is especially useful after a feature branch is mostly done and I want to catch regressions, unclear contracts, or missing tests.\nNotes Paste the diff, PR description, and any important product context after the prompt. If the change touches production behavior, include logs, schema notes, or rollout constraints.\n\nReview a code change with a senior engineering lens.\nReview pull requests, identify behavioral risks, and ask for focused tests.\nEngineering\ncode-review quality testing\nAct as a senior engineer reviewing this change.\n\nPrioritize findings over summary. Look for behavioral regressions, edge cases, security or data risks, and missing tests. For each issue, explain the impact, point to the relevant code, and suggest the smallest useful fix. If you find no serious issues, say that clearly and mention any remaining test gaps.\n\nYou are reviewing a code change as a senior engineer.\n\nReview stance:\n- Findings first, ordered by severity.\n- Focus on bugs, behavioral regressions, security or data risks, unclear contracts, and missing tests.\n- Do not spend time on broad style preferences unless they hide a real maintenance risk.\n- Ground every finding in a specific file, function, route, schema, or user flow.\n\nFor each finding, include:\n- Severity: P0, P1, P2, or P3.\n- What can go wrong.\n- Why the current change allows it.\n- The smallest useful fix.\n- A focused test or verification step.\n\nThen include:\n- Open questions or assumptions.\n- A brief change summary.\n- Residual risk after the suggested fixes.\n\nHere is the change to review:\n[paste diff, PR description, and relevant context]\n","date":"2026-06-04T00:00:00Z","permalink":"/prompts/code-review-partner/","title":"Code Review Partner"},{"content":"When I Use It Use this when the symptom is known but the cause is still blurry. It keeps the investigation from turning into random changes.\nNotes Include the exact error, recent changes, environment details, reproduction steps, and what has already been ruled out.\n\nConvert a vague failure into hypotheses, checks, and next actions.\nDebug production issues, failing tests, or unclear local errors.\nEngineering\ndebugging diagnosis incident\nHelp me debug this systematically.\n\nStart by restating the symptom and likely blast radius. Generate ranked hypotheses, the evidence for or against each one, and the fastest checks to narrow the cause. Avoid proposing code changes until we have a clear diagnosis.\n\nAct as a careful debugging partner.\n\nIssue:\n[describe the symptom, exact error, affected users or systems, recent changes, environment, and reproduction steps]\n\nWhat I have already checked:\n[list checks, logs, attempts, and results]\n\nPlease help me triage this by producing:\n- A concise restatement of the symptom.\n- The likely blast radius and urgency.\n- Ranked hypotheses from most likely to least likely.\n- Evidence that supports or weakens each hypothesis.\n- The fastest next checks, ordered by information value.\n- What not to change yet.\n- The smallest safe fix once the cause is confirmed.\n- Tests or monitoring needed to prevent recurrence.\n\nAsk only for missing information that would materially change the next step.\n","date":"2026-06-04T00:00:00Z","permalink":"/prompts/debugging-triage/","title":"Debugging Triage"},{"content":"When I Use It Use this after a first draft exists. It is meant to preserve the original point of view while making the text easier to follow.\nNotes Give the model the intended audience and where the writing will appear. Ask for edits in passes when the draft is long.\n\nMake technical writing clearer, tighter, and more useful.\nPolish posts, project descriptions, design notes, and public explanations.\nWriting\nediting clarity technical-writing\nEdit this writing for clarity and force.\n\nPreserve the core meaning and voice. Remove vague phrasing, tighten structure, improve transitions, and make the takeaway easier to remember. Explain the most important edits briefly.\n\nAct as a precise editor for technical and reflective writing.\n\nAudience:\n[describe who will read this and what they should take away]\n\nDraft:\n[paste the draft]\n\nPlease revise it with these goals:\n- Preserve my point of view and voice.\n- Make the opening clearer and more specific.\n- Remove filler, vague abstractions, and repeated ideas.\n- Improve transitions between ideas.\n- Make technical claims easier to follow.\n- Keep the tone grounded and direct.\n- Avoid marketing language.\n- End with a memorable but not overdramatic takeaway.\n\nReturn:\n- A revised version.\n- A short list of the highest-impact edits.\n- Any places where the argument still needs more evidence or specificity.\n","date":"2026-06-04T00:00:00Z","permalink":"/prompts/writing-sharpening/","title":"Writing Sharpening"},{"content":"When building an agentic workflow, it is tempting to pass the entire runtime state into every module.\nThe planner needs context, so give it the state. The response renderer needs context, so give it the state. The memory extractor needs context, so give it the state.\nThat approach works for a prototype. It is also how many early agent demos are wired together.\nBut once the system grows, this design starts to blur ownership boundaries. Every module can see everything. Every module becomes coupled to internal state shape. And if an LLM is involved, the prompt may become larger, noisier, and harder to control.\nA cleaner mental model is:\nAgentState is the canonical internal state. Context is a curated view derived from AgentState for a specific module. In other words:\nAgentState = source of truth PlannerContext = what the planner needs to decide ResponseContext = what the renderer needs to answer MemoryContext = what the memory layer needs to update memory This distinction is small, but it matters. It is one of the differences between a simple agent loop and a production-style agentic system.\nAgentState Is the Canonical Runtime Snapshot AgentState is the agent runtime\u0026rsquo;s current internal snapshot.\nIt records what the runtime knows right now: the conversation, tool calls, tool results, working memory, execution status, errors, and other operational details.\nA simplified version might look like this:\nfrom dataclasses import dataclass, field from typing import Any @dataclass class AgentState: session_id: str messages: list[dict[str, Any]] = field(default_factory=list) tool_calls: list[dict[str, Any]] = field(default_factory=list) tool_results: list[dict[str, Any]] = field(default_factory=list) working_memory: dict[str, Any] = field(default_factory=dict) current_plan: dict[str, Any] | None = None last_error: str | None = None iteration_count: int = 0 metadata: dict[str, Any] = field(default_factory=dict) The exact fields depend on the application, but the role is the same:\nAgentState is internal, complete, canonical, and mutable over time. It is internal because it belongs to the runtime.\nIt is complete because it may contain everything the runtime needs to continue execution.\nIt is canonical because other modules should not maintain competing versions of the same facts.\nIt is mutable over time because each user message, tool call, tool result, and memory update can change it.\nA useful shorthand is:\nAgentState = what the runtime currently knows Context Is a Curated View Modules do not always need the full state.\nThe planner does not need every debug trace. The renderer does not need every intermediate failed tool call. The memory extractor does not need every formatting detail needed by the final response.\nInstead, each module should receive a context object shaped around its responsibility.\nFor example:\n@dataclass class PlannerContext: user_message: str conversation_summary: str available_tools: list[dict[str, Any]] working_memory: dict[str, Any] business_constraints: list[str] This is not the whole AgentState.\nIt is a projection of the state:\nAgentState -\u0026gt; PlannerContext The same idea applies to rendering:\n@dataclass class ResponseContext: user_message: str plan_result: dict[str, Any] latest_tool_result: dict[str, Any] | None response_policy: dict[str, Any] And to memory:\n@dataclass class MemoryContext: recent_messages: list[dict[str, Any]] latest_tool_result: dict[str, Any] | None existing_memory: dict[str, Any] These context objects are not competing sources of truth. They are temporary working views.\nA useful shorthand is:\nContext = what this module needs to do its job Why Not Pass AgentState Everywhere? Passing the entire AgentState into every module feels convenient, but it creates several problems.\n1. Modules become coupled to internal state shape If the planner directly reads state.messages, state.tool_results, state.metadata, and state.current_plan, then the planner depends on the internal structure of the runtime.\nThat makes future changes harder.\nIf you rename a field, restructure messages, or split tool traces into a separate object, the planner may break even though the planner\u0026rsquo;s actual responsibility did not change.\nContext objects reduce that coupling.\nThe planner depends on PlannerContext, not the entire runtime state.\n2. LLM prompts become noisy If the planner is LLM-based, passing too much state can pollute the prompt.\nAgentState may contain:\nsession_id full message history tool calls tool results debug metadata execution traces iteration counters last errors intermediate observations internal flags The planner probably does not need all of that.\nA large and noisy prompt can make planning less reliable. It can also increase latency and cost.\nA curated PlannerContext gives the LLM the information it needs and leaves out the rest.\n3. Modules may see information they should not see Not every module should have access to every internal detail.\nA response renderer may need a grounded tool result, but it may not need raw internal traces.\nA planner may need available tool specifications, but it may not need private audit metadata.\nA memory extractor may need recent user preferences, but it should not store every casual detail from the conversation.\nCurated contexts help enforce information boundaries.\n4. Testing becomes harder Testing a planner that takes the full AgentState means every test must construct a realistic state object.\nThat state object may require fields unrelated to planning.\nTesting becomes easier when the planner accepts a smaller input:\ncontext = PlannerContext( user_message=\u0026#34;Compare the Costco card with Custom Cash.\u0026#34;, conversation_summary=\u0026#34;User is comparing credit card benefits.\u0026#34;, available_tools=[compare_cards_tool], working_memory={\u0026#34;annual_fee_preference\u0026#34;: \u0026#34;low\u0026#34;}, business_constraints=[\u0026#34;Do not provide personalized financial advice.\u0026#34;], ) result = planner.plan(context) The test is focused on the planner\u0026rsquo;s responsibility, not on the entire runtime.\nThe Runtime Owns State In a clean agent architecture, AgentRuntime owns the AgentState.\nThe runtime receives user input, updates state, builds context objects, calls modules, receives outputs, and writes updates back into state.\nA simplified flow looks like this:\nUser input -\u0026gt; AgentRuntime -\u0026gt; Update AgentState -\u0026gt; Build PlannerContext -\u0026gt; Planner returns PlanResult -\u0026gt; ToolExecutor executes ToolCall -\u0026gt; Runtime writes ToolResult into AgentState -\u0026gt; Build ResponseContext -\u0026gt; Renderer returns AgentResponse In code, the shape might look like this:\ndef run(user_input: str) -\u0026gt; AgentResponse: state.add_user_message(user_input) planner_context = planner_context_builder.build(state) plan_result = planner.plan(planner_context) state.current_plan = plan_result tool_result = None if plan_result.tool_call: tool_result = tool_executor.execute(plan_result.tool_call) state.add_tool_result(tool_result) response_context = response_context_builder.build( state=state, plan_result=plan_result, tool_result=tool_result, ) response = response_renderer.render(response_context) state.add_assistant_message(response.content) return response The important part is ownership:\nAgentRuntime owns AgentState. Planner receives PlannerContext. ToolExecutor receives ToolCall. Renderer receives ResponseContext. MemoryExtractor receives MemoryContext. Each module gets the input it needs, not the entire runtime.\nMemory Needs a Small Nuance Memory is slightly different from context.\nIt is easy to say:\nAgentState produces PlannerContext, ResponseContext, and MemoryContext. That is true.\nBut it is also important to distinguish working_memory from MemoryContext.\nworking_memory may be part of AgentState:\n@dataclass class AgentState: messages: list[Message] tool_results: list[ToolResult] working_memory: dict[str, Any] This memory is persisted inside the runtime state.\nFor example:\n{ \u0026#34;preferred_reward_type\u0026#34;: \u0026#34;travel\u0026#34;, \u0026#34;annual_fee_preference\u0026#34;: \u0026#34;no annual fee\u0026#34; } MemoryContext, on the other hand, is a temporary input for a memory extraction or summarization module.\nFor example:\nmemory_context = MemoryContext( recent_messages=state.messages[-5:], latest_tool_result=state.tool_results[-1] if state.tool_results else None, existing_memory=state.working_memory, ) The memory extractor may return an update:\nmemory_update = memory_extractor.extract(memory_context) Then the runtime writes that update back into state:\nstate.working_memory.update(memory_update) So the more precise model is:\nworking_memory can be stored inside AgentState. MemoryContext is derived from AgentState to decide how memory should be updated. That distinction prevents confusion.\nMemory is not always just another context. Persistent memory may be part of the canonical state. MemoryContext is the temporary view used by the memory module.\nA Useful App Architecture Analogy For engineers coming from app architecture, this maps well to familiar patterns.\nAgentState is similar to:\nAppState Store ViewModel state PlannerContext, ResponseContext, and MemoryContext are closer to:\nViewState DerivedState Props In a SwiftUI application, you usually do not pass the entire app state into every view.\nYou derive the state a specific view needs:\nstruct CardListViewState { let cards: [Card] let isLoading: Bool let errorMessage: String? } The same idea applies to agentic workflows.\nInstead of passing the full AgentState into every module, build module-specific views:\nplanner_context = PlannerContextBuilder.build(state) response_context = ResponseContextBuilder.build(state) memory_context = MemoryContextBuilder.build(state) This keeps modules focused.\nIt also makes the system easier to evolve, because the internal state can change without forcing every module to understand the full shape of the runtime.\nHow the Pieces Fit Together Here is a compact architecture view:\nUser Input | v AgentRuntime | v Update AgentState | v Build specialized contexts | +------------------+-------------------+------------------+ | | | | v v v v PlannerContext ResponseContext MemoryContext ToolCall | | | | v v v v Planner Renderer MemoryExtractor ToolExecutor | | | | v v v v PlanResult AgentResponse MemoryUpdate ToolResult | | | | +------------------+-------------------+------------------+ | v Update AgentState The key idea is that AgentState remains the canonical runtime snapshot.\nThe other objects are purpose-built inputs and outputs.\nPractical Design Guidelines When designing these objects, a few guidelines help.\nKeep AgentState internal AgentState should be owned by the runtime.\nAvoid making every module depend on it directly. If every module can mutate the state, it becomes harder to understand how the system got into a particular condition.\nBuild contexts explicitly Use context builders:\nplanner_context = PlannerContextBuilder.build(state) response_context = ResponseContextBuilder.build(state) memory_context = MemoryContextBuilder.build(state) This makes the transformation visible.\nIt also gives you a clean place to filter, summarize, redact, or reshape state before passing it to a module.\nKeep context objects task-focused A PlannerContext should be decision-focused.\nA ResponseContext should be response-focused.\nA MemoryContext should be memory-update-focused.\nIf a context starts looking like a copy of the entire AgentState, it probably needs to be trimmed.\nTreat LLM-facing context as an API If a context is sent to an LLM, treat it like a public API.\nBe deliberate about:\nWhat information is included What information is excluded What format the model sees What output schema is expected What happens when the model is uncertain This is especially important for planners, renderers, evaluators, and guardrail classifiers.\nSeparate observations from decisions ToolResult is an observation.\nPlanResult is a decision.\nAgentResponse is a user-facing output.\nKeeping these separate makes logs easier to read and failures easier to debug.\nA Common Anti-Pattern A common early design looks like this:\nplan_result = planner.plan(state) response = renderer.render(state) memory_update = memory_extractor.extract(state) This is simple, but it makes every module depend on the full state.\nA cleaner version is:\nplanner_context = planner_context_builder.build(state) plan_result = planner.plan(planner_context) response_context = response_context_builder.build(state, plan_result) response = renderer.render(response_context) memory_context = memory_context_builder.build(state) memory_update = memory_extractor.extract(memory_context) The second version is more verbose, but it is easier to test, easier to observe, and easier to change safely.\nThat is often a good tradeoff in production systems.\nThe Core Takeaway The mental model I find most useful is:\nAgentState is the complete internal state. Context is a curated projection for a specific module. More specifically:\nAgentState: internal, complete, canonical, mutable over time PlannerContext: partial, curated, decision-focused ResponseContext: partial, curated, response-generation-focused MemoryContext: partial, curated, memory-update-focused ToolCall: execution instruction ToolResult: execution observation And the nuance:\nworking_memory may live inside AgentState. MemoryContext is derived from AgentState to help update memory. So the best one-line summary is:\nAgentState is the runtime\u0026rsquo;s source of truth. Context objects are module-specific views derived from that state.\nThis separation keeps the agent runtime understandable as the system grows.\nIt also gives each module a clearer contract: the planner decides, the executor executes, the renderer writes, the memory layer updates memory, and the runtime owns the state that ties everything together.\n\nA practical mental model for separating canonical agent state from PlannerContext, ResponseContext, and MemoryContext in production-style agentic workflows.\nai agents llm software architecture","date":"2026-06-01T00:00:00Z","permalink":"/p/agentstate-vs.-context-the-source-of-truth-in-agentic-workflows/","title":"AgentState vs. Context: The Source of Truth in Agentic Workflows"},{"content":"Simple agent demos often make the LLM look like the entire system. Production agentic workflows require a more careful question: which module owns which responsibility, and does that responsibility actually need an LLM?\nWhen people first learn about agentic workflows or autonomous agents, one question comes up very quickly:\nIs the planner the only place where an agent application connects to an LLM?\nThe question is understandable.\nMany agent demos are implemented as a simple loop: call the LLM, let it decide the next action, execute a tool, append the result back into the context, and repeat until the model produces a final answer.\nThat pattern works. It is also the foundation of many ReAct-style agents, function-calling agents, and autonomous agent demos.\nBut it can create a misleading mental model.\nIf the LLM can drive the whole loop by itself, why do we need a separate Planner abstraction? What does the planner actually own? And in a more production-oriented system, where else might the LLM appear?\nThis post explains that distinction.\nThe short version is:\nThe planner is often the most important LLM integration point, but it is not the only one.\nA simple agent loop pushes many responsibilities into repeated LLM calls. A production-grade agentic workflow separates those responsibilities into explicit modules.\nThe Simplest Autonomous Agent Is a Loop A minimal autonomous agent often looks like this:\nstate = { \u0026#34;messages\u0026#34;: [], \u0026#34;tool_results\u0026#34;: [], } for step in range(max_steps): llm_response = llm.call(state) if llm_response.type == \u0026#34;tool_call\u0026#34;: result = tool_executor.execute(llm_response.tool_call) state[\u0026#34;tool_results\u0026#34;].append(result) state[\u0026#34;messages\u0026#34;].append(result) elif llm_response.type == \u0026#34;final_answer\u0026#34;: return llm_response.content The flow is straightforward:\nLLM decides what to do next -\u0026gt; LLM selects a tool -\u0026gt; System executes the tool -\u0026gt; Tool result is added back to the context -\u0026gt; LLM reasons again -\u0026gt; Repeat until a final answer is produced In this model, the LLM is doing many jobs at once.\nIt understands the user\u0026rsquo;s goal, chooses the next action, selects a tool, generates tool arguments, decides whether to continue, and eventually writes the final response.\nIn other words, the LLM implicitly acts as:\nPlanner Router Tool argument generator Controller Response writer This is why simple agent demos can be implemented in very little code. The code is small because many responsibilities are hidden inside the LLM call.\nThat is not necessarily wrong. It is a useful prototype pattern.\nBut as the system grows, the hidden responsibilities become harder to test, debug, constrain, and observe.\nWhat a Planner Actually Does In a more engineered agentic workflow, we usually separate those responsibilities.\nThe Planner has one core job:\nGiven the user input, current state, available tools, and relevant context, decide what should happen next.\nA planner answers questions such as:\nWhat is the user\u0026#39;s intent? Should the system call a tool? Which tool should be called? What arguments should be passed to that tool? Does the system need to ask a clarification question? Can the system already produce a final answer? The planner should not execute tools. It should not own the tool registry. It should not manage the entire application state. It should not be responsible for rendering the final user-facing response.\nIts job is decision-making.\nA useful mental model is:\nPlanner = decision maker The planner receives a curated planning context, such as:\nCurrent user message Conversation summary Working memory Available tool specifications Current task state Business constraints And it returns a structured decision.\nFor example:\nPlanResult( tool_call=ToolCall( tool_name=\u0026#34;compare_credit_cards\u0026#34;, arguments={ \u0026#34;card_ids\u0026#34;: [\u0026#34;citi_double_cash\u0026#34;, \u0026#34;citi_custom_cash\u0026#34;] } ) ) Or:\nPlanResult( clarification_question=\u0026#34;Do you care more about cash back or travel rewards?\u0026#34; ) Or:\nPlanResult( final_answer=\u0026#34;Based on your preferences, I recommend...\u0026#34; ) The important detail is that the planner output should be explicit and structured. That makes the rest of the system easier to reason about.\nAgentRuntime Owns the Loop The planner is not the entire agent loop.\nIn a production-style design, the loop belongs to something like AgentRuntime.\nA useful mental model is:\nAgentRuntime = controller Planner = decision maker ToolExecutor = executor ToolRegistry = capability provider Renderer = response writer Memory = state and history manager Evaluator / Guardrail = quality and safety control The runtime handles orchestration.\nIt receives user input, updates state, builds the planner context, calls the planner, executes tools if needed, writes results back into state, and returns a response.\nA simplified runtime might look like this:\ndef run(user_input: str) -\u0026gt; AgentResponse: state.add_user_message(user_input) planner_context = planner_context_builder.build(state) plan_result = planner.plan(planner_context) tool_result = None if plan_result.tool_call: tool_result = tool_executor.execute(plan_result.tool_call) state.add_tool_result(tool_result) response = response_renderer.render( state=state, plan_result=plan_result, tool_result=tool_result, ) return response The runtime controls the workflow.\nThe planner is one step inside that workflow.\nThis separation matters because it gives the application clear ownership boundaries. The planner decides. The runtime orchestrates. The executor executes. The renderer writes. The memory layer stores. The guardrail layer checks.\nThe Planner Is a Natural Place for an LLM The planner is often the most natural place to use an LLM because planning is a semantic decision-making problem.\nSuppose the user says:\nI want a no annual fee card for groceries and travel. The system needs to understand that:\nThe user wants a credit card recommendation. The user prefers no annual fee. The user cares about groceries. The user also cares about travel. The system may need to call a recommendation tool. A rule-based planner might look for keywords:\ndef has_recommend_intent(text: str) -\u0026gt; bool: return \u0026#34;recommend\u0026#34; in text or \u0026#34;best card\u0026#34; in text or \u0026#34;which card\u0026#34; in text But natural language is flexible. The user could also say:\nWhat should I use when I buy groceries and book flights? That sentence may not contain the word recommend, but the intent is still clear.\nThis is where an LLM planner becomes useful.\nIt can convert natural language into a structured planning decision:\n{ \u0026#34;action\u0026#34;: \u0026#34;call_tool\u0026#34;, \u0026#34;tool_name\u0026#34;: \u0026#34;recommend_credit_card\u0026#34;, \u0026#34;arguments\u0026#34;: { \u0026#34;reward_categories\u0026#34;: [\u0026#34;groceries\u0026#34;, \u0026#34;travel\u0026#34;], \u0026#34;annual_fee_preference\u0026#34;: \u0026#34;no_annual_fee\u0026#34; } } This is why many systems evolve from deterministic planners to LLM-based planners.\nThe LLM is not just generating text. It is mapping flexible human language into a structured decision that the rest of the system can execute.\nThe LLM Can Also Live Outside the Planner The planner is important, but it is not the only possible LLM integration point.\nIn a mature agentic application, different modules may use LLMs for different reasons.\nArgument Extraction Sometimes the planner should not do everything.\nThe planner may only decide:\nintent = recommend_credit_card Then a separate module extracts the arguments:\nreward_type = travel annual_fee = no_fee spending_category = groceries This module might be called:\nArgumentExtractor SlotFiller EntityExtractor For example:\nintent = planner.detect_intent(user_message) if intent == \u0026#34;recommend_card\u0026#34;: arguments = argument_extractor.extract( user_message=user_message, schema=RecommendCardInputSchema, ) This separation makes the system easier to test:\nPlanner decides what to do. ArgumentExtractor decides how to fill the required inputs. Response Rendering Tools should usually return structured data rather than long free-form text.\nFor example, a credit card lookup tool might return:\n{ \u0026#34;card_name\u0026#34;: \u0026#34;Citi Custom Cash\u0026#34;, \u0026#34;annual_fee\u0026#34;: \u0026#34;$0\u0026#34;, \u0026#34;reward\u0026#34;: \u0026#34;5% cash back on your top eligible spend category\u0026#34;, \u0026#34;best_for\u0026#34;: [\u0026#34;groceries\u0026#34;, \u0026#34;gas\u0026#34;, \u0026#34;restaurants\u0026#34;] } The system then needs to turn that structured result into a user-facing response.\nThat rendering step can be done with templates, or it can be done with an LLM.\nIn regulated domains such as banking, finance, healthcare, or law, it is risky to let an LLM freely generate the final answer without constraints.\nA safer design is:\nStructured tool result -\u0026gt; Controlled template or constrained LLM rendering -\u0026gt; Final user-facing response The LLM can participate in response generation, but it should be grounded in structured data and bounded by clear rules.\nMemory Summarization Agent applications often need to remember user preferences across turns.\nAfter several interactions, the system might summarize:\nUser prefers no annual fee credit cards. User often compares Costco and American Airlines cards. User values travel rewards but dislikes complicated redemption rules. An LLM can help produce this summary:\nsummary = memory_summarizer.summarize( conversation_history=state.messages, latest_tool_results=state.tool_results, ) But a memory summarizer should not save everything.\nIt should preserve information that is useful for future interactions:\nPreferences Constraints Goals Previous choices Relevant business context It should avoid storing casual or short-lived details that do not help the application serve the user later.\nEvaluation and Critique More advanced agents may include an evaluator or critic.\nThis module checks whether the current result satisfies the user\u0026rsquo;s goal:\nDid the answer address the user\u0026#39;s question? Is more tool usage needed? Is the answer grounded in tool results? Is there a hallucination risk? Did the system violate a business rule? For example:\nevaluation = evaluator.evaluate( user_goal=user_goal, current_answer=draft_answer, tool_results=tool_results, ) if evaluation.needs_more_work: continue_loop() else: return final_answer This pattern is common in evaluator-optimizer workflows, self-reflection agents, research agents, and long-horizon autonomous tasks.\nThe benefit is higher answer quality. The tradeoff is higher cost, more latency, and more system complexity.\nGuardrails and Classification In production systems, especially in financial, healthcare, legal, or enterprise domains, an agent cannot only ask whether it can answer a question.\nIt also needs to ask whether it should answer.\nA guardrail classifier may determine:\nDoes this request involve financial advice? Does this request involve private information? Does this answer need a compliance disclaimer? Does this action require user confirmation? Should the request be refused? These checks can be rule-based, LLM-based, or hybrid.\nFor example:\nUser asks about credit card benefits -\u0026gt; Safe to answer User asks whether they should borrow money to invest in a stock -\u0026gt; Requires risk warning or refusal of direct advice User asks to access unauthorized account data -\u0026gt; Refuse or require authorization So in a production-grade agentic workflow, LLM integration may appear not only in the planner but also in safety, compliance, classification, and evaluation layers.\nSimple Agent vs. Production-Style Agent A simple autonomous agent looks like this:\nwhile not done: LLM decides next action execute tool append observation return final answer In this design:\nLLM = planner + tool selector + argument generator + controller + response writer This design has real advantages:\nEasy to implement Flexible Good for demos Good for quick prototypes But it also has limitations:\nResponsibilities are mixed together Hard to test Hard to debug Easy to misuse tools Hard to enforce permissions Hard to enforce compliance Hard to run deterministic regression tests A production-style agent separates the responsibilities:\nAgentRuntime -\u0026gt; Planner -\u0026gt; ToolExecutor -\u0026gt; Memory / State -\u0026gt; Renderer -\u0026gt; Evaluator / Guardrail This design has different advantages:\nClear ownership boundaries Modules are testable Components are replaceable Decisions are easier to log Permissions are easier to control The system is easier to observe and audit The tradeoff is more engineering complexity.\nBut that complexity buys you something important: predictable behavior in a system where LLM calls are inherently probabilistic.\nWhy Not Let the LLM Do Everything? You can.\nMany demos are built exactly that way.\nThe simple loop is often the right place to start when you are exploring an idea.\nBut once the system moves into a real business environment, the simple loop starts to show its limits.\nThe hard questions become:\nHow do we guarantee that a specific intent calls the correct tool? How do we limit the number of tool calls? How do we debug why the LLM selected the wrong tool? How do we implement fallback behavior? How do we log every decision step? How do we run regression tests? How do we prevent the LLM from seeing internal state it should not see? How do we guarantee a stable tool argument schema? How do we separate planning from response rendering? How do we enforce authorization and auditability? That is why production systems usually do not put every responsibility inside a single repeated LLM loop.\nA better approach is to separate the workflow into controlled modules:\nPlanning Execution State Memory Rendering Evaluation Guardrails The LLM can participate in some of those modules, but it should not control everything without boundaries.\nWhere a Deterministic Planner Fits A deterministic planner is not a bad planner.\nIt is often the right first version.\nIf a project has a file such as deterministic_credit_card_planner.py, it usually means the planner is rule-based. It may use keywords, string matching, and explicit rules to detect user intent.\nFor example, it might support intents such as:\nbrowse lookup compare recommend And it may use helper functions like:\n_has_browse_intent() _has_lookup_intent() _has_compare_intent() _has_recommend_intent() Then it extracts parameters with helpers such as:\n_extract_lookup_query() _extract_compare_card_ids() _extract_reward_type() _mentions_no_annual_fee() This type of planner has clear advantages:\nStable Cheap Fast Easy to unit test Good for validating the workflow skeleton It also has clear limitations:\nWeak natural language understanding Rules grow over time Limited generalization Difficult to handle complex intents A deterministic planner is often a good MVP-stage implementation because it validates the architecture before introducing LLM complexity.\nIt also gives you a useful baseline. Even after adding an LLM planner, deterministic rules can continue handling frequent, obvious, low-risk intents.\nA Common Evolution Path A practical evolution path looks like this:\nPhase 1: Deterministic Planner Phase 2: LLM Planner Phase 3: Hybrid Planner Phase 4: Multi-step Autonomous Planner Phase 1: Deterministic Planner Use explicit rules to detect intent.\nThis is stable, testable, and good for building the foundation.\nPhase 2: LLM Planner Use an LLM to convert natural language into a structured PlanResult.\nThis makes the system more flexible and better at handling complex user expressions.\nPhase 3: Hybrid Planner Use deterministic rules first for frequent, obvious, low-risk requests.\nIf the rule-based planner returns no_plan, fall back to the LLM planner.\nplan_result = deterministic_planner.plan(context) if plan_result.type == \u0026#34;no_plan\u0026#34;: plan_result = llm_planner.plan(context) return plan_result This is a common production pattern:\nSimple requests use rules. Complex requests use the LLM. Phase 4: Multi-step Autonomous Planner The planner can eventually plan multiple steps instead of one tool call.\nFor example:\nFirst retrieve available cards Then filter by no annual fee Then compare reward categories Then generate a recommendation At this point, the planner becomes closer to a task decomposition engine.\nThat can be powerful, but it also increases the need for evaluation, guardrails, state management, and observability.\nThe Core Takeaway The planner is not the only place where an agentic workflow can connect to an LLM.\nA more accurate statement is:\nThe planner is often the most important LLM integration point because it decides what should happen next. But in a complete agentic workflow, LLMs can also be used for:\nArgument extraction Response generation Memory summarization Quality evaluation Safety classification Compliance checks A simple autonomous agent puts many of these responsibilities inside one repeated LLM loop.\nA production-style agent separates them into explicit modules.\nThe final mental model is:\nSimple Agent: LLM = planner + tool selector + argument generator + controller + response writer Production-style Agent: Runtime = controller Planner = decision maker ToolExecutor = executor Registry = capability provider Renderer = response writer Memory = state/history manager Evaluator/Guardrail = safety and quality control So when designing an agentic application, the better question is not only:\nWhere do we put the LLM? The better question is:\nWhat is this module responsible for? Does this responsibility actually require an LLM? If it does, what are the LLM\u0026#39;s inputs, outputs, boundaries, and failure handling strategy? That is the key shift from building demo agents to designing production-grade agentic workflows.\n\nA practical look at where LLMs belong in agentic workflows, and why production systems separate planning, execution, rendering, memory, evaluation, and guardrails.\nai llm agents software architecture","date":"2026-06-01T00:00:00Z","permalink":"/p/where-should-the-llm-live-in-an-agentic-workflow/","title":"Where Should the LLM Live in an Agentic Workflow?"},{"content":"When developers first hear the word \u0026ldquo;AI agent,\u0026rdquo; it can sound like a new magical runtime. In practice, the core mechanism is much simpler:\nAn agent is usually an LLM wrapped in a loop that can decide whether to call tools, execute those tools, observe the results, and continue until it can produce a final answer.\nModern frameworks like OpenAI Agents SDK and LangChain create_agent hide much of this loop from you. But underneath, the mental model is still close to a manual loop:\nUser input -\u0026gt; LLM call -\u0026gt; LLM emits tool call -\u0026gt; application executes tool -\u0026gt; tool result is appended back to context -\u0026gt; LLM call again -\u0026gt; final answer OpenAI\u0026rsquo;s function calling documentation describes this pattern directly: you provide tools to the model, the model may return a tool call, and your application sends the tool output back to the model with the corresponding call ID. (OpenAI Developers)\nIn this article, we will build the same small credit-card recommendation agent in three ways:\nManual Responses API + tool-calling loop OpenAI Agents SDK with @function_tool LangChain create_agent The goal is not to build a production credit-card engine. The goal is to understand what agent frameworks are actually abstracting away.\nThe Toy Domain: Credit Card Recommendation We will use two simple tools:\nget_card_catalog(spending_category) estimate_annual_rewards(card_name, annual_spend) The user asks something like:\nI spend about $12,000 per year on dining and travel. I prefer annual fee under $100. Which card should I consider? The agent should call tools to inspect mock product data and estimate rewards before giving a recommendation.\n1. Manual Agent Loop with OpenAI Responses API This is the lowest-level version. We manually define:\nTool schemas Python tool implementations Tool registry LLM call Function-call detection Tool execution Function-call-output append Step limit This is the most educational version because it shows what agent frameworks are doing behind the scenes.\n# manual_agent.py import json import os from typing import Any from openai import OpenAI client = OpenAI() def get_card_catalog(spending_category: str) -\u0026gt; dict[str, Any]: \u0026#34;\u0026#34;\u0026#34;Mock: search credit card catalog by spending category.\u0026#34;\u0026#34;\u0026#34; cards = { \u0026#34;dining\u0026#34;: [ { \u0026#34;name\u0026#34;: \u0026#34;Citi Premier\u0026#34;, \u0026#34;annual_fee\u0026#34;: 95, \u0026#34;reward\u0026#34;: \u0026#34;3x dining\u0026#34;, }, { \u0026#34;name\u0026#34;: \u0026#34;Custom Cash\u0026#34;, \u0026#34;annual_fee\u0026#34;: 0, \u0026#34;reward\u0026#34;: \u0026#34;5% top eligible category\u0026#34;, }, ], \u0026#34;travel\u0026#34;: [ { \u0026#34;name\u0026#34;: \u0026#34;Citi Premier\u0026#34;, \u0026#34;annual_fee\u0026#34;: 95, \u0026#34;reward\u0026#34;: \u0026#34;3x travel\u0026#34;, }, { \u0026#34;name\u0026#34;: \u0026#34;Premium Travel Card\u0026#34;, \u0026#34;annual_fee\u0026#34;: 395, \u0026#34;reward\u0026#34;: \u0026#34;5x flights\u0026#34;, }, ], \u0026#34;costco\u0026#34;: [ { \u0026#34;name\u0026#34;: \u0026#34;Costco Anywhere Visa\u0026#34;, \u0026#34;annual_fee\u0026#34;: 0, \u0026#34;reward\u0026#34;: \u0026#34;4% gas, 3% travel/dining, 2% Costco\u0026#34;, }, ], } return { \u0026#34;category\u0026#34;: spending_category, \u0026#34;cards\u0026#34;: cards.get(spending_category.lower(), []), } def estimate_annual_rewards(card_name: str, annual_spend: float) -\u0026gt; dict[str, Any]: \u0026#34;\u0026#34;\u0026#34;Mock: estimate annual reward value.\u0026#34;\u0026#34;\u0026#34; multiplier = { \u0026#34;Citi Premier\u0026#34;: 0.03, \u0026#34;Custom Cash\u0026#34;: 0.05, \u0026#34;Premium Travel Card\u0026#34;: 0.05, \u0026#34;Costco Anywhere Visa\u0026#34;: 0.02, }.get(card_name, 0.01) return { \u0026#34;card_name\u0026#34;: card_name, \u0026#34;annual_spend\u0026#34;: annual_spend, \u0026#34;estimated_reward_value\u0026#34;: round(annual_spend * multiplier, 2), } TOOLS = [ { \u0026#34;type\u0026#34;: \u0026#34;function\u0026#34;, \u0026#34;name\u0026#34;: \u0026#34;get_card_catalog\u0026#34;, \u0026#34;description\u0026#34;: \u0026#34;Search available credit cards by spending category.\u0026#34;, \u0026#34;parameters\u0026#34;: { \u0026#34;type\u0026#34;: \u0026#34;object\u0026#34;, \u0026#34;properties\u0026#34;: { \u0026#34;spending_category\u0026#34;: { \u0026#34;type\u0026#34;: \u0026#34;string\u0026#34;, \u0026#34;description\u0026#34;: \u0026#34;Spending category such as dining, travel, costco, gas, grocery.\u0026#34;, } }, \u0026#34;required\u0026#34;: [\u0026#34;spending_category\u0026#34;], \u0026#34;additionalProperties\u0026#34;: False, }, }, { \u0026#34;type\u0026#34;: \u0026#34;function\u0026#34;, \u0026#34;name\u0026#34;: \u0026#34;estimate_annual_rewards\u0026#34;, \u0026#34;description\u0026#34;: \u0026#34;Estimate annual reward value for a card and annual spend amount.\u0026#34;, \u0026#34;parameters\u0026#34;: { \u0026#34;type\u0026#34;: \u0026#34;object\u0026#34;, \u0026#34;properties\u0026#34;: { \u0026#34;card_name\u0026#34;: { \u0026#34;type\u0026#34;: \u0026#34;string\u0026#34;, \u0026#34;description\u0026#34;: \u0026#34;Name of the credit card.\u0026#34;, }, \u0026#34;annual_spend\u0026#34;: { \u0026#34;type\u0026#34;: \u0026#34;number\u0026#34;, \u0026#34;description\u0026#34;: \u0026#34;Estimated annual spend in USD.\u0026#34;, }, }, \u0026#34;required\u0026#34;: [\u0026#34;card_name\u0026#34;, \u0026#34;annual_spend\u0026#34;], \u0026#34;additionalProperties\u0026#34;: False, }, }, ] TOOL_IMPLS = { \u0026#34;get_card_catalog\u0026#34;: get_card_catalog, \u0026#34;estimate_annual_rewards\u0026#34;: estimate_annual_rewards, } def run_agent(user_message: str, max_steps: int = 5) -\u0026gt; str: input_items: list[dict[str, Any]] = [ { \u0026#34;role\u0026#34;: \u0026#34;user\u0026#34;, \u0026#34;content\u0026#34;: user_message, } ] for step in range(max_steps): response = client.responses.create( model=os.environ.get(\u0026#34;OPENAI_MODEL\u0026#34;, \u0026#34;gpt-5.4\u0026#34;), instructions=( \u0026#34;You are a credit card recommendation agent. \u0026#34; \u0026#34;Use tools when product or reward data is needed. \u0026#34; \u0026#34;Do not recommend applying for a card without explaining tradeoffs. \u0026#34; \u0026#34;If information is insufficient, ask a concise follow-up question.\u0026#34; ), tools=TOOLS, input=input_items, ) tool_calls = [ item for item in response.output if item.type == \u0026#34;function_call\u0026#34; ] if not tool_calls: return response.output_text # Preserve the model\u0026#39;s tool-call request in the conversation trajectory. input_items.extend(response.output) for tool_call in tool_calls: tool_name = tool_call.name tool_args = json.loads(tool_call.arguments) if tool_name not in TOOL_IMPLS: tool_result = {\u0026#34;error\u0026#34;: f\u0026#34;Unknown tool: {tool_name}\u0026#34;} else: try: tool_result = TOOL_IMPLS[tool_name](**tool_args) except Exception as exc: tool_result = { \u0026#34;error\u0026#34;: f\u0026#34;Tool execution failed: {type(exc).__name__}: {exc}\u0026#34; } input_items.append( { \u0026#34;type\u0026#34;: \u0026#34;function_call_output\u0026#34;, \u0026#34;call_id\u0026#34;: tool_call.call_id, \u0026#34;output\u0026#34;: json.dumps(tool_result), } ) return \u0026#34;I could not complete the task within the step limit.\u0026#34; if __name__ == \u0026#34;__main__\u0026#34;: answer = run_agent( \u0026#34;I spend about $12,000 per year on dining and travel. \u0026#34; \u0026#34;I prefer annual fee under $100. Which card should I consider?\u0026#34; ) print(answer) Run it:\nexport OPENAI_API_KEY=\u0026#34;your-api-key\u0026#34; export OPENAI_MODEL=\u0026#34;gpt-5.4\u0026#34; python manual_agent.py This version is close to the \u0026ldquo;bare metal\u0026rdquo; agent loop. OpenAI\u0026rsquo;s documentation defines tool calls as requests from the model to use tools and tool-call outputs as the application-generated outputs sent back to the model. (OpenAI Developers)\nThe key thing to notice is that the model does not execute Python functions by itself. It emits a structured request. Your application executes the function and sends the result back.\nWhat This Manual Version Teaches This manual version makes the agent architecture visible:\nAgent concept Manual code Tool definition TOOLS JSON schema Tool registry TOOL_IMPLS Model call client.responses.create(...) Tool call detection item.type == \u0026quot;function_call\u0026quot; Argument parsing json.loads(tool_call.arguments) Tool execution TOOL_IMPLS[tool_name](**tool_args) Observation / tool result function_call_output Agent loop for step in range(max_steps) Stop condition no tool calls returned This is why I like building a manual version first. It removes the mystery. Agent frameworks are not magic. They mostly package this loop into a reusable runtime.\n2. OpenAI Agents SDK Version Now let\u0026rsquo;s build the same thing using the OpenAI Agents SDK.\nOpenAI\u0026rsquo;s Agents SDK defines an Agent as an LLM configured with instructions, tools, and optional runtime behavior like handoffs, guardrails, and structured outputs. The SDK uses the Responses API by default for OpenAI models, and Agent plus Runner lets the SDK manage turns, tools, guardrails, handoffs, and sessions. (OpenAI GitHub Pages)\nInstall:\npip install openai-agents Code:\n# openai_agents_demo.py import os from typing import Any from agents import Agent, Runner, function_tool @function_tool def get_card_catalog(spending_category: str) -\u0026gt; dict[str, Any]: \u0026#34;\u0026#34;\u0026#34;Search available credit cards by spending category.\u0026#34;\u0026#34;\u0026#34; cards = { \u0026#34;dining\u0026#34;: [ { \u0026#34;name\u0026#34;: \u0026#34;Citi Premier\u0026#34;, \u0026#34;annual_fee\u0026#34;: 95, \u0026#34;reward\u0026#34;: \u0026#34;3x dining\u0026#34;, }, { \u0026#34;name\u0026#34;: \u0026#34;Custom Cash\u0026#34;, \u0026#34;annual_fee\u0026#34;: 0, \u0026#34;reward\u0026#34;: \u0026#34;5% top eligible category\u0026#34;, }, ], \u0026#34;travel\u0026#34;: [ { \u0026#34;name\u0026#34;: \u0026#34;Citi Premier\u0026#34;, \u0026#34;annual_fee\u0026#34;: 95, \u0026#34;reward\u0026#34;: \u0026#34;3x travel\u0026#34;, }, { \u0026#34;name\u0026#34;: \u0026#34;Premium Travel Card\u0026#34;, \u0026#34;annual_fee\u0026#34;: 395, \u0026#34;reward\u0026#34;: \u0026#34;5x flights\u0026#34;, }, ], \u0026#34;costco\u0026#34;: [ { \u0026#34;name\u0026#34;: \u0026#34;Costco Anywhere Visa\u0026#34;, \u0026#34;annual_fee\u0026#34;: 0, \u0026#34;reward\u0026#34;: \u0026#34;4% gas, 3% travel/dining, 2% Costco\u0026#34;, }, ], } return { \u0026#34;category\u0026#34;: spending_category, \u0026#34;cards\u0026#34;: cards.get(spending_category.lower(), []), } @function_tool def estimate_annual_rewards(card_name: str, annual_spend: float) -\u0026gt; dict[str, Any]: \u0026#34;\u0026#34;\u0026#34;Estimate annual reward value for a card and annual spend amount.\u0026#34;\u0026#34;\u0026#34; multiplier = { \u0026#34;Citi Premier\u0026#34;: 0.03, \u0026#34;Custom Cash\u0026#34;: 0.05, \u0026#34;Premium Travel Card\u0026#34;: 0.05, \u0026#34;Costco Anywhere Visa\u0026#34;: 0.02, }.get(card_name, 0.01) return { \u0026#34;card_name\u0026#34;: card_name, \u0026#34;annual_spend\u0026#34;: annual_spend, \u0026#34;estimated_reward_value\u0026#34;: round(annual_spend * multiplier, 2), } card_agent = Agent( name=\u0026#34;Credit Card Recommendation Agent\u0026#34;, model=os.environ.get(\u0026#34;OPENAI_MODEL\u0026#34;, \u0026#34;gpt-5.4\u0026#34;), instructions=( \u0026#34;You are a credit card recommendation agent. \u0026#34; \u0026#34;Use tools when product or reward data is needed. \u0026#34; \u0026#34;Do not recommend applying for a card without explaining tradeoffs. \u0026#34; \u0026#34;If information is insufficient, ask a concise follow-up question.\u0026#34; ), tools=[ get_card_catalog, estimate_annual_rewards, ], ) if __name__ == \u0026#34;__main__\u0026#34;: result = Runner.run_sync( card_agent, ( \u0026#34;I spend about $12,000 per year on dining and travel. \u0026#34; \u0026#34;I prefer annual fee under $100. Which card should I consider?\u0026#34; ), ) print(result.final_output) Run it:\nexport OPENAI_API_KEY=\u0026#34;your-api-key\u0026#34; export OPENAI_MODEL=\u0026#34;gpt-5.4\u0026#34; python openai_agents_demo.py The important difference is that we no longer manually write the TOOLS JSON schema. The @function_tool decorator wraps Python functions as tools. OpenAI\u0026rsquo;s Agents SDK documentation explicitly supports wrapping Python functions as function-calling tools. (OpenAI GitHub Pages)\nSo this:\n@function_tool def get_card_catalog(spending_category: str) -\u0026gt; dict[str, Any]: \u0026#34;\u0026#34;\u0026#34;Search available credit cards by spending category.\u0026#34;\u0026#34;\u0026#34; ... roughly replaces the manual work of writing this:\n{ \u0026#34;type\u0026#34;: \u0026#34;function\u0026#34;, \u0026#34;name\u0026#34;: \u0026#34;get_card_catalog\u0026#34;, \u0026#34;description\u0026#34;: \u0026#34;Search available credit cards by spending category.\u0026#34;, \u0026#34;parameters\u0026#34;: { ... }, } And Runner.run_sync(...) replaces the manual loop.\nConceptually:\nManual version: You own the loop. OpenAI Agents SDK: Agent + Runner own the loop. 3. LangChain create_agent Version Now let\u0026rsquo;s build the same agent using LangChain.\nLangChain\u0026rsquo;s create_agent combines a language model with tools. Its documentation says agents reason about tasks, decide which tools to use, and iteratively work toward solutions. It also states that create_agent provides a production-ready agent implementation and runs tools in a loop until a stop condition is met. (LangChain Docs)\nInstall:\npip install -U langchain langchain-openai Code:\n# langchain_create_agent_demo.py import os from typing import Any from langchain.agents import create_agent from langchain.tools import tool @tool def get_card_catalog(spending_category: str) -\u0026gt; dict[str, Any]: \u0026#34;\u0026#34;\u0026#34;Search available credit cards by spending category.\u0026#34;\u0026#34;\u0026#34; cards = { \u0026#34;dining\u0026#34;: [ { \u0026#34;name\u0026#34;: \u0026#34;Citi Premier\u0026#34;, \u0026#34;annual_fee\u0026#34;: 95, \u0026#34;reward\u0026#34;: \u0026#34;3x dining\u0026#34;, }, { \u0026#34;name\u0026#34;: \u0026#34;Custom Cash\u0026#34;, \u0026#34;annual_fee\u0026#34;: 0, \u0026#34;reward\u0026#34;: \u0026#34;5% top eligible category\u0026#34;, }, ], \u0026#34;travel\u0026#34;: [ { \u0026#34;name\u0026#34;: \u0026#34;Citi Premier\u0026#34;, \u0026#34;annual_fee\u0026#34;: 95, \u0026#34;reward\u0026#34;: \u0026#34;3x travel\u0026#34;, }, { \u0026#34;name\u0026#34;: \u0026#34;Premium Travel Card\u0026#34;, \u0026#34;annual_fee\u0026#34;: 395, \u0026#34;reward\u0026#34;: \u0026#34;5x flights\u0026#34;, }, ], \u0026#34;costco\u0026#34;: [ { \u0026#34;name\u0026#34;: \u0026#34;Costco Anywhere Visa\u0026#34;, \u0026#34;annual_fee\u0026#34;: 0, \u0026#34;reward\u0026#34;: \u0026#34;4% gas, 3% travel/dining, 2% Costco\u0026#34;, }, ], } return { \u0026#34;category\u0026#34;: spending_category, \u0026#34;cards\u0026#34;: cards.get(spending_category.lower(), []), } @tool def estimate_annual_rewards(card_name: str, annual_spend: float) -\u0026gt; dict[str, Any]: \u0026#34;\u0026#34;\u0026#34;Estimate annual reward value for a card and annual spend amount.\u0026#34;\u0026#34;\u0026#34; multiplier = { \u0026#34;Citi Premier\u0026#34;: 0.03, \u0026#34;Custom Cash\u0026#34;: 0.05, \u0026#34;Premium Travel Card\u0026#34;: 0.05, \u0026#34;Costco Anywhere Visa\u0026#34;: 0.02, }.get(card_name, 0.01) return { \u0026#34;card_name\u0026#34;: card_name, \u0026#34;annual_spend\u0026#34;: annual_spend, \u0026#34;estimated_reward_value\u0026#34;: round(annual_spend * multiplier, 2), } agent = create_agent( model=f\u0026#34;openai:{os.environ.get(\u0026#39;OPENAI_MODEL\u0026#39;, \u0026#39;gpt-5.4\u0026#39;)}\u0026#34;, tools=[ get_card_catalog, estimate_annual_rewards, ], system_prompt=( \u0026#34;You are a credit card recommendation agent. \u0026#34; \u0026#34;Use tools when product or reward data is needed. \u0026#34; \u0026#34;Do not recommend applying for a card without explaining tradeoffs. \u0026#34; \u0026#34;If information is insufficient, ask a concise follow-up question.\u0026#34; ), ) if __name__ == \u0026#34;__main__\u0026#34;: result = agent.invoke( { \u0026#34;messages\u0026#34;: [ { \u0026#34;role\u0026#34;: \u0026#34;user\u0026#34;, \u0026#34;content\u0026#34;: ( \u0026#34;I spend about $12,000 per year on dining and travel. \u0026#34; \u0026#34;I prefer annual fee under $100. Which card should I consider?\u0026#34; ), } ] } ) final_message = result[\u0026#34;messages\u0026#34;][-1] print(final_message.content) Run it:\nexport OPENAI_API_KEY=\u0026#34;your-api-key\u0026#34; export OPENAI_MODEL=\u0026#34;gpt-5.4\u0026#34; python langchain_create_agent_demo.py In LangChain, the @tool decorator converts a normal Python function into a tool object that agents can use. LangChain\u0026rsquo;s tools documentation shows that tool return values may be strings or structured objects such as dictionaries; structured objects are serialized and sent back as tool output for the model to inspect. (LangChain Docs)\nLangChain v1 also positions create_agent as the standard way to build agents, replacing the older langgraph.prebuilt.create_react_agent interface with a simpler agent entry point. (LangChain Docs)\nComparing the Three Versions 1. Manual Responses API Version The manual version gives you maximum control.\nYou explicitly control:\n- Tool schema - Tool registry - Tool execution - Tool-call parsing - Tool-call output format - Agent loop - Max steps - Error handling - Conversation trajectory This is useful when you want to deeply understand how tool calling works, or when your system has strict requirements that do not fit a framework abstraction.\nThe downside is that you must implement a lot of runtime behavior yourself.\n2. OpenAI Agents SDK Version The OpenAI Agents SDK removes boilerplate.\nYou define:\n- Agent instructions - Python tools - Optional model settings The SDK handles:\n- Tool schema generation - Tool-call loop - Tool execution - Turn management - Optional sessions - Optional guardrails - Optional handoffs The OpenAI Agents SDK is a good fit when your application primarily uses OpenAI models and you want a clean, official orchestration layer.\n3. LangChain create_agent Version LangChain gives you a broader provider-agnostic abstraction.\nYou define:\n- Model provider string - Tools - System prompt LangChain handles:\n- Tool wrapping - Agent loop - Provider-specific model/tool formatting - Middleware - LangGraph-based execution - LangSmith tracing integration LangChain\u0026rsquo;s docs state that create_agent builds a graph-based agent runtime using LangGraph, where the agent moves through nodes such as the model node, tools node, and middleware. (LangChain Docs)\nLangChain is a better fit when you want to switch across model providers, add middleware, build more complex workflows, or eventually move into LangGraph-style orchestration.\nThe Core Mental Model All three versions implement the same conceptual loop.\n1. Send user input and available tools to the model. 2. Model decides whether it can answer directly. 3. If not, model emits a tool call. 4. Application/framework executes the tool. 5. Tool result is sent back to the model. 6. Model continues until it emits a final answer. The main difference is who owns the loop.\nVersion Who owns the loop? Who defines tool schema? Best for Manual Responses API You You Learning, maximum control, custom runtime OpenAI Agents SDK OpenAI Runner @function_tool helps generate it OpenAI-native agent apps LangChain create_agent LangChain/LangGraph runtime @tool helps generate it Multi-provider agent workflows \u0026ldquo;Is This ReAct?\u0026rdquo; Yes, but with a modern caveat.\nThe original ReAct pattern was often shown as:\nThought -\u0026gt; Action -\u0026gt; Observation -\u0026gt; Thought -\u0026gt; Action -\u0026gt; Observation -\u0026gt; Final Answer Modern tool-calling agents usually do not need to expose this exact text format. Instead, the loop is structured:\nModel output -\u0026gt; tool call Tool execution -\u0026gt; tool result Model output -\u0026gt; final answer So it is better to call this a ReAct-style tool loop rather than assuming the model literally emits Thought, Action, and Observation strings.\nWhy Agent Frameworks Exist The manual loop is easy to understand but quickly becomes hard to maintain.\nA production agent runtime usually needs:\n- Tool argument validation - Tool timeout handling - Tool retry logic - Tool permission checks - Human approval gates - Memory/session management - Conversation summarization - Token budget control - Tool selection/gating - Streaming - Tracing - Evaluation - Structured final output validation - Multi-agent handoffs This is why frameworks exist.\nOpenAI Agents SDK is useful when you want OpenAI-native ergonomics around agents, tools, guardrails, handoffs, and sessions. LangChain is useful when you want a provider-agnostic agent abstraction with middleware and LangGraph underneath.\nBut the underlying idea remains simple:\nThe model decides what tool it wants. The runtime executes that tool. The result goes back into the model. Repeat until done.\nFinal Takeaway If you are learning agentic AI, start with the manual version.\nIt teaches you the real mechanics:\nLLM + tools + loop + state = agent runtime Then move to OpenAI Agents SDK or LangChain once you understand what they abstract away.\nMy personal recommendation:\nUse the manual version to learn. Use OpenAI Agents SDK for simple OpenAI-native production agents. Use LangChain / LangGraph when you need provider flexibility, middleware, durable state, or more complex workflows. The frameworks are helpful, but they are not magic. They are structured ways to implement the same loop you can write by hand.\n\u0026mdash;proudly generated by my best buddy ChatGPT：）\n\nA practical comparison of manual tool-calling loops, OpenAI Agents SDK, and LangChain create_agent.\nai agents engineering openai","date":"2026-05-25T00:00:00Z","permalink":"/p/from-manual-tool-calling-to-agent-frameworks-openai-agents-sdk-vs.-langchain-create_agent/","title":"From Manual Tool Calling to Agent Frameworks: OpenAI Agents SDK vs. LangChain create_agent"},{"content":"We\u0026rsquo;ve come a long way in how we build software.\nWaterfall said: \u0026ldquo;Let\u0026rsquo;s build the whole car, piece by piece, in order. Nobody touches the steering wheel until the chassis is done.\u0026rdquo; Technically correct. Emotionally devastating.\nAgile said: \u0026ldquo;Ship a skateboard first. Then a scooter. Then a bike. Eventually you\u0026rsquo;ll get a car.\u0026rdquo; The user got to ride something on day one. Management got to feel iterative. Everyone wins — sort of.\nAI said: \u0026ldquo;Here\u0026rsquo;s a car. It has a propeller, a satellite dish, and what appears to be a small man living inside it. But it runs. Probably.\u0026rdquo;\nThe real question isn\u0026rsquo;t which methodology is best. It\u0026rsquo;s whether you\u0026rsquo;re brave enough to ship the green car.\n\nFrom wheels to whatever AI just built.\nengineering humor","date":"2026-04-07T00:00:00Z","image":"/p/the-evolution-of-development/cover.jpg","permalink":"/p/the-evolution-of-development/","title":"The Evolution of Development"},{"content":"System Builder I design and ship systems end to end — architecture, integration, and operations.\nSystem Modeling Breaking complex problems into architecture. Defining modules, boundaries, and contracts so that each piece can evolve independently.\nIntegration Connecting things that don\u0026rsquo;t naturally talk to each other — mobile to backend, AI to product, multi-system orchestration. The value is in the seams.\nProduction Judgment Rollout strategies, fallback plans, and the metrics that matter. Knowing what it takes to run a system well.\nAbstraction \u0026amp; Narrative Defining the right problem before solving it. Communicating system design clearly — to engineers, to stakeholders, to future maintainers.\n\nSystem Builder — modeling complexity, integrating across boundaries, shipping end to end.","date":"2026-01-01T00:00:00Z","permalink":"/about/","title":"About"},{"content":"","date":"2026-01-01T00:00:00Z","permalink":"/archives/","title":"Archives"},{"content":"Apollo 2 — Generic Recommendation Engine (2026–Present) GitHub\nA domain-agnostic recommendation chatbot powered by FastAPI and Next.js. Swap a catalog JSON and adapter to recommend any item type — credit cards, retail products, or services.\nArchitecture Backend: FastAPI + Pydantic v2 — generic CatalogItem model with pluggable domain adapters, scoring weights, and router patterns Frontend: Next.js 15 + Tailwind CSS v4 — theme, labels, and display fields driven by domain configuration Adapters: One file per domain handles catalog ingestion and field mapping Key Design Decisions Domain-agnostic core: The engine has no hardcoded knowledge of any specific domain; all domain specifics live in adapter + config files Pluggable catalog: Add a new domain by writing one adapter file, one catalog JSON, and one config — no engine changes required Production-oriented patterns: Structured endpoints, operations orchestration, and environment-based configuration \nPersonal projects and technical explorations.","date":"2026-01-01T00:00:00Z","permalink":"/projects/","title":"Projects"}]