Simple agent demos often make the LLM look like the entire system. Production agentic workflows require a more careful question: which module owns which responsibility, and does that responsibility actually need an LLM?

When people first learn about agentic workflows or autonomous agents, one question comes up very quickly:

Is the planner the only place where an agent application connects to an LLM?

The question is understandable.

Many agent demos are implemented as a simple loop: call the LLM, let it decide the next action, execute a tool, append the result back into the context, and repeat until the model produces a final answer.

That pattern works. It is also the foundation of many ReAct-style agents, function-calling agents, and autonomous agent demos.

But it can create a misleading mental model.

If the LLM can drive the whole loop by itself, why do we need a separate Planner abstraction? What does the planner actually own? And in a more production-oriented system, where else might the LLM appear?

This post explains that distinction.

The short version is:

The planner is often the most important LLM integration point, but it is not the only one.

A simple agent loop pushes many responsibilities into repeated LLM calls. A production-grade agentic workflow separates those responsibilities into explicit modules.

The Simplest Autonomous Agent Is a Loop

A minimal autonomous agent often looks like this:

state = {
    "messages": [],
    "tool_results": [],
}

for step in range(max_steps):
    llm_response = llm.call(state)

    if llm_response.type == "tool_call":
        result = tool_executor.execute(llm_response.tool_call)
        state["tool_results"].append(result)
        state["messages"].append(result)

    elif llm_response.type == "final_answer":
        return llm_response.content

The flow is straightforward:

LLM decides what to do next
  -> LLM selects a tool
  -> System executes the tool
  -> Tool result is added back to the context
  -> LLM reasons again
  -> Repeat until a final answer is produced

In this model, the LLM is doing many jobs at once.

It understands the user’s goal, chooses the next action, selects a tool, generates tool arguments, decides whether to continue, and eventually writes the final response.

In other words, the LLM implicitly acts as:

Planner
Router
Tool argument generator
Controller
Response writer

This is why simple agent demos can be implemented in very little code. The code is small because many responsibilities are hidden inside the LLM call.

That is not necessarily wrong. It is a useful prototype pattern.

But as the system grows, the hidden responsibilities become harder to test, debug, constrain, and observe.

What a Planner Actually Does

In a more engineered agentic workflow, we usually separate those responsibilities.

The Planner has one core job:

Given the user input, current state, available tools, and relevant context, decide what should happen next.

A planner answers questions such as:

What is the user's intent?
Should the system call a tool?
Which tool should be called?
What arguments should be passed to that tool?
Does the system need to ask a clarification question?
Can the system already produce a final answer?

The planner should not execute tools. It should not own the tool registry. It should not manage the entire application state. It should not be responsible for rendering the final user-facing response.

Its job is decision-making.

A useful mental model is:

Planner = decision maker

The planner receives a curated planning context, such as:

Current user message
Conversation summary
Working memory
Available tool specifications
Current task state
Business constraints

And it returns a structured decision.

For example:

PlanResult(
    tool_call=ToolCall(
        tool_name="compare_credit_cards",
        arguments={
            "card_ids": ["citi_double_cash", "citi_custom_cash"]
        }
    )
)

Or:

PlanResult(
    clarification_question="Do you care more about cash back or travel rewards?"
)

Or:

PlanResult(
    final_answer="Based on your preferences, I recommend..."
)

The important detail is that the planner output should be explicit and structured. That makes the rest of the system easier to reason about.

AgentRuntime Owns the Loop

The planner is not the entire agent loop.

In a production-style design, the loop belongs to something like AgentRuntime.

A useful mental model is:

AgentRuntime = controller
Planner = decision maker
ToolExecutor = executor
ToolRegistry = capability provider
Renderer = response writer
Memory = state and history manager
Evaluator / Guardrail = quality and safety control

The runtime handles orchestration.

It receives user input, updates state, builds the planner context, calls the planner, executes tools if needed, writes results back into state, and returns a response.

A simplified runtime might look like this:

def run(user_input: str) -> AgentResponse:
    state.add_user_message(user_input)

    planner_context = planner_context_builder.build(state)

    plan_result = planner.plan(planner_context)

    tool_result = None

    if plan_result.tool_call:
        tool_result = tool_executor.execute(plan_result.tool_call)
        state.add_tool_result(tool_result)

    response = response_renderer.render(
        state=state,
        plan_result=plan_result,
        tool_result=tool_result,
    )

    return response

The runtime controls the workflow.

The planner is one step inside that workflow.

This separation matters because it gives the application clear ownership boundaries. The planner decides. The runtime orchestrates. The executor executes. The renderer writes. The memory layer stores. The guardrail layer checks.

The Planner Is a Natural Place for an LLM

The planner is often the most natural place to use an LLM because planning is a semantic decision-making problem.

Suppose the user says:

I want a no annual fee card for groceries and travel.

The system needs to understand that:

The user wants a credit card recommendation.
The user prefers no annual fee.
The user cares about groceries.
The user also cares about travel.
The system may need to call a recommendation tool.

A rule-based planner might look for keywords:

def has_recommend_intent(text: str) -> bool:
    return "recommend" in text or "best card" in text or "which card" in text

But natural language is flexible. The user could also say:

What should I use when I buy groceries and book flights?

That sentence may not contain the word recommend, but the intent is still clear.

This is where an LLM planner becomes useful.

It can convert natural language into a structured planning decision:

{
  "action": "call_tool",
  "tool_name": "recommend_credit_card",
  "arguments": {
    "reward_categories": ["groceries", "travel"],
    "annual_fee_preference": "no_annual_fee"
  }
}

This is why many systems evolve from deterministic planners to LLM-based planners.

The LLM is not just generating text. It is mapping flexible human language into a structured decision that the rest of the system can execute.

The LLM Can Also Live Outside the Planner

The planner is important, but it is not the only possible LLM integration point.

In a mature agentic application, different modules may use LLMs for different reasons.

Argument Extraction

Sometimes the planner should not do everything.

The planner may only decide:

intent = recommend_credit_card

Then a separate module extracts the arguments:

reward_type = travel
annual_fee = no_fee
spending_category = groceries

This module might be called:

ArgumentExtractor
SlotFiller
EntityExtractor

For example:

intent = planner.detect_intent(user_message)

if intent == "recommend_card":
    arguments = argument_extractor.extract(
        user_message=user_message,
        schema=RecommendCardInputSchema,
    )

This separation makes the system easier to test:

Planner decides what to do.
ArgumentExtractor decides how to fill the required inputs.

Response Rendering

Tools should usually return structured data rather than long free-form text.

For example, a credit card lookup tool might return:

{
  "card_name": "Citi Custom Cash",
  "annual_fee": "$0",
  "reward": "5% cash back on your top eligible spend category",
  "best_for": ["groceries", "gas", "restaurants"]
}

The system then needs to turn that structured result into a user-facing response.

That rendering step can be done with templates, or it can be done with an LLM.

In regulated domains such as banking, finance, healthcare, or law, it is risky to let an LLM freely generate the final answer without constraints.

A safer design is:

Structured tool result
  -> Controlled template or constrained LLM rendering
  -> Final user-facing response

The LLM can participate in response generation, but it should be grounded in structured data and bounded by clear rules.

Memory Summarization

Agent applications often need to remember user preferences across turns.

After several interactions, the system might summarize:

User prefers no annual fee credit cards.
User often compares Costco and American Airlines cards.
User values travel rewards but dislikes complicated redemption rules.

An LLM can help produce this summary:

summary = memory_summarizer.summarize(
    conversation_history=state.messages,
    latest_tool_results=state.tool_results,
)

But a memory summarizer should not save everything.

It should preserve information that is useful for future interactions:

Preferences
Constraints
Goals
Previous choices
Relevant business context

It should avoid storing casual or short-lived details that do not help the application serve the user later.

Evaluation and Critique

More advanced agents may include an evaluator or critic.

This module checks whether the current result satisfies the user’s goal:

Did the answer address the user's question?
Is more tool usage needed?
Is the answer grounded in tool results?
Is there a hallucination risk?
Did the system violate a business rule?

For example:

evaluation = evaluator.evaluate(
    user_goal=user_goal,
    current_answer=draft_answer,
    tool_results=tool_results,
)

if evaluation.needs_more_work:
    continue_loop()
else:
    return final_answer

This pattern is common in evaluator-optimizer workflows, self-reflection agents, research agents, and long-horizon autonomous tasks.

The benefit is higher answer quality. The tradeoff is higher cost, more latency, and more system complexity.

Guardrails and Classification

In production systems, especially in financial, healthcare, legal, or enterprise domains, an agent cannot only ask whether it can answer a question.

It also needs to ask whether it should answer.

A guardrail classifier may determine:

Does this request involve financial advice?
Does this request involve private information?
Does this answer need a compliance disclaimer?
Does this action require user confirmation?
Should the request be refused?

These checks can be rule-based, LLM-based, or hybrid.

For example:

User asks about credit card benefits
  -> Safe to answer

User asks whether they should borrow money to invest in a stock
  -> Requires risk warning or refusal of direct advice

User asks to access unauthorized account data
  -> Refuse or require authorization

So in a production-grade agentic workflow, LLM integration may appear not only in the planner but also in safety, compliance, classification, and evaluation layers.

Simple Agent vs. Production-Style Agent

A simple autonomous agent looks like this:

while not done:
    LLM decides next action
    execute tool
    append observation
return final answer

In this design:

LLM = planner + tool selector + argument generator + controller + response writer

This design has real advantages:

Easy to implement
Flexible
Good for demos
Good for quick prototypes

But it also has limitations:

Responsibilities are mixed together
Hard to test
Hard to debug
Easy to misuse tools
Hard to enforce permissions
Hard to enforce compliance
Hard to run deterministic regression tests

A production-style agent separates the responsibilities:

AgentRuntime
  -> Planner
  -> ToolExecutor
  -> Memory / State
  -> Renderer
  -> Evaluator / Guardrail

This design has different advantages:

Clear ownership boundaries
Modules are testable
Components are replaceable
Decisions are easier to log
Permissions are easier to control
The system is easier to observe and audit

The tradeoff is more engineering complexity.

But that complexity buys you something important: predictable behavior in a system where LLM calls are inherently probabilistic.

Why Not Let the LLM Do Everything?

You can.

Many demos are built exactly that way.

The simple loop is often the right place to start when you are exploring an idea.

But once the system moves into a real business environment, the simple loop starts to show its limits.

The hard questions become:

How do we guarantee that a specific intent calls the correct tool?
How do we limit the number of tool calls?
How do we debug why the LLM selected the wrong tool?
How do we implement fallback behavior?
How do we log every decision step?
How do we run regression tests?
How do we prevent the LLM from seeing internal state it should not see?
How do we guarantee a stable tool argument schema?
How do we separate planning from response rendering?
How do we enforce authorization and auditability?

That is why production systems usually do not put every responsibility inside a single repeated LLM loop.

A better approach is to separate the workflow into controlled modules:

Planning
Execution
State
Memory
Rendering
Evaluation
Guardrails

The LLM can participate in some of those modules, but it should not control everything without boundaries.

Where a Deterministic Planner Fits

A deterministic planner is not a bad planner.

It is often the right first version.

If a project has a file such as deterministic_credit_card_planner.py, it usually means the planner is rule-based. It may use keywords, string matching, and explicit rules to detect user intent.

For example, it might support intents such as:

browse
lookup
compare
recommend

And it may use helper functions like:

_has_browse_intent()
_has_lookup_intent()
_has_compare_intent()
_has_recommend_intent()

Then it extracts parameters with helpers such as:

_extract_lookup_query()
_extract_compare_card_ids()
_extract_reward_type()
_mentions_no_annual_fee()

This type of planner has clear advantages:

Stable
Cheap
Fast
Easy to unit test
Good for validating the workflow skeleton

It also has clear limitations:

Weak natural language understanding
Rules grow over time
Limited generalization
Difficult to handle complex intents

A deterministic planner is often a good MVP-stage implementation because it validates the architecture before introducing LLM complexity.

It also gives you a useful baseline. Even after adding an LLM planner, deterministic rules can continue handling frequent, obvious, low-risk intents.

A Common Evolution Path

A practical evolution path looks like this:

Phase 1: Deterministic Planner
Phase 2: LLM Planner
Phase 3: Hybrid Planner
Phase 4: Multi-step Autonomous Planner

Phase 1: Deterministic Planner

Use explicit rules to detect intent.

This is stable, testable, and good for building the foundation.

Phase 2: LLM Planner

Use an LLM to convert natural language into a structured PlanResult.

This makes the system more flexible and better at handling complex user expressions.

Phase 3: Hybrid Planner

Use deterministic rules first for frequent, obvious, low-risk requests.

If the rule-based planner returns no_plan, fall back to the LLM planner.

plan_result = deterministic_planner.plan(context)

if plan_result.type == "no_plan":
    plan_result = llm_planner.plan(context)

return plan_result

This is a common production pattern:

Simple requests use rules.
Complex requests use the LLM.

Phase 4: Multi-step Autonomous Planner

The planner can eventually plan multiple steps instead of one tool call.

For example:

First retrieve available cards
Then filter by no annual fee
Then compare reward categories
Then generate a recommendation

At this point, the planner becomes closer to a task decomposition engine.

That can be powerful, but it also increases the need for evaluation, guardrails, state management, and observability.

The Core Takeaway

The planner is not the only place where an agentic workflow can connect to an LLM.

A more accurate statement is:

The planner is often the most important LLM integration point because it decides what should happen next.

But in a complete agentic workflow, LLMs can also be used for:

Argument extraction
Response generation
Memory summarization
Quality evaluation
Safety classification
Compliance checks

A simple autonomous agent puts many of these responsibilities inside one repeated LLM loop.

A production-style agent separates them into explicit modules.

The final mental model is:

Simple Agent:
LLM = planner + tool selector + argument generator + controller + response writer

Production-style Agent:
Runtime = controller
Planner = decision maker
ToolExecutor = executor
Registry = capability provider
Renderer = response writer
Memory = state/history manager
Evaluator/Guardrail = safety and quality control

So when designing an agentic application, the better question is not only:

Where do we put the LLM?

The better question is:

What is this module responsible for?
Does this responsibility actually require an LLM?
If it does, what are the LLM's inputs, outputs, boundaries, and failure handling strategy?

That is the key shift from building demo agents to designing production-grade agentic workflows.