Workflow vs Agent for LLM Integration: Start With a Workflow
Key takeaways
- Start with a workflow for most business LLM integrations.
- If the tool order and stop condition are known, you want a workflow, not an agent.
- A fixed workflow with one LLM step is the usual sweet spot for support triage, CRM note normalization, and docs Q and A.
- Use an agent only when the system must decide what to do next based on what it learns during the run.
- As autonomy increases, cost variance, approval design, tracing, and failure handling all get harder.
Start with a workflow for most business LLM integrations. Use a full agent only when the system must plan, choose tools, and adapt its next step across multiple tools or ambiguous inputs.
That rule keeps cost, logging, approval, and failure handling under control. It also matches the way OpenAI documentation separates fixed workflows from more autonomous agent patterns.
Most teams should start with the smallest pattern that works
Most SaaS use cases fit one of three patterns, and two are simpler than a full agent.
- Deterministic workflow: Use this when the rules, step order, and stop condition are known.
- Workflow with one LLM step: Use this when one part needs interpretation, but the overall path is fixed.
- Full agent: Use this when the system must decide what to do next, pick tools, and branch based on new findings.
If you can describe the process as a fixed SOP, do not start with an agent. If the only fuzzy part is classification, extraction, summarization, or drafting, keep the workflow and add one model step.
Use a deterministic workflow when the rules are already known
If the step order and stop condition are known, use code and rules, not an agent.
Support routing is a common example. If routing depends on fields you already have, such as account tier, product area, plan, or incident status, encode the logic and stop there.
CRM enrichment often fits the same pattern. If the job is pulling data from known systems, validating fields, mapping values into a schema, or syncing state between tools, a deterministic workflow is easier to test and replay.
Internal docs search can also start here. If the requirement is to search approved documents and return matching links, retrieval and ranking may be enough.
The advantage is control. Every branch is explicit. Every write is deliberate. Retries are simple because the state machine belongs to your code.
Add one LLM step when only interpretation is fuzzy
If one part needs language understanding but the path is fixed, keep the workflow and insert one model step.
Support triage is a good fit. The workflow can fetch the ticket, account metadata, and recent events. The LLM can classify issue type, urgency, or likely owner. The workflow still decides what happens next, including when to send the case to a manual queue.
CRM note normalization is another good fit. The workflow can collect call notes, form submissions, and email snippets. The LLM can extract intent, summarize context, and map messy text into your schema. Your code should validate required fields before writing anything back.
Internal docs Q and A often fits this pattern too. Retrieval gets the approved sources first. The LLM answers using only those sources and a fixed response format. The workflow can require citations, block unsupported claims, and fall back to no answer when the evidence is weak.
This is where many teams get the best tradeoff. You get language understanding without giving the model open-ended control over tools or process order.
Use an agent only when the system must decide what to do next
An agent is justified only when the next tool or next step depends on what the system learns during the run.
A complex support investigation can justify it. A request might require the system to inspect billing changes, product events, deploy history, incident records, and internal runbooks in different orders depending on what each step reveals.
Some sales operations tasks can also cross the line. If the system must compare product usage, inspect open support issues, draft next actions, and create follow-up tasks based on what it finds, an agent may be the right pattern.
Basic support triage is usually not agent territory. Normal CRM enrichment is usually not agent territory. Internal docs search is usually not agent territory unless it becomes multi-system research with follow-up actions.
Agents are useful for exploratory work. They are a poor default for repetitive work.
Expect more control work as you move toward agents
Agents raise cost variance and make approval, tracing, and failure handling harder.
- Cost: Deterministic workflows are predictable because the steps are known. A workflow with one LLM step is still bounded. An agent can loop, call tools repeatedly, and expand work based on intermediate results.
- Observability: Deterministic workflows are easiest to log. A workflow with one LLM step needs prompt, response, validation, and fallback logs. An agent needs traces for planning, tool selection, tool outputs, retries, loop counts, and stop reasons.
- Approval gates: Keep human approval before meaningful side effects such as CRM updates, outbound messages, refunds, ticket closures, or task creation. The more autonomy you add, the tighter the approval gates should be.
- Failure handling: Deterministic workflows usually fail in known places. A workflow with one LLM step needs confidence thresholds and fallback paths. An agent needs tool allowlists, step caps, timeouts, and a clear way to surface partial work.
If you cannot explain how the system fails, do not ship it.
Build in this order: workflow, one LLM step, then agent
The safe build order is deterministic workflow first, then a fixed workflow with one LLM step, then an agent if the fixed path still breaks.
- Write the current manual process as a fixed SOP.
- Automate the non-LLM steps first.
- Add one LLM step only where interpretation is the real bottleneck.
- Measure accuracy, latency, fallback rate, approval rate, and write success.
- Move to an agent only if the task still needs dynamic planning or tool choice.
A simple test helps. If you can predefine the tool order and stop condition, you probably want a workflow. If you cannot predefine them without losing the point of the task, an agent may be justified.
Boltout is a software agency.
If you want a second opinion on one use case, ask Boltout for a short call to scope a single workflow and decide whether it should stay deterministic, get one LLM step, or move to an agent.
Sources
Frequently asked questions
Usually no. Support triage usually works better as a fixed workflow with one LLM step for classification, followed by deterministic routing and a manual fallback path.
Start with a bounded workflow such as support triage, CRM note normalization, or internal docs Q and A. These use cases have clear inputs, visible outputs, and easy fallback rules.
Keep approval before actions that change business records or contact customers. That includes CRM writes, outbound emails, refunds, ticket closures, and task creation in core systems.
Yes. Many strong first use cases keep system access in the workflow, not in the model. The workflow fetches approved inputs, and the LLM handles classification, extraction, summarization, or drafting inside a controlled step.
Written by
Ready to add AI to your product?
We integrate LLMs and automation into real workflows, bounded pilots, clear metrics, sensible fallbacks when the model is wrong.
Discuss your project