18.1 Prompts vs Workflow Graphs

The Limitation of Stateless Prompts

A stateless prompt interaction follows a simple pattern where you send a request, receive a response, and start fresh on the next interaction. This works well for discrete tasks like translation, summarization, or simple question answering. However, it breaks down when tasks require context continuity across multiple exchanges where the system must remember prior conversation history, intermediate state that accumulates during problem solving, conditional logic based on previous outcomes, and coordination between multiple AI capabilities.

When to Choose Prompts vs Workflows

Complexity differs significantly between the two approaches. Stateless prompts handle single tasks with one response, while workflow graphs accommodate multi-step stateful processes.

Context handling follows similar logic. Prompts require all context bundled in a single prompt, whereas workflow graphs accumulate context across multiple interactions.

Failure recovery demonstrates the resilience advantage of workflow graphs. Stateless prompts require retrying the entire operation when something fails, but workflow graphs can resume from the last checkpoint, preserving intermediate progress.

Human oversight integrates differently as well. Prompts typically require minimal or no human intervention, while workflow graphs support approval gates and interruption points for human review.

Latency tolerance varies with architecture. Stateless prompts demand low latency since they involve a single call, while workflow graphs with sequential calls typically have higher tolerance for latency.

Cost predictability also differs. Stateless prompts are easier to estimate since costs are straightforward per-call calculations, while workflow graph costs vary based on the execution path taken through the graph.

Workflow Graph Fundamentals

A workflow graph represents your AI application as a directed graph where nodes are operations (AI calls, tools, human tasks) and edges define the flow between them. Key concepts include:

# Simple workflow graph representation
class WorkflowGraph:
    def __init__(self):
        self.nodes = {}
        self.edges = []
        self.state = {}
    
    def add_node(self, name, operation):
        self.nodes[name] = operation
    
    def add_edge(self, from_node, to_node, condition=None):
        self.edges.append({
            'from': from_node,
            'to': to_node,
            'condition': condition  # Optional routing function
        })
    
    def execute(self, start_node, initial_input):
        current_node = start_node
        while current_node:
            result = self.nodes[current_node](self.state, initial_input)
            current_node = self.route(result)
        return self.state

# DataForge example: document processing workflow
def document_processing_workflow():
    wf = WorkflowGraph()
    
    wf.add_node('classify', classify_document)
    wf.add_node('extract', extract_entities)
    wf.add_node('validate', validate_extraction)
    wf.add_node('store', store_results)
    wf.add_node('notify', send_notification)
    
    wf.add_edge('classify', 'extract')
    wf.add_edge('extract', 'validate')
    wf.add_edge('validate', 'store', condition=lambda s: s['valid'])
    wf.add_edge('validate', 'notify', condition=lambda s: not s['valid'])
    wf.add_edge('store', 'notify')
    
    return wf
            

Benefits of Structured Workflow Graphs

Predictability and Testability

Workflow graphs are explicit about execution paths, making it possible to test each node independently and verify edge conditions. You can enumerate all possible paths through your application.

Observability

Each node can emit logs, metrics, and traces. You know exactly where a workflow is executing and can debug failures with precision.

Resumability

State is preserved at checkpoints. If a workflow fails midway, you can resume from the last successful checkpoint rather than restarting entirely.

Human-in-the-Loop

Approval gates and interruption points integrate naturally. Human review becomes another node type in the graph.

Composability

Sub-workflows can be encapsulated and reused. Common patterns like approval-then-continue become reusable components.

Hybrid Approaches

Most production AI applications benefit from combining prompts and workflows. Use prompts for flexible natural language generation within well-defined boundaries, rapid prototyping before committing to workflow structure, and components that genuinely benefit from maximum flexibility. Use workflow graphs for critical business logic that must execute consistently, multi-step processes with state dependencies, integration points requiring human oversight, and error recovery and retry logic.