Tool Calling (Deep Workflow)

Tool calling is contract design between a probabilistic planner (the model) and deterministic systems. Failures are usually schema, permissions, or ambiguity—not the LLM “being dumb.”

When to Offer This Workflow

Trigger conditions:

• Designing OpenAI/Anthropic-style functions, MCP tools, or internal JSON tool protocols
• Debugging wrong arguments, hallucinated parameters, or unsafe side effects
• Building agents with many tools—selection and routing problems

Initial offer:

Use six stages: (1) define tool surface, (2) schema & validation, (3) authz & safety, (4) execution semantics, (5) errors & observability, (6) evaluation & regression. Confirm side-effect class (read-only vs write).

Stage 1: Define Tool Surface

Goal: Minimize tools; maximize clarity per tool.

Principles

• One action per tool when possible—avoid mega-tools with mode flags unless necessary
• Names descriptive: search_orders not do_stuff
• Prefer idempotent operations where writes exist; separate read vs write clearly

Anti-patterns

• Exposing raw SQL or shell to the model
• Too many overlapping tools → routing errors

Exit condition: Tool list with purpose, inputs, outputs, side effects table.

Stage 2: Schema & Validation

Goal: Arguments are typed, constrained, and machine-validated before execution.

Practices

• JSON Schema: enums, min/max, patterns, required fields
• Normalize dates, IDs, currencies server-side—never trust model formatting alone
• Default behaviors explicit in description + schema

Descriptions

• Tool and parameter docstrings seen by model—precise language; examples of valid args

Exit condition: Validator rejects invalid args with actionable errors back to model or orchestrator.

Stage 3: Authorization & Safety

Goal: Every tool call runs as some principal with least privilege.

Patterns

• User-scoped credentials carried from session; tool implementation re-checks ownership (e.g., order_id belongs to user)
• Admin tools behind explicit allowlists and human approval when needed
• Rate limits per user + global circuit breakers

Data exfiltration

• Tools that read sensitive data need output filtering and logging policies

Exit condition: Threat brief: “What if model is tricked into calling tool X?” answered.

Stage 4: Execution Semantics

Goal: Clear transactionality, retries, and idempotency.

Design

• Idempotency keys for writes; dedupe window
• Timeouts and cancellation propagation
• Ordering: parallel safe vs must be serial

Long operations

• Async jobs with poll tool vs blocking calls—prefer non-blocking for UX and cost

Exit condition: Semantics documented for retry behavior (at-least-once delivery common).

Stage 5: Errors & Observability

Goal: Model (or orchestrator) can recover from failures without leaking internals.

Error messages

• Structured error codes: ORDER_NOT_FOUND, PERMISSION_DENIED
• Hints for model on how to fix—without stack traces to end users

Observability

• Trace IDs across tool calls; audit log for write tools (who/when/args hash)

Exit condition: Dashboards/alerts on tool error rate, latency, denials.

Stage 6: Evaluation & Regression

Goal: Tool changes are tested like APIs.

Harness

• Golden conversations with expected tool calls (args normalized)
• Adversarial prompts attempting privilege escalation
• Version tools; deprecate with compatibility window

Exit condition: CI or manual eval suite before deploying new tools/schemas.

Final Review Checklist

• [ ] Minimal orthogonal tool set
• [ ] Strict schema validation on server
• [ ] AuthZ enforced per call; sensitive reads controlled
• [ ] Idempotency and timeouts defined for writes
• [ ] Structured errors + observability + eval harness

Tips for Effective Guidance

• Treat tool descriptions as API docs the model reads—iterate wording like UX copy.
• Recommend two-step patterns for dangerous ops: propose → confirm (human or policy).
• When using MCP, same discipline—server must validate everything.

Handling Deviations

• Read-only RAG: fewer semantic risks—still validate query args and injection into search backends.
• Local tools (filesystem): sandbox, path allowlists, size limits.