{ "cells": [ { "cell_type": "markdown", "metadata": { }, "source": [ "*Module 3 of 12*\n", "\n", "# Tool Design \u0026 Tool Choice Strategy\n", "\n", "Once you decide to build an agent (Module 2), the model is the one picking which tool to call. This module gives you the levers that control that decision: eliminating functional overlap between tools, checking prompt instructions that bias routing, using the `tool_choice` parameter for prerequisites, strict-mode schema validation, and the agentic loop that ties it all together.\n", "\n", "\n", "## 1. Concepts: Model-Driven Routing\n", "\n", "In an agentic system, the model picks which tool to call, that decision is driven by the tool\u0027s `name`, `description`, schema, and the surrounding system prompt. Workflows hard-code the call site; agents read your tool surface and route. Four levers shape that routing:\n", "\n", "- **Tool descriptions are the primary selection mechanism.** Claude reads the `name` and `description` on each tool to choose. If two tools have overlapping purposes (e.g. `analyze_content` vs. `analyze_document`), Claude will misroute, write descriptions that disambiguate *exactly when* each tool is appropriate.\n", "- **Functional overlap must be removed, not merely explained.** If two tools do the same job with different names, rename or split them so each one owns a distinct input shape and outcome.\n", "- **System prompts can override good descriptions.** Keyword-sensitive instructions like \"for any document, call `analyze_document`\" can drag routing away from a better tool description. Audit prompt rules for accidental keyword traps.\n", "- **`tool_choice`** forces or constrains routing on a per-request basis: require *any* tool call, require a *specific* tool, or let the model decide.\n", "- **Strict mode (`strict: True`)** uses grammar-constrained sampling to force tool inputs to match your JSON schema 100%. No trailing commas, no missing required fields, no surprise keys.\n", "\n", "### 1a. Eliminating Functional Overlap\n", "\n", "\"Tool description\" is the most under-engineered part of most agentic systems. A bad description is one that tells Claude what the tool *does*; a good description tells Claude *when to pick it over its siblings*. A better design removes the sibling conflict entirely. Compare:\n", "\n", "*anti-pattern: ambiguous*" ] }, { "cell_type": "code", "execution_count": null, "metadata": { }, "outputs": [ ], "source": [ "# BAD: descriptions overlap, Claude will guess.\n", "tools = [\n", " {\"name\": \"analyze_content\", \"description\": \"Analyzes content.\"},\n", " {\"name\": \"analyze_document\", \"description\": \"Analyzes a document.\"},\n", "]" ] }, { "cell_type": "markdown", "metadata": { }, "source": [ "*good: disambiguated*" ] }, { "cell_type": "code", "execution_count": null, "metadata": { }, "outputs": [ ], "source": [ "# GOOD: each description names what the OTHER tool is NOT for.\n", "tools = [\n", " {\n", " \"name\": \"analyze_content\",\n", " \"description\": (\n", " \"Analyzes a single piece of inline text or HTML passed as a string \"\n", " \"(blog post body, email draft, social copy). Use this when the caller \"\n", " \"has the content in-hand. DO NOT use for files on disk or document IDs, \"\n", " \"use analyze_document for those.\"\n", " ),\n", " },\n", " {\n", " \"name\": \"analyze_document\",\n", " \"description\": (\n", " \"Analyzes a stored document referenced by document_id (PDF, Word, \"\n", " \"uploaded file). Use this when the caller has only an identifier or \"\n", " \"a file path. DO NOT use for raw inline text, use analyze_content.\"\n", " ),\n", " },\n", "]" ] }, { "cell_type": "markdown", "metadata": { }, "source": [ "The pattern: state the **input shape**, the **canonical use case**, and an explicit **\"DO NOT use for X, use *sibling_tool* instead\"** clause. That last clause is what eliminates misrouting between similar tools.\n", "\n", "*lab task: rename the overlapping tool*" ] }, { "cell_type": "code", "execution_count": null, "metadata": { }, "outputs": [ ], "source": [ "# STARTING POINT: both tools sound like generic analysis.\n", "tools = [\n", " {\"name\": \"analyze_content\", \"description\": \"Analyze content and return key points.\"},\n", " {\"name\": \"analyze_document\", \"description\": \"Analyze a document and return key points.\"},\n", "]\n", "\n", "# TARGET DESIGN: split by function and input source.\n", "tools = [\n", " {\n", " \"name\": \"analyze_uploaded_document\",\n", " \"description\": (\n", " \"Extracts claims and entities from an uploaded PDF/DOCX by document_id. \"\n", " \"Use only when the file already exists in document storage. Do not use \"\n", " \"for web search results or raw HTML.\"\n", " ),\n", " },\n", " {\n", " \"name\": \"extract_web_results\",\n", " \"description\": (\n", " \"Extracts title, URL, snippet, publication date, and source credibility \"\n", " \"from web_search results. Use only after web_search/web_fetch. Do not \"\n", " \"use for uploaded files or inline drafts.\"\n", " ),\n", " },\n", "]" ] }, { "cell_type": "markdown", "metadata": { }, "source": [ "### 1b. Prompt Audit for Keyword-Sensitive Instructions\n", "\n", "Even a clean toolbox can misroute if the system prompt contains broad keyword rules. Review instructions for phrases that bind routing to words in the user request instead of the tool\u0027s true preconditions.\n", "\n", "*prompt audit*\n", "```text\n", "Risky: \"Whenever the user mentions a document, call analyze_document.\"\n", "Better: \"When the user provides a stored document_id or file path, use analyze_uploaded_document.\n", "When the user asks about search output or fetched web pages, use extract_web_results.\"\n", "```\n", "\n", "The exam version of this problem is subtle: the tool descriptions look correct, but a prompt rule like \"document = analyze_document\" overrides the better choice for \"summarize this web document.\" Fix the prompt as well as the tool names.\n", "\n", "### 1c. Controlling Routing with `tool_choice`\n", "\n", "Descriptions guide the model\u0027s preference; `tool_choice` overrides it. Three values matter for the exam:\n", "\n", "- **`{\"type\": \"auto\"}`** (default), Claude decides whether to use a tool, which one, and may also reply with plain text.\n", "- **`{\"type\": \"any\"}`**, Claude *must* emit a tool call (no plain-text reply allowed). Use this when you want a structured response and any tool is acceptable.\n", "- **`{\"type\": \"tool\", \"name\": \"verify_identity\"}`**, Claude *must* call the named tool on this turn. Use this to enforce a prerequisite, e.g., identity verification before any account action.\n", "\n", "*forced prerequisite, then any-tool*" ] }, { "cell_type": "code", "execution_count": null, "metadata": { }, "outputs": [ ], "source": [ "import anthropic\n", "from dotenv import load_dotenv\n", "\n", "load_dotenv() # reads ANTHROPIC_API_KEY from your .env file\n", "\n", "client = anthropic.Anthropic()\n", "\n", "# Turn 1: force identity verification before anything else can happen.\n", "response = client.messages.create(\n", " model=\"claude-opus-4-7\",\n", " max_tokens=2048,\n", " tools=tools,\n", " tool_choice={\"type\": \"tool\", \"name\": \"verify_identity\"},\n", " messages=[{\"role\": \"user\", \"content\": \"Refund my last order.\"}],\n", ")\n", "\n", "# ... append assistant tool_use and the tool_result for verify_identity ...\n", "\n", "# Turn 2: identity is confirmed. Force a tool-based response (no chit-chat).\n", "response = client.messages.create(\n", " model=\"claude-opus-4-7\",\n", " max_tokens=2048,\n", " tools=tools,\n", " tool_choice={\"type\": \"any\"}, # must call SOME tool, model picks which\n", " messages=messages,\n", ")" ] }, { "cell_type": "markdown", "metadata": { }, "source": [ "\u003e **Tip.** Architect Tip for the Exam\n", "The exam loves the prerequisite pattern: `tool_choice={\"type\":\"tool\",\"name\":...}` on the first turn forces a specific call (verify identity, fetch customer, look up account); on subsequent turns you switch to `{\"type\":\"any\"}` or `{\"type\":\"auto\"}`. Combine this with Module 9\u0027s hooks for defense in depth, `tool_choice` nudges the model, hooks *guarantee* the prerequisite at execution time.\n", "\n", "### 1d. Dynamic Filtering \u0026 Strict Mode\n", "\n", "- **Dynamic filtering** with `web_search_20260209` alongside `code_execution_20260120` lets Claude write Python that post-processes search results *before* they enter the context window. Lower token cost, higher signal.\n", "- **Strict mode (`strict: True`)** uses grammar-constrained sampling to force tool inputs to match your JSON schema 100%. No trailing commas, no missing required fields, no surprise keys.\n", "\n", "\u003e **Tip.** ZDR Compliance Warning\n", "Standard `web_search` alone is **ZDR-eligible**. Adding `code_execution_20260120` for dynamic filtering makes the **entire request non-ZDR** because execution state is held server-side. Communicate this trade-off to regulated clients before enabling it.\n", "\n", "\u003e **Tip.** Architect Tip for the Exam\n", "The `web_search_20260209` tool version activates dynamic filtering, the older version does not. Always check the tool type version string, it is a direct exam signal for which capabilities are available.\n", "\n", "## 2. Defining the Toolbox\n", "\n", "Both `web_search` and `code_execution` are server-side built-ins, Anthropic runs them and feeds the results back to Claude. `add_lead_to_crm` is a **client-side** custom tool: when Claude calls it, your code is responsible for executing the action and returning the result.\n", "\n", "Apply the disambiguation rule from Section 1a: the `add_lead_to_crm` description states the *canonical use case* (\"a validated prospect\") and the *precondition* (\"after industry research is complete\"), so Claude won\u0027t fire it before web search has run. Note also the explicit `name` field on every entry, `code_execution_20260120` requires `name` even though it\u0027s a built-in (omitting it returns a 400 `BadRequestError`)." ] }, { "cell_type": "code", "execution_count": null, "metadata": { }, "outputs": [ ], "source": [ "# DEFINE THE TOOLS\n", "# Note: Use \u0027strict: True\u0027 for the CRM tool to guarantee valid JSON.\n", "# Note: \u0027code_execution\u0027 must have a name even if it\u0027s a built-in server tool.\n", "\n", "tools = [\n", " {\n", " \"type\": \"web_search_20260209\",\n", " \"name\": \"web_search\",\n", " \"max_uses\": 5,\n", " \"allowed_domains\": [\"gartner.com\", \"forrester.com\", \"hbr.org\"],\n", " },\n", " {\n", " \"type\": \"code_execution_20260120\",\n", " \"name\": \"code_execution\" # CRITICAL: required alongside type\n", " },\n", " {\n", " \"name\": \"add_lead_to_crm\",\n", " \"description\": \"Adds a validated prospect to the enterprise CRM. Use only after industry research is complete.\",\n", " \"strict\": True,\n", " \"input_schema\": {\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"company\": {\"type\": \"string\"},\n", " \"contact_email\": {\"type\": \"string\", \"format\": \"email\"},\n", " \"budget_range\": {\"type\": [\"string\", \"null\"]}, # Nullable to prevent hallucinations\n", " },\n", " \"required\": [\"company\", \"contact_email\", \"budget_range\"],\n", " \"additionalProperties\": False,\n", " },\n", " },\n", "]" ] }, { "cell_type": "markdown", "metadata": { }, "source": [ "## 3. The Initial Request \u0026 `stop_reason`\n", "\n", "Send the user\u0027s prospecting goal with the toolbox attached. Don\u0027t print just the final text, inspect `stop_reason` and walk the content blocks. On the first turn you should see `tool_use`, not `end_turn`." ] }, { "cell_type": "code", "execution_count": null, "metadata": { }, "outputs": [ ], "source": [ "# THE INITIAL REQUEST\n", "# On Opus 4.7, \u0027thinking\u0027 is required for complex tasks like this.\n", "response = client.messages.create(\n", " model=\"claude-opus-4-7\",\n", " max_tokens=8192,\n", " thinking={\"type\": \"adaptive\"},\n", " tools=tools,\n", " messages=[{\n", " \"role\": \"user\",\n", " \"content\": \"Find the top 5 AI consulting prospects in EMEA finance and add them to my CRM.\",\n", " }],\n", ")\n", "\n", "# SHOW OUTPUT\n", "print(f\"Stop Reason: {response.stop_reason}\") # Expected: \u0027tool_use\u0027\n", "for block in response.content:\n", " if block.type == \"thinking\":\n", " print(\"Reasoning Signature Detected.\")\n", " if block.type == \"tool_use\":\n", " print(f\"Claude requested tool: {block.name}\")\n", " print(f\"Parameters: {block.input}\")" ] }, { "cell_type": "markdown", "metadata": { }, "source": [ "## 4. Handling the Agentic Loop\n", "\n", "Web search runs **server-side**, Anthropic executes it and feeds the results back to Claude inside the same request. Your CRM write is **client-side**, your code must execute it and return a `tool_result` block before Claude can finish its turn.\n", "\n", "- **Turn 1:** Claude calls `web_search`. The API runs it server-side and silently injects results into Claude\u0027s context.\n", "- **Turn 2:** Claude reasons over the results and emits a `tool_use` block for `add_lead_to_crm`.\n", "- **Turn 3:** your application performs the database write and sends a `tool_result` block back. Claude generates the final `end_turn` response.\n", "\n", "*client-side leg*" ] }, { "cell_type": "code", "execution_count": null, "metadata": { }, "outputs": [ ], "source": [ "def handle_tool_use(block):\n", " if block.name == \"add_lead_to_crm\":\n", " # crm.write(block.input) # your client-side action\n", " return {\n", " \"type\": \"tool_result\",\n", " \"tool_use_id\": block.id,\n", " \"content\": \"ok: lead written\",\n", " }\n", " raise ValueError(f\"unhandled client-side tool: {block.name}\")\n", "\n", "# Loop until Claude is done.\n", "while response.stop_reason == \"tool_use\":\n", " tool_results = [handle_tool_use(b) for b in response.content if b.type == \"tool_use\"]\n", " response = client.messages.create(\n", " model=\"claude-opus-4-7\",\n", " max_tokens=8192,\n", " thinking={\"type\": \"adaptive\"},\n", " tools=tools,\n", " messages=[\n", " {\"role\": \"user\", \"content\": \"Find the top 5 AI consulting prospects ...\"},\n", " {\"role\": \"assistant\", \"content\": response.content},\n", " {\"role\": \"user\", \"content\": tool_results},\n", " ],\n", " )" ] }, { "cell_type": "markdown", "metadata": { }, "source": [ "## 5. Final Verification \u0026 Citations\n", "\n", "Once the loop terminates with `end_turn`, inspect the final text blocks. Web search automatically attaches citations to grounded claims, verify them so you can audit the sources Claude relied on." ] }, { "cell_type": "code", "execution_count": null, "metadata": { }, "outputs": [ ], "source": [ "# FINAL OUTPUT VERIFICATION\n", "# Web search automatically provides citations.\n", "for block in response.content:\n", " if block.type == \"text\":\n", " print(\"Final Answer:\", block.text)\n", " if hasattr(block, \"citations\") and block.citations:\n", " print(\"Sources:\", [c.url for c in block.citations])" ] }, { "cell_type": "markdown", "metadata": { }, "source": [ "\u003e **Tip.** Architectural Summary\n", "\n", "**Safety:** nullable types in the schema (`[\"string\", \"null\"]`) prevent Claude from inventing a budget figure when one isn\u0027t in the search results. \n", "**Efficiency:** `code_execution_20260120` is **free** when bundled with `web_search` or `web_fetch`, so dynamic filtering costs nothing extra in token generation. \n", "**Compliance:** that same bundling makes the request non-ZDR. Standard web search alone stays ZDR-eligible. \n", "**Selection:** tool descriptions, not names alone, drive Claude\u0027s routing. Disambiguate any tools whose purposes overlap." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.0" } }, "nbformat": 4, "nbformat_minor": 5 }