{ "cells": [ { "cell_type": "markdown", "metadata": { }, "source": [ "*Module 11 Self-Driven Lab*\r\n", "\r\n", "# Advanced Decomposition: Durability, Compaction, and Case Facts\r\n", "\r\n", "**Objective:** master adaptive re-planning, compaction block persistence, and the case facts pattern.\r\n", "\r\n", "## Challenge Outline\r\n", "\r\n", "Build a complete notebook that demonstrates the following outcomes:\r\n", "\r\n", "- **Iterative refinement:** implement an adaptive coordinator that evaluates subagent outputs for gaps, missing coverage, or contradictions, then re-delegates targeted follow-up tasks until a quality threshold is met.\r\n", "- **Case facts extraction:** create a `case_facts` block that persists `Max Budget: $50k`. Run a compaction trigger and verify the agent still knows the exact budget despite summary compression.\r\n", "- **Position-aware ordering:** build an aggregated input where `key_findings` appear first and detailed research is grouped under explicit section headers.\r\n", "- **Intelligence recovery:** simulate a research subagent failure. Ensure it returns a structured failure object with `partial_results`. Verify the coordinator uses partial data to finish the report rather than re-running the failed search.\r\n", "- **Compaction persistence:** implement the `compact_20260112` header. Verify that you pass the compaction block back in the subsequent `messages.create` call to keep the context window fresh.\r\n", "- **Effective context verification:** after the compaction block fires, call `client.messages.count_tokens` on the next outgoing payload to verify the effective context size.\r\n", "- **Thinking hygiene:** conduct a 10-turn research session with Adaptive Thinking enabled. Observe context accumulation and implement a trigger to omit thinking blocks for non-strategic summary turns.\r\n", "\r\n", "Your solution should include enough code, output, or written observations to prove each outcome worked. Keep the notebook focused on final behavior and evidence rather than a guided walkthrough.\n" ] }, { "cell_type": "markdown", "metadata": { }, "source": [ "## Student Workspace\n", "\n", "Use the sections below to build your solution. Each section maps to one required outcome from the challenge outline.\n" ] }, { "cell_type": "markdown", "metadata": { }, "source": [ "### Part 1: Iterative refinement\n", "\n", "implement an adaptive coordinator that evaluates subagent outputs for gaps, missing coverage, or contradictions, then re-delegates targeted follow-up tasks until a quality threshold is met.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { }, "outputs": [ ], "source": [ "# Part 1: Iterative refinement\n", "# Add your implementation, outputs, or notes here.\n" ] }, { "cell_type": "markdown", "metadata": { }, "source": [ "### Part 2: Case facts extraction\n", "\n", "create a `case_facts` block that persists `Max Budget: $50k`. Run a compaction trigger and verify the agent still knows the exact budget despite summary compression.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { }, "outputs": [ ], "source": [ "# Part 2: Case facts extraction\n", "# Add your implementation, outputs, or notes here.\n" ] }, { "cell_type": "markdown", "metadata": { }, "source": [ "### Part 3: Position-aware ordering\n", "\n", "build an aggregated input where `key_findings` appear first and detailed research is grouped under explicit section headers.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { }, "outputs": [ ], "source": [ "# Part 3: Position-aware ordering\n", "# Add your implementation, outputs, or notes here.\n" ] }, { "cell_type": "markdown", "metadata": { }, "source": [ "### Part 4: Intelligence recovery\n", "\n", "simulate a research subagent failure. Ensure it returns a structured failure object with `partial_results`. Verify the coordinator uses partial data to finish the report rather than re-running the failed search.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { }, "outputs": [ ], "source": [ "# Part 4: Intelligence recovery\n", "# Add your implementation, outputs, or notes here.\n" ] }, { "cell_type": "markdown", "metadata": { }, "source": [ "### Part 5: Compaction persistence\n", "\n", "implement the `compact_20260112` header. Verify that you pass the compaction block back in the subsequent `messages.create` call to keep the context window fresh.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { }, "outputs": [ ], "source": [ "# Part 5: Compaction persistence\n", "# Add your implementation, outputs, or notes here.\n" ] }, { "cell_type": "markdown", "metadata": { }, "source": [ "### Part 6: Effective context verification\n", "\n", "after the compaction block fires, call `client.messages.count_tokens` on the next outgoing payload to verify the effective context size.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { }, "outputs": [ ], "source": [ "# Part 6: Effective context verification\n", "# Add your implementation, outputs, or notes here.\n" ] }, { "cell_type": "markdown", "metadata": { }, "source": [ "### Part 7: Thinking hygiene\n", "\n", "conduct a 10-turn research session with Adaptive Thinking enabled. Observe context accumulation and implement a trigger to omit thinking blocks for non-strategic summary turns.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { }, "outputs": [ ], "source": [ "# Part 7: Thinking hygiene\n", "# Add your implementation, outputs, or notes here.\n" ] }, { "cell_type": "markdown", "metadata": { }, "source": [ "### Verification Notes\n", "\n", "Summarize the evidence that each part worked. Capture API signals, validation outcomes, errors, recovery behavior, cost observations, or comparisons required by this lab.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { }, "outputs": [ ], "source": [ "# Verification notes\n", "# Record the evidence that proves each lab outcome worked.\n" ] }, { "cell_type": "markdown", "metadata": { }, "source": [ "---\n", "\n", "## Answer Key\n", "\n", "The cells below contain the completed reference implementation/content for this module. Use this section only after attempting the self-driven lab." ] }, { "cell_type": "markdown", "metadata": { }, "source": [ "# Advanced Decomposition: Durability, Compaction, and Case Facts\n", "\n", "This module covers the advanced logic needed to maintain performance and accuracy as long-running research sessions approach the context window limit, and to ensure that multi-agent systems recover intelligently from intermediate failures rather than failing the entire workflow.\n", "\n", "## 1. Context Management: Server-Side Compaction\n", "\n", "As sessions approach the 1M-token context window limit, they can suffer from **context rot**, where the model loses focus on early instructions or misses information in the middle of long inputs, the \"lost in the middle\" effect.\n", "\n", "Use the `compact_20260112` strategy with a minimum trigger of **50,000 tokens**. Claude automatically generates a concise summary of the conversation and replaces stale content with a compaction block.\n", "\n", "Architect note: you must pass the resulting compaction block back to the API on all subsequent turns. The API then automatically drops all messages prior to it, resetting the context while preserving the narrative summary.\n", "\n", "Long-running loops should not wait for a hard crash to manage context. Poll `usage.context_window_remaining` and trigger compaction proactively at **80% utilization**.\n", "\n", "## 2. The Case Facts Pattern\n", "\n", "Compaction summaries can lose precise, non-negotiable details like confirmed budget figures, meeting dates, or transactional IDs.\n", "\n", "Extract critical facts into a persistent `case_facts` block placed after the compaction block and outside the summary.\n", "\n", "```text\n", "\u003ccompaction\u003e\n", "Summary of long-running research thread...\n", "\u003c/compaction\u003e\n", "\n", "\u003ccase_facts\u003e\n", "- Max Budget: $50k\n", "- Renewal Decision Date: 2026-09-15\n", "- CRM Account ID: acct_7781\n", "\u003c/case_facts\u003e\n", "```\n", "\n", "Rule: `case_facts` is never summarized or compacted. It keeps hard data available even after the conversational narrative is compressed.\n", "\n", "### 2a. Position-Aware Input Ordering\n", "\n", "To mitigate the \"lost in the middle\" effect before compaction is needed, place key findings summaries at the beginning of aggregated inputs and organize detailed results with explicit section headers.\n", "\n", "```text\n", "\u003ckey_findings\u003e\n", "- Max Budget: $50k\n", "- Current Competitor: ContosoAI\n", "- Decision Date: 2026-09-15\n", "\u003c/key_findings\u003e\n", "\n", "## Research Detail: Budget\n", "...\n", "\n", "## Research Detail: Competitor\n", "...\n", "```\n", "\n", "Do not bury critical facts between long raw excerpts. Put the summary first, then the detailed evidence.\n", "\n", "## 3. Decomposition: Fixed Pipelines vs. Adaptive Plans\n", "\n", "Orchestrators must choose between predictable efficiency and dynamic durability.\n", "\n", "- **Prompt chaining (fixed sequential):** a hard-coded sequence such as research -\u003e draft -\u003e review. It is predictable and fast but brittle. If research surfaces a deal-breaker, the system may still attempt to draft a useless report.\n", "- **Adaptive decomposition (dynamic):** the coordinator inspects each subagent\u0027s output and re-plans the next steps based on findings. If research finds a prospect already uses a competitor, the coordinator skips the standard pitch and spawns a competitor-positioning subagent instead.\n", "\n", "Adaptive systems cost more coordinator turns, but they are the durable pattern for open-ended work.\n", "\n", "## 4. The Subagent Failure-Report Contract\n", "\n", "A coordinator\u0027s ability to recover is bounded by the quality of the failure report it receives from a subagent.\n", "\n", "Subagents should return a structured contract instead of generic prose errors:\n", "\n", "```json\n", "{\n", " \"status\": \"failed\",\n", " \"failure_type\": \"TRANSIENT\",\n", " \"partial_results\": [\n", " \"Found pricing page\",\n", " \"Confirmed enterprise plan exists\"\n", " ],\n", " \"recommended_next_step\": \"Retry competitor review after rate limit clears.\"\n", "}\n", "```\n", "\n", "- `failure_type`: an enum such as `TRANSIENT` or `VALIDATION`, allowing the coordinator to branch without re-reading prose.\n", "- `partial_results`: lets the coordinator salvage what worked instead of wasting the entire run.\n", "- `recommended_next_step`: the subagent\u0027s local assessment of what it learned about the failure.\n", "\n", "Outer-loop signal: `isRetryable` from Module 9 is a tool-level signal. This structured report is the coordinator-level signal for orchestrating complex recoveries.\n", "\n", "## 5. Thinking Hygiene\n", "\n", "On newer models, thinking blocks are kept in context by default and can consume the window quickly during long-horizon work.\n", "\n", "To preserve context, use a lower compaction trigger or explicitly set `\"thinking_output\": \"omitted\"` on turns where narrative continuity is not required.\n", "\n", "## Lab Exercise: Managing a Long-Horizon Research Session\n", "\n", "**Objective:** master adaptive re-planning, compaction block persistence, and the case facts pattern.\n", "\n", "1. **Iterative refinement:** implement an adaptive coordinator that evaluates subagent outputs for gaps, missing coverage, or contradictions, then re-delegates targeted follow-up tasks until a quality threshold is met.\n", "2. **Case facts extraction:** create a `case_facts` block that persists `Max Budget: $50k`. Run a compaction trigger and verify the agent still knows the exact budget despite summary compression.\n", "3. **Position-aware ordering:** build an aggregated input where `key_findings` appear first and detailed research is grouped under explicit section headers.\n", "4. **Intelligence recovery:** simulate a research subagent failure. Ensure it returns a structured failure object with `partial_results`. Verify the coordinator uses partial data to finish the report rather than re-running the failed search.\n", "5. **Compaction persistence:** implement the `compact_20260112` header. Verify that you pass the compaction block back in the subsequent `messages.create` call to keep the context window fresh.\n", "6. **Effective context verification:** after the compaction block fires, call `client.messages.count_tokens` on the next outgoing payload to verify the effective context size.\n", "7. **Thinking hygiene:** conduct a 10-turn research session with Adaptive Thinking enabled. Observe context accumulation and implement a trigger to omit thinking blocks for non-strategic summary turns.\n", "\n", "\u003e **Exam tip:** compaction preserves narrative continuity; `case_facts` preserves exact facts; subagent failure reports preserve recovery options." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.0" } }, "nbformat": 4, "nbformat_minor": 5 }