Skip to main content
This feature is in beta. Configuration options and behavior may change.
Agent-in-the-loop adds an AI verification step after extraction. It compares extracted arrays against the original document page by page, identifies missing or incorrect items, and re-extracts until the array is complete. This is designed for cases where you cannot afford to miss items: transaction lists, invoice line items, portfolio holdings.

​
How It Works

  1. Initial extraction runs normally
  2. Page mapping identifies which pages contain array items
  3. Page-by-page verification compares each page’s image against extracted data
  4. Iterative correction re-extracts when discrepancies are found
  5. Merge combines verified results across all pages
The agent uses vision to read the actual page image, not just parsed text. This catches items that text extraction might miss due to OCR errors or unusual formatting.

​
Configuration

Agent-in-the-loop is configured under instructions: 
result = client.extract.run(
    input=upload.file_id,
    instructions={
        "schema": {
            "type": "object",
            "properties": {
                "transactions": {
                    "type": "array",
                    "items": {
                        "type": "object",
                        "properties": {
                            "date": {"type": "string"},
                            "description": {"type": "string"},
                            "amount": {"type": "number"}
                        }
                    }
                }
            }
        },
        "system_prompt": "Extract all transaction line items.",
        "agent_in_the_loop": {
            "enabled": True,
            "fields_to_verify": ["transactions"]
        }
    }
)
Parameters:
  • enabled: Turn on agent verification
  • fields_to_verify: The array field to verify. Must be a top-level key in your schema. Currently limited to one field.

​
Requirements

PDF only. The agent uses page images for verification, which requires PDF input. Top-level array. The field in fields_to_verify must be a top-level property in your schema, not nested: 
# Works
{"properties": {"transactions": {"type": "array", ...}}}

# Does not work
{"properties": {"data": {"properties": {"transactions": {"type": "array", ...}}}}}
No page ranges. The entire document is processed. Page range settings are ignored. No citations. Agent-in-the-loop cannot be combined with citations. Enabling both throws an error.

​
When to Use

Use agent-in-the-loop when:
  • You need complete extraction of long arrays (50+ items)
  • Standard extraction or array extraction misses items
  • Accuracy matters more than speed
Standard array extraction segments the document and extracts in parallel. It’s fast but passive: if something is missed, it stays missed. Agent-in-the-loop actively verifies each page, catching items that passive extraction misses. The tradeoff is processing time. Agent-in-the-loop runs multiple model calls per page during verification, so expect longer processing times for large documents.

​
System Prompts

The system prompt guides both initial extraction and agent verification. Be specific about what constitutes a valid item: 
# Clear boundaries
"system_prompt": "Extract individual transaction line items. Exclude summary rows, headers, totals, and balance lines."

# Vague
"system_prompt": "Extract all data."

​
Pricing

During beta, agent-in-the-loop is billed at standard extraction rates. See Credit Usage for details.