# Reducto API Reference for Coding Agents Source: https://docs.reducto.ai/agent-guide Complete, structured reference for AI coding agents integrating Reducto This page is a dense, structured reference designed for AI coding agents. It contains everything needed to integrate Reducto without navigating multiple pages. ## Fastest Successful Path Use this decision table before writing code: | Situation | Use | First command or call | | ------------------------------------------------ | ------------------------- | ------------------------------------------------------------------------------------------ | | Local file or folder in a coding-agent workspace | [CLI](/cli) | `reducto parse ./document.pdf` | | Agent should call Reducto as tools | [MCP server](/mcp-server) | `parse_document(document_url="https://cdn.reducto.ai/samples/fidelity-example.pdf")` | | App code in Python | Python SDK | `client.parse.run(input="https://cdn.reducto.ai/samples/fidelity-example.pdf")` | | App code in Node.js | Node.js SDK | `await client.parse.run({ input: "https://cdn.reducto.ai/samples/fidelity-example.pdf" })` | | No SDK allowed | REST API | `POST https://platform.reducto.ai/parse` | For a benchmark or smoke test, parse this public sample first: ```text theme={null} https://cdn.reducto.ai/samples/fidelity-example.pdf ``` Then replace it with the user's document URL or upload a local file and pass the returned `reducto://` file ID. ## Product Summary Reducto is the agentic document platform. It provides a complete toolkit for classification, parsing, extraction, splitting, editing, and workflow orchestration across documents (PDFs, images, spreadsheets, DOCX, and 30+ other formats) via a REST API. * **Base URL:** `https://platform.reducto.ai` * **Auth:** `Authorization: Bearer $REDUCTO_API_KEY` * **SDKs:** Python (`pip install reductoai`), Node.js (`npm install reductoai`), Go (`go get github.com/reductoai/reducto-go-sdk`) * **Input:** Upload a file via `/upload` to get a `file_id`, then pass it to any endpoint. You can also pass public URLs or presigned S3/GCS/Azure URLs directly. *** ## Authentication 1. Create a free account at [studio.reducto.ai](https://studio.reducto.ai/) 2. In the Studio sidebar, click **API Keys**, then **Create new API key** 3. Set the key as an environment variable: ```bash theme={null} # macOS / Linux export REDUCTO_API_KEY="your_api_key_here" # Windows (PowerShell) $env:REDUCTO_API_KEY="your_api_key_here" ``` The Python and Node.js SDKs automatically read `REDUCTO_API_KEY` from the environment. For the Go SDK, pass it explicitly: ```go theme={null} client := reducto.NewClient(option.WithAPIKey(os.Getenv("REDUCTO_API_KEY"))) ``` For direct REST calls, pass it as a Bearer token: ```bash theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{"input": "https://example.com/doc.pdf"}' ``` *** ## Supported File Types | Category | Formats | | ------------- | ---------------------------------------------------------------------------------------------------------------------------- | | PDF | `.pdf` | | Documents | `.docx`, `.doc`, `.dotx`, `.rtf`, `.txt`, `.wpd` | | Spreadsheets | `.xlsx`, `.xlsm`, `.xls`, `.xltx`, `.xltm`, `.csv`, `.qpw` | | Presentations | `.pptx`, `.ppt` | | Images | `.png`, `.jpg`/`.jpeg`, `.gif`, `.bmp`, `.tiff`, `.heic`, `.psd`, `.pcx`, `.ppm`, `.apng`, `.cur`, `.dcx`, `.ftex`, `.pixar` | Upload limit: 100MB direct, 5GB via [presigned URL](/upload/large-files). Multi-page TIFFs are processed as multi-page documents. *** ## Which Endpoint Should I Use? | I want to... | Endpoint | Method | Key config | | ----------------------------------------------------------------------------------------------------- | --------------------------------------------------------------- | ---------------- | --------------------------------------------------------------- | | Get all text, tables, and figures from a document separated into chunks with bounding box coordinates | `/parse` | POST | `enhance.agentic` for a stronger model pass to correct mistakes | | Extract specific fields into JSON using a specific schema | `/extract` | POST | `instructions.schema` (JSON Schema) | | Divide a document into named sections by page range | `/split` | POST | `split_description` (section definitions) | | Fill PDF or DOCX forms | `/edit` | POST | `edit_instructions` (natural language) | | Classify a document's type before processing | `/classify` | POST | `classification_schema` (categories + criteria) | | Upload a local file for processing | `/upload` | POST (multipart) | `file` field | | Process asynchronously with webhooks | `/parse_async`, `/extract_async`, `/split_async`, `/edit_async` | POST | `webhook` URL | | Check job status or retrieve results | `/job/{job_id}` | GET | - | *** ## Quick Start (Python) ```python theme={null} from pathlib import Path from reducto import Reducto client = Reducto() # reads REDUCTO_API_KEY from env # --- Option A: Pass a URL directly (no upload needed) --- parse_result = client.parse.run(input="https://example.com/document.pdf") # --- Option B: Upload a local file first --- upload = client.upload(file=Path("document.pdf")) parse_result = client.parse.run(input=upload.file_id) # --- Handle the response (important: check result.type for large docs) --- import requests if parse_result.result.type == "url": # Large documents return a URL instead of inline content chunks = requests.get(parse_result.result.url).json() else: chunks = parse_result.result.chunks for chunk in chunks: # Use dict access for URL results, attribute access for inline results content = chunk["content"] if isinstance(chunk, dict) else chunk.content print(content) # --- Extract: pull specific fields --- extract_result = client.extract.run( input=upload.file_id, instructions={ "schema": { "type": "object", "properties": { "invoice_number": {"type": "string", "description": "The invoice number"}, "total": {"type": "number", "description": "Total amount due"} } } } ) # result is a list, access first item for single-document extraction data = extract_result.result[0] print(data["invoice_number"], data["total"]) # --- Split: find section boundaries --- split_result = client.split.run( input=upload.file_id, split_description=[ {"name": "Summary", "description": "Executive summary section"}, {"name": "Financials", "description": "Financial statements and tables"} ] ) for split in split_result.result.splits: print(f"{split.name}: pages {split.pages}") # --- Classify: identify document type --- classify_result = client.classify.run( input=upload.file_id, classification_schema=[ {"category": "invoice", "criteria": ["billing info", "itemized charges"]}, {"category": "contract", "criteria": ["legal terms", "signatures"]} ] ) print(classify_result.result) # --- Edit: fill a form --- # NOTE: Edit uses "document_url" instead of "input" (unlike other endpoints) edit_result = client.edit.run( document_url=upload.file_id, edit_instructions="Fill Name: John Doe, Date: 2024-01-15, Check 'Yes' for US Citizen" ) print(edit_result.document_url) # URL to download filled document ``` *** ## SDK Naming Conventions | SDK | Property names | Install | Notes | | ------- | ---------------------------------------- | -------------------------------------------- | ---------------------------------------------------------------------------- | | Python | snake\_case (`array_extract`, `file_id`) | `pip install reductoai` | Client auto-reads `REDUCTO_API_KEY` env var | | Node.js | snake\_case (`array_extract`, `file_id`) | `npm install reductoai` | All methods return promises, use `await` | | Go | PascalCase (`ArrayExtract`, `FileID`) | `go get github.com/reductoai/reducto-go-sdk` | Wrap values with `reducto.F()`, use `shared.UnionString()` for document URLs | | REST | snake\_case in JSON body | - | `Authorization: Bearer $REDUCTO_API_KEY` header | *** ## Parse Parameters `POST /parse`. Convert documents into structured JSON with text, tables, and figures. ### Core Parameters | Parameter | Type | Default | Description | | --------- | ------ | ------------ | ----------------------------------------------------------------------------------- | | `input` | string | **required** | File ID (`reducto://...`), public URL, presigned URL, or `jobid://...` to reprocess | ### enhance group | Parameter | Type | Default | Description | | ---------------------------------------- | ----------------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `enhance.agentic` | array | `[]` | List of agentic scopes. Each item has a `scope` field. | | `enhance.agentic[].scope` | `"text"` \| `"table"` \| `"figure"` | - | AI correction scope. `text`: OCR cleanup for scanned docs. `table`: fix misaligned columns. `figure`: chart data extraction. Adds latency and cost. | | `enhance.agentic[].prompt` | string \| null | `null` | Custom prompt for agentic processing | | `enhance.agentic[].advanced_chart_agent` | bool | `false` | Advanced chart extraction (figure scope only): agentic extractor returning full structured series data (`chart_data`) plus a reconstruction image re-drawn from that data (`extra.chart_reconstruction`). Use `parse_async`; high latency. | | `enhance.summarize_figures` | bool | `true` | Generate natural language descriptions of figures for RAG | | `enhance.intelligent_ordering` | bool | `false` | Use vision model to improve reading order accuracy | ### retrieval group | Parameter | Type | Default | Description | | ---------------------------------- | ----------------------------------------------------------------------------------------- | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `retrieval.chunking.chunk_mode` | `"disabled"` \| `"variable"` \| `"section"` \| `"page"` \| `"block"` \| `"page_sections"` | `"disabled"` | `disabled`: one chunk for entire doc. `variable`: semantic boundaries (best for RAG). `section`: split at headers. `page`: one chunk per page. `page_sections`: sections within each page. | | `retrieval.chunking.chunk_size` | int \| null | `null` | Target chunk size in characters. Defaults to 250-1500 range in variable mode. | | `retrieval.chunking.chunk_overlap` | int | `0` | Characters of overlap between adjacent chunks | | `retrieval.filter_blocks` | string\[] | `[]` | Block types to exclude from `content`/`embed`. Options: `"Header"`, `"Footer"`, `"Title"`, `"Section Header"`, `"Page Number"`, `"List Item"`, `"Figure"`, `"Table"`, `"Key Value"`, `"Text"`, `"Comment"`, `"Signature"` | | `retrieval.embedding_optimized` | bool | `false` | Optimize output for embedding models | ### formatting group | Parameter | Type | Default | Description | | -------------------------------- | ------------------------------------------------------------------------ | ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | | `formatting.table_output_format` | `"dynamic"` \| `"html"` \| `"md"` \| `"json"` \| `"csv"` \| `"jsonbbox"` | `"dynamic"` | `dynamic`: auto-selects md or html based on complexity. `html`: best for complex/merged cells. `md`: simple tables. `json`: programmatic cell access. | | `formatting.add_page_markers` | bool | `false` | Add page markers to output | | `formatting.merge_tables` | bool | `false` | Merge consecutive tables with same column count | | `formatting.include` | string\[] | `[]` | Include: `"change_tracking"`, `"highlight"`, `"comments"`, `"hyperlinks"`, `"signatures"`, `"ignore_watermarks"` | ### spreadsheet group | Parameter | Type | Default | Description | | ---------------------------------------- | ---------------------------------------- | ------------ | ----------------------------------------------------------------------------------------- | | `spreadsheet.split_large_tables.enabled` | bool | `true` | Split large tables into smaller tables | | `spreadsheet.split_large_tables.size` | int | `50` | Rows per chunk for split tables | | `spreadsheet.clustering` | `"accurate"` \| `"fast"` \| `"disabled"` | `"accurate"` | Algorithm for splitting sheets into tables. Accurate uses more powerful models (5x cost). | | `spreadsheet.include` | string\[] | `[]` | Include: `"cell_colors"`, `"formula"`, `"dropdowns"` | ### settings group | Parameter | Type | Default | Description | | --------------------------------- | -------------------------- | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `settings.page_range` | object \| null | `null` | `{"start": 1, "end": 10}` (1-indexed). Process specific pages only. | | `settings.return_images` | string\[] | `[]` | Return image URLs for block types: `"figure"`, `"table"`, `"page"` | | `settings.ocr_system` | `"standard"` \| `"legacy"` | `"standard"` | `standard`: best multilingual OCR. `legacy`: Germanic languages only. | | `settings.extraction_mode` | `"ocr"` \| `"hybrid"` | `"hybrid"` | `hybrid`: combines OCR with embedded PDF text (recommended). `ocr`: OCR only. | | `settings.persist_results` | bool | `false` | Keep results indefinitely (default: expire after 24h) | | `settings.force_url_result` | bool | `false` | Always return results as a URL | | `settings.timeout` | float \| null | `null` | Custom timeout in seconds | | `settings.document_password` | string \| null | `null` | Password for encrypted documents | | `settings.embed_pdf_metadata` | bool | `false` | Embed OCR metadata into returned PDF | | `settings.embed_pdf_metadata_dpi` | int | `100` | Render DPI for the rasterized pages of the embedded-OCR PDF. Range 50-250. Default 100 is suitable for on-screen viewing; raise toward the source scan DPI for crisper output when zoomed past \~200%. | ### Parse Response Shape ```json theme={null} { "job_id": "uuid", "duration": 3.89, "result": { "type": "full", // "full" (inline) or "url" (fetch from URL) "chunks": [ { "content": "# Heading\n\nText content...", // Markdown-formatted "embed": "Heading. Text content...", // Embedding-optimized "blocks": [ { "type": "Title", // Title, Section Header, Text, Table, Figure, Key Value, etc. "content": "Heading", "bbox": {"left": 0.1, "top": 0.05, "width": 0.3, "height": 0.04, "page": 1}, "confidence": "high" // "high" or "low" } ] } ] }, "usage": {"num_pages": 3, "credits": 4.0}, "studio_link": "https://studio.reducto.ai/job/..." } ``` When `result.type` is `"url"`, chunks are not inline. Fetch them from the URL: ```python theme={null} import requests if parse_result.result.type == "url": chunks = requests.get(parse_result.result.url).json() # chunks are plain dicts when fetched via URL else: chunks = parse_result.result.chunks # chunks are SDK objects with attribute access ``` *** ## Extract Parameters `POST /extract`. Pull specific fields from documents into structured JSON using a schema. Extract runs Parse internally. If a value doesn't appear in the Parse output, Extract cannot extract it. | Parameter | Type | Default | Description | | ----------------------------------------- | ------------------- | ---------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | | `input` | string \| string\[] | **required** | File ID, URL, `jobid://...`, or array of job IDs to combine | | `instructions.schema` | object | `{}` | JSON Schema defining fields to extract. Field names and descriptions directly influence accuracy. | | `instructions.system_prompt` | string | `"Be precise and thorough."` | Document-level context for the LLM | | `settings.array_extract` | bool | `false` | **Deprecated.** Use `settings.deep_extract` instead. | | `settings.deep_extract` | bool | `false` | Agentic mode that iteratively refines output for near-perfect accuracy. Preferred for complex or long (array-heavy) extractions. Higher cost/latency. | | `settings.citations.enabled` | bool | `false` | Return source page, bbox, and text for each value. **Mutually exclusive with chunking.** | | `settings.citations.numerical_confidence` | bool | `true` | Include 0-1 confidence scores (vs "high"/"low") | | `settings.include_images` | bool | `false` | Include page images in extraction context | | `settings.optimize_for_latency` | bool | `false` | Higher priority processing at 2x cost | | `parsing` | object | `{}` | All Parse parameters (see above). Ignored if input is `jobid://`. | ### Extract Response Shape ```json theme={null} { "result": [ { "invoice_number": "INV-2024-001", "total": 1250.00, "line_items": [{"description": "Widget", "amount": 500.00}] } ], "job_id": "uuid", "usage": {"num_fields": 4, "num_pages": 2, "credits": 10.0}, "studio_link": "https://studio.reducto.ai/job/..." } ``` With `citations.enabled: true`, each value is wrapped: ```json theme={null} { "result": { "total": { "value": 1250.00, "citations": [ { "type": "Table", "content": "Total: $1,250.00", "bbox": {"left": 0.04, "top": 0.26, "width": 0.45, "height": 0.50, "page": 2}, "confidence": "high" } ] } } } ``` *** ## Split Parameters `POST /split`. Divide documents into named sections by page number. Split runs Parse internally, then uses an LLM to classify pages against your section descriptions. | Parameter | Type | Default | Description | | ----------------------- | ---------------------------- | ------------------------------------------------------ | ------------------------------------------------------------------- | | `input` | string | **required** | File ID, URL, or `jobid://...` | | `split_description` | array | **required** | List of `{"name": "...", "description": "..."}` section definitions | | `split_rules` | string | `"Split the document into the applicable sections..."` | Natural language rules for splitting behavior | | `settings.table_cutoff` | `"truncate"` \| `"preserve"` | `"truncate"` | `truncate`: first rows only (faster). `preserve`: all content. | | `parsing` | object | `{}` | All Parse parameters. Ignored if input is `jobid://`. | ### Split Response Shape ```json theme={null} { "result": { "splits": [ {"name": "Executive Summary", "pages": [1, 2]}, {"name": "Financial Statements", "pages": [3, 4, 5, 6]}, {"name": "Risk Factors", "pages": [7, 8, 9]} ] }, "job_id": "uuid", "usage": {"num_pages": 9, "credits": 6.0} } ``` *** ## Edit Parameters `POST /edit`. Fill PDF forms and modify DOCX documents. **Note:** Edit uses `document_url` as its input parameter, not `input` like other endpoints. | Parameter | Type | Default | Description | | ------------------------------------ | ------------- | ------------ | ------------------------------------------------------------------------------------- | | `document_url` | string | **required** | File ID or URL of the document to edit (this is `document_url`, not `input`) | | `edit_instructions` | string | **required** | Natural language instructions. Be explicit: `"Fill Name: John Doe, Date: 2024-01-15"` | | `edit_options.color` | string | `"#FF0000"` | Highlight color for edits (DOCX only) | | `edit_options.enable_overflow_pages` | bool | `false` | Create appendix pages for text exceeding field capacity (PDF only) | | `form_schema` | array \| null | `null` | Pre-defined field locations for repeatable form filling. Skips detection. | ### Edit Response Shape ```json theme={null} { "document_url": "https://storage.reducto.ai/filled-form.pdf?...", "form_schema": [ { "bbox": {"left": 0.1, "top": 0.2, "width": 0.4, "height": 0.03, "page": 1}, "description": "Name field", "type": "text" } ], "usage": {"num_pages": 2, "credits": 8} } ``` The `document_url` is a presigned URL valid for 24 hours. Save the returned `form_schema` to reuse for the same form type (skips field detection). *** ## Classify Parameters `POST /classify`. Categorize a document before processing. | Parameter | Type | Default | Description | | ----------------------- | -------------- | ------------ | --------------------------------------------------------------------------------- | | `input` | string | **required** | File ID or URL | | `classification_schema` | array | **required** | List of `{"category": "...", "criteria": ["...", "..."]}` | | `page_range` | object \| null | `null` | Pages to use for classification context. Defaults to first 5 pages. Max 10 pages. | | `document_metadata` | string \| null | `null` | Optional metadata to include in classification prompt | ### Classify Response Shape ```json theme={null} { "result": { "category": "invoice" }, "job_id": "uuid", "duration": 1.23 } ``` *** ## Async Processing Most endpoints have async variants (`/parse_async`, `/extract_async`, `/split_async`, `/edit_async`). Classify is synchronous only. Async endpoints return a `job_id` immediately and process in the background. ```python theme={null} # Submit async job job = client.parse.run_job(input=upload.file_id) print(job.job_id) # Poll for results import time while True: result = client.job.get(job.job_id) if result.status in ("Completed", "Failed"): break time.sleep(2) ``` Configure webhooks for push-based delivery instead of polling. *** ## Error Codes | HTTP Status | Meaning | Common cause | | ----------- | ---------------- | ------------------------------------------------------------- | | 401 | Unauthorized | Missing or invalid `REDUCTO_API_KEY` | | 422 | Validation error | Invalid parameters, schema too large, or constraint violation | | 429 | Rate limited | Too many concurrent requests. Retry with backoff. | | 500 | Server error | Transient issue. Retry with backoff. | *** ## Useful Links * [API Reference](/api-reference/parse): Full OpenAPI spec with request/response details * [Parse Configuration](/configs/overview): All configuration options * [Cookbooks](/cookbooks/overview): End-to-end tutorials (invoice extraction, form filling, RAG) * [Error Codes](/reference/error-codes): Complete error catalog * [Checking API Health & Usage](/reference/checking-api-health): Hosted API availability, usage data, and throttling signals * [Rate Limits](/reference/rate-limits): Request limits and quotas * [Credit Usage](/reference/credit-usage): How credits are calculated # Parse Async Source: https://docs.reducto.ai/api-reference/async-parse openapi.json post /parse_async # Cancel Job Source: https://docs.reducto.ai/api-reference/cancel-job openapi.json post /cancel/{job_id} # Classify Source: https://docs.reducto.ai/api-reference/classify openapi.json post /classify # Delete Job Source: https://docs.reducto.ai/api-reference/delete-job openapi.json delete /job/{job_id} Asynchronously delete a job's stored artifacts. Marks the job as mid-deletion (retrieval returns 409 until cleanup finishes, then 410) and enqueues the streaq cleanup task. # Delete Upload Source: https://docs.reducto.ai/api-reference/delete-upload openapi.json delete /upload/{file_id} # Edit Source: https://docs.reducto.ai/api-reference/edit openapi.json post /edit # Edit Async Source: https://docs.reducto.ai/api-reference/edit-async openapi.json post /edit_async # Extract Source: https://docs.reducto.ai/api-reference/extract openapi.json post /extract # Extract Async Source: https://docs.reducto.ai/api-reference/extract-async openapi.json post /extract_async # Get Jobs Source: https://docs.reducto.ai/api-reference/get-jobs openapi.json get /jobs # Get Version Source: https://docs.reducto.ai/api-reference/get-version openapi.json get /version # Parse Source: https://docs.reducto.ai/api-reference/parse openapi.json post /parse # Pipeline Source: https://docs.reducto.ai/api-reference/pipeline openapi.json post /pipeline # Pipeline Async Source: https://docs.reducto.ai/api-reference/pipeline-async openapi.json post /pipeline_async # Retrieve Job Source: https://docs.reducto.ai/api-reference/retrieve-parse openapi.json get /job/{job_id} # Split Source: https://docs.reducto.ai/api-reference/split openapi.json post /split # Split Async Source: https://docs.reducto.ai/api-reference/split-async openapi.json post /split_async # Upload Source: https://docs.reducto.ai/api-reference/upload openapi.json post /upload # Webhook Portal Source: https://docs.reducto.ai/api-reference/webhook-portal openapi.json post /configure_webhook # Classify Best Practices Source: https://docs.reducto.ai/classify/best-practices Write better classification schemas for more accurate results The quality of your classification depends heavily on how you define your categories and criteria. Here are guidelines for getting the best results. *** ## Be Specific with Criteria Criteria should describe concrete, observable characteristics of the document, things that would be visible on the page. Think of criteria as instructions you'd give a human reviewer: "Look for X, Y, and Z to identify this type of document." **Good criteria** describe what you'd actually see in the document: ```json theme={null} { "category": "invoice", "criteria": [ "contains an invoice number or reference number", "has line items with quantities and unit prices", "shows a total amount due or balance", "includes vendor/supplier contact information" ] } ``` **Weak criteria** are too vague or abstract: ```json theme={null} { "category": "invoice", "criteria": [ "is an invoice", "looks like an invoice", ] } ``` *** ## Make Categories Mutually Exclusive Design your categories so that a given document clearly belongs to one category over others. If categories overlap significantly, classification accuracy will suffer. ```json theme={null} [ { "category": "declaration_page", "criteria": [ "summarizes active coverage for a specific policy term", "includes policy number, named insured, and insurer details", "shows effective and expiration dates", "lists coverage types with limits and deductibles", "states total premium and billing or installment info" ] }, { "category": "explanation_of_benefits", "criteria": [ "explains how a submitted medical claim was processed", "includes member ID, claim number, and service dates", "breaks down billed amount, allowed amount, and insurer payment", "shows patient responsibility (copay, deductible, coinsurance)", "clearly indicates this is not a bill" ] } ] ``` *** ## Text vs. Image-Based Classification Classify works with both text-heavy and visually distinct documents. Your criteria can reference either textual content or visual characteristics: * **Text-based criteria**: `"contains the words 'Terms and Conditions'"`, `"includes a table of financial figures"` * **Visual/structural criteria**: `"has a photo ID section"`, `"contains handwritten notes"`, `"includes a signature block"` For documents that are primarily distinguished by layout rather than text (e.g., a passport vs. a driver's license), include structural criteria like `"contains a machine-readable zone at the bottom"` or `"has a photo in the upper-left corner"`. *** ## Use Enough Categories You must provide at least two categories. Classify returns the **best match** from your schema, so even if none of the categories are a perfect fit, it will return the closest one. If you need an escape hatch, add a catch-all category: ```json theme={null} { "category": "other", "criteria": [ "does not match any of the other document types (list all the other document types)", "unrecognized format or content" ] } ``` *** ## Use Confidence Scores to Refine Your Schema The [response confidence breakdown](/classify/response-format#structured-confidence-reasoning-you-can-act-on) tells you exactly which criteria matched or didn't for every category. Use this to iterate on your schema: 1. Run Classify on a batch of sample documents. 2. Check documents where the winning category had low confidence (e.g., below `0.7`). 3. Inspect the `criteria_confidence` to see which criteria are too broad, too narrow, or overlapping with other categories. 4. Adjust your criteria and re-run until confidence scores improve. *** ## Related Quick start, request parameters, and pipeline integration. Confidence scores, per-criterion reasoning, and all response fields. # Classify Source: https://docs.reducto.ai/classify/overview Categorize documents before you parse Classify determines what kind of document you are looking at before any downstream processing begins. You specify categories with natural language criteria, and Reducto returns the best match. Classify is typically the first step in a document workflow. It routes documents to the right pipeline with the right configurations, so that [Parse](/parse/overview), [Extract](/extract/overview), and [Split](/split) each run with settings tuned for the specific document type. *** ## When to Use Classify Classify routes your documents to the right pipeline upstream of further processing. Instead of parsing all of your documents with the same settings, you can contextualize your documents before you parse them. **Common use cases:** * **Document triage during onboarding.** Quickly classify documents into categories when users upload files on your platform. For example, sort into different types of legal documents. * **Conditional parsing configurations.** Classify handwritten doctor's notes versus other forms, then enable agentic text in Parse with specific prompts downstream for the doctor's notes. * **Schema routing for extraction.** Classify your documents upfront to apply different extraction schemas downstream. Passports get one schema, immigration forms get another. * **Pipeline branching.** Use classification results to decide whether a document needs Split, Extract, or both, and with which Parse configurations. For example, some financial documents may need specific splits, while others may not. Use Classify on its own to label documents as part of a larger user flow, or apply Classify before your existing Reducto pipeline. *** ## Quick Start Play around at [the Classify demo](https://classify.reducto.ai). ```python Python theme={null} from reducto import Reducto client = Reducto() response = client.classify.run( input="https://example.com/document.pdf", classification_schema=[ { "category": "invoice", "criteria": [ "contains billing information", "has itemized charges", "includes payment details or amounts due", ], }, { "category": "contract", "criteria": [ "contains legal terms and conditions", "includes signature lines or parties involved", "references obligations or agreements", ], }, { "category": "receipt", "criteria": [ "shows a completed transaction", "includes items purchased with prices", "has a total amount paid", ], }, ], ) print(response) ``` ```javascript Node.js theme={null} import Reducto from 'reductoai'; const client = new Reducto(); const response = await client.classify.run({ input: 'https://example.com/document.pdf', classification_schema: [ { category: 'invoice', criteria: [ 'contains billing information', 'has itemized charges', 'includes payment details or amounts due', ], }, { category: 'contract', criteria: [ 'contains legal terms and conditions', 'includes signature lines or parties involved', 'references obligations or agreements', ], }, { category: 'receipt', criteria: [ 'shows a completed transaction', 'includes items purchased with prices', 'has a total amount paid', ], }, ], }); console.log(response); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/classify \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "https://example.com/document.pdf", "classification_schema": [ { "category": "invoice", "criteria": [ "contains billing information", "has itemized charges", "includes payment details or amounts due" ] }, { "category": "contract", "criteria": [ "contains legal terms and conditions", "includes signature lines or parties involved", "references obligations or agreements" ] }, { "category": "receipt", "criteria": [ "shows a completed transaction", "includes items purchased with prices", "has a total amount paid" ] } ] }' ``` **What this does:** 1. **Upload** the document to get a `file_id` 2. **Call `/classify`** with the file reference and a list of categories, each with criteria describing what makes a document belong to that category 3. **Get back** the best-matching category ```json theme={null} { "job_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "result": { "category": "invoice" }, "usage": { "num_pages": 5, "num_categories": 3, "credits": 2.5 } } ``` Visualize your classification with [the Classify demo](https://classify.reducto.ai). Classify works on all file types we can Parse. Full breakdown of confidence scores, per-criterion reasoning, and all response fields. *** ## Request Parameters ```python Python theme={null} from reducto import Reducto client = Reducto() response = client.classify.run( input="...", # Required: upload response or URL classification_schema=[...], # Required: categories with criteria page_range={"start": 1, "end": 10}, # Optional: pages to use as context document_metadata="...", # Optional: additional context ) ``` ```javascript Node.js theme={null} import Reducto from 'reductoai'; const client = new Reducto(); const response = await client.classify.run({ input: '...', // Required: upload response or URL classification_schema: [...], // Required: categories with criteria page_range: { start: 1, end: 10 }, // Optional: pages to use as context document_metadata: '...', // Optional: additional context }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/classify \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "YOUR_FILE_ID_OR_URL", "classification_schema": [{"category": "...", "criteria": ["..."]}], "page_range": {"start": 1, "end": 10}, "document_metadata": "optional context string" }' ``` ### input (required) The document to classify. Accepts the same formats as other Reducto endpoints: | Format | Example | When to use | | --------------- | ---------------------------------------- | ---------------------------------- | | Upload response | `reducto://abc123` | Local files uploaded via `/upload` | | Public URL | `https://example.com/doc.pdf` | Publicly accessible documents | | Presigned URL | `https://bucket.s3.../doc.pdf?X-Amz-...` | Files in your cloud storage | ### classification\_schema (required) A list of categories you want to classify the document into. Each category has a name and a list of criteria describing what makes a document belong to that category. ```json theme={null} { "classification_schema": [ { "category": "invoice", "criteria": [ "contains billing information", "has itemized charges", "includes payment details" ] }, { "category": "contract", "criteria": [ "contains legal terms and conditions", "includes signature lines", "references parties and obligations" ] } ] } ``` | Field | Type | Description | | ---------- | -------------- | --------------------------------------------------------------------------------------------------------------------------------------- | | `category` | `string` | The category name/label that documents will be classified into (e.g., `"invoice"`, `"contract"`, `"receipt"`) | | `criteria` | `list[string]` | A list of criteria, keywords, or descriptions that define what characteristics a document must have to be classified into this category | ### page\_range `page_range` (optional, defaults to first 5 pages): The page range to use as context for classification. Accepts an object with `start` and `end` fields (1-indexed, inclusive). If more than 10 pages are selected, the request returns an error. Only applies to PDFs. See [Classify Configuration](/configs/classify/configuration) for examples and cost implications. ### document\_metadata `document_metadata` (optional, defaults to `null`): A metadata string to include in classification prompts. Use this to provide additional context about the document that may help with classification. *** ## How It Works 1. **Document ingestion.** Classify accepts your document and processes it to understand its content. 2. **Category evaluation.** Each category in your `classification_schema` is evaluated against the document. The criteria you provide guide what the model looks for. 3. **Best match selection.** Classify returns the single best-matching category. It compares all categories and picks the one whose criteria best describe the document. Classify is optimized for latency. It's a lightweight operation compared to full parsing or extraction, designed to return results fast enough to use as an inline routing step without adding meaningful overhead to your pipeline. Classify is synchronous only. The endpoint is optimized for low latency, so classification results return fast enough that async polling or webhooks are unnecessary. *** ## Using Classify in a Pipeline Classify is most useful when combined with other Reducto endpoints. Here's a common pattern: classify first, then route to different Parse/Extract configurations based on the result. ```python Python theme={null} from pathlib import Path from reducto import Reducto client = Reducto() # Step 1: Upload upload = client.upload(file=Path("document.pdf")) # Step 2: Classify classification = client.classify.run( input=upload.file_id, classification_schema=[ { "category": "handwritten_notes", "criteria": ["contains handwritten text", "informal layout", "pen or pencil marks"], }, { "category": "printed_form", "criteria": ["structured layout with fields", "typed text", "checkboxes or form widgets"], }, ], ) category = classification.result.category # Step 3: Route to appropriate pipeline if category == "handwritten_notes": result = client.parse.run( input=upload.file_id, enhance={"agentic": [{"scope": "text", "prompt": "This is a handwritten medical note. Pay close attention to medication names and dosages."}]}, ) else: # Standard extraction for printed forms result = client.extract.run( input=upload.file_id, instructions={ "schema": { "type": "object", "properties": { "patient_name": {"type": "string"}, "date_of_birth": {"type": "string"}, "insurance_id": {"type": "string"}, }, } }, ) print(result) ``` ```javascript Node.js theme={null} import fs from 'fs'; import Reducto from 'reductoai'; const client = new Reducto(); // Step 1: Upload const upload = await client.upload({ file: fs.createReadStream('document.pdf') }); // Step 2: Classify const classification = await client.classify.run({ input: upload.file_id, classification_schema: [ { category: 'handwritten_notes', criteria: ['contains handwritten text', 'informal layout', 'pen or pencil marks'], }, { category: 'printed_form', criteria: ['structured layout with fields', 'typed text', 'checkboxes or form widgets'], }, ], }); const category = classification.result.category; // Step 3: Route to appropriate pipeline if (category === 'handwritten_notes') { const result = await client.parse.run({ input: upload.file_id, enhance: { agentic: [{ scope: 'text', prompt: 'This is a handwritten medical note. Pay close attention to medication names and dosages.' }] }, }); console.log(result); } else { // Standard extraction for printed forms const result = await client.extract.run({ input: upload.file_id, instructions: { schema: { type: 'object', properties: { patient_name: { type: 'string' }, date_of_birth: { type: 'string' }, insurance_id: { type: 'string' }, }, }, }, }); console.log(result); } ``` ```bash cURL theme={null} # Step 1: Upload FILE_ID=$(curl -s -X POST https://platform.reducto.ai/upload \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -F "file=@document.pdf" | jq -r '.file_id') # Step 2: Classify CATEGORY=$(curl -s -X POST https://platform.reducto.ai/classify \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "'$FILE_ID'", "classification_schema": [ { "category": "handwritten_notes", "criteria": ["contains handwritten text", "informal layout", "pen or pencil marks"] }, { "category": "printed_form", "criteria": ["structured layout with fields", "typed text", "checkboxes or form widgets"] } ] }' | jq -r '.result.category') # Step 3: Route to appropriate pipeline if [ "$CATEGORY" = "handwritten_notes" ]; then curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "'$FILE_ID'", "enhance": { "agentic": [{"scope": "text", "prompt": "This is a handwritten medical note. Pay close attention to medication names and dosages. "}] } }' else curl -X POST https://platform.reducto.ai/extract \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "'$FILE_ID'", "instructions": { "schema": { "type": "object", "properties": { "patient_name": {"type": "string"}, "date_of_birth": {"type": "string"}, "insurance_id": {"type": "string"} } } } }' fi ``` *** ## FAQs Reducto can already Classify documents after parsing using Extract or Split. Extract pulls **specific data** out of a document alongside an enum-based classification. Split works best when you have multi-document packets. However, both of these endpoints need Parse. Classify is utilized **before** Parse, and as such is optimized for latency and cost. Classify also returns structured reasoning and confidence. Classify is faster than Parse or Extract because it doesn't need to do full document parsing. It focuses on high-level document understanding to match against your categories. Accuracy depends on how well your criteria describe each category. Well-defined, mutually exclusive categories with specific criteria will yield the best results. Any categories you want. You define both the category names and the criteria. Common examples include: * **Document type**: invoice, contract, receipt, tax form * **Content characteristics**: handwritten vs. typed, single-page vs. multi-page * **Domain-specific**: ACORD forms vs. declarations pages, W-2 vs. 1099 * **Processing needs**: needs OCR enhancement vs. standard processing You must provide your own categories via `classification_schema`. Classify matches your document against the categories you define. It doesn't auto-discover document types. This is by design: your categories should reflect your specific pipeline needs. A healthcare company and a law firm would classify the same document differently based on their downstream processing requirements. Yes. This is the primary use case. Classify sits at the top of your pipeline to route documents, then you call Parse, Extract, or Split with configurations tailored to each document type. See [Using Classify in a Pipeline](#using-classify-in-a-pipeline) above for a complete example. Classify is optimized for PDFs and images (PNG, JPEG, etc.), but also supports the remaining formats supported by Reducto's Parse endpoint. Classify always returns the best match from your schema, even if the fit isn't perfect. If you need to handle unrecognized documents, add an `"other"` category with criteria like `"does not match any of the other document types"`. Be sure to enumerate the types it should not match *** ## Related Confidence scores, per-criterion reasoning, and all response fields. Write better classification schemas for more accurate results. Page ranges and classification schema options. How Classify credits are calculated. # Classify Response Format Source: https://docs.reducto.ai/classify/response-format Confidence scores, per-criterion reasoning, and all response fields Classify returns a category label, a per-criterion confidence breakdown for every category in your schema, and timing information. This page explains every field in the response. *** ## Response Structure ```json theme={null} { "job_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "result": { "category": "invoice" }, "response_confidence": { "categories": [ { "category": "invoice", "confidence": 1.0, "criteria_confidence": [ {"criterion": "contains billing information", "confidence": "high"}, {"criterion": "has itemized charges", "confidence": "high"}, {"criterion": "includes payment details or amounts due", "confidence": "high"} ] }, { "category": "contract", "confidence": 0.33, "criteria_confidence": [ {"criterion": "contains legal terms and conditions", "confidence": "low"}, {"criterion": "includes signature lines or parties involved", "confidence": "low"}, {"criterion": "references obligations or agreements", "confidence": "high"} ] } ] }, "usage": { "num_pages": 5, "num_categories": 2, "credits": 2.5 }, "duration": 1.23 } ``` ### Top-Level Fields | Field | Type | Description | | --------------------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------- | | `job_id` | string | Unique identifier for the classification job | | `result.category` | string | The best-matching category from your `classification_schema`. Always one of the category names you provided. | | `response_confidence` | object | Per-category and per-criterion confidence breakdown. `null` when no criteria are provided. | | `usage` | object | Page count, category count, and credit consumption for this classification. See [Usage Fields](#usage-fields) below. May be `null`. | | `duration` | number | Time in seconds the classify request took. May be `null`. | ### Usage Fields The `usage` object contains: | Field | Type | Description | | ---------------- | ------- | ------------------------------------------------------------------------------------------------------- | | `num_pages` | integer | Number of pages used as context for classification. Defaults to 5, or fewer if the document is shorter. | | `num_categories` | integer | Number of categories in the `classification_schema` that were evaluated. | | `credits` | number | Total credits consumed. Each page costs 0.5 credits. May be `null`. | ### Category Confidence Fields Each entry in `response_confidence.categories` contains: | Field | Type | Description | | --------------------- | ------ | ------------------------------------------------------------------------------------------ | | `category` | string | The category name from your schema | | `confidence` | number | Float between 0 and 1 representing the fraction of criteria that matched for this category | | `criteria_confidence` | array | Per-criterion evaluation results | ### Criteria Confidence Fields Each entry in `criteria_confidence` contains: | Field | Type | Description | | ------------ | ------ | ------------------------------------------------------------------------------ | | `criterion` | string | The criterion text from your schema | | `confidence` | string | `"high"` (criterion matched the document) or `"low"` (criterion did not match) | *** ## Structured Confidence: Reasoning You Can Act On Classify doesn't just return a label. It returns a per-criterion confidence breakdown for **every** category you defined, not just the winner. This gives you structured, interpretable reasoning for why a document was classified the way it was. Each criterion you define becomes a yes/no evaluation. The `confidence` score for a category is the fraction of its criteria that matched (`high`). In the example above, `"invoice"` scored `1.0` because all 3 criteria matched, while `"contract"` scored `0.33` because only 1 of 3 criteria matched. This structured output is useful in several ways: * **Auditability.** You can trace exactly which criteria drove a classification decision. If an invoice was misclassified, inspect the `criteria_confidence` to see which criteria matched or didn't. * **Threshold-based routing.** Instead of blindly trusting `result.category`, check the confidence score. If the top category scores below a threshold (e.g., `0.6`), flag it for human review rather than routing it automatically. * **Ambiguity detection.** If two categories score similarly (e.g., `0.67` and `0.55`), the document may be ambiguous. Use this signal to trigger a different workflow or request additional information. * **Schema refinement.** Low-confidence classifications across your pipeline tell you which criteria need to be more specific. The per-criterion breakdown pinpoints exactly which criteria are too broad or overlapping. Think of criteria as a structured checklist. This makes the classification decision transparent and programmatically accessible, not just a black-box label. *** ## Example: Confidence-Based Routing Use the per-category confidence scores to build routing logic that handles uncertain classifications gracefully. ```python Python theme={null} import requests API_KEY = "YOUR_REDUCTO_API_KEY" HEADERS = {"Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json"} # Classify the document classify_resp = requests.post( "https://platform.reducto.ai/classify", headers=HEADERS, json={ "input": "YOUR_FILE_ID", "classification_schema": [ {"category": "invoice", "criteria": ["contains billing information", "has itemized charges"]}, {"category": "contract", "criteria": ["contains legal terms", "includes signature lines"]}, ], }, ) result = classify_resp.json() # Extract the winning category and its confidence category = result["result"]["category"] confidence = next( c for c in result["response_confidence"]["categories"] if c["category"] == category ) if confidence["confidence"] >= 0.7: # High confidence: route automatically print(f"Routing {category} automatically (confidence: {confidence['confidence']})") else: # Low confidence: flag for human review print(f"Flagging for review: {category} (confidence: {confidence['confidence']})") print("Criteria breakdown:") for c in confidence["criteria_confidence"]: print(f" {c['criterion']}: {c['confidence']}") ``` ```bash cURL theme={null} # Classify and inspect confidence RESPONSE=$(curl -s -X POST https://platform.reducto.ai/classify \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "YOUR_FILE_ID", "classification_schema": [ {"category": "invoice", "criteria": ["contains billing information", "has itemized charges"]}, {"category": "contract", "criteria": ["contains legal terms", "includes signature lines"]} ] }') # Get the category and confidence CATEGORY=$(echo $RESPONSE | jq -r '.result.category') CONFIDENCE=$(echo $RESPONSE | jq -r ".response_confidence.categories[] | select(.category == \"$CATEGORY\") | .confidence") echo "Category: $CATEGORY (confidence: $CONFIDENCE)" # Show per-criterion breakdown echo "Criteria breakdown:" echo $RESPONSE | jq -r ".response_confidence.categories[] | select(.category == \"$CATEGORY\") | .criteria_confidence[] | \" \(.criterion): \(.confidence)\"" ``` *** ## Related Quick start, request parameters, and pipeline integration. Write better classification schemas for more accurate results. # Reducto CLI Source: https://docs.reducto.ai/cli Access Reducto from your terminal. The Reducto CLI gives you direct terminal access to Reducto's document capabilities: parse, extract, split, classify, and edit. Use it for batch processing, scripting, CI/CD pipelines, and quick operations without writing application code. ## Fast path for coding agents Use the CLI when the document is already on disk or when an agent needs a terminal-first workflow. It avoids writing upload code and produces Markdown files that agents can read directly. ```bash theme={null} pip install reducto-cli reducto login curl -L -o fidelity-example.pdf https://cdn.reducto.ai/samples/fidelity-example.pdf reducto parse ./fidelity-example.pdf ``` For a local file: ```bash theme={null} reducto parse ./document.pdf ``` For a directory: ```bash theme={null} reducto parse ./documents ``` The CLI writes `.parse.md` next to each input file. For agent tool calling instead of terminal commands, use the [MCP server](/mcp-server). ## Installation Install the Reducto CLI using pip: ```bash theme={null} pip install reducto-cli ``` ## Authentication Before using the CLI, authenticate by running: ```bash theme={null} reducto login ``` This command opens [Reducto Studio](https://studio.reducto.ai/) in your browser, where you can securely authenticate your CLI session. ## Quick Examples ```bash theme={null} # Parse a single file reducto parse path/to/document.pdf # Parse an entire folder reducto parse ./docs # Extract with a schema (path or inline JSON) reducto extract ./docs/invoice.pdf -s schemas/invoice.json # Edit a single file reducto edit path/to/document.pdf --instructions "Your editing instructions here" ``` Parsed outputs are written as `.parse.md`. Extraction reuses existing parses when possible and saves `.extract.json` containing only the payload. ## Supported File Types The CLI supports the same file types as the Reducto API: | Format | Extensions | | ---------------- | -------------------------------- | | PDF | `.pdf` | | Images | `.png`, `.jpg`, `.jpeg` | | Office documents | `.doc`, `.docx`, `.ppt`, `.pptx` | | Spreadsheets | `.xls`, `.xlsx` | Commands accept either a file or a directory. Directories are scanned recursively, and only supported file types are processed. ## Parse Command The `parse` command converts documents into structured markdown output. ### Flags | Flag | Description | | ------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--agentic` | Enables all agentic options for tables, text, and figures. Increases accuracy but also increases latency. Use when document quality or complex layouts require enhanced processing. | | `--change-tracking` | Enables change tracking during parsing. Returns `` tags around strikethrough text, `` tags around underlined text, and `` tags around colored adjacent strikethrough and underlined text. Useful for documents with revision history. | | `--highlights` | Include highlighted text in the parsed output. | | `--hyperlinks` | Include embedded hyperlinks in the parsed output. | | `--comments` | Include document comments in the parsed output. | ### Examples ```bash theme={null} # Basic parse reducto parse document.pdf # Parse with maximum accuracy (slower) reducto parse document.pdf --agentic # Parse a contract with change tracking reducto parse contract.pdf --change-tracking # Parse with all metadata reducto parse document.pdf --hyperlinks --comments --highlights # Combine flags as needed reducto parse legal_doc.pdf --agentic --change-tracking --comments ``` ## Extract Command The `extract` command pulls structured data from documents according to a JSON Schema you provide. It automates information extraction by mapping complex or unstructured documents into machine-readable JSON. ### Common Use Cases * Extracting line items, totals, vendor/customer info from invoices and receipts * Pulling key fields, tables, or sections from contracts or legal documents * Capturing form field values from scanned forms or applications * Summarizing structured results from reports, statements, or medical records ### Schema Guidelines * Schemas must be valid JSON Schema documents * The top-level schema **must** be an object (`{"type": "object", ...}`) — inline strings or arrays are not permitted * Provide explicit property definitions so the extractor can map fields deterministically * Schemas may be supplied as file paths or inline JSON strings ### Example Schema ```json theme={null} { "type": "object", "properties": { "items": { "type": "array", "items": { "type": "object", "properties": { "article_number": {"type": "string"}, "description": {"type": "string"}, "quantity": {"type": "number"}, "unit_price": {"type": "number"}, "total_price": {"type": "number"} }, "required": [ "article_number", "description", "quantity", "unit_price", "total_price" ] } } }, "required": ["items"] } ``` You can reuse parses across multiple extractions: the CLI automatically detects existing `.parse.md` files, rehydrates the recorded job ID, and uses `jobid://` references to accelerate extraction jobs. ## Edit Command The `edit` command modifies documents using natural language instructions. It uploads the document, applies the specified edits, and downloads the resulting file. ### Usage ```bash theme={null} reducto edit path/to/document.pdf --instructions "Your editing instructions here" reducto edit path/to/document.pdf -i "Your editing instructions here" ``` ### Parameters | Parameter | Required | Description | | ---------------------- | -------- | ------------------------------------------------------------------------------------------ | | `path` | Yes | Path to a file or directory. Directories are scanned recursively for supported file types. | | `--instructions`, `-i` | Yes | Natural language instructions describing the edits to apply. | ### Output Edited files are saved alongside the original with the naming pattern `.edited.`. For example: * `invoice.pdf` becomes `invoice.edited.pdf` * `report.docx` becomes `report.edited.docx` ### Examples ```bash theme={null} reducto edit contract.pdf -i "Fill in the client name as 'Acme Corporation' and set the contract date to January 15, 2024" reducto edit document.pdf -i "Fill out the form with: Name: John Doe, Email: john@example.com, Select 'Yes' for newsletter subscription" ``` ### Tips for Effective Instructions For best results with the `--instructions` flag: * Be specific about what content to modify and how * Reference specific elements (headers, footers, tables, specific text) * Describe the desired outcome clearly * For bulk operations on directories, ensure instructions apply uniformly to all file types ## Next Steps Learn to use the Reducto API directly for more advanced integrations. Explore all available parsing configurations. Deep dive into structured data extraction. Learn more about document editing capabilities. # SpreadsheetViewer Source: https://docs.reducto.ai/components/spreadsheet-viewer A React component for viewing Excel spreadsheets with bounding box overlays for document extraction visualization **Enterprise Component** — The SpreadsheetViewer component is available to Reducto enterprise customers. Contact [sales@reducto.ai](mailto:sales@reducto.ai) to get access. ## Overview The `SpreadsheetViewer` component provides a powerful, read-only Excel spreadsheet viewer with support for bounding box overlays. It's designed to visualize Reducto extraction results directly on your spreadsheet documents. View `.xlsx`, `.xls`, and `.xlsm` files with full sheet navigation Overlay extraction results with clickable, colored regions CSS Variables for easy theming without Tailwind dependency Full type definitions included for excellent DX ## Installation Create or update your project's `.npmrc` file to authenticate with GitHub Packages: ```bash .npmrc theme={null} @reductoai-collab:registry=https://npm.pkg.github.com //npm.pkg.github.com/:_authToken=${REDUCTO_NPM_TOKEN} ``` Never commit your token directly to `.npmrc`. Use an environment variable as shown above. Reducto will provide you with an access token. Set it as an environment variable: Add to your shell profile (`~/.bashrc`, `~/.zshrc`, etc.): ```bash theme={null} export REDUCTO_NPM_TOKEN="your-token-from-reducto" ``` Then reload your shell: ```bash theme={null} source ~/.zshrc # or ~/.bashrc ``` Set as a system environment variable: ```powershell theme={null} [System.Environment]::SetEnvironmentVariable('REDUCTO_NPM_TOKEN', 'your-token-from-reducto', 'User') ``` Restart your terminal after setting. Add `REDUCTO_NPM_TOKEN` as a secret in your CI environment: ```yaml GitHub Actions theme={null} env: REDUCTO_NPM_TOKEN: ${{ secrets.REDUCTO_NPM_TOKEN }} ``` ```yaml GitLab CI theme={null} variables: REDUCTO_NPM_TOKEN: $REDUCTO_NPM_TOKEN ``` ```bash theme={null} npm install @reductoai-collab/components ``` ```bash theme={null} yarn add @reductoai-collab/components ``` ```bash theme={null} pnpm add @reductoai-collab/components ``` Import the CSS file in your application's entry point: ```tsx App.tsx or main.tsx theme={null} import '@reductoai-collab/components/styles/spreadsheet-viewer.css'; ``` ## Quick Start Here's a minimal example to get you started: ```tsx theme={null} import { SpreadsheetViewer } from '@reductoai-collab/components'; import '@reductoai-collab/components/styles/spreadsheet-viewer.css'; function App() { return (
{ console.log(`Loaded ${metadata.sheetCount} sheets`); }} onError={(error) => { console.error(`Error: ${error.message}`); }} />
); } ``` The `SpreadsheetViewer` requires a container with defined dimensions. Always wrap it in a parent element with explicit `height` and `width`. ## Usage Examples ### Basic Viewer Display a spreadsheet without any overlays: ```tsx theme={null} import { SpreadsheetViewer } from '@reductoai-collab/components'; function BasicViewer() { return (
{ console.log('Sheet names:', metadata.sheetNames); }} />
); } ``` ### With Bounding Boxes Highlight specific regions of the spreadsheet with colored bounding boxes: ```tsx theme={null} import { SpreadsheetViewer, type BoundingBox } from '@reductoai-collab/components'; function ViewerWithBboxes() { const bboxes: BoundingBox[] = [ { id: 'header-row', page: 1, // Sheet number (1-indexed) top: 1, // Starting row left: 1, // Starting column (A = 1) width: 5, // Number of columns height: 1, // Number of rows color: 'blue', label: 'Header', }, { id: 'data-table', page: 1, top: 2, left: 1, width: 5, height: 10, color: 'green', label: 'Sales Data', metadata: { tableId: 'tbl-001', confidence: 0.95, }, }, ]; return (
{ console.log('Clicked:', bbox.id); console.log('Custom data:', bbox.metadata); }} />
); } ``` ### Integrating with Reducto API Results Convert Reducto extraction results to bounding boxes: ```tsx theme={null} import { useState } from 'react'; import { SpreadsheetViewer, type BoundingBox } from '@reductoai-collab/components'; interface ReductoBlock { id: string; type: 'table' | 'text' | 'key_value'; bbox: { page: number; top: number; left: number; width: number; height: number; }; content: string; } interface ExtractionResult { blocks: ReductoBlock[]; } function ReductoViewer({ documentUrl, extractionResult, }: { documentUrl: string; extractionResult: ExtractionResult; }) { const [selectedBlock, setSelectedBlock] = useState(null); // Map block types to colors const colorMap: Record = { table: 'blue', text: 'green', key_value: 'purple', }; // Convert Reducto blocks to bounding boxes const bboxes: BoundingBox[] = extractionResult.blocks.map((block) => ({ id: block.id, page: block.bbox.page, top: block.bbox.top, left: block.bbox.left, width: block.bbox.width, height: block.bbox.height, color: colorMap[block.type] ?? 'gray', label: block.type, metadata: { blockId: block.id, type: block.type, content: block.content, }, })); return (
{/* Viewer */}
{ const block = extractionResult.blocks.find( (b) => b.id === bbox.metadata?.blockId ); setSelectedBlock(block ?? null); }} />
{/* Details Panel */}

Selected Block

{selectedBlock ? (

Type: {selectedBlock.type}

Content:

              {selectedBlock.content}
            
) : (

Click a bounding box to see details

)}
); } ``` ### Custom Styling with CSS Variables Override the default styles using CSS variables: ```css custom-theme.css theme={null} :root { /* Container */ --sv-background: #1a1a2e; --sv-border-color: #2d2d44; --sv-border-radius: 8px; /* Tabs */ --sv-tab-background: #2d2d44; --sv-tab-background-active: #3d3d5c; --sv-tab-color: #a0a0b0; --sv-tab-color-active: #ffffff; /* Grid */ --sv-header-background: #2d2d44; --sv-header-color: #ffffff; --sv-cell-background: #1a1a2e; --sv-cell-color: #e0e0e0; --sv-cell-border-color: #2d2d44; /* Bounding boxes */ --sv-bbox-border-width: 2px; --sv-bbox-opacity: 0.15; --sv-bbox-opacity-hover: 0.25; } ``` ```css theme={null} :root { /* Container */ --sv-font-family: system-ui, -apple-system, sans-serif; --sv-background: #ffffff; --sv-border-color: #e5e7eb; --sv-border-radius: 4px; /* Tabs */ --sv-tab-height: 36px; --sv-tab-padding: 0 16px; --sv-tab-background: #f9fafb; --sv-tab-background-hover: #f3f4f6; --sv-tab-background-active: #ffffff; --sv-tab-color: #6b7280; --sv-tab-color-active: #111827; --sv-tab-font-size: 13px; --sv-tab-font-weight: 500; /* Grid */ --sv-header-background: #f9fafb; --sv-header-color: #374151; --sv-header-font-size: 12px; --sv-header-font-weight: 600; --sv-cell-background: #ffffff; --sv-cell-color: #111827; --sv-cell-font-size: 13px; --sv-cell-padding: 4px 8px; --sv-cell-border-color: #e5e7eb; --sv-cell-min-width: 80px; --sv-cell-height: 24px; --sv-row-header-width: 50px; /* Selection */ --sv-selection-background: rgba(59, 130, 246, 0.1); --sv-selection-border-color: #3b82f6; /* Scrollbar */ --sv-scrollbar-width: 8px; --sv-scrollbar-track: #f1f1f1; --sv-scrollbar-thumb: #c1c1c1; --sv-scrollbar-thumb-hover: #a1a1a1; /* Bounding boxes */ --sv-bbox-border-width: 2px; --sv-bbox-opacity: 0.1; --sv-bbox-opacity-hover: 0.2; /* Loading & Error states */ --sv-loading-color: #6b7280; --sv-error-color: #dc2626; --sv-error-background: #fef2f2; } ``` ## API Reference ### SpreadsheetViewer Props URL to the Excel file (`.xlsx`, `.xls`, or `.xlsm`). Can be a relative path, absolute URL, or blob URL. Array of bounding boxes to overlay on the spreadsheet. Callback fired when the workbook is successfully loaded. Callback fired when an error occurs loading or parsing the file. Callback fired when a bounding box is clicked. Additional CSS class name for the container element. Inline styles for the container element. ### BoundingBox Type ```typescript theme={null} interface BoundingBox { id: string; // Unique identifier page: number; // Sheet number (1-indexed) top: number; // Starting row (1-indexed) left: number; // Starting column (1-indexed, A=1) width: number; // Number of columns height: number; // Number of rows color?: BoundingBoxColor; // Box color (default: 'blue') label?: string; // Optional label text metadata?: Record; // Custom data } type BoundingBoxColor = | 'blue' | 'green' | 'red' | 'yellow' | 'purple' | 'orange' | 'pink' | 'gray'; ``` ### WorkbookMetadata Type ```typescript theme={null} interface WorkbookMetadata { sheetCount: number; // Total number of sheets sheetNames: string[]; // Array of sheet names activeSheet: number; // Currently active sheet index } ``` ### SpreadsheetError Type ```typescript theme={null} interface SpreadsheetError { code: 'FETCH_ERROR' | 'PARSE_ERROR' | 'INVALID_FORMAT'; message: string; originalError?: Error; } ``` ## Troubleshooting Ensure your `.npmrc` is configured correctly and the `REDUCTO_NPM_TOKEN` environment variable is set: ```bash theme={null} echo $REDUCTO_NPM_TOKEN # Should print your token ``` If using a monorepo, place the `.npmrc` in the root directory. If loading files from a different domain, ensure the server sends appropriate CORS headers: ``` Access-Control-Allow-Origin: * ``` Alternatively, proxy the request through your own backend or use blob URLs. Verify that: 1. The `page` property matches the sheet number (1-indexed) 2. The `top` and `left` values are within the sheet bounds 3. You've imported the CSS file Make sure you import the CSS file in your app's entry point: ```tsx theme={null} import '@reductoai-collab/components/styles/spreadsheet-viewer.css'; ``` If using CSS modules or scoped styles, ensure the import is global. ## Support Contact **[sales@reducto.ai](mailto:sales@reducto.ai)** to get access to `@reductoai-collab/components` and receive your authentication token. For technical support, reach out to your Reducto account representative or email [support@reducto.ai](mailto:support@reducto.ai). # Classify Configuration Source: https://docs.reducto.ai/configs/classify/configuration Configure page ranges and classification behavior Classify accepts a document and a list of categories, then returns the best match. This page covers the configuration options that control how classification works. ## Page Range By default, Classify uses the first 5 pages of a document as context for classification. For most documents, the first few pages contain enough information to determine document type (cover pages, headers, introductory sections). You can increase context up to 10 pages using the `page_range` parameter when distinguishing content appears deeper in the document. ```python Python theme={null} from reducto import Reducto client = Reducto() response = client.classify.run( input="https://example.com/document.pdf", page_range={"start": 1, "end": 10}, classification_schema=[ { "category": "annual_report", "criteria": ["financial statements", "shareholder letter", "auditor's report"], }, { "category": "quarterly_filing", "criteria": ["quarterly results", "interim statements"], }, ], ) ``` ```javascript Node.js theme={null} import Reducto from 'reductoai'; const client = new Reducto(); const response = await client.classify.run({ input: 'https://example.com/document.pdf', page_range: { start: 1, end: 10 }, classification_schema: [ { category: 'annual_report', criteria: ['financial statements', 'shareholder letter', "auditor's report"], }, { category: 'quarterly_filing', criteria: ['quarterly results', 'interim statements'], }, ], }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/classify \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "https://example.com/document.pdf", "page_range": {"start": 1, "end": 10}, "classification_schema": [ { "category": "annual_report", "criteria": ["financial statements", "shareholder letter", "auditor'\''s report"] }, { "category": "quarterly_filing", "criteria": ["quarterly results", "interim statements"] } ] }' ``` * Page numbers are 1-indexed (first page is page 1). * Both `start` and `end` are inclusive. * If no `page_range` is specified, the first 5 pages are used. * If more than 10 pages are selected, the request returns an error. * Only applies to PDFs. Ignored for other document types. Each page of context costs 0.5 credits. Using the default 5 pages costs 2.5 credits per classification. Increasing to 10 pages costs 5.0 credits. Only increase when the default pages don't contain enough distinguishing content. See [Credit Usage](/reference/credit-usage) for details. ## Classification Schema The `classification_schema` parameter defines what categories Classify can return. Each category needs a name and a list of criteria. ### Writing effective criteria Criteria are natural language descriptions that tell the model what to look for. More specific criteria produce better results. **Good criteria** describe observable features: * "Contains a table of itemized charges with quantities and unit prices" * "Includes signature blocks for multiple parties" * "Has a header with 'INVOICE' or invoice number" **Weak criteria** are too generic: * "Business document" * "Has text" * "Contains information" ### Example: Financial document routing ```python theme={null} response = client.classify.run( input=upload.file_id, classification_schema=[ { "category": "invoice", "criteria": [ "itemized list of charges or line items", "total amount due", "billing and payment information", "vendor or supplier details", ], }, { "category": "bank_statement", "criteria": [ "account balance and transaction history", "deposits and withdrawals listed by date", "bank name and account number", ], }, { "category": "tax_form", "criteria": [ "tax identification numbers (SSN, EIN)", "income and deduction categories", "IRS form number (W-2, 1099, 1040)", ], }, { "category": "receipt", "criteria": [ "single transaction with date and amount", "store or merchant name", "payment method (cash, card, etc.)", ], }, ], ) ``` *** ## Related Introduction to document classification. Route classified documents to Parse and Extract. Classification pricing details. # Form Schema Source: https://docs.reducto.ai/configs/edit/form-schema Pre-define field locations for faster, more consistent PDF form filling Form schemas let you define exactly where form fields are located in a PDF, what type they are, and how they should be filled. Instead of relying on Edit to detect fields each time, you provide the field definitions upfront. This matters for two reasons: **speed** and **consistency**. With a form schema, Edit skips field detection and description generation, processing forms significantly faster. And because the same fields are targeted every time, you get deterministic results across thousands of form fills. *** ## The Workflow 1. **Run Edit once without a form\_schema** to let Reducto detect all fields 2. **Save the returned `form_schema`** from the response 3. **Use that schema for subsequent fills** of the same form type ```python Python theme={null} from reducto import Reducto import json client = Reducto() # First run: let Edit detect fields result = client.edit.run( document_url="https://example.com/w9-blank.pdf", edit_instructions="Fill with: Name: Test Corp, EIN: 12-3456789" ) # Save the detected schema (convert Pydantic objects to dicts) schema_dicts = [field.model_dump() for field in result.form_schema] with open("w9_schema.json", "w") as f: json.dump(schema_dicts, f) # Subsequent runs: much faster with saved schema with open("w9_schema.json") as f: saved_schema = json.load(f) result = client.edit.run( document_url="https://example.com/w9-blank.pdf", edit_instructions="Fill with: Name: Acme Inc, EIN: 98-7654321", form_schema=saved_schema ) ``` ```javascript Node.js theme={null} import Reducto from 'reductoai'; import fs from 'fs'; const client = new Reducto(); // First run: let Edit detect fields const result = await client.edit.run({ document_url: 'https://example.com/w9-blank.pdf', edit_instructions: 'Fill with: Name: Test Corp, EIN: 12-3456789' }); // Save the detected schema fs.writeFileSync('w9_schema.json', JSON.stringify(result.form_schema, null, 2)); // Subsequent runs: much faster with saved schema const savedSchema = JSON.parse(fs.readFileSync('w9_schema.json', 'utf-8')); const result2 = await client.edit.run({ document_url: 'https://example.com/w9-blank.pdf', edit_instructions: 'Fill with: Name: Acme Inc, EIN: 98-7654321', form_schema: savedSchema }); ``` ```bash cURL theme={null} # First run: detect fields and save schema curl -s -X POST https://platform.reducto.ai/edit \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "document_url": "https://example.com/w9-blank.pdf", "edit_instructions": "Fill with: Name: Test Corp, EIN: 12-3456789" }' | jq '.form_schema' > w9_schema.json # Subsequent runs: much faster with saved schema SCHEMA=$(cat w9_schema.json) curl -X POST https://platform.reducto.ai/edit \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "document_url": "https://example.com/w9-blank.pdf", "edit_instructions": "Fill with: Name: Acme Inc, EIN: 98-7654321", "form_schema": '"$SCHEMA"' }' ``` The first call runs field detection and generates descriptions. Subsequent calls with the schema skip those steps entirely. | Scenario | Pipeline | | -------------------- | ----------------------------------------- | | Without form\_schema | Detection → Context → Descriptions → Fill | | With form\_schema | Fill only | *** ## Schema Structure A form schema is an array of field definitions: ```python theme={null} form_schema = [ { "bbox": { "left": 0.227, # Distance from left edge (0-1) "top": 0.144, # Distance from top edge (0-1) "width": 0.15, # Width as fraction of page "height": 0.025, # Height as fraction of page "page": 1 # Page number (1-indexed) }, "description": "Bank Routing/ABA Number", "type": "text", "fill": True, # Let LLM determine value (default) "value": None # No fixed value }, { "bbox": {"left": 0.432, "top": 0.144, "width": 0.2, "height": 0.025, "page": 1}, "description": "Bank Name", "type": "text", "value": "Wells Fargo" # Fixed value bypasses LLM }, { "bbox": {"left": 0.227, "top": 0.54, "width": 0.02, "height": 0.02, "page": 1}, "description": "Domestic wire checkbox", "type": "checkbox" } ] ``` ### Field Properties | Property | Type | Required | Description | | ------------- | ------- | -------- | ------------------------------------------------ | | `bbox` | object | Yes | Normalized coordinates (0-1 range from top-left) | | `description` | string | Yes | Used by LLM to map instructions to this field | | `type` | string | Yes | `text`, `checkbox`, `dropdown`, or `barcode` | | `fill` | boolean | No | Whether to fill this field (default: `true`) | | `value` | string | No | Fixed value that bypasses LLM | ### Bounding Box Coordinates are normalized (0-1), measured from the **top-left corner**: ``` (0,0) ─────────────────────── (1,0) │ │ │ ┌─────────┐ │ │ │ Field │ left: 0.1 │ │ └─────────┘ top: 0.2 │ │ │ (0,1) ─────────────────────── (1,1) ``` Page numbers are 1-indexed. The first page is `page: 1`. *** ## Fill Control The `fill` and `value` properties control how each field is handled: | fill | value | Behavior | | ---------------- | ------------- | ------------------------------------------- | | `true` (default) | `null` | LLM determines value from instructions | | `true` | `"something"` | Uses this exact value, ignores instructions | | `false` | any | Field left empty | Use `value` for fields that should always contain the same thing (form version, tax year). Use `fill: false` for fields that should stay blank (signature boxes). *** ## Widget Types ### Text Standard text input fields. The LLM extracts the relevant value from your instructions based on the field's description. ```python theme={null} { "type": "text", "description": "Social Security Number in XXX-XX-XXXX format", "bbox": {"left": 0.55, "top": 0.72, "width": 0.4, "height": 0.03, "page": 1} } ``` Include format expectations in the description. "SSN in XXX-XX-XXXX format" produces better results than just "SSN" because the LLM knows how to format the output. ### Checkbox Boolean fields that get checked or unchecked. The LLM interprets your instructions to determine whether the box should be checked. ```python theme={null} { "type": "checkbox", "description": "US Citizen - Yes", "bbox": {"left": 0.15, "top": 0.35, "width": 0.02, "height": 0.02, "page": 1} } ``` Checkbox bounding boxes should be small and roughly square. Make descriptions explicit about what checking means: "US Citizen - Yes" is clearer than "Citizenship" when there are Yes/No checkbox pairs. ### Dropdown Selection fields with predefined options. The LLM suggests a value, and Edit selects the matching option from the PDF's dropdown. ```python theme={null} { "type": "dropdown", "description": "State of incorporation", "bbox": {"left": 0.3, "top": 0.4, "width": 0.2, "height": 0.03, "page": 1} } ``` The value must exactly match an available option. If your instructions say "CA" but the dropdown only contains "California", the field is skipped silently. Consider listing options in the description: "State (CA, NY, TX, ...)" to help the LLM match correctly. ### Barcode Special fields for barcode data. These are typically detected automatically in forms that have barcode regions and filled with encoded data. ```python theme={null} { "type": "barcode", "description": "Document tracking code", "bbox": {"left": 0.7, "top": 0.9, "width": 0.25, "height": 0.05, "page": 1} } ``` *** ## Troubleshooting Coordinates are from the top-left (0,0), with Y increasing downward. If you measured from bottom-left, flip the Y values: `correct_top = 1 - your_top - height` Start with a single field, verify it works, then add more incrementally. Edit matches schema fields to existing widgets using bounding box overlap. If overlap is less than 50%, a new widget is created. Run Edit without a schema first to see actual widget positions, then adjust your coordinates to match. 1. Ensure `type` is `"checkbox"`, not `"text"` 2. Bounding boxes should be small and square 3. Be explicit in instructions: "Check the 'Yes' checkbox for US citizenship" Check for: * `fill: false` on the field * Dropdown value not matching available options exactly * Instructions don't mention data for this field * Unsupported widget types (signatures, images) # Array Extraction Source: https://docs.reducto.ai/configs/extract/array-extraction Extract complete arrays from long documents without truncation Array Extraction is being deprecated. We recommend using [Deep Extract](/configs/extract/deep-extract) instead, which provides higher accuracy on complex and long extractions through an agentic loop. Array extraction is a mode specifically designed for extracting **arrays** (lists of items) from documents. It exists because LLMs have context limits that cause them to truncate long lists. The core problem: if you ask an LLM to extract 500 transactions from a bank statement, it might return only the first 50-100 before stopping. Array extraction solves this by splitting the document into segments, extracting the array items from each segment, then merging results. **This only affects array fields in your schema.** Scalar fields (strings, numbers, single objects) are still extracted from the full document context normally. ## When to Use ```python theme={null} result = client.extract.run( input=upload, instructions={"schema": schema}, settings={"array_extract": True} ) ``` Enable it when: * Your schema has array fields with many items (50+) * Extraction results look truncated or end abruptly * Tables span multiple pages If you're extracting a few scalar fields like "invoice\_number" and "total\_amount", you don't need this. ## How It Works 1. **Segment** the document into overlapping page ranges 2. **Extract** array items from each segment independently 3. **Merge** all array items together 4. **Deduplicate** items that appeared in overlapping regions Segments overlap at boundaries to catch items that span page breaks. If a table row starts on page 10 and continues to page 11, both segments will capture it, and deduplication removes the duplicate. ## Schema Requirements Your schema must have at least one top-level array property: ```python theme={null} schema = { "type": "object", "properties": { "transactions": { # This array is extracted segment-by-segment "type": "array", "items": { "type": "object", "properties": { "date": {"type": "string"}, "amount": {"type": "number"} } } }, "account_number": {"type": "string"} # This scalar is extracted normally } } ``` The schema root must be an object, not an array: ```python theme={null} # Wrong - will error {"type": "array", "items": {...}} # Correct {"type": "object", "properties": {"items": {"type": "array", ...}}} ``` ## Deduplication When segments overlap, the same item may be extracted twice. Reducto deduplicates using content similarity. **Problem:** If your document has legitimately identical items (two transactions with the same date and amount), deduplication might incorrectly merge them. **Solution:** Add distinguishing fields like line numbers or IDs: ```python theme={null} "items": { "type": "object", "properties": { "line_number": {"type": "integer", "description": "Row number if visible"}, "date": {"type": "string"}, "amount": {"type": "number"} } } ``` ## With Citations Array extraction works with citations. Each item retains its source location: ```python theme={null} result = client.extract.run( input=upload, instructions={"schema": schema}, settings={ "array_extract": True, "citations": {"enabled": True} } ) for item in result.result["transactions"]: if item["amount"].citations: page = item["amount"].citations[0].bbox.page print(f"${item['amount'].value} on page {page}") ``` ## Troubleshooting **Still missing items:** 1. Check Parse output first (`client.parse.run`). Extract can only find what Parse sees. 2. Add system prompt: "Extract every item. Do not skip any rows." **Duplicate items:** Add unique identifiers (line numbers, IDs) to help differentiation. **Schema error:** Ensure at least one `"type": "array"` property exists at the top level. # Citations Source: https://docs.reducto.ai/configs/extract/citations Link extracted values to their source locations in the document Citations tell you exactly where each extracted value came from in the document. When enabled, every field includes bounding box coordinates pointing to the source text. This matters for: * **Verification**: Confirm extractions are correct by checking source text * **Compliance**: Maintain audit trails for regulated workflows * **Debugging**: See where the model looked when values are wrong * **User experience**: Let users click from extracted data to the original location ```python theme={null} result = client.extract.run( input=upload, instructions={"schema": schema}, settings={ "citations": {"enabled": True} } ) ``` ## Response Structure With citations enabled, each value becomes an object with `value` and `citations`: ```json theme={null} { "result": { "invoice_total": { "value": 1575.00, "citations": [ { "type": "Table", "content": "Total Due: $1,575.00", "bbox": { "left": 0.65, "top": 0.82, "width": 0.25, "height": 0.03, "page": 1, "original_page": 1 }, "confidence": "high", "granular_confidence": { "extract_confidence": 0.95, "parse_confidence": 0.91 }, "parentBlock": { "type": "Table", "content": "Subtotal: $1,500.00\nTax: $75.00\nTotal Due: $1,575.00", "bbox": {"left": 0.60, "top": 0.75, "width": 0.35, "height": 0.12, "page": 1} } } ] } } } ``` **Fields:** * `type`: Block type (`Text`, `Table`, `Key Value`, etc.) * `content`: The source text * `bbox`: Bounding box coordinates (normalized 0-1 for PDFs/images) * `confidence`: `"high"` or `"low"` * `granular_confidence`: Numeric scores (`extract_confidence`, `parse_confidence`) between 0-1 * `parentBlock`: The larger Parse block containing this citation, for context ## Working with Citations **Accessing a scalar field:** ```python theme={null} invoice_number = result.result["invoice_number"] print(f"Value: {invoice_number.value}") if invoice_number.citations: citation = invoice_number.citations[0] print(f"Found on page {citation.bbox.page}") print(f"Source text: {citation.content}") ``` **Looping through array items:** ```python theme={null} for item in result.result["line_items"]: amount = item["amount"] print(f"${amount.value}") if amount.citations: print(f" Found on page {amount.citations[0].bbox.page}") ``` ## Bounding Box Coordinates For PDFs and images, coordinates are normalized to \[0, 1] relative to page dimensions. `left: 0.5` means halfway across the page. `page` is the page number in the processed result. `original_page` is the page number in the original document, which differs when you use page ranges. To convert to pixels, multiply by page dimensions: ```python theme={null} x_px = bbox["left"] * page_width_px y_px = bbox["top"] * page_height_px ``` ## Spreadsheet Citations Excel and CSV files use cell coordinates instead of normalized positions: * `left`: Column number (1-indexed, so 1 = A, 2 = B, 3 = C) * `top`: Row number (1-indexed, so 1 = first row) * `page`: Sheet number (1-indexed position of the sheet in the workbook) A citation with `{"left": 3, "top": 15, "page": 2}` points to cell C15 on the second sheet. All extraction modes use this convention, including `spreadsheet_agent` and `deep_extract`. ## Confidence Scores Each citation includes a `confidence` field with a categorical value (`"high"` or `"low"`). By default in v3, `numerical_confidence` is **enabled**, which adds `granular_confidence` with numeric 0-1 scores: ```json theme={null} { "confidence": "high", "granular_confidence": { "extract_confidence": 0.95, "parse_confidence": 0.91 } } ``` * `extract_confidence`: How confident the LLM is about the extraction * `parse_confidence`: How confident the OCR/parsing is about the underlying text To disable numeric scores and only get categorical confidence: ```python theme={null} settings={ "citations": { "enabled": True, "numerical_confidence": False # Only return "high"/"low" } } ``` Low `parse_confidence` suggests OCR errors. Low `extract_confidence` suggests the model was uncertain about interpretation. ## Constraints **Citations disable chunking:** The document is processed as a single unit to maintain precise coordinate mapping. **Empty citations:** A field's `citations` array can be empty in two cases. The value was inferred rather than found directly in the document, or the value was found but Reducto could not reliably localize it to a source span. The second case is rare. In both cases the `value` is still returned correctly. Always check `if field.citations:` before accessing them. For an inferred value, re-running will not add a citation, so adjust the schema or field description if you need the value tied to source text. For the rare localization failure, re-running the extraction can recover the citation. ## Studio Visualization Every response includes a `studio_link`. In Studio, citations are interactive: * Click an extracted field to highlight its source in the document * Click a highlight to jump to the corresponding field # Deep Extract Source: https://docs.reducto.ai/configs/extract/deep-extract Achieve near-perfect accuracy on complex and long extractions with our agentic loop ## What is Deep Extract? Deep Extract is an agentic extraction mode that iteratively refines its output to achieve near-perfect accuracy. Unlike standard extraction which makes a single pass over the document, Deep Extract runs an agentic loop that verifies and corrects its results against the source document until a quality threshold is met. This is especially powerful for complex documents where a single extraction pass may miss values, misalign table rows, or produce inconsistencies. Deep Extract catches these issues by checking its own work and re-extracting until the results are accurate. *** ## When to Use It Deep Extract is designed for extractions where accuracy is critical and the cost of errors is high. Common use cases include: * **Invoice line item extraction** — Ensuring every line item is captured and that totals reconcile with the sum of individual amounts * **Financial statement processing** — Extracting balance sheets, income statements, or cash flow data where numbers must be internally consistent * **Legal document extraction** — Pulling clauses, dates, and party information from contracts where missing a single field has real consequences * **Medical and insurance forms** — Capturing patient data, procedure codes, and billing amounts that must be complete and correct * **Multi-page tables** — Documents with hundreds of rows spanning many pages where standard extraction may truncate or skip entries If your extraction is simple (a few scalar fields from a short document), standard extraction is sufficient. Use Deep Extract when you need high reliability on complex or lengthy documents. *** ## How to Use It Enable Deep Extract by setting `deep_extract` to `true` in the `settings` object: ```python Python theme={null} result = client.extract.run( input=upload.file_id, instructions={ "schema": schema, "system_prompt": "Extract all line items from this invoice. Iterate until the line items sum up to the total listed in the document." }, settings={ "deep_extract": True } ) ``` ```javascript Node.js theme={null} const result = await client.extract.run({ input: upload.file_id, instructions: { schema, system_prompt: 'Extract all line items from this invoice. Iterate until the line items sum up to the total listed in the document.' }, settings: { deep_extract: true } }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/extract \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "instructions": { "schema": {...}, "system_prompt": "Extract all line items from this invoice. Iterate until the line items sum up to the total listed in the document." }, "settings": { "deep_extract": true } }' ``` *** ## Best Practices ### Add verification criteria to your system prompt The agentic loop uses your system prompt to determine when extraction is "good enough." Including explicit verification criteria gives the agent a concrete goal to iterate toward. ```python theme={null} instructions={ "schema": invoice_schema, "system_prompt": ( "Extract all line items from this invoice. " "Iterate until the line items sum up to the total listed in the document." ) } ``` Other examples of effective verification criteria: * **Financial documents:** "Verify that the sum of all transaction amounts equals the statement total." * **Multi-page tables:** "Ensure every row in the table is captured. The document states there are N entries — verify the count matches." * **Contracts:** "Confirm that all parties listed in the signature block are captured in the parties array." ### Use with a well-defined schema Deep Extract works best when your schema has clear field names and descriptions. The agent uses these to understand what it's looking for during each iteration. See [Extract Best Practices](/extraction/best-practices-extract) for schema design guidance. ### Pair with Parse configuration Deep Extract can only verify and refine what Parse sees. If the underlying parse output is missing data (e.g., a table isn't detected), Deep Extract won't be able to find it either. Consider enabling [agentic mode](/configs/parse/agentic-modes) for tables or using [HTML table output](/configs/parse/table-output-formats) for complex documents. *** ## Related Endpoint basics and parameters. Schema design and prompt writing tips. Link values to source locations. Understand how credits are calculated. # Configuration Overview Source: https://docs.reducto.ai/configs/overview Configure every step of Reducto document processing Every Reducto endpoint exposes configuration options that control how documents are processed. This section covers all available configurations, from parse-level OCR and layout settings through extraction schemas and workflow orchestration. ## Configuration by Endpoint Parse converts documents into structured content. Options are grouped by purpose: | Group | Purpose | Pages | | ------------- | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | | `enhance` | AI-powered accuracy | [Agentic Modes](/configs/parse/agentic-modes), [Chart Extraction](/configs/parse/chart-extraction) | | `retrieval` | RAG optimization | [Chunking Methods](/configs/parse/chunking-methods) | | `formatting` | Detecting styling & output format | [Table Formats](/configs/parse/table-output-formats), [Additional Document Data](/configs/parse/additional-document-data) | | `spreadsheet` | Excel/CSV handling | [Spreadsheet Processing](/configs/parse/spreadsheet) | | `settings` | Processing controls | [Processing Settings](/configs/parse/ocr-settings), [Page Ranges](/configs/parse/page-ranges) | ```python theme={null} result = client.parse.run( input=upload, enhance={...}, retrieval={...}, formatting={...}, spreadsheet={...}, settings={...} ) ``` Extract pulls structured data from documents using a JSON schema. | Group | Purpose | Pages | | -------------- | --------------------------- | ---------------------------------------------------------------------------------------------- | | `instructions` | Schema and system prompt | (base config) | | `settings` | Citations, array extraction | [Array Extraction](/configs/extract/array-extraction), [Citations](/configs/extract/citations) | | `parsing` | Document processing | All Parse options | ```python theme={null} result = client.extract.run( input=upload, instructions={"schema": {...}, "system_prompt": "..."}, settings={"deep_extract": True, "citations": {"enabled": True}}, parsing={...} ) ``` Split divides documents into logical sections. | Group | Purpose | Pages | | ------------------- | ---------------------- | --------------------------------------------------- | | `split_description` | Section definitions | [Split Configuration](/configs/split/configuration) | | `split_rules` | Splitting logic prompt | [Split Configuration](/configs/split/configuration) | | `settings` | Table handling | [Split Configuration](/configs/split/configuration) | | `parsing` | Document processing | All Parse options | ```python theme={null} result = client.split.run( input=upload, split_description=[{"name": "...", "description": "..."}], split_rules="...", settings={"table_cutoff": "truncate"} ) ``` Classify determines document type based on natural language criteria. | Group | Purpose | Pages | | ----------------------- | --------------------- | --------------------------------------------------------- | | `classification_schema` | Category definitions | [Classify Configuration](/configs/classify/configuration) | | `page_range` | Pages used as context | [Classify Configuration](/configs/classify/configuration) | ```python theme={null} response = client.classify.run( input=upload, classification_schema=[ {"category": "invoice", "criteria": ["billing info", "itemized charges"]}, {"category": "contract", "criteria": ["legal terms", "signatures"]}, ] ) ``` Edit fills forms and modifies documents. | Option | Purpose | Pages | | ------------------- | ----------------------------- | ---------------------------------------- | | `edit_instructions` | Natural language instructions | (base config) | | `form_schema` | Pre-defined field locations | [Form Schema](/configs/edit/form-schema) | | `edit_options` | Highlight color, overflow | (base config) | ```python theme={null} result = client.edit.run( document_url=upload, edit_instructions="Fill name: John Doe, date: 2024-01-15", form_schema=[...], edit_options={"color": "#FF0000"} ) ``` ## Common Patterns Variable chunking with embedding optimization for vector search: ```python theme={null} result = client.parse.run( input=upload, retrieval={ "chunking": {"chunk_mode": "variable", "chunk_size": 1000}, "embedding_optimized": True }, formatting={"table_output_format": "dynamic"} ) ``` Enable agentic mode for both text and tables: ```python theme={null} result = client.parse.run( input=upload, enhance={ "agentic": [{"scope": "text"}, {"scope": "table"}] } ) ``` Deep Extract with source locations for long or complex documents (higher cost/latency): ```python theme={null} result = client.extract.run( input=upload, instructions={"schema": schema}, settings={ "deep_extract": True, "citations": {"enabled": True} } ) ``` ## Migrating from v2 If you're using the legacy configuration format, use this converter to transform your v2 config to v3: See the [Migration Guide](/v/legacy/migration-guide) for complete mapping tables and examples. # Additional Document Data Source: https://docs.reducto.ai/configs/parse/additional-document-data Extract revision marks, comments, highlights, hyperlinks, and signatures When you parse a document, Reducto extracts the main text content by default. But documents often contain additional information layered on top: revision marks from Track Changes, margin comments, highlighted passages, hyperlinks, and signatures. The `formatting.include` option lets you extract these. ```python Python theme={null} result = client.parse.run( input=upload.file_id, formatting={ "include": ["change_tracking", "comments", "highlight", "hyperlinks", "signatures"] } ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, formatting: { include: ['change_tracking', 'comments', 'highlight', 'hyperlinks', 'signatures'] } }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "formatting": { "include": ["change_tracking", "comments", "highlight", "hyperlinks", "signatures"] } }' ``` By default, none of these are extracted. Enable only what you need, since each adds processing overhead. These formatting options are available in the Python SDK, Node.js SDK, and via cURL. The Go SDK has limited support—only `enable_underlines` (for change tracking) is currently available. ## Change Tracking Legal documents, contracts, and collaborative drafts often use underlines and strikethroughs to show what changed between versions. Reducto can detect these and wrap them in HTML tags so you can programmatically identify revisions. ```python Python theme={null} formatting={"include": ["change_tracking"]} ``` ```javascript Node.js theme={null} formatting: { include: ['change_tracking'] } ``` ```bash cURL theme={null} "formatting": {"include": ["change_tracking"]} ``` When enabled, underlined and struck-through text appears with markup: ```html theme={null} The agreement shall commence on January 1, 2024 February 15, 2024. ``` The `` tag marks strikethrough (typically deletions), `` marks underlines (typically insertions), and `` wraps the entire revision region. **How it works:** For digital PDFs and Word documents, Reducto reads the embedded formatting information. For scanned documents, it uses a segmentation model to visually detect underlines and strikethroughs on the page image. **Common uses:** * Contract review: automatically extract what changed between versions * Compliance: track modifications to policies and procedures * Editorial workflows: preserve editor suggestions in parsed output ## Comments PDF sticky notes, Word margin comments, and Excel cell notes contain reviewer feedback, questions, and instructions that are separate from the document content itself. Reducto extracts these as distinct blocks. ```python Python theme={null} formatting={"include": ["comments"]} ``` ```javascript Node.js theme={null} formatting: { include: ['comments'] } ``` ```bash cURL theme={null} "formatting": {"include": ["comments"]} ``` Each comment becomes its own block with the comment text and its position on the page: ```json theme={null} { "type": "Comment", "content": "Verify this figure with the finance team before publishing", "bbox": {"left": 0.85, "top": 0.15, "width": 0.1, "height": 0.05, "page": 1} } ``` The bounding box tells you where the comment annotation appeared (normalized to \[0, 1] relative to page size). This lets you correlate comments with nearby content if needed. ## Highlights Highlighted text usually signals importance. Reducto can detect highlighted passages and wrap them in `` tags, letting you identify what reviewers or authors emphasized. ```python Python theme={null} formatting={"include": ["highlight"]} ``` ```javascript Node.js theme={null} formatting: { include: ['highlight'] } ``` ```bash cURL theme={null} "formatting": {"include": ["highlight"]} ``` Output: ```html theme={null} The key finding was that revenue increased 47% year-over-year despite market headwinds. ``` **How it works:** For digital documents, Reducto reads highlight annotations. For scanned documents, it uses a segmentation model to detect colored highlighting (typically yellow, but other colors work too). **Common uses:** * Extract key passages from research documents * Identify what reviewers marked as significant during review * Use highlights as importance signals for summarization ## Hyperlinks Documents contain links to external resources, internal references, and citations. Reducto extracts these and converts them to markdown link format, preserving both the display text and the URL. ```python Python theme={null} formatting={"include": ["hyperlinks"]} ``` ```javascript Node.js theme={null} formatting: { include: ['hyperlinks'] } ``` ```bash cURL theme={null} "formatting": {"include": ["hyperlinks"]} ``` Output: ```markdown theme={null} For more details, see [our methodology paper](https://example.com/methodology.pdf). ``` **Common uses:** * Build reference lists from academic papers * Audit documents for broken or outdated links * Extract cited sources for verification ## Signatures Forms and contracts often contain signature fields. Reducto can detect where signatures appear, which is useful for determining whether a document has been signed or for locating signature regions for downstream processing. ```python Python theme={null} formatting={"include": ["signatures"]} ``` ```javascript Node.js theme={null} formatting: { include: ['signatures'] } ``` ```bash cURL theme={null} "formatting": {"include": ["signatures"]} ``` Detected signatures appear as blocks marking their location: ```json theme={null} { "type": "Signature", "content": "", "bbox": {"left": 0.1, "top": 0.8, "width": 0.3, "height": 0.1, "page": 2} } ``` The actual signature image is not extracted (for privacy). The block identifies where a signature was detected. **Common uses:** * Verify that forms have been signed before processing * Route unsigned documents back for completion * Classify documents as signed vs. unsigned ## Format Compatibility Not all features work with all document types: **Change tracking and highlights** work best with Word documents, which store this information natively. For PDFs, Reducto uses visual detection models, which work well but may miss subtle formatting. Scanned documents rely entirely on visual detection. **Comments** work with PDF annotations (sticky notes), Word margin comments, and Excel cell notes. Scanned documents don't have extractable comments. **Hyperlinks** work with PDFs, Word documents, and HTML files that contain embedded links. Scanned documents don't preserve hyperlink information. **Signatures** are detected visually, so they work across all document types including scans. # Agentic Modes Source: https://docs.reducto.ai/configs/parse/agentic-modes AI-powered accuracy improvements for text, tables, and figures Agentic mode adds a vision-language model review layer to catch errors that single-pass OCR misses. You enable it per content type using scopes. ## Basic Usage ```python Python theme={null} result = client.parse.run( input=upload.file_id, enhance={ "agentic": [ {"scope": "text"}, {"scope": "table"}, {"scope": "figure", "advanced_chart_agent": True} ] } ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, enhance: { agentic: [ { scope: 'text' }, { scope: 'table' }, { scope: 'figure', advanced_chart_agent: true } ] } }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "enhance": { "agentic": [ {"scope": "text"}, {"scope": "table"}, {"scope": "figure", "advanced_chart_agent": true} ] } }' ``` Each scope is independent. Enable only what you need. Agentic modes are available in the Python SDK, Node.js SDK, and via cURL. The Go SDK does not yet support agentic modes. ## Text Scope Adds a second OCR pass using a vision-language model to correct text extraction errors. ```python Python theme={null} enhance={"agentic": [{"scope": "text"}]} ``` ```javascript Node.js theme={null} enhance: { agentic: [{ scope: 'text' }] } ``` ```bash cURL theme={null} "enhance": {"agentic": [{"scope": "text"}]} ``` **What it fixes:** * Handwritten notes and signatures * Small or faded text * Special characters and mathematical notation * Mixed fonts and styles **Custom prompts** apply only to form regions (key-value fields like "Name: John Doe"), not body text: ```python Python theme={null} enhance={"agentic": [{"scope": "text", "prompt": "Extract all dates in MM/DD/YYYY format"}]} ``` ```javascript Node.js theme={null} enhance: { agentic: [{ scope: 'text', prompt: 'Extract all dates in MM/DD/YYYY format' }] } ``` ```bash cURL theme={null} "enhance": {"agentic": [{"scope": "text", "prompt": "Extract all dates in MM/DD/YYYY format"}]} ``` ## Table Scope Uses a VLM to reconstruct table structure after initial extraction. ```python Python theme={null} enhance={"agentic": [{"scope": "table"}]} ``` ```javascript Node.js theme={null} enhance: { agentic: [{ scope: 'table' }] } ``` ```bash cURL theme={null} "enhance": {"agentic": [{"scope": "table"}]} ``` **What it fixes:** * Merged cells (rowspan/colspan) * Nested or multi-level headers * Tables with missing or faint borders * Rotated text in cells **Custom prompts** guide the reconstruction: ```python Python theme={null} enhance={"agentic": [{"scope": "table", "prompt": "Preserve currency symbols. Align rows by date."}]} ``` ```javascript Node.js theme={null} enhance: { agentic: [{ scope: 'table', prompt: 'Preserve currency symbols. Align rows by date.' }] } ``` ```bash cURL theme={null} "enhance": {"agentic": [{"scope": "table", "prompt": "Preserve currency symbols. Align rows by date."}]} ``` ### Table Mode The `mode` option controls whether Reducto selectively applies agentic enrichment only to tables likely to benefit or runs agentic enrichment on every table. Defaults to `"default"`. ```python Python theme={null} enhance={"agentic": [{"scope": "table", "mode": "max"}]} ``` ```javascript Node.js theme={null} enhance: { agentic: [{ scope: 'table', mode: 'max' }] } ``` ```bash cURL theme={null} "enhance": {"agentic": [{"scope": "table", "mode": "max"}]} ``` | Mode | Behavior | | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `"default"` | Reducto selectively applies deeper table processing where it is likely to improve results, reducing latency while maintaining accuracy for complex tables. This is the default. | | `"max"` | Every table receives full agentic enrichment. Use this when every table in the job should be agentically enriched. | Use `mode: "max"` when you want every table in a job to be agentically enriched. ## Figure Scope Enables enhanced figure processing. Without additional options, it uses more powerful models and better classifies figures as charts vs. images. Note: `summarize_figures` must also be enabled for this configuration to be activated. ```python Python theme={null} enhance={"agentic": [{"scope": "figure"}]} ``` ```javascript Node.js theme={null} enhance: { agentic: [{ scope: 'figure' }] } ``` ```bash cURL theme={null} "enhance": {"agentic": [{"scope": "figure"}]} ``` To extract numerical data from charts as structured tables, enable `advanced_chart_agent`: ```python Python theme={null} enhance={"agentic": [{"scope": "figure", "advanced_chart_agent": True}]} ``` ```javascript Node.js theme={null} enhance: { agentic: [{ scope: 'figure', advanced_chart_agent: true }] } ``` ```bash cURL theme={null} "enhance": {"agentic": [{"scope": "figure", "advanced_chart_agent": true}]} ``` | Option | What it does | | ---------------------- | ---------------------------------------------- | | `advanced_chart_agent` | Extracts numerical values as structured tables | | `prompt` | Custom instructions for figure processing | For details on chart types and the extraction pipeline, see [Chart Extraction](/configs/parse/chart-extraction). ## Figure Summarization (Default) To generate generic text descriptions of images and figures, enable `summarize_figures`: ```python Python theme={null} enhance={"summarize_figures": True} # This is the default ``` ```javascript Node.js theme={null} enhance: { summarize_figures: true } // This is the default ``` ```bash cURL theme={null} "enhance": {"summarize_figures": true} ``` This is lightweight and on by default. It makes visual content searchable but doesn't extract numerical data. Note: this must be enabled in addition to figure scope for enhanced figure summarization to be activated. **To disable it:** ```python Python theme={null} enhance={"summarize_figures": False} ``` ```javascript Node.js theme={null} enhance: { summarize_figures: false } ``` ```bash cURL theme={null} "enhance": {"summarize_figures": false} ``` ## When to Use What | Situation | Configuration | | --------------------------------------- | --------------------------------------------------- | | Clean, digital-native PDFs | Skip agentic entirely | | Documents with handwriting | `{"scope": "text"}` | | Complex financial tables | `{"scope": "table"}` | | Every table requires agentic enrichment | `{"scope": "table", "mode": "max"}` | | Charts you need searchable | Default `summarize_figures` is enough | | Charts you need data from | `{"scope": "figure", "advanced_chart_agent": True}` | | All of the above | Combine all three scopes | ## Cost and Latency Each scope adds processing time and cost. For cost-sensitive workloads: * `table` scope alone handles the most common accuracy issues and defaults to selective enrichment * `table` scope with `mode: "max"` enriches every table and adds latency * `text` scope is most valuable for handwritten content * `figure` scope with `advanced_chart_agent` is expensive, so use only when you need numerical data from charts # Chart Extraction Source: https://docs.reducto.ai/configs/parse/chart-extraction Extract structured data from charts and graphs Reducto can extract numerical data from visualizations and output it as structured tables. This page covers how to configure chart extraction and what chart types are supported. ## Three Levels of Chart Processing Reducto offers three ways to process charts, each with different accuracy/cost tradeoffs: | Level | Configuration | What it does | | ------------ | --------------------------------------------------- | ------------------------------------------------------- | | **Basic** | `summarize_figures: True` (default) | Text descriptions for RAG search | | **Enhanced** | `{"scope": "figure"}` | Better models, structured extraction for simpler charts | | **Advanced** | `{"scope": "figure", "advanced_chart_agent": True}` | Multi-stage pipeline for precise numerical extraction | ## Basic: Figure Summarization Enabled by default. Generates natural language descriptions using a lightweight model: ```python Python theme={null} result = client.parse.run( input=upload.file_id, enhance={"summarize_figures": True} # Default, no need to specify ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, enhance: { summarize_figures: true } // Default, no need to specify }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "enhance": {"summarize_figures": true} }' ``` **Output example:** `"Bar chart showing Q1-Q4 revenue growth, with Q4 reaching approximately $2.5M"` Good for making charts searchable in RAG applications. Fast and cheap, but doesn't extract actual numbers. ## Enhanced: Figure Scope The `figure` scope uses more powerful models and classifies figures before processing: ```python Python theme={null} result = client.parse.run( input=upload.file_id, enhance={ "agentic": [{"scope": "figure"}] } ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, enhance: { agentic: [{ scope: 'figure' }] } }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "enhance": { "agentic": [{"scope": "figure"}] } }' ``` The pipeline: 1. Classifies whether the image is a chart or general figure 2. If chart: runs structured extraction to pull data as text 3. If not a chart: generates a detailed description using a more powerful model Better than basic summarization, but not as precise as the advanced pipeline for complex charts. ## Advanced: Chart Agent Pipeline For precise numerical extraction, enable `advanced_chart_agent`: ```python Python theme={null} result = client.parse.run( input=upload.file_id, enhance={ "agentic": [{"scope": "figure", "advanced_chart_agent": True}] } ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, enhance: { agentic: [{ scope: 'figure', advanced_chart_agent: true }] } }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "enhance": { "agentic": [{"scope": "figure", "advanced_chart_agent": true}] } }' ``` ### How the Pipeline Works The chart agent runs multiple parallel tasks, then combines results: **Stage 1: Parallel extraction** * **Component detection**: Identifies each data series (lines, bars, areas, scatter points) and their colors/styles * **OCR**: Detects all text (axis labels, titles, legends, tick values) * **Legend detection**: Maps colors to series labels * **Coordinate extraction**: Finds axis boundaries and tick positions **Stage 2: Processing** * **Masking**: Isolates each component by color/style for individual processing * **Axis functions**: Builds mathematical functions to convert pixel coordinates to actual values (handles linear, logarithmic, and time series axes) * **Tick alignment**: Maps detected points to axis tick values **Stage 3: Value extraction** * Converts pixel coordinates to actual (x, y) values using the axis functions * Falls back to a VLM for components that couldn't be processed deterministically * Outputs a consolidated markdown table ### Output Format Data is returned as a markdown table with the X-axis as rows and each component as a column: ```markdown theme={null} | Date | Revenue ($M) | Expenses ($M) | | --- | --- | --- | | 2020-01 | 125.4 | 98.2 | | 2020-02 | 142.8 | 105.1 | | 2020-03 | 168.5 | 112.7 | ``` For bar charts, values show the range: `(bottom, top)`. ## Supported Chart Types | Chart Type | Support Level | Notes | | ----------------------- | ------------- | ------------------------------------------------- | | **Vertical bar charts** | ✅ Full | Detects bar heights and x-axis categories | | **Line charts** | ✅ Full | Tracks points along each series | | **Area charts** | ✅ Full | Extracts top/bottom boundaries | | **Scatter plots** | ✅ Partial | Works for sparse plots; very dense plots may fail | | **Combination charts** | ✅ Full | Handles mixed bar/line/area in same chart | | **Time series** | ✅ Full | Supports YYYY, YYYY-MM, YYYY-MM-DD formats | | **Logarithmic axes** | ✅ Full | Correctly interprets log-scale values | | **Dual Y-axis** | ✅ Full | Maps components to primary or secondary axis | ### Not Supported The advanced pipeline will skip these chart types (falls back to VLM description): * **Horizontal bar charts**: Axis orientation not supported * **Pie charts**: No coordinate-based extraction possible * **Radar/spider charts**: Non-Cartesian coordinate system * **Density plots**: Continuous distributions don't map to discrete points * **Flow charts/diagrams**: Not data visualizations * **Multiple charts in one image**: Requires a single chart per figure * **Charts with data labels**: If values are already printed on each point, extraction is skipped (the data is already visible) ## Custom Prompts Guide figure processing with custom instructions: ```python Python theme={null} result = client.parse.run( input=upload.file_id, enhance={ "agentic": [ {"scope": "figure", "prompt": "Focus on the primary trend line, ignore confidence intervals"} ] } ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, enhance: { agentic: [ { scope: 'figure', prompt: 'Focus on the primary trend line, ignore confidence intervals' } ] } }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "enhance": { "agentic": [ {"scope": "figure", "prompt": "Focus on the primary trend line, ignore confidence intervals"} ] } }' ``` ## Combining with Other Scopes For documents with charts and complex tables: ```python Python theme={null} result = client.parse.run( input=upload.file_id, enhance={ "agentic": [ {"scope": "table"}, {"scope": "figure", "advanced_chart_agent": True} ] } ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, enhance: { agentic: [ { scope: 'table' }, { scope: 'figure', advanced_chart_agent: true } ] } }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "enhance": { "agentic": [ {"scope": "table"}, {"scope": "figure", "advanced_chart_agent": true} ] } }' ``` ## Limitations * **Resolution matters**: Higher quality source images produce more accurate extractions * **Processing time**: The advanced pipeline is significantly slower than basic summarization. For async calls, use `priority=True` to speed up processing. * **Dense charts**: Scatter plots with many overlapping points may have reduced accuracy * **Same-color styles**: Charts where solid and dashed lines share the same color can confuse component detection # Chunking Methods Source: https://docs.reducto.ai/configs/parse/chunking-methods Control how parsed content is grouped in API responses When Reducto parses a document, it extracts individual **blocks**: paragraphs, headers, tables, figures, list items. Chunking controls how these blocks are **grouped together** when returned in the API response. For a complete overview of the response structure, see [Parse Response Format](/parse/response-format). This matters for RAG pipelines: most embedding models and vector databases work best with text segments of a specific size. Too small, and you lose context. Too large, and retrieval becomes imprecise. Chunking lets you control this tradeoff without post-processing the response yourself. ## Basic Usage ```python Python theme={null} result = client.parse.run( input=upload.file_id, retrieval={ "chunking": { "chunk_mode": "variable", "chunk_size": 1000 } } ) # Response contains grouped blocks for chunk in result.result.chunks: print(chunk.content) # Combined content of all blocks in this chunk print(chunk.blocks) # Individual blocks with metadata ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, retrieval: { chunking: { chunk_mode: 'variable', chunk_size: 1000 } } }); // Response contains grouped blocks for (const chunk of result.result.chunks) { console.log(chunk.content); // Combined content of all blocks in this chunk console.log(chunk.blocks); // Individual blocks with metadata } ``` ```go Go theme={null} result, _ := client.Parse.Run(context.Background(), reducto.ParseRunParams{ ParseConfig: reducto.ParseConfigParam{ DocumentURL: reducto.F[reducto.ParseConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), Options: reducto.F(shared.BaseProcessingOptionsParam{ Chunking: reducto.F(shared.BaseProcessingOptionsChunkingParam{ ChunkMode: reducto.F(shared.BaseProcessingOptionsChunkingChunkModeVariable), }), }), }, }) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "retrieval": { "chunking": { "chunk_mode": "variable", "chunk_size": 1000 } } }' ``` ## Chunking Modes **Best for:** RAG, semantic search Groups blocks to target a specific character count while keeping semantically related content together. This is the recommended mode for most RAG applications. ```python Python theme={null} retrieval={"chunking": {"chunk_mode": "variable", "chunk_size": 1000}} ``` ```javascript Node.js theme={null} retrieval: { chunking: { chunk_mode: 'variable', chunk_size: 1000 } } ``` ```go Go theme={null} Options: reducto.F(shared.BaseProcessingOptionsParam{ Chunking: reducto.F(shared.BaseProcessingOptionsChunkingParam{ ChunkMode: reducto.F(shared.BaseProcessingOptionsChunkingChunkModeVariable), }), }) ``` ```bash cURL theme={null} "retrieval": {"chunking": {"chunk_mode": "variable", "chunk_size": 1000}} ``` The algorithm: 1. Groups blocks by document structure (new group at each title/section header) 2. Splits oversized groups at natural boundaries (points where blocks are physically separated on the page) 3. Applies adjacency rules: keeps headers with content, figures with captions, list items together 4. Merges undersized groups to reach the target range **Size behavior:** When you specify `chunk_size: 1000`, chunks will range from 750 to 1250 characters (±25%). Without a size specified, the default range is 750-1250. **Best for:** Hierarchical documents, manuals, legal docs Each chunk starts at a title or section header and contains everything until the next header. No size limits. ```python Python theme={null} retrieval={"chunking": {"chunk_mode": "section"}} ``` ```javascript Node.js theme={null} retrieval: { chunking: { chunk_mode: 'section' } } ``` ```go Go theme={null} Options: reducto.F(shared.BaseProcessingOptionsParam{ Chunking: reducto.F(shared.BaseProcessingOptionsChunkingParam{ ChunkMode: reducto.F(shared.BaseProcessingOptionsChunkingChunkModeSection), }), }) ``` ```bash cURL theme={null} "retrieval": {"chunking": {"chunk_mode": "section"}} ``` Use when document structure is meaningful and you want to preserve it. Chunks can be large if sections are long. **Best for:** Presentations, page-specific analysis, spreadsheets One chunk per page. For spreadsheets, one chunk per sheet. ```python Python theme={null} retrieval={"chunking": {"chunk_mode": "page"}} ``` ```javascript Node.js theme={null} retrieval: { chunking: { chunk_mode: 'page' } } ``` ```bash cURL theme={null} "retrieval": {"chunking": {"chunk_mode": "page"}} ``` Use when page boundaries matter or when each page/sheet is a self-contained unit. **Best for:** Documents where both page and section context matter Splits by page first, then by sections within each page. ```python Python theme={null} retrieval={"chunking": {"chunk_mode": "page_sections"}} ``` ```javascript Node.js theme={null} retrieval: { chunking: { chunk_mode: 'page_sections' } } ``` ```bash cURL theme={null} "retrieval": {"chunking": {"chunk_mode": "page_sections"}} ``` Useful when you need to know which page content came from while also preserving section structure. **Best for:** Maximum granularity, custom chunking logic Each block becomes its own chunk. Gives you the finest granularity to implement your own chunking logic downstream. ```python Python theme={null} retrieval={"chunking": {"chunk_mode": "block"}} ``` ```javascript Node.js theme={null} retrieval: { chunking: { chunk_mode: 'block' } } ``` ```go Go theme={null} Options: reducto.F(shared.BaseProcessingOptionsParam{ Chunking: reducto.F(shared.BaseProcessingOptionsChunkingParam{ ChunkMode: reducto.F(shared.BaseProcessingOptionsChunkingChunkModeBlock), }), }) ``` ```bash cURL theme={null} "retrieval": {"chunking": {"chunk_mode": "block"}} ``` **Best for:** Small documents, no chunking needed Returns all blocks as a single chunk. ```python Python theme={null} retrieval={"chunking": {"chunk_mode": "disabled"}} ``` ```javascript Node.js theme={null} retrieval: { chunking: { chunk_mode: 'disabled' } } ``` ```go Go theme={null} Options: reducto.F(shared.BaseProcessingOptionsParam{ Chunking: reducto.F(shared.BaseProcessingOptionsChunkingParam{ ChunkMode: reducto.F(shared.BaseProcessingOptionsChunkingChunkModeDisabled), }), }) ``` ```bash cURL theme={null} "retrieval": {"chunking": {"chunk_mode": "disabled"}} ``` ## Optimizing for Embeddings Tables often embed poorly because their structure doesn't translate well to vector representations. Enable `embedding_optimized` to generate natural language summaries of tables: ```python Python theme={null} retrieval={ "chunking": {"chunk_mode": "variable"}, "embedding_optimized": True } ``` ```javascript Node.js theme={null} retrieval: { chunking: { chunk_mode: 'variable' }, embedding_optimized: true } ``` ```bash cURL theme={null} "retrieval": { "chunking": {"chunk_mode": "variable"}, "embedding_optimized": true } ``` With this enabled, each chunk has two content fields: * `content`: Original format (tables as HTML/markdown) * `embed`: Optimized for embeddings (tables converted to summaries like "This table shows quarterly revenue by region...") Use `embed` for vector search, `content` for display. ## Filtering Block Types Some content types (headers, footers, page numbers) add noise to search results. Filter them out: ```python Python theme={null} retrieval={"filter_blocks": ["Header", "Footer", "Page Number"]} ``` ```javascript Node.js theme={null} retrieval: { filter_blocks: ['Header', 'Footer', 'Page Number'] } ``` ```go Go theme={null} Options: reducto.F(shared.BaseProcessingOptionsParam{ FilterBlocks: reducto.F([]shared.BaseProcessingOptionsFilterBlock{ shared.BaseProcessingOptionsFilterBlockHeader, shared.BaseProcessingOptionsFilterBlockFooter, shared.BaseProcessingOptionsFilterBlockPageNumber, }), }) ``` ```bash cURL theme={null} "retrieval": {"filter_blocks": ["Header", "Footer", "Page Number"]} ``` Filtered blocks still appear in `chunks[].blocks` with full metadata, but they're excluded from the `content` and `embed` text fields. **Available types:** `Header`, `Footer`, `Title`, `Section Header`, `Page Number`, `List Item`, `Figure`, `Table`, `Key Value`, `Text`, `Comment`, `Signature` ## Response Structure Each chunk in the response contains: ```json theme={null} { "content": "Combined markdown content of all blocks in this chunk", "embed": "Embedding-optimized version (tables summarized if embedding_optimized=True)", "blocks": [ { "type": "Text", "content": "Individual block content", "bbox": {"left": 0.1, "top": 0.2, "width": 0.8, "height": 0.05, "page": 1}, "confidence": "high" } ] } ``` The `blocks` array gives you access to individual elements with their bounding boxes and types, useful for citations or highlighting source locations. See [Parse Response Format](/parse/response-format) for complete field documentation. ## Troubleshooting Reduce `chunk_size` or switch to `block` mode and implement your own chunking logic on the individual blocks. Increase `chunk_size` or use `section` mode if your document has well-defined sections. Increase `chunk_size` to accommodate full tables. Alternatively, enable `formatting.merge_tables` to combine consecutive tables with the same column structure before chunking. Enable `embedding_optimized: True` to generate natural language summaries of tables for the `embed` field. # Processing Settings Source: https://docs.reducto.ai/configs/parse/ocr-settings Control OCR, timeouts, output options, and document handling The `settings` config group controls how documents are processed: which OCR system to use, how long to wait, what to include in the response, and how to handle special cases like password-protected files. ```python Python theme={null} result = client.parse.run( input=upload.file_id, settings={ "ocr_system": "standard", "timeout": 300, "page_range": {"start": 1, "end": 50} } ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, settings: { ocr_system: 'standard', timeout: 300, page_range: { start: 1, end: 50 } } }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "settings": { "ocr_system": "standard", "timeout": 300, "page_range": {"start": 1, "end": 50} } }' ``` ## OCR System Reducto offers two OCR systems that determine how text is extracted from images and scanned documents. ```python Python theme={null} settings={"ocr_system": "standard"} ``` ```javascript Node.js theme={null} settings: { ocr_system: 'standard' } ``` ```bash cURL theme={null} "settings": {"ocr_system": "standard"} ``` **`standard` (default):** Our primary OCR engine supporting 60+ languages. Handles mixed-language documents automatically. Afrikaans, Albanian, Arabic, Armenian, Belarusian, Bengali, Bulgarian, Catalan, Chinese, Croatian, Czech, Danish, Dutch, English, Estonian, Filipino, Finnish, French, German, Greek, Gujarati, Hebrew, Hindi, Hungarian, Icelandic, Indonesian, Italian, Japanese, Kannada, Khmer, Korean, Lao, Latvian, Lithuanian, Macedonian, Malay, Malayalam, Marathi, Nepali, Norwegian, Persian, Polish, Portuguese, Punjabi, Romanian, Russian, Serbian, Slovak, Slovenian, Spanish, Swedish, Tagalog, Tamil, Telugu, Thai, Turkish, Ukrainian, Vietnamese, Yiddish **`legacy`:** An older engine optimized for Germanic languages only. Available for backwards compatibility with existing integrations. Use `standard` for new projects. English, German, Dutch, Norwegian, Swedish, Danish, Icelandic, Afrikaans The Go SDK uses different OCR system values (`highres`, `multilingual`, `combined`). Use Python, Node.js, or cURL for the `standard` and `legacy` options. For maximum accuracy on difficult documents (handwriting, faded text, poor scans), combine with agentic text mode: ```python Python theme={null} result = client.parse.run( input=upload.file_id, settings={"ocr_system": "standard"}, enhance={"agentic": [{"scope": "text"}]} ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, settings: { ocr_system: 'standard' }, enhance: { agentic: [{ scope: 'text' }] } }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "settings": {"ocr_system": "standard"}, "enhance": {"agentic": [{"scope": "text"}]} }' ``` See [Agentic Modes](/configs/parse/agentic-modes) for details on when to enable this. ## Extraction Mode Controls how text is extracted from PDFs that have embedded text layers. ```python Python theme={null} settings={"extraction_mode": "hybrid"} ``` ```javascript Node.js theme={null} settings: { extraction_mode: 'hybrid' } ``` ```bash cURL theme={null} "settings": {"extraction_mode": "hybrid"} ``` **`hybrid` (default):** Uses good quality metadata first, then OCR. Best when processing mixed document sets where some have reliable embedded text and others don't. **`ocr`:** Uses optical character recognition only, ignoring any embedded text in the PDF. Best for scanned documents, images, or when embedded text is unreliable or corrupted. **`metadata`:** Uses embedded text from PDF metadata only, without OCR. Best for native DOCX/PDFs with reliable text layers where you want faster processing. ## Page Range Process only specific pages to save time and credits. See [Page Ranges](/configs/parse/page-ranges) for complete documentation. ```python Python theme={null} # Pages 1-10 only settings={"page_range": {"start": 1, "end": 10}} # Multiple ranges settings={"page_range": [{"start": 1, "end": 5}, {"start": 20, "end": 25}]} ``` ```javascript Node.js theme={null} // Pages 1-10 only settings: { page_range: { start: 1, end: 10 } } // Multiple ranges settings: { page_range: [{ start: 1, end: 5 }, { start: 20, end: 25 }] } ``` ```go Go theme={null} // Pages 1-10 only AdvancedOptions: reducto.F(shared.AdvancedProcessingOptionsParam{ PageRange: reducto.F[shared.AdvancedProcessingOptionsPageRangeUnionParam]( shared.PageRangeParam{ Start: reducto.F(int64(1)), End: reducto.F(int64(10)), }, ), }) ``` ```bash cURL theme={null} # Pages 1-10 only "settings": {"page_range": {"start": 1, "end": 10}} # Multiple ranges "settings": {"page_range": [{"start": 1, "end": 5}, {"start": 20, "end": 25}]} ``` ## Timeout Set a maximum processing time in seconds. If processing exceeds this limit, the request fails rather than hanging indefinitely. ```python Python theme={null} settings={"timeout": 300} # 5 minutes ``` ```javascript Node.js theme={null} settings: { timeout: 300 } // 5 minutes ``` ```bash cURL theme={null} "settings": {"timeout": 300} ``` If not specified, Reducto uses internal defaults appropriate for the document size. ## Embedded Document Properties Set `extract_document_properties` to `true` to return properties embedded in the original pre-conversion file, including PDF Info/XMP and OOXML core properties. This setting defaults to `false`. ```python Python theme={null} result = client.parse.run( input=upload.file_id, settings={"extract_document_properties": True} ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, settings: { extract_document_properties: true } }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "settings": {"extract_document_properties": true} }' ``` The response includes a top-level `document_properties` object for supported files with embedded properties. It is `null` when no properties are found or the input format is unsupported. Supported formats are PDF, DOCX, XLSX, and PPTX. Legacy binary `.doc`, `.xls`, and `.ppt` files are not supported. Each field is nullable, and dates use timezone-aware ISO-8601 strings. PDF Info/XMP populates `title`, `author`, `subject`, `keywords`, `creator`, `producer`, `created_at`, and `modified_at`. `last_modified_by` is only available from OOXML core properties. DOCX, XLSX, and PPTX can populate `title`, `author`, `subject`, `keywords`, `creator`, `last_modified_by`, `created_at`, and `modified_at`; `producer` is PDF-only. ```json theme={null} { "document_properties": { "title": "Quarterly Report", "author": "Jane Smith", "subject": "Financial results", "keywords": "finance, quarterly", "creator": "Microsoft Word", "producer": null, "last_modified_by": "Alex Chen", "created_at": "2024-01-15T09:30:00+00:00", "modified_at": "2024-02-01T16:45:12+00:00" } } ``` ## Password-Protected Documents For encrypted PDFs that require a password to open: ```python Python theme={null} settings={"document_password": "secret123"} ``` ```javascript Node.js theme={null} settings: { document_password: 'secret123' } ``` ```go Go theme={null} AdvancedOptions: reducto.F(shared.AdvancedProcessingOptionsParam{ DocumentPassword: reducto.F("secret123"), }) ``` ```bash cURL theme={null} "settings": {"document_password": "secret123"} ``` The password is used to decrypt the document before processing. It's transmitted securely but not stored. ## Return Images By default, blocks contain only extracted text. Enable `return_images` to get pre-signed URLs pointing to cropped images of specific block types: ```python Python theme={null} settings={"return_images": ["figure", "table"]} ``` ```javascript Node.js theme={null} settings: { return_images: ['figure', 'table'] } ``` ```bash cURL theme={null} "settings": {"return_images": ["figure", "table"]} ``` When enabled, applicable blocks include an `image_url` field: ```json theme={null} { "type": "Figure", "bbox": {"left": 0.1, "top": 0.2, "width": 0.8, "height": 0.4, "page": 1}, "content": "Bar chart showing quarterly revenue growth from Q1 to Q4...", "image_url": "https://storage.reducto.ai/figures/abc123.png?X-Amz-Expires=3600..." } ``` The URL is a pre-signed S3 link valid for a limited time. Download or process the image before expiration. **Options:** * `figure`: Cropped images for figure blocks (charts, diagrams, photos, illustrations) * `table`: Cropped images for table blocks ## Return OCR Data Returns the raw OCR output with word-level and line-level bounding boxes. This gives you access to the underlying text extraction before Reducto's layout analysis. ```python Python theme={null} settings={"return_ocr_data": True} ``` ```javascript Node.js theme={null} settings: { return_ocr_data: true } ``` ```go Go theme={null} AdvancedOptions: reducto.F(shared.AdvancedProcessingOptionsParam{ ReturnOcrData: reducto.F(true), }) ``` ```bash cURL theme={null} "settings": {"return_ocr_data": true} ``` The response `result` object includes an `ocr` field containing `words` and `lines` arrays: ```json theme={null} { "job_id": "parse_abc123xyz", "result": { "type": "full", "chunks": [...], "ocr": { "words": [ { "text": "Revenue", "bbox": {"left": 0.12, "top": 0.08, "width": 0.15, "height": 0.02, "page": 1}, "confidence": 0.98, "rotation": 0 } ], "lines": [ { "text": "Revenue Report Q4 2024", "bbox": {"left": 0.12, "top": 0.08, "width": 0.45, "height": 0.02, "page": 1}, "confidence": 0.97, "rotation": 0 } ] } } } ``` Each word and line includes: * `text`: The recognized text * `bbox`: Normalized bounding box (coordinates as fractions of page dimensions) * `confidence`: OCR confidence score between 0 and 1 * `rotation`: Detected rotation angle in degrees (0-360, counterclockwise) ## Persist Results By default, processed results are stored temporarily and eventually deleted. Enable persistence to keep results indefinitely in long-term storage: ```python Python theme={null} settings={"persist_results": True} ``` ```javascript Node.js theme={null} settings: { persist_results: true } ``` ```bash cURL theme={null} "settings": {"persist_results": true} ``` When enabled, you can retrieve results later using the job ID without reprocessing the document. The response includes a `job_id` that serves as the retrieval key: ```json theme={null} { "job_id": "parse_abc123xyz", "duration": 2.34, "result": {...} } ``` Retrieve stored results later: ```python Python theme={null} job = client.job.get("parse_abc123xyz") result = job.result ``` ```javascript Node.js theme={null} const job = await client.job.retrieve('parse_abc123xyz'); const result = job.result; ``` ```bash cURL theme={null} curl https://platform.reducto.ai/job/parse_abc123xyz \ -H "Authorization: Bearer $REDUCTO_API_KEY" ``` Requires opting in to Reducto Studio. Contact support to enable this feature for your organization. ## Embed PDF Metadata Embeds the OCR-extracted text back into the PDF as a hidden text layer. The response includes a `pdf_url` pointing to the enhanced PDF: ```python Python theme={null} settings={"embed_pdf_metadata": True} ``` ```javascript Node.js theme={null} settings: { embed_pdf_metadata: true } ``` ```bash cURL theme={null} "settings": {"embed_pdf_metadata": true} ``` ```json theme={null} { "job_id": "parse_abc123xyz", "pdf_url": "https://storage.reducto.ai/pdfs/abc123.pdf?...", "result": {...} } ``` ### Render DPI By default we render the source PDF at 100 DPI when building the embedded-OCR PDF — small files, sharp at default zoom in a doc viewer, slight softness only past \~200% zoom. To control this directly, pass `embed_pdf_metadata_dpi` (range 50-250): ```python Python theme={null} settings={"embed_pdf_metadata": True, "embed_pdf_metadata_dpi": 150} ``` ```javascript Node.js theme={null} settings: { embed_pdf_metadata: true, embed_pdf_metadata_dpi: 150 } ``` ```bash cURL theme={null} "settings": {"embed_pdf_metadata": true, "embed_pdf_metadata_dpi": 150} ``` Higher values preserve more detail when zoomed in but produce larger PDFs; lower values shrink the output but soften the rendered text at high zoom. Pick close to the source scan DPI for scan-derived PDFs; the default 100 is right for most digital documents. The returned PDF looks identical to the original but now supports: * Text selection and copy/paste in PDF viewers * Full-text search within the document * Accessibility features (screen readers can read the text) ## Force URL Result By default, Reducto returns the full result inline in the response. For very large documents, this is automatically switched to a URL. You can force URL mode regardless of size: ```python Python theme={null} settings={"force_url_result": True} ``` ```javascript Node.js theme={null} settings: { force_url_result: true } ``` ```go Go theme={null} Options: reducto.F(shared.BaseProcessingOptionsParam{ ForceURLResult: reducto.F(true), }) ``` ```bash cURL theme={null} "settings": {"force_url_result": true} ``` When enabled, the response contains a URL instead of inline content: ```json theme={null} { "job_id": "parse_abc123xyz", "result": { "type": "url", "url": "https://storage.reducto.ai/results/abc123.json?...", "result_id": "abc123" } } ``` Fetch the full result by downloading from the URL. The URL is pre-signed and valid for a limited time. ## Force File Extension Reducto automatically detects file types from URLs and content. Override this detection when automatic detection fails or returns incorrect results: ```python Python theme={null} settings={"force_file_extension": ".pdf"} ``` ```javascript Node.js theme={null} settings: { force_file_extension: '.pdf' } ``` ```go Go theme={null} AdvancedOptions: reducto.F(shared.AdvancedProcessingOptionsParam{ ForceFileExtension: reducto.F(".pdf"), }) ``` ```bash cURL theme={null} "settings": {"force_file_extension": ".pdf"} ``` Common scenarios: * URLs without file extensions (e.g., `https://api.example.com/document/12345`) * URLs with misleading extensions * Pre-signed URLs with complex query parameters that confuse detection Valid extensions include `.pdf`, `.png`, `.jpg`, `.docx`, `.xlsx`, `.pptx`, and all other [supported file types](/upload/overview#supported-file-types). # Page Ranges Source: https://docs.reducto.ai/configs/parse/page-ranges Process specific pages of a document Reducto allows you to specify which pages of a document to process using the `page_range` parameter in settings. You can specify a single range or multiple ranges. ## Single Range For processing a continuous range of pages, specify `start` and `end` page numbers: ```python Python theme={null} result = client.parse.run( input=upload.file_id, settings={ "page_range": {"start": 1, "end": 10} } ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, settings: { page_range: { start: 1, end: 10 } } }); ``` ```go Go theme={null} result, _ := client.Parse.Run(context.Background(), reducto.ParseRunParams{ ParseConfig: reducto.ParseConfigParam{ DocumentURL: reducto.F[reducto.ParseConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), AdvancedOptions: reducto.F(shared.AdvancedProcessingOptionsParam{ PageRange: reducto.F[shared.AdvancedProcessingOptionsPageRangeUnionParam]( shared.PageRangeParam{ Start: reducto.F(int64(1)), End: reducto.F(int64(10)), }, ), }), }, }) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "settings": { "page_range": {"start": 1, "end": 10} } }' ``` This processes pages 1 through 10. ## Multiple Ranges For non-contiguous pages, provide an array of range objects: ```python Python theme={null} result = client.parse.run( input=upload.file_id, settings={ "page_range": [ {"start": 1, "end": 5}, {"start": 10, "end": 15} ] } ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, settings: { page_range: [ { start: 1, end: 5 }, { start: 10, end: 15 } ] } }); ``` ```go Go theme={null} result, _ := client.Parse.Run(context.Background(), reducto.ParseRunParams{ ParseConfig: reducto.ParseConfigParam{ DocumentURL: reducto.F[reducto.ParseConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), AdvancedOptions: reducto.F(shared.AdvancedProcessingOptionsParam{ PageRange: reducto.F[shared.AdvancedProcessingOptionsPageRangeUnionParam]( shared.AdvancedProcessingOptionsPageRangeArrayParam{ {Start: reducto.F(int64(1)), End: reducto.F(int64(5))}, {Start: reducto.F(int64(10)), End: reducto.F(int64(15))}, }, ), }), }, }) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "settings": { "page_range": [ {"start": 1, "end": 5}, {"start": 10, "end": 15} ] } }' ``` This processes pages 1-5 and 10-15. ## Notes * Page numbers are 1-indexed (first page is page 1) * Both `start` and `end` are inclusive * If no page range is specified, the entire document is processed * The `end` page must be greater than or equal to `start` * Ranges do not need to be in order * Overlapping ranges are processed only once * If `end` exceeds the document length, processing stops at the last page ## With Split Endpoint For the Split endpoint, `page_range` is nested under `parsing.settings`: ```python Python theme={null} result = client.split.run( input=upload.file_id, split_description=[...], parsing={ "settings": { "page_range": {"start": 40, "end": 80} } } ) ``` ```javascript Node.js theme={null} const result = await client.split.run({ input: upload.file_id, split_description: [...], parsing: { settings: { page_range: { start: 40, end: 80 } } } }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/split \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "document_url": "reducto://your-file-id", "split_description": [...], "parsing": { "settings": { "page_range": {"start": 40, "end": 80} } } }' ``` # Spreadsheet Processing Source: https://docs.reducto.ai/configs/parse/spreadsheet Configure how Excel, CSV, and spreadsheet files are processed Spreadsheets present a unique challenge: a single sheet can contain multiple logical tables, empty regions, header rows, and metadata scattered across cells. The `spreadsheet` config group controls how Reducto identifies table boundaries, handles large tables, and extracts cell-level metadata like colors and formulas. ```python Python theme={null} result = client.parse.run( input=upload.file_id, spreadsheet={ "clustering": "accurate", "split_large_tables": {"enabled": True, "size": 50}, "include": ["cell_colors", "formula"], "exclude": ["hidden_sheets"] } ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, spreadsheet: { clustering: 'accurate', split_large_tables: { enabled: true, size: 50 }, include: ['cell_colors', 'formula'], exclude: ['hidden_sheets'] } }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "spreadsheet": { "clustering": "accurate", "split_large_tables": {"enabled": true, "size": 50}, "include": ["cell_colors", "formula"], "exclude": ["hidden_sheets"] } }' ``` ## Table Clustering Many spreadsheets contain multiple tables on the same sheet, separated by empty rows or columns. Clustering detects where one table ends and another begins, so each table becomes its own block in the output. ```python Python theme={null} spreadsheet={"clustering": "accurate"} ``` ```javascript Node.js theme={null} spreadsheet: { clustering: 'accurate' } ``` ```bash cURL theme={null} "spreadsheet": {"clustering": "accurate"} ``` **`accurate` (default):** Uses an LLM to analyze the sheet structure and identify table boundaries. This handles complex layouts where tables have different column structures, headers in unusual positions, or subtle separations. Costs 5x per cell compared to `fast`. **`fast`:** Uses a rule-based algorithm to find tables based on empty rows/columns. Works well for simple spreadsheets where tables are clearly separated. Standard per-cell cost. **`disabled`:** Treats the entire sheet as one table. Use this when you know each sheet contains exactly one table, or when you want raw cell data without any boundary detection. The Go SDK currently only supports `default` and `disabled` clustering modes. Use Python, Node.js, or cURL for `accurate` and `fast` modes. ```python Python theme={null} # Simple spreadsheet with obvious table boundaries spreadsheet={"clustering": "fast"} # Each sheet is a single table spreadsheet={"clustering": "disabled"} ``` ```javascript Node.js theme={null} // Simple spreadsheet with obvious table boundaries spreadsheet: { clustering: 'fast' } // Each sheet is a single table spreadsheet: { clustering: 'disabled' } ``` ```bash cURL theme={null} # Simple spreadsheet with obvious table boundaries "spreadsheet": {"clustering": "fast"} # Each sheet is a single table "spreadsheet": {"clustering": "disabled"} ``` ## Large Table Splitting Tables with many rows create problems downstream: they can exceed LLM context windows, make chunking difficult, and slow down processing. By default, Reducto splits tables that exceed 50 rows into smaller chunks, each retaining the header row for context. ```python Python theme={null} spreadsheet={ "split_large_tables": { "enabled": True, "size": 50 # Max rows per chunk } } ``` ```javascript Node.js theme={null} spreadsheet: { split_large_tables: { enabled: true, size: 50 // Max rows per chunk } } ``` ```go Go theme={null} AdvancedOptions: reducto.F(shared.AdvancedProcessingOptionsParam{ LargeTableChunking: reducto.F(shared.AdvancedProcessingOptionsLargeTableChunkingParam{ Enabled: reducto.F(true), Size: reducto.F(int64(50)), }), }) ``` ```bash cURL theme={null} "spreadsheet": { "split_large_tables": { "enabled": true, "size": 50 } } ``` Each chunk becomes a separate table block. If your original table has headers in row 1 and 200 data rows, you get 4 blocks: rows 1-50, 1+51-100, 1+101-150, 1+151-200 (header repeated in each). **When to disable:** If your downstream processing needs all rows together (for example, sorting or aggregation), disable splitting: ```python Python theme={null} spreadsheet={"split_large_tables": {"enabled": False}} ``` ```javascript Node.js theme={null} spreadsheet: { split_large_tables: { enabled: false } } ``` ```go Go theme={null} AdvancedOptions: reducto.F(shared.AdvancedProcessingOptionsParam{ LargeTableChunking: reducto.F(shared.AdvancedProcessingOptionsLargeTableChunkingParam{ Enabled: reducto.F(false), }), }) ``` ```bash cURL theme={null} "spreadsheet": {"split_large_tables": {"enabled": false}} ``` **Adjusting chunk size:** For tables where rows are highly interdependent, increase the size. For very wide tables that consume lots of tokens, decrease it: ```python Python theme={null} # More context per chunk spreadsheet={"split_large_tables": {"size": 100}} # Smaller chunks for wide tables spreadsheet={"split_large_tables": {"size": 25}} ``` ```javascript Node.js theme={null} // More context per chunk spreadsheet: { split_large_tables: { size: 100 } } // Smaller chunks for wide tables spreadsheet: { split_large_tables: { size: 25 } } ``` ```go Go theme={null} // More context per chunk AdvancedOptions: reducto.F(shared.AdvancedProcessingOptionsParam{ LargeTableChunking: reducto.F(shared.AdvancedProcessingOptionsLargeTableChunkingParam{ Size: reducto.F(int64(100)), }), }) // Smaller chunks for wide tables AdvancedOptions: reducto.F(shared.AdvancedProcessingOptionsParam{ LargeTableChunking: reducto.F(shared.AdvancedProcessingOptionsLargeTableChunkingParam{ Size: reducto.F(int64(25)), }), }) ``` ```bash cURL theme={null} # More context per chunk "spreadsheet": {"split_large_tables": {"size": 100}} # Smaller chunks for wide tables "spreadsheet": {"split_large_tables": {"size": 25}} ``` ## Including Cell Metadata By default, Reducto extracts cell values only. You can optionally include colors and formulas. ### Cell Colors Financial spreadsheets often use color to convey meaning: red for negative values, green for positive, yellow for warnings. Enable `cell_colors` to preserve this information: ```python Python theme={null} spreadsheet={"include": ["cell_colors"]} ``` ```javascript Node.js theme={null} spreadsheet: { include: ['cell_colors'] } ``` ```bash cURL theme={null} "spreadsheet": {"include": ["cell_colors"]} ``` Colors appear as inline styles in HTML table output: ```html theme={null} -$5,000 ``` The `color` property is text color; `background-color` is cell highlight/fill. ### Formulas Spreadsheets contain computational logic in formulas. Enable `formula` to capture the original formula alongside the computed value: ```python Python theme={null} spreadsheet={"include": ["formula"]} ``` ```javascript Node.js theme={null} spreadsheet: { include: ['formula'] } ``` ```bash cURL theme={null} "spreadsheet": {"include": ["formula"]} ``` Formulas appear as `data-formula` attributes in HTML output: ```html theme={null} $125,000 ``` This is useful when you need to understand how values were calculated, audit spreadsheet logic, or recreate the computation elsewhere. ## Excluding Content Spreadsheets often contain content you don't want to process: hidden sheets with intermediate calculations, hidden rows/columns, embedded images, or styling information. ```python Python theme={null} spreadsheet={"exclude": ["hidden_sheets", "hidden_rows", "hidden_cols"]} ``` ```javascript Node.js theme={null} spreadsheet: { exclude: ['hidden_sheets', 'hidden_rows', 'hidden_cols'] } ``` ```bash cURL theme={null} "spreadsheet": {"exclude": ["hidden_sheets", "hidden_rows", "hidden_cols"]} ``` **`hidden_sheets`:** Skips sheets marked as hidden in Excel. Many spreadsheets hide calculation sheets or raw data that isn't meant for display. **`hidden_rows` and `hidden_cols`:** Skips rows and columns that are hidden. Useful when spreadsheets hide detail rows in grouped/outlined sections. **`styling`:** Excludes all styling information (fonts, borders, colors). Use when you only need values. **`spreadsheet_images`:** Skips embedded images and charts. These are processed separately by default, but you can skip them if not needed. By default, hidden content IS processed. Explicitly exclude it if your spreadsheets contain sensitive data in hidden areas or if hidden content is irrelevant to your use case. ## Cell Count Limit Giant spreadsheets with millions of cells can generate unexpectedly large bills. The `max_cell_count` option lets you set a cap on the total non-empty cells across all sheets. If the count exceeds the limit, the request fails with a 422 error before any expensive processing begins. ```python Python theme={null} spreadsheet={"max_cell_count": 500000} ``` ```javascript Node.js theme={null} spreadsheet: { max_cell_count: 500000 } ``` ```bash cURL theme={null} "spreadsheet": {"max_cell_count": 500000} ``` Defaults to `null` (no limit). Only non-empty cells count. Empty cells and cells with no value inside an oversized used range do not contribute. When a spreadsheet exceeds the limit, the error response includes the actual cell count so you can adjust: ```json theme={null} { "error": "Spreadsheet cell count (1,234,567) exceeds max_cell_count limit (500,000). Set a higher max_cell_count in spreadsheet options or set to null to disable the limit." } ``` The response `usage` object includes `non_empty_cell_count` for all spreadsheet parse jobs, so you can monitor cell counts and set appropriate limits: ```json theme={null} { "usage": { "num_pages": 12, "credits": 45.0, "non_empty_cell_count": 45230 } } ``` ## Example: Financial Model Financial models typically have multiple tables per sheet, use colors for emphasis, and have hidden calculation sheets: ```python Python theme={null} result = client.parse.run( input=financial_model_url, spreadsheet={ "clustering": "accurate", # Multiple tables with complex layouts "include": ["cell_colors"], # Color indicates meaning "exclude": ["hidden_sheets"] # Skip calculation sheets } ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: financialModelUrl, spreadsheet: { clustering: 'accurate', // Multiple tables with complex layouts include: ['cell_colors'], // Color indicates meaning exclude: ['hidden_sheets'] // Skip calculation sheets } }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "https://example.com/financial_model.xlsx", "spreadsheet": { "clustering": "accurate", "include": ["cell_colors"], "exclude": ["hidden_sheets"] } }' ``` ## Example: Data Export Large data exports are typically single tables with thousands of rows: ```python Python theme={null} result = client.parse.run( input=data_export_url, spreadsheet={ "clustering": "disabled", # Single table per sheet "split_large_tables": {"enabled": True, "size": 100} # Larger chunks for context } ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: dataExportUrl, spreadsheet: { clustering: 'disabled', // Single table per sheet split_large_tables: { enabled: true, size: 100 } // Larger chunks for context } }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "https://example.com/data_export.csv", "spreadsheet": { "clustering": "disabled", "split_large_tables": {"enabled": true, "size": 100} } }' ``` # Table Output Formats Source: https://docs.reducto.ai/configs/parse/table-output-formats Control how tables are formatted in API responses Reducto extracts tables from documents and can return them in several formats. The format you choose affects how merged cells, headers, and structure are represented. ## Setting Table Format ```python Python theme={null} result = client.parse.run( input=upload.file_id, formatting={"table_output_format": "html"} ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, formatting: { table_output_format: 'html' } }); ``` ```go Go theme={null} result, _ := client.Parse.Run(context.Background(), reducto.ParseRunParams{ ParseConfig: reducto.ParseConfigParam{ DocumentURL: reducto.F[reducto.ParseConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), AdvancedOptions: reducto.F(shared.AdvancedProcessingOptionsParam{ TableOutputFormat: reducto.F(shared.AdvancedProcessingOptionsTableOutputFormatHTML), }), }, }) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "formatting": {"table_output_format": "html"} }' ``` ## Available Formats **Default.** Automatically chooses between markdown and HTML based on table complexity. * Uses **markdown** for simple tables (30 cells or fewer AND 4 merged cells or fewer) * Uses **HTML** for complex tables (more than 30 cells OR more than 4 merged cells) ```python Python theme={null} formatting={"table_output_format": "dynamic"} ``` ```javascript Node.js theme={null} formatting: { table_output_format: 'dynamic' } ``` ```go Go theme={null} AdvancedOptions: reducto.F(shared.AdvancedProcessingOptionsParam{ TableOutputFormat: reducto.F(shared.AdvancedProcessingOptionsTableOutputFormatDynamic), }) ``` ```bash cURL theme={null} "formatting": {"table_output_format": "dynamic"} ``` Best for RAG pipelines where you want clean, readable output for simple tables while preserving structure for complex ones. Full HTML table structure with proper support for merged cells. ```python Python theme={null} formatting={"table_output_format": "html"} ``` ```javascript Node.js theme={null} formatting: { table_output_format: 'html' } ``` ```bash cURL theme={null} "formatting": {"table_output_format": "html"} ``` ```html theme={null}
Q1 Results
ProductRevenue
Widget A$10,000
``` Merged cells are encoded using `rowspan` and `colspan` attributes. Use for financial statements, regulatory filings, or any tables where cell merging carries meaning.
GitHub-flavored markdown tables. ```python Python theme={null} formatting={"table_output_format": "md"} ``` ```javascript Node.js theme={null} formatting: { table_output_format: 'md' } ``` ```go Go theme={null} AdvancedOptions: reducto.F(shared.AdvancedProcessingOptionsParam{ TableOutputFormat: reducto.F(shared.AdvancedProcessingOptionsTableOutputFormatMd), }) ``` ```bash cURL theme={null} "formatting": {"table_output_format": "md"} ``` ```markdown theme={null} | Header 1 | Header 2 | | - | - | | Data 1 | Data 2 | ``` Markdown cannot represent merged cells. If your table has merged cells, they will be flattened. Use for simple tables where human readability matters. Nested arrays for programmatic access. ```python Python theme={null} formatting={"table_output_format": "json"} ``` ```javascript Node.js theme={null} formatting: { table_output_format: 'json' } ``` ```go Go theme={null} AdvancedOptions: reducto.F(shared.AdvancedProcessingOptionsParam{ TableOutputFormat: reducto.F(shared.AdvancedProcessingOptionsTableOutputFormatJson), }) ``` ```bash cURL theme={null} "formatting": {"table_output_format": "json"} ``` ```json theme={null} [ ["Header 1", "Header 2"], ["Data 1", "Data 2"] ] ``` First row contains headers. All cell values are strings. Use when you need to process table data programmatically. JSON with normalized bounding box coordinates for each cell. ```python Python theme={null} formatting={"table_output_format": "jsonbbox"} ``` ```javascript Node.js theme={null} formatting: { table_output_format: 'jsonbbox' } ``` ```go Go theme={null} AdvancedOptions: reducto.F(shared.AdvancedProcessingOptionsParam{ TableOutputFormat: reducto.F(shared.AdvancedProcessingOptionsTableOutputFormatJsonbbox), }) ``` ```bash cURL theme={null} "formatting": {"table_output_format": "jsonbbox"} ``` ```json theme={null} [ [ {"text": "Header 1", "bbox": {"x": 0.1, "y": 0.2, "width": 0.3, "height": 0.04}}, {"text": "Header 2", "bbox": {"x": 0.4, "y": 0.2, "width": 0.3, "height": 0.04}} ] ] ``` Coordinates are normalized to \[0, 1] relative to page dimensions. Use when you need to know where each cell is located on the page. Agentic table enhancement is not compatible with `jsonbbox` and will be automatically disabled when this format is used. Comma-separated values. ```python Python theme={null} formatting={"table_output_format": "csv"} ``` ```javascript Node.js theme={null} formatting: { table_output_format: 'csv' } ``` ```bash cURL theme={null} "formatting": {"table_output_format": "csv"} ``` ```csv theme={null} Header 1,Header 2 Data 1,Data 2 ``` Minimal output, easy to import into spreadsheet software. Most token-efficient format.
## Additional Options ### Merge Tables When a logical table spans multiple pages, Reducto may detect it as separate tables. Enable `merge_tables` to combine consecutive tables with the same column count: ```python Python theme={null} formatting={ "table_output_format": "html", "merge_tables": True } ``` ```javascript Node.js theme={null} formatting: { table_output_format: 'html', merge_tables: true } ``` ```go Go theme={null} AdvancedOptions: reducto.F(shared.AdvancedProcessingOptionsParam{ TableOutputFormat: reducto.F(shared.AdvancedProcessingOptionsTableOutputFormatHTML), MergeTables: reducto.F(true), }) ``` ```bash cURL theme={null} "formatting": { "table_output_format": "html", "merge_tables": true } ``` The algorithm: 1. Identifies consecutive tables with identical column counts 2. Uses a language model to determine if the second table is a continuation of the first 3. Combines them into a single table, removing duplicate headers Tables are merged based on column count and semantic analysis. Tables with the same number of columns but different structures may be incorrectly merged. Review output when using this option on complex documents. ### Add Page Markers Inserts page boundary indicators into the content: ```python Python theme={null} formatting={"add_page_markers": True} ``` ```javascript Node.js theme={null} formatting: { add_page_markers: true } ``` ```go Go theme={null} AdvancedOptions: reducto.F(shared.AdvancedProcessingOptionsParam{ AddPageMarkers: reducto.F(true), }) ``` ```bash cURL theme={null} "formatting": {"add_page_markers": true} ``` Output includes markers like: ```markdown theme={null} [[START OF PAGE 1]] # Document Title Content from page 1... [[END OF PAGE 1]] [[START OF PAGE 2]] ``` Useful for page-specific extraction or citation tracking. ### Include Additional Metadata The `formatting` group also supports extracting comments, highlights, change tracking, hyperlinks, and signatures. See [Additional Document Data](/configs/parse/additional-document-data) for details. ## Choosing the Right Format **For LLM context (RAG, summarization):** * Use `dynamic` (default). It balances readability with structure preservation. * Markdown is easier for LLMs to parse than HTML for simple tables. * Complex tables benefit from HTML to preserve relationships between cells. **For programmatic data extraction:** * Use `json` when you need to iterate over rows and cells. * Use `jsonbbox` when cell positions matter (highlighting, overlays). * Use `csv` for direct import into pandas, spreadsheets, or data pipelines. **For accuracy-critical applications:** * Use `html`. It's the only format that preserves merged cells. * Financial statements, regulatory filings, and complex reports need HTML to maintain correct structure. ## Troubleshooting Use `html` format. Markdown and JSON formats cannot represent merged cells and will flatten them. Enable `merge_tables: True` to combine consecutive tables with the same column structure. Use `csv` for minimal output. If you need structure but want fewer tokens, use `json` instead of `html`. Use `jsonbbox` format. Each cell includes normalized (x, y, width, height) coordinates. # Split Configuration Source: https://docs.reducto.ai/configs/split/configuration Configure how documents are divided into sections Split identifies sections in a document based on natural language descriptions. This page covers the configuration options that control how sections are identified and returned. For basic usage and the full workflow, see the [Split endpoint documentation](/split). ## split\_description The `split_description` array defines what sections to look for. Each entry has three fields: ```python Python theme={null} split_description=[ { "name": "Account Summary", "description": "Overview section with balances and totals at the top of the statement", "partition_key": "account_number" # Optional } ] ``` ```javascript Node.js theme={null} split_description: [ { name: 'Account Summary', description: 'Overview section with balances and totals at the top of the statement', partition_key: 'account_number' // Optional } ] ``` ```bash cURL theme={null} "split_description": [ { "name": "Account Summary", "description": "Overview section with balances and totals at the top of the statement", "partition_key": "account_number" } ] ``` **name**: The identifier returned in results. Use names that make sense for your downstream processing logic. **description**: Natural language description of the section's content. The LLM uses this to classify pages. Be specific about what makes this section recognizable: content type, position in document, visual characteristics. **partition\_key**: For sections that repeat with different identifiers (multiple accounts, multiple patients, multiple companies). When set, Split extracts the identifier value from the document and groups pages by that value. ### Writing Effective Descriptions The description is passed to an LLM that classifies each page. Vague descriptions lead to ambiguous classifications. ```python theme={null} # Vague - could match many things {"name": "Tables", "description": "Pages with tables"} # Specific - clear criteria for classification {"name": "Transaction History", "description": "Table showing individual transactions with dates, descriptions, and amounts. Usually appears after the account summary section."} ``` Include distinguishing characteristics: * Content type (tables, narrative text, forms) * Position (beginning, end, after section X) * Visual elements (headers, logos, signature lines) * What it does NOT include (to avoid confusion with similar sections) *** ## partition\_key Partition key handles a common scenario: the same section type repeating for different entities. A consolidated statement has holdings for multiple accounts. A medical record packet has intake forms for multiple patients. Without partition key, Split returns all matching pages as one group. You'd then need to figure out where one entity ends and the next begins. Partition key does this automatically. ```python Python theme={null} split_description=[ { "name": "Holdings", "description": "Investment holdings table for a specific account", "partition_key": "account number" } ] ``` ```javascript Node.js theme={null} split_description: [ { name: 'Holdings', description: 'Investment holdings table for a specific account', partition_key: 'account number' } ] ``` ```bash cURL theme={null} "split_description": [ { "name": "Holdings", "description": "Investment holdings table for a specific account", "partition_key": "account number" } ] ``` The response includes partitions with extracted identifier values: ```json theme={null} { "result": { "splits": [ { "name": "Holdings", "pages": [1, 2, 3, 7, 8, 9, 10, 11], "conf": "high", "partitions": [ {"name": "1234-5678", "pages": [1, 2, 3], "conf": "high"}, {"name": "8765-4321", "pages": [7, 8, 9, 10, 11], "conf": "high"} ] } ] } } ``` The `name` in each partition is the actual value extracted from the document. If the document shows "Account #1234-5678" on pages 1-3 and "Account #8765-4321" on pages 7-11, those become your partition names. **The partition key is semantic, not literal.** If you set `partition_key` to "account number" but the document says "Acct #1234" or "Portfolio ID: 5678", Split will still find it. Describe what the identifier represents, not the exact text format. ### When partition\_key values appear in tables By default, Split truncates table content to speed up processing. If your partition key values appear deep within tables (not in headers or the first few rows), the truncation might hide them. Set `table_cutoff` to `preserve` to keep full table content: ```python Python theme={null} result = client.split.run( input=upload.file_id, split_description=[ { "name": "Holdings", "description": "Investment holdings table", "partition_key": "account_number" } ], settings={"table_cutoff": "preserve"} ) ``` ```javascript Node.js theme={null} const result = await client.split.run({ input: upload.file_id, split_description: [ { name: 'Holdings', description: 'Investment holdings table', partition_key: 'account_number' } ], settings: { table_cutoff: 'preserve' } }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/split \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "document_url": "reducto://your-file-id", "split_description": [ { "name": "Holdings", "description": "Investment holdings table", "partition_key": "account_number" } ], "settings": {"table_cutoff": "preserve"} }' ``` This increases processing time but ensures partition keys aren't missed. *** ## split\_rules Controls how pages are assigned to sections. The default rule: ``` "Split the document into the applicable sections. Sections may only overlap at their first and last page if at all." ``` This means a page can belong to multiple sections only at boundaries. Page 5 can belong to both "Section A" and "Section B" only if it's the last page of A and the first page of B. Customize for your use case: ```python Python theme={null} # Allow full overlap (page can belong to multiple sections anywhere) split_rules="Pages can belong to multiple sections. A page with both summary data and transaction data should appear in both sections." # Force exclusive classification (each page belongs to exactly one section) split_rules="Each page must belong to exactly one section. Assign to the most relevant section." # Document-specific logic split_rules="The cover page (page 1) should not be assigned to any section. Start section detection from page 2." ``` ```javascript Node.js theme={null} // Allow full overlap split_rules: 'Pages can belong to multiple sections. A page with both summary data and transaction data should appear in both sections.' // Force exclusive classification split_rules: 'Each page must belong to exactly one section. Assign to the most relevant section.' // Document-specific logic split_rules: 'The cover page (page 1) should not be assigned to any section. Start section detection from page 2.' ``` ```bash cURL theme={null} "split_rules": "Pages can belong to multiple sections. A page with both summary data and transaction data should appear in both sections." ``` The string is passed directly to the LLM as instructions. Write it as you would write instructions for a person doing the classification. *** ## settings ### table\_cutoff Controls how table content is processed during section detection. ```python Python theme={null} settings={"table_cutoff": "truncate"} # Default settings={"table_cutoff": "preserve"} ``` ```javascript Node.js theme={null} settings: { table_cutoff: 'truncate' } // Default settings: { table_cutoff: 'preserve' } ``` ```bash cURL theme={null} "settings": {"table_cutoff": "truncate"} "settings": {"table_cutoff": "preserve"} ``` **truncate (default)**: Tables are shortened to the first few rows. Faster processing. Works for most cases where section identifiers appear in headers, titles, or surrounding text. **preserve**: Full table content is retained. Required when partition\_key values or section identifiers appear deep within table rows. Slower but more thorough. *** ## parsing Split runs Parse internally before classifying sections. The `parsing` parameter accepts all Parse configuration options. ```python Python theme={null} result = client.split.run( input=upload.file_id, split_description=[...], parsing={ "settings": { "page_range": {"start": 1, "end": 50}, "ocr_system": "standard" }, "enhance": { "agentic": [{"scope": "table"}] } } ) ``` ```javascript Node.js theme={null} const result = await client.split.run({ input: upload.file_id, split_description: [...], parsing: { settings: { page_range: { start: 1, end: 50 }, ocr_system: 'standard' }, enhance: { agentic: [{ scope: 'table' }] } } }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/split \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "document_url": "reducto://your-file-id", "split_description": [...], "parsing": { "settings": { "page_range": {"start": 1, "end": 50}, "ocr_system": "standard" }, "enhance": { "agentic": [{"scope": "table"}] } } }' ``` If you pass `jobid://` as input (reusing a previous Parse result), the `parsing` options are ignored since the document was already parsed. *** ## Response Structure ```json theme={null} { "result": { "splits": [ { "name": "Section Name", "pages": [1, 2, 3], "conf": "high", "partitions": null } ], "section_mapping": { "Section Name": [1, 2, 3] } }, "usage": { "num_pages": 10, "credits": 20.0 } } ``` **splits**: Array of found sections, one per entry in `split_description`. **splits\[].name**: The name you provided. **splits\[].pages**: Page numbers where this section appears (1-indexed). **splits\[].conf**: `"high"` or `"low"` indicating classification confidence. **splits\[].partitions**: When using `partition_key`, sub-sections grouped by extracted identifier values. Each partition has its own `name` (the extracted value), `pages`, and `conf`. **section\_mapping**: Legacy format mapping section names to page arrays. Use `splits` for new code. A section not found in the document still appears in results with an empty pages array. Always check that `pages` has content before processing. # Deep Split Source: https://docs.reducto.ai/configs/split/deep-split Achieve near-perfect accuracy on complex document splitting with our agentic loop ## What is Deep Split? Deep Split is an agentic splitting mode that iteratively refines its output to achieve near-perfect accuracy. Unlike standard split which classifies each page in a single pass, Deep Split runs an agentic loop that verifies and corrects its section assignments against the source document until a quality threshold is met. This is especially useful for complex documents where a single split pass may mislabel pages, miss boundaries between similar sections, or partition repeating sections inconsistently. Deep Split catches these issues by checking its own work and re-classifying until the results are accurate. *** ## When to Use It Deep Split is designed for splitting tasks where accuracy is critical and the cost of errors is high. Common use cases include: * **Consolidated financial statements**, where holdings, transactions, and summaries repeat across many accounts and must be partitioned correctly. * **Insurance claim packets**, where sections like medical records, billing statements, and adjuster notes are visually similar and easy to confuse. * **Loan and mortgage files**, where dozens of disclosures, appraisals, and supporting documents are interleaved and ordering matters for downstream processing. * **Multi-patient medical record bundles**, where intake forms, lab reports, and discharge summaries repeat per patient and must group cleanly by partition. * **Long legal binders**, where exhibits, contracts, and addenda span hundreds of pages and section boundaries are ambiguous. If your splitting task is simple (a handful of clearly distinct sections in a short document), standard split is sufficient. Use Deep Split when you need high reliability on complex or lengthy documents. *** ## How to Use It Enable Deep Split by setting `deep_split` to `true` inside the `settings` object: ```python Python theme={null} result = client.split.run( input=upload.file_id, split_description=[ { "name": "Account Summary", "description": "Overview section with balances and totals at the top of the statement" }, { "name": "Holdings", "description": "Investment holdings table for a specific account", "partition_key": "account_number" }, { "name": "Transaction History", "description": "Table of individual transactions with dates, descriptions, and amounts" } ], settings={"deep_split": True} ) ``` ```javascript Node.js theme={null} const result = await client.split.run({ input: upload.file_id, split_description: [ { name: 'Account Summary', description: 'Overview section with balances and totals at the top of the statement' }, { name: 'Holdings', description: 'Investment holdings table for a specific account', partition_key: 'account_number' }, { name: 'Transaction History', description: 'Table of individual transactions with dates, descriptions, and amounts' } ], settings: { deep_split: true } }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/split \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "split_description": [ { "name": "Account Summary", "description": "Overview section with balances and totals at the top of the statement" }, { "name": "Holdings", "description": "Investment holdings table for a specific account", "partition_key": "account_number" }, { "name": "Transaction History", "description": "Table of individual transactions with dates, descriptions, and amounts" } ], "settings": { "deep_split": true } }' ``` *** ## Best Practices ### Write specific, distinguishing section descriptions The agentic loop relies on your `split_description` entries to verify whether each page is in the right section. Vague descriptions give the agent nothing concrete to check, so include content, position, and visual cues that distinguish each section from its neighbors. ```python theme={null} split_description=[ { "name": "Transaction History", "description": ( "Table showing individual transactions with dates, descriptions, and amounts. " "Appears after the account summary and before the holdings detail. " "Does NOT include opening or closing balance rows." ) } ] ``` Other examples of effective distinguishing criteria: * **Repeating sections:** "Holdings table for a single account. Each block starts with an account number header and ends before the next account header." * **Visually similar sections:** "Lab report. Contains the laboratory name in the header and a results table with reference ranges. Distinct from imaging reports, which contain narrative findings instead of tables." * **Boundary-sensitive sections:** "Signature page. Always the last page of the contract block, immediately before any exhibits." ### Use with partition\_key for repeating sections When the same section repeats for different entities (multiple accounts, multiple patients, multiple companies), pair Deep Split with [`partition_key`](/configs/split/configuration#partition-key) so the agent verifies both the section assignment and the partition value extracted from the page. ### Pair with Parse configuration Deep Split can only verify what Parse sees. If the underlying parse output is missing data (for example, a table is not detected or a header is misread), Deep Split will not be able to recover the missing signal. Consider enabling [agentic mode](/configs/parse/agentic-modes) for tables, or using a higher-fidelity [OCR mode](/configs/parse/ocr-settings) when section identifiers live deep inside tables or in low-quality scans. *** ## Related Endpoint basics and parameters. split\_description, partition\_key, and table\_cutoff. The same agentic loop pattern for schema-based extraction. # Edit Source: https://docs.reducto.ai/editing/edit-overview Fill forms and modify documents with natural language instructions Edit fills PDF forms and modifies DOCX documents using natural language instructions. You describe what values should go where, and Reducto handles field detection, mapping, and insertion. Edit completes the document lifecycle. Where [Parse](/parse/overview) reads documents and [Extract](/extract/overview) pulls data out, Edit writes data back in. This enables end-to-end workflows: classify an incoming document, extract data from it, and populate a different form or template, all within a single pipeline. The Edit endpoint is available in the Python SDK, Node.js SDK, and via cURL. The Go SDK does not yet support Edit. *** ## When to Use Edit Edit solves form filling at scale. Instead of manually clicking through fillable PDFs or templating Word documents, you describe what you want in natural language. **Common use cases:** * Filling government and tax forms (W-9, I-9, G-1145) with applicant data * Completing insurance applications and claim forms * Populating legal contracts with client information * Generating customized DOCX reports from extracted data PDF libraries like PyPDF require knowing exact field names and coordinates. Edit uses AI to understand field context, so "Fill in the applicant name" works even if the PDF field is named `topmostSubform[0].Page1[0].f1_1[0]`. *** ## Quick Start ```python Python theme={null} from pathlib import Path from reducto import Reducto client = Reducto() upload = client.upload(file=Path("w9_form.pdf")) result = client.edit.run( document_url=upload.file_id, edit_instructions=""" Fill this W-9 form with: - Name: Acme Corporation - Business name: Acme Corp LLC - Tax classification: LLC - Address: 123 Main Street, San Francisco, CA 94102 - TIN: 12-3456789 """ ) print(result.document_url) # Download the filled form ``` ```javascript Node.js theme={null} import Reducto from 'reductoai'; import fs from 'fs'; const client = new Reducto(); const upload = await client.upload({ file: fs.createReadStream('w9_form.pdf'), }); const result = await client.edit.run({ document_url: upload.file_id, edit_instructions: ` Fill this W-9 form with: - Name: Acme Corporation - Business name: Acme Corp LLC - Tax classification: LLC - Address: 123 Main Street, San Francisco, CA 94102 - TIN: 12-3456789 ` }); console.log(result.document_url); // Download the filled form ``` ```bash cURL theme={null} # First upload the file FILE_ID=$(curl -s -X POST https://platform.reducto.ai/upload \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -F "file=@w9_form.pdf" | jq -r '.file_id') # Then edit curl -X POST https://platform.reducto.ai/edit \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "document_url": "'$FILE_ID'", "edit_instructions": "Fill this W-9 form with: Name: Acme Corporation, Business name: Acme Corp LLC, Tax classification: LLC, Address: 123 Main Street, San Francisco, CA 94102, TIN: 12-3456789" }' ``` **What happens:** 1. Edit detects all fillable fields in the PDF (text boxes, checkboxes, dropdowns) 2. An LLM reads your instructions and field context (labels, surrounding text) 3. Each field gets mapped to the appropriate value 4. The filled PDF is returned as a downloadable URL *** ## PDF vs DOCX ### PDF PDFs have structured form widgets (text fields, checkboxes, dropdowns). For PDFs without existing form fields, Edit uses vision to detect where fillable areas should be. | Feature | Description | | -------------- | -------------------------------------- | | Text fields | Fill any text input area | | Checkboxes | Check or uncheck based on instructions | | Dropdowns | Choose from available options | | Form detection | Finds fields even in scanned forms | | Overflow | Long text can flow to appendix pages | ### DOCX DOCX supports richer editing because the format allows inline content modification. | Feature | Description | | -------------- | ------------------------------------------- | | Form fields | Modern (2007+) and legacy form controls | | Checkboxes | Both modern and legacy controls | | Table cells | Modify or append to existing cell content | | Text insertion | Insert content at paragraph markers | | Highlighting | Edits get highlighted in configurable color | *** ## Request Parameters ```python Python theme={null} result = client.edit.run( document_url="...", # Required: file to edit edit_instructions="...", # Required: what to fill edit_options={ "color": "#FF0000", # Highlight color (DOCX only) "enable_overflow_pages": False, # Appendix for long text (PDF only) "llm_provider_preference": "openai" # LLM provider (optional) }, form_schema=[...] # Optional: predefined field locations (PDF only) ) ``` ```javascript Node.js theme={null} const result = await client.edit.run({ document_url: '...', // Required: file to edit edit_instructions: '...', // Required: what to fill edit_options: { color: '#FF0000', // Highlight color (DOCX only) enable_overflow_pages: false, // Appendix for long text (PDF only) llm_provider_preference: 'openai' // LLM provider (optional) }, form_schema: [...] // Optional: predefined field locations (PDF only) }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/edit \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "document_url": "...", "edit_instructions": "...", "edit_options": { "color": "#FF0000", "enable_overflow_pages": false, "llm_provider_preference": "openai" }, "form_schema": [...] }' ``` ### document\_url The document to edit. Accepts the same formats as Parse: `reducto://` file IDs from upload, public URLs, or presigned S3/GCS URLs. ### edit\_instructions Natural language instructions describing what to fill. Be explicit about values and which fields they belong to: ``` Fill this form with: - Full Name: John David Smith - Date of Birth: March 15, 1985 - SSN: 123-45-6789 - Check "Yes" for US Citizen - Select "California" for state ``` Vague instructions like "Fill out John's information" perform poorly. Include formatting hints when the form expects specific formats (dates, phone numbers, SSNs). ### edit\_options | Option | Default | Description | | ------------------------- | --------- | ----------------------------------------------------------------------------------------------------- | | `color` | `#FF0000` | Hex color for highlighting edits (DOCX only) | | `enable_overflow_pages` | `false` | Create appendix pages for text exceeding field capacity (PDF only) | | `llm_provider_preference` | `null` | LLM provider for form filling: `"openai"`, `"anthropic"`, or `"google"`. If null, defaults to Google. | ### form\_schema For repeatable form filling, define field locations explicitly. This skips detection, improving speed 3x and consistency. See [Form Schema](/configs/edit/form-schema). *** ## Response ```json theme={null} { "document_url": "https://storage.reducto.ai/filled-form.pdf?...", "form_schema": [ { "bbox": {"left": 0.1, "top": 0.2, "width": 0.4, "height": 0.03, "page": 1}, "description": "Name field in header section", "type": "text" } ], "usage": {"num_pages": 2, "credits": 8} } ``` | Field | Description | | --------------- | ---------------------------------------------------------------------------- | | `document_url` | Presigned URL to download edited document. Valid 24 hours. | | `form_schema` | Detected field schema (PDF only). Save this to reuse for the same form type. | | `usage.credits` | Credits charged: 4 per page. | *** ## Async Processing For larger documents or webhook delivery, use the async endpoint: ```python Python theme={null} result = client.edit.run_job( document_url=upload.file_id, edit_instructions="...", ) print(result.job_id) # Poll /job/{job_id} or wait for webhook # Poll for results import time while True: job = client.job.get(result.job_id) if job.status == "Completed": print(job.result.document_url) break time.sleep(2) ``` ```javascript Node.js theme={null} const result = await client.edit.runJob({ document_url: upload.file_id, edit_instructions: '...', }); console.log(result.job_id); // Poll /job/{job_id} or wait for webhook // Poll for results while (true) { const job = await client.job.retrieve(result.job_id); if (job.status === 'Completed') { console.log(job.result.document_url); break; } await new Promise(r => setTimeout(r, 2000)); } ``` ```bash cURL theme={null} # Submit async job JOB_ID=$(curl -s -X POST https://platform.reducto.ai/edit_async \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "document_url": "reducto://your-file-id", "edit_instructions": "..." }' | jq -r '.job_id') # Poll for results while true; do STATUS=$(curl -s https://platform.reducto.ai/job/$JOB_ID \ -H "Authorization: Bearer $REDUCTO_API_KEY" | jq -r '.status') echo "Status: $STATUS" if [ "$STATUS" = "Completed" ]; then curl -s https://platform.reducto.ai/job/$JOB_ID \ -H "Authorization: Bearer $REDUCTO_API_KEY" | jq '.result.document_url' break fi sleep 2 done ``` Sync requests get priority by default. Async can request priority with `priority=True` if your account has budget available. *** ## How It Works ### PDF 1. **Detect** form widgets (or use vision if none exist) 2. **Analyze** context around each field (labels, headers) 3. **Map** your instructions to fields based on descriptions 4. **Fill** values into the PDF With a `form_schema`, steps 1-2 are skipped since you've defined field locations and descriptions. ### DOCX 1. **Tag** editable locations (paragraphs, table cells, form controls) 2. **Analyze** document structure against your instructions 3. **Generate** specific edits (insert text, check boxes, update cells) 4. **Apply** edits with optional highlighting *** ## Troubleshooting Several things can cause unfilled fields: * **Instructions didn't match**: The LLM couldn't map your instructions to that field. Use terms that appear on the form itself. * **Dropdown mismatch**: Value must exactly match an option ("CA" vs "California") * **Detection missed it**: Use `form_schema` to explicitly define field locations The PDF has no widgets and vision couldn't detect fillable areas. Common causes: * Scanned image without clear form structure * Fields blend into background or lack clear boundaries * Document isn't actually a fillable form **Solution**: Provide a `form_schema` defining field locations. See [Form Schema](/configs/edit/form-schema). PDF fields have fixed sizes. When content exceeds capacity, it gets truncated. **Solutions:** 1. Enable overflow: `edit_options={"enable_overflow_pages": True}` creates appendix pages 2. Abbreviate in your instructions if the form expects short values Be explicit: `Check "Yes" for US Citizen` works better than `US Citizen: Yes`. The LLM needs to understand you mean check a box, not fill text. When fields have similar labels, the LLM may map incorrectly: 1. **Be more specific**: "Applicant First Name: John" rather than "Name: John" 2. **Use form\_schema**: Define exactly which field is which using coordinates 3. **Reference position**: "The name field in the top-left of page 1" No. Parse finds existing content (labels like "Name:"). Edit finds empty fillable areas (the input box next to "Name:"). Form fields are blank rectangles with nothing for Parse to detect. Use Edit once without `form_schema` to detect fields, then save and reuse the returned schema. *** ## Limitations ### Partial Success Edit returns successfully even when some fields couldn't be filled. These situations don't raise errors: | Scenario | Behavior | | --------------------------------------- | --------------------------- | | Dropdown value not in options | Field skipped silently | | Instructions don't match any field | Document returned unchanged | | Unsupported widget (signatures, images) | Widget skipped | ### Format-Specific **PDF:** * Signature and image fields not supported * Radio buttons have limited support * Heavily designed forms may detect incorrectly **DOCX:** * Requires structured documents (form controls, tables) * Very large documents (100+ pages) take longer *** ## Next Steps Pre-define field locations for faster, more consistent filling. Complete endpoint specification. # Enterprise Readiness Source: https://docs.reducto.ai/enterprise/enterprise-readiness How Reducto meets security, reliability, and scale requirements at enterprise grade Reducto is built for production at enterprise scale. With 3B+ pages processed, hybrid and air-gapped deployment options, and SOC 2 + HIPAA compliance, Reducto meets the requirements of teams processing hundreds of millions of pages per month. The Enterprise tier adds contractual SLAs, dedicated compute, white-glove field engineering support, and flexible deployment models. All Enterprise capabilities are negotiable based on your requirements. [Contact our sales team](https://reducto.ai/contact) to discuss what's right for you. ## Availability & SLAs Enterprise customers receive contractual uptime SLAs of **up to 99.99%**. SLA terms are tailored to your deployment model and workload requirements. | | Standard | Growth | Enterprise | | ----------------- | -------- | ------ | ----------------------------------- | | Uptime target | — | — | Up to 99.99% SLA (custom available) | | Contractual SLA | — | — | Included | | Custom MSA | — | — | Included | | Custom DPA | — | — | Included | | Incident response | — | — | 24/7 oncall support | Real-time service status is available at [status.reducto.ai](https://status.reducto.ai/). ## Throughput & Scaling Enterprise customers can negotiate custom concurrency baselines based on their workload. | | Enterprise | | -------------------- | ----------------------------- | | Concurrency baseline | Custom (raised tier baseline) | | Live deployments | Up to 10+ concurrent | | Annual page volume | Up to billions | Reducto throttles per account on a concurrency ceiling, not a per-second rate. See [Concurrency Throttle](/reference/throttling) for how the ceiling is calculated. [Contact sales](https://reducto.ai/contact) to discuss a custom baseline. ## Security & Compliance Reducto maintains enterprise-grade security across all tiers, with additional compliance options for Growth and Enterprise customers. | Capability | Standard | Growth | Enterprise | | ------------------------------- | -------- | --------------- | --------------- | | **SOC 2 Type II** | Yes | Yes | Yes | | **HIPAA (with signed BAA)** | — | Yes | Yes | | **Zero Data Retention** | — | Yes (Ephemeral) | Yes (Ephemeral) | | **SSO and SAML authentication** | — | — | Yes | | **Role-based access control** | — | — | Yes | | **Encryption at rest** | AES-256 | AES-256 | AES-256 | | **Encryption in transit** | TLS 1.2+ | TLS 1.2+ | TLS 1.2+ | | **Custom MSA & DPA** | — | — | Included | | **EU data residency** | — | Yes | Yes | Enterprise tier data is never used for training. For full details, see [Data policies & compliance](/security/policies). ## Deployment Options VPC and on-premise deployments are available exclusively for Enterprise customers. Deployment services are included. * **SaaS** — Fully managed by Reducto on AWS. Zero infrastructure overhead. * **Hybrid VPC** — Data stays in your VPC; processing runs on Reducto's GPU infrastructure. * **Full VPC** — Entirely hosted in your cloud environment (AWS, GCP, or Azure). See [Deployment options](/onprem/enterprise_deployment_options) for architecture details. ## Platform & Tools | | Standard | Growth | Enterprise | | --------------------------- | -------- | --------- | ---------- | | Studio seats | Up to 5 | Unlimited | Unlimited | | Custom processing pipelines | — | — | Available | | Custom model fine-tuning | — | — | Available | | Early access program | — | — | Included | | Ongoing model updates | Yes | Yes | Yes | Enterprise customers can work with Reducto's team to build custom processing pipelines and fine-tune models on their specific document types, improving accuracy for domain-specific layouts, terminology, and edge cases. [Contact sales](https://reducto.ai/contact) to learn more. ## Support Enterprise support includes up to 24/7 coverage with dedicated personnel. | | Standard | Growth | Enterprise | | ---------------------------- | -------------- | -------------- | ---------- | | Email support | Yes | Yes | Yes | | Slack channel | — | Yes | Yes | | Phone support | — | — | Yes | | Dedicated account manager | — | — | Available | | Forward deployed engineering | — | — | Available | | Support hours | Business hours | Business hours | Up to 24/7 | | Custom response time SLAs | — | — | Available | ## Volume Pricing Enterprise pricing is based on annual page volume commitments with per-page overage rates. Custom credit rates and committed-use discounts are available, scaling with volume. [Contact sales](https://reducto.ai/contact) to discuss pricing for your workload. # Extract Source: https://docs.reducto.ai/extract/overview Pull specific data from documents into structured JSON Extract pulls specific fields from documents as structured JSON. You define a schema describing the data you need, and Reducto returns values matching that schema, handling OCR, layout detection, and LLM-based field location under the hood. Extract builds on [Parse](/parse/overview), which processes the document first, then uses AI to locate and return the specified fields accurately, even across complex layouts. *** ## Parse vs Extract Both endpoints process documents, but they answer different questions. **Parse** answers: "What's in this document?" It returns all content as structured chunks with positions and types. Use Parse for RAG pipelines, document viewers, or when you need to feed full content to an LLM. **Extract** answers: "What is the value of X?" It returns only the specific fields you request. Extract runs Parse internally, then uses AI to pull out values matching your schema. The key insight is that **Extract can only return what Parse sees**. If a value doesn't appear in the Parse output (perhaps due to OCR issues or a table format problem), no amount of schema tweaking will extract it. When debugging extraction issues, always verify the data exists in the Parse result first. *** ## Quick Start Given [this investment statement](https://studio.reducto.ai/share/md726aw3w7mfs46659ttkqry0s7se3pd?processor=kh7c9e30evkfb5a4h80dq4xke17sfwck\&fileId=js7e4hrtnh2tsyjqdbz114ceyn7sf1v1), we'll extract the portfolio value change, total income, and top holdings: Finance Statement ```python Python theme={null} from pathlib import Path from reducto import Reducto client = Reducto() upload = client.upload(file=Path("fidelity-example.pdf")) result = client.extract.run( input=upload.file_id, instructions={ "schema": { "type": "object", "properties": { "portfolio_increase": { "type": "number", "description": "Increase in total portfolio value" }, "total_income_ytd": { "type": "number", "description": "Total income year-to-date" }, "top_holdings": { "type": "array", "items": {"type": "string"}, "description": "Names of top holdings" } } }, "system_prompt": "Extract financial data from this investment statement." } ) print(result.result) ``` ```javascript Node.js theme={null} import Reducto from 'reductoai'; import fs from 'fs'; const client = new Reducto(); const upload = await client.upload({ file: fs.createReadStream('fidelity-example.pdf'), }); const result = await client.extract.run({ input: upload.file_id, instructions: { schema: { type: 'object', properties: { portfolio_increase: { type: 'number', description: 'Increase in total portfolio value' }, total_income_ytd: { type: 'number', description: 'Total income year-to-date' }, top_holdings: { type: 'array', items: { type: 'string' }, description: 'Names of top holdings' } } }, system_prompt: 'Extract financial data from this investment statement.' } }); console.log(result.result); ``` ```go Go theme={null} package main import ( "context" "fmt" "io" "os" reducto "github.com/reductoai/reducto-go-sdk" "github.com/reductoai/reducto-go-sdk/option" "github.com/reductoai/reducto-go-sdk/shared" ) func main() { client := reducto.NewClient(option.WithAPIKey(os.Getenv("REDUCTO_API_KEY"))) file, _ := os.Open("fidelity-example.pdf") defer file.Close() upload, _ := client.Upload(context.Background(), reducto.UploadParams{ File: reducto.F[io.Reader](file), }) schema := map[string]interface{}{ "type": "object", "properties": map[string]interface{}{ "portfolio_increase": map[string]interface{}{ "type": "number", "description": "Increase in total portfolio value", }, "total_income_ytd": map[string]interface{}{ "type": "number", "description": "Total income year-to-date", }, "top_holdings": map[string]interface{}{ "type": "array", "items": map[string]interface{}{"type": "string"}, "description": "Names of top holdings", }, }, } result, _ := client.Extract.Run(context.Background(), reducto.ExtractRunParams{ ExtractConfig: reducto.ExtractConfigParam{ DocumentURL: reducto.F[reducto.ExtractConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), Schema: reducto.F[interface{}](schema), SystemPrompt: reducto.F("Extract financial data from this investment statement."), }, }) fmt.Printf("%+v\n", result.Result) } ``` ```bash cURL theme={null} # First upload the file FILE_ID=$(curl -s -X POST https://platform.reducto.ai/upload \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -F "file=@fidelity-example.pdf" | jq -r '.file_id') # Then extract with schema curl -X POST https://platform.reducto.ai/extract \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "'$FILE_ID'", "instructions": { "schema": { "type": "object", "properties": { "portfolio_increase": { "type": "number", "description": "Increase in total portfolio value" }, "total_income_ytd": { "type": "number", "description": "Total income year-to-date" }, "top_holdings": { "type": "array", "items": {"type": "string"}, "description": "Names of top holdings" } } }, "system_prompt": "Extract financial data from this investment statement." } }' ``` **What this does:** 1. **Upload** the PDF to get a `file_id` 2. **`Call /extract`** with the file reference and a JSON schema defining the three fields you want 3. **Get back** JSON with exactly those fields populated from the document ```json theme={null} { "result": [ { "portfolio_increase": 21000.37, "total_income_ytd": 23278.62, "top_holdings": [ "Johnson & Johnson (JNJ)", "Apple Inc (AAPL)", "NH Portfolio 2015 Delphi", "Corp Jr Sb Nt Slm Corp", "Spi Lkd Nt (OSM)" ] } ], "job_id": "9531166f-9725-4854-8096-459785a33972", "usage": {"num_fields": 7, "num_pages": 3, "credits": 10.0}, "studio_link": "https://studio.reducto.ai/job/9531166f-..." } ``` The `result` is an array containing objects matching your schema. When you enable citations, the response format changes to wrap each value with its source location. Full breakdown of result structure, citations, and usage fields. *** ## Request Parameters ```python theme={null} result = client.extract.run( input="...", # Required: file_id, jobid://, or URL instructions={ "schema": {...}, # JSON schema defining fields to extract "system_prompt": "..." # Context for the LLM about the document }, settings={ "array_extract": False, # Segment document for long arrays "citations": { "enabled": False, # Return source locations "numerical_confidence": True }, "include_images": False, "optimize_for_latency": False }, parsing={...} # Parse options (ignored if using jobid://) ) ``` ### input (required) The document to process. Accepts several formats: | Format | Example | When to use | | --------------- | ---------------------------------------- | ---------------------------------- | | Upload response | `reducto://abc123` | Local files uploaded via `/upload` | | Public URL | `https://example.com/doc.pdf` | Publicly accessible documents | | Presigned URL | `https://bucket.s3.../doc.pdf?X-Amz-...` | Files in your cloud storage | | Job ID | `jobid://7600c8c5-...` | Reuse a previous Parse result | | Job ID list | `["jobid://...", "jobid://..."]` | Combine multiple parsed documents | Using `jobid://` skips the parsing step entirely, which is useful when you want to try different extraction schemas on the same document without re-parsing, or when combining data from multiple documents into a single extraction. ```python Python theme={null} # Combine multiple parsed documents result = client.extract.run( input=["jobid://job-1", "jobid://job-2", "jobid://job-3"], instructions={"schema": schema} ) ``` ```javascript Node.js theme={null} // Combine multiple parsed documents const result = await client.extract.run({ input: ['jobid://job-1', 'jobid://job-2', 'jobid://job-3'], instructions: { schema } }); ``` ```go Go theme={null} // Combine multiple parsed documents result, _ := client.Extract.Run(context.Background(), reducto.ExtractRunParams{ ExtractConfig: reducto.ExtractConfigParam{ DocumentURL: reducto.F[reducto.ExtractConfigDocumentURLUnionParam]( reducto.ExtractConfigDocumentURLArrayParam{ shared.UnionString("jobid://job-1"), shared.UnionString("jobid://job-2"), shared.UnionString("jobid://job-3"), }, ), Schema: reducto.F[interface{}](schema), }, }) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/extract \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": ["jobid://job-1", "jobid://job-2", "jobid://job-3"], "instructions": {"schema": {...}} }' ``` ### instructions | Field | Purpose | | --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `schema` | JSON schema defining target fields and types. Field names and descriptions directly influence extraction quality because the LLM uses them to locate values. A field called `invoice_total` with description `"The total amount due, typically at the bottom of the invoice"` performs better than a generic `total` field. | | `system_prompt` | Document-level context. Describe what kind of document this is or highlight edge cases. Field-specific instructions belong in schema descriptions, not here. | ### settings | Field | Default | Purpose | | -------------------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `array_extract` | `false` | Deprecated. Use `deep_extract` instead (higher cost and latency). Previously used for documents with repeating data (line items, transactions): segments the document, extracts from each segment, and merges results. | | `deep_extract` | `false` | Agentic extraction mode that iteratively refines its output to achieve near-perfect accuracy. Best for complex documents where accuracy is critical. See [Deep Extract](/configs/extract/deep-extract). | | `citations.enabled` | `false` | Return page number, bounding box, and source text for each extracted value. Useful for verification and debugging. | | `citations.numerical_confidence` | `true` | When citations are enabled, include a 0-1 confidence score instead of just "high"/"low". | | `include_images` | `false` | Include page images in the extraction context. Can help with visually complex documents but increases cost. | | `optimize_for_latency` | `false` | Prioritize speed at 2x credit cost. Jobs get higher priority in the processing queue. | Citations cannot be used with chunking. If you enable `settings.citations.enabled`, the parsing step automatically disables chunking. This is because citations require knowing exactly where each piece of content came from, which chunking obscures. ### parsing Since Extract runs Parse internally, you can configure how parsing works. These options are ignored if your `input` is a `jobid://` reference. Common options: ```python Python theme={null} result = client.extract.run( input=upload.file_id, instructions={"schema": schema}, parsing={ "enhance": { "agentic": [{"scope": "table"}] # LLM correction for tables }, "formatting": { "table_output_format": "html" # Better for complex tables }, "settings": { "page_range": {"start": 1, "end": 10}, # Process specific pages "document_password": "secret" # For encrypted PDFs } } ) ``` ```javascript Node.js theme={null} const result = await client.extract.run({ input: upload.file_id, instructions: { schema }, parsing: { enhance: { agentic: [{ scope: 'table' }] // LLM correction for tables }, formatting: { table_output_format: 'html' // Better for complex tables }, settings: { page_range: { start: 1, end: 10 }, // Process specific pages document_password: 'secret' // For encrypted PDFs } } }); ``` ```go Go theme={null} result, _ := client.Extract.Run(context.Background(), reducto.ExtractRunParams{ ExtractConfig: reducto.ExtractConfigParam{ DocumentURL: reducto.F[reducto.ExtractConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), Schema: reducto.F[interface{}](schema), Options: reducto.F(shared.BaseProcessingOptionsParam{ TableOutputFormat: reducto.F(shared.BaseProcessingOptionsTableOutputFormatHTML), PageRange: reducto.F(shared.PageRangeParam{ Start: reducto.F(int64(1)), End: reducto.F(int64(10)), }), DocumentPassword: reducto.F("secret"), }), AdvancedOptions: reducto.F(shared.AdvancedProcessingOptionsParam{ Agentic: reducto.F([]shared.AgenticModeConfigParam{ {Scope: reducto.F(shared.AgenticModeConfigScopeTable)}, }), }), }, }) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/extract \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "instructions": {"schema": {...}}, "parsing": { "enhance": { "agentic": [{"scope": "table"}] }, "formatting": { "table_output_format": "html" }, "settings": { "page_range": {"start": 1, "end": 10}, "document_password": "secret" } } }' ``` All available parsing options. *** ## Schema vs Schemaless Extract supports two modes of operation: schema-based extraction (the default) and schemaless extraction. **Schema-based extraction** is what most users need. You define a JSON schema specifying exactly which fields to extract and their types. The model returns data matching your schema structure. This gives you predictable, typed output that integrates cleanly with your application code. ```python Python theme={null} # Schema-based: you define the exact structure result = client.extract.run( input=upload.file_id, instructions={ "schema": { "type": "object", "properties": { "invoice_number": {"type": "string"}, "total": {"type": "number"} } } } ) ``` ```javascript Node.js theme={null} // Schema-based: you define the exact structure const result = await client.extract.run({ input: upload.file_id, instructions: { schema: { type: 'object', properties: { invoice_number: { type: 'string' }, total: { type: 'number' } } } } }); ``` ```go Go theme={null} // Schema-based: you define the exact structure schema := map[string]interface{}{ "type": "object", "properties": map[string]interface{}{ "invoice_number": map[string]interface{}{"type": "string"}, "total": map[string]interface{}{"type": "number"}, }, } result, _ := client.Extract.Run(context.Background(), reducto.ExtractRunParams{ ExtractConfig: reducto.ExtractConfigParam{ DocumentURL: reducto.F[reducto.ExtractConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), Schema: reducto.F[interface{}](schema), }, }) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/extract \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "instructions": { "schema": { "type": "object", "properties": { "invoice_number": {"type": "string"}, "total": {"type": "number"} } } } }' ``` **Schemaless extraction** lets the model decide what to extract based on a natural language prompt. Instead of providing a schema, you describe what you want in plain English. The model analyzes the document and returns whatever it deems relevant. This is useful for exploration or when you don't know the document structure in advance. ```python Python theme={null} # Schemaless: the model decides what to extract result = client.extract.run( input=upload.file_id, instructions={ "system_prompt": "Extract all the key financial information from this invoice" } ) ``` ```javascript Node.js theme={null} // Schemaless: the model decides what to extract const result = await client.extract.run({ input: upload.file_id, instructions: { system_prompt: 'Extract all the key financial information from this invoice' } }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/extract \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "instructions": { "system_prompt": "Extract all the key financial information from this invoice" } }' ``` Use schema-based extraction for production workflows where you need consistent output structure. Use schemaless extraction when exploring new document types or building prototypes. Detailed guidance on schema design, naming conventions, and descriptions. *** ## Array Extraction Standard extraction works well for short documents, but for documents with many repeating items (hundreds of transactions, long invoice line items), you need array extraction. The problem: LLMs have context limits. When a document is too long, items toward the end may be truncated or missed. Array extraction solves this by segmenting the document, extracting from each segment, and merging the results. ```python Python theme={null} result = client.extract.run( input=upload.file_id, instructions={ "schema": { "type": "object", "properties": { "transactions": { "type": "array", "items": { "type": "object", "properties": { "date": {"type": "string"}, "description": {"type": "string"}, "amount": {"type": "number"} } } } } } }, settings={"array_extract": True} ) ``` ```javascript Node.js theme={null} const result = await client.extract.run({ input: upload.file_id, instructions: { schema: { type: 'object', properties: { transactions: { type: 'array', items: { type: 'object', properties: { date: { type: 'string' }, description: { type: 'string' }, amount: { type: 'number' } } } } } } }, settings: { array_extract: true } }); ``` ```go Go theme={null} schema := map[string]interface{}{ "type": "object", "properties": map[string]interface{}{ "transactions": map[string]interface{}{ "type": "array", "items": map[string]interface{}{ "type": "object", "properties": map[string]interface{}{ "date": map[string]interface{}{"type": "string"}, "description": map[string]interface{}{"type": "string"}, "amount": map[string]interface{}{"type": "number"}, }, }, }, }, } result, _ := client.Extract.Run(context.Background(), reducto.ExtractRunParams{ ExtractConfig: reducto.ExtractConfigParam{ DocumentURL: reducto.F[reducto.ExtractConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), Schema: reducto.F[interface{}](schema), ArrayExtract: reducto.F(shared.ArrayExtractConfigParam{ Enabled: reducto.F(true), }), }, }) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/extract \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "instructions": { "schema": { "type": "object", "properties": { "transactions": { "type": "array", "items": { "type": "object", "properties": { "date": {"type": "string"}, "description": {"type": "string"}, "amount": {"type": "number"} } } } } } }, "settings": {"array_extract": true} }' ``` Array extraction requires at least one top-level property of type `array` in your schema. If your schema has no arrays, the endpoint returns an error. Detailed configuration and algorithm options. *** ## Citations Citations link each extracted value back to its source location in the document. Enable them when you need to verify extractions or show users where values came from. ```python Python theme={null} result = client.extract.run( input=upload.file_id, instructions={"schema": schema}, settings={ "citations": { "enabled": True } } ) # With citations enabled, result is a dict with wrapped values field = result.result["total_amount"] print(f"Value: {field.value}") print(f"Found on page {field.citations[0].bbox.page}") print(f"Confidence: {field.citations[0].confidence}") ``` ```javascript Node.js theme={null} const result = await client.extract.run({ input: upload.file_id, instructions: { schema }, settings: { citations: { enabled: true } } }); // With citations enabled, result is an object with wrapped values const field = result.result.total_amount; console.log(`Value: ${field.value}`); console.log(`Found on page ${field.citations[0].bbox.page}`); console.log(`Confidence: ${field.citations[0].confidence}`); ``` ```go Go theme={null} result, _ := client.Extract.Run(context.Background(), reducto.ExtractRunParams{ ExtractConfig: reducto.ExtractConfigParam{ DocumentURL: reducto.F[reducto.ExtractConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), Schema: reducto.F[interface{}](schema), GenerateCitations: reducto.F(true), }, }) // With citations enabled, access wrapped values resultMap := result.Result.(map[string]interface{}) field := resultMap["total_amount"].(map[string]interface{}) fmt.Printf("Value: %v\n", field["value"]) citations := field["citations"].([]interface{}) bbox := citations[0].(map[string]interface{})["bbox"].(map[string]interface{}) fmt.Printf("Found on page %v\n", bbox["page"]) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/extract \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "instructions": {"schema": {...}}, "settings": { "citations": { "enabled": true } } }' ``` When citations are enabled, the response format changes. Each value is wrapped in an object containing `value` and `citations`: ```json theme={null} { "result": { "total_amount": { "value": 23278.62, "citations": [ { "type": "Table", "content": "Total: $23,278.62", "bbox": {"left": 0.04, "top": 0.26, "width": 0.45, "height": 0.50, "page": 3}, "confidence": "high" } ] } } } ``` Each citation includes: * **Page number** where the value was found * **Bounding box** coordinates (normalized 0-1) * **Confidence** as `"high"` or `"low"` * **Source text** the original text that was extracted from Working with bounding boxes and confidence scores. *** ## Troubleshooting LLM outputs are inherently non-deterministic. Small variations are normal. To reduce variance: 1. Use enums to constrain possible values 2. Make field descriptions more specific 3. Add examples in your system prompt If you need identical outputs for identical inputs, consider caching results by document hash. This typically happens with long documents containing arrays. Enable `array_extract` to process the full document: ```python Python theme={null} result = client.extract.run( input=upload.file_id, instructions={"schema": schema}, settings={"array_extract": True} ) ``` ```javascript Node.js theme={null} const result = await client.extract.run({ input: upload.file_id, instructions: { schema }, settings: { array_extract: true } }); ``` ```go Go theme={null} result, _ := client.Extract.Run(context.Background(), reducto.ExtractRunParams{ ExtractConfig: reducto.ExtractConfigParam{ DocumentURL: reducto.F[reducto.ExtractConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), Schema: reducto.F[interface{}](schema), ArrayExtract: reducto.F(shared.ArrayExtractConfigParam{ Enabled: reducto.F(true), }), }, }) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/extract \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "instructions": {"schema": {...}}, "settings": {"array_extract": true} }' ``` You can also add guidance in your system prompt: "Process all pages in the document, not just the beginning." When expected fields come back empty: 1. **Check the Parse output first.** Extract can only find what Parse sees. Run `client.parse.run(input=upload.file_id)` and verify the value appears in the content. 2. **If it's in Parse output**, refine your schema. Add better field descriptions that match how the value appears in the document. 3. **If it's not in Parse output**, adjust your parsing configuration. Try enabling agentic mode for tables, or changing the table output format to HTML. For long arrays, also try enabling `array_extract`. Extract returns only what's on the document. If you request calculated fields (like "annual cost" when only monthly appears), the model may fabricate values. **Solution**: Extract raw values and compute in your code: ```python Python theme={null} monthly_cost = result.result["monthly_cost"].value annual_cost = monthly_cost * 12 # Compute yourself ``` ```javascript Node.js theme={null} const monthlyCost = result.result.monthly_cost.value; const annualCost = monthlyCost * 12; // Compute yourself ``` ```go Go theme={null} resultMap := result.Result.(map[string]interface{}) monthlyCost := resultMap["monthly_cost"].(map[string]interface{})["value"].(float64) annualCost := monthlyCost * 12 // Compute yourself ``` Enable citations to verify source locations for any suspicious values. Very large schemas may exceed LLM token limits and fail with a 422 error. Solutions: 1. Flatten deeply nested structures 2. Remove unnecessary fields 3. Split into multiple extraction calls As a rule of thumb, keep schemas under 50 fields. If you need more, consider breaking the extraction into logical groups. If you see "Citations and chunking cannot be enabled at the same time", you have conflicting options. When citations are enabled, chunking is automatically disabled in the parsing step. If you're explicitly setting chunking options in `parsing.retrieval.chunking`, either remove them or disable citations. Pass the document password in parsing settings: ```python Python theme={null} result = client.extract.run( input=upload.file_id, instructions={"schema": schema}, parsing={ "settings": {"document_password": "your-password"} } ) ``` ```javascript Node.js theme={null} const result = await client.extract.run({ input: upload.file_id, instructions: { schema }, parsing: { settings: { document_password: 'your-password' } } }); ``` ```go Go theme={null} result, _ := client.Extract.Run(context.Background(), reducto.ExtractRunParams{ ExtractConfig: reducto.ExtractConfigParam{ DocumentURL: reducto.F[reducto.ExtractConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), Schema: reducto.F[interface{}](schema), Options: reducto.F(shared.BaseProcessingOptionsParam{ DocumentPassword: reducto.F("your-password"), }), }, }) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/extract \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "instructions": {"schema": {...}}, "parsing": { "settings": {"document_password": "your-password"} } }' ``` *** ## Next Steps Full breakdown of the response structure. Schema design and prompt writing tips. Handle long documents with repeating data. Trace values back to source locations. # Extract Response Format Source: https://docs.reducto.ai/extract/response-format Understanding extracted data, citations, and usage Extract returns your extracted data as structured JSON matching your schema. The response format differs depending on whether citations are enabled. *** ## Response Structure ### Without Citations (Default) When citations are disabled (the default), `result` contains an array of objects with your extracted values directly: ```json theme={null} { "job_id": "9531166f-9725-4854-8096-459785a33972", "result": [ { "invoice_number": "INV-2024-001", "total": 1575.00, "line_items": [ { "description": "Professional Services", "quantity": 10, "amount": 1500.00 }, { "description": "Materials", "quantity": 1, "amount": 75.00 } ] } ], "usage": { "num_pages": 1, "num_fields": 8, "credits": 8.0 }, "studio_link": "https://studio.reducto.ai/job/9531166f-..." } ``` ### Top-Level Fields | Field | Type | Description | | ------------------ | --------------- | --------------------------------------------------------------------------------------------------------------- | | `job_id` | string | Unique identifier for this extraction job. Use this to retrieve results later or reference in support requests. | | `result` | array or object | Without citations: an array containing your extracted data. With citations: an object with wrapped values. | | `usage.num_pages` | integer | Number of document pages processed. | | `usage.num_fields` | integer | Total number of fields extracted, including nested fields in arrays. | | `usage.credits` | number | Credits consumed for this extraction. | | `studio_link` | string | Link to view and debug this extraction in Reducto Studio. | *** ## Accessing Values ### Without Citations When citations are disabled, access values directly from the result array: ```python Python theme={null} # Access the first (usually only) result object data = result.result[0] # Access scalar fields directly invoice_number = data["invoice_number"] total = data["total"] # Access array items for item in data["line_items"]: print(f"{item['description']}: ${item['amount']}") ``` ```javascript Node.js theme={null} // Access the first (usually only) result object const data = result.result[0]; // Access scalar fields directly const invoiceNumber = data.invoice_number; const total = data.total; // Access array items for (const item of data.line_items) { console.log(`${item.description}: $${item.amount}`); } ``` ```go Go theme={null} // Access the first result object data := result.Result[0] // Access fields from the map invoiceNumber := data["invoice_number"] total := data["total"] // Access array items lineItems := data["line_items"].([]interface{}) for _, item := range lineItems { itemMap := item.(map[string]interface{}) fmt.Printf("%s: $%.2f\n", itemMap["description"], itemMap["amount"]) } ``` ### With Citations When citations are enabled, values are wrapped in objects with `value` and `citations` fields: ```python Python theme={null} # With citations, result is a dict (not an array) invoice_number = result.result["invoice_number"].value total = result.result["total"].value # Access array items for item in result.result["line_items"]: print(f"{item['description'].value}: ${item['amount'].value}") ``` ```javascript Node.js theme={null} // With citations, result is an object (not an array) const invoiceNumber = result.result.invoice_number.value; const total = result.result.total.value; // Access array items for (const item of result.result.line_items) { console.log(`${item.description.value}: $${item.amount.value}`); } ``` ```go Go theme={null} // With citations, result is a map with wrapped values invoiceNumber := result.Result["invoice_number"].(map[string]interface{})["value"] total := result.Result["total"].(map[string]interface{})["value"] // Access array items lineItems := result.Result["line_items"].([]interface{}) for _, item := range lineItems { itemMap := item.(map[string]interface{}) desc := itemMap["description"].(map[string]interface{})["value"] amount := itemMap["amount"].(map[string]interface{})["value"] fmt.Printf("%s: $%.2f\n", desc, amount) } ``` When a field cannot be extracted, it may appear as `null` or be absent entirely, depending on whether it was marked as required in your schema. *** ## Citations When `settings.citations.enabled` is `true`, the response format changes. The `result` becomes an object (not an array), and each value is wrapped with citation data: ```json theme={null} { "result": { "total": { "value": 1575.00, "citations": [ { "type": "Table", "content": "Total Due: $1,575.00", "bbox": { "left": 0.65, "top": 0.82, "width": 0.25, "height": 0.03, "page": 1, "original_page": 1 }, "confidence": "high", "granular_confidence": { "extract_confidence": 0.95, "parse_confidence": 0.91 }, "parentBlock": { "type": "Table", "content": "Invoice Total\nTotal Due: $1,575.00", "bbox": {"left": 0.60, "top": 0.78, "width": 0.35, "height": 0.08, "page": 1} } } ] } } } ``` ### Citation Fields | Field | Description | | --------------------- | --------------------------------------------------------------------------------------------------------------------------------- | | `type` | Block type where the value was found: `Text`, `Table`, `Key Value`, etc. | | `content` | The source text from which the value was extracted. May differ slightly from the extracted value due to formatting normalization. | | `bbox` | Bounding box coordinates for the source location. | | `confidence` | Overall confidence as `"high"` or `"low"`. | | `granular_confidence` | Detailed confidence breakdown with `extract_confidence` (0-1) and `parse_confidence` (0-1). | | `parentBlock` | The larger Parse block containing this citation. Useful for context when the citation is very granular. | ### Bounding Box Coordinates All coordinates are normalized to the range \[0, 1] relative to page dimensions: | Field | Description | | --------------- | ------------------------------------------------------------------------------------------------------ | | `left` | Distance from the left edge. 0 is the left margin, 1 is the right margin. | | `top` | Distance from the top edge. 0 is the top, 1 is the bottom. | | `width` | Width as a fraction of page width. | | `height` | Height as a fraction of page height. | | `page` | Page number (1-indexed) in the processed document. | | `original_page` | Page number in the original document. Differs from `page` when using `page_range` to process a subset. | To convert to pixel coordinates, multiply by the page dimensions: ```python Python theme={null} # If your page is 612x792 pixels (standard letter) bbox = citation.bbox pixel_left = bbox.left * 612 pixel_top = bbox.top * 792 pixel_width = bbox.width * 612 pixel_height = bbox.height * 792 ``` ```javascript Node.js theme={null} // If your page is 612x792 pixels (standard letter) const bbox = citation.bbox; const pixelLeft = bbox.left * 612; const pixelTop = bbox.top * 792; const pixelWidth = bbox.width * 612; const pixelHeight = bbox.height * 792; ``` ```go Go theme={null} // If your page is 612x792 pixels (standard letter) bbox := citation.Bbox pixelLeft := bbox.Left * 612 pixelTop := bbox.Top * 792 pixelWidth := bbox.Width * 612 pixelHeight := bbox.Height * 792 ``` ### Array Citations For array fields, each item in the array has its own citations. The structure mirrors the data: ```json theme={null} { "line_items": [ { "description": { "value": "Professional Services", "citations": [{"bbox": {...}, "content": "Professional Services", ...}] }, "amount": { "value": 1500.00, "citations": [{"bbox": {...}, "content": "$1,500.00", ...}] } }, { "description": { "value": "Materials", "citations": [{"bbox": {...}, "content": "Materials", ...}] }, "amount": { "value": 75.00, "citations": [{"bbox": {...}, "content": "$75.00", ...}] } } ] } ``` Each field within each array item has its own citation pointing to where that specific value was found. *** ## Spreadsheet Citations Excel and other spreadsheet formats use a different coordinate system because they have cells, not continuous pages. ### Coordinate Differences | Aspect | PDFs/Images | Spreadsheets | | ----------------- | ----------------------- | ------------------------------------------------- | | Coordinate system | Normalized 0-1 range | Cell positions (1-indexed) | | `left` | Fraction of page width | Column number (1 = A, 2 = B, etc.) | | `top` | Fraction of page height | Row number (1 = first row) | | `width` | Fraction of page width | Number of columns spanned | | `height` | Fraction of page height | Number of rows spanned | | `page` | Page number | Sheet number (1-indexed position in the workbook) | ### Example Spreadsheet Citation ```json theme={null} { "bbox": { "left": 2, // Column B (1-indexed) "top": 5, // Row 5 (1-indexed) "width": 1, // Single column "height": 1, // Single row "page": 1, // Sheet1 "original_page": 1 } } ``` This citation points to cell B5 on the first sheet. To convert to Excel notation, use `top` directly as the row number and map `left` to a letter (1 = A, 2 = B, etc.). *** ## Confidence Scores Confidence indicates how certain the extraction is about a value. Each citation includes both summary and detailed confidence information. ### Summary Confidence The `confidence` field provides a quick assessment: ```json theme={null} "confidence": "high" ``` Values are either `"high"` or `"low"` based on internal thresholds. ### Granular Confidence The `granular_confidence` object provides detailed numerical scores: ```json theme={null} "granular_confidence": { "extract_confidence": 0.95, "parse_confidence": 0.91 } ``` | Score | Description | | -------------------- | ----------------------------------------------------------------------------------------------------------- | | `extract_confidence` | How confident the extraction LLM is about this value (0-1). May be `null` for array items. | | `parse_confidence` | How confident the parsing stage was about the source text (0-1). Reflects OCR and layout detection quality. | Use granular confidence when you need to set custom thresholds or debug extraction issues. Low `parse_confidence` suggests the source document may have OCR or layout problems. Low `extract_confidence` suggests the schema description may need refinement. *** ## Usage and Credits The `usage` object shows what was processed and what it cost: ```json theme={null} { "usage": { "num_pages": 3, "num_fields": 24, "credits": 12.0 } } ``` | Field | Description | | ------------ | -------------------------------------------------------------------------------------------------------------------------------- | | `num_pages` | Document pages that were processed. Affected by `page_range` settings. | | `num_fields` | Total leaf fields extracted. A schema with 5 scalar fields and an array of 10 objects with 2 fields each would report 25 fields. | | `credits` | Credits charged. Based on pages processed plus complexity factors like agentic modes and latency optimization. | Credit calculation varies based on: * Number of pages processed * Whether agentic parsing modes were used * Whether `optimize_for_latency` was enabled (2x multiplier) * Spreadsheet complexity (cell count for Excel files) See [Credit Usage](/faq/credit-usage-overview#extract-endpoint) for detailed pricing. *** ## Complete Example ```json theme={null} { "job_id": "543d1950-068c-4e38-981d-98903326b554", "result": { "invoice_number": { "value": "INV-2024-001", "citations": [ { "type": "Text", "content": "Invoice #INV-2024-001", "bbox": {"left": 0.70, "top": 0.08, "width": 0.20, "height": 0.02, "page": 1, "original_page": 1}, "confidence": "high", "granular_confidence": {"extract_confidence": 0.98, "parse_confidence": 0.95} } ] }, "date": { "value": "2024-01-15", "citations": [ { "type": "Text", "content": "Date: January 15, 2024", "bbox": {"left": 0.70, "top": 0.11, "width": 0.15, "height": 0.02, "page": 1, "original_page": 1}, "confidence": "high", "granular_confidence": {"extract_confidence": 0.96, "parse_confidence": 0.94} } ] }, "total": { "value": 1575.00, "citations": [ { "type": "Table", "content": "Total: $1,575.00", "bbox": {"left": 0.75, "top": 0.85, "width": 0.15, "height": 0.02, "page": 1, "original_page": 1}, "confidence": "high", "granular_confidence": {"extract_confidence": 0.97, "parse_confidence": 0.91} } ] }, "line_items": [ { "description": { "value": "Professional Services", "citations": [ { "type": "Table", "content": "Professional Services", "bbox": {"left": 0.10, "top": 0.45, "width": 0.35, "height": 0.02, "page": 1, "original_page": 1}, "confidence": "high", "granular_confidence": {"extract_confidence": null, "parse_confidence": 0.93} } ] }, "amount": { "value": 1500.00, "citations": [ { "type": "Table", "content": "$1,500.00", "bbox": {"left": 0.78, "top": 0.45, "width": 0.12, "height": 0.02, "page": 1, "original_page": 1}, "confidence": "high", "granular_confidence": {"extract_confidence": null, "parse_confidence": 0.93} } ] } }, { "description": { "value": "Materials", "citations": [ { "type": "Table", "content": "Materials", "bbox": {"left": 0.10, "top": 0.48, "width": 0.20, "height": 0.02, "page": 1, "original_page": 1}, "confidence": "high", "granular_confidence": {"extract_confidence": null, "parse_confidence": 0.91} } ] }, "amount": { "value": 75.00, "citations": [ { "type": "Table", "content": "$75.00", "bbox": {"left": 0.78, "top": 0.48, "width": 0.10, "height": 0.02, "page": 1, "original_page": 1}, "confidence": "high", "granular_confidence": {"extract_confidence": null, "parse_confidence": 0.91} } ] } } ] }, "usage": { "num_pages": 1, "num_fields": 8, "credits": 6.0 }, "studio_link": "https://studio.reducto.ai/job/543d1950-068c-4e38-981d-98903326b554" } ``` *** ## Related Quick start and parameters. Working with source locations. Handle long documents with repeating data. Schema design and prompt tips. # Extract Best Practices Source: https://docs.reducto.ai/extraction/best-practices-extract Schema design and prompt writing for reliable extractions Reliable extractions come from understanding the system's architecture. Extract uses an LLM to find and pull values from parsed content, so the quality of your results depends on two things: whether the data exists in the Parse output, and whether your schema helps the LLM locate it. *** ## Start with Parse When extractions return incorrect values, the root cause is often parsing, not extraction. Extract never works directly on your original file. It only sees the structured output generated by Parse. Think of Parse as the ground truth layer. Extract is a filter on top of that layer, shaping the parsed content into your schema. If the foundation is wrong, extraction cannot fix it. If the value you need isn't in the Parse output, no amount of schema tweaking will help. You'll need to adjust your Parse configuration first. Common fixes include: * **Enabling agentic mode** for tables with misaligned columns or OCR errors * **Changing table format to HTML** for complex tables with merged cells * **Adding formatting detection** for signatures, change tracking, and hyperlinks * **Setting a document password** for password-protected PDFs Once you confirm the data exists in Parse output, then focus on improving your extraction schema. *** ## Schema Design Principles Your schema is the primary input to the extraction LLM. It determines not just the output structure, but also what the model looks for in the document. ### Use descriptive field names The LLM uses field names as search hints. A field called `po_number` will be matched against text like "PO Number" or "Purchase Order #" in the document. Generic names like `field1` or `data` give the model nothing to work with. ```python theme={null} # Effective: name matches document terminology schema = { "properties": { "invoice_total": {"type": "number"}, "due_date": {"type": "string"}, "bill_to_address": {"type": "string"} } } # Problematic: generic names schema = { "properties": { "amount": {"type": "number"}, "date": {"type": "string"}, "address": {"type": "string"} } } ``` When your document has multiple dates or amounts, specific names help the model distinguish between them. ### Write descriptions that locate values Field descriptions aren't just documentation. The LLM reads them to understand what to extract. A good description tells the model where to look and what distinguishes this field from similar ones. ```python theme={null} schema = { "properties": { "contract_date": { "type": "string", "description": "The date the contract was signed, typically found near the signature block at the end of the document" }, "effective_date": { "type": "string", "description": "The date when the contract terms take effect, usually stated in the first section" }, "expiration_date": { "type": "string", "description": "The date when the contract expires, found in the termination clause" } } } ``` Each date field is now distinguishable because the description explains where it appears and what it represents. ### Constrain values with enums When a field has a known set of possible values, use an enum. This prevents hallucination and ensures consistent output formatting. ```python theme={null} schema = { "properties": { "document_type": { "type": "string", "enum": ["invoice", "receipt", "purchase_order", "credit_memo"], "description": "The type of financial document" }, "payment_status": { "type": "string", "enum": ["paid", "unpaid", "partial", "overdue"], "description": "Current payment status" } } } ``` Without enums, the model might return "Invoice", "INVOICE", "invoice document", or other variations. Enums force a canonical format. ### Keep nesting shallow Deeply nested schemas reduce extraction accuracy. Each level of nesting adds cognitive load for the LLM, increasing the chance of structural errors. ```python theme={null} # Avoid: deeply nested schema = { "properties": { "parties": { "type": "object", "properties": { "buyer": { "type": "object", "properties": { "contact": { "type": "object", "properties": { "address": {"type": "string"} } } } } } } } } # Better: flattened schema = { "properties": { "buyer_name": {"type": "string"}, "buyer_address": {"type": "string"}, "buyer_contact_email": {"type": "string"} } } ``` If you need nested output for your application, extract flat data and restructure it in your code. ### Extract only what exists Extract can only return values that appear in the document. If you request calculated fields or inferred data, the model may hallucinate. This principle extends to any transformation: currency conversion, date formatting, string concatenation. Extract the raw data and transform it yourself. *** ## System Prompts The system prompt provides document-level context. It's where you describe what kind of document this is and how to handle ambiguity. The system prompt should have the following: * **Document type context**: "This is a commercial real estate lease agreement" or "These are bank statements from various institutions" * **Global extraction rules**: "Extract all individual transactions. Exclude summary rows, headers, and running totals." * **Edge case handling**: "Some invoices split line items across pages. Treat these as single items." * **Precision guidance**: "Be thorough and process all pages in the document." Field-specific instructions work better as descriptions because they're attached to the relevant field. For instance, putting "use YYYY-MM-DD format" in the system prompt would apply to all dates, which might not be what you want for different date fields. *** ## Citations Citations link each extracted value back to its source location in the document. Enable them when you need to audit extractions, show users where values came from, or debug extraction accuracy. ```python Python theme={null} result = client.extract.run( input=upload.file_id, instructions={"schema": schema}, settings={"citations": {"enabled": True}} ) ``` ```javascript Node.js theme={null} const result = await client.extract.run({ input: upload.file_id, instructions: { schema }, settings: { citations: { enabled: true } } }); ``` ```go Go theme={null} result, _ := client.Extract.Run(context.Background(), reducto.ExtractRunParams{ ExtractConfig: reducto.ExtractConfigParam{ DocumentURL: reducto.F[reducto.ExtractConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), Schema: reducto.F[interface{}](schema), GenerateCitations: reducto.F(true), }, }) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/extract \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "instructions": {"schema": {...}}, "settings": {"citations": {"enabled": true}} }' ``` When citations are enabled, chunking is automatically disabled because citations require knowing exactly where each piece of content came from. *** ## Related Handle long documents with repeating data. Link values to source locations. Endpoint basics and parameters. # Reducto MCP Server Source: https://docs.reducto.ai/mcp-server Connect AI agents to Reducto via the Model Context Protocol. The Reducto MCP server connects AI agents directly to Reducto. Once installed, agents in Claude Desktop, Claude Code, Codex, Cursor, VS Code, Windsurf, or any other [Model Context Protocol](https://modelcontextprotocol.io) client can classify, parse, extract, split, and edit documents as part of their reasoning loop, without custom integration code. ## Agent quick path If you are setting this up for a coding agent, choose local MCP for local files and hosted MCP for public URLs. ```bash Claude Code theme={null} uvx mcp-server-reducto --login claude mcp add -s user reducto -- uvx mcp-server-reducto ``` ```toml Codex theme={null} [mcp_servers.reducto] command = "uvx" args = ["mcp-server-reducto"] ``` ```json Hosted HTTP theme={null} { "mcpServers": { "reducto": { "type": "http", "url": "https://mcp.reducto.ai/mcp", "headers": { "Authorization": "Bearer your-api-key" } } } } ``` After setup, ask the agent to call: ```text theme={null} parse_document(document_url="https://cdn.reducto.ai/samples/fidelity-example.pdf") ``` For local files, ask the agent to call `upload_file("./document.pdf")` first, then pass the returned `reducto://` URL to `parse_document`. ## What is MCP? The Model Context Protocol is an open standard that lets AI agents call external tools. An MCP server exposes a set of tools (here, Reducto's APIs) that the agent can invoke as part of its reasoning loop. You install the server once, point your client at it, and the agent figures out which tool to call and how to chain results. If you've never used MCP before, the mental model is simple: you describe what you want in plain English, and the agent decides which Reducto tool to call, when to upload a file, and how to pass results between steps. ## When to use the MCP server Use the MCP server when you want an agent to: * Answer questions about a PDF, spreadsheet, or scanned document inside Claude Desktop or another chat client. * Read documents and write code against the result inside Claude Code, Cursor, or VS Code Copilot. * Run multi-step workflows (parse, then extract, then split) without you writing glue code. * Prototype a Reducto integration by letting the agent show you the right API calls, schemas, and response shapes. For non-agent workflows (scripting, batch jobs, CI), use the [Reducto CLI](/cli) or one of the [SDKs](/sdk/python) instead. ## Quick Start Pick one of two options. The hosted server is fastest to set up. The local server lets agents read files directly from your machine. ### Option A: Hosted server (no install) The hosted server runs at `https://mcp.reducto.ai/mcp`. Add this block to your MCP client config and replace `your-api-key` with a key from [studio.reducto.ai/api-keys](https://studio.reducto.ai/api-keys): ```json theme={null} { "mcpServers": { "reducto": { "type": "http", "url": "https://mcp.reducto.ai/mcp", "headers": { "Authorization": "Bearer your-api-key" } } } } ``` The hosted server runs in the cloud, so it cannot read files from your local filesystem. `upload_file` only accepts public URLs through the hosted server. To upload local files directly, use Option B. ### Option B: Local server (runs on your machine) The local server runs via [`uvx`](https://docs.astral.sh/uv/) and accepts local file paths, public URLs, and `reducto://` references. ```bash theme={null} curl -LsSf https://astral.sh/uv/install.sh | sh ``` ```bash theme={null} uvx mcp-server-reducto --login ``` This opens an OAuth device-code flow in your browser. Once approved, your API key is saved to `~/.reducto/config.yaml` with `chmod 600`. If you already use the [Reducto CLI](/cli), you're authenticated already. The MCP server reads from the same credential file. ```json theme={null} { "mcpServers": { "reducto": { "command": "uvx", "args": ["mcp-server-reducto"] } } } ``` No API key in the config: the server reads it from `~/.reducto/config.yaml` automatically. To pass the key explicitly instead (for CI or shared machines), use the `env` field: ```json theme={null} { "mcpServers": { "reducto": { "command": "uvx", "args": ["mcp-server-reducto"], "env": { "REDUCTO_API_KEY": "your-api-key" } } } } ``` ### Hosted vs local | | Hosted (`mcp.reducto.ai`) | Local (`uvx mcp-server-reducto`) | | ------------------------------- | ----------------------------- | -------------------------------- | | Install | None | Python 3.11+, `uvx` | | Auth | Bearer token in headers | Browser login or env var | | Local file uploads | Public URLs only | Local paths supported | | Shares creds with `reducto` CLI | No | Yes | | Best for | Web/Studio users, quick demos | Desktop agents, local workflows | ## Client Setup Most MCP clients use the same JSON shape, but the config file location differs. ### Claude Desktop Edit the config file at `~/Library/Application Support/Claude/claude_desktop_config.json` on macOS, or `%APPDATA%\Claude\claude_desktop_config.json` on Windows: ```json theme={null} { "mcpServers": { "reducto": { "command": "uvx", "args": ["mcp-server-reducto"] } } } ``` Restart Claude Desktop after saving. Reducto appears in the MCP servers list. ### Claude Code Add to your project's `.mcp.json` to share the server with collaborators, or run `claude mcp add -s user reducto -- uvx mcp-server-reducto` to enable it across all projects (stored in `~/.claude.json`): ```json theme={null} { "mcpServers": { "reducto": { "command": "uvx", "args": ["mcp-server-reducto"] } } } ``` ### Codex Codex uses TOML, not JSON. Edit `~/.codex/config.toml` (or a project-scoped `.codex/config.toml`) to add the local server: ```toml theme={null} [mcp_servers.reducto] command = "uvx" args = ["mcp-server-reducto"] ``` To use the hosted server instead, configure it as a Streamable HTTP server. Set `REDUCTO_API_KEY` in your environment first, then add: ```toml theme={null} [mcp_servers.reducto] url = "https://mcp.reducto.ai/mcp" bearer_token_env_var = "REDUCTO_API_KEY" ``` You can also add the local server via the CLI: ```bash theme={null} codex mcp add reducto -- uvx mcp-server-reducto ``` ### Cursor Add to `.cursor/mcp.json` in your project root: ```json theme={null} { "mcpServers": { "reducto": { "command": "uvx", "args": ["mcp-server-reducto"] } } } ``` ### VS Code (Copilot) Add to `.vscode/mcp.json` in your project root: ```json theme={null} { "servers": { "reducto": { "command": "uvx", "args": ["mcp-server-reducto"] } } } ``` ### Windsurf Add to `~/.codeium/windsurf/mcp_config.json`: ```json theme={null} { "mcpServers": { "reducto": { "command": "uvx", "args": ["mcp-server-reducto"] } } } ``` ### HTTP transport (self-hosted) To run the server in HTTP mode for shared use on a single machine: ```bash theme={null} REDUCTO_API_KEY=your-key \ REDUCTO_MCP_TRANSPORT=http \ REDUCTO_MCP_PORT=8000 \ uvx mcp-server-reducto ``` Connect MCP clients to `http://your-host:8000/mcp`. ## Concepts A few small ideas to internalize before reading the tool reference. Most chained operations rely on these. ### Document URL schemes Every `document_url` parameter accepts one of four schemes: | Scheme | Meaning | Where it comes from | | ---------------------- | --------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | | `https://` / `http://` | A public URL Reducto can fetch | You provide it | | `reducto://` | A file in Reducto's temporary storage (24-hour TTL) | Returned by `upload_file` | | `jobid://` | A reference to a previous processing job | Returned by `parse_document`, `extract_data`, `split_document`, `classify_document`, `edit_document` | Other schemes (`s3://`, `file://`, raw paths) fail validation. Use `upload_file` to bring local files or other URLs into Reducto. ### Job chaining with `jobid://` Every processing tool returns a `job_id`. Pass `jobid://` as `document_url` to a follow-up tool to reuse the work, with no re-upload and no re-parse: ```text theme={null} upload_file("./report.pdf") → reducto://abc (file_id) parse_document("reducto://abc") → jobid://xyz123 (job_id) extract_data("jobid://xyz123", ...) → reuses parsed text split_document("jobid://xyz123", ...) → reuses parsed text ``` This is the cheapest and fastest way to run multiple operations against the same document. ### Response shape Every tool returns a JSON object with a consistent set of fields. A representative parse response: ```json theme={null} { "job_id": "xyz123", "duration_seconds": 4.1, "studio_link": "https://studio.reducto.ai/jobs/xyz123", "usage": { "num_pages": 12, "credits": 12 }, "num_blocks": 42, "block_type_counts": { "Text": 30, "Table": 6, "Figure": 6 }, "blocks": [], "next_steps": "Use jobid://xyz123 as document_url for extract_data, split_document, or classify_document." } ``` Common fields: * `job_id`: pass to `get_job` later, or chain via `jobid://`. * `studio_link`: open in [Reducto Studio](https://studio.reducto.ai) to inspect the result visually. * `usage`: pages and credits consumed. * `next_steps`: a hint the server adds describing the recommended follow-up call. ### Large or async results When a result is too large to inline, the response contains a URL-backed payload: ```json theme={null} { "job_id": "xyz123", "result_type": "url", "result_url": "https://...", "result_access_warning": "Result content is URL-backed; call get_job(job_id='xyz123') before reading it." } ``` When `result_type` is `"url"`, call `get_job(job_id=...)` to fetch the materialized result before reading any fields. Do not read `result` directly. ### Response truncation If a response (typically the `blocks` array from `parse_document`) exceeds `REDUCTO_MCP_MAX_RESPONSE_SIZE` (defaults to `50000` characters), the server truncates it and adds: ```json theme={null} { "truncated": true, "truncation_note": "Response truncated (showing 20 of 150 blocks). Use get_job(job_id='xyz123') for full results, or narrow with page_range." } ``` When `truncated` is `true`, call `get_job(job_id=...)` for the full result, or re-run with a narrower `page_range`. ### The `options` escape hatch Most tools accept an `options` parameter for any [Reducto API field](/api-reference) not exposed as a top-level argument. * **Top-level params win.** When a top-level argument and `options` set the same key, the top-level value takes precedence. Nested config dicts get a one-level shallow merge. * **JSON strings are accepted.** Some MCP clients struggle to pass nested JSON. `options`, `schema`, and `categories` therefore each accept either a native object or a JSON-encoded string. Both work identically. ### Page range syntax `page_range` accepts a string with **1-based** page numbers: * `"1-5"`: pages 1 through 5 * `"3,7,10-12"`: pages 3, 7, 10, 11, and 12 ## Tools ### Which tool to use | If you need to... | Use | | ------------------------------------------------------------- | ------------------- | | Look up working SDK or REST examples for any Reducto endpoint | `get_documentation` | | Bring a local file or arbitrary URL into Reducto | `upload_file` | | Get text, tables, figures, and layout from a document | `parse_document` | | Pull specific fields into JSON using a schema | `extract_data` | | Divide a document into named page-range sections | `split_document` | | Categorize a document into one of N labels | `classify_document` | | Fill a form or modify a PDF or DOCX | `edit_document` | | Fetch a full, URL-backed, truncated, or async result | `get_job` | | Inspect recent jobs | `list_jobs` | ### `get_documentation` Returns Reducto SDK and REST documentation, including install commands, auth setup, working code examples, and response shapes for a specific topic. Call this **before** writing any Reducto integration code so the agent works from the current API surface, not training memory. | Name | Type | Required | Description | | ---------- | ------ | -------- | --------------------------------------------------------------------------------------- | | `topic` | string | Yes | One of: `quickstart`, `parse`, `extract`, `split`, `classify`, `edit`, `upload`, `auth` | | `language` | string | No | `node`, `python`, or `http`. Omit to receive all three. | ### `parse_document` Parses a document into structured text, tables, and figures. Supports PDFs, images, spreadsheets, DOCX, PPTX, and 30+ other formats. | Name | Type | Required | Description | | --------------------- | ------------------ | -------- | -------------------------------------------------------------------------------------------------------------------------------------- | | `document_url` | string | Yes | `https://`, `reducto://`, or `jobid://` URL | | `table_output_format` | string | No | `html`, `json`, `md`, `csv`, `dynamic`, `jsonbbox` (defaults to `dynamic`) | | `page_range` | string | No | e.g. `"1-5"` or `"3,7,10-12"` (1-based) | | `chunk_mode` | string | No | `disabled`, `variable`, `section`, `page`. See [parse reference](/api-reference/parse). | | `agentic` | list\[string] | No | Subset of `["text", "table", "figure", "layout"]`. Improves quality on hard documents (handwriting, complex tables) at higher latency. | | `add_page_markers` | bool | No | Insert page-boundary markers in the output | | `return_images` | list\[string] | No | Subset of `["figure", "table", "page"]` to return as images | | `options` | dict / JSON string | No | Any other `ParseOptions` field | ### `extract_data` Extracts structured data from a document using a JSON schema. | Name | Type | Required | Description | | ---------------- | ------------------ | -------- | ------------------------------------------------------------------------------------------------------- | | `document_url` | string | Yes | Pass `jobid://` to skip re-parsing | | `schema` | dict / JSON string | Yes | JSON Schema describing the target fields | | `system_prompt` | string | No | Custom extraction instructions | | `citations` | bool | No | Include source references for each extracted field | | `array_extract` | bool | No | Deprecated flag for repeating items. Prefer `deep_extract`, or model the array directly in your schema. | | `deep_extract` | bool | No | Iterative agentic refinement for harder documents | | `include_images` | bool | No | Add page images to the LLM context | | `page_range` | string | No | Limit pages to process | | `options` | dict / JSON string | No | Any other `ExtractOptions` field | ### `split_document` Segments a document into labeled sections by topic. Returns each section's name, page range, and a confidence score. | Name | Type | Required | Description | | -------------- | ------------------ | -------- | ----------------------------------------------------------------------------- | | `document_url` | string | Yes | Document to split | | `categories` | list / JSON string | Yes | `[{"name": "Disclosures", "description": "Legal and risk disclosures"}, ...]` | | `split_rules` | string | No | Natural-language splitting guidance | | `page_range` | string | No | Limit pages | | `options` | dict / JSON string | No | Any other `SplitOptions` field | ### `classify_document` Categorizes a document into one of the provided categories. | Name | Type | Required | Description | | ------------------- | ------------------ | -------- | ------------------------------------------------------------------------------------ | | `document_url` | string | Yes | Document to classify | | `categories` | list / JSON string | Yes | `[{"category": "invoice", "criteria": ["has billing info", "has line items"]}, ...]` | | `page_range` | string | No | Defaults to first 5 pages | | `document_metadata` | string | No | Additional context to bias classification | ### `edit_document` Fills forms or modifies a document (PDF or DOCX). Returns a download URL for the edited file plus a `form_schema` describing the fields it found. The first edit on a new form returns a `form_schema` and a hint: ```json theme={null} { "document_url": "https://.../edited.pdf", "form_schema": {}, "form_schema_note": "Cache this form_schema and pass it via options.form_schema for repeated edits." } ``` For repeated edits to the same form template, cache `form_schema` and pass it back via `options.form_schema` to skip re-detection. | Name | Type | Required | Description | | ------------------- | ------------------ | -------- | ----------------------------------------- | | `document_url` | string | Yes | Document to edit | | `edit_instructions` | string | Yes | Natural-language instructions | | `options` | dict / JSON string | No | `edit_options`, `form_schema`, `priority` | ### `upload_file` Uploads a document to Reducto's temporary storage (24-hour TTL) and returns a `reducto://` URL for use in other tools. Accepts: * **Local file paths**: `./report.pdf`, `/Users/me/docs/x.pdf`, `~/inbox/y.pdf` (local server only). * **Public URLs**: `https://example.com/report.pdf` (downloaded server-side, then uploaded). The hosted server (`mcp.reducto.ai`) does not support local paths. Use a public URL or run the server locally. | Name | Type | Required | Description | | ---------- | ------ | -------- | ----------------------------- | | `file_url` | string | Yes | Local file path or public URL | ### `get_job` Returns the status and result of a previous processing job. Use this to fetch URL-backed results, full content for truncated responses, or to poll an async job. | Name | Type | Required | Description | | -------- | ------ | -------- | -------------------------------------- | | `job_id` | string | Yes | Job ID returned by any processing tool | ### `list_jobs` Returns recent processing jobs. | Name | Type | Required | Description | | ------- | ---- | -------- | ------------------------------------- | | `limit` | int | No | Max jobs to return (defaults to `10`) | ## Usage Patterns ### Basic parse > "Parse this PDF and show me the content." The agent calls `parse_document(document_url="https://example.com/report.pdf")`. ### Extract with a schema > "Extract the invoice number, date, and total from this document." The agent calls `extract_data` with a JSON schema describing the three fields. ### End-to-end chain A full workflow using `upload_file`, `parse_document`, and chained `extract_data`: ```text theme={null} 1. upload_file("./contracts/q4-msa.pdf") → { "file_id": "reducto://abc", "next_steps": "Pass reducto://abc as document_url ..." } 2. parse_document("reducto://abc", agentic=["text"]) → { "job_id": "xyz123", "studio_link": "...", "next_steps": "Use jobid://xyz123 ..." } 3. extract_data( "jobid://xyz123", schema={ "type": "object", "properties": { "effective_date": {"type": "string"}, "parties": {"type": "array", "items": {"type": "string"}}, "termination_clauses": {"type": "array", "items": {"type": "string"}} }, "required": ["effective_date", "parties"] }, citations=True ) → structured fields, with citations ``` The same `jobid://xyz123` can also be reused by `split_document` or `classify_document` without re-parsing. ### Triage a mixed document set > "Classify each of these PDFs as `invoice`, `contract`, or `lab_report`, then run the right extraction schema for each." The agent calls `classify_document` per file, then routes each to `extract_data` with a category-specific schema. ### Fill a recurring form > "Fill out a W-9 for each of these vendors using the data in `vendors.json`." The first call to `edit_document` returns a `form_schema`. Subsequent calls pass the cached `form_schema` via `options` to skip re-detection. ### Handle large or async results If a response includes `result_type: "url"`, call `get_job` before reading fields: ```text theme={null} parse_document("reducto://big-doc") → { "job_id": "j1", "result_type": "url", "result_url": "...", "result_access_warning": "Result content is URL-backed; call get_job(job_id='j1') before reading it." } get_job(job_id="j1") → full materialized result ``` If a response includes `truncated: true`, either call `get_job(job_id=...)` for the full result or re-run with a narrower `page_range`. ## Authentication The server resolves API keys in this order: 1. **`REDUCTO_API_KEY` environment variable** (highest priority). Useful for CI or explicit config. 2. **`~/.reducto/config.yaml`**, the shared credential store with the Reducto CLI. Three ways to authenticate: ```bash theme={null} # Browser login (recommended) uvx mcp-server-reducto --login # Reuse an existing CLI session reducto login # Set the env var directly export REDUCTO_API_KEY=your-key ``` `--login` runs an OAuth device-code flow: it prints a code, opens your browser to the verification page, and waits for you to approve. Once approved, the key is written to `~/.reducto/config.yaml` with `chmod 600`. Use `--login --force` to replace an existing saved key. The hosted server (`mcp.reducto.ai`) does not use this flow. Pass your key as `Authorization: Bearer ` in your client config instead. ## Environment Variables | Variable | Required | Default | Description | | ------------------------------- | -------- | ----------------------------- | -------------------------------------------------- | | `REDUCTO_API_KEY` | No\* | none | API key (or authenticate via `--login`) | | `REDUCTO_BASE_URL` | No | `https://platform.reducto.ai` | Override the API base URL (EU region, on-prem) | | `REDUCTO_MCP_MAX_RESPONSE_SIZE` | No | `50000` | Response truncation threshold in characters | | `REDUCTO_MCP_TIMEOUT` | No | `300` | Request timeout in seconds | | `REDUCTO_MCP_TRANSPORT` | No | `stdio` | Transport mode: `stdio` or `http` | | `REDUCTO_MCP_PORT` | No | `8000` | Port when using HTTP transport | | `REDUCTO_TELEMETRY` | No | `1` | Set to `0` to opt out of anonymous usage telemetry | \*Required only if you have not run `mcp-server-reducto --login` or `reducto login`. ## Telemetry The MCP server sends a small amount of anonymous usage telemetry to PostHog so the team can prioritize improvements. Collected: * Lifecycle events: `mcp.installed` (once per machine, on first run) and `mcp.start` (once per server boot). * Per-tool invocation events: `tool..invoked` with `tool` name, `status` (`ok`, `error`, or `exception`), and `latency_ms`. * Environment fingerprint on every event: client name and version, transport (`stdio` or `hosted`), Python version, OS platform. Never collected: * Tool arguments (document URLs, file IDs, schemas, prompts, page ranges). * Tool responses or document content. * API keys. The user identifier on each event is `sha256(api_key)[:16]`, which is one-way. To opt out, set `REDUCTO_TELEMETRY=0` in the environment that runs the MCP server. ## Debugging ### MCP Inspector Test the server interactively: ```bash theme={null} npx @modelcontextprotocol/inspector -- uvx mcp-server-reducto ``` This opens a web UI where you can discover tools, call them, and inspect responses. Requires prior authentication via `--login`. ### Studio links Most parse and extract responses include a `studio_link`. Open it to inspect the same job interactively in [Reducto Studio](https://studio.reducto.ai). This is the fastest way to compare parser output side by side with the original document. ### Logs The server logs to stderr. Stdout is reserved for the MCP transport. Set `LOG_LEVEL=DEBUG` for verbose output. ## Troubleshooting **Problem**: The server starts but every tool call fails with an authentication error. **Solution**: Run `mcp-server-reducto --login`, or set `REDUCTO_API_KEY` in your client config's `env` block. **Problem**: A tool call fails with a validation error about `document_url`. **Solution**: `document_url` must start with `https://`, `http://`, `reducto://`, or `jobid://`. For local files, call `upload_file` first to get a `reducto://` URL. **Problem**: You passed a local path to `upload_file` while connected to `mcp.reducto.ai`. **Solution**: Switch to the local server (Option B above), or pass a public URL. **Problem**: After editing your client config, no Reducto tools show up. **Solution**: Restart your MCP client. Then test the server directly by running `uvx mcp-server-reducto` in your terminal to confirm it starts cleanly. **Problem**: A long-running parse or extract returns a timeout error. **Solution**: Increase `REDUCTO_MCP_TIMEOUT` (defaults to `300` seconds), or narrow the request with `page_range`. **Problem**: A response references content but the `result` field is empty or missing. **Solution**: Check for `result_type: "url"` or `truncated: true`. In either case, call `get_job(job_id=...)` to fetch the full materialized result. ## Next Steps Run parse, extract, and edit from your terminal. A self-contained API reference designed for AI coding agents. How parse turns documents into structured text, tables, and figures. Schema-driven structured data extraction. # On-premise changelog Source: https://docs.reducto.ai/onprem/changelog Release notes for on-premise deployments of Reducto * feat: Native DOCX comment extraction now returns comments anchored to the relevant document blocks. * feat: `/edit` can synthesize form fields for questionnaires that do not include them. * feat: v3 parse supports customer-managed encryption with a KMS key (alpha). * fix: Improve citation handling and page-range support for `/extract` requests. * fix: Resolve failures in document ordering and checkbox detection. * fix: Improve handling of rotated PDFs in embed and parse results. * feat: Improve reliability and performance for large spreadsheet processing. * feat: Add asynchronous job deletion endpoints. * feat: Improve extraction quality and reliability for large documents. * fix: `/edit`: preserve AcroForm through overflow-page assembly, render filled text at the correct size, and write radio button values by name. * fix: Correct embedded text-layer placement on rotated (90/270) PDF pages. * fix: Honor `/extract` page-range settings and improve citation accuracy. * fix: Improve reliability when embedding or converting large PDFs. * fix: Handle corrupt TIFF palettes in image load paths. * perf: Reduce memory usage when processing dense and scanned documents. * feat: Self-hosted Office conversion for DOCX and PPTX files (alpha). * feat: `REDUCTO_AGENTIC_URL` to point the agentic model at an on-prem endpoint. * fix: Improve text extraction in FIPS-compatible environments and from documents with complex fonts. * fix: Improve isolation of stored job results across organizations. * fix: Improve stability during layout inference and document processing. * fix: Respect EXIF orientation on image inputs and drive scanned-page render DPI off the embedded scan resolution. * fix: Reconcile AcroForm fields with filled widgets in `/edit` so forms render correctly in Preview/PDFKit. * fix: Improve resilience to temporary LLM provider failures. * perf: Reduce peak memory use across document, spreadsheet, and OCR processing. * perf: Reduce peak memory use when processing large documents. * perf: Improve PDF rendering and OCR performance. * fix: Preserve inline markup when merging table rows across page breaks. * fix: Improve reliability for text-heavy PDFs and Office document conversion. * fix: Improve the completeness and clarity of public OpenAPI schemas. * feat: Console logging is on by default, with a `NO_LOG` override. * feat: Return detected languages in OCR data when available. * perf: Bound concurrent agentic image memory, preventing worker OOMs on heavy documents. * fix: Resolve spurious `DocumentCorruptError` failures on DOCX files that reference external templates. * feat: Added `/openapi-onprem-full.json` for on-prem deployments. This schema includes the full HTTP pod API surface, including routes hidden from the hosted public API reference. * feat: Added on-prem deployment controls and worker/runtime reliability improvements for supported customer environments. * fix: Improved job finalization, result streaming, document rendering, and customer-facing error handling for better stability under load. * docs: Added RSS-ready metadata for on-prem changelog entries. The generated feed is available at [onprem-changelog-rss.reducto.ai/rss.xml](https://onprem-changelog-rss.reducto.ai/rss.xml). * feat: Add `num_pages` field to the `/classify` API response. * feat: Add `max` mode for agentic tables for higher-fidelity table extraction. * fix: Fail jobs on OCR transient errors instead of silently returning an empty response. * fix: More accurate HTTP error status codes so client-side validation failures return 4xx instead of 5xx. * fix: Multipart S3 upload for result payloads over 100MB, with a new `OversizedResultError` for limit cases. * chore: Removed legacy sandbox setup that is no longer used by supported on-prem workflows. * fix: Improved `/jobs` source redaction and added release validation for `/version` interpolation. * fix: Strengthened authentication header handling for webhook-related endpoints. * docs: Added clearer guidance for the on-prem shared security model, securing Reducto deployments, and observability access controls. See [On-prem security model](/onprem/security_model), [Securing Reducto](/onprem/securing_reducto), and [Observability & Monitoring](/onprem/observability). * feat: Extend the SIGUSR2 stack-trace dumper to the DB-queue worker pods (`reducto-worker`, `reducto-priority-worker`, `reducto-gpu-worker`). Previously the handler was only installed on HTTP and streaq worker pods; this completes coverage so `kill -s USR2 ` produces a stderr stack dump on every Reducto pod type. See [Observability](/onprem/observability#pod-stack-trace-dumps-sigusr2). * perf: Higher embed pool concurrency for improved throughput * fix: Pre-resolve OCR before vision-model citations with stricter containment threshold for improved accuracy * fix: Preserve 5xx semantics for PDF text extraction subprocess crashes * perf: Faster spreadsheet processing + fix images on text-empty sheets * perf: Higher PDF render pool concurrency with eager background respawn * fix: Replace non-ASCII chars that break obfuscation build * feat: Azure Vision OCR client migrated to the async aio SDK with hard wall-clock cancellation via `asyncio.wait_for`. The new `AZURE_VISION_TOTAL_CALL_TIMEOUT` (default 45s) closes the underlying aiohttp socket when it fires, so workers no longer wedge on slow Azure responses. Per-call connect/read timeouts and SDK retries remain configurable via `AZURE_VISION_CONNECTION_TIMEOUT`, `AZURE_VISION_READ_TIMEOUT`, and `AZURE_VISION_MAX_RETRIES`. See [LLM options → Azure Vision](/onprem/llm_options#azure-vision-ocr) for the full env var reference. * feat: SIGUSR2 stack-trace dumper installed in HTTP gunicorn workers and streaq worker processes. `kubectl exec` into a pod and run `kill -s USR2 ` to print every thread's stack to stderr (visible via `kubectl logs`) for diagnosing hung tasks, wedged event loops, or contended thread pools. See [Observability](/onprem/observability#pod-stack-trace-dumps-sigusr2). * feat: Kubernetes liveness watchdog + exec probe for `reducto-worker`, `reducto-priority-worker`, and `reducto-gpu-worker` pods. A pod is only restarted when the in-process watchdog reports an in-flight task running longer than `WORKER_STUCK_TASK_THRESHOLD_SEC` (default 1800s) or the watchdog itself stops heartbeating (event loop wedged). Idle workers are never restarted. Configurable via `worker.livenessProbe.*` Helm values. See [Operations](/onprem/operations#worker-liveness-probe). * feat: Hard-killable subprocess pool with per-renderer timeouts for PDF renders * feat: Hard-killable subprocess pool for PDF flatten + embed timeouts * fix: Skip zero-dimension crops in fine-grained citations * fix: Killable timeout for stuck PDF text extraction * fix: HTTP worker graceful shutdown timeout set to 120s * fix: Strip control characters from extracted strings * fix: Preserve overlay text selectability when embedding into existing PDFs * fix: Protect figures from signature detection * fix: Return 4xx for images with extreme aspect ratios * refactor: More reliable PDF widget ingestion in `/edit` * fix: Azure Vision OCR SDK retry behavior on-prem * perf: Spreadsheet memory optimizations * perf: Faster OCR cropping for table prediction * perf: Faster PDF render path * perf: Faster Google Cloud Vision response and rotation hint processing * feat: Make all fields in auto-generated extraction schemas optional * fix: Memory leak in PDF processing * fix: Avoid PDF library hang on dense-tiling-pattern PDFs via fallback reader * fix: Corrupt zip-based documents now return 400 instead of 500 * fix: Preserve citations on scalar-only schema adherence corrections * fix: Cap spreadsheet agent instructions to avoid input token limits * fix: Correct ordering of table enrichment relative to markup application * fix: Clean up spurious 5xx on cancels and HTML to PDF hangs * feat: New Azure Vision OCR strategy for on-prem deployments * feat: Datadog dashboard for on-prem k8s deployments * feat: Expose `keep_line_breaks` in V3 `settings.alpha` * feat: Optimized inference backend for table router with safety wrapper * fix: Faster usage logging via append-only buffer to reduce DB contention * fix: Tolerate quoted-printable colspan in HTML parsing * fix: Stabilize table router builds with pinned model export version * chore: Bound embed feature with per-call wall-clock budget * feat: Dynamic rendering for small-font metadata extraction * feat: Remove Adobe-added watermark layers from documents that have them * feat: Default `fast_embed_pdf_metadata` to True * fix: Preserve PDF optional content properties through page extraction for watermark layer stripping * fix: Extract always returns V3 shape when citations are missing * fix: Extract citations no longer dropped when `page_range` skips page 1 * fix: Clean up downloaded PDF temp files on batch failure * refactor: LLM client dependency bump for CVE remediation * refactor: Azure Vision array OCR failover with retryable errors only * chore: Patch critical and high CVEs in Docker images * chore: Return S3 links instead of raw JSON to avoid response size limits * fix: Error attribution added to `GET /job` endpoint * chore: Pass prompts as files instead of args to avoid bytes overflow * fix: Skip parse-ID overlay in `/edit` when `form_schema` is provided * fix: Filter out uninitialized providers in model config resolution * fix: Page range validation parity in parse pipeline * refactor: Classify enhancements * feat: Azure Vision multi-endpoint failover with load balancing * feat: Enhanced image-to-PDF conversion * feat: Per-chunk embed PDF in pipeline * feat: Run LLM enrichments in parallelized DAG for lower latency * fix: Correct argument handling in offline entrypoint for org/job\_id * fix: Hybrid OCR prefers metadata over garbled OCR when text is reordered * refactor: Remove unused models from on-prem image builds for slim images * fix: `max_completion_tokens` / `check_schema` escape hatch for reasoning models * fix: Office document conversion with CJK fonts * fix: Deliver webhook after `persist_results` to avoid S3 race condition * chore: Upgrade LLM client dependencies * feat: New `force_simple_page` config option * feat: Enable `fast_flatten` for legacy early flatten in pipeline orchestration * fix: `max_completion_tokens` schema validation for reasoning vs. non-reasoning models * fix: `max_tokens` and other arg handling for Azure LLM inference * feat: Increased sheet processing timeout from 600s to 900s * feat: Fast PDF flatten, selective rasterization replaces full-document rasterization * fix: Validation for custom experimental options on on-prem * fix: Reduce DB lock contention on batch completion * fix: Equation enrichment correctly preserves line/word offsets * feat: New fast embed via env settings * feat: Configurable timeouts for OCR and embed text metadata steps * feat: Per-page billing feature breakdown in parse response * feat: Auto formatting per page * perf: 10x faster PDF text overlay rendering * perf: Parallelize figure classification with higher LLM concurrency * perf: Optimized batch result aggregation * fix: CSV parsing column count uses max row width; no longer drops columns when first row is narrower * fix: OCR PDF no longer drops pages on multi-chunk documents * fix: Allow None in override schema for bool fields * fix: Correct `num_pages` reporting on jobs * fix: Image conversion error handling * fix: Strip markup noise before language detection to avoid false unknowns * fix: Avoid page overlaps in `/split` endpoint output * fix: Guard against errors in layout postprocess * fix: Reclassify corrupt PDF annotation failures as proper 4xx status codes * chore: Raise default overflow chunk limit to 500 * perf: Parallelize KV fallback to prevent task deadline breaches * feat: Native DOCX XML parsing pipeline (alpha) with `.pages` support * feat: HEIC image support and section-based chunking for Numbers files * feat: Formatting and images support for Numbers files * feat: Extract model and internal prompt overrides for v2/v3 configurations * feat: YAML extract and citations models added to Helm chart for GPU deployments * feat: KV repetition detection with Gemini fallback replacing repetition\_penalty * feat: OTEL pipeline routing and K8s metrics collection * fix: Auth for chained jobs * fix: Required fields on extraction schema * fix: Equation detection TypeError from tuple/list concatenation * fix: Move sync blocking calls off the event loop in HTTP handlers * fix: Use encrypted DB URL and disable k8s metrics in on-prem environments * fix: Fail fast on hung PDF renders * fix: Parallelize S3 batch result loading for faster retrieval * perf: Optimized layout postprocessing * perf: Optimized hybrid OCR processing * refactor: Graceful fallbacks for XML conversion issues * chore: Cron retries increased from 1 to 2 for improved reliability * chore: Send total credits for on-prem customers * feat: OCR-based table citations for deep extract * feat: Add API key prefix filter parameter for /jobs endpoint * fix: On-prem presigned URL upload path mismatch * fix: new layout postprocessing * fix: Empty OCR fallback handling * fix: Python version in sandbox runtime * chore: Upgraded enhanced figure summary models * fix: on-prem deployments start without Redis configured * feat: On-prem usage logging for customer tracking * feat: Schemaless deep extract * feat: Granular citations in deep extract * feat: Deep extract available for on-prem deployments * feat: Spreadsheet sheet-name page\_range support * feat: Suppress citation content feature flag for extract * fix: Garbled DOCX for change tracking * fix: Recursive render to handle nested tables for edit * fix: Rotation in embed metadata * fix: GCP API key requirement for on-prem * fix: Remove libpq options from DB connect\_args for RDS Proxy compatibility * fix: Lock timeout batches and non-locking last-batch check * fix: Background threads no longer block main processing * fix: Transient classify inference issues * perf: Merge tables speedup from O(N^2) to O(N) * perf: Retry improvements to avoid double work * refactor: Decompose batch pipeline into composable phases * chore: Upgrade Gemini models * fix: Middleware context propagation and ordering for proper trace handling * fix: Cron job retry logic and syntax improvements * fix: HTTP startup patched for hashlib.md5 in FIPS environments (GCS support) * perf: Decaying timeout for retries with improved retry\_on\_timeout behavior * perf: Updated timeout and max batch configurations * feat: Chunk overlap configuration for including text context from previous/next chunks * feat: summarize\_all\_figures option in v3 alpha config * feat: Deep extraction optimizations for improved structured data quality * fix: Lazy loading for HTTP/worker modules to avoid unnecessary dependency imports * fix: Guard against empty document\_url list in pipeline and split endpoints * fix: Cron job improvements * fix: More reliable page orientation detection * fix: Exception handling on deep extraction completion * refactor: Deep extract sandbox image * chore: Upgraded Anthropic models * chore: New table detection model with improved accuracy * fix: Add bounds check for page index in PDF text embedding to prevent IndexError crashes * fix: Skip cover pages for PDF portfolios during attachment concatenation * fix: Fallback to original pages when portfolio has no PDF attachments * fix: Fast-fail URL download on non-success HTTP status codes * fix: Random checkbox YOLO crash when Conv has no batch normalization * fix: HuggingFace model downloads for builds * fix: Lazily import probing modules to avoid Modal dependency in on-prem * feat: Custom agentic layout postprocessing * feat: Routing for parse batches * feat: Classify concurrency improvements * refactor: Set reducto environment to `onprem` by default * chore: Upgrade pytorch and torchvision dependencies * fix: Detect visual redlines (colored strikethrough/underline) in DOCX change tracking * fix: Use min instead of max for checkbox detection * fix: CSV parsing truncation and scientific notation for large integers * fix: Recover in-progress batches alongside pending ones * feat: Fallback to Gemini Flash for improved reliability * feat: Intelligent ordering fallbacks * feat: Schema adherence model updates * feat: Add ONNX model integrity verification with forced fresh model download * refactor: Database lock timeout for sync DB engine * feat: New OCR recognition model for Apple deployments * fix: Recover in-progress batches alongside pending ones * fix: Add ONNX model integrity verification and force fresh model download * feat: Classify endpoint with parallelized Gemini Flash Lite probes for document classification * feat: Add bucket\_name as alpha option in v3 parse config * feat: Enable bucket & KMS ARN override for hybrid VPC deployments * feat: Auto region routing for Gemini models * feat: Native office conversion alpha flag in v3 config * feat: Inference helm charts and kv-base routing * feat: Enable flatten for edit endpoint * feat: Improved models for standard figure summary * feat: Add dimension limit handling for AWS environments * fix: Memory leaks and file descriptor leaks in PIL Image handling across OCR and processing pipelines * fix: N-squared completion pattern in batch processing for significantly improved performance at scale * fix: Race condition in parse completion job processing * fix: Argument order bug in pdftext multiprocessing extraction * fix: V3 config fixes for on-prem deployments * fix: Initialize empty sheets to prevent errors on blank spreadsheets * fix: Force resize to fit AWS dimension limits for large documents * fix: Image conversion failures now return proper 415 error instead of 500 * fix: PyPDFForm version update to resolve form filling bug * fix: Classify endpoint fixes for improved reliability * fix: Offset\_in\_chunk calculation for empty blocks * fix: Exclude veryHidden sheets when exclude\_hidden\_sheets is enabled * fix: Checkbox detection bug * fix: Prioritize S3/BUCKET over GCS when both GCP\_PROJECT\_ID and BUCKET are set * fix: cron.py Kubernetes usage * fix: Distributed traces with LOGFIRE\_DISTRIBUTED\_TRACING * fix: Temperature 0.1 for promptable layout for more deterministic results * fix: local-full Dockerfile fix by adding gcc and python3-dev to apt install * perf: Hydrate SharedBatchWorker.process\_org\_batch before ThreadPoolExecutor for improved concurrency * refactor: Remove enhanced enrich tables, default to same model for simpler table processing * chore: Upgrade table models * feat: V3 config overrides for v2-only and on-prem-only settings * feat: List item support and chunk offsets in blocks for improved extraction * fix: handle\_required\_fields not adding missing fields to array items in extraction * fix: Page marker blocks now include correct page and original\_page values * fix: Multi-batch recovery when job processing is interrupted * fix: Division by zero error for images with corrupted EXIF data * fix: Restore V2 OCR defaults (highres OCR system) in V3 on-prem config for consistent behavior * chore: Bookworm image build configuration for CD pipeline * feat: Bookworm Dockerfile variant for improved on-prem DOCX→PDF conversion reliability * fix: DOCX→PDF conversion using LibreOffice from Trixie backports for improved reliability * feat: Super-agent integration into /extract pipeline for improved structured data extraction * fix: Traceparent propagation for API requests * perf: Per-image table predictions for better performance * feat: Hybrid VPC routing based on header with default AU/EU/US regions * feat: Add docx fallbacks for malformed XML and OOXML-format .doc files * feat: Change default presigned URL expiration from 1 hour to 12 hours * fix: Table edit pattern improvements and preferred edit model changes * feat: Schema adherence for required keys in extraction * feat: Improved table edit granularity * fix: Properly propagate password errors for password-protected PDFs * fix: Anthropic Bedrock on-prem edit calls * feat: Add raw XML repair fallback for malformed docx files * fix: Local parse hanging for multi-batch documents * feat: Schema Optimization Agent for improved extraction accuracy * fix: Hyperlinks being dropped when OCR extraction mode is enabled * feat: Add line level offsets when config is enabled * feat: Intelligent Ordering Model API integration * feat: Add document\_password support to pipeline API for password-protected documents * feat: Implement character-level DOCX change tracking * fix: Hidden rows and columns handling for spreadsheets * feat: Cloudflare R2 Storage Class support * fix: Settings Overrides for streamlined API config/env var customization * chore: inference parallelization * feat: OCR word and line rotation data propagation * fix: layout prediction improvements * feat: extract schema adherence * fix: empty table model output * refactor: Document fetching logic * feat: Updated ordering model * feat(settings): more streamlined customization for models and prompts via API configuration / env variables * feat: Customizable models for AWS Bedrock using environment variables * refactor: Default models updated for AWS Bedrock to `us.anthropic.claude-sonnet-4-5-20250929-v1:0` * feat: support more edge case custom file mimetypes * fix: table chunking * fix: make enrich tables more robust * refactor: optimize some DB transactions to not be left open too long * feat: Add signatures as a formatting option in v3 config * feat: extract schema adherence * refactor: optimize enrich tables latency * feat: new hybrid OCR implementation * feat: add force file mimetype to extension config option * feat: env var based customization for local KV prompt/model * feat: Add priority-based worker routing to skip shared/dedicated workers when priority is not set * chore: adding latency sensitive for fast mode in Spreadsheet Agent * feat: add OpenAI Responses LLM Provider * feat: Allow direct DataDog Tracing with Beta Headers and Logfire Service name handling * fix: table block chunking * fix: Chainguard image dependencies * fix: md5 for FIPS environments * fix: numbers file parsing * fix: allow invalid surrogates when encoding * feat: Add logfire gauge metrics for K8s queue lengths * feat: Add Azure Blob Storage authentication support for private endpoints * fix: race condition with in progress batch -> job completion enqueue * fix: logfire logging if logfire token is set * fix: embed pdf metadata * fix: persist results before webhook * fix: update cancel\_all and wipe endpoints for on-prem and secure them correctly * refactor: cron cleanup function + running frequency * fix: Chainguard image dependency issues * fix: PgDog Helm Chart application version configuration for on-premise deployments * feat: V3 API config with improved spreadsheet response format and citations support * feat: Enhanced table block chunking for better extraction of large tables * feat: Agent-in-the-loop (AITL) extraction with generalizable configuration for multiple fields * feat: Spreadsheet figure summary support for better data visualization * feat: LLM provider preference configuration for v3 API (specify OpenAI, Anthropic, Google, etc.) * feat: Helm chart PgDog dependency for PostgreSQL monitoring * feat: Affinity and topologySpreadConstraints support in Helm charts for advanced pod scheduling * fix: OCR system handling in v3 config * fix: Race condition for single batch jobs * fix: Webhook delivery on Kubernetes environments * fix: Parse job update batching for improved database performance * fix: DOCX timeout increased for large document processing * fix: Table merging with XML parsing improvements * fix: Underline/strikethrough character threshold adjustments * chore: Datadog integration for enhanced monitoring * feat: Reduce PDF output size by avoiding text layer rasterization * feat: Custom chunking response format support * fix: AITL configuration handling for proper field validation * feat: Agent-in-the-loop (AITL) documentation exposed and configuration updated to handle multiple fields * feat: Hyperlink extraction support in PDF parsing - preserves document links in output * feat: PostgreSQL Helm dependency migrated to OCI registry for better reliability * feat: Spreadsheet figure summary generation for visual data extraction * fix: OCR system switching for v3 config * fix: Change tracking for accurate document diff detection * feat: Helm charts now support affinity and topologySpreadConstraints for advanced Kubernetes pod placement control * feat: Table merging heuristics improved * fix: Webhook delivery on Kubernetes fixed for reliable notification * fix: Underline and strikethrough detection threshold adjusted for better accuracy * fix: Safe fill implementation used everywhere in PDF form filling * chore: Datadog monitoring integration * feat: V3 API config support - new configuration format for improved extraction control * feat: Naive table merging for bulk processing with better cross-page detection * feat: Figure summary enhancements and configuration via API * feat: LLM provider preference support in v3 config * feat: Tool use support for Anthropic provider * feat: /openapi.json and /openapi-legacy.json endpoints for API schema access * feat: Split implementation improvements * fix: Database transaction handling in Kubernetes - don't keep transactions open * fix: Experimental table citations now default to true in v3 * fix: Extract confidence concurrency handling * chore: Figure summarization adjusted for more thorough output * fix: Helm chart labels for retry stale jobs cronjob * fix: Build configuration cleanup * feat: Support for custom extract models via LLM service, enabling on-premise model configurations * fix: PDF form dropdown filling improvements with proper context and option handling * fix: Excel column to string conversion using openpyxl * feat: New /jobs endpoint with cursor-based pagination for efficient job listing and filtering * feat: New PDF edit flow using parse pipeline for improved form filling accuracy and performance * feat: Schema-less extraction generation - automatically infer extraction schemas when not provided * feat: Enhanced table merging across pages in HTML documents with improved row/column detection * feat: Improved spreadsheet agent with citations support and performance optimizations * feat: Parallelized batch results loading from storage for faster retrieval * fix: Multi-page TIFF and JPEG handling for proper page extraction * fix: Password-protected landscape PDF processing * fix: Text overlay visibility issues during edit flow * fix: Spreadsheet agent formatting values in preview mode * chore: Docker base image upgraded to Debian Trixie for better security and compatibility * chore: Enhanced mode set as default for better quality * feat: Priority handling for time-sensitive extraction requests with improved page mapping reasoning * feat: Improved Vertex AI Gemini region configuration * fix: Array extract error handling - prevents crashes from malformed LLM output * fix: Better concurrency management for key-value extraction * fix: Split configuration handling improvements * chore: OpenAI API retry logic for handling slow responses * chore: Exponential backoff for split operations * feat: Improved layout inference with reduced latency * feat: PDF edit overlay improvements using OCR-B font for better text rendering * fix: Timeout configuration improvements for long-running operations * fix: Worker stability improvements and bug fixes * feat: Spreadsheet extraction agent enhancements for better cell and table detection * fix: Citation formatting improvements across extraction outputs * feat: Cross-page table merging improvements with naive row merging implementation * fix: Performance optimizations for large document processing * feat: Enhanced extraction pipeline with improved data handling * fix: Error handling improvements throughout the system * feat: GCP Workload Identity support for Google Cloud deployments * feat: AWS region override configuration for flexible cloud deployments * fix: Worker stability enhancements * feat: Prometheus alerting integration for monitoring * fix: Reliability enhancements for long-running jobs * feat: opt-in or opt-out to send billing usage to license server * feat: block OpenAI invocation with BLOCK\_OPENAI env var * feat: signature detection * fix: helm chart template rendering * fix: update figure summarization to correctly override default prompt when user wants to override * refactor: update equations detection to use on premise-provided LLMs * feat: configurable S3 endpoint url * feat: character-level support for azure in hybrid mode * feat: support for docx comments * feat: split support with gemini on vertex ai * refactor: updated LLM service with vision/text * fix: ensure formatted text (i.e. underline, strikethroughs) is not subsumed by key value detection * feat: secret management in helm chart * feat: add .msg file support * feat: HEIC file format support for image processing * feat: character-level OCR detection for strikethrough and underline formatting * feat: parallelize and optimize PDF metadata embedding for improved performance * fix: add locks to prevent race conditions * fix: timeout handling for DOCX to PDF conversion with proper 400 status codes * feat: configurable S3 SSL options for boto * feat: /billing-usage API for exporting usage in air-gapped deployments * feat: Support for Google Cloud Storage gs\:// document url * feat: timeout and fail jobs and batches when queued for GLOBAL\_QUEUE\_TIMEOUT\_SEC * feat: BackendConfig support in Helm Chart for GCP * feat: generate extract schema if no schema was provided * feat: adding form schema for edit documentation * feat: improve cold starts * fix: fine-grained citation fixes * feat: DOCX improvements * feat: added schema token limits * feat: add customizations to auth via environment variables * feat: faster model inference optimizations * fix: OCR image resizing improvements * feat: implement fault-tolerant webhook delivery * feat: fix table headers for html parsing * feat: add secret metadata parameter to /job/ endpoint * feat: include config when include\_metadata is enabled for job endpoint * feat: adding sheet color to output * feat: clean up refs in extract output * feat: excel table color mapping implementation * feat: enhance merge tables * feat: table feedback loop using the enrich table flag * feat: add litellm proxy model for 'best' * feat: long-polling with timeout (seconds) query param for `/job/{job_id}` * fix: job type and add duration field in `/jobs` endpoint * fix: Preserve all decimals in md tables * feat: support for GCP * feat: presentation detection and kv-disabling * fix: Strike underline tuning * feat: implement rtf * fix: Fix offsets for tables extracted from excel sheets * feat: add optional confidence fields to OCRWord and OCRLine * feat: add source in `/jobs` * feat: initial change detection implementation * docs: clarify Excel citation coordinate system differences * feat: Integrate spreadsheet agent for extract * feat: Convert images to pdf for pdf\_url * Fix: Allow Gemini to output `` and `` fields for key-value * chore: update textract quota * feat: option for multiplatform builds for onprem * docs: Add Model Governance Policy to security section * fix: Split regex fix for subcategory * fix: Add retries to spreadsheet agent * fix: Merging splits in the new format * fix: Merge array\_extract citations based on extract results * Add chart extraction documentation page * feat: ship both small/large models in built images * feat: allow changing default use\_gpu\_ocr config value based on env var * feat: handle multipart/form-data content type errors on /split endpoint * Latency fix: Agentic unicode changes * fix: sanitize html file upload path to s3 * Add strict typing for SplitResult.splits * feat: Helm chart and values for GPU-based OCR deployment * feat: enhanced DOCX change tracking with improved underline detection and formatting accuracy * fix: optimized model server initialization to reduce startup time and improve processing performance * fix: resolved document conversion hangs caused by separate executor processes for improved reliability * feat: added cancel\_all endpoint for on-prem deployments to cancel all running jobs at once * feat: enhanced extraction with schema key normalization and improved page range references in citations * feat: added global timeout overrides for better performance control and reliability * fix: resolved document conversion hangs with global timeout implementation * fix: improved change tracking validation with proper error handling * feat: enhanced DOCX metadata extraction for improved change tracking capabilities * fix: improved Excel citation handling when OCR data is not available * feat: added on-prem licensing alerts when connection to license.reducto.ai fails * feat: implemented timeout functionality for improved processing performance and reliability * fix: improved authentication on /upload and /cancel endpoints for better security * feat: enhanced multilingual OCR text embedding with support for Latin, CJK, Cyrillic, and Devanagari scripts using custom Unifont font * feat: file-based authentication system for on-prem deployments with Kubernetes secret mounting support * feat: automatic file cleanup system with configurable retention windows (default 60 minutes) to manage storage usage * fix: improved authentication reliability with retry logic for API validation calls * fix: enhanced extraction pipeline to handle None extract\_outputs and improve data merging * fix: native office conversion now skips files over 150MB and falls back to LibreOffice for better reliability * fix: improved citation confidence handling when confidence values are null * fix: batch processing improvements to keep batches alive for large documents * feat: enhanced array extraction to work with non-array fields for improved data extraction flexibility * feat: improved LLM error handling and timeout support for more reliable model calls * feat: added support for OpenDocument Text (.odt) file uploads through existing LibreOffice conversion pipeline * fix: improved block merging logic to properly update table content during document enrichment * fix: added retry logic for database errors on job status requests to improve reliability * fix: preserve empty blocks (such as figures with no content) in final document layout * feat: default to big extract model * feat: automatic file cleanup * fix: support for azure openai * feat: add hidden sheet/row/column filtering for Excel processing * feat: Fix jsonbbox and citations for excel * fix: pdf processing timeout * fix: Fix DOCX to PDF conversion error status code from 500 to 400 * feat: enable change tracking capability * fix: remove the large figure filter in dfine layout model postprocessing * feat: Split blocks for array\_extract on excel to separate pages * feat: Persist the full result for url results to persist bucket * feat: Query jobs by user-id and fair queueing docs * fix: Edit conditionals so full tables aren't returned in citations * fix: Fix job type error when cancelling a job * fix: on-prem changelog auth on light and dark mode * fix: Rename file to include guessed extension if one isn't already included * fix: handle empty bbox arrays in layout postprocess calculations * feat: update replicated helm chart * feat: surface all table citations in v2 * feat: internal webhook via IPC on job completion * feat: expose table citations in extraction results * feat: add persist config option that persists parsebatches and results * fix: keep batch alive for large HTML documents * fix: check for empty document\_url list in extract * refactor: PDF editing with PyPDFForm for improved form handling * feat: update extraction pipeline with improved array handling * fix: f-string usage in logfire calls across the codebase * fix: OpenAI vision LLM calls in LLM router * fix: root level acroform rendering (preserve form values) * feat: add onprem config option to enable figure summaries for all figures * feat: add exclude\_configs query param to /jobs endpoint to reduce response size * fix: onprem CD now corrects the `/version` url to the latest version number Enhanced support for OCR text embedding in PDFs with `embed_text_metadata_pdf` flag. Added support for routing on-premise deployments to v2 extraction pipeline and improved exception handling for subscription errors in Stripe usage logging. Fixed Pydantic AI Agent tools configuration in the document editing functionality. Added Azure OpenAI support and improved table model with a new fallback mechanism. Enhanced webhook validation and added support for selective customer notifications via target channels parameter. Added support for LiteLLM Proxy configuration via environment variables: * `LITELLM_PROXY_URL`: URL of the LiteLLM Proxy * `LITELLM_PROXY_FAST_MODEL`: Fast model to route to via the proxy * `LITELLM_PROXY_ACCURATE_MODEL`: Accurate model to route to via the proxy When using the proxy configuration: * Both fast and accurate models must be defined if using the proxy URL * Existing LiteLLM routing options are overridden when proxy settings are active This enables easier integration with centralized proxy setups for model routing and observability. Add a `/wipe` endpoint to the On-Prem API to wipe the database of all parse jobs, batches, and tasks. This is only available to on prem customers and is a good fail safe. Ensure that this is not available or exposed to the users. Should be a backend only failsafe. Please let us know if you'd like this disabled or removed in your deployment. In this release, we make some query optimizations to significantly reduce CPU usage of the Postgres DB at high document volumes (e.g. > 1k pg/min). In Google Cloud environments, we improved the skew detection capability by updating our thresholds to more intelligently detect skew in certain cases. Some additional bug fixes were made for folks who specify an LLM provider preference. # Database configuration Source: https://docs.reducto.ai/onprem/database_configuration Configure PostgreSQL connection pooling, timeouts, and performance settings for on-premise deployments ## Connection architecture Each Reducto pod (HTTP server or worker) maintains its own SQLAlchemy connection pool to PostgreSQL. Connections are not shared across pods. | Pod type | Processes per pod | Pool size per process | Max connections per pod | | ---------- | ---------------------------- | ---------------------------- | ----------------------- | | **HTTP** | 8 gunicorn workers (default) | `pool_size` + `max_overflow` | 8 × (4 + 8) = **96** | | **Worker** | 1 | `pool_size` + `max_overflow` | 4 + 8 = **12** | The number of HTTP workers is controlled by the `HTTP_WORKERS` environment variable (default: `8`). ## Default pool settings These are the application-level defaults for on-premise deployments: | Setting | Default | Description | | ------------------------- | --------------- | ------------------------------------------------------------ | | `DB_POOL_SIZE` | `4` | Persistent connections kept open per process | | `DB_MAX_OVERFLOW` | `8` | Additional connections created under load (closed when idle) | | `DB_POOL_TIMEOUT` | `30` (seconds) | Max time to wait for a connection from the pool | | `DB_POOL_PRE_PING` | `true` | Validate connections before use (handles stale connections) | | `DB_POOL_RECYCLE` | `-1` (disabled) | Max connection lifetime in seconds before recycling | | `DB_LOCK_TIMEOUT_MS` | `15000` | PostgreSQL `lock_timeout` per transaction | | `DB_STATEMENT_TIMEOUT_MS` | `20000` | PostgreSQL `statement_timeout` per transaction | All settings can be overridden via environment variables in your Helm values. ## Connection pooling with PgBouncer or RDS Proxy We strongly recommend running an external connection pooler between Reducto and PostgreSQL. Without one, connection storms during pod scaling (especially KEDA-driven autoscaling) can overwhelm the database. ### Azure (built-in PgBouncer) Azure Database for PostgreSQL Flexible Server includes a built-in PgBouncer. Our [Azure on-prem Terraform module](https://github.com/reductoai/reducto-onprem-azure) enables it by default: ```hcl theme={null} variable "postgres_pgbouncer_enabled" { description = "Enable PgBouncer for built-in connection pooling" type = bool default = true } ``` When enabled, the database URL automatically points to port `6432` (PgBouncer) instead of `5432` (direct PostgreSQL). No application-level changes are needed. To verify PgBouncer is active, check the Azure Portal under your PostgreSQL Flexible Server > Server parameters > `pgbouncer.enabled`. ### AWS (RDS Proxy) Our [AWS on-prem Terraform module](https://github.com/reductoai/reducto-onprem-infra) provisions an RDS Proxy by default. The Helm chart automatically uses the pooled database URL: ```yaml theme={null} env: DATABASE_URL: ``` RDS Proxy handles connection multiplexing, failover, and credential rotation transparently. ## Estimating total database connections To estimate your peak connection count: ``` Total connections = (HTTP pods × HTTP_WORKERS × (DB_POOL_SIZE + DB_MAX_OVERFLOW)) + (Worker pods × (DB_POOL_SIZE + DB_MAX_OVERFLOW)) ``` **Example** with 2 HTTP pods and 10 worker pods (default settings): ``` HTTP: 2 × 8 × (4 + 8) = 192 Workers: 10 × (4 + 8) = 120 Total: 312 ``` Azure Flexible Server and AWS RDS have connection limits based on instance size. With a connection pooler, the actual backend connections will be much lower than this number, since the pooler multiplexes idle application connections onto fewer database connections. ### Right-sizing for your workload If you see connection timeout errors or pool exhaustion: 1. **Increase `DB_POOL_SIZE`** if connections are frequently at capacity during steady state 2. **Increase `DB_MAX_OVERFLOW`** if you see spikes during burst traffic 3. **Decrease `DB_POOL_RECYCLE`** (e.g., `300`) if you're behind a pooler that has its own idle timeout — this prevents the application from trying to use connections the pooler has already closed If you are using an external pooler (PgBouncer or RDS Proxy), keep `DB_POOL_PRE_PING` set to `true`. This ensures the application validates connections before use, which is important when the pooler may close idle backend connections. ## Timeout tuning The `lock_timeout` and `statement_timeout` values are set per transaction using `SET LOCAL`, which is compatible with all connection poolers (direct, PgBouncer, RDS Proxy). If you process very large documents (100+ pages) and see timeout errors, you may want to increase these: ```yaml theme={null} env: DB_STATEMENT_TIMEOUT_MS: "30000" # 30 seconds DB_LOCK_TIMEOUT_MS: "20000" # 20 seconds ``` Keep `statement_timeout` higher than `lock_timeout` so that lock contention surfaces as a lock timeout rather than a generic statement timeout. # Deployment options (Cloud, On-prem, Hybrid) Source: https://docs.reducto.ai/onprem/enterprise_deployment_options Flexible, secure deployment options designed to meet enterprise data-governance, performance, and scalability requirements Reducto offers flexible, secure deployment options designed to meet enterprise data-governance, performance, and scalability requirements. Customers can choose between a dedicated VPC deployment, a hybrid VPC model, and a fully hosted SaaS model, balancing control, cost, and compute performance. ## Option 1: Hybrid VPC deployment A hybrid model balancing privacy and compute efficiency. All data and storage reside in the customer's VPC, while ephemeral processing is handled by Reducto's dedicated GPU infrastructure. The GPU infrastructure on Reducto's side can be made single-tenant if desired or shared access can be provided. ### Architecture S3 Bucket/Database lives in a customer environment. Ephemeral workers, which do not persist data beyond memory, are deployed solely for your tenant and connect with your DB via whitelisted outgoing IP. ### Benefits * Maintains data sovereignty (storage & databases stay in the customer's cloud with separate data and compute planes) * Offloads model inference to Reducto's GPU cluster for cost and latency efficiency * Faster auto-scaling and avoid having to provision GPU capacity in your own cloud * Faster iteration speeds for feature resolution, reduced devops burden * Customers can connect their own LLMs or use Reducto's built-in ZDR agreements with processors (OpenAI, Vertex, etc.) ### Architecture variants * **Dynamic Workers** - auto-scales compute based on demand (can scale to zero) * **Always-on Workers** - reserved compute for predictable throughput ## Option 2: Reducto SaaS API Reducto offers a SaaS option that eliminates operational overhead while providing enterprise-level security and compliance, making it ideal for teams who want to focus on building applications rather than managing infrastructure. ### Architecture * Built on Amazon Web Services (AWS) and Modal Labs as the primary cloud providers * Uses AWS S3 for secure data storage with encryption at rest and in transit ### Benefits * Zero infrastructure management * HIPAA compliance available for Growth and Enterprise tiers * Zero data retention for Growth and Enterprise tiers * Automatic updates & reliability * SOC 2 Type II compliant with comprehensive security audits ## Option 3: Full VPC deployment A dedicated deployment fully hosted within the customer's cloud environment (AWS, GCP, or Azure), ensuring complete data isolation and compliance control. In this model, the customer owns the runtime security boundary. Reducto supplies the application and deployment artifacts, while the customer controls network exposure, API access, Kubernetes policy, storage policy, observability access, and egress. Review the [on-prem security model](/onprem/security_model) before production rollout. ### Architecture Reducto provides a container image containing all proprietary models. * Deployment is orchestrated via Helm chart and Terraform templates for seamless setup * Runs on Kubernetes with PostgreSQL as the minimum database requirement * GPU usage is configurable depending on workload * Reducto's models are optimized to run on CPU where possible ### Benefits Data never leaves the customer's VPC. ### Integration & options * Customers can connect their own LLMs (OpenAI, Google Vertex AI, Bedrock, etc.) * Customers can use Reducto's post-trained LLMs requiring GPU clusters * GPU-based extraction models (30B extraction, 7B citation) can be deployed alongside the main application for higher accuracy extraction. Requires H200/H100 GPUs. See [LLM & service configuration](/onprem/llm_options#gpu-based-extraction-models) for details. # Self-hosted fair queueing with Reducto Source: https://docs.reducto.ai/onprem/fair_queueing Fair queueing by user-id for on-premise deployment ### Fair queueing for Parse and Extract requests To take advantage of fair queueing on an on-premise deployment of Reducto, you can simply make your existing requests to `/parse` and `/parse_async` as normal, but additionally pass the user-id via a header parameter on your HTTP request. The header parameter is `user-id`. ```python theme={null} import requests url = "https://platform.reducto.ai/parse" payload = { "input": "https://utfs.io/f/140cc88f-6cff-4521-87fd-76ecc6532aef-es0zzc.pdf" } headers = { "user-id": "my_unique_user_id", "Authorization": "Bearer " } response = requests.post(url, json=payload, headers=headers) print(response.json()) ``` ### Filtering jobs by user-id To filter jobs by the user-id you can add the user-id via a header parameter similar to how the requests are made above. When the user-id is excluded from the header, all jobs will be returned regardless of which user it is associated with. Python example for querying jobs by user-id: ```python theme={null} import requests url = "https://platform.reducto.ai/jobs" headers = { "user-id": "my_unique_user_id", "Authorization": "Bearer " } response = requests.get(url, headers=headers) print(response.json()) ``` # Automatic file cleanup Source: https://docs.reducto.ai/onprem/file_cleanup Configure automatic deletion of uploaded files in Reducto ## Overview When `FILE_CLEANUP_ENABLED` is set, Reducto records the keys of every object written to customer storage and automatically deletes any object that is older than the configured retention window (default 60 minutes). The cleanup runs as part of the hourly `cleanup.py` cronjob. **Note:** Because cleanup is performed hourly, the minimum effective retention interval is 60 minutes (1 hour). ## Configuration | Variable | Description | Default | | ------------------------ | -------------------------------------------------------------------------------------------------------------------------------- | ------- | | `FILE_CLEANUP_ENABLED` | Set to any value to enable automatic deletion of stored objects. | – | | `FILE_RETENTION_MINUTES` | Retention window in minutes before an object is permanently removed. The minimum is 60 minutes (1 hour), as cleanup runs hourly. | `60` | Example: ```bash theme={null} export FILE_CLEANUP_ENABLED=1 export FILE_RETENTION_MINUTES=120 # keep files for two hours ``` # Hybrid VPC — AWS S3 Source: https://docs.reducto.ai/onprem/hybrid-vpc-aws Set up Hybrid VPC with AWS S3 storage and optional PrivateLink This guide covers setting up Hybrid VPC with AWS S3 as your storage backend. Reducto assumes an IAM role in your AWS account to read and write documents. ## Prerequisites * **AWS account(s)**: Can use separate accounts for development, staging, and production * **Terraform 1.2+**: For infrastructure provisioning * **Values from Reducto** (provided during onboarding): * Principal ARNs for Reducto's compute services * ExternalId for secure role assumption * Endpoint Service name and region (if using PrivateLink) * **If using PrivateLink**: Send Reducto every AWS account ID that will create a VPC endpoint, including separate dev, staging, production, or organizational accounts. Reducto must allow-list each account on the VPC Endpoint Service before endpoint creation succeeds. ### Principal ARNs Use the appropriate ARNs for your deployment region: | Environment | EKS Role ARN | Modal User ARN | | ------------- | ------------------------------------------------ | ---------------------------------------------- | | **Prod (US)** | `arn:aws:iam::731106932034:role/reducto-prod` | `arn:aws:iam::731106932034:user/modal-prod` | | **Prod-EU** | `arn:aws:iam::731106932034:role/reducto-prod-eu` | `arn:aws:iam::731106932034:user/modal-prod-eu` | ### VPC Endpoint Service Configuration If using PrivateLink, use the endpoint service closest to your region: | Environment | VPC Endpoint Service Name | Region | DNS Name | | ------------- | ------------------------------------------------------------ | -------------- | ------------------------------- | | **Prod (US)** | `com.amazonaws.vpce.us-west-2.vpce-svc-0929182c8ed77b7a8` | `us-west-2` | `hybrid.platform.reducto.ai` | | **Prod-EU** | `com.amazonaws.vpce.eu-central-1.vpce-svc-0a231d441f3a482a0` | `eu-central-1` | `hybrid.eu.platform.reducto.ai` | VPC endpoints support cross-region connections. You can create a VPC endpoint in your region that connects to any Reducto endpoint service above, regardless of your VPC's region. PrivateLink endpoint creation only works after Reducto has allow-listed the AWS account that creates the endpoint. If your organization uses separate AWS accounts for dev, staging, production, or separate business units, provide each account ID before setup. ## Setup ```bash theme={null} git clone https://github.com/reductoai-collab/reducto-hybrid-infra.git cd reducto-hybrid-infra ``` ```hcl theme={null} name_prefix = "reducto" # Use the appropriate Principal ARNs for your region reducto_principal_arns = [ "arn:aws:iam::731106932034:role/reducto-prod", "arn:aws:iam::731106932034:user/modal-prod" ] reducto_external_id = "" # Optional: customize bucket name (auto-generated if not set) # bucket_name = "my-company-reducto-data" # Optional: customize object retention (default: 1 day) # lifecycle_expiration_days = 1 tags = { Environment = "production" Project = "reducto-hybrid" } ``` ```bash theme={null} terraform init terraform plan terraform apply ``` ```bash theme={null} terraform output integration_values ``` Example output: ```json theme={null} { "bucket_name": "reducto-data-a1b2c3d4", "region": "us-east-1", "role_arn": "arn:aws:iam::987654321098:role/reducto-access", "access_mode": "assume_role", "privatelink_endpoint_id": null } ``` ### Components provisioned | Component | Purpose | Required | | ------------ | ----------------------------------------------------------- | -------- | | S3 Bucket | Document and artifact storage with configurable lifecycle | Yes | | IAM Role | Cross-account access for Reducto with ExternalId protection | Yes | | VPC Endpoint | PrivateLink endpoint for private API access | Optional | ## Access Modes The default and recommended access mode. Reducto assumes an IAM role in your account with ExternalId protection. ```hcl theme={null} access_mode = "assume_role" reducto_external_id = "your-external-id-from-reducto" ``` **Benefits:** * ExternalId prevents confused deputy attacks * Fine-grained permission control * Easy credential rotation Alternative mode that grants Reducto direct access via bucket policy. Simpler but without ExternalId protection. ```hcl theme={null} access_mode = "bucket_policy" ``` ## PrivateLink Setup (Optional) For private-only API access without traversing the public internet: Provide the following to your Reducto team: * **AWS Account ID(s)**: Every account where you'll create a VPC endpoint, including dev, staging, production, or separate organizational accounts * **Region(s)**: Where you need PrivateLink connectivity Reducto will add each account root as an allowed principal on the VPC Endpoint Service. Wait for Reducto's confirmation before you create the endpoint. Add to your Terraform configuration: ```hcl theme={null} enable_privatelink = true vpc_id = "vpc-0123456789abcdef0" subnet_ids = ["subnet-abc123", "subnet-def456"] reducto_endpoint_service_name = "com.amazonaws.vpce.us-west-2.vpce-svc-0929182c8ed77b7a8" reducto_endpoint_service_region = "us-west-2" ``` Use the region-specific DNS name matching your VPC endpoint: ```python theme={null} from reducto import Reducto client = Reducto( api_key="your-api-key", base_url="https://hybrid.platform.reducto.ai" ) ``` You **must** enable private DNS resolution in your VPC endpoint configuration. This is required for the DNS alias to resolve correctly within your VPC. ## Validation Checklist After `terraform apply`, verify your setup: * [ ] **Terraform apply succeeded** without errors * [ ] **S3 bucket has lifecycle rule**: ```bash theme={null} aws s3api get-bucket-lifecycle-configuration --bucket your-bucket-name ``` * [ ] **S3 bucket blocks public access**: All public access settings should be blocked * [ ] **IAM role trust policy is correct**: Verify Reducto principals and ExternalId condition ```bash theme={null} aws iam get-role --role-name reducto-access --query 'Role.AssumeRolePolicyDocument' ``` * [ ] **If PrivateLink enabled**: Reducto has confirmed that every endpoint-creating AWS account is allow-listed * [ ] **If PrivateLink enabled**: Endpoint status shows "available" ```bash theme={null} aws ec2 describe-vpc-endpoints --vpc-endpoint-ids vpce-xxx ``` * [ ] **Smoke test**: Run a small Reducto job and verify objects appear in the bucket ## Troubleshooting ### VPC endpoint service does not exist **Problem**: AWS returns `InvalidServiceName` or says the VPC Endpoint Service does not exist, but the service name and region match the table above. **Solution**: Send Reducto the AWS account ID for the account creating the endpoint. Reducto will allow-list the account root on the endpoint service. After Reducto confirms the change, retry endpoint creation. Repeat this for each account that will create an endpoint. ## Multi-Region Setup Deploy separate infrastructure in each region with region-specific Principal ARNs: ```bash theme={null} # US East cd environments/us-east-1 terraform apply -var="name_prefix=reducto-us" # EU (Frankfurt) cd ../eu-central-1 terraform apply -var="name_prefix=reducto-eu" ``` Each region requires its own IAM role with the region-specific Principal ARNs from the table above. ## Multi-Environment Setup For organizations with separate AWS accounts for dev/staging/prod: ``` environments/ ├── dev/ │ ├── main.tf │ └── terraform.tfvars ├── staging/ │ ├── main.tf │ └── terraform.tfvars └── prod/ ├── main.tf └── terraform.tfvars ``` Each environment should use a separate Terraform state file and its own S3 bucket and IAM role. If the environments share the same Reducto org, Reducto can register them as named Hybrid VPC environments instead of separate orgs. ## Multiple Buckets for One Reducto Org For workflows that need client-specific buckets under one Reducto organization, register each bucket/role pair as a named environment: ```json theme={null} { "default_environment": "client-a", "environments": { "client-a": { "region": "us-east-1", "bucket": "client-a-reducto-data", "role_arn": "arn:aws:iam::987654321098:role/reducto-client-a" }, "client-b": { "region": "us-east-1", "bucket": "client-b-reducto-data", "role_arn": "arn:aws:iam::987654321098:role/reducto-client-b" } } } ``` Then select the environment on each request: ```json theme={null} { "input": "s3://client-b-reducto-data/documents/invoice.pdf", "settings": { "hybrid_vpc": { "environment": "client-b" } } } ``` ## Security ### ExternalId protection The ExternalId in the IAM role trust policy prevents [confused deputy attacks](https://docs.aws.amazon.com/IAM/latest/UserGuide/confused-deputy.html). Only requests with the correct ExternalId can assume the role. ### Principle of least privilege The IAM role grants only the permissions necessary for Reducto operations: * `s3:GetObject` — Read documents * `s3:PutObject` — Write results and artifacts * `s3:DeleteObject` — Clean up temporary files * `s3:ListBucket` — List objects for batch operations * `s3:AbortMultipartUpload`, `s3:ListMultipartUploadParts` — Handle large file uploads ### Automatic data cleanup Objects expire automatically based on the lifecycle configuration (default: 24 hours). This ensures no long-term data persistence, compliance with retention policies, and automatic cleanup of intermediate artifacts. # Hybrid VPC — Azure Blob Storage Source: https://docs.reducto.ai/onprem/hybrid-vpc-azure Set up Hybrid VPC with Azure Blob Storage for data sovereignty This guide covers setting up Hybrid VPC with Azure Blob Storage as your storage backend. Reducto uses a cross-tenant service principal to read and write documents in your Azure Storage account. ## Prerequisites * **Azure subscription** with permissions to create Storage accounts and role assignments * **Terraform 1.2+** with the AzureRM provider (for automated setup) * **Values from Reducto** (provided during onboarding): * Reducto's Azure AD application ID (for cross-tenant access) * Organization ID for configuration ## Architecture ```mermaid theme={null} flowchart LR subgraph customer["Customer Azure Subscription"] direction TB blob["Azure Blob Storage
(documents, artifacts)"] rbac["RBAC Role Assignment
(Storage Blob Data Contributor)"] rbac --> blob end subgraph reducto["Reducto Infrastructure"] direction TB sp["Service Principal
(multi-tenant app)"] workers["Compute Workers"] sp --> workers end workers <--> blob ``` ## Setup Reducto provides a Terraform example for Azure setup in the [hybrid infrastructure repository](https://github.com/reductoai-collab/reducto-hybrid-infra). ```bash theme={null} git clone https://github.com/reductoai-collab/reducto-hybrid-infra.git cd reducto-hybrid-infra/examples/azure ``` ```hcl theme={null} name_prefix = "reducto" location = "eastus" resource_group_name = "reducto-hybrid-rg" # Provided by Reducto during onboarding reducto_service_principal_object_id = "" tags = { Environment = "production" Project = "reducto-hybrid" } ``` ```bash theme={null} terraform init terraform plan terraform apply ``` ```bash theme={null} terraform output integration_values ``` Provide the output values (storage account name, container name, connection string) to your Reducto team. 1. Go to **Azure Portal** → **Storage Accounts** → **Create** 2. Configure: * **Performance**: Standard * **Redundancy**: LRS (or your preferred level) * **Enable** hierarchical namespace if needed 3. Under **Networking**, set **Public network access** to your preference 4. Under **Data protection**, configure lifecycle management (recommended: 1-day expiry) 1. Open the new Storage Account 2. Go to **Containers** → **+ Container** 3. Name: `reducto-documents` 4. **Public access level**: Private 1. Go to **Storage Account** → **Access Control (IAM)** → **Add role assignment** 2. Role: **Storage Blob Data Contributor** 3. Assign access to: **User, group, or service principal** 4. Select the Reducto service principal (provided during onboarding) 1. Go to **Storage Account** → **Access keys** 2. Copy the **Connection string** 3. Share with your Reducto team (securely) ### Components provisioned | Component | Purpose | Required | | ------------------------- | ------------------------------------------------- | ----------- | | Storage Account | Azure Blob Storage with lifecycle policies | Yes | | Blob Container | Container for documents and artifacts | Yes | | RBAC Role Assignment | Cross-tenant access for Reducto service principal | Yes | | Lifecycle Management Rule | Automatic blob expiry (default: 1 day) | Recommended | ## Integration Values After setup, provide these values to Reducto: | Value | Description | Where to find | | ---------------------- | ------------------------- | ------------------------------ | | `storage_account_name` | Storage account name | Azure Portal → Storage Account | | `container_name` | Blob container name | Storage Account → Containers | | `connection_string` | Storage connection string | Storage Account → Access keys | ## Data Lifecycle Configure lifecycle management to automatically delete blobs after processing: ```json theme={null} { "rules": [ { "name": "auto-expire", "type": "Lifecycle", "definition": { "actions": { "baseBlob": { "delete": { "daysAfterModificationGreaterThan": 1 } } }, "filters": { "blobTypes": ["blockBlob"] } } } ] } ``` ## Security * **Cross-tenant access**: Reducto's service principal is granted only `Storage Blob Data Contributor` on the specific container * **No shared keys required**: RBAC-based access is more secure than shared key authentication * **Network restrictions**: Optionally restrict access to specific IP ranges or virtual networks * **Automatic cleanup**: Lifecycle management policies ensure no long-term data persistence # Hybrid VPC — Box Source: https://docs.reducto.ai/onprem/hybrid-vpc-box Set up Hybrid VPC with Box as your document storage backend This guide covers setting up Hybrid VPC with Box as your storage backend. Reducto uses a Box enterprise application with Client Credentials Grant (CCG) to read and write documents in your Box environment. This integration is ideal for organizations that already manage documents in Box and want to process them with Reducto without moving data to a separate object store. ## Prerequisites * **Box Enterprise account** with admin access * **Box Developer Console** access for creating custom apps * Values provided during onboarding from Reducto (Organization ID) ## Architecture ```mermaid theme={null} flowchart LR subgraph customer["Customer Box Environment"] direction TB folder["Box Folder
(documents, artifacts)"] app["Box Custom App
(CCG auth)"] app --> folder end subgraph reducto["Reducto Infrastructure"] direction TB workers["Compute Workers"] api["Reducto API + Database"] workers --> api end workers <--> folder ``` ## Setup 1. Go to the [Box Developer Console](https://app.box.com/developers/console) 2. Click **Create New App** → **Custom App** 3. Select **Server Authentication (Client Credentials Grant)** as the authentication method 4. Name the app (e.g., "Reducto Integration") 5. Click **Create App** In the app's **Configuration** tab: 1. Under **Application Scopes**, enable: * **Read all files and folders stored in Box** * **Write all files and folders stored in Box** 2. Under **App Access Level**, select **App + Enterprise Access** 3. Click **Save Changes** 1. Go to [Box Admin Console](https://app.box.com/master) → **Apps** → **Custom Apps** 2. Click **Add App** 3. Enter the **Client ID** from your app's Configuration tab 4. Click **Authorize** This step requires Box Enterprise Admin privileges. If you're not an admin, ask your Box admin to authorize the app. Create a folder in Box for Reducto to use: 1. In Box, create a new folder (e.g., "Reducto Processing") 2. Note the **Folder ID** from the URL (e.g., `https://app.box.com/folder/123456789` → folder ID is `123456789`) 3. Ensure the service account (created automatically with your app) has access to this folder If you want Reducto to access the root of the enterprise, use folder ID `0`. For better isolation, we recommend creating a dedicated folder. From the app's **Configuration** tab, securely share: | Value | Where to find | | ----------------- | ----------------------------------------------------------------------- | | **Client ID** | Configuration → OAuth 2.0 Credentials | | **Client Secret** | Configuration → OAuth 2.0 Credentials | | **Enterprise ID** | General Settings → Enterprise ID (or Admin Console → Account & Billing) | | **Folder ID** | URL of the target folder in Box | ## Integration Values | Value | Description | | --------------- | -------------------------------------------------------- | | `client_id` | Box app OAuth 2.0 Client ID | | `client_secret` | Box app OAuth 2.0 Client Secret | | `enterprise_id` | Your Box Enterprise ID | | `folder_id` | Target folder ID for document storage (use `0` for root) | ## How It Works 1. **Authentication**: Reducto authenticates using Client Credentials Grant (CCG) — no user interaction required. The app's service account gets an access token automatically. 2. **Document storage**: Files are stored in the configured Box folder using the processing key as the filename. Reducto creates, reads, and deletes files as needed during processing. 3. **Result access**: Processing results are written back to Box. You can access them via shared links (similar to presigned URLs) or by browsing the folder in Box. ## Processing an Existing Box File Pass a Box file ID explicitly with the `box://` scheme: ```python Python theme={null} from reducto import Reducto client = Reducto() result = client.parse.run(input="box://123456789") ``` ```javascript Node.js theme={null} import Reducto from "reductoai"; const client = new Reducto(); const result = await client.parse.run({ input: "box://123456789" }); ``` ```bash cURL theme={null} curl -X POST "https://platform.reducto.ai/parse" \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{"input":"box://123456789"}' ``` Reducto retrieves the file using the Box credentials configured for your Hybrid VPC organization. The file must be readable by that Box application. Use `box://` rather than an `app.box.com` web URL, which is intended for an interactive browser session. ## Data Lifecycle Box does not have built-in lifecycle expiration like S3 or Azure. To manage data retention: Reducto automatically deletes intermediate artifacts after processing completes. Configure the retention period during onboarding. Box Enterprise supports [retention policies](https://support.box.com/hc/en-us/articles/360043697334-Managing-Retention-Policies) that can auto-delete content after a specified period. Set up a retention policy on the Reducto folder. Periodically review and delete files from the Reducto folder. Not recommended for production use. ## Security * **No user credentials required**: CCG authentication uses app-level credentials, not user passwords * **Scoped access**: The app can be restricted to specific folders using Box's collaboration model * **Enterprise admin approval**: The app must be explicitly authorized by a Box admin * **Audit trail**: Box provides detailed audit logs of all file access and modifications * **Credential rotation**: Client Secret can be rotated in the Box Developer Console without downtime ## Limitations | Limitation | Impact | Mitigation | | -------------------------- | ----------------------------------------------------- | ------------------------------------------------------- | | No Terraform provider | Setup is manual (not automated via IaC) | Documented step-by-step process above | | Rate limits | Box API has more aggressive rate limits than S3/Azure | Reducto handles rate limiting and retries automatically | | File-based API | Box uses file/folder hierarchy, not flat key-value | Reducto maps keys to filenames transparently | | No native lifecycle expiry | Unlike S3, no automatic per-object TTL | Use Box retention policies or Reducto-managed cleanup | ## Troubleshooting Verify that: 1. The app is authorized in the Box Admin Console 2. The service account has collaborator access to the target folder 3. The app scopes include read and write permissions Reducto handles rate limiting automatically with exponential backoff. If you see persistent rate limit errors, contact Reducto support — we may need to adjust concurrency settings for your account. Files created by the service account are owned by that account. To view them in the Box web UI, add yourself as a collaborator on the Reducto folder, or use the service account's credentials to browse. # Hybrid VPC Deployment Source: https://docs.reducto.ai/onprem/hybrid-vpc-deployment Deploy Reducto with your data in your own cloud account and compute managed by Reducto Hybrid VPC deployment provides a balance between data sovereignty and operational simplicity. Your data stays in your cloud account while Reducto manages all compute infrastructure. ## Overview In a Hybrid VPC deployment: * **Data stays in your cloud account**: All documents, intermediate artifacts, and results are stored in your storage * **Compute runs on Reducto's infrastructure**: GPU processing and model inference are handled by Reducto * **Stateless by design**: Objects have a configurable lifecycle, ensuring no data persists beyond processing * **Multiple storage providers**: AWS S3, Azure Blob Storage, and Box are supported Cross-account IAM role with ExternalId protection. Optional PrivateLink for private-only API access. Cross-tenant service principal access with RBAC. Standard Azure security model. Box enterprise app with Client Credentials Grant. Ideal for organizations already using Box for document management. Cross-project service account access. Standard GCP IAM model. ### Key benefits | Benefit | Description | | --------------------- | ---------------------------------------------------------- | | Data sovereignty | Storage remains in your cloud account | | No GPU management | Offload model inference to Reducto's optimized GPU cluster | | Cost efficiency | Avoid provisioning and maintaining GPU capacity | | Fast auto-scaling | Scale to zero when idle, scale up on demand | | Reduced DevOps burden | Faster iteration, no infrastructure maintenance | ## Architecture ```mermaid theme={null} flowchart LR subgraph customer["Customer Cloud Account"] direction TB storage["Object Storage
(S3 / Azure Blob / Box)"] auth["Access Credentials
(IAM Role / Service Principal / Box App)"] auth --> storage end subgraph reducto["Reducto Infrastructure"] direction TB workers["Compute Workers"] api["Reducto API + Database"] workers --> api end workers <--> storage ``` ### Data flow 1. You upload documents to your storage (or use Reducto's `/upload` endpoint) 2. You call Reducto API with a reference to your document 3. Reducto uses your configured credentials to access the document 4. Processing occurs on Reducto's compute infrastructure 5. Results and artifacts are written back to your storage 6. Objects expire automatically based on your lifecycle configuration ## Choosing a Storage Provider Best choice if your organization already uses AWS. Provides cross-account IAM role assumption with ExternalId protection against confused deputy attacks. Optional AWS PrivateLink keeps all traffic off the public internet. Terraform module provided for automated setup. Best choice if your organization uses Azure. Uses cross-tenant service principal with RBAC role assignments. Terraform configuration provided for automated setup. Best choice if your organization already manages documents in Box. Uses Box enterprise app authentication (Client Credentials Grant). No Terraform provider available — setup is done through the Box Admin Console. Best choice if your organization uses GCP. Uses cross-project service account access with IAM bindings. Contact Reducto for setup guidance. ## Document Handoff There are multiple ways to provide documents to Reducto APIs, regardless of which storage provider you use: Document handoff is a trust boundary. Only trusted services should submit document URLs or uploads to Reducto. If a caller can submit arbitrary URLs, that caller can ask Reducto to fetch any network location reachable from the deployment or hybrid worker environment. Use your gateway, storage policy, and egress policy to limit what callers can request. Use Reducto's `/upload` endpoint to upload documents directly. Files are automatically stored in your configured storage: ```python theme={null} from pathlib import Path from reducto import Reducto client = Reducto(api_key="your-api-key") upload_response = client.upload(file=Path("contract.pdf")) result = client.parse.run(document_url=upload_response.url) ``` Generate a temporary URL from your storage provider and pass it to Reducto: ```python theme={null} from reducto import Reducto # Generate a presigned/shared URL from your storage provider # (S3 presigned URL, Azure SAS URL, or Box shared link) document_url = "https://..." client = Reducto(api_key="your-api-key") result = client.parse.run(document_url=document_url) ``` For AWS S3, pass an S3 URI directly. Reducto will use the configured IAM role to access the object: ```python theme={null} from reducto import Reducto client = Reducto(api_key="your-api-key") result = client.parse.run(document_url="s3://your-bucket/documents/contract.pdf") ``` For production workflows: * Prefer direct uploads, `reducto://` file IDs, or tightly scoped storage-provider URLs. * Keep presigned URLs short-lived and scoped to a single object. * Do not pass URLs containing long-lived credentials. * Restrict worker egress to expected document sources when your workflow allows it. * Block cloud metadata endpoints and internal admin services from document-fetching egress. For PrivateLink connections (AWS only), specify the region-specific hybrid endpoint as `base_url`: * **US**: `https://hybrid.platform.reducto.ai` * **EU**: `https://hybrid.eu.platform.reducto.ai` ## Integration Contract After setting up your storage infrastructure, provide the following values to Reducto: | Value | Description | | ------------------------- | ------------------------------------------------------------------------ | | `bucket_name` | S3 bucket name | | `region` | AWS region (e.g., `us-east-1`) | | `role_arn` | IAM role ARN for Reducto to assume | | `external_id` | ExternalId for secure role assumption | | `privatelink_endpoint_id` | VPC Endpoint ID (if using PrivateLink) | | AWS account ID(s) | Every AWS account that will create a VPC endpoint (if using PrivateLink) | | Value | Description | | ---------------------- | ---------------------------------------- | | `storage_account_name` | Azure Storage account name | | `container_name` | Blob container name | | `connection_string` | Storage connection string (or SAS token) | | Value | Description | | --------------- | ------------------------------------- | | `client_id` | Box app client ID | | `client_secret` | Box app client secret | | `enterprise_id` | Box enterprise ID | | `folder_id` | Target folder ID for document storage | ## Multi-Region Setup For organizations needing storage in multiple regions for latency or compliance requirements, see the provider-specific setup guides linked above. Each provider supports region-specific configurations that Reducto routes automatically based on the deployment area (US, EU). ## Multiple Environments If one Reducto organization needs multiple dedicated storage locations, Reducto can register named Hybrid VPC environments under the same org. Each environment points to one bucket, IAM role, and region. Then select the environment per request: ```python theme={null} from reducto import Reducto client = Reducto(api_key="your-api-key") result = client.parse.run( document_url="s3://client-a-bucket/documents/contract.pdf", settings={ "hybrid_vpc": { "environment": "client-a" } } ) ``` Use separate Reducto orgs only when you need separate API keys, admins, billing, quotas, or customer-visible tenancy. For client-specific buckets inside the same customer account, named environments are the recommended path. ## Security All storage integrations follow least-privilege principles: * **AWS**: ExternalId prevents confused deputy attacks; IAM policy limits access to S3 operations only * **Azure**: RBAC role assignment scoped to the specific storage account/container * **Box**: App access restricted to the configured folder; enterprise admin approval required * **All providers**: Automatic data cleanup via configurable lifecycle policies # Hybrid VPC — Google Cloud Storage Source: https://docs.reducto.ai/onprem/hybrid-vpc-gcs Set up Hybrid VPC with Google Cloud Storage for data sovereignty Hybrid VPC with Google Cloud Storage uses a service account that you provide to allow Reducto's compute infrastructure to read and write documents in your GCS bucket. Use Studio to configure GCS environments alongside existing AWS S3 environments. This lets you run migrations with names such as `staging`, `prod`, `staging-gcp`, and `prod-gcp` under the same Reducto organization. ## Setup 1. In Studio, go to **Settings** → **Hybrid VPC**. 2. Add a new environment and choose **Google Cloud Storage** as the provider. 3. Enter the GCP region, bucket name, project ID, and optional bucket folder. 4. Create a service account with access to the bucket and grant it both of these roles on the bucket: * `roles/storage.objectAdmin`: read, write, and delete objects * `roles/storage.legacyBucketReader`: read bucket metadata 5. Generate a JSON key for that service account and paste it into `service_account_json`. 6. Click **Verify storage access**. Reducto writes and deletes a small verification object. 7. Save the configuration after verification succeeds. | Value | Description | | ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | | `storage_type` | Use `gcs` for Google Cloud Storage environments | | `region` | GCP region, for example `us-central1` or `europe-west1` | | `bucket` | GCS bucket name | | `project_id` | GCP project ID that owns the bucket | | `bucket_folder` | Optional key prefix for all objects | | `service_account_json` | Required. Service account credentials JSON for an account with `roles/storage.objectAdmin` and `roles/storage.legacyBucketReader` on the bucket | ## Security * **Scoped IAM**: The service account is granted `roles/storage.objectAdmin` and `roles/storage.legacyBucketReader` on the specific bucket only * **Customer-provided credentials**: You supply the service account JSON in `service_account_json`; scope its IAM to the single bucket and rotate the key on your own schedule * **Lifecycle management**: Configure object lifecycle rules on the bucket for automatic cleanup # LLM & service configuration options Source: https://docs.reducto.ai/onprem/llm_options Complete guide to LLM configuration and environment variables for Reducto ## OCR service configuration For detailed OCR provider configuration (AWS Textract, Azure Vision, GCP Vision API, cross-cloud OCR, and GPU OCR deployment), see the dedicated [OCR provider configuration](/onprem/ocr_options) page. ## LLM provider environment variables Reducto supports multiple LLM providers through environment variables. Below is a complete list of supported providers and their required environment variables. ### LiteLLM proxy | Variable | Description | Required | | ------------------------------ | ---------------------------------------- | -------- | | `LITELLM_PROXY_URL` | URL of the LiteLLM Proxy | Yes | | `LITELLM_PROXY_FAST_MODEL` | Fast model to route to via the proxy | Yes | | `LITELLM_PROXY_ACCURATE_MODEL` | Accurate model to route to via the proxy | Yes | ### OpenAI | Variable | Description | Required | | ---------------- | ------------------- | -------- | | `OPENAI_API_KEY` | Your OpenAI API key | Yes | ### Azure OpenAI | Variable | Description | Required | | ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | | `AZURE_OPENAI_API_KEY` | Your Azure OpenAI API key | Yes | | `AZURE_OPENAI_ENDPOINT` | Azure OpenAI endpoint (e.g., `https://your-resource-name.openai.azure.com/`) | Yes | | `OPENAI_API_VERSION` | Azure OpenAI API version (e.g., `2024-10-21`) | Yes | | `AZURE_OPENAI_MODEL_MAP` | Comma-separated map (or single default deployment) used to translate model names to Azure deployment names. Reducto uses the following models and each should resolve to a deployment unless a single default is supplied: `gpt-4o-2024-08-06`, `gpt-4o`, `gpt-4o-mini-2024-07-18`, `gpt-4o-mini`, `gpt-4.1`, `o1`. Example mappings: `my-default-deployment` (single default) or `gpt-4o=my-prod-dep, gpt-4o-mini=gpt4o-mini-dep` | Yes | ### Anthropic | Variable | Description | Required | | ------------------- | ---------------------- | -------- | | `ANTHROPIC_API_KEY` | Your Anthropic API key | Yes | ### Google | Variable | Description | Required | | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | -------- | | `GOOGLE_APPLICATION_CREDENTIALS` | Service account key json with `roles/aiplatform.user` role for Vertex AI | Yes | | `GCP_PROJECT_ID` | GCP project for Cloud Vision API | Yes | | `GCP_REGION` | Region for Vertex AI, defaults to `us-central1` | No | | `GCP_API_KEY` | [API key](https://console.cloud.google.com/apis/credentials) with no Application or API restrictions to access Cloud Vision API | Yes | ### Gemini | Variable | Description | Required | | ---------------- | ------------------- | -------- | | `GEMINI_API_KEY` | Your Gemini API key | Yes | ### AWS Bedrock | Variable | Description | Required | | ----------------------- | ------------------------------------------------- | ----------------------- | | `USE_CLAUDE_BEDROCK` | Set to any value to enable Claude via AWS Bedrock | Yes | | `AWS_ACCESS_KEY_ID` | AWS access key ID | Yes, when using Bedrock | | `AWS_SECRET_ACCESS_KEY` | AWS secret access key | Yes, when using Bedrock | | `AWS_REGION` | AWS region name | Yes, when using Bedrock | ## GPU-based extraction models Reducto offers GPU-based models for structured data extraction and fine-grained citations. For best results, deploy both models together. Model weights are downloaded from HuggingFace using a scoped token provided by Reducto. ### Prerequisites Create a Kubernetes secret with the HuggingFace token provided by Reducto: ```bash theme={null} kubectl create secret generic reducto-hf-token --from-literal=HF_TOKEN=hf_... ``` We recommend enabling `modelStorage` to cache weights on a PVC so restarts don't re-download. ### YAML extraction model (30B) **GPU requirement:** 1x NVIDIA H200 (will not fit on H100/A100/A10G). ```yaml theme={null} yamlExtract: enabled: true gpu: "H200" modelStorage: enabled: true size: "100Gi" storageClassName: "your-storage-class" ``` When enabled, `REDUCTO_YAML_EXTRACT_URL` is automatically injected into all worker and HTTP pods. ### Citation model (7B) **GPU requirement:** 1x NVIDIA H100 or H200. ```yaml theme={null} citationModel: enabled: true gpu: "H200" # or "H100" modelStorage: enabled: true size: "50Gi" storageClassName: "your-storage-class" ``` When enabled, `REDUCTO_CITATION_URL` is automatically injected into all worker and HTTP pods. If not deployed, citations fall back to your configured external LLM provider. ### Model path overrides Both deployments expose a `modelPath` field that can be updated if Reducto ships new model weights: ```yaml theme={null} yamlExtract: modelPath: "reducto/extract_30b_0108" # update when directed by Reducto citationModel: modelPath: "reducto/citation_7b_mimo_0812" # update when directed by Reducto ``` ### Extraction without GPU models If you do not deploy either GPU model, extraction uses your configured external LLM provider (OpenAI, Anthropic, Google, Azure, or Bedrock). No additional configuration is needed. ### Fine-tuned OpenAI extraction model (alternative) | Variable | Description | Required | | ------------------------------- | --------------------------------------------------------------------- | -------- | | `LOCAL_EXTRACT_CITATIONS_MODEL` | Fine-tuned OpenAI model ID (e.g., `openai:ft:gpt-4.1-2025-04-14:...`) | No | When set, this takes priority over the self-hosted extraction model. ## Request-level LLM overrides In addition to environment variables, on-prem deployments can override LLM configuration at the request level using the `overrides` parameter in `experimental_options`. ### Key-value processing overrides Override the model and add custom instructions for key-value (form) region processing: ```json theme={null} { "document_url": "https://example.com/form.pdf", "experimental_options": { "overrides": { "key_value": { "model": "google:gemini-2.5-flash-lite", "custom_instructions": "Pay special attention to date fields. Use MM/DD/YYYY format." } } } } ``` | Field | Description | | --------------------- | ----------------------------------------------------------- | | `model` | Model alias (`fast`, `accurate`) or `provider:model` format | | `custom_instructions` | Additional instructions appended to the default prompt | ### Resolution order **Model resolution:** 1. **Request override** - `experimental_options.overrides.key_value.model` 2. **Environment variable** - `LOCAL_KV_MODEL` 3. **Code default** - Based on deployment configuration **Prompt resolution:** 1. **Base prompt** - `LOCAL_KV_PROMPT` env var, or built-in default 2. **Custom instructions** - Appended from `overrides.key_value.custom_instructions` ### Environment variable defaults | Variable | Description | Default | | ----------------- | ----------------------------------------------------- | ---------------------------- | | `LOCAL_KV_MODEL` | Override model for KV processing | None (uses built-in cascade) | | `LOCAL_KV_PROMPT` | Base prompt for KV processing (can be fully replaced) | Built-in prompt | ## AI usage tracking Reducto includes a comprehensive AI usage tracking system that monitors language model consumption throughout the document processing pipeline. This feature provides detailed insights into token usage, request counts, and model utilization for billing and optimization purposes. ### How AI usage tracking works The AI usage tracking system operates at the block level within the parsing pipeline: 1. **Token Counting**: Each AI operation (table summarization, figure analysis, key-value extraction, etc.) records token consumption 2. **Request Tracking**: The system counts API calls made to each model 3. **Model Identification**: Usage is tracked per model type with provider information 4. **Aggregation**: Usage is aggregated across all blocks and pages for comprehensive reporting ### Available via /parse API AI usage information is **currently only available through the `/parse` API endpoint** using the `custom_format` parameter. This feature is not available in other API endpoints. ### Usage information structure When enabled, the system returns an `AIUsageInfo` object containing: ```json theme={null} { "did_use_ai_models": true, "ai_usage_info": [ { "promptTokenCount": 1500, "completionTokenCount": 300, "cachedTokenCount": 0, "requestCount": 2, "modelType": "us.anthropic.claude-3-7-sonnet-20250219-v1:0", "modelProvider": "anthropic", "modelRateLimitFamily": "us.anthropic.claude-3-7-sonnet" } ] } ``` ### Field descriptions * **`did_use_ai_models`**: Boolean indicating whether any AI models were used during processing * **`ai_usage_info`**: Array of usage information objects, one per model type used * **`promptTokenCount`**: Total input tokens sent to the model * **`completionTokenCount`**: Total output tokens generated by the model * **`cachedTokenCount`**: Total cached tokens used (when supported by provider) * **`requestCount`**: Number of API calls made to this model * **`modelType`**: Standardized model identifier * **`modelProvider`**: Provider name (e.g., "anthropic", "openai") * **`modelRateLimitFamily`**: Rate limiting group for the model ### Enabling AI usage tracking To retrieve AI usage information, set the `custom_format` parameter to `"ai_usage"` in your `/parse` request: ```json theme={null} { "input": "your_document_url", "settings": { "custom_format": "ai_usage" } } ``` ### Tracked AI operations The system tracks usage from these AI-powered features: * **Table Summarization**: Analysis and description of complex tables * **Figure Summarization**: Analysis and description of images and charts * **Key-Value enrichment**: Enrichment for form-like regions within documents ### Model name standardization The system automatically standardizes model names for consistent reporting: * Internal model identifiers are mapped to standard formats * Provider information is automatically added * Rate limit families are identified for capacity planning ### Possible model identifiers The following model identifiers may appear in the `modelType` field of AI usage tracking responses, if you have OpenAI and Anthropic access enabled: #### OpenAI models * `gpt-4o-2024-08-06` * `gpt-4o-mini-2024-07-18` #### Anthropic models * `claude-haiku-4-5-20251001` * `claude-3-7-sonnet-20250219` If you enable other model providers, they have their own prefixes which will appear. # Observability & Monitoring Source: https://docs.reducto.ai/onprem/observability Built-in observability stack for on-premise Reducto deployments ## Overview On-premise Reducto deployments include a built-in observability stack called **ClickStack**, which provides: * **HyperDX**: Unified observability UI for logs, traces, and metrics * **ClickHouse**: High-performance analytics database for telemetry storage * **OTEL Collector**: OpenTelemetry collector for ingesting and routing telemetry data ClickStack is enabled by setting `clickstack.enabled: true` in your Helm values. Everything else is automatic. No additional setup required. Telemetry is part of your on-premise security boundary. Reducto emits logs to stdout and can route traces, metrics, and logs through OpenTelemetry, but you control where telemetry is stored, who can access it, and how long it is retained. See the [on-prem security model](/onprem/security_model) for the shared responsibility model. Reducto telemetry is designed for operational metadata. It should not contain document content, OCR text, extracted values, prompts, model outputs, API tokens, access keys, secrets, or other customer content. File names and URLs can reveal customer identity and should be redacted or avoided before telemetry leaves the deployment. ## Accessing HyperDX ### Default Credentials When ClickStack is enabled, a seed admin user is automatically created on first install with default credentials. Contact the Reducto team for the default login details, or configure your own credentials in your Helm values (see [Configuration](#configuration) below). Change the default password immediately after first login. ### Accessing the UI HyperDX can be exposed via: * **Ingress**: Set `clickstack.hyperdx.ingress.enabled: true` with your domain * **Tailscale**: Set `clickstack.hyperdx.exposure.tailscale.enabled: true` for private access * **Cloudflare Tunnel**: Set `clickstack.hyperdx.exposure.cloudflareTunnel.enabled: true` * **Port-forward** (for testing): `kubectl port-forward svc/-clickstack-app 3000:3000` ## Prometheus Scraping Prometheus endpoints are intended for internal scraping by monitoring systems. Do not expose `/metrics`, `/prometheus`, ClickStack, HyperDX, ClickHouse, or the OTEL collector to the public internet. To collect metrics from services that expose Prometheus endpoints (like NGINX ingress controllers), enable the Prometheus receiver on the OTEL collector together with the Target Allocator (TA) subchart. TA shards scrape targets across collector replicas via consistent-hashing so each target is scraped exactly once. See [Scaling the OTEL Collector](#scaling-the-otel-collector) for the multi-replica rationale. ```yaml theme={null} prometheusScrape: enabled: true targetAllocator: enabled: true targetAllocator: config: collector_selector: matchlabels: app.kubernetes.io/instance: app.kubernetes.io/name: otel-collector config: scrape_configs: - job_name: nginx-ingress scrape_interval: 30s static_configs: - targets: - ingress-nginx-controller-metrics.ingress-nginx.svc.cluster.local:10254 - job_name: kube-state-metrics scrape_interval: 30s static_configs: - targets: - prometheus-stack-kube-state-metrics.monitoring.svc.cluster.local:8080 ``` `scrape_configs` follows the standard Prometheus scrape config schema — any `static_configs`, `kubernetes_sd_configs`, `relabel_configs`, etc. are supported. When `targetAllocator.enabled: true`, the OTEL collector pod must expose `POD_NAME` via the downward API so each replica gets a distinct `collector_id` for consistent-hashing. Render fails fast otherwise. ```yaml theme={null} otelCollector: extraEnvs: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name ``` Scraped metrics route to ClickHouse by default. See [Routing](#routing) to fan out to other sinks. ## Scaling the OTEL Collector The OTEL collector deploys as a Deployment with `otelCollector.replicaCount` replicas (HPA optional). Two receivers need explicit coordination once `replicaCount > 1`, otherwise every replica emits the same data and ClickHouse storage grows linearly with the replica count. | Receiver | Coordination mechanism | Values key | | ---------------------------- | --------------------------------------------------------------- | ---------------------------------- | | `prometheus` (scrape) | Target Allocator subchart shards targets via consistent-hashing | `targetAllocator.enabled` | | `k8s_cluster` + `k8sobjects` | `k8s_leader_elector` extension — only the lease-holder emits | `k8sMetrics.leaderElector.enabled` | ### Leader Elector for Kubernetes Metrics & Events `k8s_cluster` (cluster-level pod/node/container metrics) and `k8sobjects` (K8s events watch) are cluster-singletons — every replica running them independently produces an identical stream. Enable the leader-elector extension so only the lease-holder emits: ```yaml theme={null} k8sMetrics: enabled: true leaderElector: enabled: true ``` The chart renders a namespace-scoped `Role` + `RoleBinding` for the `coordination.k8s.io` lease, gated on `leaderElector.enabled`. Lease hand-off on pod rollover is automatic. Both `targetAllocator.enabled` and `k8sMetrics.leaderElector.enabled` default to `false` — single-replica deployments work out of the box with no extra RBAC or subchart. ## Configuration ### Seed User Configure the admin user credentials in your Helm values: ```yaml theme={null} clickstack: hyperdx: seedUser: email: "admin@yourcompany.com" password: "your-secure-password" teamName: "Your Team" ``` For production deployments, use a Kubernetes secret instead of a plaintext password: ```yaml theme={null} clickstack: hyperdx: seedUser: email: "admin@yourcompany.com" existingSecret: "my-hyperdx-secret" secretKey: "HYPERDX_ADMIN_PASSWORD" teamName: "Your Team" ``` ### ClickHouse Storage ```yaml theme={null} clickstack: clickhouse: persistence: dataSize: 50Gi # Adjust based on expected telemetry volume logSize: 10Gi ``` ### Data Retention Telemetry data retention is controlled by the OTEL exporter TTL: ```yaml theme={null} otelConfig: exporters: clickhouse: ttl: 360h # 15 days (default: 72h) ``` ## Telemetry Pipeline The OTEL collector receives telemetry from multiple sources and routes it to configured sinks: | Source | What it collects | Default sink | | -------------------------- | ------------------------------------------------------------------------------------------------------------ | ----------------- | | Application traces/metrics | OTLP from Reducto services | All enabled sinks | | Kubernetes metrics | Cluster-level pod, node, container metrics via k8s\_cluster receiver; node-level kubelet stats via DaemonSet | All enabled sinks | | Kubernetes events | K8s events via k8sobjects receiver | All enabled sinks | | Prometheus scrape | Metrics from any Prometheus endpoint | ClickHouse | | Application logs | OTLP logs from Reducto services | ClickHouse | ### Routing Each source can be independently routed to any combination of sinks: ```yaml theme={null} otelConfig: routing: traces: [tinybird, datadog, clickhouse, iceberg] # default metrics: [tinybird, datadog, clickhouse, iceberg] # default k8sMetrics: [tinybird, datadog, clickhouse, iceberg] # default k8sEvents: [tinybird, datadog, clickhouse, iceberg] # default prometheusScrape: [clickhouse] # default logs: [clickhouse] # default ``` Available sinks: `clickhouse`, `tinybird`, `datadog`, `iceberg`. Each sink must also be enabled in `otelConfig.exporters`. The defaults list all sinks, but only sinks that are both listed **and** enabled will actually receive data, so the defaults are safe for any exporter combination. ## Pod stack trace dumps (SIGUSR2) Every Reducto worker and HTTP pod installs a `SIGUSR2` handler that dumps a per-thread stack trace to `stderr` when signalled. Use this when a pod is unresponsive (stuck event loop, hung downstream call, contended thread pool) and `kubectl logs` alone doesn't explain why. Coverage: | Pod | Process labelled as | | ---------------------------------------------------------------------------- | ---------------------------------------------------------------------------- | | `reducto-http` (gunicorn) | `http-worker-` | | `reducto-streaq-worker` | `streaq--worker` (e.g. `streaq-io-worker`, `streaq-cpu-worker`) | | `reducto-worker`, `reducto-priority-worker`, `reducto-gpu-worker` (DB-queue) | `k8s-worker` (or the value of `LOGFIRE_SERVICE_NAME` when set) | The handler is always installed, with no Helm flag to disable it. Output is written directly to `stderr` (not through the structured logger), so the trace appears even if the application logging pipeline is itself wedged. ### Triggering a dump ```bash theme={null} # Pick a pod that's misbehaving kubectl get pods -n reducto -l app=reducto-worker # Find the worker PID (gunicorn / python process) kubectl exec -n reducto -- ps -eo pid,cmd | grep -E 'gunicorn|streaq|python' # Send SIGUSR2 to that PID kubectl exec -n reducto -- kill -s USR2 # Read the dump from the pod log kubectl logs -n reducto --tail=500 ``` You'll see a single-line banner followed by one frame block per thread: ``` USR2 signal received [http-worker-42]; dumping thread stacks USR2 triggered stack trace: Thread "MainThread" (most recent call first): File "/app/.venv/bin/gunicorn", line 8, in sys.exit(run()) ... Thread "asyncio-loop-0" (most recent call first): ... ``` ### Notes * Forked gunicorn and streaq child processes each register their own handler, so signalling the main PID alone won't dump child stacks. Signal each child PID individually if you need full process-tree coverage. * `SIGUSR2` is not used by any other component in the worker/HTTP processes, so triggering a dump is safe in production. The signal handler is async-safe and only enqueues work onto a dedicated daemon thread. * For wider diagnostics (CPU profile, off-CPU sampling), consider `py-spy dump --pid ` from a debug container. `SIGUSR2` is the lowest-friction option and works without an extra binary. ## Telemetry controls Reducto telemetry is designed to avoid customer content. Logs, traces, and metrics are for debugging and performance analysis, not for storing customer data or business records. Recommended controls: * Keep observability UIs and scrape endpoints on private networks. * Use SSO, VPN, Zero Trust access, or equivalent controls for operator access. * Send telemetry only to approved sinks. * Set retention periods that match your security and compliance requirements. * Review telemetry exports for customer content, file names, URLs, prompts, model outputs, and secrets before sharing them outside your organization. # OCR provider configuration Source: https://docs.reducto.ai/onprem/ocr_options Configure cloud OCR providers for on-premise Reducto installations Reducto supports multiple cloud OCR providers. By default, Reducto uses its own local OCR models (which require GPU), but you can configure cloud OCR providers for broader language support or to avoid provisioning GPU hardware for OCR. ## Provider overview | Provider | Credentials needed | Best for | | --------------------- | ---------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- | | **Reducto local OCR** | None (default) | General-purpose, privacy-sensitive, GPU-equipped deployments | | **AWS Textract** | `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` | AWS-native deployments | | **Azure Vision Read** | `AZURE_VISION_ENDPOINT` + `AZURE_VISION_KEY` (or `AZURE_VISION_ARRAY`) | Azure-native deployments | | **GCP Vision API** | `GOOGLE_APPLICATION_CREDENTIALS` + `GCP_PROJECT_ID` (or `GCP_SERVICE_ACCOUNT_EMAIL` for workload identity) | GCP-native deployments, cross-cloud deployments wanting GCP OCR, 60+ language support | ## How OCR provider is selected The `ocr_system` parameter in API requests controls provider routing. Only two values are available to API callers: `standard` (default) and `legacy`. **`standard` (default) routing priority:** 1. GCP Vision API, if GCP credentials are configured 2. Azure Vision or AWS Textract, if configured (Azure is preferred over Textract) **`legacy` routing priority:** 1. Azure Vision or AWS Textract, if configured (Azure is preferred over Textract) 2. GCP Vision API, as fallback if only GCP credentials are available In practice: if you configure GCP credentials, `standard` requests use GCP Vision. If you only have Azure/AWS credentials, both `standard` and `legacy` use those. ### Auto-detection: GCP-only environments When **only** GCP credentials are configured (no AWS or Azure), Reducto auto-detects this and routes all OCR through GCP Vision regardless of the `ocr_system` value. This requires: * No Azure credentials (`AZURE_VISION_ENDPOINT` and `AZURE_VISION_ARRAY` both unset) * No AWS credentials (`AWS_ACCESS_KEY_ID` unset) * GCP credentials available (`GOOGLE_APPLICATION_CREDENTIALS` or `GCP_SERVICE_ACCOUNT_EMAIL`) * `GCP_PROJECT_ID` is set *** ## AWS Textract ### Environment variables | Variable | Description | Required | | ----------------------- | ------------------------------------------------------- | -------- | | `AWS_ACCESS_KEY_ID` | AWS access key ID | Yes | | `AWS_SECRET_ACCESS_KEY` | AWS secret access key | Yes | | `TEXTRACT_REGIONS` | Comma-separated `region:quota` pairs for load balancing | No | Default regions: `us-east-2:100,us-east-1:100,us-west-2:100,ap-south-1:5,eu-west-1:5` ```bash theme={null} # Format: region:quota pairs (quota defaults to 1 if omitted) TEXTRACT_REGIONS=us-east-1:50,us-west-2:25,eu-west-1:10 # Government Cloud TEXTRACT_REGIONS=us-gov-west-1:10,us-gov-east-1:10 ``` ### Required IAM permissions ```json theme={null} { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "textract:DetectDocumentText", "Resource": "*" } ] } ``` *** ## Azure Vision Read ### Environment variables **Single endpoint:** | Variable | Description | Required | | ----------------------- | ---------------------------------- | --------------------------------------- | | `AZURE_VISION_ENDPOINT` | Azure Computer Vision endpoint URL | Yes (unless using `AZURE_VISION_ARRAY`) | | `AZURE_VISION_KEY` | Azure Computer Vision API key | Yes (unless using `AZURE_VISION_ARRAY`) | **Multiple endpoints (load balancing and failover):** | Variable | Description | Required | | ----------------------------- | --------------------------------------------------------- | -------- | | `AZURE_VISION_ARRAY` | JSON array of `{"endpoint": "...", "key": "..."}` objects | No | | `AZURE_VISION_ARRAY_STRATEGY` | `load_balance` (default) or `priority` | No | ```bash theme={null} AZURE_VISION_ARRAY='[ {"endpoint": "https://vision-east.cognitiveservices.azure.com/", "key": ""}, {"endpoint": "https://vision-west.cognitiveservices.azure.com/", "key": ""} ]' ``` | Strategy | Behavior | | ------------------------ | ---------------------------------------------------------------------------------------------- | | `load_balance` (default) | Randomly selects an endpoint per request. On transient error, retries then fails over to next. | | `priority` | Tries endpoints in order. On transient error, retries then fails over to next. | **Timeout and retry tuning:** | Variable | Default | Description | | --------------------------------- | --------------- | ---------------------------------------------------------------- | | `AZURE_VISION_TOTAL_CALL_TIMEOUT` | `45` (seconds) | Hard cancellation cap per `analyze` call, including all retries. | | `AZURE_VISION_WALL_TIMEOUT` | `120` (seconds) | SDK-level wall-clock timeout (checked between retries). | | `AZURE_VISION_CONNECTION_TIMEOUT` | `10` (seconds) | Per-attempt TCP connect timeout. | | `AZURE_VISION_READ_TIMEOUT` | `30` (seconds) | Per-attempt socket read timeout. | | `AZURE_VISION_MAX_RETRIES` | `2` | Retries per endpoint. Total attempts = `1 + MAX_RETRIES`. | Transient errors (408, 429, 5xx, network errors) trigger retries and failover. Non-retryable 4xx errors surface immediately. When `AZURE_VISION_ARRAY` is set, `AZURE_VISION_ENDPOINT` and `AZURE_VISION_KEY` are ignored. *** ## GCP Vision API ### Environment variables | Variable | Description | Required | | -------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------- | | `GOOGLE_APPLICATION_CREDENTIALS` | Path to service account key JSON file, or the raw JSON content. Handles both OCR routing detection and Vision API authentication via Application Default Credentials. | Yes (unless using `GCP_SERVICE_ACCOUNT_EMAIL`) | | `GCP_PROJECT_ID` | GCP project ID for quota attribution | Yes | | `GCP_SERVICE_ACCOUNT_EMAIL` | Service account email for workload identity auth (alternative to `GOOGLE_APPLICATION_CREDENTIALS`). | No | | `GCP_API_KEY` | [API key](https://console.cloud.google.com/apis/credentials) with Cloud Vision API enabled. Optional; if set, the Vision API client uses this instead of ADC. | No | | `GCP_REGION` | Region for Vertex AI | No (default: `us-central1`) | ### Authentication methods **Option 1: Service account key (recommended)** `GOOGLE_APPLICATION_CREDENTIALS` handles everything: Reducto uses it to detect GCP availability for routing, and the Vision API client picks it up via Application Default Credentials (ADC). ```bash theme={null} GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account-key.json GCP_PROJECT_ID=your-project-id ``` `GOOGLE_APPLICATION_CREDENTIALS` can be a file path or the raw JSON content of the service account key. **Option 2: Workload identity (GKE)** Set `GCP_SERVICE_ACCOUNT_EMAIL` to use GKE workload identity. No key file needed. ```bash theme={null} GCP_SERVICE_ACCOUNT_EMAIL=ocr-sa@your-project.iam.gserviceaccount.com GCP_PROJECT_ID=your-project-id ``` ### Required GCP API and roles * **Cloud Vision API** must be enabled on the project (`GCP_PROJECT_ID`). * If using a service account, it needs at minimum the `roles/cloudvision.user` role. * If using `GOOGLE_APPLICATION_CREDENTIALS` for broader GCP features (storage, Vertex AI), the service account also needs: * `roles/aiplatform.user` (for Vertex AI / Gemini LLM calls) * `roles/storage.objectAdmin` (if using GCS for file storage) *** ## Cross-cloud OCR: using GCP Vision on non-GCP infrastructure If you run on Azure or AWS but want GCP Vision for OCR, set `GCP_OCR_ONLY=true` so Reducto uses GCP only for Vision API calls and does not initialize GCS storage. | Variable | Description | Default | | -------------- | ------------------------------------------------- | ------- | | `GCP_OCR_ONLY` | Use GCP for Vision API OCR only, not for storage. | `false` | ### Example: Azure infrastructure with GCP Vision OCR ```bash theme={null} # --- Azure storage (unchanged) --- AZURE_STORAGE_CONNECTION_STRING=DefaultEndpointsProtocol=https;AccountName=... # --- Azure Vision (optional, keep as fallback or remove) --- # If you want to remove Azure Vision entirely, delete these. # If you keep them, 'legacy' OCR requests will still use Azure Vision, # while 'standard' (default) requests will use GCP Vision. AZURE_VISION_ENDPOINT=https://your-vision.cognitiveservices.azure.com/ AZURE_VISION_KEY=your-azure-vision-key # --- GCP Vision OCR --- GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account-key.json GCP_PROJECT_ID=your-gcp-project-id GCP_OCR_ONLY=true ``` With this configuration: * **Default API requests** (`ocr_system=standard` or unset) route to **GCP Vision API** * **`ocr_system=legacy` requests** route to **Azure Vision** (if Azure Vision credentials are still set) or **GCP Vision** (if Azure Vision credentials are removed) * **File storage** remains on **Azure Blob Storage** ### Example: AWS infrastructure with GCP Vision OCR ```bash theme={null} # --- AWS storage --- AWS_ACCESS_KEY_ID=your-aws-key AWS_SECRET_ACCESS_KEY=your-aws-secret BUCKET=your-s3-bucket # --- GCP Vision OCR --- GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account-key.json GCP_PROJECT_ID=your-gcp-project-id GCP_OCR_ONLY=true ``` ### Migration path: switching from Azure Vision to GCP Vision If you are currently using Azure Vision and want to switch to GCP Vision as your primary OCR: 1. **Add GCP credentials** to your deployment: ```bash theme={null} GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account-key.json GCP_PROJECT_ID=your-gcp-project-id GCP_OCR_ONLY=true ``` 2. **Test with a few requests.** The default `ocr_system=standard` will now route to GCP Vision. Verify OCR quality meets your expectations. 3. **Optionally remove Azure Vision credentials.** If you no longer need Azure Vision as a fallback for `legacy` requests, remove `AZURE_VISION_ENDPOINT` and `AZURE_VISION_KEY` (or `AZURE_VISION_ARRAY`). This simplifies the configuration. 4. **`GCP_OCR_ONLY` should remain set** as long as you are using non-GCP file storage. *** ## GPU OCR deployment (`OCR_ONLY` mode) For deployments that want to use Reducto's own OCR models (which run on GPU) as a dedicated service, set `OCR_ONLY=true`. This starts a lightweight service exposing only the `/ocr` endpoint. | Variable | Description | Default | | ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- | | `OCR_ONLY` | Enable OCR-only mode | `false` | | `OFFLINE` | Must be `1` for on-prem deployments | - | | `HTTP_WORKERS` | HTTP worker processes | `8` | | `MAX_PROCESSING_REQUESTS` | Max concurrent OCR requests per worker | `1` | | `MAX_QUEUED_REQUESTS` | Max queued requests per worker | `0` | | `OCR_THREADS` | Thread pool size for OCR | `5` | | `NUM_GPUS` | GPUs available (for CUDA device assignment) | `0` | | `REQUIRE_GPU` | Fail startup if ONNX Runtime cannot reach the GPU. Defaults to on for GPU images. Set `false` to intentionally run a GPU image on CPU nodes. | (GPU images: `true`) | GPU images refuse to start when ONNX Runtime cannot actually reach the GPU, rather than silently falling back to CPU at a fraction of the throughput. The startup log names the cause and the commands to confirm it. The most common cause on NVSwitch hardware (HGX/SXM H100, H200, A100) is `nvidia-fabricmanager` not running, or running at a version that does not exactly match the installed driver. Note that `nvidia-smi` still reports a perfectly healthy GPU in this state, because NVML does not touch fabric state. To confirm on the node: ```sh theme={null} nvidia-smi -q | grep -i -A3 fabric # expect "State: Completed" systemctl status nvidia-fabricmanager cat /proc/driver/nvidia/version # must match the fabricmanager package version ``` Repair the node before upgrading the image: a deployment that was previously serving degraded CPU-only OCR will now fail its readiness check instead. If you need to run a GPU image on CPU nodes deliberately, set `REQUIRE_GPU=false`. ```yaml theme={null} # Helm values gpuOcr: enabled: true replicaCount: 1 resources: requests: nvidia.com/gpu: 1 limits: nvidia.com/gpu: 1 ``` The Helm chart sets `OCR_ONLY=true` and `OFFLINE=1` automatically. # Operations Source: https://docs.reducto.ai/onprem/operations Runtime configuration for on-premise Reducto worker health and restart behavior Runtime knobs operators use to keep on-prem Reducto worker pods healthy: liveness behavior, restart policy, and watchdog tuning. Related references: * [Observability → Pod stack trace dumps](/onprem/observability#pod-stack-trace-dumps-sigusr2) for live thread-stack diagnostics via `SIGUSR2`. * [Database configuration](/onprem/database_configuration) for DB pool sizing and timeouts. * [LLM options → Azure Vision](/onprem/llm_options#azure-vision-ocr) for OCR provider timeouts, retries, and failover. ## Worker liveness probe The DB-queue worker pods (`reducto-worker`, `reducto-priority-worker`, `reducto-gpu-worker`) ship with a Kubernetes liveness probe that restarts a pod **only** when it is stuck mid-processing, not when it is idle waiting for work. ### How it works 1. An asyncio `WorkerWatchdog` task runs alongside the worker's job loops. 2. Every 5 seconds it writes a heartbeat file `/tmp/worker-state` containing ` `. Idle workers emit `-1` for the age. 3. The kubelet runs `bin/worker-liveness.sh` as an `exec` probe. The script fails (exit non-zero) and triggers a restart when **either**: * the heartbeat file itself is older than `WORKER_LIVENESS_WATCHDOG_STALE_SEC` (event loop is wedged so the watchdog can't tick); **or** * the oldest in-flight task age exceeds `WORKER_LIVENESS_STUCK_THRESHOLD_SEC` (a real job has hung beyond the threshold). 4. Idle workers always pass the probe. The watchdog ticks even with no work, and the age sentinel `-1` is always treated as healthy. A file-based heartbeat is used rather than an in-process HTTP `/health` endpoint because an HTTP server can keep returning `200` while the asyncio event loop is blocked on a syscall. The watchdog has to be alive to refresh the file, so the probe directly tests the thing we care about. ### Helm configuration Configure the probe via `worker.livenessProbe.*` in your Helm values: ```yaml theme={null} worker: livenessProbe: enabled: true # set to false to disable the probe entirely stuckTaskThresholdSec: 1800 # restart if any in-flight task runs longer than this watchdogStaleSec: 60 # restart if the watchdog heartbeat hasn't ticked in this long periodSeconds: 30 # how often kubelet runs the probe timeoutSeconds: 5 # exec probe timeout initialDelaySeconds: 120 # grace period after pod start before probing begins failureThreshold: 2 # consecutive probe failures before pod restart ``` The same Helm partial applies the probe to all three worker deployments, so a single block configures `reducto-worker`, `reducto-priority-worker`, and `reducto-gpu-worker` together. ### Defaults | Knob | Default | When to change | | ----------------------- | --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `stuckTaskThresholdSec` | `1800` (30 min) | Raise if your workload includes legitimately long single-task work (large multi-thousand-page documents, long extraction prompts). Lower if you'd rather fail fast and rely on client retries. | | `watchdogStaleSec` | `60` | Rarely needs tuning. Lower bounds how quickly an event-loop wedge is caught; should stay several times larger than the 5s tick interval to avoid false positives. | | `periodSeconds` | `30` | Lower for faster detection at the cost of more probe overhead. | | `initialDelaySeconds` | `120` | Raise if your pods take longer to come up (large image pulls, slow init containers). | | `failureThreshold` | `2` | Raise to make restart decisions more conservative. | ### Environment variables The Helm chart pipes the values above into env vars that the Python watchdog and shell probe both read, so the two stay in sync. You normally configure these via Helm, but you can override directly when running outside the chart: | Variable | Read by | Default | Purpose | | ------------------------------------- | ------------------------ | ------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `WORKER_STUCK_TASK_THRESHOLD_SEC` | Python watchdog | `1800` | Drives the per-task age comparison **and** the `logfire.warn` emitted when oldest age exceeds 80% of threshold (early breadcrumb for SRE before kubelet restarts the pod). | | `WORKER_LIVENESS_STUCK_THRESHOLD_SEC` | `bin/worker-liveness.sh` | `1800` | Per-task age limit used by the exec probe itself. Helm sets this from `worker.livenessProbe.stuckTaskThresholdSec`. | | `WORKER_LIVENESS_WATCHDOG_STALE_SEC` | `bin/worker-liveness.sh` | `60` | Maximum heartbeat-file age before the probe fails. Helm sets this from `worker.livenessProbe.watchdogStaleSec`. | | `WORKER_WATCHDOG_STATE_PATH` | Both | `/tmp/worker-state` | Heartbeat file location. Almost never needs to change. | ### Disabling the probe If you're operating in a constrained environment that can't run exec probes, or you'd rather rely on external orchestration, set: ```yaml theme={null} worker: livenessProbe: enabled: false ``` The Python watchdog still runs and emits `logfire.warn` when tasks exceed 80% of the threshold. The probe just doesn't trigger restarts. ### Verifying it's installed After deploy, confirm the probe is wired up: ```bash theme={null} kubectl describe pod -n reducto -l app=reducto-worker | grep -A 4 Liveness # Liveness: exec [bin/worker-liveness.sh] delay=120s timeout=5s period=30s ... ``` And confirm the watchdog file is being refreshed: ```bash theme={null} kubectl exec -n reducto -- cat /tmp/worker-state # 1778631873 -1 <- idle (age sentinel -1 is healthy) # 1778631878 42 <- busy with a 42-second-old task ``` # Securing Reducto Source: https://docs.reducto.ai/onprem/securing_reducto Learn how to secure on-premise Reducto deployments ## Authentication Reducto on-premise should run behind your private network boundary and should be called only by trusted services. Authentication protects access to the deployment, but Reducto does not provide per-end-user authorization inside a single on-premise tenant. Review the [on-prem security model](/onprem/security_model) before exposing Reducto to additional services or networks. Reducto can read one or more tokens used for authenticating API calls from a file. On Kubernetes, a Secret can be mounted as a file in the `http` Pod - when this Secret is updated, Kubernetes automatically updates the mounted file using an eventually-consistent approach, allowing Reducto to reload the new value without requiring a restart. Multiple API keys can be specified by placing each key on a separate line. This enables progressive secret rotation: add the new key, roll your clients over, then remove the old key, all without downtime. Enable API authentication for production on-premise deployments. If you need different permissions for different users, applications, or business units, enforce those permissions in your gateway or run separate Reducto deployments. ### Overview When enabled, Reducto reads the authentication secret from a mounted file specified by the `AUTH_SECRET_PATH` environment variable. This provides better security isolation and follows Kubernetes best practices for secret management. ### Setup instructions You can set up authentication using either manual secret creation or secret creation via Helm Chart. #### Option 1: Manual secret creation ##### 1. Create a Kubernetes secret First, create a Kubernetes secret containing your API key(s). For a single key: ```bash theme={null} kubectl create secret generic reducto-auth-secret \ --from-literal=secret=your-api-key-here \ --namespace reducto ``` For multiple keys (e.g., during rotation), separate them with newlines: ```bash theme={null} kubectl create secret generic reducto-auth-secret \ --from-literal=secret=$'current-key\nnew-key' \ --namespace reducto ``` ##### 2. Configure Helm values Add the following configuration to your Helm values file: ```yaml theme={null} auth: secretPath: enabled: true secretName: "reducto-auth-secret" mountPath: "/etc/auth" filename: "secret" ``` #### Option 2: Secret creation via Helm Chart ##### 1. Configure values for creation via Helm Chart Add the following configuration to your Helm values file to let the Helm chart create the secret: ```yaml theme={null} auth: secretPath: enabled: true createSecret: true apiKey: "your-api-key-here" # Must specify when createSecret is true. secretName: "reducto-auth-secret" mountPath: "/etc/auth" filename: "secret" ``` **Note**: With `createSecret: true` you must specify `apiKey` #### 2. Deploy Reducto (for both options) Deploy or upgrade your Reducto installation: ```bash theme={null} helm upgrade --install reducto oci://registry.reducto.ai/reducto-api/reducto \ --namespace reducto \ --values your-values.yaml ``` ### Configuration options | Parameter | Description | Default | | ------------------------------ | ---------------------------------------------------- | ----------------------- | | `auth.secretPath.enabled` | Enable file-based authentication | `false` | | `auth.secretPath.createSecret` | Create the Kubernetes secret | `false` | | `auth.secretPath.apiKey` | API key, must be specified when `createSecret: true` | - | | `auth.secretPath.secretName` | Name of the Kubernetes secret | `"reducto-auth-secret"` | | `auth.secretPath.mountPath` | Mount path for the secret file | `"/etc/auth"` | | `auth.secretPath.filename` | Filename within the secret | `"secret"` | ### How it works 1. When `auth.secretPath.enabled` is `true`, the Helm chart mounts the specified secret as a volume 2. The `AUTH_SECRET_PATH` environment variable is automatically set to the full file path 3. Reducto's authentication middleware reads all keys from the file (one per line) and authenticates API requests against any of them 4. The file is monitored for changes, supporting secret rotation without restarts 5. If the provided API key doesn't match any key in the file, a 401 error is returned ### Secret rotation To rotate keys without downtime: 1. Add the new key on a new line alongside the existing key 2. Update the Kubernetes secret (Reducto picks up the change automatically) 3. Migrate your clients to use the new key 4. Remove the old key from the secret ## Network access Do not expose Reducto directly to the public internet. Put the API behind an internal load balancer, private ingress, service mesh, or API gateway that matches your organization's access policy. Restrict access to: * Reducto API endpoints, including `/job`, `/jobs`, `/upload`, `/cancel`, `/configure_webhook`, and billing export endpoints * Metrics and Prometheus endpoints * PostgreSQL, Redis, object storage, model services, and observability UIs If users or browser clients need to initiate work, route them through your own application backend. That backend should authenticate the user, authorize the workflow, and then call Reducto as a trusted service. # On-prem security model Source: https://docs.reducto.ai/onprem/security_model Understand the shared security responsibilities and trust boundaries for Reducto on-premise deployments Reducto on-premise runs inside infrastructure that you control. Reducto provides the application, Helm chart, and deployment guidance. You control the network boundary, caller authentication, Kubernetes policy, storage policy, telemetry sinks, and access to the deployment. ## Trust boundary Reducto on-premise is designed for a single customer tenant. It should be deployed behind your private network boundary and called by trusted services inside your environment. Do not expose the Reducto API directly to the public internet or to untrusted browser clients. If users or applications outside your trusted network need access, place Reducto behind your own gateway, identity provider, firewall, and rate limiting controls. In the on-premise model, Reducto does not provide per-end-user authorization inside a single deployment. API keys authenticate access to the deployment, and resources such as jobs, uploaded files, webhook configuration, and usage data belong to the deployment's tenant. If you need isolation between business units, applications, or users, run separate deployments or enforce that isolation in your gateway before traffic reaches Reducto. ## Shared responsibility | Area | Reducto responsibility | Customer responsibility | | -------------------- | ------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------- | | Application code | Ship secure application code, fixes, and supported configuration options | Keep the deployment upgraded and apply security releases promptly | | Network exposure | Document intended network placement and internal endpoints | Keep Reducto APIs, metrics, databases, object storage, and observability UIs private unless intentionally exposed through your controls | | API authentication | Support bearer token authentication and file-based key rotation | Enable authentication, manage secrets, rotate keys, and restrict who can call the service | | Caller authorization | Enforce tenant-level access within the Reducto deployment | Enforce per-user or per-application authorization before requests reach Reducto | | Document input | Process documents and URLs provided by authenticated callers | Ensure callers only submit documents and URLs they are allowed to process | | URL fetching | Fetch customer-provided document URLs as part of document processing | Control egress, restrict metadata services, and use allowlists or network policy where needed | | Kubernetes hardening | Provide Helm values and supported runtime guidance | Apply pod security standards, network policies, ingress policy, secret management, and cluster logging policy | | Observability | Emit metadata-only logs, metrics, traces, and operational endpoints | Route telemetry to approved sinks, set retention, and restrict observability access to operators | ## API authentication Enable file-based API key authentication for production deployments. See [Securing Reducto](/onprem/securing_reducto) for setup and rotation steps. Use one or more long random bearer tokens stored in a Kubernetes Secret. Treat the token as a deployment-level credential. During rotation, configure both the current key and the new key, migrate clients, then remove the old key. If you operate multiple caller identities, do not rely on Reducto to distinguish those callers inside one on-premise deployment. Put an API gateway, service mesh, or internal service in front of Reducto to authenticate callers, authorize requested workflows, and attach audit context. ## Network placement Place Reducto services on private cluster networks. At minimum: * Expose the API only to trusted internal services or through a controlled ingress. * Keep PostgreSQL, Redis, object storage, and model services private. * Keep `/metrics`, `/prometheus`, and observability UIs private to monitoring and operations networks. * Block direct internet access to internal admin or scrape endpoints. * Use Kubernetes NetworkPolicy, cloud security groups, private endpoints, or equivalent controls to restrict east-west and egress traffic. ## Document URLs and uploads Reducto processes documents supplied by authenticated callers. This includes direct uploads, `reducto://` file IDs, presigned URLs, customer storage URLs, and internal URLs when your deployment allows access to them. Because URL fetching is part of the product, your network design decides which internal resources Reducto can reach. Treat document URL submission as a privileged capability. Only trusted services should be able to ask Reducto to fetch URLs. Recommended controls: * Prefer direct uploads or tightly scoped presigned URLs when possible. * Keep presigned URLs short-lived and limited to a single object. * Block cloud metadata endpoints and other sensitive infrastructure endpoints from worker egress. * Use egress allowlists for document source domains when your workflows allow it. * Avoid passing URLs that embed long-lived credentials. ## Internal endpoints Some endpoints are intended for tenant operators and internal automation, not public callers. Examples include job listing, webhook configuration, billing export, metrics, and Prometheus scraping endpoints. In a single-tenant deployment, these endpoints operate at the tenant boundary. They are not a substitute for per-end-user access control. Restrict them to trusted services and operators through your network and gateway policy. ## Logs, metrics, and traces Reducto emits application logs to stdout and can send traces, metrics, and logs through OpenTelemetry. See [Observability & Monitoring](/onprem/observability) for configuration. Reducto telemetry is metadata-only observability, not a data pipeline. Logs, traces, and metrics should not include document text, OCR output, extracted values, prompts, model responses, API tokens, access keys, secrets, or other customer content. File names and URLs can reveal customer identity and should be redacted or avoided in telemetry. You own the retention period, sink selection, access policy, and export policy for on-premise telemetry. Telemetry may include job IDs, counts, statuses, durations, sanitized error categories, and other debugging metadata. Route it only to systems approved by your organization. ## Deployment checklist * Enable API authentication with `auth.secretPath.enabled`. * Put Reducto behind a private ingress or internal gateway. * Document which services are allowed to call Reducto. * Restrict metrics, Prometheus, HyperDX, PostgreSQL, Redis, object storage, and model endpoints to internal networks. * Configure egress controls for URL fetching and block metadata services. * Use Kubernetes Secrets or your secret manager for API keys and provider credentials. * Configure telemetry retention, routing, and access controls. * Keep the Helm chart and Reducto image current with security releases. * Run separate deployments or gateway-level authorization if you need isolation between internal callers. # Air-gapped usage for billing Source: https://docs.reducto.ai/onprem/usage_for_billing Export usage data from air-gapped environments for billing purposes ## Overview Usage data from Reducto deployments in air-gapped environments can be exported as a compressed CSV file and sent to Reducto for billing purposes. When you invoke the `/billing-usage` API, usage data is exported from the database to a file and uploaded to the `BUCKET/usage_for_billing` prefix in object storage. A presigned URL is returned, which you can use to download the file and send it to Reducto. **Note**: To avoid data loss, ensure that the exported file is downloaded and shared before any lifecycle rules expire and delete the object. ## Enable usage persistence To export usage data, it must first be persisted in the database. This can be enabled by setting the following environment variable: ```bash theme={null} STORE_USAGE_FOR_BILLING="yes" ``` ## API reference ### POST `/billing-usage` Exports usage data to object storage and returns a presigned URL for download. #### Request parameters | Parameter | Type | Default | Description | | --------- | ------- | --------- | ------------------------------------------------------ | | `count` | integer | 1,000,000 | The number of records to fetch and export (minimum: 1) | | `delete` | boolean | false | Whether to delete the records after exporting | #### Request example ```bash theme={null} curl -X POST "https:///billing-usage" \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "count": 100000, "delete": false }' ``` #### Response format ```json theme={null} { "presigned_url": "" } ``` If usage persistence is not enabled or no data is available, the response will be: ```json theme={null} { "presigned_url": null } ``` ## Exported data format The exported CSV file contains the following columns: * `idempotency_key`: Unique identifier for each usage record * `type`: Type of usage (e.g., "parse") * `num_pages`: Number of pages processed * `attributes`: Additional metadata as JSON * `timestamp`: When the usage occurred (ISO format) ## Usage workflow 1. **Enable persistence**: Set `STORE_USAGE_FOR_BILLING="yes"` in your environment 2. **Process documents**: Use Reducto normally; usage data will be automatically stored 3. **Export all data**: Call the `/billing-usage` API repeatedly with `"delete": true` until the presigned URL is null 4. **Download files**: Use each returned presigned URL to download the compressed CSV files 5. **Send to Reducto**: Share all downloaded files with Reducto for billing ### Complete data export process To ensure all usage data is exported, you must call the API repeatedly with `delete: true` until no more data remains: ```bash theme={null} set -eu REDUCTO_ENDPOINT="https://" REDUCTO_API_KEY="your-api-key" # Call this repeatedly until presigned_url is null while true; do echo "Exporting ..." response=$(curl -s -X POST "${REDUCTO_ENDPOINT}/billing-usage" \ -H "Authorization: Bearer ${REDUCTO_API_KEY}" \ -H "Content-Type: application/json" \ -d '{"count": 1000000, "delete": true}') presigned_url=$(echo "$response" | jq -r '.presigned_url') if [ "$presigned_url" = "null" ]; then echo "All data exported successfully" break fi echo "Downloading: $presigned_url" # Download the file using the presigned URL curl -OJ "$presigned_url" done ``` **Important**: Always use `"delete": true` when performing complete exports to avoid re-exporting the same data and to free up database storage. ## Important notes * Records are exported in chronological order (oldest first) * The presigned URL expires after 6 days or until lifecycle rule deletes the object * If `delete: true` is used, only successfully exported records are deleted from database * To regenerate presigned URL of already exported data visit `BUCKET/usage_for_billing` prefix on cloud console for Bucket # Overview Source: https://docs.reducto.ai/overview The agentic document platform for leading AI teams Reducto is the agentic document platform for AI teams who need production-grade document processing at enterprise scale. It provides a complete toolkit for the full document lifecycle, from classification and extraction to editing and workflow orchestration, powered by custom in-house and leading frontier models. **What you can build:** * **Automated intake pipelines** that classify, split, and route documents without manual triage * **Structured data extraction** from invoices, contracts, medical records, and any document type at scale * **Document generation and editing** that fills forms, modifies templates, and produces new documents programmatically * **RAG-ready content pipelines** with layout-aware chunking optimized for LLM consumption * **Multi-step workflows** that chain classification, parsing, extraction, and editing into single API calls *** ## Platform capabilities Reducto covers the full lifecycle of document work. Each capability is available as a standalone API endpoint or composed into pipelines. Route documents by type before processing. Define categories in natural language. Convert documents into structured JSON with text, tables, and figures, with layout-aware chunking. Pull specific fields into structured JSON using a schema. Define what you need, get exactly that. Divide documents into logical sections using natural language descriptions. Fill PDF forms and modify DOCX files programmatically with natural language instructions. Chain multiple steps into reusable, single-call workflows deployed from Studio. Supports [30+ file types](/upload/overview#supported-file-types) including PDFs, images, spreadsheets, presentations, and scanned documents. *** ## How to use Reducto For developers building automated pipelines. Available in Python, Node.js, Go, and REST. For visual pipeline building. Configure, test, and deploy document workflows in your browser. **Agent-ready tooling:** Reducto integrates directly into AI agent workflows via the [MCP server](/mcp-server), [CLI](/cli), and native SDKs. One-page reference for coding agents using Reducto. Give agents Reducto tools directly in Claude Code, Codex, Cursor, and other MCP clients. Parse local files and folders from a terminal with minimal setup. *** ## Performance Reducto orchestrates a pipeline of specialized models, including custom in-house models and frontier VLMs, with agentic multipasses that correct errors iteratively. This architecture delivers accuracy on the long tail of real-world documents: handwritten forms, rotated pages, nested tables, multi-column layouts, and degraded scans. Every result links to the [Studio citation viewer](https://studio.reducto.ai) where you can inspect outputs against source documents at the bounding-box level. *** ## Built for production Audited security controls. Signed BAA available. Documents deleted within 24h. * **Deployment flexibility.** SaaS, hybrid VPC, full VPC, or air-gapped on-premises * **Scale.** 3B+ pages processed, with autoscaling for production workloads. * **Security.** Encryption at rest (AES-256) and in transit (TLS 1.2+). EU data residency. [Learn more →](/security/policies) * **Support.** Dedicated field engineering, custom model fine-tuning, and 24/7 oncall for Enterprise customers [See Enterprise readiness →](/enterprise/enterprise-readiness) *** ## Get started Parse your first document in 5 minutes with Python, Node.js, Go, or REST. Build and deploy an extraction pipeline visually, no code required. # Parse Best Practices Source: https://docs.reducto.ai/parse/best-practices Key practices for getting the best results from Parse ## 1. Use Variable Chunking for RAG The default chunking mode (`disabled`) returns the entire document as one chunk. For RAG applications you need smaller chunks that can be embedded and retrieved independently. Variable chunking splits at semantic boundaries like section headers, tables, and figures, keeping related content together while creating chunks sized for embedding models. ```python Python theme={null} result = client.parse.run( input=upload.file_id, retrieval={ "chunking": {"chunk_mode": "variable"}, "embedding_optimized": True } ) # Use embed field for vector database, content for display for chunk in result.result.chunks: vector_db.insert( embedding=embed(chunk.embed), metadata={"content": chunk.content} ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, retrieval: { chunking: { chunk_mode: 'variable' }, embedding_optimized: true } }); // Use embed field for vector database, content for display for (const chunk of result.result.chunks) { await vectorDb.insert({ embedding: embed(chunk.embed), metadata: { content: chunk.content } }); } ``` ```go Go theme={null} result, _ := client.Parse.Run(context.Background(), reducto.ParseRunParams{ ParseConfig: reducto.ParseConfigParam{ DocumentURL: reducto.F[reducto.ParseConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), }, Retrieval: reducto.F(reducto.RetrievalParam{ Chunking: reducto.F(reducto.RetrievalChunkingUnionParam{ OfVariableChunking: &reducto.VariableChunkingConfigParam{ ChunkMode: reducto.F(reducto.VariableChunkingConfigChunkModeVariable), }, }), EmbeddingOptimized: reducto.F(true), }), }) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "retrieval": { "chunking": {"chunk_mode": "variable"}, "embedding_optimized": true } }' ``` The `embed` field contains table and figure summaries as natural language, which embeds better than raw Markdown tables. The `content` field preserves the original formatting for display. *** ## 2. Enable Agentic Mode Only When Needed Agentic mode uses an LLM to review and correct parsing output. It adds latency with additional credit usage, so only enable it when needed. **When to enable `scope: "text"`:** * Handwritten documents or signatures * Faded or low-quality scans * Documents with unusual fonts * When you see garbled characters in output **When to enable `scope: "table"`:** * Tables with misaligned columns after parsing * Merged cells that didn't parse correctly * Numbers appearing in wrong columns * Financial documents where accuracy is critical **When to enable `scope: "figure"`:** * Charts and graphs that need data extraction * [Advanced chart extraction](https://reducto.ai/blog/reducto-chart-extraction) with structured data output * Diagrams requiring detailed descriptions * Visual elements where you need numeric data from bar charts, line graphs, or pie charts ```python Python theme={null} result = client.parse.run( input=upload.file_id, enhance={ "agentic": [ {"scope": "text"}, {"scope": "table"}, {"scope": "figure", "advanced_chart_agent": True} ] } ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, enhance: { agentic: [ { scope: 'text' }, { scope: 'table' }, { scope: 'figure', advanced_chart_agent: true } ] } }); ``` ```go Go theme={null} result, _ := client.Parse.Run(context.Background(), reducto.ParseRunParams{ ParseConfig: reducto.ParseConfigParam{ DocumentURL: reducto.F[reducto.ParseConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), }, Enhance: reducto.F(reducto.EnhanceParam{ Agentic: reducto.F([]reducto.AgenticScopeParam{ {Scope: reducto.F(reducto.AgenticScopeScopeText)}, {Scope: reducto.F(reducto.AgenticScopeScopeTable)}, {Scope: reducto.F(reducto.AgenticScopeScopeFigure), AdvancedChartAgent: reducto.F(true)}, }), }), }) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "enhance": { "agentic": [ {"scope": "text"}, {"scope": "table"}, {"scope": "figure", "advanced_chart_agent": true} ] } }' ``` Clean digital PDFs (native text, not scanned) parse correctly without agentic mode. Test your document types without it first, then enable selectively. *** ## 3. Set Priority for Async Requests Parse has sync (`/parse`) and async (`/parse_async`) endpoints. **Async requests without `priority: true` enter a queue** and may experience delays during high traffic. If you're using async for latency-sensitive requests (user-facing features, real-time processing), always set priority. ```python Python theme={null} job = client.parse.run_job( input=upload.file_id, async_config={"priority": True} ) ``` ```javascript Node.js theme={null} const job = await client.parse.runJob({ input: upload.file_id, asyncConfig: { priority: true } }); ``` ```go Go theme={null} job, _ := client.Parse.RunJob(context.Background(), reducto.ParseRunJobParams{ ParseConfig: reducto.ParseConfigParam{ DocumentURL: reducto.F[reducto.ParseConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), }, Options: reducto.F(reducto.ParseRunJobParamsOptionsParam{ Priority: reducto.F(true), }), }) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse_async \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "options": {"priority": true} }' ``` Use async with priority or sync for documents that need speed. Use async without priority for batch processing where latency doesn't matter. *** ## 4. Use HTML for Complex Tables The default table format (`dynamic`) auto-selects HTML or Markdown based on complexity. For documents with complex tables (merged cells, nested headers, multi-row cells), explicitly request HTML. ```python Python theme={null} result = client.parse.run( input=upload.file_id, formatting={ "table_output_format": "html" } ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, formatting: { table_output_format: 'html' } }); ``` ```go Go theme={null} result, _ := client.Parse.Run(context.Background(), reducto.ParseRunParams{ ParseConfig: reducto.ParseConfigParam{ DocumentURL: reducto.F[reducto.ParseConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), }, Formatting: reducto.F(reducto.FormattingParam{ TableOutputFormat: reducto.F(reducto.FormattingTableOutputFormatHTML), }), }) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "formatting": { "table_output_format": "html" } }' ``` Markdown tables can't represent merged cells or complex structures. If your tables look broken, switching to HTML usually fixes it. For programmatic access to cell data, use `json` format instead. *** ## 5. Filter Headers and Footers for RAG Page headers, footers, and page numbers add noise to RAG retrieval. When a user asks about invoice totals, you don't want to retrieve chunks containing "Page 1 of 5" or "Confidential - Do Not Distribute". ```python Python theme={null} result = client.parse.run( input=upload.file_id, retrieval={ "filter_blocks": ["Header", "Footer", "Page Number"] } ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, retrieval: { filter_blocks: ['Header', 'Footer', 'Page Number'] } }); ``` ```go Go theme={null} result, _ := client.Parse.Run(context.Background(), reducto.ParseRunParams{ ParseConfig: reducto.ParseConfigParam{ DocumentURL: reducto.F[reducto.ParseConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), }, Retrieval: reducto.F(reducto.RetrievalParam{ FilterBlocks: reducto.F([]reducto.RetrievalFilterBlock{ reducto.RetrievalFilterBlockHeader, reducto.RetrievalFilterBlockFooter, reducto.RetrievalFilterBlockPageNumber, }), }), }) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "retrieval": { "filter_blocks": ["Header", "Footer", "Page Number"] } }' ``` The filtered blocks still appear in `chunks[].blocks` metadata (so you can access them if needed), but they're excluded from `content` and `embed` fields. *** ## Common Pitfalls If you're using agentic mode, it adds latency since it runs an LLM pass over the output. Disable it and only enable for document types that actually need correction. For async calls, make sure you have `priority: true` set. Check your chunking. If you're using `disabled` (default), the entire document is one chunk. Switch to `variable` for semantic chunking. Switch from `dynamic` to `html` format. Markdown can't handle merged cells. You forgot to set `priority: true`. Without it, jobs enter a queue. The PDF file may be malformed or use unsupported encryption. Try opening the file in a PDF viewer to verify it's valid. If the file opens but Reducto fails, the PDF may use non-standard formatting. Re-save it using a tool like Adobe Acrobat or a PDF printer, then retry. *** ## Configuration Reference For complete details on all options mentioned above, see the dedicated configuration pages: All chunking modes and their use cases. When and how to use LLM-assisted parsing. HTML, Markdown, JSON, CSV options. Full reference of all configuration options. *** ## Related Quick start and basic usage. Understanding chunks, blocks, and bounding boxes. # Parse Source: https://docs.reducto.ai/parse/overview Convert documents into structured JSON with text, tables, and figures Parse is Reducto's foundational endpoint. It converts documents into structured JSON by running OCR, detecting layout (headers, paragraphs, tables, figures), and organizing content into chunks ready for LLM consumption, RAG pipelines, or downstream extraction. Each element includes its type, page position, and confidence score. Parse handles multi-column text, nested tables, forms with handwriting, rotated pages, and documents mixing text with charts and images, using agentic VLM multipasses to correct errors on difficult content. **Try it live:** See Parse in action with a [sample bank statement in Reducto Studio](https://studio.reducto.ai). **File size limits:** Upload files up to 100MB directly via the [Upload endpoint](/upload), or up to 5GB via [presigned URL](/upload/large-files). You can also pass public URLs or presigned S3/GCS/Azure URLs directly. *** ## Quick Start ```python Python theme={null} from pathlib import Path from reducto import Reducto client = Reducto() upload = client.upload(file=Path("invoice.pdf")) result = client.parse.run(input=upload.file_id) for chunk in result.result.chunks: print(chunk.content) ``` ```javascript Node.js theme={null} import Reducto from 'reductoai'; import fs from 'fs'; const client = new Reducto(); const upload = await client.upload({ file: fs.createReadStream('invoice.pdf'), }); const result = await client.parse.run({ input: upload.file_id }); for (const chunk of result.result.chunks) { console.log(chunk.content); } ``` ```go Go theme={null} package main import ( "context" "fmt" "io" "os" reducto "github.com/reductoai/reducto-go-sdk" "github.com/reductoai/reducto-go-sdk/option" "github.com/reductoai/reducto-go-sdk/shared" ) func main() { client := reducto.NewClient(option.WithAPIKey(os.Getenv("REDUCTO_API_KEY"))) file, _ := os.Open("invoice.pdf") defer file.Close() upload, _ := client.Upload(context.Background(), reducto.UploadParams{ File: reducto.F[io.Reader](file), }) result, _ := client.Parse.Run(context.Background(), reducto.ParseRunParams{ ParseConfig: reducto.ParseConfigParam{ DocumentURL: reducto.F[reducto.ParseConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), }, }) if result.Result.Type == shared.ParseResponseResultTypeFull { chunks := result.Result.Chunks.([]shared.ParseResponseResultFullResultChunk) for _, chunk := range chunks { fmt.Println(chunk.Content) } } } ``` ```bash cURL theme={null} # Upload FILE_ID=$(curl -s -X POST https://platform.reducto.ai/upload \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -F "file=@invoice.pdf" | jq -r '.file_id') # Parse curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d "{\"input\": \"$FILE_ID\"}" ``` *** ## What You Get Back ```json theme={null} { "job_id": "7600c8c5-a52f-49d2-8a7d-d75d1b51e141", "duration": 3.89, "result": { "type": "full", "chunks": [ { "content": "# Invoice\n\nBill To: Acme Corp\n123 Main St...", "embed": "# Invoice\n\nBill To: Acme Corp...", "blocks": [ { "type": "Title", "content": "Invoice", "bbox": { "left": 0.1, "top": 0.05, "width": 0.3, "height": 0.04, "page": 1 }, "confidence": "high" } ] } ] }, "usage": { "num_pages": 1, "credits": 2.0 }, "studio_link": "https://studio.reducto.ai/job/7600c8c5-..." } ``` **Key fields:** | Field | What it is | | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `chunks[].content` | The extracted content, formatted as Markdown (headers become `#`, tables become Markdown/HTML tables). Ready to pass to an LLM. | | `chunks[].embed` | Same content but optimized for embeddings. When figure/table summaries are enabled, this field contains natural language descriptions instead of raw table markup. | | `chunks[].blocks` | The individual elements (paragraphs, tables, figures) with their positions and types. Useful for highlighting or linking back to source. | | `result.type` | Either `"full"` (content inline) or `"url"` (content at a URL). Large documents return `"url"` to avoid HTTP size limits. | Full breakdown of chunks, blocks, bounding boxes, and confidence scores. *** ## Input Options The `input` field accepts four formats: 1. **Upload response** (`reducto://...`): After uploading via `/upload`, use the returned `file_id`. This is the most common method for local files. 2. **Public URL**: Any publicly accessible URL. Reducto fetches the file directly. 3. **Presigned URL**: S3, GCS, or Azure Blob presigned URLs work. Useful when files are in your cloud storage. 4. **Previous job ID** (`jobid://...`): Reprocess a document from a previous parse job without re-uploading. Useful for testing different configurations. ```python Python theme={null} # From upload result = client.parse.run(input=upload.file_id) # Public URL result = client.parse.run(input="https://example.com/doc.pdf") # Presigned S3 URL result = client.parse.run(input="https://bucket.s3.amazonaws.com/doc.pdf?X-Amz-...") # Reprocess previous job result = client.parse.run(input="jobid://7600c8c5-a52f-49d2-8a7d-d75d1b51e141") ``` ```javascript Node.js theme={null} // From upload const result = await client.parse.run({ input: upload.file_id }); // Public URL const result = await client.parse.run({ input: 'https://example.com/doc.pdf' }); // Presigned S3 URL const result = await client.parse.run({ input: 'https://bucket.s3.amazonaws.com/doc.pdf?X-Amz-...' }); // Reprocess previous job const result = await client.parse.run({ input: 'jobid://7600c8c5-a52f-49d2-8a7d-d75d1b51e141' }); ``` ```go Go theme={null} // From upload result, _ := client.Parse.Run(context.Background(), reducto.ParseRunParams{ ParseConfig: reducto.ParseConfigParam{ DocumentURL: reducto.F[reducto.ParseConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), }, }) // Public URL result, _ := client.Parse.Run(context.Background(), reducto.ParseRunParams{ ParseConfig: reducto.ParseConfigParam{ DocumentURL: reducto.F[reducto.ParseConfigDocumentURLUnionParam]( shared.UnionString("https://example.com/doc.pdf"), ), }, }) ``` ```bash cURL theme={null} # From upload curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{"input": "reducto://your-file-id"}' # Public URL curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{"input": "https://example.com/doc.pdf"}' # Reprocess previous job curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{"input": "jobid://7600c8c5-a52f-49d2-8a7d-d75d1b51e141"}' ``` *** ## Sync vs Async Parse has both synchronous (`/parse`) and asynchronous (`/parse_async`) endpoints. Use async for large documents or when you need webhook delivery. When to use each, how priority works, webhook setup. *** ## Configuration Parse has several configuration groups. Here are the most commonly changed options: ### Chunking By default, Parse returns the entire document as one chunk. For RAG applications, you want smaller chunks that can be embedded and retrieved independently. ```python Python theme={null} result = client.parse.run( input=upload.file_id, retrieval={ "chunking": {"chunk_mode": "variable"} } ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, retrieval: { chunking: { chunk_mode: 'variable' } } }); ``` ```go Go theme={null} result, _ := client.Parse.Run(context.Background(), reducto.ParseRunParams{ ParseConfig: reducto.ParseConfigParam{ DocumentURL: reducto.F[reducto.ParseConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), }, Retrieval: reducto.F(reducto.RetrievalParam{ Chunking: reducto.F(reducto.RetrievalChunkingUnionParam{ OfVariableChunking: &reducto.VariableChunkingConfigParam{ ChunkMode: reducto.F(reducto.VariableChunkingConfigChunkModeVariable), }, }), }), }) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "retrieval": { "chunking": {"chunk_mode": "variable"} } }' ``` | Mode | Behavior | | ---------- | ------------------------------------------------------------------------------------ | | `disabled` | One chunk for the whole document (default) | | `variable` | Splits at semantic boundaries (sections, tables, figures stay intact). Best for RAG. | | `page` | One chunk per page | | `section` | Splits at section headers | [Full chunking options →](/configs/parse/chunking-methods) ### Table Output Format Controls how tables appear in the output. ```python Python theme={null} result = client.parse.run( input=upload.file_id, formatting={ "table_output_format": "html" } ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, formatting: { table_output_format: 'html' } }); ``` ```go Go theme={null} result, _ := client.Parse.Run(context.Background(), reducto.ParseRunParams{ ParseConfig: reducto.ParseConfigParam{ DocumentURL: reducto.F[reducto.ParseConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), }, Formatting: reducto.F(reducto.FormattingParam{ TableOutputFormat: reducto.F(reducto.FormattingTableOutputFormatHTML), }), }) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "formatting": { "table_output_format": "html" } }' ``` | Format | When to use | | --------- | ----------------------------------------------------------------- | | `dynamic` | Auto-selects HTML or Markdown based on table complexity (default) | | `html` | Complex tables with merged cells, nested headers | | `md` | Simple tables, Markdown-based workflows | | `json` | Programmatic processing, need cell-level access | | `csv` | Export to spreadsheets | [Full table format options →](/configs/parse/table-output-formats) ### Figure Summaries By default, Parse uses a vision model to generate descriptions for figures and images. This helps with RAG (the `embed` field contains the description) but adds latency. ```python Python theme={null} result = client.parse.run( input=upload.file_id, enhance={ "summarize_figures": True } ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, enhance: { summarize_figures: true } }); ``` ```go Go theme={null} result, _ := client.Parse.Run(context.Background(), reducto.ParseRunParams{ ParseConfig: reducto.ParseConfigParam{ DocumentURL: reducto.F[reducto.ParseConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), }, Enhance: reducto.F(reducto.EnhanceParam{ SummarizeFigures: reducto.F(true), }), }) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "enhance": { "summarize_figures": true } }' ``` ### Agentic Mode Uses an LLM to review and correct parsing output. Adds latency with additional credit usage. Enable it when: * **`scope: "text"`**: Handwritten text, faded scans, documents with unusual fonts, or when you see garbled characters in the output. * **`scope: "table"`**: Tables with misaligned columns, merged cells that didn't parse correctly, or numbers that appear in wrong columns. * **`scope: "figure"`**: Charts and graphs that need data extraction, including [advanced chart extraction](https://reducto.ai/blog/reducto-chart-extraction) with structured data output. ```python Python theme={null} result = client.parse.run( input=upload.file_id, enhance={ "agentic": [ {"scope": "text"}, {"scope": "table"}, {"scope": "figure", "advanced_chart_agent": True} ] } ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, enhance: { agentic: [ { scope: 'text' }, { scope: 'table' }, { scope: 'figure', advanced_chart_agent: true } ] } }); ``` ```go Go theme={null} result, _ := client.Parse.Run(context.Background(), reducto.ParseRunParams{ ParseConfig: reducto.ParseConfigParam{ DocumentURL: reducto.F[reducto.ParseConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), }, Enhance: reducto.F(reducto.EnhanceParam{ Agentic: reducto.F([]reducto.AgenticScopeParam{ {Scope: reducto.F(reducto.AgenticScopeScopeText)}, {Scope: reducto.F(reducto.AgenticScopeScopeTable)}, {Scope: reducto.F(reducto.AgenticScopeScopeFigure), AdvancedChartAgent: reducto.F(true)}, }), }), }) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "enhance": { "agentic": [ {"scope": "text"}, {"scope": "table"}, {"scope": "figure", "advanced_chart_agent": true} ] } }' ``` Don't enable for clean digital PDFs (native text, not scanned). They parse correctly without it and you'll just add latency. ### Filter Blocks Remove specific content types from the output. The blocks still appear in `blocks` metadata but are excluded from `content` and `embed`. ```python Python theme={null} result = client.parse.run( input=upload.file_id, retrieval={ "filter_blocks": ["Header", "Footer", "Page Number"] } ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, retrieval: { filter_blocks: ['Header', 'Footer', 'Page Number'] } }); ``` ```go Go theme={null} result, _ := client.Parse.Run(context.Background(), reducto.ParseRunParams{ ParseConfig: reducto.ParseConfigParam{ DocumentURL: reducto.F[reducto.ParseConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), }, Retrieval: reducto.F(reducto.RetrievalParam{ FilterBlocks: reducto.F([]reducto.RetrievalFilterBlock{ reducto.RetrievalFilterBlockHeader, reducto.RetrievalFilterBlockFooter, reducto.RetrievalFilterBlockPageNumber, }), }), }) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "retrieval": { "filter_blocks": ["Header", "Footer", "Page Number"] } }' ``` Useful for RAG when headers/footers would pollute search results. ### Page Range Process only specific pages. ```python Python theme={null} result = client.parse.run( input=upload.file_id, settings={ "page_range": {"start": 1, "end": 10} } ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, settings: { page_range: { start: 1, end: 10 } } }); ``` ```go Go theme={null} result, _ := client.Parse.Run(context.Background(), reducto.ParseRunParams{ ParseConfig: reducto.ParseConfigParam{ DocumentURL: reducto.F[reducto.ParseConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), }, Settings: reducto.F(reducto.SettingsParam{ PageRange: reducto.F(reducto.PageRangeParam{ Start: reducto.F(int64(1)), End: reducto.F(int64(10)), }), }), }) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "settings": { "page_range": {"start": 1, "end": 10} } }' ``` ### Return Images Get image URLs for figures and tables in the document. ```python Python theme={null} result = client.parse.run( input=upload.file_id, settings={ "return_images": ["figure", "table"] } ) # Access images from blocks for chunk in result.result.chunks: for block in chunk.blocks: if block.image_url: print(f"{block.type}: {block.image_url}") ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, settings: { return_images: ['figure', 'table'] } }); // Access images from blocks for (const chunk of result.result.chunks) { for (const block of chunk.blocks) { if (block.image_url) { console.log(`${block.type}: ${block.image_url}`); } } } ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "settings": { "return_images": ["figure", "table"] } }' ``` Options: `["figure"]`, `["table"]`, or `["figure", "table"]`. By default, no images are returned. ### Additional Settings | Setting | Default | Description | | ----------------------------- | ------- | ------------------------------------------------------------------ | | `persist_results` | `false` | Keep results indefinitely instead of expiring after 24 hours | | `timeout` | `null` | Custom timeout in seconds for processing | | `force_url_result` | `false` | Always return results as a URL (useful for consistent handling) | | `embed_pdf_metadata` | `false` | Embed OCR metadata into returned PDF | | `extract_document_properties` | `false` | Return properties embedded in the original pre-conversion document | Set `extract_document_properties` to `true` to include a top-level `document_properties` object in the response. The setting defaults to `false` and reads metadata from the original file before conversion. ```python Python theme={null} result = client.parse.run( input=upload.file_id, settings={"extract_document_properties": True} ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, settings: { extract_document_properties: true } }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "settings": {"extract_document_properties": true} }' ``` The response includes `document_properties` for supported files with embedded properties. It is `null` when no properties are found or the input format is unsupported. Supported formats are PDF, DOCX, XLSX, and PPTX. Legacy binary `.doc`, `.xls`, and `.ppt` files are not supported. All fields are nullable, and dates use timezone-aware ISO-8601 strings. PDF Info/XMP populates `title`, `author`, `subject`, `keywords`, `creator`, `producer`, `created_at`, and `modified_at`. `last_modified_by` is only available from OOXML core properties. DOCX, XLSX, and PPTX can populate `title`, `author`, `subject`, `keywords`, `creator`, `last_modified_by`, `created_at`, and `modified_at`; `producer` is PDF-only. ```json theme={null} { "document_properties": { "title": "Quarterly Report", "author": "Jane Smith", "subject": "Financial results", "keywords": "finance, quarterly", "creator": "Microsoft Word", "producer": null, "last_modified_by": "Alex Chen", "created_at": "2024-01-15T09:30:00+00:00", "modified_at": "2024-02-01T16:45:12+00:00" } } ``` ```python theme={null} result = client.parse.run( input=upload.file_id, settings={ "persist_results": True, "timeout": 120, "force_url_result": True } ) ``` For complete configuration reference including OCR settings, spreadsheet options, and more, see the [Configuration section](/configs/parse/ocr-settings). *** ## Troubleshooting Try `formatting.table_output_format: "html"`. HTML handles merged cells and complex headers better than Markdown. Still broken? Enable `enhance.agentic: [{"scope": "table"}]` to use an LLM for alignment fixes. Main causes: * `enhance.agentic` can add latency with higher accuracy * `enhance.summarize_figures` adds latency with figures * Large documents take longer linearly * `async_priority` should be True for faster priority processing For fastest processing, disable what you don't need. See [Best Practices](/parse/best-practices). ```python Python theme={null} result = client.parse.run( input=upload.file_id, settings={"document_password": "your-password"} ) ``` ```javascript Node.js theme={null} const result = await client.parse.run({ input: upload.file_id, settings: { document_password: 'your-password' } }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-file-id", "settings": {"document_password": "your-password"} }' ``` Large documents return `result.type: "url"` instead of inline content to avoid HTTP size limits. Fetch the content: ```python Python theme={null} import requests if result.result.type == "url": chunks = requests.get(result.result.url).json() else: chunks = result.result.chunks ``` ```javascript Node.js theme={null} let chunks; if (result.result.type === 'url') { const response = await fetch(result.result.url); chunks = await response.json(); } else { chunks = result.result.chunks; } ``` ```bash cURL theme={null} # If result.type is "url", fetch from the URL curl -s "$RESULT_URL" | jq '.chunks' ``` To always get a URL (consistent handling): `settings.force_url_result: true` *** ## Next Steps Full breakdown of chunks, blocks, and bounding boxes. Optimization by document type, latency tips. # Parse Response Format Source: https://docs.reducto.ai/parse/response-format Understanding chunks, blocks, and bounding boxes Parse returns structured content in a format optimized for RAG and LLM applications. This page explains every field in the response. *** ## Response Structure ```json theme={null} { "job_id": "7600c8c5-a52f-49d2-8a7d-d75d1b51e141", "duration": 3.89, "result": { "type": "full", "chunks": [ ... ] }, "usage": { "num_pages": 1, "credits": 2.0 }, "pdf_url": "https://...", "studio_link": "https://studio.reducto.ai/job/..." } ``` ### Top-Level Fields | Field | Type | Description | | ----------------- | ------- | ------------------------------------------- | | `job_id` | string | Unique identifier for this job | | `duration` | number | Processing time in seconds | | `result` | object | Contains parsed content (see below) | | `usage.num_pages` | integer | Number of pages processed | | `usage.credits` | number | Credits consumed | | `pdf_url` | string | Temporary URL to download the processed PDF | | `studio_link` | string | Link to view results in Reducto Studio | *** ## Result Types: Full vs URL Parse can return results in two ways: ```json theme={null} { "result": { "type": "full", "chunks": [ { "content": "...", "blocks": [...] } ] } } ``` **When returned:** Documents under \~6MB response size (most documents). **How to use:** Access `result.chunks` directly. ```json theme={null} { "result": { "type": "url", "url": "https://storage.reducto.ai/chunks/abc123.json" } } ``` **When returned:** Large documents that would exceed HTTP response limits. **How to use:** Fetch the JSON from `result.url`. ```python Python theme={null} import requests if result.result.type == "url": chunks_data = requests.get(result.result.url).json() else: chunks_data = result.result.chunks ``` ```javascript Node.js theme={null} let chunksData; if (result.result.type === 'url') { const response = await fetch(result.result.url); chunksData = await response.json(); } else { chunksData = result.result.chunks; } ``` ```go Go theme={null} var chunksData interface{} if result.Result.Type == shared.ParseResponseResultTypeURL { resp, _ := http.Get(result.Result.URL) defer resp.Body.Close() json.NewDecoder(resp.Body).Decode(&chunksData) } else { chunksData = result.Result.Chunks } ``` ```bash cURL theme={null} # Check if result.type is "url" and fetch if needed RESULT_TYPE=$(echo $RESPONSE | jq -r '.result.type') if [ "$RESULT_TYPE" = "url" ]; then CHUNKS=$(curl -s $(echo $RESPONSE | jq -r '.result.url')) else CHUNKS=$(echo $RESPONSE | jq '.result.chunks') fi ``` To always receive a URL (useful for consistent handling), set `force_url_result: true` in your request. **Result URLs expire after 1 hour.** Download or process the content promptly after receiving the response. This applies to: * `result.url` (when `type: "url"`) * `pdf_url` (processed PDF) * `image_url` on blocks (when `return_images` is enabled) *** ## Understanding Chunks Chunks are the primary output unit, optimized for RAG and embedding workflows. ```json theme={null} { "content": "# Invoice\n\nBill To: Acme Corp\n123 Main St...", "embed": "# Invoice\n\nBill To: Acme Corp\n123 Main St...", "blocks": [ ... ], "enriched": null, "enrichment_success": false } ``` | Field | Description | | -------------------- | ---------------------------------------------------------------- | | `content` | Markdown-formatted content of this chunk | | `embed` | Embedding-optimized version (may include table/figure summaries) | | `blocks` | Array of individual content blocks with positions | | `enriched` | AI-enriched content (when enrich config enabled) | | `enrichment_success` | Whether enrichment completed successfully | ### content vs embed * **`content`**: Raw extracted content, preserves original text * **`embed`**: Optimized for vector embeddings, may include: * Table summaries (natural language descriptions) * Figure summaries (AI-generated descriptions) **For RAG:** Use `embed` for your vector database, `content` for display. *** ## Understanding Blocks Blocks are the atomic content elements within each chunk. Every paragraph, table, header, and figure is a separate block. ```json theme={null} { "type": "Table", "content": "...
DateAmount
", "bbox": { "left": 0.076, "top": 0.427, "width": 0.834, "height": 0.432, "page": 1, "original_page": 1 }, "confidence": "high", "granular_confidence": { "parse_confidence": 0.834, "extract_confidence": null }, "image_url": null } ``` ### Block Types | Type | Description | Example Content | | ---------------- | ----------------------- | -------------------------------- | | `Title` | Document title | "Invoice #12345" | | `Section Header` | Section headings | "Payment Terms" | | `Header` | Page headers | "Page 1 of 5" | | `Footer` | Page footers | "Confidential" | | `Text` | Body paragraphs | "Thank you for your business..." | | `Table` | Tabular data | HTML/Markdown table | | `Figure` | Images and charts | Caption or AI description | | `Key Value` | Label-value pairs | "Total: \$1,234.56" | | `List Item` | Bulleted/numbered items | "• First item" | | `Checkbox` | Form checkboxes | "☑ Agree to terms" | ### Block Fields | Field | Type | Description | | --------------------- | ------------ | ----------------------------------------------- | | `type` | string | Block type (see table above) | | `content` | string | The actual content | | `bbox` | object | Position and size on the page | | `confidence` | string | "high" or "low" | | `granular_confidence` | object | Numeric confidence scores | | `image_url` | string\|null | URL to block image (if `return_images` enabled) | *** ## Bounding Box Coordinates Every block includes a `bbox` object describing its position on the page. ```json theme={null} { "left": 0.076, "top": 0.427, "width": 0.834, "height": 0.432, "page": 1, "original_page": 1 } ``` ### Coordinate System All coordinates are **normalized to \[0, 1]** relative to page dimensions: ``` (0,0) ─────────────────────────── (1,0) │ │ │ ┌─────────────────┐ │ │ │ Block │ │ │ │ left=0.1 │ │ │ │ top=0.2 │ │ │ │ width=0.5 │ │ │ │ height=0.3 │ │ │ └─────────────────┘ │ │ │ (0,1) ─────────────────────────── (1,1) ``` | Field | Description | | --------------- | ------------------------------------------------------- | | `left` | Distance from left edge (0 = left edge, 1 = right edge) | | `top` | Distance from top edge (0 = top, 1 = bottom) | | `width` | Block width as fraction of page width | | `height` | Block height as fraction of page height | | `page` | Page number (1-indexed) in the processed output | | `original_page` | Page number in the source document | `page` and `original_page` differ when using `page_range` to process a subset of pages. For example, if you process pages 5-10, page 5 becomes `page: 1` but `original_page: 5`. *** ## Confidence Scores Parse provides confidence scores to help you identify potentially problematic extractions. ### String Confidence ```json theme={null} "confidence": "high" // or "low" ``` * **`high`**: Extraction is reliable * **`low`**: May need review; consider enabling agentic mode ### Granular Confidence ```json theme={null} "granular_confidence": { "parse_confidence": 0.834, "extract_confidence": null } ``` | Field | Description | | -------------------- | ---------------------------------------------------------- | | `parse_confidence` | Numeric score (0-1) for parsing accuracy | | `extract_confidence` | Numeric score for extraction (when using Extract endpoint) | *** ## Complete Example Here's a complete example showing the full structure: ```json theme={null} { "job_id": "7600c8c5-a52f-49d2-8a7d-d75d1b51e141", "duration": 3.89, "result": { "type": "full", "chunks": [ { "content": "# Bank Statement\n\nAccount: 12345678\nPeriod: Jan 1 - Jan 31, 2024\n\n## Transactions\n\n| Date | Description | Amount |\n|------|-------------|--------|\n| 01/05 | Direct Deposit | $2,500.00 |\n| 01/10 | Grocery Store | -$85.32 |", "embed": "# Bank Statement\n\nAccount: 12345678\nPeriod: Jan 1 - Jan 31, 2024\n\n## Transactions\n\nThis table shows transactions including a direct deposit of $2,500 and a grocery purchase of $85.32.", "blocks": [ { "type": "Title", "content": "Bank Statement", "bbox": { "left": 0.35, "top": 0.02, "width": 0.30, "height": 0.03, "page": 1, "original_page": 1 }, "confidence": "high", "granular_confidence": { "parse_confidence": 0.95, "extract_confidence": null }, "image_url": null }, { "type": "Key Value", "content": "Account: 12345678", "bbox": { "left": 0.10, "top": 0.08, "width": 0.25, "height": 0.02, "page": 1, "original_page": 1 }, "confidence": "high", "granular_confidence": { "parse_confidence": 0.92, "extract_confidence": null }, "image_url": null }, { "type": "Section Header", "content": "Transactions", "bbox": { "left": 0.10, "top": 0.15, "width": 0.20, "height": 0.02, "page": 1, "original_page": 1 }, "confidence": "high", "granular_confidence": { "parse_confidence": 0.88, "extract_confidence": null }, "image_url": null }, { "type": "Table", "content": "| Date | Description | Amount |\n|------|-------------|--------|\n| 01/05 | Direct Deposit | $2,500.00 |\n| 01/10 | Grocery Store | -$85.32 |", "bbox": { "left": 0.10, "top": 0.20, "width": 0.80, "height": 0.25, "page": 1, "original_page": 1 }, "confidence": "high", "granular_confidence": { "parse_confidence": 0.78, "extract_confidence": null }, "image_url": null } ], "enriched": null, "enrichment_success": false } ] }, "usage": { "num_pages": 1, "credits": 2.0 }, "pdf_url": "https://storage.reducto.ai/pdfs/7600c8c5.pdf?...", "studio_link": "https://studio.reducto.ai/job/7600c8c5-a52f-49d2-8a7d-d75d1b51e141" } ``` *** ## Related Quick start and basic usage. Optimize for your document types. Control how content is split. HTML, Markdown, JSON, CSV options. # API Quickstart Source: https://docs.reducto.ai/quickstart Parse your first document with Reducto in 5 minutes. This guide walks you through your first Reducto API call. You will parse a document and get back structured JSON ready for LLMs, downstream extraction, or any other processing step in your pipeline. *** ## Fastest path for coding agents If you are using Claude Code, Codex, Cursor, or another coding agent, start here. This path avoids Studio clicks and extra docs navigation. 1. Set `REDUCTO_API_KEY`. 2. Choose one interface: * Local file or folder: use the [Reducto CLI](/cli). * Agent tool calling: use the [Reducto MCP server](/mcp-server). * Application code: use the Python, Node.js, Go, or cURL examples below. 3. Parse the sample PDF first, then replace the URL or file path with your document. ```bash CLI theme={null} pip install reducto-cli reducto login curl -L -o fidelity-example.pdf https://cdn.reducto.ai/samples/fidelity-example.pdf reducto parse ./fidelity-example.pdf ``` ```python Python theme={null} from reducto import Reducto client = Reducto() result = client.parse.run(input="https://cdn.reducto.ai/samples/fidelity-example.pdf") print(result.job_id) print(result.result.chunks[0].content[:1000]) ``` ```javascript Node.js theme={null} import Reducto from "reductoai"; const client = new Reducto(); const result = await client.parse.run({ input: "https://cdn.reducto.ai/samples/fidelity-example.pdf", }); console.log(result.job_id); console.log(result.result.chunks[0].content.slice(0, 1000)); ``` ```bash cURL theme={null} curl -X POST "https://platform.reducto.ai/parse" \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{"input":"https://cdn.reducto.ai/samples/fidelity-example.pdf"}' ``` For MCP, install once with `uvx mcp-server-reducto --login`, then ask the agent to call `parse_document(document_url="https://cdn.reducto.ai/samples/fidelity-example.pdf")`. *** ## What we're going to parse We'll use a financial statement PDF that contains multiple tables, headers, account summaries, and formatted text. This is the kind of complex document that's difficult to process manually but straightforward with Reducto. Finance Statement [View the sample PDF in Studio](https://studio.reducto.ai/share/md726aw3w7mfs46659ttkqry0s7se3pd?processor=kh7c9e30evkfb5a4h80dq4xke17sfwck\&fileId=js7e4hrtnh2tsyjqdbz114ceyn7sf1v1) or [download it directly](https://cdn.reducto.ai/samples/fidelity-example.pdf) to follow along. **What we want to extract:** * The portfolio value table with beginning and ending values * Account information including account numbers and types * Income summary broken down by tax category * Top holdings with values and percentages By the end of this guide, you'll have all of this data in structured JSON that you can use in your application. For structured field extraction (e.g., extracting specific account numbers or values into typed fields), see the [/extract endpoint](/extract/overview) after completing this quickstart. *** ## Prerequisites Go to [studio.reducto.ai](https://studio.reducto.ai/) and sign up for a free account. In the Studio sidebar, click **API Keys**, then **Create new API key**. Give it a name and copy the key. Reducto Studio sidebar showing API Keys option This allows the SDK to authenticate automatically without hardcoding the key in your code. ```bash theme={null} export REDUCTO_API_KEY="your_api_key_here" ``` ```powershell theme={null} $env:REDUCTO_API_KEY="your_api_key_here" ``` You can also copy the below snippet for your AI coding agent to connect to Reducto via the [MCP Server](/mcp-server).
````markdown theme={null} ## Add Reducto MCP Server ### 1. Authenticate (one-time) ```bash uvx mcp-server-reducto --login ``` This opens your browser to approve access. Your API key is saved to `~/.reducto/config.yaml`. ### 2. Add to your MCP client **Claude Code:** ```bash claude mcp add reducto -- uvx mcp-server-reducto ``` **Claude Desktop**: edit `~/Library/Application Support/Claude/claude_desktop_config.json`: ```json { "mcpServers": { "reducto": { "command": "uvx", "args": ["mcp-server-reducto"] } } } ``` **Cursor**: edit `.cursor/mcp.json`: ```json { "mcpServers": { "reducto": { "command": "uvx", "args": ["mcp-server-reducto"] } } } ``` **VS Code**: edit `.vscode/mcp.json`: ```json { "servers": { "reducto": { "command": "uvx", "args": ["mcp-server-reducto"] } } } ``` ### 3. Use it The server provides these tools: | Tool | What it does | |------|-------------| | `upload_file` | Upload a local file or URL to Reducto (returns `reducto://` URL) | | `parse_document` | Parse a document into structured text, tables, figures | | `extract_data` | Extract structured JSON from a document using a schema | | `split_document` | Segment a document into labeled sections | | `classify_document` | Categorize a document type | | `edit_document` | Fill forms or modify a PDF/DOCX | **Local files:** Use `upload_file` first, e.g. `upload_file("./report.pdf")`, then pass the returned `reducto://` URL to other tools. **Chain operations:** `parse_document` returns a `job_id`. Pass `jobid://` to `extract_data` or `split_document` to skip re-parsing. ````
*** ## Install the SDK Choose your language and install the Reducto SDK: ```bash theme={null} pip install reductoai ``` Requires Python 3.8+. ```bash theme={null} npm install reductoai ``` ```bash theme={null} go get github.com/reductoai/reducto-go-sdk ``` *** ## Parse the document Now let's write the code to parse our financial statement. We'll go through each part step by step. First, we import the Reducto client. When you create a `Reducto()` client without passing an API key, it automatically reads from the `REDUCTO_API_KEY` environment variable you set earlier. ```python theme={null} from reducto import Reducto # The client reads REDUCTO_API_KEY from your environment client = Reducto() ``` Before parsing, you need to upload the document to Reducto's servers. The `upload()` method accepts a file path (as a string) and returns a reference that you'll use in the next step. You can download the sample PDF from [here](https://cdn.reducto.ai/samples/fidelity-example.pdf). ```python theme={null} from pathlib import Path # Upload the PDF file to Reducto upload = client.upload(file=Path("fidelity-example.pdf")) print(f"Uploaded: {upload.file_id}") ``` You can also pass a URL directly to the parse method if your document is already hosted somewhere accessible, like an S3 bucket: ```python theme={null} result = client.parse.run(input="https://cdn.reducto.ai/samples/fidelity-example.pdf") ``` Now we call the `parse.run()` method with the uploaded file reference. This sends the document through Reducto's processing pipeline, which runs OCR, detects layout, extracts tables, and structures everything into chunks. ```python theme={null} # Parse the uploaded document result = client.parse.run(input=upload.file_id) # Check what we got back print(f"Job ID: {result.job_id}") print(f"Pages processed: {result.usage.num_pages}") print(f"Credits used: {result.usage.credits}") print(f"Number of chunks: {len(result.result.chunks)}") ``` The response contains `chunks`, which are logical sections of the document. Each chunk has a `content` field with the full text and a `blocks` field with individual elements like tables, headers, and paragraphs. ```python theme={null} # Loop through each chunk for i, chunk in enumerate(result.result.chunks): print(f"\n=== Chunk {i + 1} ===") print(chunk.content[:500]) # First 500 characters # Look at individual blocks within this chunk for block in chunk.blocks: print(f" [{block.type}] on page {block.bbox.page}") # Tables are returned as HTML by default if block.type == "Table": print(f" Table content: {block.content[:200]}...") ``` Each block has a `type` that tells you what kind of content it is: `Title`, `Section Header`, `Text`, `Table`, `Figure`, `Key Value`, and others. The `bbox` field contains the bounding box coordinates so you know exactly where on the page this content came from. **Complete code:** ```python theme={null} from pathlib import Path from reducto import Reducto client = Reducto() upload = client.upload(file=Path("fidelity-example.pdf")) result = client.parse.run(input=upload.file_id) print(f"Processed {result.usage.num_pages} pages") for chunk in result.result.chunks: print(chunk.content) for block in chunk.blocks: if block.type == "Table": print(f"Found table on page {block.bbox.page}") ``` All Node.js examples use `await` and must be run inside an `async` function, or in a file with top-level await enabled (ES modules with Node.js 14.8+). Import the Reducto client and the `fs` module for reading files. The client automatically uses the `REDUCTO_API_KEY` environment variable for authentication. ```javascript theme={null} import Reducto from 'reductoai'; import fs from 'fs'; // The client reads REDUCTO_API_KEY from your environment const client = new Reducto(); ``` Use `createReadStream` to upload the file to Reducto. This returns a reference you'll use when calling the parse endpoint. You can download the sample PDF from [here](https://cdn.reducto.ai/samples/fidelity-example.pdf). ```javascript theme={null} // Upload the PDF file to Reducto const upload = await client.upload({ file: fs.createReadStream("fidelity-example.pdf") }); console.log(`Uploaded: ${upload.file_id}`); ``` Call `parse.run()` with the uploaded file reference. Reducto processes the document and returns structured content. ```javascript theme={null} // Parse the uploaded document const result = await client.parse.run({ input: upload.file_id }); console.log(`Job ID: ${result.job_id}`); console.log(`Pages processed: ${result.usage.num_pages}`); console.log(`Credits used: ${result.usage.credits}`); console.log(`Number of chunks: ${result.result.chunks.length}`); ``` Loop through the chunks and blocks to access the extracted text, tables, and other elements. ```javascript theme={null} // Loop through each chunk for (let i = 0; i < result.result.chunks.length; i++) { const chunk = result.result.chunks[i]; console.log(`\n=== Chunk ${i + 1} ===`); console.log(chunk.content.substring(0, 500)); // Look at individual blocks within this chunk for (const block of chunk.blocks) { console.log(` [${block.type}] on page ${block.bbox.page}`); if (block.type === "Table") { console.log(` Table content: ${block.content.substring(0, 200)}...`); } } } ``` **Complete code:** ```javascript theme={null} import Reducto from 'reductoai'; import fs from 'fs'; const client = new Reducto(); async function main() { const upload = await client.upload({ file: fs.createReadStream("fidelity-example.pdf") }); const result = await client.parse.run({ input: upload.file_id }); console.log(`Processed ${result.usage.num_pages} pages`); for (const chunk of result.result.chunks) { console.log(chunk.content); for (const block of chunk.blocks) { if (block.type === "Table") { console.log(`Found table on page ${block.bbox.page}`); } } } } main(); ``` The Go SDK is currently in alpha (`v0.1.0-alpha.1`). The API may change in future releases. Import the Reducto client and the option package for configuration. The Go SDK requires you to pass the API key explicitly using `option.WithAPIKey()`. ```go theme={null} package main import ( "context" "fmt" "io" "os" reducto "github.com/reductoai/reducto-go-sdk" "github.com/reductoai/reducto-go-sdk/option" "github.com/reductoai/reducto-go-sdk/shared" ) func main() { // Initialize client with API key from environment client := reducto.NewClient(option.WithAPIKey(os.Getenv("REDUCTO_API_KEY"))) } ``` Open the file and upload it to Reducto. The upload returns a file ID that you'll use for parsing. You can download the sample PDF from [here](https://cdn.reducto.ai/samples/fidelity-example.pdf). ```go theme={null} file, err := os.Open("fidelity-example.pdf") if err != nil { fmt.Printf("Error opening file: %v\n", err) return } defer file.Close() upload, err := client.Upload(context.Background(), reducto.UploadParams{ File: reducto.F[io.Reader](file), }) if err != nil { fmt.Printf("Upload error: %v\n", err) return } fmt.Printf("Uploaded: %s\n", upload.FileID) ``` Call `Parse.Run()` with the file ID. The Go SDK requires you to wrap the file ID with `shared.UnionString()` and then with `reducto.F[...]()` because the SDK uses strongly-typed union parameters. ```go theme={null} result, err := client.Parse.Run(context.Background(), reducto.ParseRunParams{ ParseConfig: reducto.ParseConfigParam{ // The file ID must be wrapped in shared.UnionString() and reducto.F[...]() DocumentURL: reducto.F[reducto.ParseConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), }, }) if err != nil { fmt.Printf("Parse error: %v\n", err) return } fmt.Printf("Job ID: %s\n", result.JobID) fmt.Printf("Pages: %d\n", result.Usage.NumPages) // Note: To view in Studio, construct the URL: https://studio.reducto.ai/job/{job_id} ``` The result contains chunks with extracted content. The `Chunks` field is typed as `interface{}`, so you need to type assert it to `[]shared.ParseResponseResultFullResultChunk` before you can iterate over it. When checking block types, use the SDK constants instead of string comparisons. ```go theme={null} if result.Result.Type == shared.ParseResponseResultTypeFull { // Type assert Chunks from interface{} to the actual type chunks, ok := result.Result.Chunks.([]shared.ParseResponseResultFullResultChunk) if ok { for _, chunk := range chunks { fmt.Println(chunk.Content) for _, block := range chunk.Blocks { // Use SDK constants for block type comparisons if block.Type == shared.ParseResponseResultFullResultChunksBlocksTypeTable { fmt.Printf("Found table on page %d\n", block.Bbox.Page) } } } } } ``` **Complete code:** ```go theme={null} package main import ( "context" "fmt" "io" "os" reducto "github.com/reductoai/reducto-go-sdk" "github.com/reductoai/reducto-go-sdk/option" "github.com/reductoai/reducto-go-sdk/shared" ) func main() { client := reducto.NewClient(option.WithAPIKey(os.Getenv("REDUCTO_API_KEY"))) file, _ := os.Open("fidelity-example.pdf") defer file.Close() upload, _ := client.Upload(context.Background(), reducto.UploadParams{ File: reducto.F[io.Reader](file), }) result, _ := client.Parse.Run(context.Background(), reducto.ParseRunParams{ ParseConfig: reducto.ParseConfigParam{ DocumentURL: reducto.F[reducto.ParseConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), }, }) fmt.Printf("Processed %d pages\n", result.Usage.NumPages) if result.Result.Type == shared.ParseResponseResultTypeFull { chunks, _ := result.Result.Chunks.([]shared.ParseResponseResultFullResultChunk) for _, chunk := range chunks { fmt.Println(chunk.Content) } } } ``` If you prefer not to use an SDK, you can call the API directly with cURL or any HTTP client. First, upload the file to get a file reference: ```bash theme={null} curl -X POST "https://platform.reducto.ai/upload" \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -F "file=@fidelity-example.pdf" ``` This returns a JSON response with a `file_id`: ```json theme={null} {"file_id": "reducto://abc123def456.pdf"} ``` Use the `file_id` from the previous step as the `input` parameter: ```bash theme={null} curl -X POST "https://platform.reducto.ai/parse" \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{"input": "reducto://abc123def456.pdf"}' ``` You can also skip the upload step if your document is already hosted at a public URL: ```bash theme={null} curl -X POST "https://platform.reducto.ai/parse" \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{"input": "https://cdn.reducto.ai/samples/fidelity-example.pdf"}' ``` *** ## Understanding the response Here's what we got back from parsing our financial statement: ```json theme={null} { "job_id": "5df31070-8d98-4caa-9a5b-c5c511a03f71", "duration": 11.35, "usage": { "num_pages": 3, "credits": 4.0 }, "result": { "chunks": [ { "content": "# *** SAMPLE STATEMENT ***\nFor informational purposes only\n\nFidelity\nINVESTMENTS\n\n## Your Portfolio Value:\n\n$274,222.20\n\n| | This Period | Year-to-Date |\n|-|-|-|\n| Beginning Portfolio Value | $253,221.83 | $232,643.16 |\n| Additions | 59,269.64 | 121,433.55 |...", "blocks": [ { "type": "Title", "content": "*** SAMPLE STATEMENT ***\nFor informational purposes only", "bbox": {"page": 1, "left": 0.351, "top": 0.029, "width": 0.296, "height": 0.057}, "confidence": "high" }, { "type": "Section Header", "content": "Your Portfolio Value:", "bbox": {"page": 1, "left": 0.517, "top": 0.163, "width": 0.153, "height": 0.015}, "confidence": "high" }, { "type": "Table", "content": "| | This Period | Year-to-Date |\n|-|-|-|\n| Beginning Portfolio Value | $253,221.83 | $232,643.16 |\n| Additions | 59,269.64 | 121,433.55 |\n| Subtractions | -45,430.74 | -98,912.58 |\n| Transaction Costs, Fees & Charges | -139.77 | -625.87 |\n| Change in Investment Value* | 7,161.47 | 19,058.07 |\n| Ending Portfolio Value** | $274,222.20 | $274,222.20 |", "bbox": {"page": 1, "left": 0.516, "top": 0.261, "width": 0.444, "height": 0.158}, "confidence": "high" } ] } ] }, "studio_link": "https://studio.reducto.ai/job/5df31070-8d98-4caa-9a5b-c5c511a03f71" } ``` **Key fields:** | Field | What it contains | | ------------------ | ---------------------------------------------------------------------------------------------- | | `job_id` | Unique identifier for this job. Use it to retrieve results later or debug in Studio. | | `usage.num_pages` | Number of pages that were processed. | | `usage.credits` | Credits consumed by this request. | | `chunks` | Logical sections of the document, optimized for feeding into LLMs. | | `chunks[].content` | The full text content of this chunk. | | `chunks[].blocks` | Individual elements (tables, headers, text) with their types and positions. | | `blocks[].type` | What kind of element this is: `Title`, `Table`, `Section Header`, `Text`, `Figure`, etc. | | `blocks[].bbox` | Bounding box with normalized coordinates (0-1) showing where this element appears on the page. | | `studio_link` | Direct link to view this job in Reducto Studio for visual debugging. | *** ## Customizing the output The default settings work well for most documents, but you can customize the parsing behavior for specific use cases. You can pass configuration options as `TypedDict` imports from `reducto.types` or as plain dictionaries: ```python theme={null} from reducto.types import EnhanceParam, FormattingParam, SettingsParam result = client.parse.run( input=upload.file_id, enhance=EnhanceParam( # Use AI to clean up OCR errors in scanned documents agentic=[{"scope": "text"}], # Generate descriptions for charts and images summarize_figures=True ), formatting=FormattingParam( # Get tables as HTML, md, json, or csv table_output_format="md" ), settings=SettingsParam( # Only process pages 1-5 page_range={"start": 1, "end": 5} ) ) ``` You can also pass plain dictionaries instead of `TypedDict` imports. Both work identically. ```javascript theme={null} const result = await client.parse.run({ input: upload.file_id, enhance: { agentic: [{scope: "text"}], summarize_figures: true }, formatting: { table_output_format: "md" }, settings: { page_range: {start: 1, end: 5} } }); ``` ```go theme={null} result, err := client.Parse.Run(context.Background(), reducto.ParseRunParams{ ParseConfig: reducto.ParseConfigParam{ DocumentURL: reducto.F[reducto.ParseConfigDocumentURLUnionParam]( shared.UnionString(upload.FileID), ), // Output formatting - use SDK constants for table format AdvancedOptions: reducto.F(shared.AdvancedProcessingOptionsParam{ TableOutputFormat: reducto.F(shared.AdvancedProcessingOptionsTableOutputFormatMd), }), // Chunking options Options: reducto.F(shared.BaseProcessingOptionsParam{ Chunking: reducto.F(shared.BaseProcessingOptionsChunkingParam{ ChunkMode: reducto.F(shared.BaseProcessingOptionsChunkingChunkModeVariable), }), }), }, }) ``` The Go SDK uses different parameter names than Python and Node.js: | Python/Node.js | Go SDK | | -------------------------------- | ----------------------------------- | | `formatting.table_output_format` | `AdvancedOptions.TableOutputFormat` | | `settings.page_range` | `AdvancedOptions.PageRange` | | `retrieval.chunking.chunk_mode` | `Options.Chunking.ChunkMode` | Use SDK constants like `AdvancedProcessingOptionsTableOutputFormatMd` instead of strings, and wrap values with `reducto.F()`. ```bash theme={null} curl -X POST "https://platform.reducto.ai/parse" \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://abc123def456.pdf", "enhance": { "agentic": [{"scope": "text"}], "summarize_figures": true }, "formatting": { "table_output_format": "md" }, "settings": { "page_range": {"start": 1, "end": 5} } }' ``` **What these options do:** * **`enhance.agentic`**: Runs AI-powered cleanup on the specified scope. Use `"text"` for OCR correction on scanned documents, or `"table"` to improve table structure detection. * **`enhance.summarize_figures`**: Generates natural language descriptions of charts, graphs, and images. Useful for RAG pipelines where you need to search figure content. * **`formatting.table_output_format`**: Controls how tables are returned. Options are `html`, `md` (markdown), `json`, `csv`, `dynamic` (default, returns markdown for simple tables and HTML for complex ones), or `jsonbbox`. * **`settings.page_range`**: Limits processing to specific pages. Useful for large documents where you only need certain sections. For the full list of options, see the [Parse configuration reference](/configs/overview). *** ## What's next Now that you can parse documents, explore the other Reducto endpoints: Define a JSON schema and extract specific fields from your documents. Divide long documents into sections based on content type. Fill PDF forms and modify DOCX documents programmatically. Process documents asynchronously with webhooks for high-volume workloads. *** ## Troubleshooting This means your API key is missing or invalid. Check that the `REDUCTO_API_KEY` environment variable is set correctly and that the key hasn't expired in Studio. Some complex tables need extra help. Enable `enhance.agentic` with `[{"scope": "table"}]` for AI-powered table reconstruction, or try `formatting.table_output_format` set to `"html"` or `"json"` for more structured output. For scanned documents or low-quality PDFs, enable the agentic text enhancement: `enhance.agentic: [{"scope": "text"}]`. If the document is password-protected, pass the password in `settings.document_password`. This may also be due to bad metadata polluting the output, in which case, reach out to Reducto support. Every response includes a `studio_link` that opens the job in Reducto Studio. Use it to visually inspect what was extracted and debug any issues. # Checking API Health & Usage Source: https://docs.reducto.ai/reference/checking-api-health Check Reducto hosted API availability, usage, and throttling signals Use this page to check the availability of the Reducto API and monitor your usage and throttling signals. ## Check hosted API availability If you use the hosted Reducto API at `platform.reducto.ai`, check [status.reducto.ai](https://status.reducto.ai) for uptime and incident information. Subscribe to status updates there to receive notifications about incidents and maintenance. For a lightweight connectivity check, request `GET /version`. It returns the current API version string: ```bash theme={null} curl "https://platform.reducto.ai/version" \ -H "Authorization: Bearer $REDUCTO_API_KEY" ``` To also verify that your API key is valid, request `GET /jobs?limit=1`: ```bash theme={null} curl "https://platform.reducto.ai/jobs?limit=1" \ -H "Authorization: Bearer $REDUCTO_API_KEY" ``` A `200` response confirms the API is reachable and your key is valid. A `401` indicates an invalid or expired API key. A `403` with an HTML error page means the request was sent without an `Authorization` header. For transient failures, retry according to your client's retry policy. ## Self-hosted and on-premise deployments If you run Reducto in your own environment, reach out to the Reducto team at [support@reducto.ai](mailto:support@reducto.ai) for monitoring and observability guidance specific to your deployment. ## Check usage and throttling * Use the [Usage Export API](/reference/usage-export-api) for programmatic usage and credit data. * Use [Rate Limits](/reference/rate-limits) to identify request-rate `429` responses. * Use [Concurrency Throttle](/reference/throttling) to understand queued work and latency caused by account-level concurrency limits. # Credit Usage Source: https://docs.reducto.ai/reference/credit-usage Understanding how credits are calculated across Reducto endpoints Credits are the billing unit for Reducto API usage. This page explains how credits are calculated for each endpoint and document type. ## Parse Endpoint ### Documents and Images Supported formats: PDF, DOCX, DOC, PPTX, PPT, PNG, JPEG, GIF, TIFF, HEIC, and more. | Processing Type | Credits/Page | When Applied | | ------------------------ | ------------ | ---------------------------------------------------------------------------------- | | **Standard** | 1 | Text, layout, simple tables, OCR | | **Complex** | 2 | VLM-enhanced pages (complex tables, key-value regions, figures with summarization) | | **Agentic - Standard** | 2 | Agentic mode enabled on simple pages | | **Agentic - Complex** | 4 | Agentic mode enabled on complex pages | | **Advanced chart agent** | +4 | Per chart when `advanced_chart_agent: true` | Reducto automatically classifies page complexity. You don't choose "standard" vs "complex". Complexity is determined by the content. **What makes a page complex?** * Tables with merged cells or nested headers * Key-value form regions * Figures when `summarize_figures: true` (default) ### Batch Queue Discount Parse jobs submitted with `queue_priority: "batch"` consume **20% fewer credits** than the same parse on the standard lane. The discount applies to all parse credit types in the tables above, and the credit usage returned in `/job/{id}` already reflects the reduced amount. Batch jobs run asynchronously with a 12-hour completion guarantee. See [Batch Queue](/workflows/batch-queue). ### Spreadsheets Supported formats: XLSX, XLS, CSV, XLSM | Clustering Mode | Credits | Description | | ---------------------- | ----------------- | --------------------------- | | **Accurate** (default) | 1 per 1,000 cells | Intelligent table detection | | **Fast** | 1 per 5,000 cells | Basic clustering | | **Disabled** | 1 per 5,000 cells | Single table output | ### Text Files Supported formats: HTML, TXT, RTF | Format | Credits/Page | | ------------ | ------------ | | Text formats | 0.5 | *** ## Extract Endpoint | Mode | Credits | Notes | | ----------------------- | ------------------------------------------------------------------------------- | -------------------------------------- | | **Standard** | 2 per page | Schema-based extraction | | **Deep Extract** (Beta) | 4 per page + 0.1 per field extracted,
minimum of 30 credits per document\* | Agentic loop for near-perfect accuracy |

\*Beta pricing, subject to change.

If you pass a URL or file directly to Extract (instead of a `jobid://`), Parse credits are also charged. To avoid double-charging, parse first, then extract using the job ID. ```python theme={null} # ❌ Charges for both Parse AND Extract result = client.extract.run( input=upload.file_id, instructions={"schema": schema} ) # ✅ Only charges for Extract (reuses parsed content) parse_result = client.parse.run(input=upload.file_id) result = client.extract.run( input=f"jobid://{parse_result.job_id}", instructions={"schema": schema} ) ``` *** ## Split Endpoint | Mode | Credits/Page | Notes | | -------------- | ------------ | ------------------------------------------------------------------------------------ | | **Standard** | 2 | | | **Deep Split** | 4 | Agentic loop for near-perfect accuracy. See [Deep Split](/configs/split/deep-split). | Same as Extract: passing a URL charges Parse + Split. Use `jobid://` to avoid double-charging. *** ## Classify Endpoint | Mode | Credits/Page | Notes | | ------------ | ------------ | ------------------------ | | **Standard** | 0.5 | Per page of context used | Classify reads the first 5 pages of a document by default, costing 2.5 credits per classification. You can configure context up to 10 pages (5.0 credits). *** ## Edit Endpoint | Mode | Credits/Page | Notes | | -------- | ------------ | ----------------- | | **Beta** | 4 | Subject to change | *** ## Pipeline Endpoint Pipelines combine multiple operations. Credits are the sum of all operations in the pipeline: ``` Pipeline credits = Parse + Extract (if configured) + Split (if configured) ``` *** ## Credit Optimization Tips ### 1. Reuse Parse Results Parse once, then run multiple Extract or Split calls using the job ID: ```python theme={null} # Parse once parse = client.parse.run(input=upload.file_id) # Extract multiple schemas without re-parsing invoice_data = client.extract.run( input=f"jobid://{parse.job_id}", instructions={"schema": invoice_schema} ) vendor_data = client.extract.run( input=f"jobid://{parse.job_id}", instructions={"schema": vendor_schema} ) ``` ### 2. Disable Agentic Mode When Not Needed Agentic mode doubles credit usage. Only enable it for: * Handwritten content * Low-quality scans * Complex tables that parse incorrectly ```python theme={null} # Standard parsing (1-2 credits/page) result = client.parse.run(input=upload.file_id) # Only enable agentic when needed (2-4 credits/page) result = client.parse.run( input=upload.file_id, enhance={"agentic": [{"scope": "text"}]} ) ``` ### 3. Use Page Ranges Process only the pages you need: ```python theme={null} # Only process pages 1-10 result = client.parse.run( input=upload.file_id, settings={"page_range": {"start": 1, "end": 10}} ) ``` ### 4. Choose Appropriate Spreadsheet Clustering For large spreadsheets where you don't need intelligent table detection: ```python theme={null} result = client.parse.run( input=upload.file_id, # spreadsheet file spreadsheet={"clustering": "fast"} # 5x cheaper than "accurate" ) ``` ### 5. Use the Batch Queue for Non-Urgent Parsing Submit async parse jobs with `queue_priority: "batch"` to consume 20% fewer credits, with a 12-hour completion guarantee: ```python theme={null} # 20% credit discount on parsing job = client.parse.run_job( input=upload.file_id, queue_priority="batch" ) ``` See [Batch Queue](/workflows/batch-queue) for details. *** ## Monitoring Usage Track your credit usage in [Reducto Studio](https://studio.reducto.ai/): * **Usage dashboard**: View credits consumed over time * **Job logs**: See credits charged per job * **Usage alerts**: Set threshold notifications * **[Usage Export API](/reference/usage-export-api)**: Programmatic access to usage data via API key Each API response also includes credit information: ```json theme={null} { "job_id": "abc123", "usage": { "num_pages": 10, "credits": 15.0 } } ``` *** ## Related Request limits and optimization. Reuse parse results to save credits. # Error Codes Source: https://docs.reducto.ai/reference/error-codes Understanding and handling Reducto API errors When something goes wrong, Reducto returns an HTTP status code and error message. ## Error Response Format ```json theme={null} { "error": { "message": "Failed to download file from url. Please check the document url and try again", "type": "file_download_failure", "code": 400 } } ``` ## Client Errors (4xx) These errors indicate a problem with your request. | Code | Name | Description | Solution | | ---- | ----------------------- | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | | 400 | JSON decoding error | LLM response too large to decode | Enable `array_extract: true` for large extraction schemas | | 400 | Invalid URL format | S3 URL malformed | Use format `s3://bucket-name/key` | | 400 | File download failure | Cannot fetch document from URL | Verify URL is accessible and not expired | | 400 | Invalid page range | Specified pages don't exist | Page range is 1-indexed. Check document has requested pages | | 403 | Permission error | URL access forbidden | Ensure presigned URL hasn't expired or credentials are valid | | 403 | PDF processing error | Cannot write output | Internal error, contact support | | 404 | File access error | File not found | Verify file exists at the specified path/URL | | 413 | Image too large | Image exceeds maximum dimensions | Resize image to under 50 megapixels total and 15,000 pixels per axis. See [image limits](/upload/overview#supported-file-types) | | 415 | File conversion error | Cannot convert to images | Check file is a valid, uncorrupted PDF | | 415 | PDF handling error | Cannot process PDF | File may be corrupted. Try re-saving with Adobe Acrobat | | 422 | Schema validation error | Invalid JSON schema | Validate your schema is proper JSON Schema format | | 422 | Table processing error | Table extraction failed | Retry, or enable agentic table mode for complex tables | | 442 | Document access error | Password protected | Provide password via `settings.document_password` | | 429 | Rate limit exceeded | Too many requests | Implement backoff, or switch to async endpoints | ## Server Errors (5xx) These indicate a problem on Reducto's side. Most are automatically retried by the SDK. | Code | Name | Description | Retriable? | | ---- | --------------------------- | -------------------------- | ---------- | | 500 | Content extraction error | Extraction failed | ❌ No | | 500 | Citation extraction error | Citation extraction failed | ❌ No | | 500 | PDF metadata error | PDF metadata corrupted | ❌ No | | 502 | LLM service error | LLM provider error | ✅ Yes | | 503 | Service unavailable | Temporary overload | ✅ Yes | | 503 | Table processing error | Table service error | ✅ Yes | | 504 | LLM timeout error | LLM provider timed out | ✅ Yes | | 504 | Document conversion timeout | Conversion took too long | ✅ Yes | ## Automatic Retries The Reducto SDKs automatically retry requests that fail with these status codes: ``` 408, 409, 429, and all 5xx errors (500+) ``` Default retry behavior: * **Max retries**: 2 * **Backoff**: Exponential with jitter * **Timeout**: 1 hour per request (3600 seconds) You can customize retry behavior: ```python Python theme={null} from reducto import Reducto client = Reducto( max_retries=5, timeout=120.0 # seconds ) ``` ```javascript Node.js theme={null} import Reducto from 'reductoai'; const client = new Reducto({ maxRetries: 5, timeout: 120000 // milliseconds }); ``` ## Handling Errors ```python Python theme={null} from reducto import ( Reducto, APIError, BadRequestError, RateLimitError, APIConnectionError ) client = Reducto() try: result = client.parse.run(input=upload.file_id) except BadRequestError as e: # 400-level errors (your request has a problem) print(f"Bad request: {e.message}") except RateLimitError as e: # 429 - too many requests print("Rate limited, waiting...") time.sleep(60) except APIConnectionError as e: # Network issues print(f"Connection failed: {e}") except APIError as e: # Other API errors print(f"API error {e.status_code}: {e.message}") ``` ```javascript Node.js theme={null} import Reducto from 'reductoai'; const client = new Reducto(); try { const result = await client.parse.run({ input: upload.file_id }); } catch (error) { if (error.status === 400) { console.log('Bad request:', error.message); } else if (error.status === 429) { console.log('Rate limited, waiting...'); await new Promise(r => setTimeout(r, 60000)); } else if (error.status >= 500) { console.log('Server error, will retry automatically'); } } ``` ```bash cURL theme={null} # Check response status code RESPONSE=$(curl -s -w "\n%{http_code}" -X POST "https://platform.reducto.ai/parse" \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -d '{"input": "https://example.com/document.pdf"}') STATUS=$(echo "$RESPONSE" | tail -n 1) BODY=$(echo "$RESPONSE" | sed '$d') if [ "$STATUS" -eq 200 ]; then echo "Success: $BODY" elif [ "$STATUS" -eq 429 ]; then echo "Rate limited, waiting..." sleep 60 else echo "Error $STATUS: $BODY" fi ``` ## Common Issues **Cause**: The URL you provided isn't in a recognized format. **Solutions**: * For S3: Use `s3://bucket-name/key` format * For presigned URLs: Ensure the full URL including query parameters is provided * For reducto:// URLs: Use the exact string returned from `/upload` **Cause**: Reducto cannot access the file at the URL you provided. **Solutions**: * Check that presigned URLs haven't expired * Verify the URL is publicly accessible or properly authenticated * For S3, ensure the bucket policy allows access from Reducto's IPs **Cause**: You've exceeded the request rate limit. **Solutions**: * Implement exponential backoff * Switch to async endpoints (`/parse_async`) which have higher limits * Use separate API keys for different applications * Contact support for higher limits **Cause**: An image in your document exceeds the maximum supported dimensions (50 megapixels total or 15,000 pixels on any single axis). **Solutions**: * Resize or re-export images to stay under 50 megapixels (e.g., under \~8200x6100) * Ensure no single axis exceeds 15,000 pixels * For scanned documents, reduce the scan DPI **Cause**: The document took too long to process. **Solutions**: * Use async endpoints for large documents * Process fewer pages using `page_range` * Disable agentic modes if not needed * Check if the document is unusually complex *** ## Related Understanding request limits. Avoid timeouts with async endpoints. # Frequently Asked Questions Source: https://docs.reducto.ai/reference/faq Common questions about using Reducto ## Documents and Processing Set chunking to `disabled` (the default). The entire document will be returned as a single chunk with all content in the `content` field as Markdown. ```python theme={null} result = client.parse.run( input=upload.file_id, retrieval={"chunking": {"chunk_mode": "disabled"}} ) # Full document as markdown markdown = result.result.chunks[0].content ``` Tables will be formatted according to your `table_output_format` setting (default: `dynamic`, which uses Markdown for simple tables). Both fields contain the chunk's text, but optimized for different purposes: * **`content`**: Raw extraction with original formatting. Tables appear as HTML or Markdown. Use for display. * **`embed`**: Optimized for vector embeddings. When `embedding_optimized: true`, tables become natural language summaries like "This table shows quarterly revenue..." which embed better. **For RAG:** Use `embed` for your vector database, `content` for displaying results to users. See [Understanding Chunks](/parse/response-format#understanding-chunks) for details. 1. **Parse with variable chunking** to get semantically meaningful segments 2. **Enable embedding optimization** so tables become natural language 3. **Filter noise** like headers and footers 4. **Store chunks** in your vector database ```python theme={null} result = client.parse.run( input=upload.file_id, retrieval={ "chunking": {"chunk_mode": "variable", "chunk_size": 1000}, "embedding_optimized": True, "filter_blocks": ["Header", "Footer", "Page Number"] } ) for chunk in result.result.chunks: your_vector_db.insert( embedding=your_embedding_function(chunk.embed), metadata={"content": chunk.content, "blocks": chunk.blocks} ) ``` See [Parse Best Practices](/parse/best-practices) for more. **Adds significant latency:** * `enhance.agentic` with any scope (runs VLM passes) * `enhance.agentic[].advanced_chart_agent: true` (detailed chart analysis) * Large documents with `embedding_optimized: true` **Moderate impact:** * `settings.return_images` (generates cropped images) * `settings.embed_pdf_metadata` (modifies PDF) **Minimal impact:** * Chunking mode changes * Table output format changes * Block filtering For latency-sensitive applications, disable agentic modes and use async with priority for the fastest response. No. Reducto manages model selection internally to optimize for accuracy, cost, and latency. The models used may change as we improve the system. For on-premise deployments, you can configure which LLM providers are available. See [LLM Configuration](/onprem/llm_options). ## URLs and Retention * **Image URLs** (`image_url` from `return_images`): Valid for 1 hour * **PDF URLs** (`pdf_url`): Valid for 1 hour * **Result URLs** (when `type: "url"`): Valid for 1 hour Download or process any URLs promptly after receiving them. By default, job results are deleted after **12 hours** per Reducto's zero data retention (ZDR) policy. To keep results longer: ```python theme={null} result = client.parse.run( input=upload.file_id, settings={"persist_results": True} ) ``` With `persist_results: true`, results are stored indefinitely and can be retrieved anytime using the job ID. This requires opting in to Reducto Studio. When the response exceeds approximately 6MB, Reducto returns `result.type: "url"` instead of `result.type: "full"`. Fetch the content from `result.url`: ```python theme={null} if result.result.type == "url": import requests chunks = requests.get(result.result.url).json() else: chunks = result.result.chunks ``` To always get URL responses (for consistent handling): ```python theme={null} result = client.parse.run( input=upload.file_id, settings={"force_url_result": True} ) ``` Jobs are deleted after 12 hours per the zero data retention policy. If you're looking for a job from more than 12 hours ago, it has been automatically deleted. Jobs can also be explicitly deleted via `DELETE /job/{job_id}`. A 410 response means the job was permanently deleted. A 409 response means deletion is in progress. To prevent data loss: * Process results immediately when you receive them * Store results in your own database * Use `persist_results: true` to keep results indefinitely See [Deleting Jobs & Uploads](/workflows/deletion) for more on manual deletion. ## API and Integration * **Direct upload** (`/upload`): 100MB * **Presigned URL upload**: 5GB * **URL passthrough**: No limit (Reducto fetches the file) For files over 100MB, use the [presigned URL method](/upload/large-files). For files over 5GB, host them on S3 or another storage service and pass the URL directly. Visit [status.reducto.ai](https://status.reducto.ai) for real-time status of all Reducto services, uptime history, and incident reports. Subscribe to updates to get notified of any service disruptions. * **Email**: [support@reducto.ai](mailto:support@reducto.ai) * **Slack**: Available for enterprise customers * **Studio**: Use the feedback button in [Reducto Studio](https://studio.reducto.ai/) *** ## Still Have Questions? Understand and resolve API errors. API request limits and optimization. How credits are calculated. Key terms and concepts. # Glossary Source: https://docs.reducto.ai/reference/glossary Key terms and concepts in Reducto ## A An enhancement that uses vision language models (VLMs) to review and correct parsing output. Available for three scopes: `text` (handwritten content, signatures), `table` (complex table structures), and `figure` (charts and diagrams). Adds latency and credits but improves accuracy for difficult documents. See [Agentic Modes](/configs/parse/agentic-modes). API endpoints that return immediately with a `job_id` instead of waiting for processing to complete. Examples: `/parse_async`, `/extract_async`. Use these for large documents, batch processing, or when you want webhook notifications. See [Async Processing](/workflows/async-overview). ## B Coordinates describing where a block appears on a page. All values are normalized to \[0, 1] relative to page dimensions. Fields: `left`, `top`, `width`, `height`, `page`, `original_page`. See [Response Format](/parse/response-format#bounding-box-coordinates). The atomic content element in Reducto's output. Every paragraph, table, header, figure, list item, etc. is a separate block with its own type, content, bounding box, and confidence score. Blocks are grouped into chunks. ## C A group of related blocks returned by Parse. Chunking mode determines how blocks are grouped. Each chunk has `content` (raw Markdown), `embed` (optimized for vector embeddings), and `blocks` (individual elements with metadata). See [Chunking Methods](/configs/parse/chunking-methods). A measure of parsing accuracy. String confidence is `"high"` or `"low"`. Granular confidence provides numeric scores (0-1) in `parse_confidence` and `extract_confidence` fields. The `content` field in a chunk contains the raw Markdown representation of extracted text. Tables appear in their original format (HTML/Markdown). Use for display purposes. The billing unit for Reducto API usage. Different endpoints and configurations consume different credit amounts. See [Credit Usage](/reference/credit-usage). ## E The `embed` field in a chunk is optimized for vector embeddings. When `embedding_optimized: true`, tables are converted to natural language summaries that embed better than raw Markdown. Use for vector databases and semantic search. The `/extract` endpoint pulls structured data from documents according to a JSON schema you define. Returns typed fields with values and citations. Different from Parse, which returns the full document content. ## F Setting `force_url_result: true` makes the API always return results as a URL pointing to JSON, rather than inline in the response. Useful for consistent handling or very large documents. See [Processing Settings](/configs/parse/ocr-settings#force-url-result). ## J A unique identifier returned when you submit an async request or complete any API call. Format: UUID like `7600c8c5-a52f-49d2-8a7d-d75d1b51e141`. Used to retrieve results, check status, or chain endpoints. A special URL format (`jobid://abc123`) that lets you reference a previously parsed document without re-uploading or re-parsing it. Use with Extract or Split to avoid duplicate processing. ## O The text extraction engine. `standard` (default) is the best multilingual OCR system. `legacy` only supports Germanic languages and exists for backwards compatibility. See [OCR Settings](/configs/parse/ocr-settings). ## P The `/parse` endpoint extracts text, tables, figures, and structure from documents. Returns chunks and blocks with positional metadata. The foundation for most Reducto workflows. Setting `persist_results: true` stores job results indefinitely (requires Studio). Without this, results are deleted after 12 hours per the zero data retention (ZDR) policy. A saved configuration in Reducto Studio that bundles parsing, extraction, or other operations into a single API call. Call via `/pipeline` endpoint with a `pipeline_id`. Async jobs can be submitted with `priority: true` to process ahead of non-priority async jobs. Sync requests are always prioritized over async. ## R A special URL format (`reducto://abc123`) returned by the `/upload` endpoint. Use this to reference uploaded files in subsequent API calls without re-uploading. ## S A JSON Schema definition used with the Extract endpoint to specify what data to pull from documents. Defines field names, types, and structure. The `/split` endpoint divides documents into logical sections based on natural language descriptions you provide. Returns page ranges for each section with confidence scores. [Reducto Studio](https://studio.reducto.ai/) is the web interface for configuring pipelines, viewing results, managing API keys, and monitoring usage. Pipelines created in Studio can be called via the API. API endpoints that block until processing completes and return the full result. Examples: `/parse`, `/extract`. Best for interactive applications with smaller documents. ## U The `/upload` endpoint accepts direct file uploads and returns a `reducto://` URL for use in other API calls. Use for local files rather than URLs. When a Parse response is too large for inline delivery, `result.type` is `"url"` and you must fetch the content from `result.url`. The threshold is approximately 6MB response size. ## Z Reducto's default data policy. Uploaded documents and job results are deleted within 12 hours. Enable `persist_results: true` to keep results longer (requires Studio opt-in). # Per-Page Billing Breakdown Source: https://docs.reducto.ai/reference/page-billing-breakdown Understand exactly which billing features were applied to each page of your document Every Parse response includes a `page_billing_breakdown` field inside `usage` that tells you exactly which billable features were applied to each page. This gives you granular visibility into your credit consumption. ## Response Format The `page_billing_breakdown` is a map from **1-indexed page number** (as a string) to an array of billing feature tags applied to that page: ```json theme={null} { "usage": { "num_pages": 5, "credits": 15.0, "credit_breakdown": { "page": 5.0, "agentic": 5.0, "complex": 3.0 }, "page_billing_breakdown": { "1": ["agentic", "page"], "2": ["agentic", "complex", "page"], "3": ["agentic", "chart_agent", "page"], "4": ["agentic", "chart_agent", "complex", "page"], "5": ["agentic", "page"] } } } ``` ## Billing Features Each page can have one or more of the following feature tags: | Feature | Description | When Applied | | ---------------------------- | ------------------------------------------------ | --------------------------------------------------------------------- | | `page` | Standard page processing | Every PDF/image page | | `html_page` | HTML document page | HTML and text file pages | | `docx_native_page` | Native DOCX page | Word documents processed natively | | `agentic` | Agentic extraction enabled | When any agentic mode is configured (text, tables, figures) | | `complex` | Complex page processing | Pages with complex tables, key-value regions, or figure summarization | | `chart_agent` | Advanced chart extraction | Pages where the chart agent ran | | `billable_spreadsheet_pages` | Spreadsheet page billing derived from cell count | Spreadsheet pages (XLSX, CSV, etc.) | ### Feature Collapsing Several internal work types are collapsed into simplified categories for clarity: * **`complex`** combines: `figure_summary`, `table_summary`, and `key_value` work types. If a page has multiple complex operations (e.g., both a complex table and a figure summary), it still appears as a single `complex` tag. * **`page`**, **`html_page`**, and **`docx_native_page`** are mutually exclusive base page types depending on the document format. * **`billable_spreadsheet_pages`** is the current spreadsheet billing tag. During rollout compatibility, older persisted responses may still show `spreadsheet_cells`. ## Related Full credit rates for all endpoints. Complete parse response structure. # Rate Limits Source: https://docs.reducto.ai/reference/rate-limits Per-second request caps at the API edge and the 429s they return Reducto enforces two independent limit mechanisms. This page covers **edge rate limits**. For the per-account concurrency throttle that queues parallel parse work, see [Concurrency Throttle](/reference/throttling). | Mechanism | What it limits | Behavior on exceeded | Returns | | --------------------------------------------- | -------------------------------------------------- | ----------------------------------------- | ------------------ | | **Edge rate limits** | Requests per second to the API | Request is rejected at the ingress | `429` | | [Concurrency throttle](/reference/throttling) | Parse batches running in parallel for your account | Work queues until a slot frees, then runs | `200` (after wait) | Edge rate limits live at the ingress and protect against accidental floods. They cap **requests per second**, not concurrent work. Requests above the cap return `429` immediately. The SDKs retry `429` responses with exponential backoff automatically. ## Per API Key | Endpoint scope | Limit | 429 response body | | ------------------- | --------------------- | ---------------------------------------------------------------------------------------- | | All endpoints | 1,000 req/s sustained | `{"message": "[CODE 1000] rate limit exceeded, retry with exponential backoff"}` | | `GET /job/{job_id}` | 200 req/s sustained | `{"message": "[CODE 2000] rate limit exceeded, please use webhooks instead of polling"}` | If you hit `[CODE 2000]`, you're polling job status faster than necessary. Switch to [webhooks](/workflows/async-overview) so Reducto pushes results when ready. ## Rare Infra-Shed 429s In rare conditions, Reducto returns 429 from the application layer to protect core infrastructure when it is under load. These 429s do not carry a `[CODE …]` body. They are not a normal-operation signal that you have exceeded any limit; treat them as a transient infrastructure event and retry with backoff. ## How to Tell Which Limit You're Hitting | Symptom | Mechanism | Action | | ---------------------------------------------------------- | --------------------------------------------------- | ---------------------------------------------------------------------------------------------- | | Immediate 429 with body `[CODE 1000]` | Edge rate limit, per-API-key request rate | Slow your submission rate or split traffic across API keys. | | Immediate 429 with body `[CODE 2000]` | Edge rate limit, per-API-key polling rate on `/job` | Switch to webhooks instead of polling. | | Immediate 429 with no `[CODE …]` body | Rare infra shed (see above) | Retry with backoff. | | Slow P90 latency, no 4xx | [Concurrency throttle](/reference/throttling) | Submit steadier traffic so `earned_base` grows, or [contact sales](mailto:support@reducto.ai). | | Sync request hanging close to the 15-minute (900s) timeout | [Concurrency throttle](/reference/throttling) | Switch the call to async. | ## Related How Reducto queues parallel parse work and the tier baselines. Submit jobs and receive results via webhook. Full reference for client and server errors. # Concurrency Throttle Source: https://docs.reducto.ai/reference/throttling How Reducto queues parallel parse work per account, baselines by tier, and when you'll see slow P90 Reducto enforces two independent limit mechanisms. This page covers the **concurrency throttle**. For the per-second request rate caps that return `429` at the edge, see [Rate Limits](/reference/rate-limits). | Mechanism | What it limits | Behavior on exceeded | Returns | | ------------------------------------- | -------------------------------------------------- | ----------------------------------------- | ------------------ | | [Rate limits](/reference/rate-limits) | Requests per second to the API | Request is rejected at the ingress | `429` | | **Concurrency throttle** | Parse batches running in parallel for your account | Work queues until a slot frees, then runs | `200` (after wait) | If you submit more parse work than your account's concurrency ceiling allows, Reducto **queues** the excess rather than rejecting it. You see added latency, not 4xx. ## Your Ceiling ``` ceiling = earned_base + burst_headroom ``` * `earned_base`: capacity sized for your sustained recent traffic, starting from your tier baseline. * `burst_headroom`: short-term slack on top of `earned_base` so a sudden spike does not immediately queue. The unit is **concurrent batches**. Reducto splits a parse job into one or more batches, typically around 10 pages each. A 5-page document runs as a single batch. A 200-page document runs as roughly 20 concurrent batches. ## Tier Baselines The baseline is the starting allocation for `earned_base` in the region you're hitting. With little or no recent traffic, your ceiling sits around this baseline; sustained traffic grows it above. Baselines vary per region because shared compute capacity is sized for the typical regional load. | Tier | US | EU | | ---------- | ------------- | ------------- | | Standard | 200 | 60 | | Growth | 350 | 120 | | Enterprise | 500+ (custom) | 275+ (custom) | *All values are in concurrent batches. The actual raw cap at any moment is higher than the baseline (burst headroom on top) and scales further with sustained traffic. Enterprise baselines are negotiable upward; [contact sales](mailto:support@reducto.ai) to discuss.* Multi-region customers get the tier's baseline in *each* region they hit. ## How Earned Capacity Grows Submit consistent traffic and your ceiling grows above the baseline. Reducto measures your submission rate over a short trailing window and sizes your ceiling for that rate plus burst headroom. When you stop submitting, the ceiling decays back toward the baseline over the same window. Bursty traffic gets less headroom than steady traffic at the same average rate. ## Tenant Throttling For multi-tenant applications, you can pass `settings.tenant_throttling` on parse requests to bound how much of your account's concurrency a single one of your own customers, workspaces, or organizations can consume. Tag each request with the tenant it belongs to: ```json theme={null} { "input": "https://example.com/document.pdf", "settings": { "tenant_throttling": { "tenant_id": "workspace_123", "max_share": 0.5 } } } ``` * `tenant_id` — your identifier for the tenant. Requests with the same id share one tenant-level throttle inside your account. * `max_share` — the maximum fraction of your account's concurrency ceiling this tenant may use, between 0 (exclusive) and 1. Optional; defaults to `0.5`. You can pass different values for different tenants — for example `0.2` on a backfill tenant's requests to keep more headroom for interactive traffic. Your account-level concurrency throttle still applies first; the tenant throttle only divides capacity inside it. If `tenant_throttling` is omitted, Reducto uses the existing account-level behavior only. ## Sync vs Async Under Throttle Requests above your ceiling do not fail. They queue until a slot frees, then run. The wait surfaces differently depending on endpoint type: * **Async** (`/parse_async`, `/extract_async`, `/split_async`, `/edit_async`). The job is accepted immediately, a `job_id` returned, and the work queued. Latency shows up between submission and webhook delivery, never as a 4xx. * **Sync** (`/parse`, `/extract`, `/split`, `/edit`). The HTTP request blocks until a slot opens and the job completes. Total response time includes queue wait. The edge has a 15-minute (900s) hard timeout, so a sustained burst on sync endpoints risks the HTTP connection timing out before the job finishes. For bursty workloads, use async with [webhooks](/workflows/async-overview). Your client doesn't hold open HTTP connections during queue wait. ```python theme={null} import asyncio from reducto import AsyncReducto async def submit_burst(files: list[str]): client = AsyncReducto() jobs = await asyncio.gather(*[ client.parse.run_job(input=f, async_={"webhook": {"mode": "svix"}}) for f in files ]) return [job.job_id for job in jobs] ``` ## Related Per-second request caps at the API edge. Submit jobs and receive results via webhook. Patterns for processing many documents. # Usage Export API Source: https://docs.reducto.ai/reference/usage-export-api Programmatically export usage and credit data from your Reducto account The Usage Export API returns the same usage data available on the [Studio usage dashboard](/studio-account#usage), accessible programmatically via your PropelAuth API key. ## Authentication Authenticate with a Bearer token using your PropelAuth API key from [Studio API Keys](https://studio.reducto.ai/): ```bash theme={null} curl -H "Authorization: Bearer YOUR_API_KEY" \ "https://studio.reducto.ai/api/v1/usage/export" ``` This is your personal API key from the Studio API Keys page, not the `REDUCTO_API_KEY` used for document processing. ## Endpoint ``` GET https://studio.reducto.ai/api/v1/usage/export ``` ## Query Parameters | Parameter | Type | Default | Description | | ----------- | -------- | ------------- | ----------------------------------------------------------------------------------- | | `startDate` | `string` | 30 days ago | Start date in `yyyy-mm-dd` format | | `endDate` | `string` | Today | End date in `yyyy-mm-dd` format | | `groupBy` | `string` | `product` | Dimension to group results by. One of: `product`, `feature`, `api_key`, `file_type` | | `orgId` | `string` | API key's org | Target organization ID. Required if your account belongs to multiple organizations. | | `product` | `string` | (none) | Filter by product. Repeatable for multiple values. | | `feature` | `string` | (none) | Filter by feature. Repeatable for multiple values. | | `apiKey` | `string` | (none) | Filter by API key. Repeatable for multiple values. | | `fileType` | `string` | (none) | Filter by file type. Repeatable for multiple values. | ## Response ```json theme={null} { "orgId": "org_abc123", "startDate": "2026-05-19", "endDate": "2026-06-18", "groupBy": "product", "data": [ { "date": "2026-05-19", "group": "parse", "credits": 150.0, "requestCount": 75 }, { "date": "2026-05-19", "group": "extract", "credits": 80.0, "requestCount": 20 } ] } ``` Each entry in `data` represents one day and one group value. `credits` is the total credits consumed and `requestCount` is the number of API requests made. ## Examples ### Default: last 30 days grouped by product ```python Python theme={null} import requests response = requests.get( "https://studio.reducto.ai/api/v1/usage/export", headers={"Authorization": "Bearer YOUR_API_KEY"}, ) data = response.json() for row in data["data"]: print(f"{row['date']} | {row['group']}: {row['credits']} credits, {row['requestCount']} requests") ``` ```javascript Node.js theme={null} const response = await fetch( "https://studio.reducto.ai/api/v1/usage/export", { headers: { Authorization: "Bearer YOUR_API_KEY" } } ); const data = await response.json(); for (const row of data.data) { console.log(`${row.date} | ${row.group}: ${row.credits} credits, ${row.requestCount} requests`); } ``` ```bash cURL theme={null} curl -H "Authorization: Bearer YOUR_API_KEY" \ "https://studio.reducto.ai/api/v1/usage/export" ``` ### Custom date range grouped by file type ```python Python theme={null} import requests response = requests.get( "https://studio.reducto.ai/api/v1/usage/export", headers={"Authorization": "Bearer YOUR_API_KEY"}, params={ "startDate": "2026-06-01", "endDate": "2026-06-15", "groupBy": "file_type", }, ) data = response.json() ``` ```javascript Node.js theme={null} const params = new URLSearchParams({ startDate: "2026-06-01", endDate: "2026-06-15", groupBy: "file_type", }); const response = await fetch( `https://studio.reducto.ai/api/v1/usage/export?${params}`, { headers: { Authorization: "Bearer YOUR_API_KEY" } } ); const data = await response.json(); ``` ```bash cURL theme={null} curl -H "Authorization: Bearer YOUR_API_KEY" \ "https://studio.reducto.ai/api/v1/usage/export?startDate=2026-06-01&endDate=2026-06-15&groupBy=file_type" ``` ### Filter by specific products Use repeatable query parameters to filter results: ```python Python theme={null} import requests response = requests.get( "https://studio.reducto.ai/api/v1/usage/export", headers={"Authorization": "Bearer YOUR_API_KEY"}, params=[ ("groupBy", "feature"), ("product", "parse"), ("product", "extract"), ], ) data = response.json() ``` ```javascript Node.js theme={null} const params = new URLSearchParams(); params.append("groupBy", "feature"); params.append("product", "parse"); params.append("product", "extract"); const response = await fetch( `https://studio.reducto.ai/api/v1/usage/export?${params}`, { headers: { Authorization: "Bearer YOUR_API_KEY" } } ); const data = await response.json(); ``` ```bash cURL theme={null} curl -H "Authorization: Bearer YOUR_API_KEY" \ "https://studio.reducto.ai/api/v1/usage/export?groupBy=feature&product=parse&product=extract" ``` ## Multi-Organization Access If your account belongs to multiple organizations, pass the `orgId` parameter to specify which organization's usage to retrieve. Without it, the API defaults to the organization associated with your API key. ```bash theme={null} curl -H "Authorization: Bearer YOUR_API_KEY" \ "https://studio.reducto.ai/api/v1/usage/export?orgId=org_abc123" ``` You can only query organizations you are a member of. Requesting an org you don't belong to returns a `403` error. ## Error Responses | Status | Meaning | | ------ | ------------------------------------------ | | `401` | Missing or invalid API key | | `400` | Invalid `groupBy` value | | `403` | Not a member of the requested organization | | `500` | Server error | ## Related How credits are calculated per endpoint. Manage API keys, usage alerts, and billing in Studio. # Version Pinning Source: https://docs.reducto.ai/reference/version-pinning How Reducto model versioning works and how to pin versions Reducto continuously improves the models powering document processing. This page explains how versioning works and how to control which version you use. For the current status of each model version, see [Model Versions](/reference/model-versions). ## How Versioning Works Each model version progresses through four stages: 1. **Alpha**: New version available for opt-in testing via the `alpha` config 2. **Default**: Automatically used for all requests 3. **Deprecated**: Old default, still accessible via `alpha` config 4. **Removed**: No longer available Requesting an unknown or removed version returns an error listing the available versions. ## Rollout Process New model versions follow a 4-week rollout window: 1. **Weeks 1-2 (Alpha)**: The new version is available for opt-in testing. You can pin to the new version via the `alpha` config to evaluate it against your workloads before it becomes the default. 2. **Week 3 (Default)**: The new version becomes the default for all requests. The previous default is demoted to deprecated status. 3. **Weeks 3-4 (Deprecated)**: The previous default remains accessible via version pinning for 2 weeks after being replaced. Use this window to migrate any workflows that depend on the old version's behavior. This gives you a guaranteed 4-week window, 2 weeks before and 2 weeks after a version becomes the default, to test and transition between versions. Reducto is deployed as a monolith, so fully frozen snapshots of previous versions are not available outside of on-prem/VPC deployments. Version pinning during the rollout window is the supported mechanism for managing behavioral changes. ## Pinning a Version To pin a specific version, pass it in your request's `settings.alpha` config: ```python Python theme={null} # Layout version pinning (parse) result = client.parse.run( input="https://example.com/document.pdf", settings={ "alpha": { "layout_model": "v2" } } ) # Extract version pinning (requires citations enabled) result = client.extract.run( input="https://example.com/document.pdf", instructions={"schema": {...}}, settings={ "citations": {"enabled": True}, "alpha": { "extract_model": "v2" } } ) # Deep Extract version pinning (requires deep_extract enabled) result = client.extract.run( input="https://example.com/document.pdf", instructions={"schema": {...}}, settings={ "deep_extract": True, "alpha": { "deep_extract_model": "v3" } } ) # Deep Split version pinning (requires deep_split enabled) # For split, alpha is a top-level option, not nested under settings. result = client.split.run( input="https://example.com/document.pdf", split_description=[...], settings={ "deep_split": True }, alpha={ "deep_split_model": "v2" } ) ``` ```javascript Node.js theme={null} // Layout version pinning (parse) const result = await client.parse.run({ input: 'https://example.com/document.pdf', settings: { alpha: { layout_model: 'v2' } } }); // Extract version pinning (requires citations enabled) const extractResult = await client.extract.run({ input: 'https://example.com/document.pdf', instructions: { schema: {...} }, settings: { citations: { enabled: true }, alpha: { extract_model: 'v2' } } }); // Deep Extract version pinning (requires deep_extract enabled) const deepExtractResult = await client.extract.run({ input: 'https://example.com/document.pdf', instructions: { schema: {...} }, settings: { deep_extract: true, alpha: { deep_extract_model: 'v3' } } }); // Deep Split version pinning (requires deep_split enabled) // For split, alpha is a top-level option, not nested under settings. const deepSplitResult = await client.split.run({ input: 'https://example.com/document.pdf', split_description: [...], settings: { deep_split: true }, alpha: { deep_split_model: 'v2' } }); ``` ```bash cURL theme={null} # Layout version pinning (parse) curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "https://example.com/document.pdf", "settings": { "alpha": { "layout_model": "v2" } } }' # Extract version pinning (requires citations enabled) curl -X POST https://platform.reducto.ai/extract \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "https://example.com/document.pdf", "instructions": {"schema": {...}}, "settings": { "citations": {"enabled": true}, "alpha": { "extract_model": "v2" } } }' # Deep Extract version pinning (requires deep_extract enabled) curl -X POST https://platform.reducto.ai/extract \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "https://example.com/document.pdf", "instructions": {"schema": {...}}, "settings": { "deep_extract": true, "alpha": { "deep_extract_model": "v3" } } }' # Deep Split version pinning (requires deep_split enabled) # For split, alpha is a top-level option, not nested under settings. curl -X POST https://platform.reducto.ai/split \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "https://example.com/document.pdf", "split_description": [], "settings": { "deep_split": true }, "alpha": { "deep_split_model": "v2" } }' ``` Model versioning only changes processing behavior. It does not break the API contract. If a pinned version is unknown or has been removed, the request returns an error listing the available versions. # EU data residency & processing Source: https://docs.reducto.ai/security/eu-data-residency For customers requiring that all document data stays strictly within EU For customers who would like to utilize the EU region only, Reducto guarantees that *all customer document data is processed, stored, and deleted exclusively within the EU*, with no exceptions. This applies to ingestion, temporary storage, computation, and deletion workflows. ## EU regions utilized Reducto operates across multiple cloud providers with EU-only data processing: **AWS**: Frankfurt (eu-central-1), Ireland (eu-west-1), Paris (eu-west-3), Stockholm (eu-north-1) **GCP**: Belgium (europe-west1), Frankfurt (europe-west3), Netherlands (europe-west4) **Azure**: Paris (francecentral), Frankfurt (germanywestcentral), Netherlands (westeurope), Sweden (swedencentral) ## Data boundary guarantee No customer document data ever leaves the EU. Only aggregated operational metrics (containing no document content) may be processed outside the EU. ## Subprocessor transparency Reducto maintains a strict EU-only data boundary for all document content. ### Document-processing subprocessors (EU-only execution) | Company | Description | | ------------------------- | ---------------------------------------------- | | Amazon Web Services (AWS) | Compute, storage, networking (EU regions only) | | Google Cloud | Compute, storage, networking (EU regions only) | | Modal Labs | Compute infrastructure (EU regions only) | | OpenAI | Compute (ZDR, EU region only) | ### Vendors receiving only operational telemetry The following vendors receive only aggregated, anonymous service-level metrics and never receive document content: | Company | Description | | ------- | ----------------- | | Sentry | Error monitoring | | PostHog | Product analytics | ## Data retention, expiry & deletion Reducto enforces a strict 24-hour maximum retention window for all customer-submitted data. ### Retention policy All documents, extracted text, and intermediate outputs expire within 24 hours. Jobs are purged automatically every 12 hours. All S3 data is encrypted at rest (AES-256) and in transit (TLS 1.2+). ### Deletion guarantees Automatic, irreversible deletion occurs after the 24-hour window. No backups or long-term archives are maintained. Logs and cache entries containing customer data are never persisted. # Filing Complaints Source: https://docs.reducto.ai/security/filing-complaints At Reducto AI, we are committed to protecting your privacy and complying with HIPAA regulations. We have established a process for individuals to submit complaints regarding our HIPAA policies, procedures, or compliance. **How to Submit a Complaint** If you have a complaint related to our HIPAA policies, procedures, or compliance, please follow these steps: Compose an email with the following information: Subject line: "HIPAA Complaint" Your name and contact information A detailed description of your complaint Any relevant dates, times, or other specific information Send your email to: [support@reducto.ai](mailto:support@reducto.ai) **What Happens Next** Upon receiving your complaint: We will log and record your complaint in our system. You will receive an acknowledgment of your complaint within 3 business days. Our privacy officer will review your complaint and investigate as necessary. We will provide a response to your complaint within 30 days, unless additional time is required for a thorough investigation. We take all HIPAA-related complaints seriously and are committed to addressing them promptly and appropriately. For any questions about this process, please contact our privacy officer at [security@reducto.ai](mailto:privacy@reducto.ai). # Data policies & compliance Source: https://docs.reducto.ai/security/policies At Reducto, we take data security and privacy extremely seriously. We understand the importance of protecting our customers' sensitive information and have implemented robust measures to ensure the highest level of security. This report outlines our data storage practices, encryption protocols, and compliance adherence. ## Data storage 1. **Storage Location**: Reducto utilizes the cloud infrastructure providers listed in our [authorized subprocessors](https://trust.reducto.ai) for storing and processing data. All data is encrypted at rest and in transit. 2. **Access Permissions**: Access to stored data is strictly limited to Reducto's authorized processing services. This ensures that only authenticated processes can interact with the stored data, minimizing the risk of unauthorized access. 3. **Data Retention**: We have a Zero Data Retention policy (ZDR) for users on our "Growth" tier and above, meaning all data submitted via API is set to expire within 24 hours. This means that any data older than 24 hours is automatically deleted, reducing the amount of data we retain and minimizing the potential impact of any data breaches. 4. **Data Usage**: For users on our "Growth" tier and above, we never use any of their data for training purposes. We respect the privacy of our customers and ensure only they have access to the data from their requests. ## Encryption 1. **Encryption at Rest**: All stored data is encrypted at rest using industry-standard encryption algorithms. This means that even if unauthorized individuals were to gain access to the stored data, they would not be able to decipher it without the proper encryption keys. 2. **Encryption in Transit**: We employ encryption protocols to protect data in transit. All communication between our systems and data storage is conducted over secure channels using encryption mechanisms such as SSL/TLS. This ensures that data remains confidential and tamper-proof during transmission. ## Compliance 1. **SOC 2 Type 2**: We have completed our SOC 2 Type I and Type II process. Please reach out to receive the report. This rigorous certification demonstrates our commitment to maintaining a secure and reliable system. It involves a comprehensive audit of our security controls, policies, and procedures by an independent third party. 2. **HIPAA Compliance**: We currently offer a HIPAA compliant processing pipeline for Growth and Enterprise tier customers. By adhering to HIPAA regulations, we ensure that any PHI processed by our system is handled with the utmost care and in compliance with the stringent security and privacy standards set forth by HIPAA. Please reach out to us via email to sign a BAA with us. We continuously monitor and update our security measures to stay ahead of evolving threats and maintain the highest level of protection for our customers' data. Our dedicated security team regularly conducts assessments, penetration testing, and vulnerability scans to identify and address any potential weaknesses in our system. If you have any further questions or require additional information regarding our security practices, please don't hesitate to reach out to [support@reducto.ai](mailto:support@reducto.ai). ### List of authorized subprocessors For a current list of authorized subprocessors, please visit [trust.reducto.ai](https://trust.reducto.ai). # Single sign-on (SSO) FAQ Source: https://docs.reducto.ai/security/sso-saml-faq How to configure SAML or OIDC single sign-on for your Reducto organization. Reducto supports Enterprise SSO, letting your team log in with your identity provider (IdP) credentials. This page covers how setup works and answers common questions from IT and security teams. ## How SSO works at Reducto Users sign in at [accounts.reducto.ai](https://accounts.reducto.ai). Enterprise SSO is configured per organization through a guided setup page. Once SSO is enabled for your organization, members can no longer log in with other methods such as email/password or magic link. To get started, contact your Reducto account team or support. We will send you a shareable setup link that should guide you through the setup process. ## Setup process Contact your Reducto account team to enable Enterprise SSO for your organization. We can also generate a setup link for the person configuring your IdP, even if they do not have a Reducto account. The setup page provides step-by-step guides for Entra ID (Azure AD), Okta, Google, OneLogin, JumpCloud, Duo, Rippling, and Ping Identity, plus a generic guide for any other SAML 2.0 or OIDC provider. It supplies the values to enter into your IdP, including the ACS URL and SP Entity ID. For SAML, you enter your IdP SSO URL, IdP Entity ID, and the token signing certificate. The guide then walks through mapping user attributes and, optionally, roles. After clicking Finish & Go Live, the setup page shows your organization's login URL in the form `https://auth.reducto.ai/saml/{org_slug}/login`. ## Frequently asked questions ### SAML support Yes. Reducto supports SAML 2.0, with both SP-initiated and IdP-initiated logins. OIDC is also supported. Guided setup is available for Entra ID (Azure AD), Okta, Google, OneLogin, JumpCloud, Duo, Rippling, and Ping Identity. Any other IdP that supports SAML 2.0 works through the generic SAML integration. Both values are shown in the guided setup page for your organization. Enter them into your IdP when creating the application (in Entra ID, these are the Reply URL and Identifier fields in Basic SAML Configuration). No. In Entra ID's Basic SAML Configuration, only the Identifier (Entity ID) and Reply URL (ACS URL) are required. The Sign on URL is only used when starting login from a bookmark or the My Apps portal tile. You may leave it blank, or set it to your organization's login URL: `https://auth.reducto.ai/saml/{org_slug}/login`. Yes. After completing the setup guide and clicking Finish & Go Live, take your organization's login URL (`https://auth.reducto.ai/saml/{org_slug}/login`) and replace `/login` with `/metadata`. Navigating to that URL downloads the SP metadata XML, which can be used for a Relying Party Trust configuration. No. The IdP signing certificate is provided during setup. When your IdP rotates its token signing certificate, update the SAML connection with the new certificate by revisiting the setup page. We recommend configuring certificate expiry notifications in your IdP. ### Attributes and provisioning The SAML response should include: | Attribute | Required | Notes | | ---------- | ----------- | -------------------------------------------------- | | Email | Yes | Also used as the NameID / subject | | First name | Recommended | Mapped in the setup guide | | Last name | Recommended | Mapped in the setup guide | | Role | Optional | IdP roles or groups can be mapped to Reducto roles | Yes. The setup guides include directions for mapping roles from your IdP, including mapping based on group membership. Yes. Users are provisioned automatically on their first SSO login. SCIM is available for Okta, Entra ID, OneLogin, JumpCloud, and Ping Identity. Contact your Reducto account team to confirm availability for your organization. ### Login behavior No. Once Enterprise SSO is enabled for your organization, members (and users sharing your organization's email domain) can only log in through your IdP. Yes. Both SP-initiated and IdP-initiated logins are supported. Users with a role that includes the Enterprise SSO permission can configure it from the organization settings page. ## Need help? Contact [support@reducto.ai](mailto:support@reducto.ai) or your Reducto account team for SSO setup assistance. # Split Source: https://docs.reducto.ai/split Divide documents into logical sections for targeted processing Split identifies which pages contain which sections of a document. You describe sections in natural language, and Reducto returns the page numbers where each section lives. Use Split to route document segments into different processing pipelines or to target extraction at specific sections. Under the hood, Split runs [Parse](/parse/overview) to understand the document, then uses an LLM to classify pages against your descriptions. **Split is not chunking.** Split returns page numbers telling you where sections live. Chunking (configured via Parse) breaks content into smaller pieces for embeddings or retrieval. They solve different problems: Split identifies locations, chunking divides content. ![Split endpoint workflow](https://cdn.reducto.ai/documentation_images/SplitGraphic.png) *** ## Quick Start ```python Python theme={null} from pathlib import Path from reducto import Reducto client = Reducto() upload = client.upload(file=Path("financial_report.pdf")) result = client.split.run( input=upload.file_id, split_description=[ { "name": "Executive Summary", "description": "High-level overview and key findings at the beginning of the report" }, { "name": "Financial Statements", "description": "Balance sheet, income statement, and cash flow tables" }, { "name": "Risk Factors", "description": "Section discussing business risks and uncertainties" } ] ) for split in result.result.splits: print(f"{split.name}: pages {split.pages}") ``` ```javascript Node.js theme={null} import Reducto from 'reductoai'; import fs from 'fs'; const client = new Reducto(); const upload = await client.upload({ file: fs.createReadStream('financial_report.pdf'), }); const result = await client.split.run({ input: upload.file_id, split_description: [ { name: 'Executive Summary', description: 'High-level overview and key findings at the beginning of the report' }, { name: 'Financial Statements', description: 'Balance sheet, income statement, and cash flow tables' }, { name: 'Risk Factors', description: 'Section discussing business risks and uncertainties' } ] }); for (const split of result.result.splits) { console.log(`${split.name}: pages ${split.pages}`); } ``` ```go Go theme={null} package main import ( "context" "encoding/json" "fmt" "io" "os" reducto "github.com/reductoai/reducto-go-sdk" "github.com/reductoai/reducto-go-sdk/option" "github.com/reductoai/reducto-go-sdk/shared" ) func main() { client := reducto.NewClient(option.WithAPIKey(os.Getenv("REDUCTO_API_KEY"))) file, _ := os.Open("financial_report.pdf") defer file.Close() upload, _ := client.Upload(context.Background(), reducto.UploadParams{ File: reducto.F[io.Reader](file), }) result, _ := client.Split.Run(context.Background(), reducto.SplitRunParams{ DocumentURL: reducto.F[reducto.SplitRunParamsDocumentURLUnion]( shared.UnionString(upload.FileID), ), SplitDescription: reducto.F([]shared.SplitCategoryParam{ { Name: reducto.F("Executive Summary"), Description: reducto.F("High-level overview and key findings"), }, { Name: reducto.F("Financial Statements"), Description: reducto.F("Balance sheet, income statement, and cash flow tables"), }, { Name: reducto.F("Risk Factors"), Description: reducto.F("Section discussing business risks and uncertainties"), }, }), }) // Access results via SectionMapping for name, pages := range result.Result.SectionMapping { fmt.Printf("%s: pages %v\n", name, pages) } // Or print full result as JSON resultJSON, _ := json.MarshalIndent(result, "", " ") fmt.Println(string(resultJSON)) } ``` ```bash cURL theme={null} # First upload the file FILE_ID=$(curl -s -X POST https://platform.reducto.ai/upload \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -F "file=@financial_report.pdf" | jq -r '.file_id') # Then split curl -X POST https://platform.reducto.ai/split \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "document_url": "'$FILE_ID'", "split_description": [ { "name": "Executive Summary", "description": "High-level overview and key findings at the beginning of the report" }, { "name": "Financial Statements", "description": "Balance sheet, income statement, and cash flow tables" }, { "name": "Risk Factors", "description": "Section discussing business risks and uncertainties" } ] }' ``` This request asks Split to find three sections in a financial report. Split returns the page numbers where each section appears, along with a confidence score for each match. ### Sample Response ```json theme={null} { "result": { "splits": [ {"name": "Executive Summary", "pages": [1, 2], "conf": "high", "partitions": null}, {"name": "Financial Statements", "pages": [15, 16, 17, 18], "conf": "high", "partitions": null}, {"name": "Risk Factors", "pages": [8, 9, 10, 11, 12], "conf": "high", "partitions": null} ], "section_mapping": { "Executive Summary": [1, 2], "Financial Statements": [15, 16, 17, 18], "Risk Factors": [8, 9, 10, 11, 12] } }, "usage": {"num_pages": 25, "credits": 50.0} } ``` Pages are 1-indexed, meaning the first page is page 1, not 0. *** ## Two Ways to Split Split handles two fundamentally different scenarios. ### Scenario 1: Different Sections Need Different Treatment Your document contains distinct sections that each need their own extraction schema or processing logic. A financial report has an executive summary (extract key metrics), financial tables (extract line items), and risk disclosures (extract risk categories). These are different types of content requiring different approaches. For this, you define multiple entries in `split_description`, each describing a different section: ```python Python theme={null} result = client.split.run( input=upload.file_id, split_description=[ { "name": "Account Summary", "description": "Overview section with account balances and totals" }, { "name": "Transaction History", "description": "Table listing individual transactions with dates and amounts" }, { "name": "Disclosures", "description": "Legal disclosures and terms at the end of the statement" } ] ) # Route each section to appropriate processing for split in result.result.splits: if split.name == "Transaction History": transactions = client.extract.run( input=f"jobid://{parse_job_id}", instructions={"schema": transaction_schema}, settings={"array_extract": True}, parsing={"settings": {"page_range": {"start": split.pages[0], "end": split.pages[-1]}}} ) ``` ```javascript Node.js theme={null} const result = await client.split.run({ input: upload.file_id, split_description: [ { name: 'Account Summary', description: 'Overview section with account balances and totals' }, { name: 'Transaction History', description: 'Table listing individual transactions with dates and amounts' }, { name: 'Disclosures', description: 'Legal disclosures and terms at the end of the statement' } ] }); // Route each section to appropriate processing for (const split of result.result.splits) { if (split.name === 'Transaction History') { const transactions = await client.extract.run({ input: `jobid://${parseJobId}`, instructions: { schema: transactionSchema }, settings: { array_extract: true }, parsing: { settings: { page_range: { start: split.pages[0], end: split.pages.at(-1) } } } }); } } ``` ```go Go theme={null} result, _ := client.Split.Run(context.Background(), reducto.SplitRunParams{ DocumentURL: reducto.F[reducto.SplitRunParamsDocumentURLUnion]( shared.UnionString(upload.FileID), ), SplitDescription: reducto.F([]shared.SplitCategoryParam{ { Name: reducto.F("Account Summary"), Description: reducto.F("Overview section with account balances and totals"), }, { Name: reducto.F("Transaction History"), Description: reducto.F("Table listing individual transactions with dates and amounts"), }, { Name: reducto.F("Disclosures"), Description: reducto.F("Legal disclosures and terms at the end of the statement"), }, }), }) // Route each section to appropriate processing for name, pages := range result.Result.SectionMapping { if name == "Transaction History" { fmt.Printf("Processing %s on pages %v\n", name, pages) // Extract with appropriate schema for this section } } ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/split \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "document_url": "reducto://your-file-id", "split_description": [ {"name": "Account Summary", "description": "Overview section with account balances and totals"}, {"name": "Transaction History", "description": "Table listing individual transactions with dates and amounts"}, {"name": "Disclosures", "description": "Legal disclosures and terms at the end of the statement"} ] }' ``` ### Scenario 2: Repeating Sections with Unknown Count Your document contains the same type of section repeated multiple times, but you don't know in advance how many. A consolidated financial statement might have holdings for 3 accounts or 30. A medical records packet might contain intake forms for 5 patients or 50. This is where `partition_key` becomes essential. Without a partition key, Split returns all pages containing "account holdings" as a single group. You'd then need to figure out where one account ends and the next begins. The partition key tells Split to look for a specific identifier within each section and group the pages by that identifier. ```python Python theme={null} result = client.split.run( input=upload.file_id, split_description=[ { "name": "Account Holdings", "description": "Investment holdings table for a specific account", "partition_key": "account_number" } ] ) ``` ```javascript Node.js theme={null} const result = await client.split.run({ input: upload.file_id, split_description: [ { name: 'Account Holdings', description: 'Investment holdings table for a specific account', partition_key: 'account_number' } ] }); ``` ```go Go theme={null} result, _ := client.Split.Run(context.Background(), reducto.SplitRunParams{ DocumentURL: reducto.F[reducto.SplitRunParamsDocumentURLUnion]( shared.UnionString(upload.FileID), ), SplitDescription: reducto.F([]shared.SplitCategoryParam{ { Name: reducto.F("Account Holdings"), Description: reducto.F("Investment holdings table for a specific account"), PartitionKey: reducto.F("account_number"), }, }), }) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/split \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "document_url": "reducto://your-file-id", "split_description": [ { "name": "Account Holdings", "description": "Investment holdings table for a specific account", "partition_key": "account_number" } ] }' ``` The response now includes a `partitions` array that breaks down the section by the values Split found in the document: ```json theme={null} { "result": { "splits": [ { "name": "Account Holdings", "pages": [1, 2, 3, 7, 8, 9, 10, 11], "conf": "high", "partitions": [ {"name": "1234-5678", "pages": [1, 2, 3], "conf": "high"}, {"name": "8765-4321", "pages": [7, 8, 9, 10, 11], "conf": "high"} ] } ], "section_mapping": { "Account Holdings 1234-5678": [1, 2, 3], "Account Holdings 8765-4321": [7, 8, 9, 10, 11] } } } ``` The `name` in each partition is the actual value Split extracted from the document. If the document shows "Account #1234-5678" on pages 1-3 and "Account #8765-4321" on pages 7-11, those become your partition names. The partition key describes what to look for semantically, not an exact string to match. If you set `partition_key` to "account number" but the document says "Acct #1234", Split will still find it. *** ## Connecting Split to Parse and Extract Split is rarely used in isolation. The typical workflow is Parse → Split → Extract, where each step builds on the previous one. You can reuse a Parse result across multiple Split and Extract calls by passing the job ID. Since Parse is often the slowest step, this saves significant time and credits. ```python Python theme={null} # Step 1: Parse the document once parse_result = client.parse.run(input=upload.file_id) job_id = parse_result.job_id # Step 2: Split using the job ID (no re-parsing) split_result = client.split.run( input=f"jobid://{job_id}", split_description=[ {"name": "Summary", "description": "Account summary with balances"}, {"name": "Transactions", "description": "Transaction history table"} ] ) # Step 3: Extract from each section with the appropriate schema summary_schema = { "type": "object", "properties": { "account_number": {"type": "string"}, "current_balance": {"type": "number"}, "available_balance": {"type": "number"} } } transaction_schema = { "type": "object", "properties": { "transactions": { "type": "array", "items": { "type": "object", "properties": { "date": {"type": "string"}, "description": {"type": "string"}, "amount": {"type": "number"} } } } } } for split in split_result.result.splits: schema = summary_schema if split.name == "Summary" else transaction_schema extract_result = client.extract.run( input=f"jobid://{job_id}", instructions={"schema": schema}, parsing={"settings": {"page_range": {"start": split.pages[0], "end": split.pages[-1]}}} ) print(f"{split.name}: {extract_result.result}") ``` ```javascript Node.js theme={null} // Step 1: Parse the document once const parseResult = await client.parse.run({ input: upload.file_id }); const jobId = parseResult.job_id; // Step 2: Split using the job ID (no re-parsing) const splitResult = await client.split.run({ input: `jobid://${jobId}`, split_description: [ { name: 'Summary', description: 'Account summary with balances' }, { name: 'Transactions', description: 'Transaction history table' } ] }); // Step 3: Extract from each section with the appropriate schema const summarySchema = { type: 'object', properties: { account_number: { type: 'string' }, current_balance: { type: 'number' }, available_balance: { type: 'number' } } }; const transactionSchema = { type: 'object', properties: { transactions: { type: 'array', items: { type: 'object', properties: { date: { type: 'string' }, description: { type: 'string' }, amount: { type: 'number' } } } } } }; for (const split of splitResult.result.splits) { const schema = split.name === 'Summary' ? summarySchema : transactionSchema; const extractResult = await client.extract.run({ input: `jobid://${jobId}`, instructions: { schema }, parsing: { settings: { page_range: { start: split.pages[0], end: split.pages.at(-1) } } } }); console.log(`${split.name}:`, extractResult.result); } ``` ```go Go theme={null} // Step 1: Parse the document once parseResult, _ := client.Parse.Run(context.Background(), reducto.ParseRunParams{ DocumentURL: reducto.F[reducto.ParseRunParamsDocumentURLUnion]( shared.UnionString(upload.FileID), ), }) jobID := parseResult.JobID // Step 2: Split using the job ID (no re-parsing) splitResult, _ := client.Split.Run(context.Background(), reducto.SplitRunParams{ DocumentURL: reducto.F[reducto.SplitRunParamsDocumentURLUnion]( shared.UnionString("jobid://" + jobID), ), SplitDescription: reducto.F([]shared.SplitCategoryParam{ {Name: reducto.F("Summary"), Description: reducto.F("Account summary with balances")}, {Name: reducto.F("Transactions"), Description: reducto.F("Transaction history table")}, }), }) // Step 3: Extract from each section using SectionMapping for name, pages := range splitResult.Result.SectionMapping { fmt.Printf("%s: processing pages %v\n", name, pages) // Use pages to set page_range for extraction } ``` ```bash cURL theme={null} # Step 1: Parse the document once PARSE_RESPONSE=$(curl -s -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{"document_url": "reducto://your-file-id"}') JOB_ID=$(echo $PARSE_RESPONSE | jq -r '.job_id') # Step 2: Split using the job ID SPLIT_RESPONSE=$(curl -s -X POST https://platform.reducto.ai/split \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "document_url": "jobid://'$JOB_ID'", "split_description": [ {"name": "Summary", "description": "Account summary with balances"}, {"name": "Transactions", "description": "Transaction history table"} ] }') echo $SPLIT_RESPONSE | jq '.result.splits' ``` When you pass `jobid://` as input, the parsing step is skipped entirely. Any `parsing` options you include won't re-parse the document; they only affect how the already-parsed content is filtered (like limiting which pages to consider for extraction). *** ## Request Parameters ### input (required) The document to process. Accepts: | Format | Example | Description | | --------------- | ------------------------------------------ | ---------------------------------------- | | Upload response | `upload.file_id` or `"reducto://abc123"` | File uploaded via `/upload` | | Public URL | `"https://example.com/doc.pdf"` | Publicly accessible document | | Presigned URL | `"https://bucket.s3.../doc.pdf?X-Amz-..."` | Cloud storage with temporary credentials | | Job ID | `"jobid://7600c8c5-..."` | Reuse a previous Parse result | ### split\_description (required) An array defining the sections to find. Each entry has: | Field | Required | Description | | --------------- | -------- | ------------------------------------------------------------------------------------ | | `name` | Yes | Identifier for this section in the response | | `description` | Yes | Natural language description of what the section contains | | `partition_key` | No | Identifier to look for when a section repeats (e.g., "account number", "patient ID") | Write descriptions that match how the content actually appears in the document. If the section has visual characteristics ("blue header", "signature line at bottom"), mention them. ### split\_rules A prompt that controls how Split handles page classification. The default is: ``` "Split the document into the applicable sections. Sections may only overlap at their first and last page if at all." ``` This default means a page can only belong to multiple sections if it's at the boundary between them. Page 5 can belong to both "Section A" and "Section B" only if it's the last page of A and the first page of B. You can customize this behavior for your use case: ```python Python theme={null} # Allow full overlap when content genuinely spans multiple categories result = client.split.run( input=upload.file_id, split_description=[...], split_rules="Pages can belong to multiple sections. A page with both summary information and transaction data should be included in both sections." ) # Force exclusive classification result = client.split.run( input=upload.file_id, split_description=[...], split_rules="Each page must belong to exactly one section. Choose the most relevant section for each page." ) ``` ```javascript Node.js theme={null} // Allow full overlap const result = await client.split.run({ input: upload.file_id, split_description: [...], split_rules: 'Pages can belong to multiple sections. A page with both summary information and transaction data should be included in both sections.' }); // Force exclusive classification const result = await client.split.run({ input: upload.file_id, split_description: [...], split_rules: 'Each page must belong to exactly one section. Choose the most relevant section for each page.' }); ``` ```go Go theme={null} // Allow full overlap result, _ := client.Split.Run(context.Background(), reducto.SplitRunParams{ DocumentURL: reducto.F[reducto.SplitRunParamsDocumentURLUnion]( shared.UnionString(upload.FileID), ), SplitDescription: reducto.F([]shared.SplitCategoryParam{...}), SplitRules: reducto.F("Pages can belong to multiple sections."), }) // Force exclusive classification result, _ := client.Split.Run(context.Background(), reducto.SplitRunParams{ DocumentURL: reducto.F[reducto.SplitRunParamsDocumentURLUnion]( shared.UnionString(upload.FileID), ), SplitDescription: reducto.F([]shared.SplitCategoryParam{...}), SplitRules: reducto.F("Each page must belong to exactly one section."), }) ``` ```bash cURL theme={null} # Allow full overlap curl -X POST https://platform.reducto.ai/split \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "document_url": "reducto://your-file-id", "split_description": [...], "split_rules": "Pages can belong to multiple sections." }' ``` The `split_rules` string is passed directly to the LLM as instructions, so write it as you would write instructions for a person doing the classification. ### parsing Configuration for how the document is parsed. These options are inherited from [Parse](/parse/overview) and are ignored if your `input` is a `jobid://` reference (since the document was already parsed). ```python Python theme={null} result = client.split.run( input=upload.file_id, split_description=[...], parsing={ "settings": { "page_range": {"start": 1, "end": 50} # Only analyze first 50 pages } } ) ``` ```javascript Node.js theme={null} const result = await client.split.run({ input: upload.file_id, split_description: [...], parsing: { settings: { page_range: { start: 1, end: 50 } // Only analyze first 50 pages } } }); ``` ```go Go theme={null} result, _ := client.Split.Run(context.Background(), reducto.SplitRunParams{ DocumentURL: reducto.F[reducto.SplitRunParamsDocumentURLUnion]( shared.UnionString(upload.FileID), ), SplitDescription: reducto.F([]shared.SplitCategoryParam{...}), Options: reducto.F(shared.BaseProcessingOptionsParam{ PageRange: reducto.F(shared.PageRangeParam{ Start: reducto.F(int64(1)), End: reducto.F(int64(50)), }), }), }) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/split \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "document_url": "reducto://your-file-id", "split_description": [...], "options": { "page_range": {"start": 1, "end": 50} } }' ``` ### settings | Field | Values | Default | Description | | -------------- | ---------------------------- | ------------ | -------------------------------------------------- | | `table_cutoff` | `"truncate"` or `"preserve"` | `"truncate"` | How to handle table content when classifying pages | When analyzing tables, Split truncates them by default to improve speed. This works fine for most cases, but if your `partition_key` values appear deep within tables (row 50 of a 200-row table), you need the full content: ```python Python theme={null} result = client.split.run( input=upload.file_id, split_description=[ { "name": "Holdings", "description": "Investment holdings table", "partition_key": "account_number" } ], settings={"table_cutoff": "preserve"} ) ``` ```javascript Node.js theme={null} const result = await client.split.run({ input: upload.file_id, split_description: [ { name: 'Holdings', description: 'Investment holdings table', partition_key: 'account_number' } ], settings: { table_cutoff: 'preserve' } }); ``` ```go Go theme={null} result, _ := client.Split.Run(context.Background(), reducto.SplitRunParams{ DocumentURL: reducto.F[reducto.SplitRunParamsDocumentURLUnion]( shared.UnionString(upload.FileID), ), SplitDescription: reducto.F([]shared.SplitCategoryParam{ { Name: reducto.F("Holdings"), Description: reducto.F("Investment holdings table"), PartitionKey: reducto.F("account_number"), }, }), // Note: table_cutoff setting may be in experimental options }) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/split \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "document_url": "reducto://your-file-id", "split_description": [ { "name": "Holdings", "description": "Investment holdings table", "partition_key": "account_number" } ], "settings": {"table_cutoff": "preserve"} }' ``` The tradeoff is latency. Preserving tables means more content for the LLM to process. *** ## Sample Response ```json theme={null} { "result": { "splits": [ { "name": "Section Name", "pages": [1, 2, 3], "conf": "high", "partitions": null } ], "section_mapping": { "Section Name": [1, 2, 3] } }, "usage": { "num_pages": 10, "credits": 20.0 } } ``` | Field | Type | Description | | ---------------------------- | ------------- | ------------------------------------------------------------------------------------ | | `result.splits` | array | Array of found sections, one per entry in your `split_description` | | `result.splits[].name` | string | The name you provided | | `result.splits[].pages` | array | Page numbers where this section appears (1-indexed) | | `result.splits[].conf` | string | Either `"high"` or `"low"` indicating match confidence | | `result.splits[].partitions` | array \| null | When using `partition_key`, sub-sections with their own names, pages, and confidence | | `result.section_mapping` | object | Legacy format mapping section names to page arrays. Use `splits` for new code. | | `usage.num_pages` | number | Total pages in the document | | `usage.credits` | number | Credits consumed (2 per page, plus Parse credits if not using `jobid://`) | A section that isn't found still appears in the response with an empty pages array. Always check that `pages` has content before processing: ```python Python theme={null} for split in result.result.splits: if not split.pages: print(f"Warning: {split.name} not found in document") continue # Process the section ``` ```javascript Node.js theme={null} for (const split of result.result.splits) { if (!split.pages || split.pages.length === 0) { console.log(`Warning: ${split.name} not found in document`); continue; } // Process the section } ``` ```go Go theme={null} for name, pages := range result.Result.SectionMapping { if len(pages) == 0 { fmt.Printf("Warning: %s not found in document\n", name) continue } fmt.Printf("Processing %s on pages %v\n", name, pages) } ``` *** ## Async Processing For large documents or batch processing, use the async pattern to avoid timeouts: ```python Python theme={null} import time submission = client.split.run_job( input=upload.file_id, split_description=[...] ) while True: job = client.job.get(submission.job_id) if job.status == "Completed": break if job.status == "Failed": raise Exception(f"Split failed: {job.reason}") time.sleep(2) for split in job.result.splits: print(f"{split.name}: {split.pages}") ``` ```javascript Node.js theme={null} const submission = await client.split.runJob({ input: upload.file_id, split_description: [...] }); let job; while (true) { job = await client.job.retrieve(submission.job_id); if (job.status === 'Completed') break; if (job.status === 'Failed') throw new Error(`Split failed: ${job.reason}`); await new Promise(resolve => setTimeout(resolve, 2000)); } for (const split of job.result.splits) { console.log(`${split.name}: ${split.pages}`); } ``` ```go Go theme={null} import "time" submission, _ := client.Split.RunJob(context.Background(), reducto.SplitRunJobParams{ DocumentURL: reducto.F[reducto.SplitRunJobParamsDocumentURLUnion]( shared.UnionString(upload.FileID), ), SplitDescription: reducto.F([]shared.SplitCategoryParam{...}), }) for { job, _ := client.Job.Get(context.Background(), submission.JobID) if job.Status == "Completed" { // Access results via SectionMapping resultJSON, _ := json.MarshalIndent(job, "", " ") fmt.Println(string(resultJSON)) break } if job.Status == "Failed" { fmt.Println("Job failed:", job.Reason) break } time.Sleep(2 * time.Second) } ``` ```bash cURL theme={null} # Submit async job JOB_ID=$(curl -s -X POST https://platform.reducto.ai/split_async \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "document_url": "reducto://your-file-id", "split_description": [...] }' | jq -r '.job_id') # Poll for completion while true; do STATUS=$(curl -s "https://platform.reducto.ai/job/$JOB_ID" \ -H "Authorization: Bearer $REDUCTO_API_KEY" | jq -r '.status') if [ "$STATUS" = "Completed" ]; then break; fi if [ "$STATUS" = "Failed" ]; then echo "Job failed"; exit 1; fi sleep 2 done # Get results curl -s "https://platform.reducto.ai/job/$JOB_ID" \ -H "Authorization: Bearer $REDUCTO_API_KEY" | jq '.result' ``` Documents over 100 pages should use async to avoid HTTP timeouts. The `.run_job()` method accepts the same parameters as `.run()`. *** ## Troubleshooting If you're unsure of the document structure, start with broad, generic descriptions: ```python Python theme={null} result = client.split.run( input=upload.file_id, split_description=[ {"name": "Introduction", "description": "Opening sections, executive summary, or overview"}, {"name": "Main Content", "description": "Core content, analysis, or detailed information"}, {"name": "Tables/Data", "description": "Tables, figures, numerical data, or structured information"}, {"name": "Appendix", "description": "Supporting materials, references, or supplementary content"} ] ) ``` ```javascript Node.js theme={null} const result = await client.split.run({ input: upload.file_id, split_description: [ { name: 'Introduction', description: 'Opening sections, executive summary, or overview' }, { name: 'Main Content', description: 'Core content, analysis, or detailed information' }, { name: 'Tables/Data', description: 'Tables, figures, numerical data, or structured information' }, { name: 'Appendix', description: 'Supporting materials, references, or supplementary content' } ] }); ``` ```go Go theme={null} result, _ := client.Split.Run(context.Background(), reducto.SplitRunParams{ DocumentURL: reducto.F[reducto.SplitRunParamsDocumentURLUnion]( shared.UnionString(upload.FileID), ), SplitDescription: reducto.F([]shared.SplitCategoryParam{ {Name: reducto.F("Introduction"), Description: reducto.F("Opening sections, executive summary, or overview")}, {Name: reducto.F("Main Content"), Description: reducto.F("Core content, analysis, or detailed information")}, {Name: reducto.F("Tables/Data"), Description: reducto.F("Tables, figures, numerical data, or structured information")}, {Name: reducto.F("Appendix"), Description: reducto.F("Supporting materials, references, or supplementary content")}, }), }) for name, pages := range result.Result.SectionMapping { fmt.Printf("%s: pages %v\n", name, pages) } ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/split \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "document_url": "reducto://your-file-id", "split_description": [ {"name": "Introduction", "description": "Opening sections, executive summary, or overview"}, {"name": "Main Content", "description": "Core content, analysis, or detailed information"}, {"name": "Tables/Data", "description": "Tables, figures, numerical data, or structured information"}, {"name": "Appendix", "description": "Supporting materials, references, or supplementary content"} ] }' ``` Alternatively, Parse the document first and inspect the content to understand its structure, then create targeted split descriptions. When a section returns with no pages: 1. **Check your description.** Is it specific enough? "Transaction table" might not match if the document calls it "Activity History". Include terms that appear in the actual document. 2. **Verify the section exists.** Run Parse first and inspect the content. If the section isn't visible to Parse, Split won't find it either. 3. **Broaden your description.** Start general ("any table with dates and amounts") and narrow down once you confirm Split can find it. When `partitions` is null despite setting `partition_key`: 1. **Check table\_cutoff.** If the partition key appears inside tables, set `settings.table_cutoff` to `"preserve"`. The default truncation might be hiding the values. 2. **Verify the key exists.** The partition key value must actually appear in the document. If you're looking for "account number" but the document uses "portfolio ID", adjust your partition key. 3. **Check for consistent structure.** Partition detection works best when repeating sections have similar layouts. Inconsistent formatting can confuse the classifier. When Split returns pages that don't contain the expected content: 1. **Make descriptions more specific.** If multiple sections have similar content, add distinguishing details: "the transaction table in the Account Activity section" rather than just "transaction table". 2. **Check confidence scores.** Low confidence suggests the match was ambiguous. The LLM made its best guess but wasn't certain. 3. **Adjust split\_rules.** The default overlap rules might be affecting page assignment. If a page legitimately belongs to multiple sections, customize `split_rules` to allow it. Split can timeout on documents over 100 pages: 1. **Use async processing.** Replace `.run()` with `.run_job()` and poll for results. 2. **Parse first, then split.** If you're not already using `jobid://`, parse the document separately and pass the job ID. This isolates the slow parsing step. 3. **Limit page range.** If you know the sections you need are in a specific range, set `parsing.settings.page_range` to process only those pages. *** ## Next Steps Understand what Split is analyzing under the hood. Pull structured data from the sections Split identifies. Control which pages get processed at each step. Handle large documents without timeouts. # Studio Quickstart Source: https://docs.reducto.ai/studio-quickstart Build and deploy your first document workflow in Reducto Studio. [Studio](https://studio.reducto.ai/) is Reducto's visual interface. Build document workflows by configuring classification, parsing, extraction, and editing steps, test on real documents with the citation viewer, then deploy as a Pipeline ID callable from code. Studio also manages API keys, account settings, and team access. If you just want an API Key, visit your [account's API Keys page](https://studio.reducto.ai/api-keys) to create and manage your keys. Follow our [API quickstart here](https://docs.reducto.ai/quickstart). [Classify](/classify/overview) is available via the [API](/api-reference/classify) but is not yet integrated into Studio. You can try it interactively at [classify.reducto.ai](https://classify.reducto.ai). This guide walks you through your first structured extraction from a document of your choice. *** ## 1. Visit Studio Sign-up or log-in at [studio.reducto.ai](https://studio.reducto.ai/). If you're the first to sign-up from your organization, you'll be asked to create and set a name for the organization. You should then land in Studio, where you can explore demo pipelines or create your own. Studio Empty State ## 2. Create an Extract Pipeline The goal here is to upload a document and see the structured output Reducto produces. Extract reads your document, and extracts the data you want in a schema shape you define. Click **Create pipeline**, and select **Extract**. Create Pipeline Drag and drop a file or upload directly. Upload File Hit **Generate** and then choose between **Fast** and **Enhanced** mode. You can use natural language to describe what type of data you want to extract from your document. Reducto will automatically create a well formed schema for you. Alternatively, you can manually configure your schema by building one in the schema builder directly. Generate Schema Hit **Run**. Once the pipeline is done, the **Results** tab will contain the Extract results. Extract Results Review results and compare them to your original document. If you enable [citations](https://docs.reducto.ai/configs/extract/citations), bounding boxes will appear on the original document, linking them to where the data was extracted from. While most customers find value from our default configurations, you may need to adjust yours depending on your documents, schema, and goals. See details [here](https://docs.reducto.ai/configs/overview) on our endpoint configurations. ## 3. Create an API Key Once you're ready to move into production, go back to the homepage and hit **API Keys**. Here you'll be able to create, manage, and edit your account's API keys. Api Keys Once you have an API key, deploying your new pipeline into production is easy. In any pipeline, hit **Deploy** to get a small code snippet, easily copyable into your workflow. *Congrats, you've just created your first pipeline! 🎉* *** ## Next steps Learn how to use Reducto's SDKs to call your pipelines in production. Define JSON schemas to pull structured data from documents, with automatic field detection and confidence scoring. Learn how to use Reducto's CLI to easily let agents use our endpoints. Learn how to Deploy your pipeline with a Pipeline ID. # Uploading Large Files Source: https://docs.reducto.ai/upload/large-files Upload files up to 5GB using presigned URLs For files larger than 100MB, use the presigned URL method. This uploads directly to cloud storage, bypassing the 100MB limit of the standard [Upload endpoint](/upload). | Method | Max Size | When to Use | | ------------------------- | -------- | ---------------------------------------------- | | [Direct upload](/upload) | 100MB | Most files | | Presigned URL (this page) | 5GB | Large PDFs, high-res scans, large spreadsheets | *** ## How It Works ```mermaid theme={null} sequenceDiagram participant You participant Reducto API participant Cloud Storage You->>Reducto API: 1. Request presigned URL Reducto API-->>You: file_id + presigned_url You->>Cloud Storage: 2. Upload file to presigned URL Cloud Storage-->>You: 200 OK You->>Reducto API: 3. Use file_id with Parse/Split/Extract ``` 1. **Request a presigned URL** from Reducto (no file attached) 2. **Upload your file** directly to cloud storage using the presigned URL 3. **Use the file\_id** with Parse, Split, or Extract endpoints *** ## Step 1: Request a Presigned URL Call the upload endpoint *without* attaching a file: ```python Python theme={null} import os import requests response = requests.post( "https://platform.reducto.ai/upload", headers={"Authorization": f"Bearer {os.environ.get('REDUCTO_API_KEY')}"} ) data = response.json() file_id = data["file_id"] presigned_url = data["presigned_url"] print(f"File ID: {file_id}") print(f"Presigned URL: {presigned_url[:80]}...") ``` ```javascript Node.js theme={null} const response = await fetch('https://platform.reducto.ai/upload', { method: 'POST', headers: { 'Authorization': `Bearer ${process.env.REDUCTO_API_KEY}`, }, }); const data = await response.json(); const fileId = data.file_id; const presignedUrl = data.presigned_url; console.log(`File ID: ${fileId}`); console.log(`Presigned URL: ${presignedUrl.slice(0, 80)}...`); ``` ```go Go theme={null} import ( "encoding/json" "net/http" "os" ) req, _ := http.NewRequest("POST", "https://platform.reducto.ai/upload", nil) req.Header.Set("Authorization", "Bearer "+os.Getenv("REDUCTO_API_KEY")) resp, _ := http.DefaultClient.Do(req) defer resp.Body.Close() var data struct { FileID string `json:"file_id"` PresignedURL string `json:"presigned_url"` } json.NewDecoder(resp.Body).Decode(&data) fmt.Printf("File ID: %s\n", data.FileID) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/upload \ -H "Authorization: Bearer $REDUCTO_API_KEY" ``` **Response:** ```json theme={null} { "file_id": "reducto://50c07046-3bac-4844-8c4b-d1428ed9c8f4", "presigned_url": "https://prod-storage.s3.amazonaws.com/50c07046-3bac-4844-8c4b-d1428ed9c8f4?X-Amz-Algorithm=AWS4-HMAC-SHA256&..." } ``` **Save the `file_id` now.** You'll need it in Step 3. The presigned URL is only for uploading — you can't use it to process the document. *** ## Step 2: Upload to Presigned URL Upload your file using a PUT request to the presigned URL: ```python Python theme={null} import requests with open("large_document.pdf", "rb") as f: response = requests.put(presigned_url, data=f) if response.status_code == 200: print("Upload successful!") ``` ```javascript Node.js theme={null} import fs from 'fs'; const fileBuffer = fs.readFileSync('large_document.pdf'); const response = await fetch(presignedUrl, { method: 'PUT', body: fileBuffer, }); if (response.ok) { console.log('Upload successful!'); } ``` ```go Go theme={null} import ( "bytes" "io/ioutil" ) fileBytes, _ := ioutil.ReadFile("large_document.pdf") req, _ := http.NewRequest("PUT", presignedUrl, bytes.NewReader(fileBytes)) resp, _ := http.DefaultClient.Do(req) if resp.StatusCode == 200 { fmt.Println("Upload successful!") } ``` ```bash cURL theme={null} curl -X PUT "$PRESIGNED_URL" -T large_document.pdf ``` **No Content-Type header needed.** When uploading to presigned URLs, you don't need to set a Content-Type header — the file will be accepted as-is. *** ## Step 3: Process with Parse, Split, or Extract Use the `file_id` from Step 1 (not the presigned URL) with any Reducto endpoint: ```python Python theme={null} from reducto import Reducto client = Reducto() # Use the file_id from Step 1 result = client.parse.run(input=file_id) print(f"Processed {result.usage.num_pages} pages") ``` ```javascript Node.js theme={null} import Reducto from 'reductoai'; const client = new Reducto(); // Use the fileId from Step 1 const result = await client.parse.run({ input: fileId }); console.log(`Processed ${result.usage.num_pages} pages`); ``` ```go Go theme={null} import ( reducto "github.com/reductoai/reducto-go-sdk" "github.com/reductoai/reducto-go-sdk/option" "github.com/reductoai/reducto-go-sdk/shared" ) client := reducto.NewClient(option.WithAPIKey(os.Getenv("REDUCTO_API_KEY"))) // Use the FileID from Step 1 result, _ := client.Parse.Run(context.Background(), reducto.ParseRunParams{ ParseConfig: reducto.ParseConfigParam{ DocumentURL: reducto.F[reducto.ParseConfigDocumentURLUnionParam]( shared.UnionString(data.FileID), // file_id from Step 1 ), }, }) fmt.Printf("Processed %d pages\n", result.Usage.NumPages) ``` ```bash cURL theme={null} # Use the FILE_ID from Step 1 curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d "{\"input\": \"$FILE_ID\"}" ``` *** ## Complete Example Here's the full workflow in one script: ```python Python theme={null} import os import requests from reducto import Reducto # Step 1: Get presigned URL response = requests.post( "https://platform.reducto.ai/upload", headers={"Authorization": f"Bearer {os.environ.get('REDUCTO_API_KEY')}"} ) data = response.json() file_id = data["file_id"] presigned_url = data["presigned_url"] # Step 2: Upload to presigned URL with open("large_document.pdf", "rb") as f: requests.put(presigned_url, data=f) # Step 3: Process with Reducto client = Reducto() result = client.parse.run(input=file_id) print(f"Successfully processed {result.usage.num_pages} pages") for chunk in result.result.chunks: print(chunk.content[:200]) ``` ```javascript Node.js theme={null} import Reducto from 'reductoai'; import fs from 'fs'; // Step 1: Get presigned URL const uploadResponse = await fetch('https://platform.reducto.ai/upload', { method: 'POST', headers: { 'Authorization': `Bearer ${process.env.REDUCTO_API_KEY}` }, }); const { file_id: fileId, presigned_url: presignedUrl } = await uploadResponse.json(); // Step 2: Upload to presigned URL await fetch(presignedUrl, { method: 'PUT', body: fs.readFileSync('large_document.pdf'), }); // Step 3: Process with Reducto const client = new Reducto(); const result = await client.parse.run({ input: fileId }); console.log(`Successfully processed ${result.usage.num_pages} pages`); ``` ```bash cURL theme={null} #!/bin/bash # Step 1: Get presigned URL UPLOAD_RESPONSE=$(curl -s -X POST https://platform.reducto.ai/upload \ -H "Authorization: Bearer $REDUCTO_API_KEY") FILE_ID=$(echo $UPLOAD_RESPONSE | jq -r '.file_id') PRESIGNED_URL=$(echo $UPLOAD_RESPONSE | jq -r '.presigned_url') # Step 2: Upload to presigned URL curl -X PUT "$PRESIGNED_URL" -T large_document.pdf # Step 3: Process with Reducto curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d "{\"input\": \"$FILE_ID\"}" ``` *** ## Troubleshooting **Cause:** The presigned URL has expired. **Fix:** Presigned URLs expire after a short time (typically 1 hour). Request a new presigned URL and try again. **Cause:** You might be passing the `presigned_url` instead of the `file_id`. **Fix:** Always use the `file_id` (starts with `reducto://`) with Parse, Split, or Extract — not the presigned URL. **Cause:** Large files on slow connections can timeout. **Fix:** * Use a wired connection if possible * Consider chunked/multipart upload for files >1GB * Implement retry logic with exponential backoff **Cause:** Using incompatible upload methods or headers. **Fix:** * Don't include a Content-Type header — presigned URLs don't require it * For cURL, use `-T filename` instead of `--data-binary @filename` * In Go, use `bytes.NewReader()` to ensure proper Content-Length handling *** ## Related For files under 100MB — simpler, one-step upload. Extract text, tables, and figures from uploaded documents. Process many large files in parallel. Use webhooks for long-running jobs. # Upload Source: https://docs.reducto.ai/upload/overview Upload documents to Reducto for processing The Upload endpoint transfers documents to Reducto, returning a unique `reducto://` identifier you can pass to any downstream endpoint: Classify, Parse, Split, Extract, or Edit. **Files over 100MB?** Use the [presigned URL method](/upload/large-files) which supports files up to 5GB. *** ## When to Use Upload | Your situation | What to do | | ---------------------------- | ---------------------------------------------------------------- | | Local file under 100MB | Use Upload (this page) | | Local file over 100MB | Use [Presigned URL Upload](/upload/large-files) | | File already hosted at a URL | [Skip upload entirely](#url-passthrough) — pass the URL directly | | Presigned S3/GCS/Azure URL | [Skip upload entirely](#url-passthrough) — pass the URL directly | ### Supported File Types | Category | Formats | | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **PDF** | PDF (Portable Document Format) | | **Documents** | DOCX (Word Open XML), DOC (Word Binary), DOTX (Word Template), RTF (Rich Text Format), TXT (Plain Text), WPD (WordPerfect) | | **Spreadsheets** | XLSX (Excel Open XML), XLSM (Excel Macro-Enabled), XLS (Excel Binary), XLTX (Excel Template), XLTM (Excel Macro-Enabled Template), CSV (Comma-Separated Values), QPW (Quattro Pro) | | **Presentations** | PPTX (PowerPoint Open XML), PPT (PowerPoint Binary) | | **Images** | PNG (Portable Network Graphics), JPEG/JPG (Joint Photographic Experts Group), GIF (Graphics Interchange Format), BMP (Bitmap), TIFF (Tagged Image File Format), HEIC (High Efficiency Image Codec), PSD (Adobe Photoshop), PCX (PC Paintbrush), PPM (Portable Pixmap), APNG (Animated PNG), CUR (Windows Cursor), DCX (Multi-page PCX), FTEX (3D Textures), PIXAR (Pixar Image) | **Multi-page images:** TIFF files with multiple pages are processed as multi-page documents. **Image dimension limits:** The dimension limit for images is 50 megapixels total (e.g., \~8200×6100), with a per-axis maximum of 15,000 pixels. *** ## Quick Start ```python Python theme={null} from pathlib import Path from reducto import Reducto client = Reducto() # Reads REDUCTO_API_KEY from environment upload = client.upload(file=Path("document.pdf")) print(upload.file_id) # Output: reducto://a8f8ead1-e360-4ec6-9ccd-b3277421b9ef.pdf # Now use it with Parse, Split, or Extract result = client.parse.run(input=upload.file_id) ``` ```javascript Node.js theme={null} import Reducto from 'reductoai'; import fs from 'fs'; const client = new Reducto(); // Reads REDUCTO_API_KEY from environment const upload = await client.upload({ file: fs.createReadStream('document.pdf'), }); console.log(upload.file_id); // Output: reducto://a8f8ead1-e360-4ec6-9ccd-b3277421b9ef.pdf // Now use it with Parse, Split, or Extract const result = await client.parse.run({ input: upload.file_id }); ``` ```go Go theme={null} package main import ( "context" "fmt" "io" "os" reducto "github.com/reductoai/reducto-go-sdk" "github.com/reductoai/reducto-go-sdk/option" ) func main() { client := reducto.NewClient(option.WithAPIKey(os.Getenv("REDUCTO_API_KEY"))) file, _ := os.Open("document.pdf") defer file.Close() upload, _ := client.Upload(context.Background(), reducto.UploadParams{ File: reducto.F[io.Reader](file), }) fmt.Println(upload.FileID) // Output: reducto://a8f8ead1-e360-4ec6-9ccd-b3277421b9ef.pdf } ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/upload \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -F "file=@document.pdf" # Response: {"file_id": "reducto://a8f8ead1-e360-4ec6-9ccd-b3277421b9ef.pdf"} ``` *** ## URL Passthrough If your document is already accessible via URL, skip the upload step entirely: ```python Python theme={null} # No upload needed — pass URL directly result = client.parse.run(input="https://example.com/document.pdf") ``` ```javascript Node.js theme={null} // No upload needed — pass URL directly const result = await client.parse.run({ input: 'https://example.com/document.pdf', }); ``` ```go Go theme={null} // No upload needed — pass URL directly result, _ := client.Parse.Run(context.Background(), reducto.ParseRunParams{ ParseConfig: reducto.ParseConfigParam{ DocumentURL: reducto.F[reducto.ParseConfigDocumentURLUnionParam]( shared.UnionString("https://example.com/document.pdf"), ), }, }) ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{"input": "https://example.com/document.pdf"}' ``` **This works with:** * Public URLs (https\://...) * Presigned S3, GCS, or Azure Blob URLs * Any URL that returns the file directly when accessed *** ## Response Format ```json theme={null} { "file_id": "reducto://a8f8ead1-e360-4ec6-9ccd-b3277421b9ef.pdf" } ``` | Field | Description | | --------- | -------------------------------------------------------------------------------- | | `file_id` | Unique identifier in `reducto://` format. Pass this to Parse, Split, or Extract. | **Files expire after 24 hours.** The `reducto://` URI becomes invalid after expiration. Re-upload if you need to process the file again. **Reuse the file\_id:** You can process the same document multiple times with different configurations without re-uploading — just use the same `file_id`. *** ## Common Questions | Method | Max Size | | ------------------------------------------- | -------- | | Direct upload (this page) | 100MB | | [Presigned URL upload](/upload/large-files) | 5GB | For files over 100MB, see [Uploading Large Files](/upload/large-files). Files expire **24 hours** after upload. The `reducto://` URI becomes invalid after expiration. Need to process the same file again after 24 hours? Re-upload it. The Upload endpoint accepts one file per request. For batch uploads, make parallel requests: ```python theme={null} from concurrent.futures import ThreadPoolExecutor files = ["doc1.pdf", "doc2.pdf", "doc3.pdf"] with ThreadPoolExecutor(max_workers=10) as executor: uploads = list(executor.map( lambda f: client.upload(file=Path(f)), files )) ``` See our [batch processing guide](/workflows/batch-processing) for production patterns. Yes. Use `DELETE /upload/{file_id}` to remove an uploaded file immediately. Files are also automatically deleted within 24 hours under Reducto's [ZDR policy](/security/policies) for Growth tier and above. See [Deleting Jobs & Uploads](/workflows/deletion) for details. *** ## Troubleshooting **Error:** `Unsupported file format` **Fix:** Check that your file extension matches the [supported file types](#supported-file-types). When uploading programmatically, ensure the filename includes the extension. **Error:** `File size exceeds maximum allowed` **Fix:** Direct upload is limited to 100MB. For larger files, use the [presigned URL method](/upload/large-files) which supports up to 5GB. **Error:** `Request timeout` **Fix:** * For files approaching 100MB, consider using [presigned URL upload](/upload/large-files) * Check your network connection * Implement retry logic with exponential backoff *** ## Next Steps Convert documents into text, tables, and figures. Pull specific fields into structured JSON using a schema. Divide documents into sections for targeted processing. Upload files over 100MB using presigned URLs. # Async Processing Source: https://docs.reducto.ai/workflows/async-overview Process documents asynchronously with job queues, polling, and webhooks Reducto offers two processing modes: synchronous and asynchronous. The SDK provides `run()` and `run_job()` methods that map to different API endpoints: | SDK Method | API Endpoint | Returns | | ------------------------ | ------------------- | ----------------------------------- | | `client.parse.run()` | `POST /parse` | Full result (blocks until complete) | | `client.parse.run_job()` | `POST /parse_async` | Job ID (returns immediately) | The same pattern applies to all endpoints: `/extract` vs `/extract_async`, `/split` vs `/split_async`, and `/pipeline` vs `/pipeline_async`. The Go SDK is currently in alpha and has limited async support. Go users should use the REST API directly for async operations. See the cURL examples below. ## run() vs run\_job() | Method | Behavior | Best for | | ----------- | ------------------------------------------ | --------------------------------------------------- | | `run()` | Calls sync endpoint, blocks until complete | Interactive applications, smaller documents | | `run_job()` | Calls async endpoint, returns job ID | Large documents, high volume, background processing | Both methods produce the same results. The difference is whether you wait synchronously or retrieve results later. ### Synchronous: run() ```python Python theme={null} from reducto import Reducto client = Reducto() # Blocks until parsing completes (may take seconds to minutes) result = client.parse.run(input="https://example.com/document.pdf") print(result.result.chunks) ``` ```typescript TypeScript theme={null} import Reducto from "reductoai"; const client = new Reducto(); // Blocks until parsing completes const result = await client.parse.run({ input: "https://example.com/document.pdf" }); console.log(result.result.chunks); ``` ```go Go theme={null} package main import ( "context" "fmt" "os" reducto "github.com/reductoai/reducto-go-sdk" "github.com/reductoai/reducto-go-sdk/option" "github.com/reductoai/reducto-go-sdk/shared" ) func main() { client := reducto.NewClient(option.WithAPIKey(os.Getenv("REDUCTO_API_KEY"))) // Blocks until parsing completes result, err := client.Parse.Run(context.Background(), reducto.ParseRunParams{ ParseConfig: reducto.ParseConfigParam{ DocumentURL: reducto.F[reducto.ParseConfigDocumentURLUnionParam]( shared.UnionString("https://example.com/document.pdf"), ), }, }) if err != nil { panic(err) } fmt.Println(result.Result.Chunks) } ``` ```bash cURL theme={null} curl -X POST "https://platform.reducto.ai/parse" \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{"input": "https://example.com/document.pdf"}' ``` The `run()` method handles the job lifecycle internally. If the document takes too long, the request may time out. For documents over 50 pages or complex processing, consider using `run_job()` instead. ### Asynchronous: run\_job() ```python Python theme={null} from reducto import Reducto client = Reducto() # Returns immediately with job ID submission = client.parse.run_job(input="https://example.com/document.pdf") print(f"Job submitted: {submission.job_id}") # Retrieve results later via polling or webhook ``` ```typescript TypeScript theme={null} import Reducto from "reductoai"; const client = new Reducto(); // Returns immediately with job ID const submission = await client.parse.runJob({ input: "https://example.com/document.pdf" }); console.log(`Job submitted: ${submission.job_id}`); // Retrieve results later via polling or webhook ``` ```bash cURL theme={null} # Call the async endpoint directly curl -X POST "https://platform.reducto.ai/parse_async" \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{"input": "https://example.com/document.pdf"}' # Response: {"job_id": "abc123-def456"} ``` The `run_job()` method has no limit on concurrent submissions. You can queue thousands of documents and process them in parallel without managing connections or timeouts. ## Job lifecycle When you submit a job via the async endpoint, it moves through these states: | Status | Meaning | | ------------ | -------------------------------------------- | | `Pending` | Job is queued, waiting for a worker | | `InProgress` | A worker is actively processing the document | | `Completing` | Processing finished, results being saved | | `Completed` | Results are ready to retrieve | | `Failed` | Processing failed (check error message) | Jobs typically spend most of their time in `Pending` (waiting for capacity) or `InProgress` (actual processing). ## Polling for results The simplest way to get results from an async job is to poll the job status: ```python Python theme={null} import time from reducto import Reducto client = Reducto() # Submit job submission = client.parse.run_job(input="https://example.com/document.pdf") # Poll until complete while True: job = client.job.get(submission.job_id) if job.status == "Completed": print("Success:", job.result) break elif job.status == "Failed": print("Failed:", job.error) break time.sleep(2) ``` ```typescript TypeScript theme={null} import Reducto from "reductoai"; const client = new Reducto(); // Submit job const submission = await client.parse.runJob({ input: "https://example.com/document.pdf" }); // Poll until complete while (true) { const job = await client.job.retrieve(submission.job_id); if (job.status === "Completed") { console.log("Success:", job.result); break; } else if (job.status === "Failed") { console.log("Failed:", job.error); break; } await new Promise(resolve => setTimeout(resolve, 2000)); } ``` ```bash cURL theme={null} # Submit job JOB_ID=$(curl -s -X POST "https://platform.reducto.ai/parse_async" \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{"input": "https://example.com/document.pdf"}' | jq -r '.job_id') echo "Job submitted: $JOB_ID" # Poll until complete while true; do RESPONSE=$(curl -s "https://platform.reducto.ai/job/$JOB_ID" \ -H "Authorization: Bearer $REDUCTO_API_KEY") STATUS=$(echo $RESPONSE | jq -r '.status') if [ "$STATUS" = "Completed" ]; then echo "Success" echo $RESPONSE | jq '.result' break elif [ "$STATUS" = "Failed" ]; then echo "Failed" break fi sleep 2 done ``` Polling is straightforward but requires keeping a process running. For production systems processing many documents, webhooks are more efficient. ## Priority processing By default, synchronous (`run()`) jobs are prioritized over asynchronous (`run_job()`) jobs. This ensures interactive requests get fast responses while background jobs process when capacity is available. You can request priority processing for async jobs if your account has priority budget available: ```python Python theme={null} submission = client.parse.run_job( input="urgent-document.pdf", async_={ "priority": True } ) ``` ```typescript TypeScript theme={null} const submission = await client.parse.runJob({ input: "https://example.com/urgent-document.pdf", async: { priority: true } }); ``` ```bash cURL theme={null} curl -X POST "https://platform.reducto.ai/parse_async" \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "https://example.com/urgent-document.pdf", "async": {"priority": true} }' ``` Priority jobs are processed before non-priority async jobs but may still queue behind synchronous requests. ## Async endpoints Every Reducto endpoint has a corresponding async variant: | Sync Endpoint | Async Endpoint | SDK Method | | ---------------- | ---------------------- | --------------------------- | | `POST /parse` | `POST /parse_async` | `client.parse.run_job()` | | `POST /extract` | `POST /extract_async` | `client.extract.run_job()` | | `POST /split` | `POST /split_async` | `client.split.run_job()` | | `POST /pipeline` | `POST /pipeline_async` | `client.pipeline.run_job()` | ```python Python theme={null} # Parse parse_job = client.parse.run_job(input="https://example.com/document.pdf") # Extract extract_job = client.extract.run_job( input="https://example.com/document.pdf", instructions={"schema": your_schema} ) # Split split_job = client.split.run_job( input="https://example.com/document.pdf", split_description=[{"name": "Section A", "description": "..."}] ) # Pipeline pipeline_job = client.pipeline.run_job( input="https://example.com/document.pdf", pipeline_id="your_pipeline_id" ) ``` ```typescript TypeScript theme={null} // Parse const parseJob = await client.parse.runJob({ input: "https://example.com/document.pdf" }); // Extract const extractJob = await client.extract.runJob({ input: "https://example.com/document.pdf", instructions: { schema: yourSchema } }); // Split const splitJob = await client.split.runJob({ input: "https://example.com/document.pdf", splitDescription: [{ name: "Section A", description: "..." }] }); // Pipeline const pipelineJob = await client.pipeline.runJob({ input: "document.pdf", pipelineId: "your_pipeline_id" }); ``` ```bash cURL theme={null} # Parse async curl -X POST "https://platform.reducto.ai/parse_async" \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{"input": "https://example.com/document.pdf"}' # Extract async curl -X POST "https://platform.reducto.ai/extract_async" \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "https://example.com/document.pdf", "instructions": {"schema": {"field": "string"}} }' # Split async curl -X POST "https://platform.reducto.ai/split_async" \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "https://example.com/document.pdf", "split_description": [{"name": "Section A", "description": "..."}] }' # Pipeline async curl -X POST "https://platform.reducto.ai/pipeline_async" \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "https://example.com/document.pdf", "pipeline_id": "your_pipeline_id" }' ``` ## Using metadata Include metadata with your job submission to help identify and route results: ```python Python theme={null} submission = client.parse.run_job( input="https://example.com/document.pdf", async_={ "metadata": { "user_id": "user_123", "document_type": "invoice", "batch_id": "batch_456" } } ) ``` ```typescript TypeScript theme={null} const submission = await client.parse.runJob({ input: "https://example.com/document.pdf", async: { metadata: { userId: "user_123", documentType: "invoice", batchId: "batch_456" } } }); ``` ```bash cURL theme={null} curl -X POST "https://platform.reducto.ai/parse_async" \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "https://example.com/document.pdf", "async": { "metadata": { "user_id": "user_123", "document_type": "invoice", "batch_id": "batch_456" } } }' ``` The metadata is included in webhook notifications, making it easy to match results back to your application context without maintaining a separate mapping. ## When to use async **Use `run()` when:** * Processing single documents interactively * Document size is small (under 20 pages) * You need results immediately in the same request * Testing and development **Use `run_job()` / async endpoints when:** * Processing many documents in parallel * Documents are large or complex * You want fire-and-forget with webhook notification * Building batch processing pipelines * Processing in background workers ## Job Retention **Jobs are deleted after 12 hours.** This is part of Reducto's zero data retention (ZDR) policy. If you query a job ID from more than 12 hours ago, you'll receive a "Job not found" error. **Default behavior:** Job results are retained for 12 hours. After this window, you'll need to reprocess the document. **For longer retention:** Enable `persist_results` to keep results indefinitely: ```python Python theme={null} result = client.parse.run( input="document.pdf", settings={"persist_results": True} ) # This job's results will be stored indefinitely ``` ```typescript TypeScript theme={null} const result = await client.parse.run({ input: "document.pdf", settings: { persist_results: true } }); // This job's results will be stored indefinitely ``` ```bash cURL theme={null} curl -X POST "https://platform.reducto.ai/parse" \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "https://example.com/document.pdf", "settings": {"persist_results": true} }' ``` `persist_results` requires opting in to Reducto Studio. Contact support to enable this feature for your organization. **Best practice:** Always store results in your own database when you receive them via polling or webhook, rather than relying on Reducto's retention. *** You can also delete job artifacts on demand using `DELETE /job/{job_id}`. This removes the stored output before the automatic retention window. See [Deleting Jobs & Uploads](/workflows/deletion) for details. ## API Reference See the full API documentation for async endpoints: * [Parse Async](/api-reference/async-parse) - `POST /parse_async` * [Extract Async](/api-reference/extract-async) - `POST /extract_async` * [Split Async](/api-reference/split-async) - `POST /split_async` * [Pipeline Async](/api-reference/pipeline-async) - `POST /pipeline_async` * [Get Job](/api-reference/get-jobs) - `GET /job/{job_id}` * [Delete Job](/api-reference/delete-job) - `DELETE /job/{job_id}` *** ## Related Get notified when jobs complete instead of polling. Process many documents in parallel with run(). Reuse parsed documents across multiple calls. Bundle workflows into a single API call. # Batch Processing Source: https://docs.reducto.ai/workflows/batch-processing Process multiple documents in parallel When you need to process many documents, batch processing lets you run multiple requests concurrently. This is faster than processing documents sequentially and more suitable for immediate results than webhooks. **Save 20% on bulk parsing:** for non-urgent workloads, submit jobs through the [batch queue](/workflows/batch-queue) (`queue_priority: "batch"`) for a 20% credit discount with a 12-hour completion guarantee. ## When to use batch processing | Approach | Best for | | ---------------------------------------- | ------------------------------------------------------- | | **Batch processing** (this page) | Processing many documents, need results immediately | | **[Webhooks](/workflows/svix-webhooks)** | Fire-and-forget, long documents, notification when done | | **Sequential** | Simple scripts, debugging, rate-limited scenarios | ## Async Python (recommended) Use `AsyncReducto` with `asyncio` for the best performance. The semaphore controls concurrency to avoid overwhelming the API. ### Processing URLs If your documents are already hosted (S3, web server, etc.), process URLs directly: ```python theme={null} import asyncio from reducto import AsyncReducto client = AsyncReducto() async def batch_parse_urls(urls: list[str], max_concurrency: int = 50): """Parse multiple URLs concurrently.""" sem = asyncio.Semaphore(max_concurrency) async def process(url: str): async with sem: result = await client.parse.run(input=url) return {"url": url, "pages": result.usage.num_pages} tasks = [process(url) for url in urls] return await asyncio.gather(*tasks) # Usage urls = [ "https://example.com/doc1.pdf", "https://example.com/doc2.pdf", "https://example.com/doc3.pdf", ] results = asyncio.run(batch_parse_urls(urls)) ``` ### Processing local files For local files, upload first then parse: ```python theme={null} import asyncio from pathlib import Path from reducto import AsyncReducto client = AsyncReducto() async def batch_parse_files(files: list[Path], max_concurrency: int = 50): """Parse multiple local files concurrently.""" sem = asyncio.Semaphore(max_concurrency) async def process(path: Path): async with sem: upload = await client.upload(file=path) result = await client.parse.run(input=upload) return {"file": path.name, "pages": result.usage.num_pages} tasks = [process(path) for path in files] return await asyncio.gather(*tasks) # Usage files = list(Path("documents").glob("*.pdf")) results = asyncio.run(batch_parse_files(files)) for r in results: print(f"{r['file']}: {r['pages']} pages") ``` ### With progress bar ```python theme={null} import asyncio from pathlib import Path from reducto import AsyncReducto from tqdm.asyncio import tqdm client = AsyncReducto() async def batch_parse_with_progress(files: list[Path], max_concurrency: int = 50): sem = asyncio.Semaphore(max_concurrency) async def process(path: Path): async with sem: upload = await client.upload(file=path) result = await client.parse.run(input=upload) return {"file": path.name, "pages": result.usage.num_pages} tasks = [process(path) for path in files] return await tqdm.gather(*tasks, desc="Processing documents") files = list(Path("documents").glob("*.pdf")) results = asyncio.run(batch_parse_with_progress(files)) ``` ### With error handling Some documents may fail (corrupt files, unsupported formats). Handle errors gracefully to avoid losing all results: ```python theme={null} import asyncio from pathlib import Path from reducto import AsyncReducto client = AsyncReducto() async def batch_parse_safe(files: list[Path], max_concurrency: int = 50): """Parse files with error handling.""" sem = asyncio.Semaphore(max_concurrency) async def process(path: Path): async with sem: try: upload = await client.upload(file=path) result = await client.parse.run(input=upload) return {"file": path.name, "success": True, "pages": result.usage.num_pages} except Exception as e: return {"file": path.name, "success": False, "error": str(e)} tasks = [process(path) for path in files] results = await asyncio.gather(*tasks) successes = [r for r in results if r["success"]] failures = [r for r in results if not r["success"]] print(f"Processed {len(successes)} successfully, {len(failures)} failed") return results files = list(Path("documents").glob("*.pdf")) results = asyncio.run(batch_parse_safe(files)) ``` ## Sync Python with threading If you can't use async, use `ThreadPoolExecutor` with the synchronous client: ```python theme={null} from concurrent.futures import ThreadPoolExecutor, as_completed from pathlib import Path from reducto import Reducto client = Reducto() def batch_parse_sync(files: list[Path], max_workers: int = 10): """Parse files using thread pool.""" def process(path: Path): upload = client.upload(file=path) result = client.parse.run(input=upload) return {"file": path.name, "pages": result.usage.num_pages} results = [] with ThreadPoolExecutor(max_workers=max_workers) as executor: futures = {executor.submit(process, f): f for f in files} for future in as_completed(futures): try: results.append(future.result()) except Exception as e: results.append({"file": futures[future].name, "error": str(e)}) return results files = list(Path("documents").glob("*.pdf")) results = batch_parse_sync(files) ``` ## Batch extraction The same patterns work for extraction. Define your schema once and apply it to all documents: ```python theme={null} import asyncio from reducto import AsyncReducto client = AsyncReducto() INVOICE_SCHEMA = { "type": "object", "properties": { "invoice_number": {"type": "string", "description": "Invoice number"}, "date": {"type": "string", "description": "Invoice date"}, "total": {"type": "number", "description": "Total amount"} } } async def batch_extract(urls: list[str], schema: dict, max_concurrency: int = 20): sem = asyncio.Semaphore(max_concurrency) async def extract(url: str): async with sem: result = await client.extract.run( input=url, instructions={"schema": schema} ) return {"url": url, "data": result.result} tasks = [extract(url) for url in urls] return await asyncio.gather(*tasks) urls = ["https://example.com/invoice1.pdf", "https://example.com/invoice2.pdf"] results = asyncio.run(batch_extract(urls, INVOICE_SCHEMA)) ``` ## JavaScript / TypeScript ```javascript theme={null} import Reducto from 'reductoai'; import fs from 'fs'; import { glob } from 'glob'; const client = new Reducto(); async function batchParse(files) { const results = await Promise.all( files.map(async (file) => { try { const upload = await client.upload({ file: fs.createReadStream(file) }); const result = await client.parse.run({ input: upload }); return { file, pages: result.usage.num_pages, success: true }; } catch (error) { return { file, error: error.message, success: false }; } }) ); return results; } const files = glob.sync('documents/*.pdf'); const results = await batchParse(files); console.log(results); ``` ## Saving results Save results as you process to avoid losing work: ```python theme={null} import asyncio import json from pathlib import Path from reducto import AsyncReducto client = AsyncReducto() async def batch_parse_and_save(files: list[Path], output_dir: Path, max_concurrency: int = 50): output_dir.mkdir(exist_ok=True) sem = asyncio.Semaphore(max_concurrency) async def process(path: Path): async with sem: upload = await client.upload(file=path) result = await client.parse.run(input=upload) # Save immediately output_path = output_dir / f"{path.stem}.json" output_path.write_text(result.model_dump_json(indent=2)) return {"file": path.name, "output": str(output_path)} tasks = [process(path) for path in files] return await asyncio.gather(*tasks) files = list(Path("documents").glob("*.pdf")) results = asyncio.run(batch_parse_and_save(files, Path("output"))) ``` ## Concurrency limits | Method | Recommended concurrency | | ---------------------- | -------------------------- | | `AsyncReducto` | 50-200 concurrent requests | | `ThreadPoolExecutor` | 10-50 workers | | `run_job()` (webhooks) | Unlimited | Higher concurrency means faster processing but may hit rate limits. Start with lower values and increase as needed. ## What about cURL? Batch processing requires programming constructs (loops, concurrency control, error handling) that aren't practical in cURL. For single-document processing via cURL, see the [API reference](/api-reference/parse). For batch workflows without writing code, consider: * [Reducto CLI](/cli) for scripting * [Studio pipelines](/studio-quickstart) for visual configuration ## Best practices 1. **Use async when possible**: `AsyncReducto` is more efficient than threading 2. **Handle errors gracefully**: Don't let one failure stop the entire batch 3. **Save incrementally**: Write results to disk as they complete 4. **Monitor progress**: Use `tqdm` or logging to track progress 5. **Set reasonable concurrency**: Start low (20-50) and increase if stable For very large batches or long-running documents, consider [webhooks](/workflows/svix-webhooks) instead. They're better suited for fire-and-forget processing where you don't need immediate results. # Batch Queue Source: https://docs.reducto.ai/workflows/batch-queue Discounted async processing for non-urgent workloads The batch queue is an opt-in lane for async jobs that don't need immediate turnaround. Jobs run when there's spare capacity and are guaranteed to complete within a published SLA. In return, they consume fewer credits than the standard lane. ## When to use the batch queue | Approach | Best for | | -------------------------------------------- | ---------------------------------------------------------------------------------------------- | | **Batch queue** (this page) | Bulk uploads, backfills, overnight runs, anything where minutes-to-hours latency is acceptable | | **[Async parse](/workflows/async-overview)** | Standard async: immediate processing, normal pricing | | **[Sync parse](/parse)** | Small documents, interactive flows: get the result back in one request | Pick the batch queue when you can trade latency for cost. [Webhooks](/workflows/svix-webhooks) work with both async lanes (standard and batch), and we generally recommend them over polling `/job/{id}`. You get the result pushed to you as soon as it's ready instead of paying for the round-trips. ## How to opt in The batch queue currently works with **`/parse_async` only**. Set `queue_priority: "batch"` on a `/parse_async` request to use it. No other endpoint supports the batch queue today. `queue_priority` has no effect on synchronous `/parse` or on the other async endpoints (`/extract_async`, `/split_async`, `/edit_async`). ```python Python theme={null} from reducto import AsyncReducto client = AsyncReducto() response = await client.parse.run_job( input="https://example.com/large-doc.pdf", queue_priority="batch", ) ``` ```javascript Node.js theme={null} import Reducto from "reductoai"; const client = new Reducto(); const response = await client.parse.runJob({ input: "https://example.com/large-doc.pdf", queue_priority: "batch", }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse_async \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "document_url": "https://example.com/large-doc.pdf", "queue_priority": "batch" }' ``` The response shape is identical to a regular `/parse_async` submission. You get back a `job_id` and poll `/job/{id}` for the result. ## SLA Batch jobs are guaranteed to complete within **12 hours** of submission. The 12-hour window is the firm commitment. ## Credit Consumption Batch jobs consume **20% fewer credits** than the same parse on the standard lane. The discount is applied to the number of credits used, not to the monetary cost of each credit. The credit usage returned in `/job/{id}` already reflects the reduced amount, so no separate invoicing step is needed. Because the reduction applies to credit consumption, it stacks on top of any existing per-organisation credit rate. Customers on legacy or volume pricing keep their existing per-credit rate and additionally consume 20% fewer credits when they use the batch queue. ## Back-off and capacity limits If the batch queue is saturated, `/parse_async` returns HTTP 503 with a `Retry-After: 300` header. Well-behaved clients will pause for the suggested interval before retrying; SDKs handle this automatically. The cap is a safety valve, and under normal operation you should never see it. ## Out of scope today * The batch queue is supported on `/parse_async` only. It is not available on synchronous `/parse` or on the other async endpoints (`/extract_async`, `/split_async`, `/edit_async`); these do not honour `queue_priority` and run at standard priority and pricing. * Per-customer batch-discount tiers are not available; the 20% is global. # Chaining API Calls Source: https://docs.reducto.ai/workflows/chaining-endpoints Combine Classify, Parse, and Extract to build document processing workflows Reducto endpoints can be chained together to build multi-step document processing workflows. A common pattern is Classify first to determine document type, then Parse and Extract with the right configuration for that type. When you call Parse, Reducto returns a `job_id` that represents the parsed document. You can pass this job ID to subsequent Extract or Split calls using the `jobid://` prefix, which skips re-parsing and uses the cached result. This saves both time and credits when you need to run multiple operations on the same document. ## The jobid:// protocol After parsing a document, the response includes a `job_id`: ```python theme={null} from pathlib import Path upload = client.upload(file=Path("document.pdf")) parse_result = client.parse.run(input=upload) print(parse_result.job_id) # "7600c8c5-a52f-49d2-8a7d-d75d1b51e141" ``` To reuse this parsed content in Extract or Split, prefix the job ID with `jobid://`: ```python theme={null} # Extract using the parsed document (no re-parsing) extract_result = client.extract.run( input=f"jobid://{parse_result.job_id}", instructions={"schema": your_schema} ) ``` When Reducto sees `jobid://`, it retrieves the cached parse result instead of processing the document again. Any parsing options you include in the request are ignored since the document was already parsed. ## Common chaining patterns ### Parse → Extract The most common pattern. Parse once, then run one or more extractions with different schemas: ```python theme={null} from pathlib import Path from reducto import Reducto client = Reducto() # Step 1: Upload and parse the document upload = client.upload(file=Path("financial-report.pdf")) parse_result = client.parse.run(input=upload) job_id = parse_result.job_id # Step 2: Extract summary metrics summary = client.extract.run( input=f"jobid://{job_id}", instructions={"schema": { "type": "object", "properties": { "total_revenue": {"type": "number"}, "net_income": {"type": "number"} } }} ) # Step 3: Extract detailed line items (same parsed document) line_items = client.extract.run( input=f"jobid://{job_id}", instructions={"schema": { "type": "object", "properties": { "expenses": {"type": "array", "items": {"type": "object"}} } }}, settings={"deep_extract": True} ) ``` Without chaining, each Extract call would re-parse the document. With chaining, you parse once and pay for parsing credits once. ### Parse → Split → Extract For documents with distinct sections that need different extraction schemas: ```python theme={null} from pathlib import Path # Step 1: Upload and parse upload = client.upload(file=Path("contract.pdf")) parse_result = client.parse.run(input=upload) job_id = parse_result.job_id # Step 2: Split into sections split_result = client.split.run( input=f"jobid://{job_id}", split_description=[ {"name": "Terms", "description": "Terms and conditions section"}, {"name": "Pricing", "description": "Pricing and payment terms"}, {"name": "SLA", "description": "Service level agreement"} ] ) # Step 3: Extract from specific sections for section in split_result.result.splits: if section.pages: extract_result = client.extract.run( input=f"jobid://{job_id}", instructions={"schema": get_schema_for_section(section.name)}, parsing={"settings": {"page_range": { "start": section.pages[0], "end": section.pages[-1] }}} ) ``` ### Classify → Parse → Extract When you need to determine document type before choosing an extraction schema, use Classify first. ```python Python theme={null} from reducto import Reducto client = Reducto() # Step 1: Classify the document type classification = client.classify.run( input=document_url, classification_schema=[ { "category": "invoice", "criteria": ["billing information", "itemized charges", "payment details"], }, { "category": "receipt", "criteria": ["single transaction", "store or merchant name", "payment method"], }, { "category": "purchase_order", "criteria": ["order number", "requested items", "delivery instructions"], }, ], ) doc_type = classification.result.category # Step 2: Parse the document parse_result = client.parse.run(input=document_url) job_id = parse_result.job_id # Step 3: Extract with the right schema for this document type if doc_type == "invoice": schema = invoice_schema elif doc_type == "receipt": schema = receipt_schema else: schema = purchase_order_schema result = client.extract.run( input=f"jobid://{job_id}", instructions={"schema": schema} ) ``` ```javascript Node.js theme={null} import Reducto from 'reductoai'; const client = new Reducto(); // Step 1: Classify the document type const classification = await client.classify.run({ input: documentUrl, classification_schema: [ { category: 'invoice', criteria: ['billing information', 'itemized charges', 'payment details'], }, { category: 'receipt', criteria: ['single transaction', 'store or merchant name', 'payment method'], }, { category: 'purchase_order', criteria: ['order number', 'requested items', 'delivery instructions'], }, ], }); const docType = classification.result.category; // Step 2: Parse the document const parseResult = await client.parse.run({ input: documentUrl }); const jobId = parseResult.job_id; // Step 3: Extract with the right schema for this document type let schema; if (docType === 'invoice') { schema = invoiceSchema; } else if (docType === 'receipt') { schema = receiptSchema; } else { schema = purchaseOrderSchema; } const result = await client.extract.run({ input: `jobid://${jobId}`, instructions: { schema } }); ``` ```bash cURL theme={null} # Step 1: Classify the document type CLASSIFY_RESULT=$(curl -s -X POST https://platform.reducto.ai/classify \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "https://example.com/document.pdf", "classification_schema": [ {"category": "invoice", "criteria": ["billing information", "itemized charges"]}, {"category": "receipt", "criteria": ["single transaction", "merchant name"]}, {"category": "purchase_order", "criteria": ["order number", "requested items"]} ] }') DOC_TYPE=$(echo $CLASSIFY_RESULT | jq -r '.result.category') echo "Document type: $DOC_TYPE" # Step 2: Parse the document PARSE_RESULT=$(curl -s -X POST https://platform.reducto.ai/parse \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{"input": "https://example.com/document.pdf"}') JOB_ID=$(echo $PARSE_RESULT | jq -r '.job_id') # Step 3: Extract with an appropriate schema based on classification # (choose your schema based on $DOC_TYPE) curl -X POST https://platform.reducto.ai/extract \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d "{ \"input\": \"jobid://$JOB_ID\", \"instructions\": { \"schema\": { \"type\": \"object\", \"properties\": { \"total\": {\"type\": \"number\", \"description\": \"Total amount\"} } } } }" ``` This pattern is useful when processing mixed document types from a single upload queue. Classify is purpose-built for document routing and costs only 0.5 credits per page of context, compared to using Extract as a workaround. ## Multiple job IDs Extract also accepts a list of job IDs, which combines the parsed content from multiple documents into a single extraction context: ```python theme={null} # Parse multiple documents (documents can be URLs or uploaded file IDs) job_ids = [] for doc in documents: result = client.parse.run(input=doc) job_ids.append(result.job_id) # Extract across all documents combined_result = client.extract.run( input=[f"jobid://{jid}" for jid in job_ids], instructions={"schema": aggregation_schema} ) ``` This behaves like [multi-document pipelines](/workflows/multi-document-pipelines): the extraction sees all documents together and returns a single result. Design your schema accordingly if you want data from each document. ## Supported endpoints | Endpoint | Accepts jobid:// | Notes | | -------- | ---------------- | ----------------------------------- | | Parse | Yes | Reprocesses with different settings | | Extract | Yes | Single ID or list of IDs | | Split | Yes | Single ID only | | Classify | No | Accepts URLs and upload responses | | Edit | No | Requires actual document URL | ## Credit savings When you use `jobid://`, you only pay parse credits once regardless of how many subsequent calls you make: | Without chaining | With chaining | | -------------------------- | ---------------------- | | Parse (4 credits) | Parse (4 credits) | | Extract #1 (4 + 2 credits) | Extract #1 (2 credits) | | Extract #2 (4 + 2 credits) | Extract #2 (2 credits) | | **Total: 16 credits** | **Total: 8 credits** | The savings scale with document size and number of operations. ## Job ID retention Parse job IDs are retained for 12 hours by default. If you need to chain calls after this window, you'll need to re-parse the document. For workflows that span longer periods, consider storing the parsed content or using [pipelines](/workflows/pipeline-basics) which handle this automatically. *** ## Related Categorize documents by type before processing. Bundle multi-step workflows into a single API call. Divide documents into sections for targeted extraction. Pull structured data from parsed documents. # Deleting Jobs & Uploads Source: https://docs.reducto.ai/workflows/deletion Remove job artifacts and uploaded files before the end of the automatic retention window. Reducto lets you delete job artifacts and uploaded files on demand instead of waiting for the end of the automatic retention window. Organizations on Growth and Enterprise tiers already benefit from Reducto's [Zero Data Retention (ZDR) policy](/security/policies), which typically purges all API processing artifacts within 24 hours. These endpoints are for cases where you need to delete artifacts before the 24-hour period. ## What gets deleted `DELETE /job/{job_id}` deletes the **API-layer artifacts** that Reducto stores in live systems when processing a job: | Artifact type | Examples | | ------------------ | ------------------------------------------ | | Job output | Parsed JSON, extracted data, split results | | Intermediate files | Converted images, page renders | Uploaded files are managed separately. To remove a file you uploaded via `POST /upload`, use `DELETE /upload/{file_id}`. When cleanup finishes, Reducto tombstones the job record. Tombstoning marks the record as deleted so Reducto no longer serves it through job retrieval endpoints; residual copies may remain briefly in backups or object-store versions until they are removed through ordinary backup rotation. **What is not affected:** * **Studio data.** Jobs created through Studio pipelines are managed separately. These endpoints only delete artifacts stored by the Reducto API. * **Your source documents.** If you provided a URL to an externally hosted file, that file is untouched. * **Completed downstream results.** If another job already consumed this job's output (e.g. chained extract after parse), the downstream job's own artifacts remain until you delete them separately. * **Child jobs of a pipeline.** Each pipeline step (parse, split, extract, edit) runs as its own job with its own ID and its own artifacts. Deleting the parent pipeline job does not delete them. *** ## Delete a job `DELETE /job/{job_id}` is asynchronous. It validates your request, marks the job as pending deletion, enqueues a background cleanup task, and returns `202 Accepted`. Artifact cleanup runs in the background. ```python Python theme={null} import os import requests api_key = os.environ["REDUCTO_API_KEY"] base_url = "https://platform.reducto.ai" response = requests.delete( f"{base_url}/job/{job_id}", headers={"Authorization": f"Bearer {api_key}"}, ) print(response.status_code) # 202 print(response.json()) # {"job_id": "abc123"} ``` ```javascript Node.js theme={null} const apiKey = process.env.REDUCTO_API_KEY; const baseUrl = "https://platform.reducto.ai"; const response = await fetch(`${baseUrl}/job/${jobId}`, { method: "DELETE", headers: { Authorization: `Bearer ${apiKey}` }, }); console.log(response.status); // 202 console.log(await response.json()); // { job_id: "abc123" } ``` ```bash cURL theme={null} curl -X DELETE "https://platform.reducto.ai/job/{job_id}" \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -w "\nHTTP status: %{http_code}\n" # HTTP status: 202 ``` ### HTTP status codes Job deletion is asynchronous, so the status codes reflect the lifecycle of the deletion process. | Code | Meaning | When | | ---------------------- | ------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------- | | **202 Accepted** | Deletion request accepted | Returned by `DELETE /job/{job_id}`. Cleanup is running in the background. | | **409 Conflict** | Deletion in progress | Returned by `GET /job/{job_id}` while background cleanup is still running. | | **410 Gone** | Job deleted | Returned by `GET /job/{job_id}` after cleanup finishes and the job is tombstoned. | | **404 Not Found** | Job does not exist | The job ID is invalid or belongs to a different organization. | | **400 Bad Request** | Job not in a terminal state | The job is still pending, in progress, or completing. Only completed or failed jobs can be deleted. | | **422 Not applicable** | Deletion not available for your tier | On-demand deletion is only available to Growth and Enterprise organizations. The response body carries error code `NOT_APPLICABLE`. | ### Deleting persisted artifacts If you used `persist_results: true` when creating the job, persisted result artifacts are kept by default even after deletion according to your account settings or agreement with Reducto. Pass `include_persisted=true` to also delete those long-retention artifacts: ```python Python theme={null} response = requests.delete( f"{base_url}/job/{job_id}", headers={"Authorization": f"Bearer {api_key}"}, params={"include_persisted": True}, ) ``` ```javascript Node.js theme={null} const response = await fetch( `${baseUrl}/job/${jobId}?include_persisted=true`, { method: "DELETE", headers: { Authorization: `Bearer ${apiKey}` }, } ); ``` ```bash cURL theme={null} curl -X DELETE "https://platform.reducto.ai/job/{job_id}?include_persisted=true" \ -H "Authorization: Bearer $REDUCTO_API_KEY" ``` ### Response | Field | Type | Description | | -------- | -------- | ------------------------------- | | `job_id` | `string` | The ID of the job being deleted | Store the deletion response with your own request logs if you need evidence that Reducto accepted the deletion request. ### Failure recovery If the background cleanup task fails, the deletion marker is rolled back automatically. The job becomes retrievable again via `GET /job/{job_id}` and you can retry the `DELETE` request. *** ## Delete an uploaded file `DELETE /upload/{file_id}` removes the file you uploaded via `POST /upload`. This operation is synchronous and returns immediately. Pass the full `reducto://` file ID returned by `POST /upload`. ```python Python theme={null} import os import requests api_key = os.environ["REDUCTO_API_KEY"] base_url = "https://platform.reducto.ai" response = requests.delete( f"{base_url}/upload/{file_id}", headers={"Authorization": f"Bearer {api_key}"}, ) print(response.json()) # {"file_id": "reducto://abc-123-def"} ``` ```javascript Node.js theme={null} const apiKey = process.env.REDUCTO_API_KEY; const baseUrl = "https://platform.reducto.ai"; const response = await fetch(`${baseUrl}/upload/${fileId}`, { method: "DELETE", headers: { Authorization: `Bearer ${apiKey}` }, }); console.log(await response.json()); // { file_id: "reducto://abc-123-def" } ``` ```bash cURL theme={null} curl -X DELETE "https://platform.reducto.ai/upload/{file_id}" \ -H "Authorization: Bearer $REDUCTO_API_KEY" ``` ### Response | Field | Type | Description | | --------- | -------- | ----------------------------------- | | `file_id` | `string` | The normalized `reducto://` file ID | Jobs that already processed this file are not affected. Delete each job separately if needed. *** ## Relationship to data retention These deletion endpoints complement, not replace, Reducto's automatic data retention policies. For the full scope of Reducto's data handling obligations, please refer to Reducto's Data Processing Agreement (DPA). | Tier | Automatic retention | Manual deletion | | ---------------------- | ------------------------------------------------------------------------------------------ | -------------------------------------------- | | Growth and Enterprise | ZDR: all API processing artifacts typically purged within 24 hours | Job deletion via `DELETE /job` | | With `persist_results` | Results stored for the period specified in your account settings or as agreed with Reducto | Pass `include_persisted=true` to also delete | For full details on data retention and security policies, see [Data policies & compliance](/security/policies). *** ## Troubleshooting Yes. Deleting a pipeline job does not cascade to the jobs its steps created. A pipeline run creates one parent job plus one child job per step. Each child job stores its own artifacts under its own job ID. `DELETE /job/{job_id}` only deletes the artifacts belonging to the job ID you pass, so deleting the parent leaves every child's artifacts in place. Collect the child job IDs from the pipeline response payload and delete each one, then delete the parent: ```python Python theme={null} import os import requests from reducto import Reducto api_key = os.environ["REDUCTO_API_KEY"] base_url = "https://platform.reducto.ai" client = Reducto(api_key=api_key) result = client.pipeline.run(pipeline_id=pipeline_id, input=file_id) child_job_ids = [] parse = result.result.parse if isinstance(parse, list): child_job_ids.extend(item.job_id for item in parse) elif parse: child_job_ids.append(parse.job_id) extract = result.result.extract if isinstance(extract, list): child_job_ids.extend(split.result.job_id for split in extract) elif extract: child_job_ids.append(extract.job_id) for job_id in [*child_job_ids, result.job_id]: response = requests.delete( f"{base_url}/job/{job_id}", headers={"Authorization": f"Bearer {api_key}"}, ) print(job_id, response.status_code) # 202 per job ``` ```javascript Node.js theme={null} import Reducto from "reductoai"; const apiKey = process.env.REDUCTO_API_KEY; const baseUrl = "https://platform.reducto.ai"; const client = new Reducto({ apiKey }); const result = await client.pipeline.run({ pipeline_id: pipelineId, input: fileId }); const childJobIds = []; const parse = result.result.parse; if (Array.isArray(parse)) { childJobIds.push(...parse.map((item) => item.job_id)); } else if (parse) { childJobIds.push(parse.job_id); } const extract = result.result.extract; if (Array.isArray(extract)) { childJobIds.push(...extract.map((split) => split.result.job_id)); } else if (extract) { childJobIds.push(extract.job_id); } for (const jobId of [...childJobIds, result.job_id]) { const response = await fetch(`${baseUrl}/job/${jobId}`, { method: "DELETE", headers: { Authorization: `Bearer ${apiKey}` }, }); console.log(jobId, response.status); // 202 per job } ``` ```bash cURL theme={null} # Delete each child job ID from the pipeline response, then the parent job ID. for job_id in parse-456 extract-789 pipeline-abc123; do curl -X DELETE "https://platform.reducto.ai/job/$job_id" \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -w "\n$job_id: %{http_code}\n" done ``` Check the status code for every request. A job you skip keeps its artifacts until the automatic retention window ends. Deletion is asynchronous, so a `202` means cleanup was accepted, not finished. Poll `GET /job/{job_id}` for each ID until it returns `410 Gone` if you need confirmation that cleanup completed. Yes, deleting a job impacts Reducto's ability to provide support for a specific job because most indicators of job success (i.e. results) are deleted from our storage. No. Jobs created through Studio pipelines are managed separately. These endpoints only delete artifacts stored by the Reducto API. No. Deleted jobs are not recoverable through the API. Back up results separately before deleting them in Reducto's database if you need them later. The job's artifacts are being cleaned up in the background. Wait and retry your `GET /job/{job_id}` request. Once cleanup finishes, the status changes to `410 Gone`. The job was deleted from live systems (either manually via this endpoint or by automatic retention). The job record is tombstoned and is no longer retrievable through the API. Only completed or failed jobs can be deleted. If the job is still processing, wait for it to finish or cancel it first via `POST /cancel/{job_id}`. The deletion marker is rolled back automatically. The job becomes retrievable again and you can retry the `DELETE` request. # Direct Webhooks Source: https://docs.reducto.ai/workflows/direct-webhooks Simple HTTP POST webhooks for prototyping and basic integrations For production applications, use [Svix webhooks](/workflows/svix-webhooks) instead. Svix provides cryptographic signing, advanced retries, and a debugging dashboard. Direct webhooks are best for prototyping or simple internal integrations. Direct webhooks send HTTP POST requests directly to your endpoint when jobs complete. Reducto retries failed deliveries up to 3 times with exponential backoff. ## Testing with webhook.site For quick testing, use [webhook.site](https://webhook.site) to get a temporary endpoint URL. It shows you the exact payload Reducto sends. ## Submitting jobs Include your webhook URL in the async configuration: ```python Python theme={null} from reducto import Reducto client = Reducto() job = client.parse.run_job( input="https://example.com/document.pdf", async_={ "webhook": { "mode": "direct", "url": "https://your-app.com/webhook" }, "metadata": { "user_id": "123", "document_type": "invoice" } } ) print(f"Job ID: {job.job_id}") ``` ```typescript TypeScript theme={null} import Reducto from "reductoai"; const client = new Reducto(); const job = await client.parse.runJob({ input: "https://example.com/document.pdf", async: { webhook: { mode: "direct", url: "https://your-app.com/webhook" }, metadata: { userId: "123", documentType: "invoice" } } }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse_async \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "https://example.com/document.pdf", "async": { "webhook": {"mode": "direct", "url": "https://your-app.com/webhook"}, "metadata": {"user_id": "123"} } }' ``` Works with all async endpoints: `/parse_async`, `/extract_async`, `/split_async`, `/pipeline_async`. ## Webhook payload When the job completes, your endpoint receives: ```json theme={null} { "status": "Completed", "job_id": "204a39e4-dd10-4c83-a978-0cee4af8cde2", "metadata": { "user_id": "123", "document_type": "invoice" } } ``` The `status` is either `Completed` or `Failed`. Use `job_id` to retrieve results with `client.job.get()`. ## Handling webhooks ```python Python theme={null} from flask import Flask, request, jsonify from reducto import Reducto app = Flask(__name__) client = Reducto() @app.route('/webhook', methods=['POST']) def handle_webhook(): payload = request.json if payload['status'] == "Completed": job = client.job.get(payload['job_id']) # Process job.result print(f"Processed job: {payload['job_id']}") return jsonify({"received": True}), 200 ``` ```typescript TypeScript theme={null} import express from 'express'; import Reducto from "reductoai"; const app = express(); const client = new Reducto(); app.use(express.json()); app.post('/webhook', async (req, res) => { const { job_id, status } = req.body; if (status === "Completed") { const job = await client.job.retrieve(job_id); // Process job.result console.log(`Processed job: ${job_id}`); } res.status(200).json({ received: true }); }); ``` ## Validating requests Since direct webhooks lack cryptographic signing, validate requests using a secret token in metadata: ```python Python theme={null} import os from flask import Flask, request, jsonify, abort from reducto import Reducto app = Flask(__name__) client = Reducto() WEBHOOK_SECRET = os.environ["WEBHOOK_SECRET"] # When submitting jobs, include the secret def submit_job(): return client.parse.run_job( input="https://example.com/document.pdf", async_={ "webhook": {"mode": "direct", "url": "https://your-app.com/webhook"}, "metadata": {"secret": WEBHOOK_SECRET, "user_id": "123"} } ) # When handling webhooks, verify the secret @app.route('/webhook', methods=['POST']) def handle_webhook(): payload = request.json # Validate secret if payload.get('metadata', {}).get('secret') != WEBHOOK_SECRET: abort(401) if payload['status'] == "Completed": job = client.job.get(payload['job_id']) # Process job.result return jsonify({"received": True}), 200 ``` ```typescript TypeScript theme={null} import express from 'express'; import Reducto from "reductoai"; const app = express(); const client = new Reducto(); const WEBHOOK_SECRET = process.env.WEBHOOK_SECRET!; app.use(express.json()); // When submitting jobs, include the secret async function submitJob() { return client.parse.runJob({ input: "document.pdf", async: { webhook: { mode: "direct", url: "https://your-app.com/webhook" }, metadata: { secret: WEBHOOK_SECRET, userId: "123" } } }); } // When handling webhooks, verify the secret app.post('/webhook', async (req, res) => { // Validate secret if (req.body.metadata?.secret !== WEBHOOK_SECRET) { return res.status(401).json({ error: "Unauthorized" }); } if (req.body.status === "Completed") { const job = await client.job.retrieve(req.body.job_id); // Process job.result } res.status(200).json({ received: true }); }); ``` ## Troubleshooting 1. Verify your endpoint URL is publicly accessible (not localhost) 2. Ensure your endpoint returns 2xx status codes 3. Check your server logs for incoming requests Job IDs expire after 12 hours. Retrieve results promptly after receiving the webhook. Reducto retries failed deliveries. Make your handler idempotent by tracking processed job IDs. ## Best practices 1. **Use HTTPS** for your webhook endpoint 2. **Validate requests** using the token-in-metadata pattern 3. **Return quickly**: Return 2xx immediately, process results asynchronously 4. **Be idempotent**: Handle duplicate deliveries gracefully 5. **Log everything**: Direct webhooks have no dashboard, so log for debugging For production applications with reliability requirements, use [Svix webhooks](/workflows/svix-webhooks). # Multi-document Pipelines Source: https://docs.reducto.ai/workflows/multi-document-pipelines Process related documents together with shared context for extraction Multi-document pipelines let you pass several documents to one pipeline call. Reducto parses each document in parallel, combines the parsed content, then runs a single extraction across all of them. This is useful when related information spans multiple files, such as a contract split across several PDFs or supporting documents that reference each other. ## How it works Instead of passing a single document URL, you pass a list: ```python Python theme={null} result = client.pipeline.run( input=[upload1.file_id, upload2.file_id, upload3.file_id], pipeline_id="your_pipeline" ) ``` ```typescript TypeScript theme={null} const result = await client.pipeline.run({ input: [upload1.file_id, upload2.file_id, upload3.file_id], pipeline_id: "your_pipeline" }); ``` ```bash cURL theme={null} curl -X POST "https://platform.reducto.ai/pipeline" \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": [ "reducto://file1.pdf", "reducto://file2.pdf", "reducto://file3.pdf" ], "pipeline_id": "your_pipeline" }' ``` ```go Go theme={null} // Go SDK does not yet support pipelines; use HTTP directly payload := map[string]interface{}{ "input": []string{"reducto://file1.pdf", "reducto://file2.pdf", "reducto://file3.pdf"}, "pipeline_id": "your_pipeline", } body, _ := json.Marshal(payload) req, _ := http.NewRequest("POST", "https://platform.reducto.ai/pipeline", bytes.NewReader(body)) req.Header.Set("Authorization", "Bearer "+os.Getenv("REDUCTO_API_KEY")) req.Header.Set("Content-Type", "application/json") resp, _ := http.DefaultClient.Do(req) defer resp.Body.Close() ``` Reducto then: 1. Parses all documents in parallel 2. Combines the parsed content into a single context 3. Runs extraction once across the combined content 4. Returns parse results as an array (one per document) and extract as a single result ## Understanding the response The key difference from single-document pipelines is that `result.parse` becomes an array: ```json theme={null} { "job_id": "pipeline-abc123", "usage": {"num_pages": 3, "credits": 8.0}, "result": { "parse": [ {"job_id": "parse-001", "result": {...}, "usage": {"num_pages": 1}}, {"job_id": "parse-002", "result": {...}, "usage": {"num_pages": 1}}, {"job_id": "parse-003", "result": {...}, "usage": {"num_pages": 1}} ], "extract": { "job_id": "extract-combined", "result": { "fieldName": {"value": "...", "citations": [...]} } } } } ``` The parse array maintains the same order as your input documents. The extract result is a single object representing one extraction across all combined content. ## When to use multi-document pipelines Multi-document pipelines work best when your documents are related and the extraction benefits from seeing them together: | Good use cases | Why it works | | ----------------------------------------------- | ----------------------------------------------------------------------- | | Contract with exhibits split into separate PDFs | Extract can reference terms from main contract when processing exhibits | | Multi-page report scanned as individual files | Reassemble logical document from physical pages | | Application with supporting documents | Extract can cross-reference between application form and attachments | Multi-document pipelines are not the same as batch processing. If you have independent documents that don't relate to each other (like 100 different customer invoices), use [batch processing](/workflows/batch-processing) instead. Batch processing runs your pipeline independently on each document. ## Schema design matters Because extraction runs once across combined content, your schema determines what you get back. Consider three invoices with amounts $1,000, $2,000, and \$3,000: **Singular field schema:** ```json theme={null} {"total_amount": "number"} ``` Result: `{"total_amount": 1000}` — picks one value, not a sum **Aggregate field schema:** ```json theme={null} {"total_spend": "The sum of all invoice amounts across all documents"} ``` Result: `{"total_spend": 6000}` — LLM computes the aggregate **Array field schema:** ```json theme={null} {"invoices": [{"invoice_number": "string", "amount": "number"}]} ``` Result: `{"invoices": [{"invoice_number": "A001", "amount": 1000}, ...]}` — extracts from each The LLM sees all documents together, so it can answer questions that span them. But you need to design your schema to ask the right questions. ## Requirements and limitations **Requirements:** * Pipeline must include an Extract step * At least one document required * All documents must be accessible URLs or uploaded files **Limitations:** * Split is not supported with multi-document pipelines * Only one Extract step is supported * Edit pipelines don't support multi-document input ## Credits Multi-document pipelines bill for: * Parse credits for each document (based on page count) * Extract credits once for the combined extraction If you need to run the same extraction independently on many documents, batch processing is more appropriate and gives you separate results per document. *** ## Related Understanding pipeline patterns and response structures. Process many independent documents in parallel. Extract arrays of items from documents. Full API documentation for pipeline calls. # Pipeline Basics Source: https://docs.reducto.ai/workflows/pipeline-basics Compose Reducto capabilities into reusable, single-call workflows A pipeline composes multiple Reducto steps into a single workflow. Design it visually in [Studio](/studio-quickstart), deploy it to get a `pipeline_id`, then call it from your code with one API request. The pipeline orchestrates Classify, Parse, Extract, Split, and Edit steps behind the scenes, giving you a complete document workflow in a single call. Deploy Pipeline Modal ## Why pipelines? Without pipelines, building a multi-step document workflow means writing separate API calls for each step. You call Parse, wait for the result, feed that into Extract, handle errors at each stage, and manage all the configuration in your application code. This works, but it couples your code tightly to Reducto's API structure and makes configuration changes require code deployments. Pipelines solve this by moving the workflow definition out of your code and into Studio. You configure the steps visually, test with real documents, and deploy. Your code then reduces to a single call: ```python theme={null} result = client.pipeline.run( input=upload, pipeline_id="k9798h9mwt0wmq5qz5e45qxbfx7yj4bq" ) ``` When you need to adjust extraction logic or add a processing step, you update the pipeline in Studio and redeploy. Your application code stays unchanged. ## Creating a pipeline in Studio Build your pipeline in [Studio](/studio-quickstart) by adding steps and configuring each one. The Studio guides cover each step type in detail: * [Parse](/studio-parse) for document conversion * [Extract](/studio-extract) for structured data extraction * [Split](/studio-split) for document sectioning * [Edit](/studio-edit) for form filling Once you're satisfied with the results, click **Deploy** in the top right. Select **Pipeline** as the deployment type and optionally provide a version name to track your changes. Deploy Pipeline 1 Studio generates a `pipeline_id` that you can copy directly into your code. This ID points to your exact configuration, so API calls always match what you tested in Studio. Changes made in Studio don't affect production until you deploy. This lets you iterate and test without impacting live systems. ## Updating a pipeline When you need to modify a deployed pipeline, make your changes in Studio and test with sample documents. Then click **Deploy**, select **Pipeline**, update the version name, and click **Redeploy**. The update takes effect immediately, and all API calls using that `pipeline_id` will use the new configuration. The **Activity** option shows all previous versions of your pipeline, letting you track what changed and when: Pipeline Activity Logs The **Execution** **Logs** tab shows logs for when an API call hits your pipeline ID. For more details on pipeline management, see [Deploy to Production](/studio-deploy-pipeline). ## Pipeline types Studio determines the pipeline type based on which steps you add: | Type | Steps | Use case | | --------------------------- | -------------------------- | -------------------------------------------- | | **Parse** | Parse only | Convert documents to markdown, chunk for RAG | | **Parse → Extract** | Parse + Extract | Pull specific fields as JSON | | **Parse → Split → Extract** | Parse + Split + Extract(s) | Different schemas per document section | | **Edit** | Edit only | Fill forms, modify documents | ## Basic usage ```python Python theme={null} from pathlib import Path from reducto import Reducto client = Reducto() # Upload and run pipeline in one flow upload = client.upload(file=Path("document.pdf")) result = client.pipeline.run( input=upload.file_id, pipeline_id="your_pipeline_id" ) # Access results based on pipeline type if result.result.extract: # For Parse→Split→Extract pipelines, extract is a list if isinstance(result.result.extract, list): for section in result.result.extract: print(f"{section.split_name}: {section.result}") else: # For Parse→Extract pipelines, extract is an object print(result.result.extract.result) elif result.result.parse: for chunk in result.result.parse.result.chunks: print(chunk.content) ``` ```typescript TypeScript theme={null} import Reducto from "reductoai"; import fs from "fs"; const client = new Reducto(); const upload = await client.upload({ file: fs.createReadStream("document.pdf") }); const result = await client.pipeline.run({ input: upload.file_id, pipeline_id: "your_pipeline_id" }); if (result.result.extract) { // For Parse→Split→Extract pipelines, extract is an array if (Array.isArray(result.result.extract)) { result.result.extract.forEach(section => { console.log(`${section.split_name}: ${JSON.stringify(section.result)}`); }); } else { // For Parse→Extract pipelines, extract is an object console.log(result.result.extract.result); } } else if (result.result.parse) { result.result.parse.result.chunks.forEach(chunk => { console.log(chunk.content); }); } ``` ```bash cURL theme={null} curl -X POST "https://platform.reducto.ai/pipeline" \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "reducto://your-uploaded-file.pdf", "pipeline_id": "your_pipeline_id" }' ``` ```go Go theme={null} package main import ( "bytes" "encoding/json" "fmt" "io" "net/http" "os" ) func main() { // Go SDK does not yet support pipelines; use HTTP directly payload := map[string]string{ "input": "reducto://your-uploaded-file.pdf", "pipeline_id": "your_pipeline_id", } body, _ := json.Marshal(payload) req, _ := http.NewRequest("POST", "https://platform.reducto.ai/pipeline", bytes.NewReader(body)) req.Header.Set("Authorization", "Bearer "+os.Getenv("REDUCTO_API_KEY")) req.Header.Set("Content-Type", "application/json") resp, err := http.DefaultClient.Do(req) if err != nil { fmt.Printf("Error: %v\n", err) return } defer resp.Body.Close() result, _ := io.ReadAll(resp.Body) fmt.Println(string(result)) } ``` The Go SDK does not yet support the Pipeline endpoint. Use the HTTP example above or the cURL snippet as a reference for Go implementations. ## Response structure Every pipeline returns a `PipelineResponse` with the same shape. Which fields are populated depends on the pipeline type you configured in Studio. ```json theme={null} { "job_id": "pipeline-abc123", "usage": {"num_pages": 3, "credits": 6.0}, "result": { "parse": {...}, "extract": {...}, "split": {...}, "edit": {...} } } ``` The `parse` field is present for Parse, Parse→Extract, and Parse→Split→Extract pipelines. The `extract` field appears as an object for Parse→Extract pipelines, or as an array for Parse→Split→Extract pipelines where each entry corresponds to a section. The `split` field only appears when Split is configured. The `edit` field only appears for Edit pipelines. ```json theme={null} { "job_id": "abc123", "usage": {"num_pages": 3, "credits": 4.0}, "result": { "parse": { "job_id": "parse-456", "result": { "chunks": [ {"content": "# Title\n\nParagraph text...", "blocks": [...]} ] }, "usage": {"num_pages": 3, "credits": 4.0} }, "extract": null, "split": null } } ``` ```json theme={null} { "job_id": "abc123", "usage": {"num_pages": 3, "credits": 6.0}, "result": { "parse": {"job_id": "parse-456", "result": {...}, "usage": {...}}, "extract": { "job_id": "extract-789", "result": { "invoiceNumber": {"value": "INV-001", "citations": [...]}, "totalAmount": {"value": "$1,500.00", "citations": [...]} }, "usage": {"credits": 2.0} }, "split": null } } ``` When Split is involved, `extract` becomes an array with one entry per section: ```json theme={null} { "job_id": "abc123", "usage": {"num_pages": 10, "credits": 12.0}, "result": { "parse": {...}, "split": { "result": { "splits": [ {"name": "Summary", "pages": [1, 2]}, {"name": "Details", "pages": [3, 4, 5]} ] } }, "extract": [ { "split_name": "Summary", "page_range": [1, 2], "result": {"totalValue": {"value": "$274,222", "citations": [...]}} }, { "split_name": "Details", "page_range": [3, 4, 5], "result": {"holdings": [...]} } ] } } ``` Edit pipelines return a URL to the modified document. Note that for edit pipelines, the `input` parameter contains the edit instructions rather than a document URL. The document to edit is configured in Studio as part of the pipeline. ```json theme={null} { "job_id": "abc123", "usage": {"num_pages": 0, "credits": null}, "result": { "parse": null, "extract": null, "split": null, "edit": { "job_id": "edit-999", "result": {"file_url": "https://storage.reducto.ai/edited-doc.pdf"} } } } ``` ## Next steps Build and deploy your first pipeline in Studio. Process multiple documents in a single call. Manage pipeline versions and view execution logs. Run pipelines asynchronously with webhooks. # Svix Webhooks Source: https://docs.reducto.ai/workflows/svix-webhooks Enterprise-grade webhook delivery with signing, retries, and dashboard Svix webhooks provide cryptographic request signing, automatic retries with exponential backoff, and a delivery dashboard for debugging. Use Svix for production applications. ## Accessing the Svix dashboard You can access your Svix webhook dashboard in two ways: In [Reducto Studio](https://studio.reducto.ai), go to **Account → Webhooks**. This opens your Svix dashboard directly. Call the `/configure_webhook` endpoint to get a dashboard URL: ```python Python theme={null} import requests response = requests.post( "https://platform.reducto.ai/configure_webhook", headers={"Authorization": "Bearer YOUR_API_KEY"}, ) print(response.text) # Returns Svix dashboard URL ``` ```typescript TypeScript theme={null} const response = await fetch("https://platform.reducto.ai/configure_webhook", { method: "POST", headers: { "Authorization": "Bearer YOUR_API_KEY" } }); console.log(await response.text()); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/configure_webhook \ -H "Authorization: Bearer $REDUCTO_API_KEY" ``` ## Adding an endpoint In the Svix dashboard, click **+ Add Endpoint** to configure where webhooks are delivered: Add endpoint dialog in Svix Enter your endpoint URL (must be HTTPS for production). You can use [webhook.site](https://webhook.site) for testing. Once added, you'll see your endpoint in the list: Svix endpoints list ## Submitting jobs with webhooks When submitting async jobs, include the webhook configuration: ```python Python theme={null} from reducto import Reducto client = Reducto() job = client.parse.run_job( input="https://example.com/document.pdf", async_={ "webhook": { "mode": "svix", "channels": [] # Optional: route to specific endpoints }, "metadata": { "user_id": "123", "document_type": "invoice" } } ) print(f"Job ID: {job.job_id}") ``` ```typescript TypeScript theme={null} import Reducto from "reductoai"; const client = new Reducto(); const job = await client.parse.runJob({ input: "https://example.com/document.pdf", async: { webhook: { mode: "svix", channels: [] }, metadata: { userId: "123", documentType: "invoice" } } }); ``` ```bash cURL theme={null} curl -X POST https://platform.reducto.ai/parse_async \ -H "Authorization: Bearer $REDUCTO_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "https://example.com/document.pdf", "async": { "webhook": {"mode": "svix", "channels": []}, "metadata": {"user_id": "123"} } }' ``` Works with all async endpoints: `/parse_async`, `/extract_async`, `/split_async`, `/pipeline_async`. ## Viewing deliveries When a job completes, Reducto sends an `async.update` event to Svix, which delivers it to your endpoint. You can monitor deliveries in the dashboard: Successful webhook delivery in Svix The dashboard shows: * **Delivery stats**: Success/failure rates * **Message attempts**: Each delivery with timestamp and status * **Signing secret**: For verifying webhooks in your handler ## Webhook payload Your endpoint receives: ```json theme={null} { "status": "Completed", "job_id": "a1b9090e-c9ae-420b-9726-f658afbbe338", "metadata": { "user_id": "123", "document_type": "invoice" } } ``` The `status` is either `Completed` or `Failed`. Use `job_id` to retrieve results. ## Handling webhooks with signature verification Always verify webhook signatures in production. Get your signing secret from the Svix dashboard (starts with `whsec_`): ```python Python theme={null} from flask import Flask, request, jsonify from reducto import Reducto from svix.webhooks import Webhook, WebhookVerificationError app = Flask(__name__) client = Reducto() WEBHOOK_SECRET = "whsec_your_secret" # From Svix dashboard @app.route('/webhook', methods=['POST']) def handle_webhook(): # Verify signature wh = Webhook(WEBHOOK_SECRET) try: payload = wh.verify(request.data, { 'svix-id': request.headers.get('svix-id'), 'svix-timestamp': request.headers.get('svix-timestamp'), 'svix-signature': request.headers.get('svix-signature') }) except WebhookVerificationError: return jsonify({"error": "Invalid signature"}), 401 # Process webhook if payload['status'] == "Completed": job = client.job.get(payload['job_id']) # Use job.result return jsonify({"received": True}), 200 ``` ```typescript TypeScript theme={null} import express from 'express'; import Reducto from "reductoai"; import { Webhook } from "svix"; const app = express(); const client = new Reducto(); const WEBHOOK_SECRET = "whsec_your_secret"; // From Svix dashboard app.post('/webhook', express.raw({ type: 'application/json' }), async (req, res) => { const wh = new Webhook(WEBHOOK_SECRET); let payload; try { payload = wh.verify(req.body, { 'svix-id': req.headers['svix-id'] as string, 'svix-timestamp': req.headers['svix-timestamp'] as string, 'svix-signature': req.headers['svix-signature'] as string }); } catch { return res.status(401).json({ error: "Invalid signature" }); } if (payload.status === "Completed") { const job = await client.job.retrieve(payload.job_id); // Use job.result } res.status(200).json({ received: true }); }); ``` ## Channel routing Use channels to route webhooks to different endpoints (e.g., production vs development): ```python theme={null} # Production endpoint receives this client.parse.run_job( input="https://example.com/document.pdf", async_={"webhook": {"mode": "svix", "channels": ["production"]}} ) # Development endpoint receives this client.parse.run_job( input="https://example.com/test.pdf", async_={"webhook": {"mode": "svix", "channels": ["development"]}} ) ``` Configure which channels each endpoint listens to in the Svix dashboard under endpoint settings. ## Troubleshooting 1. Check the Svix dashboard "Message Attempts" for delivery status 2. Verify your endpoint URL is publicly accessible 3. Ensure your endpoint returns 2xx status codes within 15 seconds 1. Verify you're using the correct secret from the Svix dashboard 2. Pass the raw request body (not parsed JSON) to verification 3. Check all three headers are present: `svix-id`, `svix-timestamp`, `svix-signature` Return 2xx immediately and process results asynchronously. Svix expects responses within 15 seconds. In the Svix dashboard, click on a failed message attempt and use the "Resend" button to replay it. ## Best practices 1. **Always verify signatures** in production 2. **Return quickly** (within 15 seconds), process results asynchronously 3. **Be idempotent**: Svix may retry, use `svix-id` header as idempotency key 4. **Use HTTPS** for your webhook endpoint For simpler use cases or prototyping, see [Direct Webhooks](/workflows/direct-webhooks). # Parallel Document Processing Source: https://docs.reducto.ai/cookbooks/batch-processing Process hundreds of documents concurrently using AsyncReducto with progress tracking and error handling Process a real invoice dataset from Hugging Face using `AsyncReducto` with progress tracking and error handling. *** ## Sample Dataset We'll use the [Northwind Invoices dataset](https://huggingface.co/datasets/AyoubChLin/northwind_invocies) from Hugging Face, which contains 831 PDF invoices. Each invoice includes customer information, line items, and totals. Sample Northwind invoice showing customer details, line items, and totals *** ## Create API Key Go to [studio.reducto.ai](https://studio.reducto.ai) and sign in. From the home page, click **API Keys** in the left sidebar. Studio home page with API Keys in sidebar The API Keys page shows your existing keys. Click **+ Create new API key** in the top right corner. API Keys page with Create button In the modal, enter a name for your key and set an expiration policy (or select "Never" for no expiration). Click **Create**. New API Key modal with name and expiration fields Copy your new API key and store it securely. You won't be able to see it again after closing this dialog. Copy API key dialog Set the key as an environment variable: ```bash theme={null} export REDUCTO_API_KEY="your-api-key-here" ``` *** ## Download the Dataset First, download the Northwind invoices dataset using the Hugging Face libraries: ```bash Python theme={null} pip install datasets ``` ```bash JavaScript theme={null} npm install @huggingface/hub ``` ```python Python theme={null} from datasets import load_dataset from pathlib import Path # Load the dataset dataset = load_dataset("AyoubChLin/northwind_invocies") # Create output directory output_dir = Path("./northwind_invoices") output_dir.mkdir(exist_ok=True) # Save PDFs to disk for i, sample in enumerate(dataset["train"]): pdf_path = output_dir / f"invoice_{i:04d}.pdf" pdf = sample["pdf"] pdf.stream.seek(0) with open(pdf_path, "wb") as f: f.write(pdf.stream.read()) print(f"Downloaded {len(dataset['train'])} invoices to {output_dir}") ``` ```javascript JavaScript theme={null} import { listFiles, downloadFile } from "@huggingface/hub"; import fs from "fs"; import path from "path"; // Load the dataset const repo = { type: "dataset", name: "AyoubChLin/northwind_invocies" }; // Create output directory const outputDir = "./northwind_invoices"; fs.mkdirSync(outputDir, { recursive: true }); // List and download PDF files let count = 0; for await (const fileInfo of listFiles({ repo })) { if (fileInfo.path.endsWith(".pdf")) { const response = await downloadFile({ repo, path: fileInfo.path }); const buffer = Buffer.from(await response.arrayBuffer()); const outputPath = path.join(outputDir, `invoice_${String(count).padStart(4, "0")}.pdf`); fs.writeFileSync(outputPath, buffer); count++; if (count % 100 === 0) console.log(`Downloaded ${count} invoices...`); } } console.log(`Downloaded ${count} invoices to ${outputDir}`); ``` ``` Downloaded 831 invoices to ./northwind_invoices ``` You now have 831 PDF invoices ready to process. *** ## Process the Batch Document processing is network-bound, not CPU-bound. While your code waits for one API response, it could be uploading and processing other documents. Python uses `AsyncReducto` with `asyncio`, while JavaScript uses `Promise.all()` with the `p-limit` package for concurrency control. ```python Python theme={null} import asyncio import json from pathlib import Path from reducto import AsyncReducto from tqdm.asyncio import tqdm client = AsyncReducto() async def process_invoices( input_dir: Path, output_dir: Path, max_concurrency: int = 50 ): """Process all invoices concurrently with progress tracking.""" output_dir.mkdir(exist_ok=True) files = list(input_dir.glob("*.pdf")) print(f"Found {len(files)} invoices to process") # Semaphore limits concurrent requests sem = asyncio.Semaphore(max_concurrency) async def process(path: Path): async with sem: try: upload = await client.upload(file=path) result = await client.parse.run(input=upload.file_id) # Save result immediately output_path = output_dir / f"{path.stem}.json" output_path.write_text(json.dumps({ "source": path.name, "pages": result.usage.num_pages, "content": [c.content for c in result.result.chunks] }, indent=2)) return {"file": path.name, "success": True, "pages": result.usage.num_pages} except Exception as e: return {"file": path.name, "success": False, "error": str(e)} # Process all files with progress bar tasks = [process(f) for f in files] results = await tqdm.gather(*tasks, desc="Processing invoices") # Summary successes = [r for r in results if r["success"]] failures = [r for r in results if not r["success"]] total_pages = sum(r["pages"] for r in successes) print(f"\nCompleted: {len(successes)} succeeded, {len(failures)} failed") print(f"Total pages processed: {total_pages}") if failures: print("\nFailed files:") for f in failures[:5]: # Show first 5 failures print(f" - {f['file']}: {f['error']}") return results # Run the batch results = asyncio.run(process_invoices( input_dir=Path("./northwind_invoices"), output_dir=Path("./parsed_invoices") )) ``` ```javascript JavaScript theme={null} import Reducto from "reductoai"; import fs from "fs"; import path from "path"; import pLimit from "p-limit"; // npm install p-limit const client = new Reducto(); async function processInvoices(inputDir, outputDir, maxConcurrency = 50) { fs.mkdirSync(outputDir, { recursive: true }); const files = fs.readdirSync(inputDir) .filter(f => f.endsWith(".pdf")) .map(f => path.join(inputDir, f)); console.log(`Found ${files.length} invoices to process`); // p-limit controls concurrency (like Python's Semaphore) const limit = pLimit(maxConcurrency); async function processFile(filePath) { try { const upload = await client.upload({ file: fs.createReadStream(filePath) }); const result = await client.parse.run({ input: upload.file_id }); // Save result immediately const outputPath = path.join(outputDir, `${path.basename(filePath, ".pdf")}.json`); fs.writeFileSync(outputPath, JSON.stringify({ source: path.basename(filePath), pages: result.usage.num_pages, content: result.result.chunks.map(c => c.content) }, null, 2)); return { file: path.basename(filePath), success: true, pages: result.usage.num_pages }; } catch (e) { return { file: path.basename(filePath), success: false, error: e.message }; } } // Process all files with concurrency limit const results = await Promise.all(files.map(f => limit(() => processFile(f)))); // Summary const successes = results.filter(r => r.success); const failures = results.filter(r => !r.success); const totalPages = successes.reduce((sum, r) => sum + r.pages, 0); console.log(`\nCompleted: ${successes.length} succeeded, ${failures.length} failed`); console.log(`Total pages processed: ${totalPages}`); if (failures.length > 0) { console.log("\nFailed files:"); failures.slice(0, 5).forEach(f => console.log(` - ${f.file}: ${f.error}`)); } return results; } // Run the batch const results = await processInvoices("./northwind_invoices", "./parsed_invoices"); ``` **Output:** ``` Found 831 invoices to process Processing invoices: 100%|██████████| 831/831 [03:42<00:00, 3.73it/s] Completed: 831 succeeded, 0 failed Total pages processed: 831 ``` *** ## Extract Structured Data To extract specific fields like invoice numbers, totals, and line items, use the Extract API with a schema: ```python Python theme={null} import asyncio import json from pathlib import Path from reducto import AsyncReducto from tqdm.asyncio import tqdm client = AsyncReducto() # Schema for Northwind invoices invoice_schema = { "type": "object", "properties": { "order_id": {"type": "string", "description": "Order ID at top of invoice"}, "customer_name": {"type": "string", "description": "Ship To customer name"}, "order_date": {"type": "string", "description": "Order date"}, "shipped_date": {"type": "string", "description": "Shipped date"}, "ship_address": {"type": "string", "description": "Full shipping address"}, "line_items": { "type": "array", "description": "Products ordered", "items": { "type": "object", "properties": { "product": {"type": "string"}, "quantity": {"type": "number"}, "unit_price": {"type": "number"}, "discount": {"type": "number"}, "extended_price": {"type": "number"} } } }, "subtotal": {"type": "number"}, "freight": {"type": "number"}, "total": {"type": "number"} } } async def extract_invoices( input_dir: Path, output_dir: Path, max_concurrency: int = 30 ): """Extract structured data from invoices.""" output_dir.mkdir(exist_ok=True) files = list(input_dir.glob("*.pdf"))[:100] # Process first 100 for demo print(f"Extracting from {len(files)} invoices") sem = asyncio.Semaphore(max_concurrency) async def extract(path: Path): async with sem: try: upload = await client.upload(file=path) result = await client.extract.run( input=upload.file_id, instructions={"schema": invoice_schema} ) output_path = output_dir / f"{path.stem}.json" output_path.write_text(json.dumps(result.result, indent=2)) return {"file": path.name, "success": True, "data": result.result} except Exception as e: return {"file": path.name, "success": False, "error": str(e)} tasks = [extract(f) for f in files] results = await tqdm.gather(*tasks, desc="Extracting") successes = [r for r in results if r["success"]] print(f"\nExtracted {len(successes)} invoices") # Calculate totals across all invoices grand_total = sum(r["data"].get("total", 0) or 0 for r in successes) print(f"Grand total across invoices: ${grand_total:,.2f}") return results results = asyncio.run(extract_invoices( input_dir=Path("./northwind_invoices"), output_dir=Path("./extracted_invoices") )) ``` ```javascript JavaScript theme={null} import Reducto from "reductoai"; import fs from "fs"; import path from "path"; import pLimit from "p-limit"; const client = new Reducto(); // Schema for Northwind invoices const invoiceSchema = { type: "object", properties: { order_id: { type: "string", description: "Order ID at top of invoice" }, customer_name: { type: "string", description: "Ship To customer name" }, order_date: { type: "string", description: "Order date" }, shipped_date: { type: "string", description: "Shipped date" }, ship_address: { type: "string", description: "Full shipping address" }, line_items: { type: "array", description: "Products ordered", items: { type: "object", properties: { product: { type: "string" }, quantity: { type: "number" }, unit_price: { type: "number" }, discount: { type: "number" }, extended_price: { type: "number" } } } }, subtotal: { type: "number" }, freight: { type: "number" }, total: { type: "number" } } }; async function extractInvoices(inputDir, outputDir, maxConcurrency = 30) { fs.mkdirSync(outputDir, { recursive: true }); const files = fs.readdirSync(inputDir) .filter(f => f.endsWith(".pdf")) .slice(0, 100); // Process first 100 for demo console.log(`Extracting from ${files.length} invoices`); const limit = pLimit(maxConcurrency); async function extractFile(fileName) { const filePath = path.join(inputDir, fileName); try { const upload = await client.upload({ file: fs.createReadStream(filePath) }); const result = await client.extract.run({ input: upload.file_id, instructions: { schema: invoiceSchema } }); const outputPath = path.join(outputDir, `${path.basename(fileName, ".pdf")}.json`); fs.writeFileSync(outputPath, JSON.stringify(result.result, null, 2)); return { file: fileName, success: true, data: result.result[0] }; } catch (e) { return { file: fileName, success: false, error: e.message }; } } const results = await Promise.all(files.map(f => limit(() => extractFile(f)))); const successes = results.filter(r => r.success); console.log(`\nExtracted ${successes.length} invoices`); // Calculate totals across all invoices const grandTotal = successes.reduce((sum, r) => sum + (r.data?.total || 0), 0); console.log(`Grand total across invoices: $${grandTotal.toFixed(2)}`); return results; } const results = await extractInvoices("./northwind_invoices", "./extracted_invoices"); ``` **Output:** ``` Extracting from 100 invoices Extracting: 100%|██████████| 100/100 [01:45<00:00, 1.05s/it] Extracted 100 invoices Grand total across invoices: $128,347.52 ``` *** ## Cost Optimization with Job Chaining Parse once, extract multiple times. When you need different extractions from the same documents, reuse the parse job ID to avoid re-parsing: ```python Python theme={null} import asyncio from pathlib import Path from reducto import AsyncReducto client = AsyncReducto() async def extract_multiple_schemas(files: list[Path]): """Parse once, extract with multiple schemas.""" results = [] for path in files: # Parse once upload = await client.upload(file=path) parse_result = await client.parse.run(input=upload.file_id) job_id = parse_result.job_id # Extract headers (reuses parse) header_result = await client.extract.run( input=f"jobid://{job_id}", instructions={"schema": { "type": "object", "properties": { "invoice_number": {"type": "string"}, "customer_name": {"type": "string"}, "date": {"type": "string"} } }} ) # Extract line items (still reuses parse - no extra parse cost!) items_result = await client.extract.run( input=f"jobid://{job_id}", instructions={"schema": { "type": "object", "properties": { "line_items": { "type": "array", "items": { "type": "object", "properties": { "product": {"type": "string"}, "quantity": {"type": "number"}, "price": {"type": "number"} } } }, "total": {"type": "number"} } }} ) results.append({ "file": path.name, "header": header_result.result, "items": items_result.result }) return results ``` ```javascript JavaScript theme={null} import Reducto from "reductoai"; import fs from "fs"; const client = new Reducto(); async function extractMultipleSchemas(files) { const results = []; for (const filePath of files) { // Parse once const upload = await client.upload({ file: fs.createReadStream(filePath) }); const parseResult = await client.parse.run({ input: upload.file_id }); const jobId = parseResult.job_id; // Extract headers (reuses parse) const headerResult = await client.extract.run({ input: `jobid://${jobId}`, instructions: { schema: { type: "object", properties: { invoice_number: { type: "string" }, customer_name: { type: "string" }, date: { type: "string" } } } } }); // Extract line items (still reuses parse - no extra parse cost!) const itemsResult = await client.extract.run({ input: `jobid://${jobId}`, instructions: { schema: { type: "object", properties: { line_items: { type: "array", items: { type: "object", properties: { product: { type: "string" }, quantity: { type: "number" }, price: { type: "number" } } } }, total: { type: "number" } } } } }); results.push({ file: filePath, header: headerResult.result[0], items: itemsResult.result[0] }); } return results; } ``` The `jobid://` prefix tells Reducto to reuse an existing parse result. You only pay for parsing once, regardless of how many extractions you run. *** ## Handling Failures Add retry logic with exponential backoff to handle transient network errors: ```python Python theme={null} import asyncio import random from pathlib import Path from reducto import AsyncReducto client = AsyncReducto() async def process_with_retry(path: Path, max_retries: int = 3): """Process a document with exponential backoff on failures.""" for attempt in range(max_retries): try: upload = await client.upload(file=path) result = await client.parse.run(input=upload.file_id) return {"file": path.name, "success": True, "pages": result.usage.num_pages} except Exception as e: if attempt == max_retries - 1: return {"file": path.name, "success": False, "error": str(e)} wait_time = (2 ** attempt) + random.uniform(0, 1) await asyncio.sleep(wait_time) async def process_batch_with_retries(input_dir: Path, max_concurrency: int = 50): """Process a batch with automatic retries.""" files = list(input_dir.glob("*.pdf")) sem = asyncio.Semaphore(max_concurrency) async def process(path): async with sem: return await process_with_retry(path) tasks = [process(f) for f in files] results = await asyncio.gather(*tasks) successes = [r for r in results if r["success"]] failures = [r for r in results if not r["success"]] print(f"Completed: {len(successes)} succeeded, {len(failures)} failed") if failures: print("Failed files:") for f in failures: print(f" - {f['file']}: {f['error']}") return results ``` ```javascript JavaScript theme={null} import Reducto from "reductoai"; import fs from "fs"; import pLimit from "p-limit"; const client = new Reducto(); async function processWithRetry(filePath, maxRetries = 3) { for (let attempt = 0; attempt < maxRetries; attempt++) { try { const upload = await client.upload({ file: fs.createReadStream(filePath) }); const result = await client.parse.run({ input: upload.file_id }); return { file: filePath, success: true, pages: result.usage.num_pages }; } catch (e) { if (attempt === maxRetries - 1) { return { file: filePath, success: false, error: e.message }; } const waitTime = Math.pow(2, attempt) * 1000 + Math.random() * 1000; await new Promise(r => setTimeout(r, waitTime)); } } } async function processBatchWithRetries(inputDir, maxConcurrency = 50) { const files = fs.readdirSync(inputDir) .filter(f => f.endsWith(".pdf")) .map(f => `${inputDir}/${f}`); const limit = pLimit(maxConcurrency); const results = await Promise.all(files.map(f => limit(() => processWithRetry(f)))); const successes = results.filter(r => r.success); const failures = results.filter(r => !r.success); console.log(`Completed: ${successes.length} succeeded, ${failures.length} failed`); if (failures.length > 0) { console.log("Failed files:"); failures.forEach(f => console.log(` - ${f.file}: ${f.error}`)); } return results; } ``` *** ## Monitoring Job Status When using webhooks or async jobs, you can poll for job status: ```python Python theme={null} import asyncio from reducto import AsyncReducto client = AsyncReducto() async def wait_for_job(job_id: str, timeout: int = 300): """Poll until job completes or times out.""" for _ in range(timeout): status = await client.job.get(job_id) if status.status == "Completed": return {"success": True, "status": status} elif status.status == "Failed": return {"success": False, "status": status} await asyncio.sleep(1) return {"success": False, "error": "Timeout"} # Example: Submit job and wait async def submit_and_wait(path): upload = await client.upload(file=path) job = await client.parse.run_job(input=upload.file_id) print(f"Submitted job {job.job_id}, waiting...") result = await wait_for_job(job.job_id) if result["success"]: print(f"Job completed!") else: print(f"Job failed or timed out") return result ``` ```javascript JavaScript theme={null} import Reducto from "reductoai"; import fs from "fs"; const client = new Reducto(); async function waitForJob(jobId, timeout = 300) { for (let i = 0; i < timeout; i++) { const status = await client.job.retrieve(jobId); if (status.status === "Completed") { return { success: true, status }; } else if (status.status === "Failed") { return { success: false, status }; } await new Promise(r => setTimeout(r, 1000)); } return { success: false, error: "Timeout" }; } // Example: Submit job and wait async function submitAndWait(filePath) { const upload = await client.upload({ file: fs.createReadStream(filePath) }); const job = await client.parse.runJob({ input: upload.file_id }); console.log(`Submitted job ${job.job_id}, waiting...`); const result = await waitForJob(job.job_id); if (result.success) { console.log("Job completed!"); } else { console.log("Job failed or timed out"); } return result; } ``` For production workloads, prefer webhooks over polling. Webhooks are more efficient and don't require keeping connections open. *** ## Why Concurrency Control? The semaphore (`asyncio.Semaphore` in Python, `p-limit` in JavaScript) limits how many requests run simultaneously. Without it, submitting 831 files would create 831 concurrent connections, overwhelming both your system and the API. The concurrency limiter acts as a queue, letting only `max_concurrency` requests proceed at once. | File Size | Recommended Concurrency | | ------------------- | ----------------------- | | Small (\< 5 pages) | 50-100 | | Medium (5-50 pages) | 20-50 | | Large (50+ pages) | 10-20 | Start conservative and increase if stable. Larger files consume more memory per request, so lower concurrency prevents memory issues. *** ## Fire-and-Forget with Webhooks For very large batches where you don't want to wait for results, use webhooks. Submit all jobs immediately and receive results as they complete via HTTP callbacks. ```python Python theme={null} import asyncio from pathlib import Path from reducto import AsyncReducto client = AsyncReducto() async def submit_with_webhooks( input_dir: Path, webhook_url: str, max_concurrency: int = 100 ): """Submit documents for processing with webhook notifications.""" files = list(input_dir.glob("*.pdf")) sem = asyncio.Semaphore(max_concurrency) async def submit(path: Path): async with sem: upload = await client.upload(file=path) job = await client.parse.run_job( input=upload.file_id, async_={ "webhook": {"mode": "direct", "url": webhook_url}, "metadata": {"filename": path.name} } ) return {"file": path.name, "job_id": job.job_id} tasks = [submit(f) for f in files] submissions = await asyncio.gather(*tasks) print(f"Submitted {len(submissions)} jobs - results will arrive via webhook") return submissions # Submit batch asyncio.run(submit_with_webhooks( input_dir=Path("./northwind_invoices"), webhook_url="https://your-app.com/webhook/reducto" )) ``` ```javascript JavaScript theme={null} import Reducto from "reductoai"; import fs from "fs"; import path from "path"; import pLimit from "p-limit"; const client = new Reducto(); async function submitWithWebhooks(inputDir, webhookUrl, maxConcurrency = 100) { const files = fs.readdirSync(inputDir) .filter(f => f.endsWith(".pdf")) .map(f => path.join(inputDir, f)); const limit = pLimit(maxConcurrency); async function submitFile(filePath) { const upload = await client.upload({ file: fs.createReadStream(filePath) }); const job = await client.parse.runJob({ input: upload.file_id, async_: { webhook: { mode: "direct", url: webhookUrl }, metadata: { filename: path.basename(filePath) } } }); return { file: path.basename(filePath), job_id: job.job_id }; } const submissions = await Promise.all(files.map(f => limit(() => submitFile(f)))); console.log(`Submitted ${submissions.length} jobs - results will arrive via webhook`); return submissions; } // Submit batch await submitWithWebhooks("./northwind_invoices", "https://your-app.com/webhook/reducto"); ``` Your webhook receives a payload when each job completes: ```json theme={null} { "status": "Completed", "job_id": "abc123", "metadata": {"filename": "invoice_0001.pdf"} } ``` Fetch the result in your handler with `client.job.get(job_id)`. *** ## Sync Alternative (Python) If you can't use async in Python, use threading with the synchronous `Reducto` client. This section is Python-specific. The JavaScript SDK is async by default—all methods return Promises and work naturally with `async/await` and `Promise.all()`. No threading is needed in JavaScript. ```python theme={null} from concurrent.futures import ThreadPoolExecutor, as_completed from pathlib import Path from reducto import Reducto client = Reducto() def process_sync(input_dir: Path, max_workers: int = 10): files = list(input_dir.glob("*.pdf")) def process(path: Path): upload = client.upload(file=path) result = client.parse.run(input=upload.file_id) return {"file": path.name, "pages": result.usage.num_pages} results = [] with ThreadPoolExecutor(max_workers=max_workers) as executor: futures = {executor.submit(process, f): f for f in files} for future in as_completed(futures): try: results.append(future.result()) except Exception as e: results.append({"file": futures[future].name, "error": str(e)}) return results ``` Threading works but is less efficient than async for I/O-bound work. Use lower concurrency (10-20 workers) to avoid thread overhead. *** ## Next Steps Deep dive into async jobs and job lifecycle Production webhook setup with Svix Understand per-account concurrency limits for batch sizing Monitor and manage async jobs # Layout & Table Extraction to Analytics Source: https://docs.reducto.ai/cookbooks/financial-analysis Extract and analyze financial data from 10-K annual reports using Parse, Extract, and table extraction Extract structured financial data from SEC 10-K filings for investment analysis, competitive intelligence, or compliance automation. *** ## Sample Document