LLM & service configuration options

OCR service configuration

For OCR functionality, Reducto uses AWS Textract with automatic region selection and load balancing across multiple regions.

Textract region configuration

Variable	Description	Required	Default
`TEXTRACT_REGIONS`	Comma-separated list of AWS regions and their quotas for Textract operations	No	`us-east-2:100,us-east-1:100,us-west-2:100,ap-south-1:5,eu-west-1:5`

Format

The TEXTRACT_REGIONS environment variable accepts a comma-separated list of region:quota pairs:

TEXTRACT_REGIONS=us-gov-west-1:10,us-gov-east-1:5,eu-central-1:15

Format Rules:

With quota: region:quota (e.g., us-east-1:100)
Without quota: region (defaults to quota of 1)
Mixed: us-east-1:50,us-west-2,eu-west-1:25

Use cases

This configuration is particularly useful for:

Government Cloud deployments requiring specific regions (e.g., us-gov-west-1, us-gov-east-1)
Data residency requirements restricting processing to certain geographic regions
Custom quota management when you have different service limits across regions
Network restrictions where only certain regions are accessible from your deployment

Examples

Government Cloud:

TEXTRACT_REGIONS=us-gov-west-1:10,us-gov-east-1:10

Europe-only processing:

TEXTRACT_REGIONS=eu-west-1:20,eu-central-1:15,eu-north-1:10

Single region:

TEXTRACT_REGIONS=us-west-2:50

When this variable is set, Reducto will use weighted random selection based on the quotas to distribute Textract requests across the specified regions, providing both load balancing and compliance with regional requirements. With automatic quota handling.

Azure Vision OCR

For OCR functionality using Azure Computer Vision service.

Single endpoint

Variable	Description	Required	Default
`AZURE_VISION_ENDPOINT`	Azure Computer Vision endpoint URL	Yes	None
`AZURE_VISION_KEY`	Azure Computer Vision API key	Yes	None

Multiple endpoints (failover and load balancing)

For setups with multiple endpoints, use AZURE_VISION_ARRAY. If any endpoint returns a transient error (rate limits, connection failures, server errors), Reducto automatically tries the remaining endpoints before failing. Non-retryable errors (e.g. 400, 401, 403, 404) on the primary surface immediately — they will reproduce on every endpoint, so cross-region retries would only add latency.

Variable	Description	Required	Default
`AZURE_VISION_ARRAY`	JSON array of Azure Vision endpoint objects	Yes	None
`AZURE_VISION_ARRAY_STRATEGY`	`load_balance`, `priority`, or `primary_with_fanout`	No	`load_balance`

Strategies

Strategy	Behavior
`load_balance` (default)	Pick a random endpoint per request; on transient error, retry that endpoint up to 3 times with exponential backoff (60-second per-endpoint wall-clock cap), then fail over to the next endpoint and repeat. Use when all endpoints have similar capacity and you want to spread steady-state load.
`priority`	Try endpoints in the order listed; on transient error, retry that endpoint up to 3 times with exponential backoff (60-second per-endpoint wall-clock cap), then fail over to the next and repeat. Use when you have a preferred region and want others used only as fallbacks.
`primary_with_fanout`	Try the primary (index 0) once; on a `408`/`429`/`5xx`/network error, immediately probe each secondary in order with a single attempt each. If every endpoint returns a transient error, return to the primary and retry with exponential backoff (3 tries, 60-second cap). Use when endpoints have unequal capacity and you want to keep secondaries from absorbing sustained overflow — the higher-capacity primary recovers first under backoff while secondaries see only one probe per overloaded request.

Retry classification (all strategies):

HTTP status	Retryable?	Behavior on primary
`408` Request Timeout	Yes	retry / fail over
`429` Too Many Requests	Yes	retry / fail over
`5xx` (500–599)	Yes	retry / fail over
Network / connection errors	Yes	retry / fail over
`4xx` other than `408`/`429`	No	surface immediately, no failover

Under primary_with_fanout only, a non-retryable error on a secondary (e.g. a misconfigured key on one secondary) is logged and skipped so a single bad secondary does not poison a request the primary could otherwise serve once it recovers.

# Load balance across regions (default — equal-capacity endpoints)
AZURE_VISION_ARRAY='[
  {"endpoint": "https://my-vision-uscentral.cognitiveservices.azure.com/", "key": "<key-1>"},
  {"endpoint": "https://my-vision-useast.cognitiveservices.azure.com/", "key": "<key-2>"},
  {"endpoint": "https://my-vision-uswest.cognitiveservices.azure.com/", "key": "<key-3>"}
]'

# Priority order: primary first, secondaries only on transient failure
AZURE_VISION_ARRAY_STRATEGY=priority
AZURE_VISION_ARRAY='[
  {"endpoint": "https://my-vision-uscentral.cognitiveservices.azure.com/", "key": "<key-1>"},
  {"endpoint": "https://my-vision-useast.cognitiveservices.azure.com/", "key": "<key-2>"}
]'

# Primary with fanout: high-capacity primary, low-quota secondaries only absorb spillover
AZURE_VISION_ARRAY_STRATEGY=primary_with_fanout
AZURE_VISION_ARRAY='[
  {"endpoint": "https://my-vision-primary.cognitiveservices.azure.com/", "key": "<key-1>"},
  {"endpoint": "https://my-vision-overflow-1.cognitiveservices.azure.com/", "key": "<key-2>"},
  {"endpoint": "https://my-vision-overflow-2.cognitiveservices.azure.com/", "key": "<key-3>"}
]'

When AZURE_VISION_ARRAY is set, AZURE_VISION_ENDPOINT and AZURE_VISION_KEY are ignored.

LLM provider environment variables

Reducto supports multiple LLM providers through environment variables. Below is a complete list of supported providers and their required environment variables.

LiteLLM proxy

Variable	Description	Required
`LITELLM_PROXY_URL`	URL of the LiteLLM Proxy	Yes
`LITELLM_PROXY_FAST_MODEL`	Fast model to route to via the proxy	Yes
`LITELLM_PROXY_ACCURATE_MODEL`	Accurate model to route to via the proxy	Yes

OpenAI

Variable	Description	Required
`OPENAI_API_KEY`	Your OpenAI API key	Yes

Azure OpenAI

Variable	Description	Required
`AZURE_OPENAI_API_KEY`	Your Azure OpenAI API key	Yes
`AZURE_OPENAI_ENDPOINT`	Azure OpenAI endpoint (e.g., `https://your-resource-name.openai.azure.com/`)	Yes
`OPENAI_API_VERSION`	Azure OpenAI API version (e.g., `2024-10-21`)	Yes
`AZURE_OPENAI_MODEL_MAP`	Comma-separated map (or single default deployment) used to translate model names to Azure deployment names. Reducto uses the following models and each should resolve to a deployment unless a single default is supplied: `gpt-4o-2024-08-06`, `gpt-4o`, `gpt-4o-mini-2024-07-18`, `gpt-4o-mini`, `gpt-4.1`, `o1`. Example mappings: `my-default-deployment` (single default) or `gpt-4o=my-prod-dep, gpt-4o-mini=gpt4o-mini-dep`	Yes

Anthropic

Variable	Description	Required
`ANTHROPIC_API_KEY`	Your Anthropic API key	Yes

Google

Variable	Description	Required
`GOOGLE_APPLICATION_CREDENTIALS`	Service account key json with `roles/aiplatform.user` role for Vertex AI	Yes
`GCP_PROJECT_ID`	GCP project for Cloud Vision API	Yes
`GCP_REGION`	Region for Vertex AI, defaults to `us-central1`	No
`GCP_API_KEY`	API key with no Application or API restrictions to access Cloud Vision API	Yes

Gemini

Variable	Description	Required
`GEMINI_API_KEY`	Your Gemini API key	Yes

AWS Bedrock

Variable	Description	Required
`USE_CLAUDE_BEDROCK`	Set to any value to enable Claude via AWS Bedrock	Yes
`AWS_ACCESS_KEY_ID`	AWS access key ID	Yes, when using Bedrock
`AWS_SECRET_ACCESS_KEY`	AWS secret access key	Yes, when using Bedrock
`AWS_REGION`	AWS region name	Yes, when using Bedrock

GPU-based extraction models

Reducto offers GPU-based models for structured data extraction and fine-grained citations. For best results, deploy both models together. Model weights are downloaded from HuggingFace using a scoped token provided by Reducto.

Prerequisites

Create a Kubernetes secret with the HuggingFace token provided by Reducto:

kubectl create secret generic reducto-hf-token --from-literal=HF_TOKEN=hf_...

We recommend enabling modelStorage to cache weights on a PVC so restarts don’t re-download.

YAML extraction model (30B)

GPU requirement: 1x NVIDIA H200 (will not fit on H100/A100/A10G).

yamlExtract:
  enabled: true
  gpu: "H200"
  modelStorage:
    enabled: true
    size: "100Gi"
    storageClassName: "your-storage-class"

When enabled, REDUCTO_YAML_EXTRACT_URL is automatically injected into all worker and HTTP pods.

Citation model (7B)

GPU requirement: 1x NVIDIA H100 or H200.

citationModel:
  enabled: true
  gpu: "H200"  # or "H100"
  modelStorage:
    enabled: true
    size: "50Gi"
    storageClassName: "your-storage-class"

When enabled, REDUCTO_CITATION_URL is automatically injected into all worker and HTTP pods. If not deployed, citations fall back to your configured external LLM provider.

Model path overrides

Both deployments expose a modelPath field that can be updated if Reducto ships new model weights:

yamlExtract:
  modelPath: "reducto/extract_30b_0108"  # update when directed by Reducto

citationModel:
  modelPath: "reducto/citation_7b_mimo_0812"  # update when directed by Reducto

Extraction without GPU models

If you do not deploy either GPU model, extraction uses your configured external LLM provider (OpenAI, Anthropic, Google, Azure, or Bedrock). No additional configuration is needed.

Fine-tuned OpenAI extraction model (alternative)

Variable	Description	Required
`LOCAL_EXTRACT_CITATIONS_MODEL`	Fine-tuned OpenAI model ID (e.g., `openai:ft:gpt-4.1-2025-04-14:...`)	No

When set, this takes priority over the self-hosted extraction model.

Request-level LLM overrides

In addition to environment variables, on-prem deployments can override LLM configuration at the request level using the overrides parameter in experimental_options.

Key-value processing overrides

Override the model and add custom instructions for key-value (form) region processing:

{
  "document_url": "https://example.com/form.pdf",
  "experimental_options": {
    "overrides": {
      "key_value": {
        "model": "google:gemini-2.5-flash-lite",
        "custom_instructions": "Pay special attention to date fields. Use MM/DD/YYYY format."
      }
    }
  }
}

Field	Description
`model`	Model alias (`fast`, `accurate`) or `provider:model` format
`custom_instructions`	Additional instructions appended to the default prompt

Resolution order

Model resolution:

Request override - experimental_options.overrides.key_value.model
Environment variable - LOCAL_KV_MODEL
Code default - Based on deployment configuration

Prompt resolution:

Base prompt - LOCAL_KV_PROMPT env var, or built-in default
Custom instructions - Appended from overrides.key_value.custom_instructions

Environment variable defaults

Variable	Description	Default
`LOCAL_KV_MODEL`	Override model for KV processing	None (uses built-in cascade)
`LOCAL_KV_PROMPT`	Base prompt for KV processing (can be fully replaced)	Built-in prompt

AI usage tracking

Reducto includes a comprehensive AI usage tracking system that monitors language model consumption throughout the document processing pipeline. This feature provides detailed insights into token usage, request counts, and model utilization for billing and optimization purposes.

How AI usage tracking works

The AI usage tracking system operates at the block level within the parsing pipeline:

Token Counting: Each AI operation (table summarization, figure analysis, key-value extraction, etc.) records token consumption
Request Tracking: The system counts API calls made to each model
Model Identification: Usage is tracked per model type with provider information
Aggregation: Usage is aggregated across all blocks and pages for comprehensive reporting

Available via /parse API

AI usage information is currently only available through the /parse API endpoint using the custom_format parameter. This feature is not available in other API endpoints.

Usage information structure

When enabled, the system returns an AIUsageInfo object containing:

{
  "did_use_ai_models": true,
  "ai_usage_info": [
    {
      "promptTokenCount": 1500,
      "completionTokenCount": 300,
      "cachedTokenCount": 0,
      "requestCount": 2,
      "modelType": "us.anthropic.claude-3-7-sonnet-20250219-v1:0",
      "modelProvider": "anthropic",
      "modelRateLimitFamily": "us.anthropic.claude-3-7-sonnet"
    }
  ]
}

Field descriptions

did_use_ai_models: Boolean indicating whether any AI models were used during processing
ai_usage_info: Array of usage information objects, one per model type used
promptTokenCount: Total input tokens sent to the model
completionTokenCount: Total output tokens generated by the model
cachedTokenCount: Total cached tokens used (when supported by provider)
requestCount: Number of API calls made to this model
modelType: Standardized model identifier
modelProvider: Provider name (e.g., “anthropic”, “openai”)
modelRateLimitFamily: Rate limiting group for the model

Enabling AI usage tracking

To retrieve AI usage information, set the custom_format parameter to "ai_usage" in your /parse request:

{
  "input": "your_document_url",
  "settings": {
    "custom_format": "ai_usage"
  }
}

Tracked AI operations

The system tracks usage from these AI-powered features:

Table Summarization: Analysis and description of complex tables
Figure Summarization: Analysis and description of images and charts
Key-Value enrichment: Enrichment for form-like regions within documents

Model name standardization

The system automatically standardizes model names for consistent reporting:

Internal model identifiers are mapped to standard formats
Provider information is automatically added
Rate limit families are identified for capacity planning

Possible model identifiers

The following model identifiers may appear in the modelType field of AI usage tracking responses, if you have OpenAI and Anthropic access enabled:

OpenAI models

gpt-4o-2024-08-06
gpt-4o-mini-2024-07-18

Anthropic models

claude-haiku-4-5-20251001
claude-3-7-sonnet-20250219

If you enable other model providers, they have their own prefixes which will appear.

Get Started

Developer Tools

Core Functions

Workflows and Pipelines

Configurations

Reference

Components

Enterprise Resources

Security and privacy

On-premise Resources

Documentation Index

​OCR service configuration

​Textract region configuration

​Format

​Use cases

​Examples

​Azure Vision OCR

​Single endpoint

​Multiple endpoints (failover and load balancing)

Strategies

​LLM provider environment variables

​LiteLLM proxy

​OpenAI

​Azure OpenAI

​Anthropic

​Google

​Gemini

​AWS Bedrock

​GPU-based extraction models

​Prerequisites

​YAML extraction model (30B)

​Citation model (7B)

​Model path overrides

​Extraction without GPU models

​Fine-tuned OpenAI extraction model (alternative)

​Request-level LLM overrides

​Key-value processing overrides

​Resolution order

​Environment variable defaults

​AI usage tracking

​How AI usage tracking works

​Available via /parse API

​Usage information structure

​Field descriptions

​Enabling AI usage tracking

​Tracked AI operations

​Model name standardization

​Possible model identifiers

​OpenAI models

​Anthropic models

OCR service configuration

Textract region configuration

Format

Use cases

Examples

Azure Vision OCR

Single endpoint

Multiple endpoints (failover and load balancing)

LLM provider environment variables

LiteLLM proxy

OpenAI

Azure OpenAI

Anthropic

Google

Gemini

AWS Bedrock

GPU-based extraction models

Prerequisites

YAML extraction model (30B)

Citation model (7B)

Model path overrides

Extraction without GPU models

Fine-tuned OpenAI extraction model (alternative)

Request-level LLM overrides

Key-value processing overrides

Resolution order

Environment variable defaults

AI usage tracking

How AI usage tracking works

Available via /parse API

Usage information structure

Field descriptions

Enabling AI usage tracking

Tracked AI operations

Model name standardization

Possible model identifiers

OpenAI models

Anthropic models