Skip to main content

OCR service configuration

For detailed OCR provider configuration (AWS Textract, Azure Vision, GCP Vision API, cross-cloud OCR, and GPU OCR deployment), see the dedicated OCR provider configuration page.

LLM provider environment variables

Reducto supports multiple LLM providers through environment variables. Below is a complete list of supported providers and their required environment variables. Features that call an LLM, such as intelligent spreadsheet clustering (spreadsheet_table_clustering set to intelligent), use the providers that you configure on this page. If none of them can serve intelligent clustering, Reducto uses default clustering.

LiteLLM proxy

Per-request tags

Callers can attach key/value tags to a request with settings.llm_tags. Reducto forwards each pair as a key:value entry in LiteLLM metadata.tags on every downstream LLM call the job makes, including retries and async jobs. Use it for cost attribution in LiteLLM (for example accountId and userId). Tags do not affect parsing output or caching. Limits: at most 16 entries, keys and values up to 128 characters, no : in keys.
This produces the LiteLLM tags accountId:acc_1 and userId:usr_2. The field is also available under parsing.settings.llm_tags on /extract and /split. On the legacy v2 API, pass the same object under experimental_options.llm_tags:
Tags from LITELLM_PROXY_DEFAULT_TAGS are merged with per-request tags. When a key appears in both, the request value wins.

OpenAI

Route all LLM traffic through a gateway

One gateway config covers every OpenAI call that Reducto makes, from the workers and from the Deep Extract agent service. Use it for an OpenAI-compatible gateway that you run, for example to hold the provider key, enforce a budget, or log requests. Set the three variables once, in the Secret or values that the workers read:
Example for Portkey, an OpenAI-compatible gateway:
Reducto sends each header value unchanged, so a value can hold inline JSON, such as an x-portkey-config routing config. A malformed OPENAI_DEFAULT_HEADERS value fails at startup. Workers. Parse, Extract, Split, and Edit apply both variables to their OpenAI clients (Chat Completions and Responses APIs). Super-agent sandbox (SANDBOX_ENDPOINT). The optional sandbox sidecar uses both variables too, and reaches Anthropic through ANTHROPIC_BASE_URL when set. Deep Extract agent service. With agentService.enabled, the Helm chart gives the agent service the same config, with no extra values:
  • The base URL is OPENAI_BASE_URL from env or secretEnv.stringData, without the trailing /v1. It must end with /v1 and use https, or the render fails.
  • If only an envFrom Secret holds OPENAI_BASE_URL, the chart cannot read it. The agent service then reads OPENAI_BASE_URL from the last envFrom Secret at startup, with the same rules: it must end with /v1 and use https, or the service does not start.
  • The headers come from the OPENAI_DEFAULT_HEADERS key of the Secret that the workers read: secretEnv, else the last envFrom Secret. When that Secret has no such key, the agent service sends no extra headers. If the key is in another envFrom Secret, set agentService.providerKeys.headers.openai to the key name and agentService.providerKeys.existingSecret to that Secret. A plain env.OPENAI_DEFAULT_HEADERS value fails the render: move it to a Secret.
  • The headers go only to your gateway, never to api.openai.com. With no OPENAI_BASE_URL that the chart or the service can read, the chart does not give the headers to the agent service. If the service still has headers but no gateway URL, it does not start. The exception is OpenAI-Organization and OpenAI-Project, which are OpenAI’s own headers.
  • If OPENAI_BASE_URL is in an envFrom Secret other than the last one, set agentService.tuning.gateway.base_urls.openai to the same URL without /v1.
  • An explicit agentService.tuning.gateway.base_urls.openai or agentService.providerKeys.headers.openai wins over the worker config.
The agent service adds the headers only to its OpenAI requests, never to another provider. They stay in the service process: the agent and its sandbox never receive them, and logs show only the header names. A header that the service sets itself stops the service at startup, for example authorization, x-api-key, host, content-length, content-type, accept-encoding, or a hop-by-hop header such as connection. OpenAI-Organization and OpenAI-Project are allowed. The service reads the headers once at startup, so restart its pods after you change the Secret. When the gateway refuses a request. If the gateway refuses a model call with status 401, 402, 403, 412, 429, or 446, for example an invalid key, an exhausted budget, or a rate limit, the Deep Extract job fails with LLM_PROVIDER_ERROR and the message “The configured LLM gateway refused the request (HTTP <status>)”. The response body is not shown, because it can echo the gateway config. The agent retries a 429 up to three times before it gives up. What does not use this config:
  • Azure OpenAI and the LiteLLM proxy. Each has its own variables below.
  • Other model providers, such as Anthropic, Google, and AWS Bedrock.
  • The agent service in a deployment with agentService.providerRegion: eu. The render fails when OPENAI_BASE_URL is set there, and the service does not start when it finds OPENAI_BASE_URL in an envFrom Secret.

Azure OpenAI

To call Azure OpenAI without a key, for a resource with key authentication disabled, set AZURE_AI_SERVICES_AUTH=workload_identity and remove AZURE_OPENAI_API_KEY. Reducto then sends an Entra ID token of the pod’s workload identity. Use the resource’s custom subdomain as AZURE_OPENAI_ENDPOINT, and assign the identity Cognitive Services OpenAI User on the resource. See Azure workload identity.

Anthropic

Google

Use google:<model>, for example google:gemini-3.7-flash, to call Gemini on Vertex AI.

Gemini API through Google AI Studio

Use google_aistudio:<model>. Set GEMINI_API_KEY to your Google AI Studio key, and name the provider in a deployment setting: LOCAL_KV_MODEL, AGENTIC_TABLES_MODEL, or LOCAL_EXTRACT_CITATIONS_MODEL. An API request cannot set it:
Reducto never falls back between Vertex AI and Google AI Studio. This provider is not available when LITELLM_PROXY_URL is set, and GCP_REGION does not apply to it.

AWS Bedrock

GPU-based extraction models

Reducto offers GPU-based models for structured data extraction and fine-grained citations. For best results, deploy both models together. Each model is one gpuModels.<key> block in your values. The chart ships a block per model with its image, weights pin and engine settings filled in. You set enabled, gpu, and the storage for the weights cache.

Deploy a model

Each enabled block renders a Deployment, a Service named <release>-reducto-<key>, and a PersistentVolumeClaim for the weights cache. The chart injects the model’s URL into every worker and HTTP pod as the env var in urlEnv: REDUCTO_CITATION_URL for citation, REDUCTO_YAML_EXTRACT_URL for yaml-extract. Extraction uses the YAML extract model when its URL is set, and citations use the citation model when its URL is set. Without them, both fall back to your configured external LLM provider. GPU requirements. yaml-extract (30B) needs one NVIDIA H200. citation (7B) needs one H100 or H200. Both engines are vLLM 0.31.0, built on CUDA 13, so GPU nodes need NVIDIA driver 580 or newer; upgrade node drivers before upgrading the chart, or leave the model disabled until you do. Each block lists its qualified GPUs in allowedGpus; a gpu outside that list fails the render instead of scheduling a pod that cannot load the model. gpu selects the engine flags for that GPU; it does not pin the pod to it. On a cluster with more than one GPU type, add the node label your cluster uses to the block’s nodeSelector (for example nvidia.com/gpu.product: NVIDIA-H200 from the GPU feature discovery, or your node pool label) so the pod lands on the GPU you set in gpu.

Images

Each release ships the GPU engine image with the chart, pulled through Reducto’s registry proxy as <release>-vllm; release packaging writes that image into each block’s image.repository and image.tag (and records them as image.packagedRepository and image.packagedTag). Air-gapped installs mirror that tag. To use your own mirror, set image.repository to the mirror and keep the published image.tag; to run a custom engine, set both fields. The chart rejects a changed tag on the packaged Reducto repository, since an upstream version tag does not exist there. The engine runs as uid 1000; the model pods set fsGroup: 1000 and mount a writable home directory, so a custom podSecurityContext.runAsUser also works.

Model weights

By default (weights.source: oci) an init container copies the weights from a data-only weights image, verifies its checksums and caches them on the claim. The image reference is <weights.oci.repository>:<weights.oci.tag>, with weights.oci.repository falling back to the chart’s image.repository: mirror the weights image into the same repository as the application image, or set weights.oci.repository. The weights images ship with the chart release that lists their tags in values.yaml; until your release carries one, use the hf source below. Alternatives: For the hf source, create the secret first:
weights.cache.mode: emptyDir copies the weights on every pod start instead of keeping a claim; set weights.cache.sizeLimit with it.

Block reference

Fields marked “From Reducto” come from the model’s registry file through the chart release. Leave them unless Reducto directs you to change them.

Legacy yamlExtract and citationModel blocks

Values written before the gpuModels block, with yamlExtract.enabled or citationModel.enabled, keep working unchanged: the same Service names, claim names (<release>-yaml-extract-model-pvc, <release>-citation-model-pvc), env vars and container settings render as before, including modelStorage, whose init container syncs the weights at start. Those blocks keep their own engines: citationModel runs the vLLM engine shipped as <release>-vllm, yamlExtract the SGLang engine shipped as <release>-sglang. They serve the pinned commit in pinnedRevision for the default modelPath; a different modelPath serves that repository’s default branch, or the commit you set in revision. Set offline: true for an air-gapped install with a pre-populated model cache; hfTokenSecret controls authentication only, so public repositories download without a token. To move a cluster to the pinned gpuModels block, disable the legacy block and enable its twin (citationModel to gpuModels.citation, yamlExtract to gpuModels.yaml-extract) in the same upgrade. The Service name and the env var stay the same. The weights are cached on a new claim, and the upgrade deletes the legacy claim (<release>-citation-model-pvc or <release>-yaml-extract-model-pvc) with its download, so have the weights image mirrored, or the token secret in place for the hf source, before the upgrade. Enabling a legacy block and its twin together fails the render. The model pods set fsGroup: 1000 by default, so the weights cache is writable for the init container (user 65532 on the Chainguard chart line, where minimalContainerSecurityContext.enabled is true) and readable for the engine (user 1000). A podSecurityContext of your own keeps that fsGroup unless it sets one; a storage driver that ignores fsGroup needs weights.cache.mode: emptyDir or a prepared claim in weights.cache.existingClaim. The chart serves a pinned commit of the default repositories. A different modelPath serves that repository’s default branch, or the commit you set in revision.

Extraction without GPU models

If you do not deploy either GPU model, extraction uses your configured external LLM provider (OpenAI, Anthropic, Google, Azure, or Bedrock). No additional configuration is needed.

Fine-tuned OpenAI extraction model (alternative)

When set, this takes priority over the self-hosted extraction model.

Request-level LLM overrides

In addition to environment variables, on-prem deployments can override LLM configuration at the request level using the overrides parameter in experimental_options.

Key-value processing overrides

Override the model and add custom instructions for key-value (form) region processing:

Resolution order

Model resolution:
  1. Request override - experimental_options.overrides.key_value.model
  2. Environment variable - LOCAL_KV_MODEL
  3. Code default - Based on deployment configuration
Prompt resolution:
  1. Base prompt - LOCAL_KV_PROMPT env var, or built-in default
  2. Custom instructions - Appended from overrides.key_value.custom_instructions

Environment variable defaults

AI usage tracking

Reducto includes a comprehensive AI usage tracking system that monitors language model consumption throughout the document processing pipeline. This feature provides detailed insights into token usage, request counts, and model utilization for billing and optimization purposes.

How AI usage tracking works

The AI usage tracking system operates at the block level within the parsing pipeline:
  1. Token Counting: Each AI operation (table summarization, figure analysis, key-value extraction, etc.) records token consumption
  2. Request Tracking: The system counts API calls made to each model
  3. Model Identification: Usage is tracked per model type with provider information
  4. Aggregation: Usage is aggregated across all blocks and pages for comprehensive reporting

Available via /parse API

AI usage information is currently only available through the /parse API endpoint using the custom_format parameter. This feature is not available in other API endpoints.

Usage information structure

When enabled, the system returns an AIUsageInfo object containing:

Field descriptions

  • did_use_ai_models: Boolean indicating whether any AI models were used during processing
  • ai_usage_info: Array of usage information objects, one per model type used
  • promptTokenCount: Total input tokens sent to the model
  • completionTokenCount: Total output tokens generated by the model
  • cachedTokenCount: Total cached tokens used (when supported by provider)
  • requestCount: Number of API calls made to this model
  • modelType: Standardized model identifier
  • modelProvider: Provider name (e.g., “anthropic”, “openai”)
  • modelRateLimitFamily: Rate limiting group for the model

Enabling AI usage tracking

To retrieve AI usage information, set the custom_format parameter to "ai_usage" in your /parse request:

Tracked AI operations

The system tracks usage from these AI-powered features:
  • Table Summarization: Analysis and description of complex tables
  • Figure Summarization: Analysis and description of images and charts
  • Key-Value enrichment: Enrichment for form-like regions within documents

Model name standardization

The system automatically standardizes model names for consistent reporting:
  • Internal model identifiers are mapped to standard formats
  • Provider information is automatically added
  • Rate limit families are identified for capacity planning

Possible model identifiers

The following model identifiers may appear in the modelType field of AI usage tracking responses, if you have OpenAI and Anthropic access enabled:

OpenAI models

  • gpt-4o-2024-08-06
  • gpt-4o-mini-2024-07-18

Anthropic models

  • claude-haiku-4-5-20251001
  • claude-3-7-sonnet-20250219
If you enable other model providers, they have their own prefixes which will appear.