LLM & service configuration options

OCR service configuration

For detailed OCR provider configuration (AWS Textract, Azure Vision, GCP Vision API, cross-cloud OCR, and GPU OCR deployment), see the dedicated OCR provider configuration page.

LLM provider environment variables

Reducto supports multiple LLM providers through environment variables. Below is a complete list of supported providers and their required environment variables.

LiteLLM proxy

Variable	Description	Required
`LITELLM_PROXY_URL`	URL of the LiteLLM Proxy	Yes
`LITELLM_PROXY_FAST_MODEL`	Fast model to route to via the proxy	Yes
`LITELLM_PROXY_ACCURATE_MODEL`	Accurate model to route to via the proxy	Yes
`LITELLM_PROXY_DEFAULT_TAGS`	JSON object of key/value tags added to every proxy request as `key:value` `metadata.tags`, for example `{"deployment": "prod-us"}`	No

Per-request tags

Callers can attach key/value tags to a request with settings.llm_tags. Reducto forwards each pair as a key:value entry in LiteLLM metadata.tags on every downstream LLM call the job makes, including retries and async jobs. Use it for cost attribution in LiteLLM (for example accountId and userId). Tags do not affect parsing output or caching. Limits: at most 16 entries, keys and values up to 128 characters, no : in keys.

from reducto import Reducto

client = Reducto()

result = client.parse.run(
    input="https://example.com/doc.pdf",
    settings={"llm_tags": {"accountId": "acc_1", "userId": "usr_2"}},
)

import Reducto from "reductoai";

const client = new Reducto();

const result = await client.parse.run({
  input: "https://example.com/doc.pdf",
  settings: { llm_tags: { accountId: "acc_1", userId: "usr_2" } },
});

curl -X POST https://your-reducto-host/parse \
  -H "Authorization: Bearer $REDUCTO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "https://example.com/doc.pdf",
    "settings": {
      "llm_tags": {"accountId": "acc_1", "userId": "usr_2"}
    }
  }'

This produces the LiteLLM tags accountId:acc_1 and userId:usr_2. The field is also available under parsing.settings.llm_tags on /extract and /split. On the legacy v2 API, pass the same object under experimental_options.llm_tags:

from reducto import Reducto

client = Reducto()

result = client.parse.run(
    document_url="https://example.com/doc.pdf",
    experimental_options={
        "llm_tags": {"accountId": "acc_1", "userId": "usr_2"}
    },
)

import Reducto from "reductoai";

const client = new Reducto();

const result = await client.parse.run({
  document_url: "https://example.com/doc.pdf",
  experimental_options: {
    llm_tags: { accountId: "acc_1", userId: "usr_2" },
  },
});

curl -X POST https://your-reducto-host/parse \
  -H "Authorization: Bearer $REDUCTO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "document_url": "https://example.com/doc.pdf",
    "experimental_options": {
      "llm_tags": {"accountId": "acc_1", "userId": "usr_2"}
    }
  }'

Tags from LITELLM_PROXY_DEFAULT_TAGS are merged with per-request tags. When a key appears in both, the request value wins.

OpenAI

Variable	Description	Required
`OPENAI_API_KEY`	Your OpenAI API key	Yes
`OPENAI_BASE_URL`	Base URL for all OpenAI requests. Set this to route traffic through an OpenAI-compatible gateway or proxy. Defaults to `https://api.openai.com/v1`	No
`OPENAI_DEFAULT_HEADERS`	JSON object of extra headers sent on every OpenAI request. Use this for gateway authentication or routing headers. Keys and values must be strings	No

Route OpenAI through a gateway

Set all three variables on every Reducto workload:

OPENAI_API_KEY=sk-...
OPENAI_BASE_URL='https://<gateway-host>/v1'
OPENAI_DEFAULT_HEADERS='{"<header-name>": "<header-value>"}'

Example for Portkey:

OPENAI_BASE_URL=https://api.portkey.ai/v1
OPENAI_DEFAULT_HEADERS='{"x-portkey-api-key": "<portkey-key>", "x-portkey-provider": "openai"}'

Reducto applies OPENAI_BASE_URL and OPENAI_DEFAULT_HEADERS to the OpenAI clients used by Parse, Extract, Split, and Edit (Chat Completions and Responses APIs). They do not apply to Azure OpenAI, to the LiteLLM proxy, or to the optional super-agent sandbox (SANDBOX_ENDPOINT). A malformed OPENAI_DEFAULT_HEADERS value fails at startup.

Azure OpenAI

Variable	Description	Required
`AZURE_OPENAI_API_KEY`	Your Azure OpenAI API key	Yes
`AZURE_OPENAI_ENDPOINT`	Azure OpenAI endpoint (e.g., `https://your-resource-name.openai.azure.com/`)	Yes
`OPENAI_API_VERSION`	Azure OpenAI API version (e.g., `2024-10-21`)	Yes
`AZURE_OPENAI_MODEL_MAP`	Comma-separated map (or single default deployment) used to translate model names to Azure deployment names. Reducto uses the following models and each should resolve to a deployment unless a single default is supplied: `gpt-4o-2024-08-06`, `gpt-4o`, `gpt-4o-mini-2024-07-18`, `gpt-4o-mini`, `gpt-4.1`, `o1`. Example mappings: `my-default-deployment` (single default) or `gpt-4o=my-prod-dep, gpt-4o-mini=gpt4o-mini-dep`	Yes

Anthropic

Variable	Description	Required
`ANTHROPIC_API_KEY`	Your Anthropic API key	Yes

Google

Variable	Description	Required
`GOOGLE_APPLICATION_CREDENTIALS`	Service account key json with `roles/aiplatform.user` role for Vertex AI	Yes
`GCP_PROJECT_ID`	GCP project for Cloud Vision API	Yes
`GCP_REGION`	Region for Vertex AI. When unset, empty, or `global`, calls use the `global` endpoint and fall back to the Gemini API (`GEMINI_API_KEY`) on a quota or server error. Any other value pins every call to that region with no Gemini API fallback. Most Gemini 3.x models are served only from `global` and the `us` and `eu` multi-region endpoints, so a `us-` or `europe-` pin routes those calls to `us` or `eu`. The Gemini 3.x preview models are served only from `global`. A pin that cannot serve a model logs a warning, and the call fails.	No
`GCP_API_KEY`	API key with no Application or API restrictions to access Cloud Vision API	Yes

Gemini

Variable	Description	Required
`GEMINI_API_KEY`	Your Gemini API key	Yes

AWS Bedrock

Variable	Description	Required
`USE_CLAUDE_BEDROCK`	Set to any value to enable Claude via AWS Bedrock	Yes
`AWS_ACCESS_KEY_ID`	AWS access key ID	Yes, when using Bedrock
`AWS_SECRET_ACCESS_KEY`	AWS secret access key	Yes, when using Bedrock
`AWS_REGION`	AWS region name	Yes, when using Bedrock

GPU-based extraction models

Reducto offers GPU-based models for structured data extraction and fine-grained citations. For best results, deploy both models together. Each model is one gpuModels.<key> block in your values. The chart ships a block per model with its image, weights pin and engine settings filled in. You set enabled, gpu, and the storage for the weights cache.

Deploy a model

gpuModels:
  citation:
    enabled: true
    gpu: "H200"            # or "H100"; must be in the block's allowedGpus
    weights:
      cache:
        size: "50Gi"
        storageClassName: "your-storage-class"
  yaml-extract:
    enabled: true
    gpu: "H200"
    weights:
      cache:
        size: "100Gi"
        storageClassName: "your-storage-class"

Each enabled block renders a Deployment, a Service named <release>-reducto-<key>, and a PersistentVolumeClaim for the weights cache. The chart injects the model’s URL into every worker and HTTP pod as the env var in urlEnv: REDUCTO_CITATION_URL for citation, REDUCTO_YAML_EXTRACT_URL for yaml-extract. Extraction uses the YAML extract model when its URL is set, and citations use the citation model when its URL is set. Without them, both fall back to your configured external LLM provider. GPU requirements. yaml-extract (30B) needs one NVIDIA H200. citation (7B) needs one H100 or H200. Both engines are vLLM 0.31.0, built on CUDA 13, so GPU nodes need NVIDIA driver 580 or newer; upgrade node drivers before upgrading the chart, or leave the model disabled until you do. Each block lists its qualified GPUs in allowedGpus; a gpu outside that list fails the render instead of scheduling a pod that cannot load the model. gpu selects the engine flags for that GPU; it does not pin the pod to it. On a cluster with more than one GPU type, add the node label your cluster uses to the block’s nodeSelector (for example nvidia.com/gpu.product: NVIDIA-H200 from the GPU feature discovery, or your node pool label) so the pod lands on the GPU you set in gpu.

Images

Each release ships the GPU engine image with the chart, pulled through Reducto’s registry proxy as <release>-vllm; release packaging writes that image into each block’s image.repository and image.tag (and records them as image.packagedRepository and image.packagedTag). Air-gapped installs mirror that tag. To use your own mirror, set image.repository to the mirror and keep the published image.tag; to run a custom engine, set both fields. The chart rejects a changed tag on the packaged Reducto repository, since an upstream version tag does not exist there. The engine runs as uid 1000; the model pods set fsGroup: 1000 and mount a writable home directory, so a custom podSecurityContext.runAsUser also works.

Model weights

By default (weights.source: oci) an init container copies the weights from a data-only weights image, verifies its checksums and caches them on the claim. The image reference is <weights.oci.repository>:<weights.oci.tag>, with weights.oci.repository falling back to the chart’s image.repository: mirror the weights image into the same repository as the application image, or set weights.oci.repository. The weights images ship with the chart release that lists their tags in values.yaml; until your release carries one, use the hf source below. Alternatives:

`weights.source`	Behavior	Needs
`oci` (default)	Copy from the weights image and verify	The weights image in your registry
`hf`	Download the pinned revision from Hugging Face in the server image	A Kubernetes secret with the token Reducto provides, in `weights.hf.tokenSecret.name`
`prepopulated`	The cache was filled out of band; only its sentinel is checked	A claim in `weights.cache.existingClaim`

For the hf source, create the secret first:

kubectl create secret generic reducto-hf-token --from-literal=HF_TOKEN=hf_...

gpuModels:
  citation:
    enabled: true
    weights:
      source: hf
      hf:
        tokenSecret:
          name: reducto-hf-token

weights.cache.mode: emptyDir copies the weights on every pod start instead of keeping a claim; set weights.cache.sizeLimit with it.

Block reference

Field	Default	Description
`enabled`	`false`	Render the model
`gpu`	The first entry of `allowedGpus`	The GPU this cluster schedules the model on
`allowedGpus`	From Reducto	The GPUs the model is qualified on
`image.repository`, `image.tag`	From Reducto	The engine image. Empty values use the application image repository and `<image.tag>-<engine.runtime>`
`model.repo`, `model.revision`, `model.servedNames`	From Reducto	The weights pin and every name the engine serves
`engine.runtime`, `engine.args`, `engine.env`, `engine.profiles`	From Reducto	The engine (`vllm` or `sglang`), its flags, its env, and per-GPU flags appended for the selected `gpu`
`weights`	`source: oci`	Where the weights come from and how they are cached (see above)
`resources`	Per model	Requests and limits; keep `nvidia.com/gpu: "1"`
`replicaCount`	`1`	Replicas
`nodeSelector`, `tolerations`, `affinity`, `podAnnotations`, `podLabels`	Per model	Scheduling
`extraEnv`	`[]`	Extra env entries for the engine container
`port`, `service.port`	`8000`, `80`	The container port and the Service port
`urlEnv`	`REDUCTO_<KEY>_URL`	The env var the worker and HTTP pods receive
`probe.startupSeconds`	`630` (vllm), `960` (sglang)	Time the engine may take to load before the pod restarts

Fields marked “From Reducto” come from the model’s registry file through the chart release. Leave them unless Reducto directs you to change them.

Legacy `yamlExtract` and `citationModel` blocks

Values written before the gpuModels block, with yamlExtract.enabled or citationModel.enabled, keep working unchanged: the same Service names, claim names (<release>-yaml-extract-model-pvc, <release>-citation-model-pvc), env vars and container settings render as before, including modelStorage, whose init container syncs the weights at start. Those blocks keep their own engines: citationModel runs the vLLM engine shipped as <release>-vllm, yamlExtract the SGLang engine shipped as <release>-sglang. They serve the pinned commit in pinnedRevision for the default modelPath; a different modelPath serves that repository’s default branch, or the commit you set in revision. Set offline: true for an air-gapped install with a pre-populated model cache; hfTokenSecret controls authentication only, so public repositories download without a token. To move a cluster to the pinned gpuModels block, disable the legacy block and enable its twin (citationModel to gpuModels.citation, yamlExtract to gpuModels.yaml-extract) in the same upgrade. The Service name and the env var stay the same. The weights are cached on a new claim, and the upgrade deletes the legacy claim (<release>-citation-model-pvc or <release>-yaml-extract-model-pvc) with its download, so have the weights image mirrored, or the token secret in place for the hf source, before the upgrade. Enabling a legacy block and its twin together fails the render. The model pods set fsGroup: 1000 by default, so the weights cache is writable for the init container (user 65532 on the Chainguard chart line, where minimalContainerSecurityContext.enabled is true) and readable for the engine (user 1000). A podSecurityContext of your own keeps that fsGroup unless it sets one; a storage driver that ignores fsGroup needs weights.cache.mode: emptyDir or a prepared claim in weights.cache.existingClaim. The chart serves a pinned commit of the default repositories. A different modelPath serves that repository’s default branch, or the commit you set in revision.

Extraction without GPU models

If you do not deploy either GPU model, extraction uses your configured external LLM provider (OpenAI, Anthropic, Google, Azure, or Bedrock). No additional configuration is needed.

Fine-tuned OpenAI extraction model (alternative)

Variable	Description	Required
`LOCAL_EXTRACT_CITATIONS_MODEL`	Fine-tuned OpenAI model ID (e.g., `openai:ft:gpt-4.1-2025-04-14:...`)	No

When set, this takes priority over the self-hosted extraction model.

Request-level LLM overrides

In addition to environment variables, on-prem deployments can override LLM configuration at the request level using the overrides parameter in experimental_options.

Key-value processing overrides

Override the model and add custom instructions for key-value (form) region processing:

{
  "document_url": "https://example.com/form.pdf",
  "experimental_options": {
    "overrides": {
      "key_value": {
        "model": "google:gemini-3.5-flash-lite",
        "custom_instructions": "Pay special attention to date fields. Use MM/DD/YYYY format."
      }
    }
  }
}

Field	Description
`model`	Model alias (`fast`, `accurate`) or `provider:model` format
`custom_instructions`	Additional instructions appended to the default prompt

Resolution order

Model resolution:

Request override - experimental_options.overrides.key_value.model
Environment variable - LOCAL_KV_MODEL
Code default - Based on deployment configuration

Prompt resolution:

Base prompt - LOCAL_KV_PROMPT env var, or built-in default
Custom instructions - Appended from overrides.key_value.custom_instructions

Environment variable defaults

Variable	Description	Default
`LOCAL_KV_MODEL`	Override model for KV processing	None (uses built-in cascade)
`LOCAL_KV_PROMPT`	Base prompt for KV processing (can be fully replaced)	Built-in prompt

AI usage tracking

Reducto includes a comprehensive AI usage tracking system that monitors language model consumption throughout the document processing pipeline. This feature provides detailed insights into token usage, request counts, and model utilization for billing and optimization purposes.

How AI usage tracking works

The AI usage tracking system operates at the block level within the parsing pipeline:

Token Counting: Each AI operation (table summarization, figure analysis, key-value extraction, etc.) records token consumption
Request Tracking: The system counts API calls made to each model
Model Identification: Usage is tracked per model type with provider information
Aggregation: Usage is aggregated across all blocks and pages for comprehensive reporting

Available via /parse API

AI usage information is currently only available through the /parse API endpoint using the custom_format parameter. This feature is not available in other API endpoints.

Usage information structure

When enabled, the system returns an AIUsageInfo object containing:

{
  "did_use_ai_models": true,
  "ai_usage_info": [
    {
      "promptTokenCount": 1500,
      "completionTokenCount": 300,
      "cachedTokenCount": 0,
      "requestCount": 2,
      "modelType": "us.anthropic.claude-3-7-sonnet-20250219-v1:0",
      "modelProvider": "anthropic",
      "modelRateLimitFamily": "us.anthropic.claude-3-7-sonnet"
    }
  ]
}

Field descriptions

did_use_ai_models: Boolean indicating whether any AI models were used during processing
ai_usage_info: Array of usage information objects, one per model type used
promptTokenCount: Total input tokens sent to the model
completionTokenCount: Total output tokens generated by the model
cachedTokenCount: Total cached tokens used (when supported by provider)
requestCount: Number of API calls made to this model
modelType: Standardized model identifier
modelProvider: Provider name (e.g., “anthropic”, “openai”)
modelRateLimitFamily: Rate limiting group for the model

Enabling AI usage tracking

To retrieve AI usage information, set the custom_format parameter to "ai_usage" in your /parse request:

{
  "input": "your_document_url",
  "settings": {
    "custom_format": "ai_usage"
  }
}

Tracked AI operations

The system tracks usage from these AI-powered features:

Table Summarization: Analysis and description of complex tables
Figure Summarization: Analysis and description of images and charts
Key-Value enrichment: Enrichment for form-like regions within documents

Model name standardization

The system automatically standardizes model names for consistent reporting:

Internal model identifiers are mapped to standard formats
Provider information is automatically added
Rate limit families are identified for capacity planning

Possible model identifiers

The following model identifiers may appear in the modelType field of AI usage tracking responses, if you have OpenAI and Anthropic access enabled:

OpenAI models

gpt-4o-2024-08-06
gpt-4o-mini-2024-07-18

Anthropic models

claude-haiku-4-5-20251001
claude-3-7-sonnet-20250219

If you enable other model providers, they have their own prefixes which will appear.

​OCR service configuration

​LLM provider environment variables

​LiteLLM proxy

​Per-request tags

​OpenAI

​Route OpenAI through a gateway

​Azure OpenAI

​Anthropic

​Google

​Gemini

​AWS Bedrock

​GPU-based extraction models

​Deploy a model

​Images

​Model weights

​Block reference

​Legacy yamlExtract and citationModel blocks

​Extraction without GPU models

​Fine-tuned OpenAI extraction model (alternative)

​Request-level LLM overrides

​Key-value processing overrides

​Resolution order

​Environment variable defaults

​AI usage tracking

​How AI usage tracking works

​Available via /parse API

​Usage information structure

​Field descriptions

​Enabling AI usage tracking

​Tracked AI operations

​Model name standardization

​Possible model identifiers

​OpenAI models

​Anthropic models