> ## Documentation Index
> Fetch the complete documentation index at: https://docs.reducto.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Azure Blob Storage authentication

> Configure account-key or workload-identity signing for Azure Blob Storage in a self-hosted deployment.

Reducto can sign Azure Blob Storage result links without an account key. Use workload-identity signing when your storage account disables shared-key access. This configuration applies to the default storage backend in a self-hosted deployment. Hybrid VPC Entra connections require separate support.

## Choose signing credentials

Reducto selects the signing method automatically. There is no signing-mode setting.

* If an account key is configured, Reducto signs with that key. A key in `AccountKey` in the connection string or in `AZURE_STORAGE_ACCOUNT_KEY` takes precedence over workload identity. Reducto resolves the account name from `AccountName` in the connection string, then `AZURE_STORAGE_ACCOUNT_NAME`, then the Azure SDK client. If the name cannot be resolved, URL signing fails instead of selecting delegation; normal storage I/O and startup remain available.
* Without account-key signing credentials, Reducto uses `WorkloadIdentityCredential` to request a user-delegation key. It signs a read-only SAS for one blob, valid for 12 hours by default.

**Remove the dummy account key at cutover.** Remove both `AccountKey` from the connection string and `AZURE_STORAGE_ACCOUNT_KEY` from the pod configuration. A dummy key still selects account-key signing and produces unusable links. A failed account-key signature does not trigger a fallback to workload identity.

Normal uploads, downloads, and deletes continue to use `AZURE_STORAGE_CONNECTION_STRING`. For an account with shared-key access disabled, supply a SAS connection string that permits those operations. The signing identity is separate from these I/O credentials.

## Opt in without changing existing deployments

Without an account key, leaving `AZURE_CLIENT_ID`, `AZURE_TENANT_ID`, and `AZURE_FEDERATED_TOKEN_FILE` unset preserves the existing startup and connection-string I/O behavior. Only a request that needs a presigned result link fails with the missing-signing-credentials error. `AZURE_AUTHORITY_HOST` alone does not enable workload-identity signing.

Supplying any of the three identity variables opts in. All three must then be nonblank, or startup fails with `StartupError` naming the missing settings. Explicitly empty or whitespace-only values count as an attempted opt-in. A configured account key continues to take precedence and requires no identity configuration.

## Configure the customer identity

Create a managed identity in your Entra tenant. Add a federated identity credential that trusts your Kubernetes cluster's OIDC issuer and the pod's service account subject. For Helm installations, the chart creates `<release>-reducto`, so use `system:serviceaccount:<namespace>:<release>-reducto`. For example, release `reducto` in namespace `reducto` uses `system:serviceaccount:reducto:reducto-reducto`. For pods managed outside Helm, use their actual namespace and `serviceAccountName`. Set the federated credential audience to `api://AzureADTokenExchange`.

Assign these roles to the signing identity:

| Role | Scope | Purpose |
| - | - | - |
| Storage Blob Delegator | Storage account | Request user-delegation keys |
| Storage Blob Data Reader | Each container that contains result artifacts | Authorize blob reads through signed links |

A different data role can supply the required read permission, but Reader is sufficient for signing download links. The identity that supplies normal I/O credentials separately needs permission for its upload, download, and delete operations.

## Mount the federated token

Choose one of the following paths. Both mount a rotating Kubernetes service-account token file. The Azure Identity library exchanges that token for an Entra access token. It does not create the mount or inject environment variables itself. Keep `serviceAccount.create: true` so the pod's subject matches the federated credential above.

### AKS workload-identity webhook

On AKS with the OIDC issuer and workload-identity webhook enabled, the existing chart supports webhook injection through service-account annotations and pod labels. Leave `azureWorkloadIdentity.enabled` disabled (its default). Merge these settings into your existing values, keeping each streaq worker's other settings:

```yaml Webhook values theme={null}
serviceAccount:
  create: true
  annotations:
    azure.workload.identity/client-id: "<customer-managed-identity-client-id>"
    azure.workload.identity/tenant-id: "<customer-tenant-id>"
http:
  podLabels:
    azure.workload.identity/use: "true"
worker:
  podLabels:
    azure.workload.identity/use: "true"
streaqWorkerDefaults:
  podLabels:
    azure.workload.identity/use: "true"
```

The worker labels also apply to priority workers. The upgrade-preflight hook copies only `azure.workload.identity/use` from HTTP pod labels, so the webhook can inject its identity settings without applying unrelated HTTP labels to the hook. Streaq defaults apply to all pools, including full-image CPU workers; preserve the label if a pool overrides it. Add the same label to other enabled application components such as GPU workers. The webhook injects the token volume, mount, and `AZURE_*` identity variables. Configure the webhook for the appropriate Azure cloud. See [Microsoft's AKS workload identity configuration](https://learn.microsoft.com/azure/aks/workload-identity-overview).

### Explicit projection without a webhook

For a chart that includes `azureWorkloadIdentity`, merge these values into your deployment:

```yaml Explicit projection values theme={null}
serviceAccount:
  create: true
azureWorkloadIdentity:
  enabled: true
  clientId: "<customer-managed-identity-client-id>"
  tenantId: "<customer-tenant-id>"
  authorityHost: https://login.microsoftonline.com/
  audience: api://AzureADTokenExchange
  expirationSeconds: 3600
```

The chart mounts `/var/run/secrets/azure/tokens/azure-identity-token` read-only and sets `AZURE_CLIENT_ID`, `AZURE_TENANT_ID`, `AZURE_AUTHORITY_HOST`, and `AZURE_FEDERATED_TOKEN_FILE`. Shared application helpers cover HTTP, legacy workers, priority workers, all streaq pools, and application cron/preflight pods. The token mount uses no `subPath`, so kubelet can replace the file as it rotates.

The `cleanup` and `retry-stale` CronJobs also run `cron.py`, which imports storage at startup. With no account key and none of the three identity variables set, these jobs start unchanged. If any identity variable is supplied, all three must be nonblank, even though the jobs do not sign result links. The startup check only validates the variable values; it does not open the token file. The `azureWorkloadIdentity` block supplies the complete identity configuration and token mount to both jobs. The chart renders `retry-stale` when a legacy, priority, or GPU worker is enabled.

| Value | Default and requirement |
| - | - |
| `enabled` | `false`; opt in to explicit projection |
| `clientId` | Empty; required when enabled |
| `tenantId` | Empty; required when enabled |
| `authorityHost` | `https://login.microsoftonline.com/`; HTTPS authority host |
| `audience` | `api://AzureADTokenExchange`; must match the federated credential |
| `expirationSeconds` | `3600`; supported range `3600` through `86400` |

Explicit projection needs a reachable OIDC issuer and matching Entra federated trust. It does not require the AKS workload-identity webhook, its service-account annotations, or its pod labels. Do not combine this block with webhook injection or manually configured identity variables in `env`/`extraEnv`.

### Older charts: Kustomize post-renderer

Until the updated chart ships, use a Helm post-renderer to add the token volume and mount. Do not set `azureWorkloadIdentity` on an older chart, whose schema rejects that value. When opting in, set the identity variables through the existing top-level `env` values below so every application Deployment and both CronJobs receive the complete configuration. Pods with none of the three identity variables continue to start in legacy mode; pods with only some of them fail startup validation.

```yaml Older-chart identity values theme={null}
serviceAccount:
  create: true
env:
  AZURE_CLIENT_ID: "<customer-managed-identity-client-id>"
  AZURE_TENANT_ID: "<customer-tenant-id>"
  AZURE_AUTHORITY_HOST: https://login.microsoftonline.com/
  AZURE_FEDERATED_TOKEN_FILE: /var/run/secrets/azure/tokens/azure-identity-token
```

Save the following as `azure-token-post-renderer.sh`. It requires `kubectl` with Kustomize support and Bash. Set `RELEASE_NAME` to the Helm release name. `AZURE_DEPLOYMENTS` lists deployment suffixes; include every custom streaq pool key and any other enabled application deployment. The default list covers the standard application tiers. The renderer preserves existing volumes, mounts, env, sidecars, and Helm hook annotations.

```bash azure-token-post-renderer.sh theme={null}
#!/usr/bin/env bash
set -euo pipefail
: "${RELEASE_NAME:?Set RELEASE_NAME to the Helm release name}"
work_dir=$(mktemp -d)
trap 'rm -rf "$work_dir"' EXIT
cat > "$work_dir/rendered.yaml"
cat > "$work_dir/kustomization.yaml" <<'YAML'
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources: [rendered.yaml]
patches:
YAML

patch_pod() {
  local kind=$1 suffix=$2 container=$3 indent=$4 api=apps/v1
  if [[ "$kind" != Deployment ]]; then api=batch/v1; fi
  local name="${RELEASE_NAME}-reducto-${suffix}"
  cat >> "$work_dir/kustomization.yaml" <<YAML
- target:
    kind: $kind
    name: $name
  patch: |-
    apiVersion: $api
    kind: $kind
    metadata:
      name: $name
    spec:
YAML
  if [[ "$kind" == CronJob ]]; then
    printf '      jobTemplate:\n        spec:\n' >> "$work_dir/kustomization.yaml"
  fi
  cat >> "$work_dir/kustomization.yaml" <<YAML
${indent}template:
${indent}  spec:
${indent}    serviceAccountName: ${RELEASE_NAME}-reducto
${indent}    volumes:
${indent}    - name: azure-identity-token
${indent}      projected:
${indent}        sources:
${indent}        - serviceAccountToken:
${indent}            audience: api://AzureADTokenExchange
${indent}            expirationSeconds: 3600
${indent}            path: azure-identity-token
${indent}    containers:
${indent}    - name: $container
${indent}      volumeMounts:
${indent}      - name: azure-identity-token
${indent}        mountPath: /var/run/secrets/azure/tokens
${indent}        readOnly: true
YAML
}

for suffix in ${AZURE_DEPLOYMENTS:-http http-canary worker priority-worker gpu-worker gpu-ocr inference router-controller streaq-ui streaq-io streaq-cpu streaq-cpu-large}; do
  container="reducto-$suffix"
  if [[ "$suffix" == http-canary ]]; then container=reducto-http; fi
  patch_pod Deployment "$suffix" "$container" '      '
done
patch_pod Job onprem-upgrade-preflight preflight '      '
patch_pod CronJob cleanup reducto-cleanup '          '
patch_pod CronJob retry-stale reducto-retry-stale '          '
kubectl kustomize "$work_dir"
```

Use the same post-renderer on every install and upgrade until you migrate to the chart values:

```bash theme={null}
chmod +x ./azure-token-post-renderer.sh
export RELEASE_NAME=reducto
helm template "$RELEASE_NAME" ./reducto-chart --namespace reducto \
  -f values.yaml --post-renderer ./azure-token-post-renderer.sh > rendered.yaml
# Inspect rendered.yaml, then install the same chart and values:
helm upgrade --install "$RELEASE_NAME" ./reducto-chart --namespace reducto \
  -f values.yaml --post-renderer ./azure-token-post-renderer.sh
```

For a custom audience or token lifetime, change those fields in the renderer and align the federated credential. When the chart update ships, remove the post-renderer and the four manually configured identity variables, then enable `azureWorkloadIdentity` instead.

### Pods managed outside Helm

For deployments that do not use the Reducto Helm chart, merge this fragment into the actual pod template of each API and worker Deployment that uses Azure storage. Replace the container name and identity IDs with your values. Create the example `reducto` service account in the pod's namespace, or replace `serviceAccountName` with your existing account. Its federated subject must match `system:serviceaccount:<namespace>:reducto` for this example. Helm installations instead use `<release>-reducto` and the chart procedures above; pasting this fragment into Helm values does not configure the token mount.

```yaml Standalone pod-spec fragment theme={null}
spec:
  serviceAccountName: reducto
  containers:
    - name: reducto
      env:
        - name: AZURE_CLIENT_ID
          value: "<customer-managed-identity-client-id>"
        - name: AZURE_TENANT_ID
          value: "<customer-tenant-id>"
        - name: AZURE_AUTHORITY_HOST
          value: https://login.microsoftonline.com/
        - name: AZURE_FEDERATED_TOKEN_FILE
          value: /var/run/secrets/azure/tokens/azure-identity-token
      volumeMounts:
        - name: azure-identity-token
          mountPath: /var/run/secrets/azure/tokens
          readOnly: true
  volumes:
    - name: azure-identity-token
      projected:
        sources:
          - serviceAccountToken:
              audience: api://AzureADTokenExchange
              expirationSeconds: 3600
              path: azure-identity-token
```

This fragment needs no workload-identity webhook. Keep your existing storage connection-string secret and container configuration as described below.

### Storage credentials remain separate

Supply `AZURE_STORAGE_CONNECTION_STRING` through your existing secret configuration and set `AZURE_STORAGE_CONTAINER`. The connection string should contain `BlobEndpoint=https://<account>.blob.core.windows.net/;SharedAccessSignature=<customer-supplied-SAS>`, with no `AccountKey`. Keep its SAS valid for normal I/O for the pod's lifetime, or replace pods with fresh I/O credentials before it expires. This feature does not refresh the connection-string SAS.

| Variable | Requirement or default |
| - | - |
| `AZURE_CLIENT_ID` | Required for delegation signing; customer managed identity's client ID |
| `AZURE_TENANT_ID` | Required for delegation signing; customer Entra tenant ID |
| `AZURE_FEDERATED_TOKEN_FILE` | Required for delegation signing; readable mounted-token path |
| `AZURE_AUTHORITY_HOST` | Optional; Azure Identity defaults to `https://login.microsoftonline.com` |
| `AZURE_STORAGE_ACCOUNT_NAME` | Optional if the account name can be inferred from the blob endpoint |

For Azure Government, set `azureWorkloadIdentity.authorityHost: https://login.microsoftonline.us/` for explicit chart projection, or `AZURE_AUTHORITY_HOST=https://login.microsoftonline.us/` for the older-chart stopgap. For webhook injection, configure the webhook for Azure Government. Use `https://<account>.blob.core.usgovcloudapi.net/` in the connection string. The authority, tenant, identity, and storage endpoint must belong to the same cloud. Use the same audience on the federated credential and in the chart: `api://AzureADTokenExchange` works in Azure Government, but if your federated credential uses `api://AzureADTokenExchangeUSGov`, set `azureWorkloadIdentity.audience` to match. Validate token exchange and signed downloads in your Government staging environment before rollout.

## Credential and key lifecycle

Reducto creates a new workload-identity credential and a separate signing client whenever it fetches or renews a delegation key. It caches the delegation key in memory and reuses it across links. Normal blob operations retain their separate connection-string client.

For the default 12-hour links, Reducto renews the key when it cannot cover the link expiry plus a one-hour safety margin. Key and delegated SAS start times are backdated by five minutes to allow for clock skew. Backdating does not shorten the requested link lifetime.

## Troubleshoot startup and signing

Entra authentication failures and other non-retryable HTTP errors during delegation-key acquisition return non-retryable `InvalidConfigError` (HTTP 400) with federated-trust and storage-role checks. Azure 429/5xx and network failures remain retryable.

| Symptom | Action |
| - | - |
| `StartupError` lists missing identity variables | Without an account key, supplying any of `AZURE_CLIENT_ID`, `AZURE_TENANT_ID`, or `AZURE_FEDERATED_TOKEN_FILE` requires all three to be nonblank. Supply the settings named in the error, or unset all three to preserve legacy startup and I/O without delegation signing. Account-key deployments require none of them. |
| URL signing fails because the account name cannot be resolved | Supply `AZURE_STORAGE_ACCOUNT_NAME`, particularly for a custom endpoint. Keep the name and endpoint aligned. Startup and ordinary storage I/O remain available; Reducto does not switch from a configured key to workload identity. |
| Token file is missing or unreadable | Verify the path, volume mount, and file permissions in each API and worker pod. Startup validation checks the configured variable, not file existence or federated trust; file access happens when signing needs a key. |
| Entra rejects the token, including AADSTS700213 / AADSTS70021 | Check the federated credential's issuer and subject (`system:serviceaccount:<namespace>:<release>-reducto` for Helm), plus the audience, which must equal `azureWorkloadIdentity.audience` (default `api://AzureADTokenExchange`). Verify the tenant, client ID, and cloud authority. Reducto reports `InvalidConfigError`. |
| Delegation-key request returns 401/403 | Reducto reports `InvalidConfigError`. Check federated issuer/subject and Storage Blob Delegator on the storage account, plus Storage Blob Data Reader on the result container. |
| A generated link returns 403 | Without Storage Blob Data Reader on the result container, key acquisition can succeed but every signed link returns 403 when opened. Check that role, link expiry, and whether a dummy account key remains configured. |
| Azure returns 429 or 5xx during key acquisition | Reducto reports a transient error. Check Azure availability and throttling. |

Do not include token contents, account keys, delegation-key values, SAS URLs, or connection strings in diagnostic logs. Share error codes and redacted configuration instead.

To return to account-key signing, restore a valid account key and restart the pods. This requires shared-key access to be enabled on the storage account; a key cannot bypass `allowSharedKeyAccess=false`.