Hybrid VPC — AWS S3

This guide covers setting up Hybrid VPC with AWS S3 as your storage backend. Reducto assumes an IAM role in your AWS account to read and write documents.

Prerequisites

AWS account(s): Can use separate accounts for development, staging, and production
Terraform 1.2+: For infrastructure provisioning
Values from Reducto (provided during onboarding):
- ExternalId for secure role assumption
- Endpoint Service name and region (if using PrivateLink, also listed below)
If using PrivateLink: Send Reducto every AWS account ID that will create a VPC endpoint, including separate dev, staging, production, or organizational accounts. Reducto must allow-list each account on the VPC Endpoint Service before endpoint creation succeeds.

Reducto regions

Reducto runs separate deployments in the US, EU, and AU. Each deployment uses only buckets in its own area and assumes your IAM role with its own principals. Find the Reducto region for your bucket’s AWS region, then use that row’s principal ARNs and API base URL.

Reducto region	Bucket AWS regions	API base URL
US	`us-`, `ca-`	`https://platform.reducto.ai`
EU	`eu-*`	`https://eu.platform.reducto.ai`
AU	`ap-southeast-2`	`https://au.platform.reducto.ai`

Buckets in other AWS regions are not supported.

Principal ARNs

Trust all three ARNs for the Reducto region that serves your bucket:

Reducto region	EKS Role ARN	Modal ARNs
US	`arn:aws:iam::731106932034:role/reducto-prod`	`arn:aws:iam::731106932034:user/modal-prod`, `arn:aws:iam::731106932034:role/modal-prod`
EU	`arn:aws:iam::731106932034:role/reducto-prod-eu`	`arn:aws:iam::731106932034:user/modal-prod-eu`, `arn:aws:iam::731106932034:role/modal-prod-eu`
AU	`arn:aws:iam::731106932034:role/reducto-prod-au`	`arn:aws:iam::731106932034:user/modal-prod-au`, `arn:aws:iam::731106932034:role/modal-prod-au`

Principal ARNs are region-specific. The US ARNs do not work for an EU or AU bucket, and the EU and AU ARNs work only in their own region. If your role trusts another region’s principals, every job fails with AccessDenied on sts:AssumeRole.

Trust both Modal ARNs for your region. Reducto’s Modal compute assumes your role either as the IAM user or as the IAM role of the same name. Omitting the role ARN causes AccessDenied on sts:AssumeRole for jobs that finalize on Modal.

VPC Endpoint Service Configuration

If using PrivateLink, use the endpoint service for the Reducto region that serves your bucket, and create your VPC endpoint in one of its supported VPC regions:

Reducto region	VPC Endpoint Service Name	Service region	Supported VPC regions	DNS Name
US	`com.amazonaws.vpce.us-west-2.vpce-svc-0929182c8ed77b7a8`	`us-west-2`	`us-east-1`, `us-east-2`, `us-west-1`, `us-west-2`, `eu-west-1`	`hybrid.platform.reducto.ai`
EU	`com.amazonaws.vpce.eu-central-1.vpce-svc-0a231d441f3a482a0`	`eu-central-1`	`eu-central-1`, `eu-west-1`, `eu-west-2`, `eu-west-3`, `eu-north-1`	`hybrid.eu.platform.reducto.ai`
AU	`com.amazonaws.vpce.ap-southeast-2.vpce-svc-0da3ceba709035c36`	`ap-southeast-2`	`ap-southeast-2`	`hybrid.au.platform.reducto.ai`

If your VPC is in a different region from the service region, the endpoint connects cross-region. Set reducto_endpoint_service_region to the service region. To connect from a VPC region that is not listed, contact Reducto.

PrivateLink endpoint creation only works after Reducto has allow-listed the AWS account that creates the endpoint. If your organization uses separate AWS accounts for dev, staging, production, or separate business units, provide each account ID before setup.

Setup

Clone the infrastructure repository

git clone https://github.com/reductoai-collab/reducto-hybrid-infra.git
cd reducto-hybrid-infra

Create terraform.tfvars

Use the principal ARNs for the Reducto region that serves your bucket:

name_prefix = "reducto-us"

reducto_principal_arns = [
  "arn:aws:iam::731106932034:role/reducto-prod",
  "arn:aws:iam::731106932034:user/modal-prod",
  "arn:aws:iam::731106932034:role/modal-prod"
]
reducto_external_id = "<external-id-from-reducto>"

name_prefix = "reducto-eu"

reducto_principal_arns = [
  "arn:aws:iam::731106932034:role/reducto-prod-eu",
  "arn:aws:iam::731106932034:user/modal-prod-eu",
  "arn:aws:iam::731106932034:role/modal-prod-eu"
]
reducto_external_id = "<external-id-from-reducto>"

name_prefix = "reducto-au"

reducto_principal_arns = [
  "arn:aws:iam::731106932034:role/reducto-prod-au",
  "arn:aws:iam::731106932034:user/modal-prod-au",
  "arn:aws:iam::731106932034:role/modal-prod-au"
]
reducto_external_id = "<external-id-from-reducto>"

Optional variables:

# Bucket name (default: <name_prefix>-reducto-<random suffix>)
# bucket_name = "my-company-reducto-data"

# Object retention in days (default: 1, minimum: 1)
# lifecycle_expiration_days = 1

tags = {
  Environment = "production"
  Project     = "reducto-hybrid"
}

Initialize and apply

The root module has no provider block, so Terraform creates the bucket in the region from your AWS environment. Set it to your bucket’s region:

export AWS_REGION=ap-southeast-2   # your bucket's region
terraform init
terraform plan
terraform apply

Share outputs with Reducto

terraform output -json integration_values

Example output:

{
  "access_mode": "assume_role",
  "account_id": "987654321098",
  "bucket_name": "reducto-au-reducto-a1b2c3d4",
  "privatelink_endpoint_id": null,
  "region": "ap-southeast-2",
  "role_arn": "arn:aws:iam::987654321098:role/reducto-au-reducto-access"
}

Send this output to your Reducto team. Reducto registers the bucket and role for your organization and confirms when it is active.

Components provisioned

Component	Purpose	Required
S3 Bucket	Document and artifact storage with lifecycle expiration, SSE-S3 encryption, and public access blocked	Yes
IAM Role	`<name_prefix>-reducto-access`: cross-account access for Reducto with ExternalId protection	Yes
VPC Endpoint	PrivateLink endpoint for private API access, with a security group allowing HTTPS from the VPC CIDR	Optional

Access Mode

Keep the default, access_mode = "assume_role". Reducto reaches your bucket by assuming the IAM role with your ExternalId.

The module also accepts access_mode = "bucket_policy", which grants the principals access through a bucket policy and creates no IAM role. Reducto’s Hybrid VPC storage requires a role ARN and ExternalId, so it cannot use a bucket set up this way. Do not use bucket_policy for Hybrid VPC.

PrivateLink Setup (Optional)

For private-only API access without traversing the public internet:

Request PrivateLink enablement

Provide the following to your Reducto team:

AWS Account ID(s): Every account where you’ll create a VPC endpoint, including dev, staging, production, or separate organizational accounts
Region(s): The AWS regions of the VPCs that will connect, and the Reducto region that serves your bucket

Reducto will add each account root as an allowed principal on the VPC Endpoint Service. Wait for Reducto’s confirmation before you create the endpoint.

Configure VPC endpoint

Add to your terraform.tfvars, using the service name and service region for your Reducto region from the table above:

enable_privatelink              = true
vpc_id                          = "vpc-0123456789abcdef0"
subnet_ids                      = ["subnet-abc123", "subnet-def456"]
reducto_endpoint_service_name   = "com.amazonaws.vpce.ap-southeast-2.vpce-svc-0da3ceba709035c36"
reducto_endpoint_service_region = "ap-southeast-2"

The module creates a security group that allows HTTPS from the VPC’s primary CIDR. If clients connect from other networks, such as peered VPCs, Transit Gateway, or VPN, pass your own security groups in privatelink_security_group_ids.

Configure your client

Use the DNS name for your Reducto region:

from reducto import Reducto

client = Reducto(
    api_key="your-api-key",
    base_url="https://hybrid.au.platform.reducto.ai",  # US: hybrid.platform.reducto.ai, EU: hybrid.eu.platform.reducto.ai
)

Private DNS must be enabled on the VPC endpoint so the hybrid.*.platform.reducto.ai name resolves to it inside your VPC. The module enables it. If you create the endpoint another way, enable private DNS yourself.

Require PrivateLink

By default, your organization can also call the public API base URL. Ask Reducto to enable PrivateLink-only access for your organization. Reducto then rejects requests that do not arrive through PrivateLink with HTTP 403.

Validation Checklist

After terraform apply, verify your setup:

Terraform apply succeeded without errors

S3 bucket has lifecycle rule:

aws s3api get-bucket-lifecycle-configuration --bucket your-bucket-name

S3 bucket blocks public access: All four settings are true

aws s3api get-public-access-block --bucket your-bucket-name

IAM role trust policy is correct: The principals are all three ARNs for your Reducto region, and the sts:ExternalId condition matches the ExternalId from Reducto
aws iam get-role --role-name "$(terraform output -raw assumable_role_name)" --query 'Role.AssumeRolePolicyDocument'
If PrivateLink enabled: Reducto has confirmed that every endpoint-creating AWS account is allow-listed

If PrivateLink enabled: Endpoint status shows “available”

aws ec2 describe-vpc-endpoints --vpc-endpoint-ids vpce-xxx

Smoke test: After Reducto confirms the setup, run a small job against your Reducto region’s API base URL and verify objects appear in the bucket

Troubleshooting

AccessDenied on sts:AssumeRole

Problem: Jobs fail because Reducto gets AccessDenied when it assumes your role. Solution: Check the trust policy with the aws iam get-role command from the Validation Checklist. The common causes are:

Wrong region’s principals: The role trusts the US ARNs, but your bucket is served by EU or AU (or the reverse). Set reducto_principal_arns to the Principal ARNs row for your bucket’s Reducto region and run terraform apply again.
Missing Modal role ARN: The role trusts user/modal-* but not role/modal-*. Add the role ARN for your region.
ExternalId mismatch: The sts:ExternalId value in the trust policy must exactly match the ExternalId Reducto registered for your organization.
Role ARN mismatch: The role ARN Reducto registered must match the role you created. The module names it <name_prefix>-reducto-access.

AccessDenied on S3 after the role is assumed

Problem: Reducto assumes the role, but reads or writes fail with AccessDenied. Solution: Check that no bucket policy denies the role. If the bucket uses SSE-KMS with a customer managed key, grant the KMS permissions in Encryption. To read s3:// inputs from a bucket other than the Reducto bucket, add s3:GetObject on that bucket to the role’s policy.

Region cannot be used with this deployment

Problem: The API returns an error that your Hybrid VPC region or environment cannot be used with this deployment, or that your organization is not configured for this region. Solution: Call the API base URL for the Reducto region that serves your bucket (see Reducto regions). With PrivateLink, use that region’s endpoint service and DNS name.

VPC endpoint service does not exist

Problem: AWS returns InvalidServiceName or says the VPC Endpoint Service does not exist, but the service name and region match the table above. Solution: Send Reducto the AWS account ID for the account creating the endpoint. Reducto will allow-list the account root on the endpoint service. After Reducto confirms the change, retry endpoint creation. Repeat this for each account that will create an endpoint. Also confirm that your VPC’s region is one of the service’s supported VPC regions.

Multi-Region Setup

Deploy one bucket and IAM role per region. Each role must trust the principals of the Reducto region that serves its bucket. Give each region its own name_prefix, because the IAM role name (<name_prefix>-reducto-access) must be unique in an AWS account. Copy the minimal example into one directory per region. Each directory keeps its own Terraform state:

mkdir -p regions
cp -r examples/minimal regions/us-east-1
cp -r examples/minimal regions/eu-central-1
cp -r examples/minimal regions/ap-southeast-2

Create a terraform.tfvars in each directory with that region’s principal ARNs:

# regions/us-east-1/terraform.tfvars
aws_region  = "us-east-1"
name_prefix = "reducto-us"
reducto_principal_arns = [
  "arn:aws:iam::731106932034:role/reducto-prod",
  "arn:aws:iam::731106932034:user/modal-prod",
  "arn:aws:iam::731106932034:role/modal-prod"
]
reducto_external_id = "<external-id-from-reducto>"

# regions/eu-central-1/terraform.tfvars
aws_region  = "eu-central-1"
name_prefix = "reducto-eu"
reducto_principal_arns = [
  "arn:aws:iam::731106932034:role/reducto-prod-eu",
  "arn:aws:iam::731106932034:user/modal-prod-eu",
  "arn:aws:iam::731106932034:role/modal-prod-eu"
]
reducto_external_id = "<external-id-from-reducto>"

# regions/ap-southeast-2/terraform.tfvars
aws_region  = "ap-southeast-2"
name_prefix = "reducto-au"
reducto_principal_arns = [
  "arn:aws:iam::731106932034:role/reducto-prod-au",
  "arn:aws:iam::731106932034:user/modal-prod-au",
  "arn:aws:iam::731106932034:role/modal-prod-au"
]
reducto_external_id = "<external-id-from-reducto>"

Apply each region and send Reducto every integration_values output:

cd regions/us-east-1 && terraform init && terraform apply
cd ../eu-central-1 && terraform init && terraform apply
cd ../ap-southeast-2 && terraform init && terraform apply

Each Reducto region then uses the bucket in its own area. For example, a request to https://au.platform.reducto.ai reads and writes your ap-southeast-2 bucket.

Multi-Environment Setup

For organizations with separate AWS accounts for dev/staging/prod, create one directory per environment under examples/multi-environment/, and copy main.tf and terraform.tfvars.example (renamed to terraform.tfvars) into each. The template’s module source (../../../) expects this location:

examples/multi-environment/
├── dev/
│   ├── main.tf
│   └── terraform.tfvars
├── staging/
│   ├── main.tf
│   └── terraform.tfvars
└── prod/
    ├── main.tf
    └── terraform.tfvars

Each environment should use a separate Terraform state file and its own S3 bucket and IAM role. If the environments share the same Reducto org, Reducto can register them as named Hybrid VPC environments instead of separate orgs.

Multiple Buckets for One Reducto Org

For workflows that need client-specific buckets under one Reducto organization, Reducto registers each bucket/role pair as a named environment:

{
  "default_environment": "client-a",
  "environments": {
    "client-a": {
      "region": "us-east-1",
      "bucket": "client-a-reducto-data",
      "role_arn": "arn:aws:iam::987654321098:role/reducto-client-a"
    },
    "client-b": {
      "region": "us-east-1",
      "bucket": "client-b-reducto-data",
      "role_arn": "arn:aws:iam::987654321098:role/reducto-client-b"
    }
  }
}

An environment’s region must belong to the Reducto region you call. For environments in more than one Reducto region, Reducto can set a default per region with us_default_environment, eu_default_environment, and au_default_environment. If your organization has several environments, set a default or name an environment on every request. Without a default, a request that names no environment either fails with an error that lists the available environments or, if Reducto also registered regional storage for your organization, uses that storage. Select the environment on each request. Where the setting goes depends on the endpoint:

Parse
Extract and Split
Upload

{
  "input": "s3://client-b-reducto-data/documents/invoice.pdf",
  "settings": {
    "hybrid_vpc": {
      "environment": "client-b"
    }
  }
}

{
  "input": "s3://client-b-reducto-data/documents/invoice.pdf",
  "parsing": {
    "settings": {
      "hybrid_vpc": {
        "environment": "client-b"
      }
    }
  }
}

/upload has no JSON body, so pass the environment as a query parameter:

POST /upload?hybrid_vpc_environment=client-b

On /extract and /split, a top-level settings.hybrid_vpc is ignored and the request uses your default environment. Put it under parsing.settings.

Reducto reads s3:// inputs with the selected environment’s role, so that role needs s3:GetObject on the input bucket.

Security

ExternalId protection

The ExternalId in the IAM role trust policy prevents confused deputy attacks. Only requests with the correct ExternalId can assume the role.

Role sessions

Reducto assumes the role with the session name reducto-hybrid-vpc and a one-hour session that it renews automatically. CloudTrail in your account records these AssumeRole calls. URLs that Reducto returns for objects in your bucket, such as URL results and images, are presigned with this session. They stop working when the session expires, within one hour, so download them promptly.

Principle of least privilege

The IAM role grants only the permissions necessary for Reducto operations:

s3:GetObject: read documents
s3:PutObject: write results and artifacts
s3:DeleteObject: delete objects, for example when you delete a job
s3:ListBucket, s3:GetBucketLocation: bucket-level access
s3:AbortMultipartUpload, s3:ListMultipartUploadParts, s3:ListBucketMultipartUploads: handle large file uploads

Encryption

The module turns on default encryption with SSE-S3 (AES256). If you switch the bucket to SSE-KMS with a customer managed key, add kms:Decrypt and kms:GenerateDataKey on that key to the role’s policy, and allow the role in the key policy. To have Reducto request that key on every write, send Reducto the key ARN.

Automatic data cleanup

Objects expire automatically based on the lifecycle configuration (default: 1 day), and incomplete multipart uploads are aborted after 1 day. This ensures no long-term data persistence, compliance with retention policies, and automatic cleanup of intermediate artifacts. The rule covers the whole bucket. It also deletes source documents that you store there, and async job results can no longer be retrieved after they expire. Keep documents you want to retain in another bucket.

Get Started

Developer Tools

Core Functions

Workflows and Pipelines

Configurations

Reference

Components

Enterprise Resources

Security and privacy

On-premise Resources

Hybrid VPC — AWS S3

Prerequisites

Reducto regions

Principal ARNs

VPC Endpoint Service Configuration

Setup

Components provisioned

Access Mode

PrivateLink Setup (Optional)

Require PrivateLink

Validation Checklist

Troubleshooting

AccessDenied on sts:AssumeRole

AccessDenied on S3 after the role is assumed

Region cannot be used with this deployment

VPC endpoint service does not exist

Multi-Region Setup

Multi-Environment Setup

Multiple Buckets for One Reducto Org

Security

ExternalId protection

Role sessions

Principle of least privilege

Encryption

Automatic data cleanup

​Prerequisites

​Reducto regions

​Principal ARNs

​VPC Endpoint Service Configuration

​Setup

​Components provisioned

​Access Mode

​PrivateLink Setup (Optional)

​Require PrivateLink

​Validation Checklist

​Troubleshooting

​AccessDenied on sts:AssumeRole

​AccessDenied on S3 after the role is assumed

​Region cannot be used with this deployment

​VPC endpoint service does not exist

​Multi-Region Setup

​Multi-Environment Setup

​Multiple Buckets for One Reducto Org

​Security

​ExternalId protection

​Role sessions

​Principle of least privilege

​Encryption

​Automatic data cleanup

Prerequisites

Reducto regions

Principal ARNs

VPC Endpoint Service Configuration

Setup

Components provisioned

Access Mode

PrivateLink Setup (Optional)

Require PrivateLink

Validation Checklist

Troubleshooting

AccessDenied on sts:AssumeRole

AccessDenied on S3 after the role is assumed

Region cannot be used with this deployment

VPC endpoint service does not exist

Multi-Region Setup

Multi-Environment Setup

Multiple Buckets for One Reducto Org

Security

ExternalId protection

Role sessions

Principle of least privilege

Encryption

Automatic data cleanup