Deploy a self-hosted Braintrust data plane on AWS, GCP, or Azure
AWS
GCP
Azure
Deploy the Braintrust data plane in your AWS account using the Braintrust Terraform module. This is the recommended way to self-host Braintrust on AWS.Braintrust recommends deploying in a dedicated AWS account. AWS enforces account-level Lambda concurrency limits, and Braintrust runs Lambda functions for user-defined functions (scorers and tools) and for the AI proxy that serves their LLM calls. Sharing an account with other workloads can lead to throttling and service disruptions. A dedicated account also aligns with AWS best practices for workload isolation and security.
To test infrastructure provisioning before committing to production-sized resources, use the sandbox example. It uses minimal instance sizes and has deletion protection disabled for easy teardown. It is not suitable for performance or load testing.
In provider.tf, configure your AWS account and region.Supported regions: ap-northeast-1, ap-south-1, ap-southeast-2, ca-central-1, eu-central-1, eu-west-1, eu-west-2, eu-west-3, sa-east-1, us-east-1, us-east-2, and us-west-2. If you require support for a different region, contact Braintrust.
In terraform.tf, set up your remote backend (typically S3 and DynamoDB).
In main.tf, append ?ref= with a version tag to the module source to pin the module version. Use v6.5.0 or later, the minimum recommended version for new deployments:
In main.tf, set enable_ecs_api = true to serve API traffic from ECS.The example configuration does not set this variable, which leaves API traffic on the Lambda path. ECS is the recommended runtime and costs about 90% less than Lambda for high-traffic data planes. To size the ECS services, see Scaling and storage.
In main.tf, customize the remaining Braintrust deployment settings.The defaults are suitable for a large production-sized deployment. Adjust them based on your needs, but keep in mind the hardware requirements.
Two settings fail the apply if they are wrong:
deployment_name must be unique within the same AWS account (max 18 characters). The default is "braintrust", so change it if you have multiple deployments. Resource names (IAM roles, RDS instances, S3 buckets) are prefixed with this value and collide if duplicated.
Brainstore instance types must have local NVMe storage for caching (for example, the c8gd, c5d, m5d, i3, and i4i families). Generic instance types without local storage (t3, m5, c5) are not supported and fail at plan time.
To keep LLM requests and cached completions on your own infrastructure, you can also run the Braintrust Gateway in your data plane. See Deploy the Braintrust Gateway for the prerequisites and values.
Pass the key to Terraform. The recommended approach is to store the license key in AWS Secrets Manager and reference it using a Terraform data source:
data "aws_secretsmanager_secret_version" "brainstore_license" { secret_id = "braintrust/brainstore-license-key"}
Then pass data.aws_secretsmanager_secret_version.brainstore_license.secret_string as the brainstore_license_key value in the module.Alternatively, you can pass the key without storing it in Secrets Manager:
Set TF_VAR_brainstore_license_key=your-key in your environment.
Pass it via command line: terraform apply -var 'brainstore_license_key=your-key'.
Add it to an uncommitted terraform.tfvars or .auto.tfvars file.
Do not commit the license key to your git repository.
The first terraform apply may fail with transient errors such as ASG health check timeouts (while instances are still booting) or Lambda rate limits. Re-running terraform apply resolves these.
This will create all necessary AWS resources including:
Two isolated VPCs:
Main VPC: Hosts Braintrust services (API, database, Redis, Brainstore)
Quarantine VPC: Runs user-defined functions (scorers, tools) in network isolation. This creates ~30 Lambda functions across multiple runtimes. This is required for most production use cases.
ECS services and Lambda functions for the Braintrust API
Public CloudFront endpoint, API Gateway, and an internal Application Load Balancer
EC2 Auto-scaling group for Brainstore
PostgreSQL database, Redis cache, and S3 buckets
KMS key for encryption
With enable_ecs_api = true, CloudFront routes API traffic to the internal load balancer in front of the ECS services. The API Gateway and API Lambda functions are still provisioned and kept warm for rollback. A future module release removes them.
Connect your Braintrust organization to your newly deployed data plane.
Changing your live organization’s API URL can disrupt access for existing users. If you are testing, create a new Braintrust organization for your data plane instead of updating your live environment.
If your deployment is accessed through a VPN or is otherwise on a private network (not accessible from the public internet), enable Data plane is on a private network. This enables Chrome’s Local Network Access permission handling, which is required for browser access to private network resources. When enabled, Chrome will prompt users to grant permission for the Braintrust UI to access your self-hosted data plane. See Grant browser permissions for details.
Select Save.
The UI will automatically test the connection to your new data plane. Verify that the ping to each endpoint is successful.
These options control what the Terraform module provisions in your AWS account, such as deploying into an existing VPC, KMS encryption keys, and resource tags. To tune how a running deployment behaves, including access control, connectivity, scaling, and telemetry, see Configure your deployment.
At least 3 private subnets across different availability zones
At least 1 public subnet
Internet and NAT gateways with properly configured route tables
The module manages its own security groups. To also use an existing quarantine VPC, set existing_quarantine_vpc_id and the corresponding existing_quarantine_private_subnet_*_id variables.
These tags will be applied to all resources including Brainstore EC2 instances, volumes, and ENIs. The deployment name variable automatically prefixes resource names and applies a BraintrustDeploymentName tag across all resources.
Use the custom_tags parameter instead of the AWS provider’s default_tags configuration. Due to a Terraform limitation, default_tags are not applied to resources that use launch templates, such as Brainstore instances.
Terraform module v6.2.0 and later reserve the aws-apn-id tag key, setting it to Braintrust’s AWS Partner Network identifier on every resource and overriding any aws-apn-id value in custom_tags. If your account restricts tag keys with an SCP or IAM policy, allow aws-apn-id, or resource creation and modification will fail.
Important for AWS: Avoid using burstable Redis instances (t-family instances like cache.t4g.micro) in production. These instances use CPU credits that can be exhausted during high-load periods, leading to performance throttling.Instead, use non-burstable instances like cache.r7g.large, cache.r6g.medium, or cache.r5.large for predictable performance. Even if these instances seem oversized initially, they provide consistent performance without the risk of CPU credit exhaustion.
The API Handler and AI Proxy Lambda functions default to 10240 MB (the Lambda maximum). You can reduce these to lower costs in environments with tighter memory quotas, though Braintrust recommends keeping the defaults for production workloads.Which variable matters depends on where API traffic runs:
api_handler_memory_limit applies only while API traffic runs on the Lambda path. With enable_ecs_api = true, size the ECS services instead. See Scaling and storage.
ai_proxy_memory_limit applies even with API traffic on ECS. As of Terraform module v6.5.0, LLM calls from user-authored scorers and tools running in the quarantine environment route through the AI Proxy Lambda.
module "braintrust-data-plane" { source = "github.com/braintrustdata/terraform-aws-braintrust-data-plane" api_handler_memory_limit = 10240 # default, valid range 1–10240 MB ai_proxy_memory_limit = 10240 # default, valid range 1–10240 MB # ... other configuration ...}
postgres_backup_window and postgres_maintenance_window, added in Terraform module v6.5.1, shift the RDS backup and maintenance windows to times that avoid peak traffic for your deployment. They default to "00:00-00:30" for daily backups and "Mon:08:00-Mon:11:00" for weekly maintenance, both in UTC. AWS requires that the two windows not overlap.
The brainstore_wal_footer_version variable controls the WAL footer format written by Brainstore. It defaults to "" (unset) and should not be changed outside of a planned upgrade sequence.
Do not set brainstore_wal_footer_version without following the upgrade guide. Setting it at the same time as a version bump can cause Brainstore nodes still rolling out to fail to read the new WAL format.
When kms_key_arn is configured, all managed S3 buckets (Brainstore, code-bundle, and Lambda responses) enforce blocked_encryption_types = ["NONE"], preventing unencrypted object uploads. This policy is applied automatically as of v4.5.0 — upgrading from an earlier version will include this change in your terraform plan.
As of v4.5.0, the x-bt-use-gateway header is included in the AI Proxy Lambda function URL CORS allowed headers. Browser clients can send this header without triggering a CORS preflight rejection. When the Braintrust Gateway is enabled, requests route through it by default, and x-bt-use-gateway: false sends an individual request straight to the provider instead.
Deploy the Braintrust data plane in your GCP project using the Braintrust Terraform module and Helm chart. This is the recommended way to self-host Braintrust on GCP.
The Braintrust Terraform module contains all the necessary resources for a self-hosted Braintrust data plane. A dedicated Google Cloud project for your Braintrust deployment is recommended but not required.
In provider.tf, configure your Google Cloud project and region.
In backend.tf, set up your remote backend (typically a GCS bucket).
In main.tf, customize the Braintrust deployment settings.The defaults are suitable for a large production-sized deployment. Adjust them based on your needs, but keep in mind the hardware requirements.
Create a helm-values.yaml file for your deployment. Refer to the Helm chart documentation for configuration options.Deploy the Braintrust Helm chart to your cluster:
To keep LLM requests and cached completions on your own infrastructure, you can also run the Braintrust Gateway in your data plane. See Deploy the Braintrust Gateway for the prerequisites and values.
The data plane requires a publicly reachable HTTPS endpoint with a valid TLS certificate. The Helm chart deploys the API as a ClusterIP service. You are expected to provide your own ingress solution that terminates TLS and routes traffic to the braintrust-api service on port 8000.Common approaches:
GCP Application Load Balancer with a Google-managed certificate (requires a custom domain)
GKE Gateway API with cert-manager and Let’s Encrypt
Cloud Run NGINX proxy with a VPC Connector for SSL termination (no custom domain required)
Istio/ASM Gateway - the Helm chart includes native VirtualService support (see virtualService in values.yaml)
Any reverse proxy or load balancer that terminates TLS and forwards HTTP to the API service
After configuring your ingress, save the resulting HTTPS URL. You’ll need it to configure your Braintrust organization.If your ingress uses a private or self-signed certificate, pass the CA bundle to the bt CLI using the --ca-cert <PATH> flag or the BRAINTRUST_CA_CERT environment variable so that bt commands can connect to your data plane.
Connect your Braintrust organization to your newly deployed data plane.
Changing your live organization’s API URL can disrupt access for existing users. If you are testing, create a new Braintrust organization for your data plane instead of updating your live environment.
If your deployment is accessed through a VPN or is otherwise on a private network (not accessible from the public internet), enable Data plane is on a private network. This enables Chrome’s Local Network Access permission handling, which is required for browser access to private network resources. When enabled, Chrome will prompt users to grant permission for the Braintrust UI to access your self-hosted data plane. See Grant browser permissions for details.
Select Save.
The UI will automatically test the connection to your new data plane. Verify that the ping to each endpoint is successful.
The Terraform module automatically configures Workload Identity for your GKE cluster and creates two service accounts with the following IAM grants.The module creates two GCS buckets:
Brainstore bucket (<deployment_name>-brainstore-*): Brainstore data storage.
API bucket (<deployment_name>-api-*): Contains two storage paths — code-bundle/ (API layer writes) and brainstore-cache/ (ephemeral Brainstore cache, automatically deleted after 1 day by a GCS lifecycle rule).
The brainstore-cache/ objects are ephemeral and managed automatically. Operators do not need to manage or back up this data.
Choose one of the following authentication methods for the API service:
Native GCS auth (recommended)
Native GCS authentication uses the @google-cloud/storage SDK and Workload Identity. This is the recommended approach for enhanced security as it eliminates the need to manage service account keys.Requirements:
Helm chart version 3.1.0 or later
Workload Identity configured (automatic with Terraform module)
Kubernetes secrets:For native GCS authentication, create secrets without GCS credentials:
Refer to the Terraform outputs for the connection strings. The Brainstore license key can be found at Settings > Data plane. Only organization owners can access this page.
Helm configuration:In your Helm values file, enable native GCS authentication and configure the Google service account:
The Terraform module outputs the service account email as braintrust_service_account. Run terraform output braintrust_service_account to get the full email address.The enableGcsAuth setting defaults to false for backwards compatibility. Contact Braintrust if you want to enable native GCS authentication by default.
S3 compatibility mode (legacy)
S3 compatibility mode uses HMAC keys to access GCS through the S3-compatible API. This is the legacy authentication method.Kubernetes secrets:For S3 compatibility mode, include HMAC credentials:
Refer to the Terraform outputs for the connection strings. The Brainstore license key can be found at Settings > Data plane. Only organization owners can access this page.
Helm configuration:In your Helm values file, ensure enableGcsAuth is not set or set to false. Do not configure googleServiceAccount when using HMAC keys:
api: enableGcsAuth: false # or omit this line
Helm chart v5.0.1+ automatically sets AWS_REQUEST_CHECKSUM_CALCULATION and AWS_RESPONSE_CHECKSUM_VALIDATION to WHEN_REQUIRED when enableGcsAuth is disabled. This ensures AWS SDK compatibility with the GCS S3-compatible endpoint. No additional configuration is required.
When api.enableGcsAuth: true, the API service uses the @google-cloud/storage SDK and applies the SDK’s default retry policy for transient GCS errors. To override these defaults, set any of the following environment variables on the API container. If none are set, the SDK defaults apply. See Cloud Storage retry strategy for the upstream defaults and the Node.js client behavior.
Environment variable
Type
Description
GCS_RETRY_OPTIONS_AUTO_RETRY
boolean
Enable or disable automatic retries.
GCS_RETRY_OPTIONS_MAX_RETRIES
integer
Maximum number of retry attempts per request.
GCS_RETRY_OPTIONS_RETRY_DELAY_MULTIPLIER
float
Exponential backoff multiplier between retries.
GCS_RETRY_OPTIONS_TOTAL_TIMEOUT
integer (seconds)
Total time allowed across all retries for a request.
GCS_RETRY_OPTIONS_MAX_RETRY_DELAY
integer (seconds)
Maximum delay between consecutive retries.
GCS_RETRY_OPTIONS_IDEMPOTENCY_STRATEGY
string
One of retry-always, retry-conditional, or retry-never.
Set these through api.extraEnvVars in your Helm values file:
Requires data plane v2.0.0 or later. These environment variables are read only when api.enableGcsAuth: true and have no effect in S3 compatibility mode. They apply to the API service only. Brainstore manages its GCS client independently.
The Terraform module outputs the service account email as brainstore_service_account. Run terraform output brainstore_service_account to get the full email address.
These options control what the Terraform module and Helm chart provision, such as cloud identity, resource labels, and cluster IP ranges. To tune how a running deployment behaves, including access control, connectivity, scaling, and telemetry, see Configure your deployment.
If Brainstore needs to access GCS buckets or other GCP resources in another project that are restricted to a specific service account identity, use brainstore_impersonation_targets to grant the Brainstore Kubernetes service account the ability to impersonate one or more Google Cloud service accounts.
This grants roles/iam.serviceAccountTokenCreator on each target service account to the Brainstore Kubernetes service account, enabling Brainstore to generate short-lived tokens and act as those accounts. Values must use the full resource name format projects/{project_id}/serviceAccounts/{service_account_email}, not bare email addresses. The default is [] (no impersonation).
This variable is only needed if you are not separately granting the Brainstore service account IAM access to the target accounts. The Terraform executor must have roles/iam.serviceAccountAdmin or roles/resourcemanager.projectIamAdmin on each target service account (or equivalent project-level permissions). Target service accounts must also exist before running terraform apply.
Labels are applied to Cloud SQL, Memorystore Redis, Cloud Storage buckets, the GKE cluster, and the KMS key. The module’s built-in braintrustdeploymentname label is always preserved when merged with your custom labels.
Keys must start with a lowercase letter and contain only lowercase letters, numbers, underscores, or dashes (max 63 characters). Values must contain only lowercase letters, numbers, underscores, or dashes, do not require a leading letter, and may be empty (max 63 characters). Maximum 63 custom labels.
When the module creates the VPC (create_vpc = true), Cloud SQL and Memorystore connect over a Private Service Access range. By default, the module allocates a /16 range and lets Google pick the starting address. To avoid overlap with existing VPCs, peering connections, or corporate networks, use private_service_access_prefix_length and private_service_access_address:
private_service_access_prefix_length accepts a value from 8 through 24 (default 16). Lower prefix lengths create larger ranges. Smaller ranges work, but reduce future expansion headroom, so choose the size with your Braintrust architecture team.
private_service_access_address sets the starting address of the range. If unset, Google selects an available range.
Choose these values before your first deployment. Changing the Private Service Access range later can require rebuilding Cloud SQL, Memorystore, and other dependent resources.
To control the secondary IP ranges that the GKE cluster uses for Pods and Services, set either a CIDR block or the name of a pre-existing secondary range. By default, GKE allocates these ranges automatically.
module "braintrust_data_plane" { source = "github.com/braintrustdata/terraform-google-braintrust-data-plane" # Option A: let the module create the ranges from CIDR blocks gke_pods_ipv4_cidr_block = "10.20.0.0/20" gke_services_ipv4_cidr_block = "10.30.0.0/22" # Option B: reference secondary ranges that already exist on the subnet # gke_pods_secondary_range_name = "braintrust-pods" # gke_services_secondary_range_name = "braintrust-services" # ... other configuration ...}
The CIDR variables accept a full CIDR ("10.20.0.0/20") or a netmask size only ("/20").
The secondary range name variables reference ranges that already exist on the subnet. They do not create the ranges.
For each range type (Pods and Services), set the CIDR block or the secondary range name, but not both. The module rejects setting both forms for the same range type.
Most deployments should leave the Service range unset unless they intentionally need a custom Service CIDR.
Choose these values before your first deployment. Changing GKE secondary ranges later requires recreating the cluster.
When the module creates the VPC (create_vpc = true), you can enable VPC flow logs on the subnet by setting subnet_flow_logs_config. Each field is optional and takes the default listed below when omitted.
subnet_flow_logs_config can only be set when create_vpc = true. To enable flow logs on a subnet you supply yourself, configure them directly on that subnet.
To capture access logs for the Brainstore or API GCS buckets, set gcs_brainstore_logging_config, gcs_api_logging_config, or both. Each variable points access logs at an existing log bucket. Both default to null (disabled).
Name of an existing GCS bucket to write access logs to. Must be non-empty.
log_object_prefix
no
Prefix applied to log object names. Defaults to the name of the bucket being logged. Set it to group logs under a shorter or folder-style prefix instead.
Before you apply this configuration, confirm the log bucket meets Cloud Storage’s requirements (if it doesn’t, logging fails):
The log bucket already exists. Terraform does not create it.
The log bucket and the bucket it receives logs for are in the same location (region, dual-region, or multi-region).
Both are in the same organization, or the same project if there is no organization.
Both are in the same VPC Service Controls perimeter, if you use them.
The cloud-storage-analytics@google.com group has the roles/storage.objectCreator role on the log bucket, which is what lets Cloud Storage write the logs.
Deploy the Braintrust data plane in your Azure subscription using the Braintrust Terraform module and Helm chart. This is the recommended way to self-host Braintrust on Azure.
Requirements: Terraform >= 1.10.0 and the azurerm provider ~> 4.0 are required. If you have an existing deployment using azurerm 3.x, run terraform init -upgrade before applying and review the azurerm v4 upgrade guide.
The Braintrust Terraform module contains all the necessary resources for a self-hosted Braintrust data plane. A dedicated Azure subscription for your Braintrust deployment is recommended but not required.
In provider.tf, configure your Azure subscription and tenant details.
In terraform.tf, set up your remote backend (typically Azure Blob Storage).
In main.tf, customize the Braintrust deployment settings.The defaults are suitable for a large production-sized deployment. Adjust them based on your needs, but keep in mind the hardware requirements.The module provisions two AKS node pools:
brainstore pool (aks_brainstore_pool_vm_size): runs Brainstore pods. Must be a VM SKU with local NVMe SSD (e.g. Standard_D32ds_v6). The Azure Container Storage extension is automatically installed to configure RAID0 across the local disks.
services pool (aks_services_pool_vm_size): runs API and other application pods. Does not require local SSD (e.g. Standard_D16s_v6).
Initially set enable_front_door = false in main.tf. You’ll enable this later after configuring the load balancer.
The Terraform module configures PostgreSQL extensions (pg_cron, pg_partman) and sets cron.database_name to the Braintrust database. These are static parameters that require a server restart to take effect. The Terraform provider is configured to not restart the server automatically, since automatic restarts on configuration changes could cause unintended downtime in production.After your first terraform apply, restart the PostgreSQL server before proceeding:
az postgres flexible-server restart \ --resource-group <resource-group-name> \ --name <postgres-database-server-name>
You can find the resource group and server name in your Terraform outputs.
This step is only required on the initial deployment. Subsequent terraform apply runs do not require a restart unless you modify the PostgreSQL extension configuration.
To keep LLM requests and cached completions on your own infrastructure, you can also run the Braintrust Gateway in your data plane. See Deploy the Braintrust Gateway for the prerequisites and values.
In the Azure Portal, find the private link service named <deployment>-aks-api-pls and manually approve it.
This manual approval step is an Azure platform requirement — Front Door cannot automatically approve private link connections to resources in a different subscription or tenant. The deployment will appear to succeed but Front Door traffic will not flow until the connection is approved.
Front Door deployment takes up to 45 minutes after the Terraform apply completes. Wait for the deployment to finish before proceeding.
Connect your Braintrust organization to your newly deployed data plane.
Changing your live organization’s API URL can disrupt access for existing users. If you are testing, create a new Braintrust organization for your data plane instead of updating your live environment.
If your deployment is accessed through a VPN or is otherwise on a private network (not accessible from the public internet), enable Data plane is on a private network. This enables Chrome’s Local Network Access permission handling, which is required for browser access to private network resources. When enabled, Chrome will prompt users to grant permission for the Braintrust UI to access your self-hosted data plane. See Grant browser permissions for details.
Select Save.
The UI will automatically test the connection to your new data plane. Verify that the ping to each endpoint is successful.
AWS
GCP
Azure