Skip to main content

Amazon Bedrock

Overview​

Amazon Bedrock provides both LLM capabilities and advanced re-ranking models that can be used to improve the relevance of generated content and reduce inconsistencies from document indexing.

Important: Model Compatibility Requirements​

Before configuring any Bedrock model in AI Gateway, verify these requirements:

  1. On-Demand Throughput Support - Not all models support this mode
  2. Regional Availability - Model must be available in your AWS region
  3. Access Permissions - You must have granted access in AWS Console

Step-by-Step Bedrock Setup​

1. Check Model Compatibility​

Navigate to AWS Console → Amazon Bedrock → Model access

Look for these indicators:

  • "On-demand throughput: Supported"
  • Available in your current AWS region
  • Access status shows "Available to request" or "Access granted"

2. Request Model Access​

  1. Click "Modify model access"
  2. Select your desired models
  3. Click "Request model access"
  4. Wait for approval (can take minutes to hours)
  5. Verify "Access granted" status appears

These models are confirmed to work with on-demand throughput:

Language Models:

ai:
  models:
    languageModel: amazon-bedrock|anthropic.claude-sonnet-4-5-20250929-v1:0
    # OR
    languageModel: amazon-bedrock|anthropic.claude-sonnet-4-20250514-v1:0

Re-ranking Models:

ai:
  models:
    rerankingModel: amazon-bedrock|cohere.rerank-v3-5:0

Configuration​

To use Amazon Bedrock configure it in the Config Manager under the AI Gateway section:

ai:
  providers:
    amazonBedrock:
      region: us-east-1
      accessKeyId: your-access-key-id
      secretAccessKey: your-secret-access-key
      sessionToken: your-session-token # Optional, if using temporary credentials
      baseURL: https://ai-gateway.example.com/bedrock # Optional, to point at a gateway
      headers: # Optional, e.g. for authentication with gateway
        - key: X-Custom-Header
          value: custom-value

For Language Models​

Set the default language model:

ai:
  models:
    languageModel: amazon-bedrock|anthropic.claude-sonnet-4-5-20250929-v1:0

For Re-ranking​

Optionally configure Bedrock for re-ranking:

ai:
  models:
    rerankingModel: amazon-bedrock|cohere.rerank-v3-5:0

Common Error Messages & Solutions​

Error: Invocation of model ID [model] with on-demand throughput isn't supported

Cause: This model doesn't support on-demand throughput Solution: Choose a different model from our recommended list above

Error: You don't have access to the model with the specified model ID

Cause: Model access not granted in AWS Bedrock console Solution:

  1. Go to AWS Console → Bedrock → Model access
  2. Click "Modify model access"
  3. Select the model and request access
  4. Wait for approval confirmation

Error: Model capacity exceeded or 429 errors

Cause: The model has reached its managed capacity limit Solution:

  • Wait a few minutes and retry
  • Switch to a different model temporarily
  • Consider provisioned throughput for high-volume usage

Testing Your Configuration​

After setup, test your configuration with a simple request to verify everything works before production use.

Regional Availability Notes​

Model availability varies by AWS region. If you encounter regional errors:

  1. Check the AWS Bedrock User Guide for model availability by region
  2. Consider switching to a region where your desired model is available
  3. Update your AWS credentials/configuration accordingly