> ## Documentation Index
> Fetch the complete documentation index at: https://backstage.spotify.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Amazon Bedrock

> Set up Amazon Bedrock as an AI Gateway provider for LLM and re-ranking models, including model compatibility checks and region-specific access.

## Overview

Amazon Bedrock provides both LLM capabilities and advanced re-ranking models that can be used to improve the relevance of generated content and reduce inconsistencies from document indexing.

### Important: Model Compatibility Requirements

**Before configuring any Bedrock model in AI Gateway, verify these requirements:**

1. **On-Demand Throughput Support** - Not all models support this mode
2. **Regional Availability** - Model must be available in your AWS region
3. **Access Permissions** - You must have granted access in AWS Console

## Step-by-Step Bedrock Setup

### 1. Check Model Compatibility

Navigate to AWS Console → Amazon Bedrock → Model access

**Look for these indicators:**

* "On-demand throughput: Supported"
* Available in your current AWS region
* Access status shows "Available to request" or "Access granted"

### 2. Request Model Access

1. Click **"Modify model access"**
2. Select your desired models
3. Click **"Request model access"**
4. Wait for approval (can take minutes to hours)
5. Verify "Access granted" status appears

### 3. Recommended Models (Verified Compatible)

These models are confirmed to work with on-demand throughput:

**Language Models:**

```
Language Model: amazon-bedrock|anthropic.claude-sonnet-4-5-20250929-v1:0
# OR
Language Model: amazon-bedrock|anthropic.claude-sonnet-4-20250514-v1:0
```

**Re-ranking Models:**

```
Reranking Model: amazon-bedrock|cohere.rerank-v3-5:0
```

## Configuration

To use Amazon Bedrock, configure it in the Plugin settings page under the `AI Gateway` section:

| Field                 | Description                                    | Example                                           |
| --------------------- | ---------------------------------------------- | ------------------------------------------------- |
| **Region**            | AWS region for Bedrock                         | `us-east-1`                                       |
| **Access Key ID**     | Your AWS access key ID                         | `your-access-key-id`                              |
| **Secret Access Key** | Your AWS secret access key                     | `your-secret-access-key`                          |
| **Session Token**     | Optional, if using temporary credentials       | `your-session-token`                              |
| **Base URL**          | Optional, to point at a gateway                | `https://ai-gateway.example.com/bedrock`          |
| **Headers**           | Optional, e.g. for authentication with gateway | Key: `X-Custom-Header`<br />Value: `custom-value` |

### For Language Models

Set the default language model:

```
Language Model: amazon-bedrock|anthropic.claude-sonnet-4-5-20250929-v1:0
```

### For Re-ranking

Optionally configure Bedrock for re-ranking:

```
Reranking Model: amazon-bedrock|cohere.rerank-v3-5:0
```

### Common Error Messages & Solutions

**Error:** `Invocation of model ID [model] with on-demand throughput isn't supported`

**Cause:** This model doesn't support on-demand throughput
**Solution:** Choose a different model from our recommended list above

**Error:** `You don't have access to the model with the specified model ID`

**Cause:** Model access not granted in AWS Bedrock console
**Solution:**

1. Go to AWS Console → Bedrock → Model access
2. Click "Modify model access"
3. Select the model and request access
4. Wait for approval confirmation

**Error:** `Model capacity exceeded` or `429 errors`

**Cause:** The model has reached its managed capacity limit
**Solution:**

* Wait a few minutes and retry
* Switch to a different model temporarily
* Consider provisioned throughput for high-volume usage

### Testing Your Configuration

After setup, test your configuration with a simple request to verify everything works before production use.

### Regional Availability Notes

Model availability varies by AWS region. If you encounter regional errors:

1. Check the [AWS Bedrock User Guide](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html) for model availability by region
2. Consider switching to a region where your desired model is available
3. Update your AWS credentials/configuration accordingly