Skip to main content

Amazon Bedrock

Overview​

Amazon Bedrock provides both LLM provider capabilities and advanced re-ranking models that can be used with AiKA to improve the relevance of generated content and reduce inconsistencies from document indexing.

Important: Model Compatibility Requirements​

Before configuring any Bedrock model in AiKA, verify these requirements:

  1. On-Demand Throughput Support - Not all models support this mode
  2. Regional Availability - Model must be available in your AWS region
  3. Access Permissions - You must have granted access in AWS Console

Step-by-Step Bedrock Setup​

1. Check Model Compatibility​

Navigate to AWS Console → Amazon Bedrock → Model access

Look for these indicators:

  • "On-demand throughput: Supported"
  • Available in your current AWS region
  • Access status shows "Available to request" or "Access granted"

2. Request Model Access​

  1. Click "Modify model access"
  2. Select your desired models
  3. Click "Request model access"
  4. Wait for approval (can take minutes to hours)
  5. Verify "Access granted" status appears

These models are confirmed to work with on-demand throughput:

Language Models:

aika:
 models:
   language: amazon-bedrock|claude-3-5-sonnet-20241022-v2:0
   # OR
   language: amazon-bedrock|claude-3-haiku-20240307-v1:0
   # OR
   language: amazon-bedrock|titan-text-express-v1

**Reranking Models:**
yaml
aika:
 models:
   reranking: amazon-bedrock|cohere.rerank-v3-5:0

Configuration​

To use Amazon Bedrock for LLM provider capabilities in AiKA, set the following in the config manager. The language field will be formatted as provider|model:

aika:
  models:
    language: amazon-bedrock|claude-3-5-sonnet-20241022-v2:0

To optionally use Amazon Bedrock for re-ranking in AiKA, set the following in your configuration:

aika:
  models:
    reranking: amazon-bedrock|cohere.rerank-v3-5:0

Common Error Messages & Solutions​

Error: Invocation of model ID [model] with on-demand throughput isn't supported

Cause: This model doesn't support on-demand throughput Solution: Choose a different model from our recommended list above

Error: You don't have access to the model with the specified model ID

Cause: Model access not granted in AWS Bedrock console Solution:

  1. Go to AWS Console → Bedrock → Model access
  2. Click "Modify model access"
  3. Select the model and request access
  4. Wait for approval confirmation

Error: Model capacity exceeded or 429 errors

Cause: The model has reached its managed capacity limit Solution:

  • Wait a few minutes and retry
  • Switch to a different model temporarily
  • Consider provisioned throughput for high-volume usage

Testing Your Configuration​

After setup, test your configuration with a simple request to verify everything works before production use.

Regional Availability Notes​

Model availability varies by AWS region. If you encounter regional errors:

  1. Check the AWS Bedrock User Guide for model availability by region
  2. Consider switching to a region where your desired model is available
  3. Update your AWS credentials/configuration accordingly