Skip to main content

AI Gateway

Overview​

What is AI Gateway?​

The AI Gateway is a centralized plugin for configuring and managing AI capabilities in Backstage. It allows you to set up and manage various AI models, providers, and re-ranking capabilities that can be used across different plugins, such as AiKA.

Supported Providers​

AI Gateway currently supports the following providers

  • AWS Bedrock
  • OpenAI
  • Cohere
  • Ollama

Installation​

Enabling AI Gateway in Portal​

You can enable or disable the AI Gateway at any time through the Config Manager.

Configuring your integrations​

We provide support for a variety of AI providers and models. You can configure these in the Config Manager under the AI Gateway section.

Screenshot detailing AiKA Config

tip

You can have one or multiple providers configured at the same time. Plugins will allow you to select which provider and model to use, or you can set a default provider to use across all plugins.

ai:
  providers:
    amazonBedrock:
      region: us-east-1
      accessKeyId: your-access-key-id
      secretAccessKey: your-secret-access-key
      sessionToken: your-session-token # Optional, if using temporary credentials
    cohere:
      apiKey: your-cohere-api-key
    openai:
      apiKey: your-openai-api-key
    ollama:
      baseURL: http://localhost:11434

Configuring a Default Provider​

You can set a default provider that will be used across all plugins that support AI Gateway. The model fields are formatted as provider|model.

ai:
  models:
    languageModel: openai|gpt-4
    rerankingModel: cohere|rerank-001
    embedding: openai|text-embedding-ada-002
    imageModel: ollama|llama3