Skip to main content

Documentation Index

Fetch the complete documentation index at: https://backstage.spotify.com/docs/llms.txt

Use this file to discover all available pages before exploring further.

The HTTP integration plugin for Soundcheck supports the collection of the facts from HTTP endpoints. The HTTP fact collector is intentionally simple and should be used for simple use cases. It doesn’t support rate limiting, pagination and any custom logic besides a simple transformation of the HTTP response body. In case you have a more complex use-case, consider implementing a custom integration.

​
Prerequisites

​
Configure HTTP secrets in Backstage

If your endpoints use a token or an API key authentication (or any other secret data), you may configure the secrets at the root level of app-config.yaml. Here’s an example configuration: 
soundcheck:
  collectors:
    http:
      secrets:
        - key: grafanaApiToken
          value: ${API_TOKEN}
Referencing a secret in an HTTP header template (e.g. Authorization: Bearer {{ secrets.grafanaApiToken }}) will send the secret value to the target server. Make sure the request URL points to a trusted endpoint before including any secrets in the request.

​
Configure reusable config values in Backstage

If your endpoints need non-secret configurable values you can reuse across templates, you may configure them at the root level of app-config.yaml. Here’s an example configuration: 
soundcheck:
  collectors:
    http:
      config:
        - key: title
          value: Soundcheck
The config template variable previously exposed the entire Backstage app-config.yaml (e.g. {{ config.app.title }}). It now only exposes the key-value pairs defined under soundcheck.collectors.http.config. Existing templates that reference other app-config.yaml fields will need to be migrated to use explicitly declared config entries.

​
Add the HttpFactCollector to Soundcheck

First, add the @spotify/backstage-plugin-soundcheck-backend-module-http package: 
yarn workspace backend add @spotify/backstage-plugin-soundcheck-backend-module-http
Then add the following to your packages/backend/src/index.ts file:
packages/backend/src/index.ts
const backend = createBackend();

backend.add(import('@spotify/backstage-plugin-soundcheck-backend'));
backend.add(import('@spotify/backstage-plugin-soundcheck-backend-module-http'));
// ...

backend.start();
Consult the Soundcheck Backend documentation for additional details on setting up the Soundcheck backend.

​
Plugin Configuration

The collection of HTTP facts is driven by configuration. To learn more about the configuration, consult the Defining HTTP Fact Collectors section. HTTP Fact Collector can be configured via YAML or No-Code UI. If you configure it via both YAML and No-Code UI, the configurations will be merged. It’s preferable to choose a single source for the Fact Collectors configuration (either No-Code UI or YAML) to avoid confusing merge results.

​
No-Code UI Configuration Option

  1. Make sure the prerequisite Configure HTTP secrets in Backstage is completed if your endpoints require any secret data.
  2. Make sure the prerequisite Configure reusable config values in Backstage is completed if your endpoints use reusable config values in templates.
  3. To enable the HTTP Integration, go to Soundcheck > Integrations > HTTP and click the Configure button. To learn more about the No-Code UI config, see the Configuring a fact collector (integration) via the no-code UI.
HTTP Integration

​
YAML Configuration Option

  1. Create a http-facts-collectors.yaml file in the root of your Backstage repository and fill in all your HTTP Fact Collectors. A simple example HTTP fact collector is listed below.
- factName: grafana_dashboard
  request:
    method: GET
    url: http://grafana.com/api/dashboards/uid/{{ entity.metadata.annotations['grafana.com/dashboard-uid'] }}
    headers:
      Accept: application/json
      Content-Type: application/json
      Authorization: Bearer {{ secrets.grafanaApiToken }}
  response:
    transform: dashboard
    emptyOnStatuses: [404]
    schema: |
      {
        "title": "Grafana Dashboard",
        "description": "Grafana Dashboard",
        "type": "object",
        "properties": {
          "id": {
            "type": "string"
          },
          "uid": {
            "type": "string"
          },
          "title": {
            "type": "string"
          },
          "tags": {
            "type": "array",
            "items": {
              "type": "string"
            }
          }
        }
      }
  retries:
    maxRetries: 3
    retryDelay: 5000
    retryOnStatuses: [408, 502, 503, 504]
- factName: owners_failed_check_results
  filter:
    kind: 'Component'
    spec.type: 'service'
  request:
    targetPluginId: soundcheck
    method: GET
    url: '{{ targetPluginBaseUrl }}/results?entityRef={{ entity.relations | where: "type", "ownedBy" | first | map: "targetRef" }}'
    headers:
      Authorization: Bearer {{ targetPluginAuthToken }}
  response:
    transform: results[state='failed']
    emptyOnStatuses: [404]
    schema: |
      {
        "title": "Check Results",
        "description": "Soundcheck Check Results",
        "type": "object",
        "properties": {
          "results": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "entityRef": {
                  "type": "string"
                },
                "checkId": {
                  "type": "string"
                },
                "scope": {
                  "type": "string"
                },
                "state": {
                  "type": "string"
                },
                "timestamp": {
                  "type": "string"
                }
              }
            }
          }
        }
      }
Note: this file will be loaded at runtime along with the rest of your Backstage configuration files, so make sure it’s available in deployed environments in the same way as your app-config.yaml files.
  1. Add a Soundcheck collects field to the app-config.yaml and reference the newly created http-facts-collectors.yaml file.
# app-config.yaml
soundcheck:
  collectors:
    http:
      collects:
        $include: ./http-facts-collectors.yaml

​
Defining HTTP Fact Collectors

This section describes the data shape and semantics of HTTP Fact Collectors.

​
Shape Of HTTP Fact Collector

The following is an example of HTTP Fact Collector YAML configuration: 
soundcheck:
  collectors:
    http:
      collects:
        - factName: grafana_dashboard
          request:
            method: GET
            url: http://grafana.com/api/dashboards/uid/{{ entity.metadata.annotations['grafana.com/dashboard-uid'] }}
            headers:
              Accept: application/json
              Content-Type: application/json
              Authorization: Bearer {{ secrets.grafanaApiToken }}
          response:
            transform: dashboard
            emptyOnStatuses: [404]
            schema: |
              {
                "title": "Grafana Dashboard",
                "description": "Grafana Dashboard",
                "type": "object",
                "properties": {
                  "id": {
                    "type": "string"
                  },
                  "uid": {
                    "type": "string"
                  },
                  "title": {
                    "type": "string"
                  },
                  "tags": {
                    "type": "array",
                    "items": {
                      "type": "string"
                    }
                  }
                }
              }
          retries:
            maxRetries: 3
            retryDelay: 5000
            retryOnStatuses: [408, 502, 503, 504]
          filter:
            kind: 'Component'
            spec.type: 'service'
          cache: false
Below are the details for each field.

​
factName [required]

The name of the fact to be collected.
  • Minimum length of 1
  • Maximum length of 100
  • Alphanumeric with single separator instances of periods, dashes, underscores, or forward slashes

​
request [required]

HTTP request configuration:
  • targetPluginId [optional] - Optional, provide if calling a plugin’s HTTP endpoint. If provided, liquid template tags {{ targetPluginBaseUrl }} and {{ targetPluginAuthToken }} will be populated with the plugin’s data.
  • method [optional] - HTTP method (GET or POST). Optional, if not provided the method will be defaulted to GET.
  • url [required] - HTTP request URL. Supports liquid templating. The following variables are available for use in templates:
    • entity - The entity the fact is collected for. Usage example: {{ entity.metadata.name }}
    • config - The reusable config values. Usage example: {{ config.title }}
    • targetPluginBaseUrl - Base URL of the plugin with ID provided in targetPluginId. Usage example: {{ targetPluginBaseUrl }}
  • headers [optional] - HTTP request headers. Header keys and values support liquid templating. The following variables are available for use in templates:
    • entity - The entity the fact is collected for. Usage example: {{ entity.metadata.name }}
    • config - The reusable config values. Usage example: {{ config.title }}
    • secrets - The HTTP secrets configuration. Usage example: {{ secrets.grafanaApiToken }}
    • targetPluginAuthToken - Auth token that can be used for authenticating calls towards plugin with ID provided in targetPluginId. Usage example: {{ targetPluginAuthToken }}
  • body [optional] - HTTP request body. Supports liquid templating. The following variables are available for use in templates:
    • entity - The entity the fact is collected for. Usage example: {{ entity.metadata.name }}
    • config - The reusable config values. Usage example: {{ config.title }}

​
response [optional]

HTTP response configuration:
  • transform [optional] - JSONata expression to transform the HTTP response body. Optional, if not provided an unmodified response body will be used as the fact data.
  • emptyOnStatuses [optional] - A list of HTTP status codes to return an empty response on (when received, an empty fact will be collected instead of throwing a collection error). Optional, if not provided all non-OK responses will result in collection errors.
  • schema [optional] - A JSON schema describing the HTTP response body. Optional, primarily used for Check Creation No-Code UI autocomplete. Not used for any validations.

​
retries [optional]

HTTP request retries configuration:
  • maxRetries [optional] - Max number of retries. Optional, if not provided failed requests won’t be retried.
  • retryDelay [optional] - Retry delay in milliseconds. Optional, if not provided failed requests will be retried immediately.
  • retryOnStatuses [optional] - A list of HTTP status codes to retry the request on. Optional, if not provided all failed requests will be retried.

​
frequency [optional]

The frequency at which the fact collection should be executed. Possible values are either a cron expression { cron: ... } or HumanDuration. If provided, it overrides the default frequency provided at the top level. If not provided, it defaults to the frequency provided at the top level. If neither collector’s frequency nor default frequency is provided, the fact will only be collected on demand. Example: 
frequency:
  minutes: 10

​
initialDelay [optional]

The amount of time that should pass before the first invocation happens. Possible values are either a cron expression { cron: ... } or HumanDuration. Example: 
initialDelay:
  seconds: 30

​
batchSize [optional]

The number of entities to collect facts for at once. Optional, the default value is 1. Example: 
batchSize: 100

​
filter [optional]

A filter specifying which entities to collect the specified facts for. Matches the filter format used by the Catalog API. If provided, it overrides the default filter provided at the top level. If not provided, it defaults to the filter provided at the top level. If neither collector’s filter nor default filter is provided, the fact will be collected for all entities. See filters for more details.

​
exclude [optional]

Entities matching this filter will be skipped during the fact collection process. Can be used in combination with filter. Matches the filter format used by the Catalog API. 
filter:
  - kind: component
exclude:
  - spec.type: documentation

​
cache [optional]

If the collected facts should be cached, and if so for how long. Possible values are either true or false or a nested { duration: HumanDuration } field. If provided it, overrides the default cache config provided at the top level. If not provided, it defaults to the cache config provided at the top level. If neither collector’s cache nor default cache config is provided, the fact will not be cached. Example: 
cache:
  duration:
    hours: 24

​
Rate Limiting

This fact collector doesn’t support rate limiting. If your target HTTP endpoint has rate limits it’s recommended to implement a custom fact collector that will respect the configurable limits. Check out some recommendations on how to avoid exceeding API rate limits.

​
Shape of HTTP Fact

The shape of HTTP Fact is based on the Fact Schema. The following is an example of the collected HTTP fact: 
factRef: http:default/grafana_dashboard
entityRef: component:default/queue-proxy
scope: default
timestamp: 2025-01-09T15:50+00Z
data:
  id: null
  uid: null
  title: Production Overview
  tags:
    - production
  timezone: browser
  schemaVersion: 16
  refresh: 25s
The resulting fact data will contain the HTTP response.

​
Shape of HTTP Fact Check

The shape of HTTP Fact Check matches the Shape of a Fact Check. The following is an example of the HTTP fact check: 
soundcheck:
  checks:
    - id: has-production-dashboard-tag
      name: Grafana dashboard has production tag
      ownerEntityRef: group:default/toast-infra
      description: Grafana dashboard has production tag
      filter:
        kind: Component
        spec.type: service
      rule:
        factRef: http:default/grafana_dashboard
        path: $.tags
        operator: any:equal
        value: production
      passedMessage: Success! Grafana dashboard has production tag
      failedMessage: Failure! Grafana dashboard doesn't have production tag