Skip to main content

HTTP

The HTTP integration plugin for Soundcheck supports the collection of the facts from HTTP endpoints. The HTTP fact collector is intentionally simple and should be used for simple use cases. It doesn't support rate limiting, pagination and any custom logic besides a simple transformation of the HTTP response body. In case you have a more complex use-case, consider implementing a custom integration.

Prerequisites​

Configure HTTP secrets in Backstage​

If your endpoints use a token or an API key authentication (or any other secret data), you may configure the secrets at the root level of app-config.yaml. Here's an example configuration:

soundcheck:
  collectors:
    http:
      secrets:
        grafanaApiToken: ${API_TOKEN}

Add the HttpFactCollector to Soundcheck​

First, add the @spotify/backstage-plugin-soundcheck-backend-module-http package:

yarn workspace backend add @spotify/backstage-plugin-soundcheck-backend-module-http

Then add the following to your packages/backend/src/index.ts file:

packages/backend/src/index.ts
const backend = createBackend();

backend.add(import('@spotify/backstage-plugin-soundcheck-backend'));
backend.add(import('@spotify/backstage-plugin-soundcheck-backend-module-http'));
// ...

backend.start();

Consult the Soundcheck Backend documentation for additional details on setting up the Soundcheck backend.

Plugin Configuration​

The collection of HTTP facts is driven by configuration. To learn more about the configuration, consult the Defining HTTP Fact Collectors section.

HTTP Fact Collector can be configured via YAML or No-Code UI. If you configure it via both YAML and No-Code UI, the configurations will be merged. It's preferable to choose a single source for the Fact Collectors configuration (either No-Code UI or YAML) to avoid confusing merge results.

No-Code UI Configuration Option​

  1. Make sure the prerequisite Configure HTTP secrets in Backstage is completed if your endpoints require any secret data.

  2. To enable the HTTP Integration, go to Soundcheck > Integrations > HTTP and click the Configure button. To learn more about the No-Code UI config, see the Configuring a fact collector (integration) via the no-code UI.

HTTP Integration

YAML Configuration Option​

  1. Create a http-facts-collectors.yaml file in the root of your Backstage repository and fill in all your HTTP Fact Collectors. A simple example HTTP fact collector is listed below.
- factName: grafana_dashboard
  request:
    method: GET
    url: http://grafana.com/api/dashboards/uid/{{ entity.metadata.annotations['grafana.com/dashboard-uid'] }}
    headers:
      Accept: application/json
      Content-Type: application/json
      Authorization: Bearer {{ secrets.grafanaApiToken }}
  response:
    transform: dashboard
    emptyOnStatuses: [404]
    schema: |
      {
        "title": "Grafana Dashboard",
        "description": "Grafana Dashboard",
        "type": "object",
        "properties": {
          "id": {
            "type": "string"
          },
          "uid": {
            "type": "string"
          },
          "title": {
            "type": "string"
          },
          "tags": {
            "type": "array",
            "items": {
              "type": "string"
            }
          }
        }
      }
  retries:
    maxRetries: 3
    retryDelay: 5000
    retryOnStatuses: [408, 502, 503, 504]
- factName: owners_failed_check_results
  filter:
    kind: 'Component'
    spec.type: 'service'
  request:
    targetPluginId: soundcheck
    method: GET
    url: '{{ targetPluginBaseUrl }}/results?entityRef={{ entity.relations | where: "type", "ownedBy" | first | map: "targetRef" }}'
    headers:
      Authorization: Bearer {{ targetPluginAuthToken }}
  response:
    transform: results[state='failed']
    emptyOnStatuses: [404]
    schema: |
      {
        "title": "Check Results",
        "description": "Soundcheck Check Results",
        "type": "object",
        "properties": {
          "results": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "entityRef": {
                  "type": "string"
                },
                "checkId": {
                  "type": "string"
                },
                "scope": {
                  "type": "string"
                },
                "state": {
                  "type": "string"
                },
                "timestamp": {
                  "type": "string"
                }
              }
            }
          }
        }
      }

Note: this file will be loaded at runtime along with the rest of your Backstage configuration files, so make sure it's available in deployed environments in the same way as your app-config.yaml files.

  1. Add a Soundcheck collects field to the app-config.yaml and reference the newly created http-facts-collectors.yaml file.
# app-config.yaml
soundcheck:
  collectors:
    http:
      collects:
        $include: ./http-facts-collectors.yaml

Defining HTTP Fact Collectors​

This section describes the data shape and semantics of HTTP Fact Collectors.

Shape Of HTTP Fact Collector​

The following is an example of HTTP Fact Collector YAML configuration:

soundcheck:
  collectors:
    http:
      collects:
        - factName: grafana_dashboard
          request:
            method: GET
            url: http://grafana.com/api/dashboards/uid/{{ entity.metadata.annotations['grafana.com/dashboard-uid'] }}
            headers:
              Accept: application/json
              Content-Type: application/json
              Authorization: Bearer {{ secrets.grafanaApiToken }}
          response:
            transform: dashboard
            emptyOnStatuses: [404]
            schema: |
              {
                "title": "Grafana Dashboard",
                "description": "Grafana Dashboard",
                "type": "object",
                "properties": {
                  "id": {
                    "type": "string"
                  },
                  "uid": {
                    "type": "string"
                  },
                  "title": {
                    "type": "string"
                  },
                  "tags": {
                    "type": "array",
                    "items": {
                      "type": "string"
                    }
                  }
                }
              }
          retries:
            maxRetries: 3
            retryDelay: 5000
            retryOnStatuses: [408, 502, 503, 504]
          filter:
            kind: 'Component'
            spec.type: 'service'
          cache: false

Below are the details for each field.

factName [required]​

The name of the fact to be collected.

  • Minimum length of 1
  • Maximum length of 100
  • Alphanumeric with single separator instances of periods, dashes, underscores, or forward slashes

request [required]​

HTTP request configuration:

  • targetPluginId [optional] - Optional, provide if calling a plugin's HTTP endpoint. If provided, liquid template tags {{ targetPluginBaseUrl }} and {{ targetPluginAuthToken }} will be populated with the plugin's data.

  • method [optional] - HTTP method (GET or POST). Optional, if not provided the method will be defaulted to GET.

  • url [required] - HTTP request URL. Supports liquid templating. The following variables are available for use in templates:

    • entity - The entity the fact is collected for. Usage example: {{ entity.metadata.name }}
    • config - The configuration from app-config.yaml. Usage example: {{ config.app.title }}
    • targetPluginBaseUrl - Base URL of the plugin with ID provided in targetPluginId. Usage example: {{ targetPluginBaseUrl }}

  • headers [optional] - HTTP request headers. Header keys and values support liquid templating. The following variables are available for use in templates:

    • entity - The entity the fact is collected for. Usage example: {{ entity.metadata.name }}
    • config - The configuration from app-config.yaml. Usage example: {{ config.app.title }}
    • secrets - The HTTP secrets configuration. Usage example: {{ secrets.grafanaApiToken }}
    • targetPluginAuthToken - Auth token that can be used for authenticating calls towards plugin with ID provided in targetPluginId. Usage example: {{ targetPluginAuthToken }}

  • body [optional] - HTTP request body. Supports liquid templating. The following variables are available for use in templates:

    • entity - The entity the fact is collected for. Usage example: {{ entity.metadata.name }}
    • config - The configuration from app-config.yaml. Usage example: {{ config.app.title }}

response [optional]​

HTTP response configuration:

  • transform [optional] - JSONata expression to transform the HTTP response body. Optional, if not provided an unmodified response body will be used as the fact data.

  • emptyOnStatuses [optional] - A list of HTTP status codes to return an empty response on (when received, an empty fact will be collected instead of throwing a collection error). Optional, if not provided all non-OK responses will result in collection errors.

  • schema [optional] - A JSON schema describing the HTTP response body. Optional, primarily used for Check Creation No-Code UI autocomplete. Not used for any validations.

retries [optional]​

HTTP request retries configuration:

  • maxRetries [optional] - Max number of retries. Optional, if not provided failed requests won't be retried.

  • retryDelay [optional] - Retry delay in milliseconds. Optional, if not provided failed requests will be retried immediately.

  • retryOnStatuses [optional] - A list of HTTP status codes to retry the request on. Optional, if not provided all failed requests will be retried.

frequency [optional]​

The frequency at which the fact collection should be executed. Possible values are either a cron expression { cron: ... } or HumanDuration. If provided, it overrides the default frequency provided at the top level. If not provided, it defaults to the frequency provided at the top level. If neither collector's frequency nor default frequency is provided, the fact will only be collected on demand. Example:

frequency:
  minutes: 10

initialDelay [optional]​

The amount of time that should pass before the first invocation happens. Possible values are either a cron expression { cron: ... } or HumanDuration.

Example:

initialDelay:
  seconds: 30

batchSize [optional]​

The number of entities to collect facts for at once. Optional, the default value is 1.

Example:

batchSize: 100

filter [optional]​

A filter specifying which entities to collect the specified facts for. Matches the filter format used by the Catalog API. If provided, it overrides the default filter provided at the top level. If not provided, it defaults to the filter provided at the top level. If neither collector's filter nor default filter is provided, the fact will be collected for all entities.

See filters for more details.

exclude [optional]​

Entities matching this filter will be skipped during the fact collection process. Can be used in combination with filter. Matches the filter format used by the Catalog API.

filter:
  - kind: component
exclude:
  - spec.type: documentation

cache [optional]​

If the collected facts should be cached, and if so for how long. Possible values are either true or false or a nested { duration: HumanDuration } field. If provided it, overrides the default cache config provided at the top level. If not provided, it defaults to the cache config provided at the top level. If neither collector's cache nor default cache config is provided, the fact will not be cached. Example:

cache:
  duration:
    hours: 24

Rate Limiting​

This fact collector doesn't support rate limiting. If your target HTTP endpoint has rate limits it's recommended to implement a custom fact collector that will respect the configurable limits. Check out some recommendations on how to avoid exceeding API rate limits.

Shape of HTTP Fact​

The shape of HTTP Fact is based on the Fact Schema.

The following is an example of the collected HTTP fact:

factRef: http:default/grafana_dashboard
entityRef: component:default/queue-proxy
scope: default
timestamp: 2025-01-09T15:50+00Z
data:
  id: null
  uid: null
  title: Production Overview
  tags:
    - production
  timezone: browser
  schemaVersion: 16
  refresh: 25s

The resulting fact data will contain the HTTP response.

Shape of HTTP Fact Check​

The shape of HTTP Fact Check matches the Shape of a Fact Check.

The following is an example of the HTTP fact check:

soundcheck:
  checks:
    - id: has-production-dashboard-tag
      name: Grafana dashboard has production tag
      ownerEntityRef: group:default/toast-infra
      description: Grafana dashboard has production tag
      filter:
        kind: Component
        spec.type: service
      rule:
        factRef: http:default/grafana_dashboard
        path: $.tags
        operator: any:equal
        value: production
      passedMessage: Success! Grafana dashboard has production tag
      failedMessage: Failure! Grafana dashboard doesn't have production tag