PagerDuty - Spotify for Backstage

The PagerDuty integration plugin for Soundcheck supports the extraction of the following facts:

Service
Standards
Incidents

Prerequisites

Configure PagerDuty integration in Backstage

Integrations are configured at the root level of the app-config.yaml file. Here’s an example configuration for PagerDuty:

soundcheck:
  collectors:
    pagerduty:
      token: ${PAGERDUTY_TOKEN}
      server: api.pagerduty.com

Add the PagerDutyFactCollector to Soundcheck

First, add the @spotify/backstage-plugin-soundcheck-backend-module-pagerduty package:

yarn workspace backend add @spotify/backstage-plugin-soundcheck-backend-module-pagerduty

Then add the following to your packages/backend/src/index.ts file:

packages/backend/src/index.ts

const backend = createBackend();

backend.add(import('@spotify/backstage-plugin-soundcheck-backend'));
backend.add(
  import('@spotify/backstage-plugin-soundcheck-backend-module-pagerduty'),
);
// ...

backend.start();

Consult the Soundcheck Backend documentation for additional details on setting up the Soundcheck backend.

Plugin Configuration

The collection of PagerDuty facts is driven by the configuration. To learn more about the configuration, consult the Defining PagerDuty Fact Collectors section. PagerDuty Fact Collector can be configured via YAML or No-Code UI. If you configure it via both YAML and No-Code UI, the configurations will be merged. It’s preferable to choose a single source for the Fact Collectors configuration (either No-Code UI or YAML) to avoid confusing merge results.

No-Code UI Configuration Option

Make sure the prerequisite Configure PagerDuty integration in Backstage is completed and PagerDuty instance details are configured.
To enable the PagerDuty Integration, go to Soundcheck > Integrations > PagerDuty and click the Configure button. To learn more about the No-Code UI config, see the Configuring a fact collector (integration) via the no-code UI.

YAML Configuration Option

Create a pagerduty-facts-collectors.yaml file in the root of your Backstage repository and fill in all your PagerDuty Fact Collectors. A simple example PagerDuty fact collector is listed below.
```
---
token: dummy
server: api.pagerduty.com
collects:
  - type: Standards
    filter:
      - spec.lifecycle: 'production'
        spec.type: 'website'
    cache: false
```
Note: this file will be loaded at runtime along with the rest of your Backstage configuration files, so make sure it’s available in deployed environments in the same way as your app-config.yaml files.

Add a Soundcheck collectors field to the app-config.yaml file and reference the newly created pagerduty-facts-collectors.yaml file.

# app-config.yaml
soundcheck:
  collectors:
    pagerduty:
      $include: ./pagerduty-facts-collectors.yaml

Rate Limiting (Optional)

This fact collector can be rate limited in Soundcheck using the following configuration:

soundcheck:
  job:
    workers:
      pagerduty:
        limiter:
          max: 400
          duration: 60000

PagerDuty API has a limit of 960 requests per minute. We recommend setting your rate limit to something below this, i.e. in the example above, we set the rate limit to 400 executions every minute. This fact collector handles 429 rate limit errors from PagerDuty. Soundcheck will automatically wait and retry requests that are rate limited.

Defining PagerDuty Fact Collectors

This section describes the data shape and semantics of PagerDuty Fact Collectors.

Overall Shape Of A PagerDuty Fact Collector

The following is an example of a descriptor file for a PagerDuty Fact Collector:

---
frequency:
  cron: '0 * * * *'
filter:
  kind: 'Component'
cache:
  duration:
    hours: 2
collects:
  - type: Service
    filter:
      - spec.lifecycle: 'production'
        spec.type: 'website'
    cache: false

Below are the details for each field.

`frequency` [optional]

The frequency at which the collector should be executed. Possible values are either a cron expression { cron: ... } or HumanDuration. This is the default frequency for each extractor.

`filter` [optional]

A filter specifying which entities to collect the specified facts for. Matches the filter format used by the Catalog API. This is the default filter for each extractor.

`exclude` [optional]

Entities matching this filter will be skipped during the fact collection process. Can be used in combination with filter. Matches the filter format used by the Catalog API.

filter:
  - kind: component
exclude:
  - spec.type: documentation

`cache` [optional]

If the collected facts should be cached, and if so, for how long. Possible values are either true or false or a nested { duration: HumanDuration } field. This is the default cache config for each extractor.

`collects` [required]

An array describing which facts to collect and how to extract them. See below for details on the overall shape of a fact extractor.

Overall Shape Of A Fact Extractor

Each extractor supports the fields described below.

`type` [required]

The type of the extractor (e.g. Service, Standards).

`frequency` [optional]

The frequency at which the fact extraction should be executed. Possible values are either a cron expression { cron: ... } or HumanDuration. If provided, it overrides the default frequency provided at the top level. If not provided, it defaults to the frequency provided at the top level. If neither extractor’s frequency, nor default frequency is provided, the fact will only be collected on demand. Example:

frequency:
  minutes: 10

`initialDelay` [optional]

The amount of time that should pass before the first invocation happens. Possible values are either a cron expression { cron: ... } or HumanDuration. Example:

initialDelay:
  seconds: 30

`batchSize` [optional]

The number of entities to collect facts for at once. Optional, the default value is 1. Note: Fact collection for a batch of entities is still considered as one hit towards the rate limits by the Soundcheck Rate Limiting engine, while the actual number of hits will be equal to the batchSize. Example:

batchSize: 100

`filter` [optional]

A filter specifying which entities to collect the specified facts for. Matches the filter format used by the Catalog API. If provided, it overrides the default filter provided at the top level. If not provided, it defaults to the filter provided at the top level. If neither extractor’s filter, nor default filter is provided, the fact will be collected for all entities. See filters for more details.

`exclude` [optional]

Entities matching this filter will be skipped during the fact collection process. Can be used in combination with filter. Matches the filter format used by the Catalog API.

filter:
  - kind: component
exclude:
  - spec.type: documentation

`cache` [optional]

If the collected facts should be cached, and if so, for how long. Possible values are either true or false or a nested { duration: HumanDuration } field. If provided, it overrides the default cache config provided at the top level. If not provided, it defaults to the cache config provided at the top level. If neither extractor’s cache, nor default cache config is provided, the fact will not be cached. Example:

cache:
  duration:
    hours: 24

Entity configuration

In your catalog-info.yaml file, add following metadata annotation to allow the plugin to map an entity to a service in PagerDuty.

metadata:
  annotations:
    pagerduty.com/service-id: pd-test-service-id

Alternatively, you can also configure an integration key.

metadata:
  annotations:
    pagerduty.com/integration-key: pd-test-integration-key

Collecting Service Facts

A Service Fact contains information about a service from the PagerDuty Service API.

Shape of A Service Fact Collector

The shape of a Service Fact Collector matches the Overall Shape Of A PagerDuty Fact Collector (restriction: type: Service). Here’s an example of a Service Fact Collector configuration:

- type: Service
  cache: true
  frequency:
    cron: '0 * * * *'

Shape of A Service Fact

The shape of a Service Fact is based on the Soundcheck Fact Schema.

Shape of A Service Fact Check

The shape of a Service Fact Check matches the Shape of a Fact Check. Here’s an example of a Service Fact Check configuration:

soundcheck:
  checks:
    - id: requires_type_to_be_service
      description: Requires type to be service
      passedMessage: The check has passed!
      failedMessage: The check has failed!
      rule:
        factRef: pagerduty:default/service
        path: $.type
        operator: equal
        value: service

The following is an example response from the Service Fact collector. See PagerDuty’s documentation for more detail.

{
  "id": "PIJ90N7",
  "type": "service",
  "summary": "My Application Service",
  "self": "https://api.pagerduty.com/services/PIJ90N7",
  "html_url": "https://subdomain.pagerduty.com/service-directory/PIJ90N7",
  "name": "My Application Service",
  "auto_resolve_timeout": 14400,
  "acknowledgement_timeout": 600,
  "created_at": "2015-11-06T11:12:51-05:00",
  "status": "active",
  "alert_creation": "create_alerts_and_incidents",
  "integrations": [
    {
      "id": "PQ12345",
      "type": "generic_email_inbound_integration_reference",
      "summary": "Email Integration",
      "self": "https://api.pagerduty.com/services/PIJ90N7/integrations/PQ12345",
      "html_url": "https://subdomain.pagerduty.com/services/PIJ90N7/integrations/PQ12345"
    }
  ],
  "escalation_policy": {
    "id": "PT20YPA",
    "type": "escalation_policy_reference",
    "summary": "Another Escalation Policy",
    "self": "https://api.pagerduty.com/escalation_policies/PT20YPA",
    "html_url": "https://subdomain.pagerduty.com/escalation_policies/PT20YPA"
  },
  "teams": [
    {
      "id": "PQ9K7I8",
      "type": "team_reference",
      "summary": "Engineering",
      "self": "https://api.pagerduty.com/teams/PQ9K7I8",
      "html_url": "https://subdomain.pagerduty.com/teams/PQ9K7I8"
    }
  ],
  "incident_urgency_rule": {
    "type": "use_support_hours",
    "during_support_hours": {
      "type": "constant",
      "urgency": "high"
    },
    "outside_support_hours": {
      "type": "constant",
      "urgency": "low"
    }
  },
  "support_hours": {
    "type": "fixed_time_per_day",
    "time_zone": "America/Lima",
    "start_time": "09:00:00",
    "end_time": "17:00:00",
    "days_of_week": [1, 2, 3, 4, 5]
  },
  "scheduled_actions": [
    {
      "type": "urgency_change",
      "at": {
        "type": "named_time",
        "name": "support_hours_start"
      },
      "to_urgency": "high"
    }
  ],
  "auto_pause_notifications_parameters": {
    "enabled": true,
    "timeout": 300
  }
}

Collecting Standards Fact

The Standards Fact contains information about standards from PagerDuty Standards API.

Shape of A Standards Fact Collector

The shape of a Standards Fact Collector matches the Overall Shape Of A PagerDuty Fact Collector (restriction: type: Standards). Here’s an example of a Standards Fact Collector configuration:

- type: Standards
  cache: true
  frequency:
    cron: '0 * * * *'

Shape of A Standards Fact

The shape of a Standards Fact is based on the Soundcheck Fact Schema.

Shape of A Standards Fact Check

The shape of a Standards Fact Check matches the Shape of a Fact Check. Here’s an example of a Standards Fact Check:

soundcheck:
  checks:
    - id: requires_resource_type_to_be_technical_service
      description: Requires resource type to be technical_service
      passedMessage: The check has passed!
      failedMessage: The check has failed!
      rule:
        factRef: pagerduty:default/standards
        path: $.resource_type
        operator: equal
        value: technical_service

The following is an example response from the Standards Fact collector. See PagerDuty’s documentation for more detail.

{
  "standards": [
    {
      "active": true,
      "description": "A description provides critical context about what a service represents or is used for to inform team members and responders. The description should be kept concise and understandable by those without deep knowledge of the service.",
      "id": "01CXX38Q0U8XKHO4LNKXUJTBFG",
      "pass": true,
      "name": "Service has a name",
      "type": "has_technical_service"
    },
    {
      "active": true,
      "description": "Ensure that no incident goes unaddressed, even if the on-call responder on the first level of the escalation policy is unavailable.",
      "exclusions": [],
      "id": "01CXX38Q0Y8D9IYFAEDCH5F53L",
      "inclusions": [],
      "name": "Service has an escalation policy with 2 or more unique levels",
      "resource_type": "technical_service",
      "type": "minimum_escalation_policy_rule_depth"
    }
  ]
}

Collecting Incidents Fact

The Incidents Fact contains information about incidents from PagerDuty Incidents API.

Shape of A Incidents Fact Collector

The shape of an Incidents Fact Collector matches the Overall Shape Of A PagerDuty Fact Collector (restriction: type: Incidents). Here’s an example of an Incidents Fact Collector configuration:

- type: Incidents
  statuses:
    - triggered
    - acknowledged
    - resolved
  cache: true
  frequency:
    cron: '0 * * * *'

Provide a factName to define mutliple configurations of the Incidents Fact Collector:

- factName: triggered_incidents
  type: Incidents
  statuses:
    - triggered
  cache: true
  frequency:
    cron: '0 * * * *'
- factName: acknowledged_and_recieved_incidents
  type: Incidents
  statuses:
    - acknowledged
    - resolved
  cache: true
  frequency:
    cron: '0 * * * *'

Shape of An Incidents Fact

The shape of an Incidents Fact is based on the Soundcheck Fact Schema.

Shape of An Incidents Fact Check

The shape of an Incidents Fact Check matches the Shape of a Fact Check. Here’s an example of an Incidents Fact Check:

soundcheck:
  checks:
    - id: has_less_than_5_incidents
      rule:
        factRef: pagerduty:default/Incidents # default fact name ref
        path: $.total
        operator: lessThan
        value: 5
    - id: has_less_than_5_triggered_incidents
      rule:
        factRef: pagerduty:default/triggered_incidents # custom factName ref
        path: $.total
        operator: lessThan
        value: 5

The following is an example response from the Incidents Fact Collector. See PagerDuty’s documentation for more detail.

"total": 1,
"incidents": [
  {
    "id": "PT4KHLK",
    "type": "incident",
    "summary": "[#1234] The server is on fire.",
    "self": "https://api.pagerduty.com/incidents/PT4KHLK",
    "html_url": "https://subdomain.pagerduty.com/incidents/PT4KHLK",
    "incident_number": 1234,
    "title": "The server is on fire.",
    "created_at": "2015-10-06T21:30:42Z",
    "updated_at": "2015-10-06T21:40:23Z",
    "status": "triggered",
    "incident_key": "baf7cf21b1da41b4b0221008339ff357",
    "service": {
      "id": "pd-test-service-id",
      "type": "service_reference",
      "summary": "My Mail Service",
      "self": "https://api.pagerduty.com/services/PIJ90N7",
      "html_url": "https://subdomain.pagerduty.com/service-directory/PIJ90N7",
    },
    "assignments": [],
    "assigned_via": "escalation_policy",
    "last_status_change_at": "2015-10-06T21:38:23Z",
    "resolved_at": "2015-10-06T21:38:23Z",
    "first_trigger_log_entry": {
      "id": "Q02JTSNZWHSEKV",
      "type": "trigger_log_entry_reference",
      "summary": "Triggered through the API",
      "self": "https://api.pagerduty.com/log_entries/Q02JTSNZWHSEKV?incident_id=PT4KHLK",
      "html_url":
        "https://subdomain.pagerduty.com/incidents/PT4KHLK/log_entries/Q02JTSNZWHSEKV",
    },
    "alert_counts": {
      "all": 1,
      "triggered": 1,
      "resolved": 0,
    },
    "is_mergeable": true,
    "escalation_policy": {
      "id": "PT20YPA",
      "type": "escalation_policy_reference",
      "summary": "Another Escalation Policy",
      "self": "https://api.pagerduty.com/escalation_policies/PT20YPA",
      "html_url": "https://subdomain.pagerduty.com/escalation_policies/PT20YPA",
    },
    "teams": [
      {
        "id": "PQ9K7I8",
        "type": "team_reference",
        "summary": "Engineering",
        "self": "https://api.pagerduty.com/teams/PQ9K7I8",
        "html_url": "https://subdomain.pagerduty.com/teams/PQ9K7I8",
      },
    ],
    "pending_actions": [],
    "acknowledgements": [],
    "alert_grouping": {
      "grouping_type": "advanced",
      "started_at": "2015-10-06T21:30:42Z",
      "ended_at": null,
      "alert_grouping_active": true,
    },
    "last_status_change_by": {
      "id": "PXPGF42",
      "type": "user_reference",
      "summary": "Earline Greenholt",
      "self": "https://api.pagerduty.com/users/PXPGF42",
      "html_url": "https://subdomain.pagerduty.com/users/PXPGF42",
    },
    "priority": {
      "id": "P53ZZH5",
      "type": "priority_reference",
      "summary": "P2",
      "self": "https://api.pagerduty.com/priorities/P53ZZH5",
    },
    "resolve_reason": null,
    "conference_bridge": {
      "conference_number": "+1-415-555-1212,,,,1234#",
      "conference_url": "https://example.com/acb-123",
    },
    "incidents_responders": [],
    "responder_requests": [],
    "urgency": "high",
  },
]

Shape of A PagerDuty Track

The following is an example of the Soundcheck track that utilizes these checks

soundcheck:
  tracks:
    - id: demo
      name: Demo
      ownerEntityRef: group:default/owning_group
      description: Demonstration of Soundcheck PagerDuty Fact Collector
      levels:
        - ordinal: 1
          name: First level
          description: Checks leveraging SoundCheck's PagerDuty Fact Collector
          checks:
            - id: requires_resource_type_to_be_technical_service
              name: Requires resource type to be technical service
              description: Requires resource type to be technical service
            - id: requires_type_to_be_service
              name: Requires type to be service
              description: Requires type to be service
      filter:
        catalog:
          metadata.tags: some-tag

For a description of the data collected about a service, refer to the PagerDuty Service API documentation.

​Prerequisites

​Configure PagerDuty integration in Backstage

​Add the PagerDutyFactCollector to Soundcheck

​Plugin Configuration

​No-Code UI Configuration Option

​YAML Configuration Option

​Rate Limiting (Optional)

​Defining PagerDuty Fact Collectors

​Overall Shape Of A PagerDuty Fact Collector

​frequency [optional]

​filter [optional]

​exclude [optional]

​cache [optional]

​collects [required]

​Overall Shape Of A Fact Extractor

​type [required]

​frequency [optional]

​initialDelay [optional]

​batchSize [optional]

​filter [optional]

​exclude [optional]

​cache [optional]

​Entity configuration

​Collecting Service Facts

​Shape of A Service Fact Collector

​Shape of A Service Fact

​Shape of A Service Fact Check

​Collecting Standards Fact

​Shape of A Standards Fact Collector

​Shape of A Standards Fact

​Shape of A Standards Fact Check

​Collecting Incidents Fact

​Shape of A Incidents Fact Collector

​Shape of An Incidents Fact

​Shape of An Incidents Fact Check

​Shape of A PagerDuty Track

Prerequisites

Configure PagerDuty integration in Backstage

Add the PagerDutyFactCollector to Soundcheck

Plugin Configuration

No-Code UI Configuration Option

YAML Configuration Option

Rate Limiting (Optional)

Defining PagerDuty Fact Collectors

Overall Shape Of A PagerDuty Fact Collector

`frequency` [optional]

`filter` [optional]

`exclude` [optional]

`cache` [optional]

`collects` [required]

Overall Shape Of A Fact Extractor

`type` [required]

`frequency` [optional]

`initialDelay` [optional]

`batchSize` [optional]

`filter` [optional]

`exclude` [optional]

`cache` [optional]

Entity configuration

Collecting Service Facts

Shape of A Service Fact Collector

Shape of A Service Fact

Shape of A Service Fact Check

Collecting Standards Fact

Shape of A Standards Fact Collector

Shape of A Standards Fact

Shape of A Standards Fact Check

Collecting Incidents Fact

Shape of A Incidents Fact Collector

Shape of An Incidents Fact

Shape of An Incidents Fact Check

Shape of A PagerDuty Track