> ## Documentation Index
> Fetch the complete documentation index at: https://backstage.spotify.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# PagerDuty

> Extract PagerDuty service details, standards compliance, and incident data as Soundcheck facts for operational readiness checks.

The PagerDuty integration plugin for [Soundcheck](https://www.npmjs.com/package/@spotify/backstage-plugin-soundcheck-backend-module-pagerduty) supports the extraction of the following facts:

* [Service](#collecting-service-facts)
* [Standards](#collecting-standards-fact)
* [Incidents](#collecting-incidents-fact)

## Prerequisites

### Configure PagerDuty integration in Backstage

Integrations are configured at the root level of the `app-config.yaml` file. Here's an example configuration for PagerDuty:

```yaml theme={"theme":{"light":"github-light","dark":"dracula"}}
soundcheck:
  collectors:
    pagerduty:
      token: ${PAGERDUTY_TOKEN}
      server: api.pagerduty.com
```

### Add the PagerDutyFactCollector to Soundcheck

First, add the `@spotify/backstage-plugin-soundcheck-backend-module-pagerduty` package:

```bash theme={"theme":{"light":"github-light","dark":"dracula"}}
yarn workspace backend add @spotify/backstage-plugin-soundcheck-backend-module-pagerduty
```

Then add the following to your `packages/backend/src/index.ts` file:

```ts packages/backend/src/index.ts highlight={4} theme={"theme":{"light":"github-light","dark":"dracula"}}
const backend = createBackend();

backend.add(import('@spotify/backstage-plugin-soundcheck-backend'));
backend.add(
  import('@spotify/backstage-plugin-soundcheck-backend-module-pagerduty'),
);
// ...

backend.start();
```

Consult the [Soundcheck Backend documentation](../../../setup-and-installation#backend-setup) for additional details on setting up the Soundcheck backend.

## Plugin Configuration

The collection of PagerDuty facts is driven by the configuration. To learn more about the configuration, consult the [Defining PagerDuty Fact Collectors](https://backstage.spotify.com/docs/plugins/soundcheck/core-concepts/fact-collectors/3p-integrations/pagerduty#defining-pagerduty-fact-collectors) section.

PagerDuty Fact Collector can be configured via YAML or No-Code UI. If you configure it via both YAML and No-Code UI, the configurations will be merged.
It's preferable to choose a single source for the Fact Collectors configuration (either No-Code UI or YAML) to avoid confusing merge results.

### No-Code UI Configuration Option

1. Make sure the prerequisite [Configure PagerDuty integration in Backstage](#configure-pagerduty-integration-in-backstage) is completed and PagerDuty instance details are configured.

2. To enable the PagerDuty Integration, go to `Soundcheck > Integrations > PagerDuty` and click the `Configure` button. To learn more about the No-Code UI config, see the [Configuring a fact collector (integration) via the no-code UI](../index#configuring-a-fact-collector-integration-via-the-no-code-ui).

<Frame>
  <img
    src="https://mintcdn.com/spotify-89f50c35/ulmNVnZzSFL3BsWR/plugins/soundcheck/images/collectors/pagerduty-collector-ncui.png?fit=max&auto=format&n=ulmNVnZzSFL3BsWR&q=85&s=42dfa668ba71fb312affdc60f4835f3a"
    alt="PagerDuty
Integration"
    width="3430"
    height="1920"
    data-path="plugins/soundcheck/images/collectors/pagerduty-collector-ncui.png"
  />
</Frame>

### YAML Configuration Option

1. Create a `pagerduty-facts-collectors.yaml` file in the root of your Backstage repository and fill in all your **PagerDuty Fact Collectors**.
   A simple example PagerDuty fact collector is listed below.

   ```yaml theme={"theme":{"light":"github-light","dark":"dracula"}}
   ---
   token: dummy
   server: api.pagerduty.com
   collects:
     - type: Standards
       filter:
         - spec.lifecycle: 'production'
           spec.type: 'website'
       cache: false
   ```

   **Note:** this file will be loaded at runtime along with the rest of your Backstage configuration files, so make sure it's available in deployed environments in the same way as your `app-config.yaml` files.

2. Add a Soundcheck collectors field to the `app-config.yaml` file and reference the newly created `pagerduty-facts-collectors.yaml` file.

   ```yaml theme={"theme":{"light":"github-light","dark":"dracula"}}
   # app-config.yaml
   soundcheck:
     collectors:
       pagerduty:
         $include: ./pagerduty-facts-collectors.yaml
   ```

### Rate Limiting (Optional)

This fact collector can be rate limited in Soundcheck using the following configuration:

```yaml theme={"theme":{"light":"github-light","dark":"dracula"}}
soundcheck:
  job:
    workers:
      pagerduty:
        limiter:
          max: 400
          duration: 60000
```

PagerDuty API has a limit of [960 requests per minute](https://developer.pagerduty.com/docs/72d3b724589e3-rest-api-rate-limits). We recommend setting your rate limit to something below this, i.e. in the example above, we set the rate limit to 400 executions every minute.

This fact collector handles 429 rate limit errors from PagerDuty. Soundcheck will automatically wait and retry requests that are rate limited.

## Defining PagerDuty Fact Collectors

This section describes the data shape and semantics of **PagerDuty Fact Collectors**.

### Overall Shape Of A PagerDuty Fact Collector

The following is an example of a descriptor file for a **PagerDuty Fact Collector**:

```yaml theme={"theme":{"light":"github-light","dark":"dracula"}}
---
frequency:
  cron: '0 * * * *'
filter:
  kind: 'Component'
cache:
  duration:
    hours: 2
collects:
  - type: Service
    filter:
      - spec.lifecycle: 'production'
        spec.type: 'website'
    cache: false
```

Below are the details for each field.

#### `frequency` \[optional]

The frequency at which the collector should be executed. Possible values are either a cron expression `{ cron: ... }` or [HumanDuration](https://backstage.io/docs/reference/types.humanduration).
This is the default frequency for each extractor.

#### `filter` \[optional]

A filter specifying which entities to collect the specified facts for. Matches the [filter format](https://backstage.io/docs/reference/catalog-client.entityfilterquery) used by the Catalog API.
This is the default filter for each extractor.

#### `exclude` \[optional]

Entities matching this filter will be skipped during the fact collection process. Can be used in combination with filter. Matches the [filter format](https://backstage.io/docs/reference/catalog-client.entityfilterquery) used by the Catalog API.

```yaml theme={"theme":{"light":"github-light","dark":"dracula"}}
filter:
  - kind: component
exclude:
  - spec.type: documentation
```

#### `cache` \[optional]

If the collected facts should be cached, and if so, for how long. Possible values are either `true` or `false` or a nested `{ duration:` [HumanDuration](https://backstage.io/docs/reference/types.humanduration) `}` field.
This is the default cache config for each extractor.

#### `collects` \[required]

An array describing which facts to collect and how to extract them. See below for details on the overall shape of a fact extractor.

### Overall Shape Of A Fact Extractor

Each extractor supports the fields described below.

#### `type` \[required]

The type of the extractor (e.g. Service, Standards).

#### `frequency` \[optional]

The frequency at which the fact extraction should be executed. Possible values are either a cron expression `{ cron: ... }` or [HumanDuration](https://backstage.io/docs/reference/types.humanduration).
If provided, it overrides the default frequency provided at the top level. If not provided, it defaults to the frequency provided at the top level. If neither extractor's frequency, nor default frequency is provided, the fact will only be collected on demand.
Example:

```yaml theme={"theme":{"light":"github-light","dark":"dracula"}}
frequency:
  minutes: 10
```

#### `initialDelay` \[optional]

The amount of time that should pass before the first invocation happens. Possible values are either a cron expression `{ cron: ... }` or [HumanDuration](https://backstage.io/docs/reference/types.humanduration).

Example:

```yaml theme={"theme":{"light":"github-light","dark":"dracula"}}
initialDelay:
  seconds: 30
```

#### `batchSize` \[optional]

The number of entities to collect facts for at once. Optional, the default value is 1.

**Note**: Fact collection for a batch of entities is still considered as one hit towards the rate limits
by the Soundcheck [Rate Limiting](#rate-limiting-optional) engine, while the actual number of hits
will be equal to the `batchSize`.

Example:

```yaml theme={"theme":{"light":"github-light","dark":"dracula"}}
batchSize: 100
```

#### `filter` \[optional]

A filter specifying which entities to collect the specified facts for. Matches the [filter format](https://backstage.io/docs/reference/catalog-client.entityfilterquery) used by the Catalog API.
If provided, it overrides the default filter provided at the top level. If not provided, it defaults to the filter provided at the top level. If neither extractor's filter, nor default filter is provided, the fact will be collected for all entities.

See [filters](/docs/plugins/soundcheck/core-concepts/filters) for more details.

#### `exclude` \[optional]

Entities matching this filter will be skipped during the fact collection process. Can be used in combination with filter. Matches the [filter format](https://backstage.io/docs/reference/catalog-client.entityfilterquery) used by the Catalog API.

```yaml theme={"theme":{"light":"github-light","dark":"dracula"}}
filter:
  - kind: component
exclude:
  - spec.type: documentation
```

#### `cache` \[optional]

If the collected facts should be cached, and if so, for how long. Possible values are either `true` or `false` or a nested `{ duration:` [HumanDuration](https://backstage.io/docs/reference/types.humanduration) `}` field.
If provided, it overrides the default cache config provided at the top level. If not provided, it defaults to the cache config provided at the top level. If neither extractor's cache, nor default cache config is provided, the fact will not be cached.
Example:

```yaml theme={"theme":{"light":"github-light","dark":"dracula"}}
cache:
  duration:
    hours: 24
```

## Entity configuration

In your `catalog-info.yaml` file, add following metadata annotation to allow the plugin to map an entity to a service in PagerDuty.

```yaml theme={"theme":{"light":"github-light","dark":"dracula"}}
metadata:
  annotations:
    pagerduty.com/service-id: pd-test-service-id
```

Alternatively, you can also configure an integration key.

```yaml theme={"theme":{"light":"github-light","dark":"dracula"}}
metadata:
  annotations:
    pagerduty.com/integration-key: pd-test-integration-key
```

## Collecting Service Facts

A **Service Fact** contains information about a *service* from the [PagerDuty Service API](https://developer.pagerduty.com/api-reference/165ad96a22ffd-get-a-service).

### Shape of A Service Fact Collector

The shape of a **Service Fact Collector** matches the [Overall Shape Of A PagerDuty Fact Collector](#overall-shape-of-a-pagerduty-fact-collector) (restriction: `type: Service`).

Here's an example of a **Service Fact Collector** configuration:

```yaml theme={"theme":{"light":"github-light","dark":"dracula"}}
- type: Service
  cache: true
  frequency:
    cron: '0 * * * *'
```

### Shape of A Service Fact

The shape of a **Service Fact** is based on the Soundcheck [Fact Schema](#overall-shape-of-a-pagerduty-fact-collector).

### Shape of A Service Fact Check

The shape of a **Service Fact Check** matches the [Shape of a Fact Check](https://backstage.spotify.com/docs/plugins/soundcheck/core-concepts/checks#overall-shape-of-a-check).

Here's an example of a **Service Fact Check** configuration:

```yaml theme={"theme":{"light":"github-light","dark":"dracula"}}
soundcheck:
  checks:
    - id: requires_type_to_be_service
      description: Requires type to be service
      passedMessage: The check has passed!
      failedMessage: The check has failed!
      rule:
        factRef: pagerduty:default/service
        path: $.type
        operator: equal
        value: service
```

The following is an example response from the Service Fact collector.
See PagerDuty's [documentation](https://developer.pagerduty.com/api-reference/165ad96a22ffd-get-a-service#Responses) for more detail.

```json theme={"theme":{"light":"github-light","dark":"dracula"}}
{
  "id": "PIJ90N7",
  "type": "service",
  "summary": "My Application Service",
  "self": "https://api.pagerduty.com/services/PIJ90N7",
  "html_url": "https://subdomain.pagerduty.com/service-directory/PIJ90N7",
  "name": "My Application Service",
  "auto_resolve_timeout": 14400,
  "acknowledgement_timeout": 600,
  "created_at": "2015-11-06T11:12:51-05:00",
  "status": "active",
  "alert_creation": "create_alerts_and_incidents",
  "integrations": [
    {
      "id": "PQ12345",
      "type": "generic_email_inbound_integration_reference",
      "summary": "Email Integration",
      "self": "https://api.pagerduty.com/services/PIJ90N7/integrations/PQ12345",
      "html_url": "https://subdomain.pagerduty.com/services/PIJ90N7/integrations/PQ12345"
    }
  ],
  "escalation_policy": {
    "id": "PT20YPA",
    "type": "escalation_policy_reference",
    "summary": "Another Escalation Policy",
    "self": "https://api.pagerduty.com/escalation_policies/PT20YPA",
    "html_url": "https://subdomain.pagerduty.com/escalation_policies/PT20YPA"
  },
  "teams": [
    {
      "id": "PQ9K7I8",
      "type": "team_reference",
      "summary": "Engineering",
      "self": "https://api.pagerduty.com/teams/PQ9K7I8",
      "html_url": "https://subdomain.pagerduty.com/teams/PQ9K7I8"
    }
  ],
  "incident_urgency_rule": {
    "type": "use_support_hours",
    "during_support_hours": {
      "type": "constant",
      "urgency": "high"
    },
    "outside_support_hours": {
      "type": "constant",
      "urgency": "low"
    }
  },
  "support_hours": {
    "type": "fixed_time_per_day",
    "time_zone": "America/Lima",
    "start_time": "09:00:00",
    "end_time": "17:00:00",
    "days_of_week": [1, 2, 3, 4, 5]
  },
  "scheduled_actions": [
    {
      "type": "urgency_change",
      "at": {
        "type": "named_time",
        "name": "support_hours_start"
      },
      "to_urgency": "high"
    }
  ],
  "auto_pause_notifications_parameters": {
    "enabled": true,
    "timeout": 300
  }
}
```

## Collecting Standards Fact

The **Standards Fact** contains information about *standards* from [PagerDuty Standards API](https://developer.pagerduty.com/api-reference/f339354b607d5-list-a-resource-s-standards-scores).

### Shape of A Standards Fact Collector

The shape of a **Standards Fact Collector** matches the [Overall Shape Of A PagerDuty Fact Collector](#overall-shape-of-a-pagerduty-fact-collector) (restriction: `type: Standards`).

Here's an example of a **Standards Fact Collector** configuration:

```yaml theme={"theme":{"light":"github-light","dark":"dracula"}}
- type: Standards
  cache: true
  frequency:
    cron: '0 * * * *'
```

### Shape of A Standards Fact

The shape of a **Standards Fact** is based on the Soundcheck [Fact Schema](#overall-shape-of-a-pagerduty-fact-collector).

### Shape of A Standards Fact Check

The shape of a **Standards Fact Check** matches the [Shape of a Fact Check](https://backstage.spotify.com/docs/plugins/soundcheck/core-concepts/checks#overall-shape-of-a-check).

Here's an example of a **Standards Fact Check**:

```yaml theme={"theme":{"light":"github-light","dark":"dracula"}}
soundcheck:
  checks:
    - id: requires_resource_type_to_be_technical_service
      description: Requires resource type to be technical_service
      passedMessage: The check has passed!
      failedMessage: The check has failed!
      rule:
        factRef: pagerduty:default/standards
        path: $.resource_type
        operator: equal
        value: technical_service
```

The following is an example response from the Standards Fact collector.
See PagerDuty's [documentation](https://developer.pagerduty.com/api-reference/dbed9a0ff9355-list-standards#Responses) for more detail.

```json theme={"theme":{"light":"github-light","dark":"dracula"}}
{
  "standards": [
    {
      "active": true,
      "description": "A description provides critical context about what a service represents or is used for to inform team members and responders. The description should be kept concise and understandable by those without deep knowledge of the service.",
      "id": "01CXX38Q0U8XKHO4LNKXUJTBFG",
      "pass": true,
      "name": "Service has a name",
      "type": "has_technical_service"
    },
    {
      "active": true,
      "description": "Ensure that no incident goes unaddressed, even if the on-call responder on the first level of the escalation policy is unavailable.",
      "exclusions": [],
      "id": "01CXX38Q0Y8D9IYFAEDCH5F53L",
      "inclusions": [],
      "name": "Service has an escalation policy with 2 or more unique levels",
      "resource_type": "technical_service",
      "type": "minimum_escalation_policy_rule_depth"
    }
  ]
}
```

## Collecting Incidents Fact

The **Incidents Fact** contains information about *incidents* from [PagerDuty Incidents API](https://developer.pagerduty.com/api-reference/9d0b4b12e36f9-list-incidents).

### Shape of A Incidents Fact Collector

The shape of an **Incidents Fact Collector** matches the [Overall Shape Of A PagerDuty Fact Collector](#overall-shape-of-a-pagerduty-fact-collector) (restriction: `type: Incidents`).

Here's an example of an **Incidents Fact Collector** configuration:

```yaml theme={"theme":{"light":"github-light","dark":"dracula"}}
- type: Incidents
  statuses:
    - triggered
    - acknowledged
    - resolved
  cache: true
  frequency:
    cron: '0 * * * *'
```

Provide a **factName** to define mutliple configurations of the **Incidents Fact Collector**:

```yaml theme={"theme":{"light":"github-light","dark":"dracula"}}
- factName: triggered_incidents
  type: Incidents
  statuses:
    - triggered
  cache: true
  frequency:
    cron: '0 * * * *'
- factName: acknowledged_and_recieved_incidents
  type: Incidents
  statuses:
    - acknowledged
    - resolved
  cache: true
  frequency:
    cron: '0 * * * *'
```

### Shape of An Incidents Fact

The shape of an **Incidents Fact** is based on the Soundcheck [Fact Schema](https://www.npmjs.com/package/@spotify/backstage-plugin-soundcheck-backend#fact-schema).

### Shape of An Incidents Fact Check

The shape of an **Incidents Fact Check** matches the [Shape of a Fact Check](https://www.npmjs.com/package/@spotify/backstage-plugin-soundcheck-backend#shape-of-a-fact-check).

Here's an example of an **Incidents Fact Check**:

```yaml theme={"theme":{"light":"github-light","dark":"dracula"}}
soundcheck:
  checks:
    - id: has_less_than_5_incidents
      rule:
        factRef: pagerduty:default/Incidents # default fact name ref
        path: $.total
        operator: lessThan
        value: 5
    - id: has_less_than_5_triggered_incidents
      rule:
        factRef: pagerduty:default/triggered_incidents # custom factName ref
        path: $.total
        operator: lessThan
        value: 5
```

The following is an example response from the **Incidents Fact Collector**.
See PagerDuty's [documentation](https://developer.pagerduty.com/api-reference/9d0b4b12e36f9-list-incidents#Responses) for more detail.

```json theme={"theme":{"light":"github-light","dark":"dracula"}}
"total": 1,
"incidents": [
  {
    "id": "PT4KHLK",
    "type": "incident",
    "summary": "[#1234] The server is on fire.",
    "self": "https://api.pagerduty.com/incidents/PT4KHLK",
    "html_url": "https://subdomain.pagerduty.com/incidents/PT4KHLK",
    "incident_number": 1234,
    "title": "The server is on fire.",
    "created_at": "2015-10-06T21:30:42Z",
    "updated_at": "2015-10-06T21:40:23Z",
    "status": "triggered",
    "incident_key": "baf7cf21b1da41b4b0221008339ff357",
    "service": {
      "id": "pd-test-service-id",
      "type": "service_reference",
      "summary": "My Mail Service",
      "self": "https://api.pagerduty.com/services/PIJ90N7",
      "html_url": "https://subdomain.pagerduty.com/service-directory/PIJ90N7",
    },
    "assignments": [],
    "assigned_via": "escalation_policy",
    "last_status_change_at": "2015-10-06T21:38:23Z",
    "resolved_at": "2015-10-06T21:38:23Z",
    "first_trigger_log_entry": {
      "id": "Q02JTSNZWHSEKV",
      "type": "trigger_log_entry_reference",
      "summary": "Triggered through the API",
      "self": "https://api.pagerduty.com/log_entries/Q02JTSNZWHSEKV?incident_id=PT4KHLK",
      "html_url":
        "https://subdomain.pagerduty.com/incidents/PT4KHLK/log_entries/Q02JTSNZWHSEKV",
    },
    "alert_counts": {
      "all": 1,
      "triggered": 1,
      "resolved": 0,
    },
    "is_mergeable": true,
    "escalation_policy": {
      "id": "PT20YPA",
      "type": "escalation_policy_reference",
      "summary": "Another Escalation Policy",
      "self": "https://api.pagerduty.com/escalation_policies/PT20YPA",
      "html_url": "https://subdomain.pagerduty.com/escalation_policies/PT20YPA",
    },
    "teams": [
      {
        "id": "PQ9K7I8",
        "type": "team_reference",
        "summary": "Engineering",
        "self": "https://api.pagerduty.com/teams/PQ9K7I8",
        "html_url": "https://subdomain.pagerduty.com/teams/PQ9K7I8",
      },
    ],
    "pending_actions": [],
    "acknowledgements": [],
    "alert_grouping": {
      "grouping_type": "advanced",
      "started_at": "2015-10-06T21:30:42Z",
      "ended_at": null,
      "alert_grouping_active": true,
    },
    "last_status_change_by": {
      "id": "PXPGF42",
      "type": "user_reference",
      "summary": "Earline Greenholt",
      "self": "https://api.pagerduty.com/users/PXPGF42",
      "html_url": "https://subdomain.pagerduty.com/users/PXPGF42",
    },
    "priority": {
      "id": "P53ZZH5",
      "type": "priority_reference",
      "summary": "P2",
      "self": "https://api.pagerduty.com/priorities/P53ZZH5",
    },
    "resolve_reason": null,
    "conference_bridge": {
      "conference_number": "+1-415-555-1212,,,,1234#",
      "conference_url": "https://example.com/acb-123",
    },
    "incidents_responders": [],
    "responder_requests": [],
    "urgency": "high",
  },
]
```

## Shape of A PagerDuty Track

The following is an example of the Soundcheck track that utilizes these checks

```yaml theme={"theme":{"light":"github-light","dark":"dracula"}}
soundcheck:
  tracks:
    - id: demo
      name: Demo
      ownerEntityRef: group:default/owning_group
      description: Demonstration of Soundcheck PagerDuty Fact Collector
      levels:
        - ordinal: 1
          name: First level
          description: Checks leveraging SoundCheck's PagerDuty Fact Collector
          checks:
            - id: requires_resource_type_to_be_technical_service
              name: Requires resource type to be technical service
              description: Requires resource type to be technical service
            - id: requires_type_to_be_service
              name: Requires type to be service
              description: Requires type to be service
      filter:
        catalog:
          metadata.tags: some-tag
```

For a description of the data collected about a service, refer to the [PagerDuty Service API documentation](https://developer.pagerduty.com/api-reference/165ad96a22ffd-get-a-service).
