Tracks

Tracks are composed of levels, checks and certifications. Tracks encourage alignment to architectural best practices and standards and are analogous to an organization’s long-term tech health initiatives. Tracks are composed of one or more levels, and each level is composed of one or more checks. Each level of checks sets the standard for an organization and how they want to determine what passes and what fails in the individual track.

Default Configuration

Prerequisites:

• Soundcheck database is empty (no checks and tracks have been configured before).
• SCM and GitHub integrations are installed and configured.

To add the default initial configuration of GitHub and SCM tracks on startup, the following flags must be set to true in the app-config.yaml file:

soundcheck:
  addStartingConfigurations:
    collectors: true
    checks: true
    tracks: true

• The collectors: true configuration is required to be able to collect the GitHub and SCM facts necessary for the default checks.
• The checks: true configuration is required to create the default GitHub and SCM checks necessary for the default tracks.
• The tracks: true configuration is required to create the default GitHub and SCM tracks.

Note: The configuration will be stored in the database and will be configurable via No-Code UI. Default tracks will be added only if your database is empty (no checks and tracks have been configured before). If you'd like to add this configuration via yaml, you should add the config from the Appendix instead.

Managing tracks via the no-code UI

Creating a new track

To create a track, give your track a meaningful name and a description, and select the filters and checks that you want to use.

Filters allow a user further customization for their tracks to only be applicable to specific entities in the Software Catalog. This allows users to tie their tracks to specific tags such as “Java”, “service” or “production” to name a few. You can have as many or as few selections per type as you want to include in each track. Each type of filter will display a dropdown menu with your choice for selection. You can select any number of filters per section as desired.

Checks are searchable, and can be dragged-and-dropped from the list of checks into the level(s) of the track. Levels can be added to a track with the “+ Add Level” button shown above. Organizing checks into levels allows for a more nuanced read of what checks cleared and which ones didn’t. Most use levels by order of difficulty to clear, where level one means hardest to pass, level two is medium hard to pass, and level three is relatively easy to pass.

Editing a track

Once a track is created, you will be able to manage and edit your track on its detail page. From the tracks listing page, you will see an option to edit your track, which will bring you to the details page shown below.

As with checks, you can make as many changes as desired when editing tracks. Just make sure to save your track in order for changes to take effect. Also, similar to checks,, editing a track and saving will immediately reflect onto anything that has been assigned to said track.

Export to YAML

Checks can be saved as YAML via the dropdown menu on the track cards.

Associated checks are not exported. You must export these seperately and ensure the id fields match.

Managing tracks via yaml configuration

Tracks can be created in code via yaml configuration files.

Overall Shape Of A Track

The following is an example of a descriptor file for a Soundcheck Track:

- id: test-certified
  name: Test Certified
  ownerEntityRef: group:default/example-owner
  description: >
    Improve quality and reliability of your software component by measuring the use of testing best practices.
  documentationURL: https://www.backstage.io
  filter:
    metadata.tags: python
  exclude:
    spec.lifecycle: deprecated
  badge:
    type: medal
  levels:
    - ordinal: 1
      checks:
        - id: python_service_runs_tests
          name: The python service runs pytest
          description: >
            This service is currently running pytest as part of its GitHub actions workflow as defined in the build.yaml
    - ordinal: 2
      checks:
        - id: github_actions_tests_passing
          name: CI/CD Tests passing
          description: >
            The last tests run in GitHub Actions was successful.

See below for details about these fields.

Adding yaml tracks to your track library

To add tracks defined in a yaml file to your track library, you need to source them in you app-config.yaml. You have the option of storing them locally or remotely.

Here is an example configuration should your tracks yaml files be local to your Backstage project:

app-config.yaml

soundcheck:
  programs:
    - $include: ./path-to-local-folder/tracks-file-1.yaml
    - $include: ./path-to-local-folder/tracks-file-2.yaml
    - $include: ./path-to-local-folder/tracks-file-3.yaml

To maintain your tracks files in a remote repo, you can source them in app-config.yaml in a similar manner. Here is an example configuration:

app-config.yaml

soundcheck:
  programs:
    - https://github.com/user/repo/blob/main/tracks-file-1.yaml
    - https://github.com/user/repo/blob/main/tracks-file-2.yaml
    - https://github.com/user/repo/blob/main/tracks-file-3.yaml
  remote_file_updates:
    disabled: false
    frequency:
      minutes: 5

Note: you cannot combine local and remote sourcing for tracks files. This includes the usage of $include within a remotely sourced tracks file. The below option will NOT work:

app-config.yaml

soundcheck:
  programs:
    - https://github.com/user/repo/blob/main/tracks-file-1.yaml
    - https://github.com/user/repo/blob/main/tracks-file-2.yaml
    - $include: ./path-to-local-folder/tracks-file-3.yaml

Remote file update options

The remote_file_updates object is optional configuration allowing you to control if and when Soundcheck looks for updates within your remote files. This configuration is global for Soundcheck, so will apply to both track and check files. If not explicitly set, see the below for default values.

Key	Default	Description
disabled	false	Enables Soundcheck to look for updates in remote files
frequency	minutes: 5	The frequency at Soundcheck will look for updates in remote files. Possible values are either a cron expression { cron: ... } or HumanDuration.

Track Fields

The Track object is composed of some top level summary fields as well as more complex nested fields for filters and levels.

Field	Required	Description
id	Yes	Unique identifier for this particular Soundcheck Track.
name	Yes	Human-readable identifier for this particular Soundcheck Track.
description	Yes	Additional details about the motivation behind this set of checks.
ownerEntityRef	Yes	Relates this Soundcheck Track to the person or team responsible for maintaining it.
documentationURL	No	Link users can follow for more details about this track.
group	No	Connects logically similar but functionally disjoint tracks into groups.
filter	No	Specifies the type of entities that a particular track applies to.
badge	No	Specifies the type of badges that will visualize the statuses of the track and its levels.
levels	Yes	Offers an additional layer of specificity on top of the track definition.
draft	No	A boolean flag that enables / disables a draft mode. Draft tracks are not visible by default.

Level Fields

The level object is composed of several top-level summary fields and a repeated check definition field.

Field	Required	Description
ordinal	Yes	Uniquely identifies a particular level within a track.
name	No	Provides a human-readable title for the level.
description	No	Allows further expression regarding the purpose of a level.
badge	No	Specifies the type of the badge that will vizualize the status of the level.
checks	Yes	Specifies all the checks an entity must pass to be certified at this level.

Check Fields

The check object defines the atomic unit of work within the Soundcheck certification process.

Field	Required	Description
id	Yes	Links this definition to the actual check results.
name	Yes	Provides the human-readable title for this check.
description	Yes	Provides additional context on the motivation and implementation of check.
filter	No	Allows further filtering of checks within a track definition.

Badge Fields

Field	Required	Description
type	No	Specifies the type of the badge (medal or status). Default value is medal.
color	No	Allows to specify a custom HEX color code for a badge with type medal. Default values are bronze (#A36652), silver (#D9D9D9) and gold (#FCE54F).

Filter

filter is used to narrow down applicability for both tracks and checks using catalog filters, with the exception of the CATALOG_FILTER_EXISTS symbol.

See filters for more details.

Draft Tracks

You can now mark tracks as draft. Draft tracks are perfect for testing and verification, allowing you to refine your track before making it visible by default. Draft tracks are hidden from Soundcheck Entity Card, Soundcheck Entity Tab and Soundcheck Overview pages. Besides the visibility, draft tracks function the same as non-draft tracks (applicable entities are certified and certifications are stored in the database).

To view the draft tracks on Soundcheck Entity Tab, click on the small gear icon beside the list of tracks for the component, and select Show Draft Tracks and Campaigns:

Removing the draft flag from a track or setting its value to false will make this track visible on Soundcheck pages listed above.

Integrating with Soundcheck

Soundcheck enables developers to evaluate software components against standards through the creation of tracks, levels, and checks.

Fact Framework

Soundcheck's fact framework collects information about an entity using Fact Collectors or the Facts API.

Fact Collectors

Collector	Description
Catalog	Collects facts from Backstage's Software Catalog as fact data.
Soundcheck	Exposes Soundcheck track certifications as facts.

Third Party Integrations

Integration	Description
Source Control Management (SCM)	Provides integration with source control management providers.
GitHub	Enables gathering facts regarding GitHub repositories.

Fact Checks

Fact checks define rules that determine whether a check result should pass or fail.

Field	Description
id	Unique identifier for this fact check.
name	A name for the fact check suitable for display.
description	A description of the fact check.
rule	Conditions that determine check pass/fail.
schedule	How often and on which entities to run the check.

Executing Checks

Fact checks are executed, or triggered, by Soundcheck in the following ways:

1. A dependent fact is updated, either through fact collectors or via the Facts API.
2. A fact check is scheduled, in which case Soundcheck will automatically execute the check on the specified frequency.
3. A fact check is manually triggered via the Checks API.

REST API

We include a REST API for Tracks. See API Reference for details.

Certifications and Badges

Certification is the outcome of passing all applicable checks within a level. Badges are visualizing certification status for tracks and their levels.

Medal Badges

L1 L2 L3

Medal badges incentivize Engineers to complete levels through Gamification. Component owners can achieve bronze, silver, and gold badges upon completion of related levels. A level is completed when all checks within the level are passing and all previous levels are completed.

Status Badges

Status badges show how many checks are passing within each level and the entire track. A status of one level does not depend on statuses of other levels within the track.

Certification History

By default, Soundcheck retains certification history for entities and their applicable tracks. This feature allows tracking of the highest certified levels for every entity and their applicable tracks over time. The default retention period is 120 days, and is configurable.

Certification history data is used by Soundcheck Tech Health page to visualize snapshots and trends of the highest certified levels for every entity and their applicable tracks. It's also possible for other systems to integrate with certification history by reading from the Soundcheck database directly.

Modifying Certification History Settings

To disable certification history, set soundcheck.certifications.history.enable to false in config:

app-config.yaml

soundcheck:
  certifications:
    history:
      enable: false

Soundcheck only updates history when a certification status changes.

History is retained for a default time of 120 days. You can modify this setting using retentionTimeInDays. For example, this config will instruct Soundcheck to retain history for the past year:

app-config.yaml

soundcheck:
  certifications:
    history:
      enable: true
      retentionTimeInDays: 365

Soundcheck will automatically clean up old certification results when the retention time is reached. The cadence of this cleanup is configurable, as is the time it's allowed to run before being terminated as a hung process. The default values are as follows, and can be changed according to the needs of your organization:

app-config.yaml

  certifications:
    history:
      enable: true
      retentionTimeInDays: 120
      cleanupFrequencyCron: '0 0 0 * * *' // Daily at midnight
      cleanupTimeoutInMinutes: 5

Reading Certification History

Certification history is stored in the certification_history table in the Soundcheck database. Unless overridden in Backstage configuration, the Soundcheck database will be part of the main Backstage database, and named backstage_plugin_soundcheck.

SELECT *
FROM certification_history
WHERE entity_ref = 'component:default/example-component';

Certification History Schema

• certification_id - Unique identifier for the certification (primary key).
• entity_ref - Entity reference that the certification relates to.
• owner_entity_ref [optional] - Software catalog entity that owns the entity this certification is for.
• track_id - Unique identifier for the track this certification is for.
• scope - The scope of check results this certification was created for.
• highest_level [optional] - Rank of the highest certified level. Null if no level is certified.
• certification_date - The date of the certification.
• last_updated - The timestamp indicating the date and time when the certification was last updated.

Appendix

Default GitHub Track

Default GitHub track is based on the default GitHub checks configuration and defined as follows:

id: 140ea084-d918-4d41-9035-95d72516ec4b
name: Recommended GitHub Settings
ownerEntityRef: group:default/backstage
levels:
  - ordinal: 1
    checks:
      - id: 013091a6-2bc8-46dc-8d34-58207c5b7370
        name: Force Pushes Disabled
        description: >-
          Check to ensure that the repository does not allow force pushes to the
          main branch.
        filter:
          - kind:
              - component
      - id: ca05baf0-ff68-4ebd-9559-58a94de84ac3
        name: Pull Request Requires Review
        description: Pull requests should require at least one reviewer.
        filter:
          - kind:
              - component
      - id: b113ad76-bb17-41af-980e-1f515f061004
        name: Deletions Not Allowed
        description: Check to ensure that the repository does not allow deletions.
        filter:
          - kind:
              - component
      - id: 21ed0e9c-7fd9-4e59-89dd-33902b1e6bcb
        name: Branch Is Not Locked
        description: This check ensures that the default branch is not locked.
        filter:
          - kind:
              - component
    name: Minimum
  - ordinal: 2
    checks:
      - id: 0d602bee-f46b-4ff9-8dad-198d82308fa4
        name: Admins Must Follow Rules
        description: >-
          This check that ensures that even administrators cannot bypass branch
          rules.
        filter:
          - kind:
              - component
    name: Recommended
  - ordinal: 3
    checks:
      - id: b812cac0-8f80-4fa0-83cd-6a725c16ac2a
        name: Repository Has Owner
        description: Verifies that the repository has an owner set.
        filter:
          - kind:
              - component
    name: Ownership
description: >-
  This track defines recommended GitHub settings used at Spotify and serves as
  an example track.
filter:
  - kind:
      - component

Default SCM Track

Default SCM track is based on the default SCM checks configuration and defined as follows:

id: 993a9b69-3445-4d42-a950-2a847d618825
name: Basic SCM Compliance
ownerEntityRef: group:default/backstage
levels:
  - ordinal: 1
    checks:
      - id: 8ab4de4f-01f3-46b0-8994-14178614543e
        name: Has API Report
        description: Check to ensure the component has an 'api-report.md' file.
        filter:
          - kind:
              - component
      - id: fe5dbf8f-a7dc-43a0-bdc1-4423c0726649
        name: Has Readme File
        description: Check to ensure the component has a readme file.
        filter:
          - kind:
              - component
      - id: pre-canned-contributions-file-check
        name: Has Contributions File
        description: Check to ensure the component has a CONTRIBUTING.md file.
        filter:
          - kind:
              - component
      - id: pre-canned-license-file-check
        name: Has License File
        description: Check to ensure the component has a license file.
        filter:
          - kind:
              - component
    name: File Existence
  - ordinal: 2
    checks:
      - id: 81c5e85f-cd52-4a0d-a025-2577f4a30db0
        name: API Report File Compliance
        description: >-
          Ensures that the api-report file contains the text "Do not edit this
          file".
        filter:
          - kind:
              - component
    name: File Contents
description: >-
  This track looks at compliance with a few of the basic SCM checks used at
  Spotify.
filter:
  - kind:
      - component