Start Measuring Catalog Maturity in 1 hour with the SCM Compliance Track

Estimated Time: 1 hour

Introduction

Is your organization seeking to automatically measure your organization's catalog maturity without manual spreadsheet tracking? This tutorial is for you!

In this tutorial, we're helping your organization measure catalog maturity through a "best hits playlist" of our most popular checks specifically for SCM tools. We base this track on what we've observed both our internal usage of Soundcheck @ Spotify and across many of our adopters.

Requirements

1. Completed Soundcheck Backend Setup
2. Completed Soundcheck Frontend Setup
3. Completed SCM Fact Collector Prerequisites

Understanding the Basic SCM Compliance Track

NOTE: If you've been using Soundcheck for a while, you may not have the pre-populated Basic SCM Compliance track. We've provided the YAML configuration files below. Please download the files and see our instructions on how to add checks and tracks via YAML.

https://21894833.fs1.hubspotusercontent-na1.net/hubfs/21894833/.COM-resources/basic-scm-compliance-tutorial-files/check-has-api-report.yaml
https://21894833.fs1.hubspotusercontent-na1.net/hubfs/21894833/.COM-resources/basic-scm-compliance-tutorial-files/check-has-readme-file.yaml
https://21894833.fs1.hubspotusercontent-na1.net/hubfs/21894833/.COM-resources/basic-scm-compliance-tutorial-files/check-has-contributions-file.yaml
https://21894833.fs1.hubspotusercontent-na1.net/hubfs/21894833/.COM-resources/basic-scm-compliance-tutorial-files/check-api-report-file-compliance.yaml
https://21894833.fs1.hubspotusercontent-na1.net/hubfs/21894833/.COM-resources/basic-scm-compliance-tutorial-files/basic-scm-compliance-track.yaml
https://21894833.fs1.hubspotusercontent-na1.net/hubfs/21894833/.COM-resources/basic-scm-compliance-tutorial-files/check-has-license.yaml

What is a Track?

Tracks encourage alignment to architectural best practices and standards and are analogous to an organization’s long-term tech health initiatives.

What is a Check?

Checks indicate where the software aligns with or deviates from organizational standards and best practices.

Soundcheck populates a Basic SCM Compliance track with a set of pre-populated checks on the first startup of Soundcheck. It inspires what a real track & set of checks may look like in the wild.

If you're interested in learning more about the shape of a track or check, we recommend checking out the following documentation for further reading:

• Overall Shape of a Track
• Overall Shape of a Check

Ready-to-Go Collector YAML

What is a Collector?

A fact collector collects one or more facts about entities, and Soundcheck can be extended with additional fact collectors. Collectors are responsible for integrating with TPIs or other Backstage plugins to gather information related to Backstage entities in your Software Catalog.

What is a Fact?

Facts represent data related to an entity in your Software Catalog. The data can be formatted in any desired format, defined by the fact collector. Each fact collector will have its fact schema representing what type of facts are returned to the collector.

Initializing the SCM Fact Collector

Here's an example of the SCM Fact Collector configuration.

These files should be copied into your Backstage directory at the following locations at the top of the file, commented out.

# ./packages/backend/src/plugins/soundcheck/collectors/scm-fact-collector.yaml
---
frequency:
  seconds: 15 # recommended a different value for production use
initialDelay:
  seconds: 5
filter:
  kind: component
cache:
  duration:
    hours: 24
collects:
  - factName: required_files_exist
    type: exists
    data:
      - name: readme
        path: /README.md
      - name: license
        path: /LICENSE
      - name: code_of_conduct
        path: /CODE_OF_CONDUCT.md
      - name: api_report
        path: /api-report.md
      - name: contributing
        path: /CONTRIBUTING.md

  - factName: api-report-has-no-edit-warning
    type: regex
    path: api-report.md
    regex: .*Do not edit this file.*

# app-config.yaml
soundcheck:
  results:
    history:
      enable: true
      retentionTimeInDays: 90
      cleanupFrequencyCron: '0 0 0 * * *'
      cleanupTimeoutInMinutes: 5
  certifications:
    history:
      enable: true
      retentionTimeInDays: 90
      cleanupFrequencyCron: '0 0 0 * * *'
      cleanupTimeoutInMinutes: 5
  collectors:
    scm:
      - $include: ./packages/backend/src/plugins/soundcheck/collectors/scm-fact-collector.yaml

Soundcheck scheduling best practices

When you schedule a collector, it collects facts on all entities that match the collector's filter. If the fact for an entity is updated compared to its cached version (or if there is no cached fact, Soundcheck finds all checks that depend on that fact and automatically executes them to produce check results.

When you schedule a check, it executes on all entities that match the check's filter, Soundcheck will request a collection of the required facts on all entities matching that filter if the fact isn't already cached, then execute the check to produce a check result.

Scheduling using collectors is the highly recommended approach. Collector scheduling respects rate-limiting settings whereas check scheduling does not.

Most facts come from a fact collector. The only exceptions to this are the internal-built Catalog facts & Soundcheck certification facts provided natively by the core Soundcheck module. For these facts, users only have the option to schedule checks (which then go fetch relevant facts).

Voilà!

Once the configuration is added, feel free to start Backstage! You'll know you're successful when you visit an entity to see its check results showing pass / fail results in your Soundcheck UI.

Congratulations! You've set up your first SCM Compliance track!

What's next?

Great question! From here, you have a few paths available to you:

• Organizational standards vary. We encourage adopters to use this track as a starting place for adding new facts & checks to enforce technical standards important for their organization. This example provides everything needed to use as inspiration for additional checks & facts.
• This may have inspired ideas for new tracks for other initiatives besides SCM Compliance. We have other tutorials & other excellent documentation for configuring additional fact collectors for integrations with other important integrations your organization cares about.