Skip to main content

SonarQube

The SonarQube integration plugin for Soundcheck supports the extraction of the following facts:

Prerequisites​

Configure SonarQube integration in Backstage​

Integrations are configured at the root level of app-config.yaml. Here's an example configuration for SonarQube:

soundcheck:
  collectors:
    sonarqube:
      token: Token

      # Alternatively you may use username and password
      # username:
      # password:

      # If you wish to override the api URL
      # baseUrl: https://sonarqube.com

Additionally, you can use multiple instances of SonarQube. When using multiple instances, the instance is determined by the sonarqube.org/project-key annotation from the entity's metadata configuration.

soundcheck:
  collectors:
    sonarqube:
      - name: instance1
        baseUrl: http://sonarqube-instance1.example.com # Is not required to be different for each instance
        token: Token
      - name: instance2
        baseUrl: http://sonarqube-instance2.example.com # Is not required to be different for each instance
        token: Token

Add the SonarQubeFactCollector to Soundcheck​

First, add the package:

yarn workspace backend add @spotify/backstage-plugin-soundcheck-backend-module-sonarqube

Then add the following to your packages/backend/src/index.ts file:

packages/backend/src/index.ts
const backend = createBackend();

backend.add(import('@spotify/backstage-plugin-soundcheck-backend'));
backend.add(
  import('@spotify/backstage-plugin-soundcheck-backend-module-sonarqube'),
);
// ...

backend.start();

Consult the Soundcheck Backend documentation for additional details on setting up the Soundcheck backend.

Legacy Backend​

warning

If you are still using the Legacy Backend you can follow these instructions but we highly recommend migrating to the New Backend System.

First add the package: yarn workspace backend add @spotify/backstage-plugin-soundcheck-backend-module-sonarqube

If you are still using the legacy backend, in packages/backend/src/plugins/soundcheck.ts, add the SonarQubeFactCollector:

import { SoundcheckBuilder } from '@spotify/backstage-plugin-soundcheck-backend';
import { Router } from 'express';
import { PluginEnvironment } from '../types';
+ import { SonarQubeFactCollector } from '@spotify/backstage-plugin-soundcheck-backend-module-sonarqube';

export default async function createPlugin(
  env: PluginEnvironment,
): Promise<Router> {
  return SoundcheckBuilder.create({ ...env })
    .addFactCollectors(
+      SonarQubeFactCollector.create(env.logger)
    )
    .build();
}

New Backend System​

Plugin Configuration​

The collection of SonarQube facts is driven by configuration. To learn more about the configuration, consult the Defining SonarQube Fact Collectors section.

SonarQube Fact Collector can be configured via YAML or No-Code UI. If you configure it via both YAML and No-Code UI, the configurations will be merged. It's preferable to choose a single source for the Fact Collectors configuration (either No-Code UI or YAML) to avoid confusing merge results.

No-Code UI Configuration Option​

  1. Make sure the prerequisite Configure SonarQube integration in Backstage is completed and SonarQube instance details are configured.

  2. To enable the SonarQube Integration, go to Soundcheck > Integrations > SonarQube and click the Configure button. To learn more about the No-Code UI config, see the Configuring a fact collector (integration) via the no-code UI.

SonarQube Integration

YAML Configuration Option​

  1. Create a sonarqube-facts-collectors.yaml file in the root of your Backstage repository and fill in all your SonarQube Fact Collectors. A simple example SonarQube fact collector is listed below.

    ---
    token: dummy
    collects:
      - type: projects
        filter:
          - spec.lifecycle: 'production'
            spec.type: 'website'
        cache: false

    Note: this file will be loaded at runtime along with the rest of your Backstage configuration files, so make sure it's available in deployed environments in the same way as your app-config.yaml files.

  2. Add a Soundcheck collectors field to the app-config.yaml and reference the newly created sonarqube-facts-collectors.yaml file.

    # app-config.yaml
    soundcheck:
      collectors:
        sonarqube:
          $include: ./sonarqube-facts-collectors.yaml
    # You can also utilize multiple instances with the Soundcheck collectors:
    # soundcheck:
    #   collectors:
    #     sonarqube:
    #       - $include: ./sonarqube-facts-collectors-instance1.yaml
    #       - $include: ./sonarqube-facts-collectors-instance2.yaml

Rate Limiting (Optional)​

This fact collector can be rate limited in Soundcheck using the following configuration:

soundcheck:
  job:
    workers:
      sonarqube:
        limiter:
          max: 1000
          duration: 60000

In this example, the fact collector is limited to 1000 executions per minute.

This fact collector handles 429 rate limit errors from SonarQube. Soundcheck will automatically wait and retry requests that are rate limited.

Defining SonarQube Fact Collectors​

This section describes the data shape and semantics of SonarQube Fact Collectors.

Overall Shape Of A SonarQube Fact Collector​

The following is an example of a descriptor file for a SonarQube Fact Collector:

---
baseUrl: https://sonarqube.com
token: dummy
collects:
  - type: project
    filter:
      - spec.lifecycle: 'production'
        spec.type: 'website'
    cache: false

Below are the details for each field.

name [optional]​

The unique identifier used by the plugin to determine the instance needed to collect facts based on the sonarqube.org/project-key annotation value in an entity's metadata configuration. If this value is not provided, a default value of sonarqube will be used. Additionally, if the instance names are not unique then the last entry will be the one used to collect the facts about an entity.

baseUrl [optional]​

The base URL of the SonarQube instance to use. If not provided, the plugin will attempt to use the default URL https://sonarqube.com.

token [optional]​

The SonarQube token to use for authentication. If not provided, the plugin will attempt to use username and password instead.

username [optional]​

The SonarQube username to use for authentication. If not provided, the plugin will attempt to use token instead.

password [optional]​

The SonarQube password to use for authentication. If not provided, the plugin will attempt to use token instead.

collects [required]​

An array describing which facts to collect and how to extract them. See below for details about the overall shape of a fact extractor.

Overall Shape Of A Fact Extractor​

Each extractor supports the fields described below.

type [required]​

The type of the extractor (e.g. projects, project-tags, issues).

frequency [optional]​

The frequency at which the fact extraction should be executed. Possible values are either a cron expression { cron: ... } or HumanDuration. If provided, it overrides the default frequency provided at the top level. If not provided, it defaults to the frequency provided at the top level. If neither extractor's frequency, nor default frequency is provided, the fact will only be collected on demand. Example:

frequency:
  minutes: 10

initialDelay [optional]​

The amount of time that should pass before the first invocation happens. Possible values are either a cron expression { cron: ... } or HumanDuration.

Example:

initialDelay:
  seconds: 30

batchSize [optional]​

The number of entities to collect facts for at once. Optional, the default value is 1.

Note: Fact collection for a batch of entities is still considered as one hit towards the rate limits by the Soundcheck Rate Limiting engine, while the actual number of hits will be equal to the batchSize.

Example:

batchSize: 100

filter [optional]​

A filter specifying which entities to collect the specified facts for. Matches the filter format used by the Catalog API. If provided, it overrides the default filter provided at the top level. If not provided, it defaults to the filter provided at the top level. If neither extractor's filter nor default filter is provided, the fact will be collected for all entities.

exclude [optional]​

Entities matching this filter will be skipped during the fact collection process. Can be used in combination with filter. Matches the filter format used by the Catalog API.

filter:
  - kind: component
exclude:
  - spec.type: documentation

cache [optional]​

If the collected facts should be cached, and if so for how long. Possible values are either true or false or a nested { duration: HumanDuration } field. If provided it, overrides the default cache config provided at the top level. If not provided, it defaults to the cache config provided at the top level. If neither extractor's cache nor default cache config is provided, the fact will not be cached. Example:

cache:
  duration:
    hours: 24

Entity configuration​

In your catalog-info.yaml file, add the following metadata annotation to allow the plugin to map an entity to a project in SonarQube.

metadata:
  annotations:
    sonarqube.org/project-key: test-project-key

When using SonarQube Cloud, add the following metadata annotation to allow the plugin to map an entity to an organization in SonarQube Cloud.

metadata:
  annotations:
    sonarqube.org/organization-key: test-organization-key

Collecting Projects Fact​

A Projects Fact contains information about projects from SonarQube Projects API.

Shape of A Projects Fact Collector​

The shape of a Projects Fact Collector matches the Overall Shape Of A SonarQube Fact Collector (restriction: type: projects).

Here's an example of a Projects Fact Collector configuration:

- type: projects
  cache: true
  frequency:
    cron: '0 * * * *'

Shape of A Projects Fact Check​

The shape of a projects Fact Check matches the Shape of a Fact Check.

The following is an example of the projects fact checks:

soundcheck:
  checks:
    - id: requires_project_to_exist
      description: Requires project to exist in sonarqube
      passedMessage: The check has passed!
      failedMessage: The check has failed!
      rule:
        factRef: sonarqube:default/projects
        path: $.paging.total
        operator: greaterThan
        value: 0
      schedule:
        frequency:
          minutes: 1
        filter:
          metadata.tags: some-tag

The following is an example response from the Projects Fact collector. See Sonarqube documentation for more detail.

{
  "paging": {
    "pageIndex": 1,
    "pageSize": 100,
    "total": 2
  },
  "components": [
    {
      "key": "project-key-1",
      "name": "Project Name 1",
      "qualifier": "TRK",
      "visibility": "public",
      "lastAnalysisDate": "2017-03-01T11:39:03+0300",
      "revision": "cfb82f55c6ef32e61828c4cb3db2da12795fd767",
      "managed": false
    },
    {
      "key": "project-key-2",
      "name": "Project Name 2",
      "qualifier": "TRK",
      "visibility": "private",
      "lastAnalysisDate": "2017-03-02T15:21:47+0300",
      "revision": "7be96a94ac0c95a61ee6ee0ef9c6f808d386a355",
      "managed": false
    }
  ]
}

Collecting Project Tags Fact​

A Project Tags Fact contains information about project tags from SonarQube Project Tags API.

Shape of A Project Tags Fact Collector​

The shape of a Project Tags Fact Collector matches the Overall Shape Of A SonarQube Fact Collector (restriction: type: project-tags).

Here's an example of a Project Tags Fact Collector configuration:

- type: project-tags
  cache: true
  frequency:
    cron: '0 * * * *'

Shape of A Project Tags Fact Check​

The shape of a project-tags Fact Check matches the Shape of a Fact Check.

The following is an example of the project-tags fact checks:

soundcheck:
  checks:
    - id: requires_project_tag_to_exist
      description: Requires project tag to exist
      passedMessage: The check has passed!
      failedMessage: The check has failed!
      rule:
        factRef: sonarqube:default/project-tags
        path: $.tags
        operator: hasLengthOf
        value: 1
      schedule:
        frequency:
          minutes: 1
        filter:
          metadata.tags: some-tag

The following is an example response from the Project Tags Fact collector. See Sonarqube documentation for more detail.

{
  "tags": ["official", "offshore", "playoff"]
}

Collecting Issues Fact​

An Issues Fact contains information about issues from SonarQube Issues API.

Shape of an Issues Fact Collector​

The shape of an Issues Fact Collector matches the Overall Shape Of A SonarQube Fact Collector (restriction: type: issues).

Here's an example of an Issues Fact Collector configuration:

- type: issues
  cache: true
  frequency:
    cron: '0 * * * *'

Shape of an Issues Fact Check​

The shape of a issues Fact Check matches the Shape of a Fact Check.

The following is an example of the issues fact checks:

soundcheck:
  checks:
    - id: requires_to_have_no_issues
      description: Requires to have no issues
      passedMessage: The check has passed!
      failedMessage: The check has failed!
      rule:
        factRef: sonarqube:default/issues
        path: $.paging.total
        operator: equal
        value: 0
      schedule:
        frequency:
          minutes: 1
        filter:
          metadata.tags: some-tag

The following is an example response from the Issues Fact collector. See Sonarqube documentation for more detail.

{
  "paging": {
    "pageIndex": 1,
    "pageSize": 100,
    "total": 1
  },
  "issues": [
    {
      "key": "01fc972e-2a3c-433e-bcae-0bd7f88f5123",
      "component": "com.github.kevinsawicki:http-request:com.github.kevinsawicki.http.HttpRequest",
      "project": "com.github.kevinsawicki:http-request",
      "rule": "java:S1144",
      "cleanCodeAttribute": "CLEAR",
      "cleanCodeAttributeCategory": "INTENTIONAL",
      "issueStatus": "ACCEPTED",
      "impacts": [
        {
          "softwareQuality": "SECURITY",
          "severity": "HIGH"
        }
      ],
      "message": "Remove this unused private \"getKee\" method.",
      "messageFormattings": [
        {
          "start": 0,
          "end": 4,
          "type": "CODE"
        }
      ],
      "line": 81,
      "hash": "a227e508d6646b55a086ee11d63b21e9",
      "author": "Developer 1",
      "effort": "2h1min",
      "creationDate": "2013-05-13T17:55:39+0200",
      "updateDate": "2013-05-13T17:55:39+0200",
      "tags": ["bug"],
      "comments": [
        {
          "key": "7d7c56f5-7b5a-41b9-87f8-36fa70caa5ba",
          "login": "john.smith",
          "htmlText": "Must be &quot;public&quot;!",
          "markdown": "Must be \"public\"!",
          "updatable": false,
          "createdAt": "2013-05-13T18:08:34+0200"
        }
      ],
      "attr": {
        "jira-issue-key": "SONAR-1234"
      },
      "transitions": ["reopen"],
      "actions": ["comment"],
      "textRange": {
        "startLine": 2,
        "endLine": 2,
        "startOffset": 0,
        "endOffset": 204
      },
      "flows": [
        {
          "locations": [
            {
              "textRange": {
                "startLine": 16,
                "endLine": 16,
                "startOffset": 0,
                "endOffset": 30
              },
              "msg": "Expected position: 5",
              "msgFormattings": [
                {
                  "start": 0,
                  "end": 4,
                  "type": "CODE"
                }
              ]
            }
          ]
        },
        {
          "locations": [
            {
              "textRange": {
                "startLine": 15,
                "endLine": 15,
                "startOffset": 0,
                "endOffset": 37
              },
              "msg": "Expected position: 6",
              "msgFormattings": []
            }
          ]
        }
      ],
      "quickFixAvailable": false,
      "ruleDescriptionContextKey": "spring",
      "codeVariants": ["windows", "linux"]
    }
  ],
  "components": [
    {
      "key": "com.github.kevinsawicki:http-request:src/main/java/com/github/kevinsawicki/http/HttpRequest.java",
      "enabled": true,
      "qualifier": "FIL",
      "name": "HttpRequest.java",
      "longName": "src/main/java/com/github/kevinsawicki/http/HttpRequest.java",
      "path": "src/main/java/com/github/kevinsawicki/http/HttpRequest.java"
    },
    {
      "key": "com.github.kevinsawicki:http-request",
      "enabled": true,
      "qualifier": "TRK",
      "name": "http-request",
      "longName": "http-request"
    }
  ],
  "rules": [
    {
      "key": "java:S1144",
      "name": "Unused \"private\" methods should be removed",
      "status": "READY",
      "lang": "java",
      "langName": "Java"
    }
  ],
  "users": [
    {
      "login": "admin",
      "name": "Administrator",
      "active": true,
      "avatar": "ab0ec6adc38ad44a15105f207394946f"
    }
  ],
  "facets": [
    {
      "property": "cleanCodeAttributeCategories",
      "values": [
        {
          "val": "INTENTIONAL",
          "count": 6912
        },
        {
          "val": "CONSISTENT",
          "count": 4950
        },
        {
          "val": "ADAPTABLE",
          "count": 899
        },
        {
          "val": "RESPONSIBLE",
          "count": 27
        }
      ]
    },
    {
      "property": "impactSeverities",
      "values": [
        {
          "val": "HIGH",
          "count": 435
        },
        {
          "val": "LOW",
          "count": 7858
        },
        {
          "val": "MEDIUM",
          "count": 4495
        }
      ]
    }
  ]
}

Shape of A SonarQube Collector Track​

The following is an example of the Soundcheck track that utilizes these checks

soundcheck:
  programs:
    - id: demo
      name: Demo
      ownerEntityRef: group:default/owning_group
      description: Demonstration of Soundcheck SonarQube Fact Collector
      levels:
        - ordinal: 1
          name: First level
          description: Checks leveraging SoundCheck's DataDog Fact Collector
          checks:
            - id: requires_to_have_no_issues
              name: Requires to have no issues
              description: Requires to have no issues
            - id: requires_project_tag_to_exist
              name: Requires project tag to exist
              description: Requires project tag to exist
            - id: requires_project_to_exist
              name: Requires project to exist in sonarqube
              description: Requires project to exist in sonarqube
      filter:
        catalog:
          metadata.tags: some-tag

Collecting Measures Fact​

A measures fact contains information about measures from SonarQube Measures API.

Shape of A Measures Fact Collector​

The shape of a Measures Fact Collector matches the Overall Shape Of A SonarQube Fact Collector ( restriction: type: measures).

The following is an example of the Measures Fact Collector config:

- type: measures
  cache: true
  frequency:
    cron: '0 * * * *'

Configure metric keys​

A Measures fact can be configured to which metrics should be fetched. By default, it will fetch the following metric keys:

  • ncloc
  • complexity
  • violations
  • open_issues

To configure it, add following section in your YAML configuration. Note the metrics will completely override the default ones, so you need to include them if you wish to fetch them as well.

soundcheck:
  collectors:
    sonarqube:
      collects:
        - factName: extendedMeasures
          type: measures
          metrics:
            - ncloc
            - complexity
            - violations
            - open_issues
            - bugs
            - code_smells

You can see a full list of available metrics in your instance with the following HTTP request:

GET https://sonarqube.com/api/metrics/search?ps=500
Authorization: Token
Accept: application/json

Shape of A SonarQube Track​

The following is an example of the Soundcheck track that utilizes these checks

soundcheck:
  programs:
    - id: demo
      name: Demo
      ownerEntityRef: group:default/owning_group
      description: Demonstration of Soundcheck SonarQube Fact Collector
      levels:
        - ordinal: 1
          name: First level
          description: Checks leveraging SoundCheck's DataDog Fact Collector
          checks:
            - id: requires_to_have_no_issues
              name: Requires to have no issues
              description: Requires to have no issues
            - id: requires_project_tag_to_exist
              name: Requires project tag to exist
              description: Requires project tag to exist
            - id: requires_project_to_exist
              name: Requires project to exist in sonarqube
              description: Requires project to exist in sonarqube
      filter:
        catalog:
          metadata.tags: some-tag