Troubleshooting

This section covers the most common problems our first time adopters face and how to troubleshoot them. This guide covers the Open Source Backstage + Soundcheck and is not intended for Portal + Soundcheck (although some answers may be the same).

Why isn't my check running?

See the answer

There are 3 ways to make a check run. For testing your check you can use option 3, for a sustainable check execution use option 1.

NOTE: Executing a check is NOT the same as collecting a fact. Fact collection is what fetches data from an API and is subject to caching. Check execution is simply looking at that data and verifying if it satisfies the check criteria. However, after a new fact is collected the checks will execute.

Option 1 - Schedule fact collection (Recommended)

Each integration has its own documentation page - Github example here. In most cases to schedule a fact collection you just need to add following section to your YAML but please refer to the specific fact collector documentation. Once a new fact is collected, soundcheck will trigger all reliant checks automatically, so you don't need to schedule them separately.

---
frequency:
cron: '0 * * * *'

If you are using the No-Code UI, some integration can be configured to collect facts. Go to Soundcheck -> Integrations and then click on Configure next to the collector you want to configure. From there you can schedule fact collection like on the screenshot below.

NOTE: If the Configure button is disabled or not present that means you either configured the integration from the YAML or that this integration does not support No Code UI yet.

Option 2 - Schedule check (Not recommended)

We recommend you use the first option when possible. However, for some integrations, for example catalog, you will need to schedule the check instead. This is described in checks section here.

Option 3 - Manually run check (Recommended for testing only)

To manually trigger a check to run, navigate to a component page, next click on Soundcheck tab and then click the Execute Check button.

Why don't I see any history for my checks or certifications?

See the answer

In new versions, the history is enabled by default however you need to make sure you haven't disabled history.

I am seeing rate limit issues, what can I do?

See the answer

There is a couple of things to try when faced with rate limits.

Check your frequency settings

Depending on your needs you may tweak the frequency of the fact collection to reduce the number of requests being made. The trade off here is you will have data that is potentially stale. Refer to the specific collector on how to change the frequency. For example this is how you do it in Github Collector

Enable cache for the fact collector

Refer to the specific collector on how to do that. For example this is how you do it for Github Collector

Enable Rate limiting

You can enable limiting the amount of requests that soundcheck will make from a specific collector. It is highly recommended to also enable redis queues so that the limits apply across all pods.

Use API to submit check results

Normally soundcheck works in pull model, where it fetches facts on a schedule and then executes checks. Soundcheck also has an API that enables it to work in push model where other systems such as CI pipelines can push data to soundcheck.

You can either push new facts or push check results.

I have a lot of catalog entities and my Backstage instance is now slowing down, what can I do?

See the answer

Refer to the Scaling Backstage documentation.

Why can't I use all the filters available in YAML in the No Code UI?

See the answer

We are working on bringing the functionality parity to the No code UI. Check our release notes to see latest improvements.

Why am I getting errors with Isolated VM?

See the answer

Isolated VM version 5.0.1 works with node version 22. Isolated VM version 4.7.2 works with node version 20. Isolated VM version 4.6.0 works with node version 18.

If you're having issues with Isolated VM errors, check the version of Node you're using, and manually pin the version of Isolated VM in your package.json to the appropriate version.