Setup & Installation
Prerequisites
Configure the Sign-In Resolver
To allow Insights to identify the user entity associated with the signed-in user, your SignInResolver must issue an identity token with a sub (subject) claim pointing to the user entity in the catalog corresponding to the signed-in user. Check out Backstage user identity documentation on backstage.io for complete details on how to supplly an identity resolver.
While many authentication providers handle this automatically, if you're using a custom provider, you might need to implement this behavior yourself. Refer to examples in the Google auth sign-in resolver or the GitHub auth sign-in resolver. For more information on configuring a sign-in resolver, see the identity resolver documentation on backstage.io.
Backend Installation
Obtain the Plugin
Add the Insights backend as a dependency to your Backstage backend app:
yarn workspace backend add @spotify/backstage-plugin-insights-backend
Integrate the Plugin
Create a new file named packages/backend/src/plugins/backstageInsights.ts
with the following content:
import { Router } from 'express';
import { PluginEnvironment } from '../types';
import { createRouter } from '@spotify/backstage-plugin-insights-backend';
export default async function createPlugin(
env: PluginEnvironment,
): Promise<Router> {
return await createRouter({
...env,
});
}
Next, wire up the Insights backend in packages/backend/src/index.ts
. Import the module from the previous step, create a plugin environment, and add the router to the Express app.
...
import proxy from './plugins/proxy';
import techdocs from './plugins/techdocs';
import search from './plugins/search';
import backstageInsights from './plugins/backstageInsights';
...
const techdocsEnv = createEnv('techdocs');
const searchEnv = createEnv('search');
const appEnv = createEnv('app');
const backstageInsightsEnv = createEnv('backstageInsights');
...
apiRouter.use('/techdocs', await techdocs(techdocsEnv));
apiRouter.use('/proxy', await proxy(proxyEnv));
apiRouter.use('/search', await search(searchEnv));
apiRouter.use('/backstage-insights', await backstageInsights(backstageInsightsEnv));
Note: be sure to use "backstage-insights" exactly as your route in the above file
New Backend System
If you are using the New Backend System, you can integrate the Insights backend plugin with your Backstage backend like this:
import { createBackend } from '@backstage/backend-defaults';
const backend = createBackend();
backend.add(import('@backstage/plugin-app-backend/alpha'));
backend.add(import('@spotify/backstage-plugin-insights-backend'));
// ...
backend.start();
Create an API Token for Your Organization
To secure communication between your Backstage backend and Spotify's servers, you need to create an API token. Visit backstage.spotify.com/account/tokens and create a token with a descriptive name, such as backstage-insights-production-token
. Note down the token value as you'll need it in the next step.
Configure the Spotify Backstage API Token
In your app-config.yaml
, add the following entry to enable communication with Spotify's servers:
spotify:
apiToken: ${SPOTIFY_BACKSTAGE_API_TOKEN}
Set the SPOTIFY_BACKSTAGE_API_TOKEN
environment variable to the API token you created earlier. Ensure that you treat this value as a secret and avoid checking it into version control.
Additional Configuration
Extra Entity Metadata Capture
The Insights backend integrates with the Software Catalog to enhance analytics events with additional entity metadata. This allows filtering or grouping data based on attributes like the entity's type
. By default, the following fields are captured on analytics events associated with catalog entities: kind
, metadata.namespace
, spec.type
, spec.lifecycle
, and spec.owner
. To capture extra fields, provide them in the configuration. Common fields can be found in the Catalog's YAML File Format documentation. You can also reference custom metadata stored on entities.
backstageInsights:
captureEntityFields:
- spec.someCustomField
Similarly, certain fields are captured on all analytics events related to the signed-in user, such as metadata.namespace
and spec.memberOf
. You can configure additional user metadata capture in the configuration as well:
backstageInsights:
captureIdentityFields:
- spec.profile.someCustomField
Note that the Insights backend applies redaction logic to avoid collecting personal data, but you should understand the data collected when configuring these additional fields.
Identity groups filter
By default, membership groups for users are captured with analytic events, which are then used in the frontend to filter and segment the data on dashboards. However, some groups may not be logical representations of groups that you'd want to segment by. As an example, you may populate these groups from an external source that is used for application management, and therefore have no relation to Backstage usage.
For this reason, we provide a captureIdentityGroupsFilter config option to specify which types of groups you want to capture.
Some examples:
- Only capture specific group types
backstageInsights:
captureIdentityGroupsFilter:
'spec.type':
- department
- guild
- team
- Only capture groups annotated with an insights flag
backstageInsights:
captureIdentityGroupsFilter:
'metadata.includeInInsights': true
Total Users Count Filter
The Insights frontend uses the Software Catalog to determine the total number of users who can access Backstage. This number is essential for various dashboards and views. If the count is incorrect due to factors like retaining deactivated users or gradual rollouts, you can apply filters using an EntityFilterQuery. Examples include counting members of a specific group or users under a certain namespace.
backstageInsights:
totalUsersEntityQueryFilter:
'relations.memberOf': 'group:default/rnd'
For more details, refer to the EntityFilterQuery documentation on backstage.io.
Host Filter
While it's recommended that, in local and pre-production environments, you set backstageInsights.debug
to true
, there may be a variety of scenarios where that's not possible or practical to standardize. You can set a host filter to ensure that dashboards only display data that's been collected from your production environment.
For instance, if your production Backstage app is accessible to employees at backstage.acme.net
, you could add the following in your app-config.yaml
:
backstageInsights:
hostFilter: backstage.acme.net
Frontend Installation - Adding the Insights API
Get the Plugin
Add the Insights Analytics Module as a dependency to your Backstage frontend app:
yarn workspace app add @spotify/backstage-plugin-analytics-module-insights
Supply the BackstageInsights API implementation in your app
This is typically done in /packages/app/src/apis.ts
, or less often directly in the call to createApp()
.
...
import { BackstageInsights } from '@spotify/backstage-plugin-analytics-module-insights';
export const apis = [
...
BackstageInsights.createDefaultApiFactory(),
];
Configure the Backstage Insights API
Generate a random string, for example with the following terminal one-liner:
node -p 'require("crypto").randomBytes(24).toString("base64")'
Then add the following configuration to your app-config.yaml
, substituting the random string you just generated for the salt
value.
backstageInsights:
salt: randomly-generated-string-here
debug: true
The debug
property is set to true
, which will output data to local logs instead of sending it to the production API. Don't forget to supply an override in your app-config.production.yaml
:
backstageInsights:
debug: false
Note: The backstageInsights.salt value is used to hash user identifiers in the insights event data that is sent to Spotify. This value is secret to Spotify (meaning: Spotify should never have access to this value), but it does not need to be treated as a secret within your organization (meaning: feel free to check this value in).
Check everything is working
Start up backstage locally. For the sake of testing, you could provide any necessary overrides as environment variables inline, like this:
SPOTIFY_BACKSTAGE_API_TOKEN="{your token}" \
APP_CONFIG_backstageInsights_debug=false \
yarn dev
Once the app is built and running, open the app in your browser and open the browser's network console; check for requests being made against the insights backend with a path containing backstage-insights/events
. As you navigate around Backstage, those requests should receive a 204
response if everything is configured correctly.
Once you've verified your configuration locally, deploy your changes to production (ensuring that you properly set the SPOTIFY_BACKSTAGE_API_TOKEN
variable in your production environment). Perform the same network request checks in your production Backstage.
Please note that data is processed in batches with a slight delay. The dashboards only show full complete days so events from the present day are not displayed.
Troubleshooting
If you're not seeing what's described in the Check everything is working section
, here are a few common misconfigurations to be aware of:
- Are you seeing a
401
or404
response instead of a204
? There are two key items to check:- Check that you've set the
SPOTIFY_BACKSTAGE_API_TOKEN
environment variable as expected. - Be sure that you set the path correctly when you wired up the Insights backend. It should be
/backstage-insights
, as inapiRouter.use('/backstage-insights', /* etc */)
.
- Check that you've set the
- Are you not seeing any matching network requests at all? Check that you've set the
backstageInsights.salt
value as expected. A missing value could result in the analytics API instantiation failing silently. It's also possible thatbackstageInsights.debug
is mistakenly set totrue
(if this is the case, you'll see backstageInsights logs in your browser's console log). Make sure that your app is loading in all relevant configuration files (app-config.yaml
as well asapp-config.production.yaml
). - Are you seeing an error similar to
tried to register duplicate or forbidden API factory
when you load the frontend? This is likely because you already have an analytics implementation supplied in yourapis.ts
file. To resolve, you can use theMultipleAnalyticsApi
class to supply your existing analytics implementation alongside Backstage Insights. In place ofBackstageInsights.createDefaultApiFactory()
, you could do something similar to the following:
import {
analyticsApiRef,
configApiRef,
createApiFactory,
discoveryApiRef,
fetchApiRef,
identityApiRef,
storageApiRef,
} from '@backstage/core-plugin-api';
import { MultipleAnalyticsApi } from '@backstage/core-app-api';
import { BackstageInsightsApi } from '@spotify/backstage-plugin-analytics-module-insights';
import { YourExistingAnalyticsApi } from '@somewhere/else';
export const apis = [
...
createApiFactory({
api: analyticsApiRef,
deps: {
configApi: configApiRef,
discoveryApi: discoveryApiRef,
fetchApi: fetchApiRef,
identityApi: identityApiRef,
storageApi: storageApiRef,
},
factory: ({
configApi,
discoveryApi,
fetchApi,
identityApi,
storageApi,
}) =>
MultipleAnalyticsApi.fromApis([
BackstageInsightsApi.fromConfig(configApi, {
discoveryApi,
fetchApi,
identityApi,
storageApi,
}),
YourExistingAnalyticsApi.fromConfig(configApi, {
...
}),
]),
...
}),
];
Frontend Installation - Adding the UI
Get the Plugin
Add the Insights frontend as a dependency to your Backstage frontend app:
yarn workspace app add @spotify/backstage-plugin-insights
Install the Insights frontend
Wire up routing to the InsightsPage
entry page. The route should look something like this:
import {
InsightsPage,
insightsPlugin,
InsightsSurveyLoader,
} from '@spotify/backstage-plugin-insights';
...
/* Configure the route location for the Search plugin inside bindRoutes */
bindRoutes({ bind }) {
...
bind(insightsPlugin.externalRoutes, {
searchPage: searchPlugin.routes.root,
});
...
}
...
const routes = (
<FlatRoutes>
<Navigate key="/" to="catalog" />
...
<Route path="/insights" element={<InsightsPage />} />
...
</FlatRoutes>
);
/* Add the SurveyLoader, which renders configured pop-up surveys */
export default app.createRoot(
<>
<AlertDisplay />
<OAuthRequestDialog />
<AppRouter>
<InsightsSurveyLoader />
<Root>{routes}</Root>
</AppRouter>
</>,
);
Add a sidebar menu item that routes to the path setup in the previous step:
...
import { InsightsSidebarItem } from '@spotify/backstage-plugin-insights'
...
<SidebarScrollWrapper>
<SidebarItem icon={MapIcon} to="tech-radar" text="Tech Radar" />
...
<InsightsSidebarItem />
...
</SidebarScrollWrapper>;
...
Configure Access Controls
The Insights plugin integrates with Backstage's permission framework and the RBAC plugin. This integration enables restricting which users/groups can view Insights dashboards as well as Create, Read, Update, or Delete (CRUD) Insights surveys and responses.
If you currently permission access to catalog resources, you'll likey want to configure who can access Insights as it could potentially expose information from the catalog that users might not normally have permissions to see.
The recommended approaches to setting access controls for this plugin require the Backstage permissions framework. You may have set up this up already when configuring other plugins from the Spotify bundle, but if not, follow this guide to setting up the permissions framework.
Option 1: Use the RBAC Plugin (Recommended)
"The RBAC plugin is a no-code management UI for restricting access to plugins, routes, and data within Backstage. Admins can quickly define roles, assign users and groups, and configure permissions to encode authorization decisions according to your organization's evolving security and compliance needs."
To have access to Insights permissions within the RBAC UI, be sure to add backstage-insights
to the list of permissionedPlugins
in your app-config.yaml
.
permission:
enabled: true
permissionedPlugins:
- catalog
- scaffolder
- backstage-insights
...
Using the RBAC plugin, it's easy to define access to Insights by configuring Allow
for all of the Insights permissions:
See the Available Permissions section for details on what the permissions entail.
Here is a basic sample any-allow
policy that allows general access to the catalog and templates but restricts access to the Insights plugin to the group backstage-admins
:
name: Sample Insights Policy
description: null
options:
resolutionStrategy: any-allow
roles:
- name: Default user
members: '*'
permissions:
- match:
resourceType: scaffolder-action
decision: allow
- match:
resourceType: scaffolder-template
decision: allow
- match:
resourceType: catalog-entity
decision: allow
- name: Admin
members:
- group:default/backstage-admins
permissions:
- match: '*'
decision: allow
To get started with this sample policy, save the content as a
yaml
file and follow the RBAC import policy instructions.
Option 2: Use Custom Policy Handlers
If you don't use the RBAC plugin, you can define custom policy handlers for permission framework policies.
class ExamplePolicy implements PermissionPolicy {
async handle(request: PolicyQuery): Promise<PolicyDecision> {
if (request.permission.name === 'insights.read') {
const isAllowed = await someEvaluationFunction();
if (isAllowed) {
return {
result: AuthorizeResult.ALLOW,
};
}
}
return { result: AuthorizeResult.DENY };
}
}
Integrate the Plugin with Your Frontend App
Install the Insights routes in your app within packages/app/src/App.tsx
and configure the route location for the Search plugin. You can also add a sidebar item for easy access to Insights if needed.
Verify Functionality
After setting up access controls, users with the insights.read
permission should see an Insights sidebar menu item. Ensure that the Backend plugin is correctly configured and data has been collected for at least 48 hours for the dashboards to display meaningful information.
Available Permissions
The Insights plugin has the following permissions that can be used to control access to the plugin and its features:
Permission | Description |
---|---|
insights.read | Allows a user to access the Insights UI, view dashboards and data. This is also required in tandem with all of the following permissions |
insights.survey.read | Allows a user to view Insights surveys and any associated responses |
insights.survey.update | Allows a user to edit or close an existing Insights survey |
insights.survey.delete | Allows a user to delete an existing Insights survey and all associated responses |
insights.survey.create | Allows a user to create an Insights survey via the Insights Feedback UI. Also allows a user to duplicate an existing survey |