How we measure Backstage proof of value at Expedia Group
- Author
- Sneha Kumar, Expedia Group
- Published
Expedia Group’s (EG) vision is to power global travel for everyone, everywhere. In the quest to do so, we are focusing on simplification and enhancing foundational platform capabilities to empower our developers, suppliers, partners, and travelers.
EG Developer Experience proudly adopted Backstage as our developer front door in August 2020 after a successful proof of concept to evaluate search, technical documentation, and plugin contribution capabilities. An amazing partnership blossomed between Spotify, EG, and the broader adopter community as we evolved Backstage for our internal implementation and contributions to Backstage open source.
An important question emerged as Backstage continues to gain momentum with our 6K+ developer community at EG — how to show the value of Backstage to them in a meaningful way?
Let’s first examine the EG internal developer journey
The EG Developer Experience Team is part of a broader organization called Development Platform & Infrastructure that was chartered to simplify the developer journey.
We kicked off a journey map workshop to refine our developer journey map representing the next 6–12 month horizon. We focused on getting these three concepts right:
- Align our journey map phases to the DevOps process and software development lifecycle
- Identify developer jobs to be done by phase
- Identify metrics by phase
Developer jobs to be done by phase
Our team identified developer jobs to be done (JTBD) by phase to define critical workflows. These jobs to be done follow a user story format. An example JTBD looks like “When onboarding as a developer, I want intuitive search so that I can discover EG Platform content quickly.”
This JTBD exercise enables a common steel thread exercise, which means testing developer JTBD on the critical path with an aim to reduce friction. Our goal is to enable as many jobs to be done as possible across the journey within Backstage.
Journey metrics by phase
We then envisioned a set of overall metrics by journey phase. Example metrics include increasing the percentage of people shipping on day one and reducing the lead time for change during the build, validate, and test phases.
And now where things get exciting — measuring Backstage proof of value
Proof of value fundamentally means answering the “why” for Backstage in a way that resonates with our customers. These metrics are a subset of our overall journey metrics. Our working hypothesis is that increased metrics transparency will be valuable for existing customers and increase willingness for new customers to adopt Backstage and contribute plugins/template actions.
We have oriented our Backstage product roadmap in Q4 (and beyond) to enabling value creation for our customers and simplifying the developer journey by aligning to these metrics.
The proof of value metrics we are starting with:
| Metric | Description | High Level Work Planned in Q4 |
|---|---|---|
| Ship on day one | Time from first commit to deploying app code in production | Add curriculum content to developer bootcamp for new hires to leverage Backstage to create a new “hello world” app in production on day one Partner with EG IT to automate access to common developer-facing tools when developers receive laptop in hand |
| Reduced context switching by increasing number of integrated tools | Measure number of developer-facing tools integrated with Backstage to reduce context switching for developers since they do not need to toggle between multiple tools for developer jobs to be done | Continue plans to integrate with internal developer-facing tools with Backstage in Q4 (e.g., Runtime Compute Platform) |
| Search result relevance | Ranking of relevant content developer is searching for out of all search results Monitored by clickthroughs of search results | Refine proof of concept on search modal to easily view quick results across all content types in Backstage and make contribution back to OS community Enhance user analytics instrumentation for search |
| Technical documentation reach | Number of developers who have viewed a document | N/A – initial reporting provided by current user analytics instrumentation |
We are at the initial phase of establishing a baseline for our Backstage proof-of-value metrics. We look forward to sharing an update with the adopter community in the coming months! In the meantime, if you missed it, you can read about the metrics that Spotify uses to measure Backstage success.