We all experienced this situation in a meeting :
Manager: " How many active users do we have this month? "
Data Analyst: "I have 35k monthly active users according to the marketing dashboard."
Data Scientist: "Hum, well, I just ran a quick query and found that there is only 15k monthly active users."
Manager: "Ok, then, who can I trust?"
This situation is really painful, yet so common. Having multiple business/technical definitions for the same KPI (Key Performance Indicator) happens in every company.
KPIs are impactful numbers, used as management tools to improve decision making. KPIs are helpful in understanding if you’re hitting your business objectives, improving over time, and helping to forecast future growth.
KPIs need to be crystal clear to the whole company. KPIs need to be uniquely defined both in a technical way (query) and business way (classic definition).
As the complexity and the size of your company grows, the number of key metrics and dimensional cuts teams track grows, as well. Tribal knowledge of important data does not scale to large teams, and all teams may not have the data engineering skills to build and maintain pipelines.
KPI and business metrics can be hard to build, maintain, and consume:
Working as a data scientist at Ubisoft, I realized that most metrics are, in the end, dimensional cuts of "Master Metrics". Let's take the example of Uber. The Master Metrics can be "rides" or "waiting time". Those Master Metrics can then be cut into dimensions:
To ensure a proper metric/KPI management, providing clear definitions of Master Metrics and Dimensions is essential. Those definitions need to be both technical and business-oriented. They need to be actionable: you can copy/paste the query in the SQL editor to get the metric results.
This is why we built, Castor, a product to do just that (among other things). Castor lets you reference your own set of Master Metric and attach frequently used dimensions. We always recommend adding the code used to build the metric. Castor also enables data people to attach metric definitions to specific tables or dashboards.
You feel that metric management is a challenge in your team? To test Castor, just send me an email: xavier@castordoc.com