CastorDoc and BigQuery

Google's serverless, highly scalable enterprise data warehouse, is designed to make data analysts more productive.

integration mockup

Why BigQuery and Castor?

BigQuery is Google Cloud's fully managed, petabyte-scale, and cost-effective analytics data warehouse that lets you run analytics over vast amounts of data in near real-time.

Your might have hundreds or thousands of data tables on BigQuery. Discovery, documentation, ownership and governance is hard with fast growing data assets and data teams. Castor is a plug-and-play layer that help your data team power self-service BI.

What does Castor help you with?

Castor enables you to scale your self-service analytics strategy without losing control. We are designed with real use-cases in mind :

🔎   You work with data you don't know

Your boss asks to build a report on "Churn for Premium Users in 2021". You need to find the relevant dataset, understand the meaning of its column, and use it fast.

✅ Reduce by 95% the time to find the right data asset (source : Lyft)

🧬   A key employee is leaving

Mike, the data engineer that built the entire data infrastructure is leaving at the end of the month. All the knowledge is in his head. He needs to write it down.

✅ 42% of the work not recovered without knowledge management (source : 360Learning)

👩🏽‍🌾  A new employee onboarding

Elsa, data analyst, arrived last week. She has no idea what data the company stores or how it is used. She spends hours asking around to gather knowledge.

✅ New hires are autonomous after day 1

💣  A data pipeline is late

Nelson, customer success analyst, refreshes the "daily active users" dashboard every two minutes. The data hasn't arrived yet. He wants to know what is happening.

✅ 5x less Slack messages on #ask_data

🗺️  No one knows where personal information are

Camila, from data governance, has to map all personal information to comply to GDPR requirements. She needs a list of all data assets and their location.

✅ 70% of employees have access to data that they shouldn’t (source)

BigQuery Set Up

You need to have the needed permissions to create a Google Service Account for Castor. Otherwise refer to someone in your organisation with these permissions.

Client creates a service account for Castor from the Google Console. See how to here

Client grants the needed access to this new service account with the following roles. See how to here

Bigquery Read Session User

Why: To access the cluster via SQL. We use BigQuery sessions to read metadata. This requires the sessions.* permissions. readSessionUser allows read capabilities via SQL yet does not grant by itself any data access.

Bigquery Metadata Viewer

Scope: for all the projects where we need to access metadata to be able to fetch the schemas of your data

Why: to fetch the schemas of your data

Bigquery Job User

Scope: for all the projects where we need to fetch the query log to be able to parse queries and compute lineage and usage stats

Why: to parse queries and compute lineage and usage stats

Bigquery Resource Viewer

Scope: for all the projects where we need to fetch query log for the role which is necessary to access query logs ()

Why: to parse queries and compute lineage and usage stats

These roles only allow Castor to see the data model, the queries and the users, by running queries on the metadata. It is the minimum roles allowing us to read your metadata while not being able to read your data.


snowflake icon
redshift icon
bigquery icon
synapse icon
postgreSQL icon
mysql icon
databricks icon
dbt icon
looker icon
tableau icon
powerbi icon
slack icon

Get in Touch to Learn More

See Why Users Love CastorDoc
Fantastic tool for data discovery and documentation

“[I like] The easy to use interface and the speed of finding the relevant assets that you're looking for in your database. I also really enjoy the score given to each table, [which] lets you prioritize the results of your queries by how often certain data is used.” - Michal P., Head of Data