As you read the title, you probably thought: “Great, and another job hits the stack: what on earth is an Analytics Engineer now?”
You almost kept scrolling, but if you made it up to here, it’s probably because the weird juxtaposition of “analytics” next to “engineer” caught your attention. And rightly so: for many of us analysts are not exactly highly technical engineering roles and vice versa. But let’s face it: the world has jumped forward and today, Python, R and SQL are basic requirements to many jobs. The worlds of engineering and analytics are slowly merging to become one, and today we want to discuss the intersection of both areas: here sits the Analytics Engineer.
And yes, we agree with you, these days it seems that any excuse is a good one to create “new, essential tech jobs”, making it all so overwhelming to keep track of, but more seriously it also gives the impression that these roles are all very much Fugazi, Foogazi
But this time, everything seems to indicate that the Analytics Engineer is here to stay.
Here is the deal:
Analytics has developed from a support to an advisory role, taking a driver seat in product innovations (setting and determining product KPIs, roadmaps, etc.). On the other hand, the data engineering role can, to a large extent, be automated.
Why does it matter?
Because whether you have a handful of data engineers or only a few data analysts with some coding skills on your team*,* when these worlds converge, you will need Analytics Engineers to act as the bridge between the producers and the consumers of your data.
In the following article, we’ll cover the rise of the Analytics Engineer, we’ll define the key responsibilities of Analytics Engineers and guide you in identifying your next Analytics Engineer. All of this to ultimately help you understand whether your organization needs one.
To grasp why Analytics Engineers are here to stay, we must understand where it all started. Originally coined two years ago, by Michael Kaminsky, the term came from the ground-up, when data people experienced a shift in their job: they went from handling data engineer/scientist/analyst’s tasks to spending most of their time fixing, cleaning, and transforming data. And so, they (mainly members of the dbt community) created a terminology to describe this middle seat role: the Analytics Engineer. The real question becomes: How long ago and why did we experience this?
Three big changes in the data tooling stack are at the core of the development of the Analytics Engineer:
These shifts made access to data easier than ever. Every company now collects, tracks, and reports large amounts of data on everything but also on anything... This explosion of data comes with the cost of extracting the relevant data. To achieve this, companies must be sure to build the right data teams.
Today, the distinct jobs that existed in traditional data teams have converged: the line between the roles of data engineering, data science, and data analysis is blurry. But we cannot expect data scientists or analysts to be great working software engineers and vice-versa. Although this is exactly what we see happening…
Today, most data teams resemble something like this:
The gaps mentioned above are exactly where Analytics Engineers walk in.
An Analytics Engineer builds, transforms, tests and deploys models that support the queries of the rest of the data team. The objective of the job is to enable data products to talk to data consumers: the Analytics Engineer stands as the technical moderator of business and technical teams. It all comes down to this: Analytics Engineers allow their teams to navigate quickly the modern stack, all while leveraging the right data to answer their needs.
At the intersection of business, data engineering and analytics, the Analytics Engineer travels back to SQL land to bring data models to life. In their day-to-day job Analytics Engineers will:
Obviously, as the modern stack continues its development evolve, we are not done figuring out how to use all these new tools, who should use them, how we should report around them, etc. It only makes sense that the exact responsibilities of Analytics Engineers are still in the works. But the skillset required for such a role is straightforward.
The next Analytics Engineer at your company may have a ton of software engineering skills or may very well be proficient in only one scripting language. What is important is that the Analytics Engineers bring these two worlds together. They won’t be the best software engineer out there, nor will they be the business go-to deal marker. And they really shouldn’t be: all the value they will add comes from their ability to blend technical and domain expertise in their daily tasks.
So how do you identify a good analytics engineer? Look at the skill sets.
Your Analytics Engineer should have:
One interesting point was put forward: soft skills are almost more looked into than technical skills in this analyst role. Claire Carroll, a senior Analytics Engineer from dbt Fishtown Analytics developed that from her experience, the best Analytics Engineers she met were “people who do not come from the engineering background but instead from the operations/consulting/financial area”.
The skills needed to be a great Analytics Engineer spread across a wide spectrum requiring both technical expertise and solid business acumen. A certain level of experience calls for looking for a somewhat senior profile. At first sight, you may think it is impossible to find someone with such qualifications – but for an Analytics Engineer, the importance is to hire someone that will grow in their role, especially as Danielle Leong, Senior Analytics Engineer at Starry said, the role is more about “the mindset than the skillset”.
Since not long ago, companies experience that, thanks to data, the sky is the limit: their business impact is directly correlated to their ability to navigate the modern stack efficiently and make strategic use of the available data. This only stresses the importance of creating the right data team in place. The Analytics Engineer should be an integral part of that team because:
Too often, we tend to see the world as binary: we think a career path is choosing between getting more specialized or going into management. But there are many different paths to seeing birth in the world of data, and Analytics Engineers are just one of the first. The importance of defining key data roles goes far beyond simply saying ‘oh I want someone to do something with my data' in an organization. Any stakeholder will want to understand the key roles his team players impersonate before building/joining a company. This relates directly to other topics we have already discussed as having the right data strategy, different tribes of data scientists and figuring out how to build a great data team.
We write about all the processes involved when leveraging data assets: from the modern data stack to data teams composition, to data governance. Our blog covers the technical and the less technical aspects of creating tangible value from data.
At Castor, we are building a data documentation tool for the Notion, Figma, Slack generation. We designed our catalog software to be easy to use, delightful and friendly.
Want to check it out? Reach out to us and we will show you a demo.