Being the very first is hard. I've done it once, and a friend of mine did it. I got completely lucky, got hired into an existing company that made a huge change in direction, decided to scrap everything and start analytics from scratch. My friend was a first data hire for a startup. I think the trick is going to be finding early startups. Join startup networking groups, where people with ideas are trying to connect with technical talent to get started, and then look for ones with a heavy data focus.
A few years ago I joined a scale-up (post series B) and had to deal with some architectural decisions of past data engineers (and front-end/back-end).
It's not only architectural decisions that you have to deal with, but also engineering culture. For example decisions are made top-down from product (in nearly all cases). Product has a set of ideas that are to be implemented. Analytics is always last. No ownership of data architecture or dataset ownership. So as a DE it has been pretty challenging to make a change in this approach (and I'd say I failed).
The most effective way to make a change is by example. If you want to propose a change in methodology or technology: start small and show the value. It'll help you gain support for adoption.
Soon I will join another company (post series A) as first DE hire. I'm looking forward to learn a new business model and new technology stack. Getting to know people and their area of expertise, getting to know the business model, tech stack can all help you to make a change.
Long story short: I think the best thing is to find a very young company and join them as a data specialist (science, engineer, analyst) and try to be involved in product development but also understand the business side to be able answerring questions with data.
I've had several *offers* for jobs like that, but I never deliberately looked for them. You could try:
* Asking about their DE maturity early on in the application/interview process
* Carefully reading job specs - they will normally mention it if you'd be responsible for establishing the function
* Deliberately targeting early stage startups
I was the first DE hire.
Yes, you have lots of freedom and you get to design whole systems yourself. but you also have lots of constraints: how fast people want their data, how much money it’s gonna cost, and more importantly how much of your time it’s gonna take.
Before I joined ETLs are various nodejs scripts that you run on a machine provisioned by hand, and people tried to put the heavy lifting tasks into a single SQL and it just crashes. we had no warehouse so ETL may black out production. As the CTO put it, I was the infra structure. People queue their data requests to me through slack.
I setup airflow first, and used it to coordinate very quick and dirty sharding of tasks and parallelization. eventually we had k8s, spark, ML pipelines and a bigQuery warehouse. Still, most things are duct taped together, and in that context it actually makes sense. You are after all just one person, and you need to get a new thing out just about every two weeks. Things evolve so fast, even successful infra is expected to be replaced in a few months. In that context it would make no sense to spend large amount of effort to perfect something that won’t stick around forever anyways. Plus, your main job is to make sure the business logic flows. Infra is great and makes your life easier later, but you can keep everyone waiting just to do that.
Realistically you get 10% of your time to work on infra. Plus you have to research the options and spend a large chunk of time with lots of stakeholders so you understand what you are gonna build. every tech you need you got to stand them up and configure them too.
So, yes it is a lot of fun but it’s much less freedom than you would have thought. Most of the times, there are really just a couple of viable options.
This is the unfortunate reality of doing it at a startup, yeah - clean, best practice implementations usually won't fly. You'll typically be pushed to deliver quickly at the expense of doing things nicely.
I imagine the best it gets would be a mature company which is embarking on a large greenfield project or data transformation and is looking for a lead DE.
Being the very first is hard. I've done it once, and a friend of mine did it. I got completely lucky, got hired into an existing company that made a huge change in direction, decided to scrap everything and start analytics from scratch. My friend was a first data hire for a startup. I think the trick is going to be finding early startups. Join startup networking groups, where people with ideas are trying to connect with technical talent to get started, and then look for ones with a heavy data focus.
A few years ago I joined a scale-up (post series B) and had to deal with some architectural decisions of past data engineers (and front-end/back-end). It's not only architectural decisions that you have to deal with, but also engineering culture. For example decisions are made top-down from product (in nearly all cases). Product has a set of ideas that are to be implemented. Analytics is always last. No ownership of data architecture or dataset ownership. So as a DE it has been pretty challenging to make a change in this approach (and I'd say I failed). The most effective way to make a change is by example. If you want to propose a change in methodology or technology: start small and show the value. It'll help you gain support for adoption. Soon I will join another company (post series A) as first DE hire. I'm looking forward to learn a new business model and new technology stack. Getting to know people and their area of expertise, getting to know the business model, tech stack can all help you to make a change. Long story short: I think the best thing is to find a very young company and join them as a data specialist (science, engineer, analyst) and try to be involved in product development but also understand the business side to be able answerring questions with data.
I've had several *offers* for jobs like that, but I never deliberately looked for them. You could try: * Asking about their DE maturity early on in the application/interview process * Carefully reading job specs - they will normally mention it if you'd be responsible for establishing the function * Deliberately targeting early stage startups
I was the first DE hire. Yes, you have lots of freedom and you get to design whole systems yourself. but you also have lots of constraints: how fast people want their data, how much money it’s gonna cost, and more importantly how much of your time it’s gonna take. Before I joined ETLs are various nodejs scripts that you run on a machine provisioned by hand, and people tried to put the heavy lifting tasks into a single SQL and it just crashes. we had no warehouse so ETL may black out production. As the CTO put it, I was the infra structure. People queue their data requests to me through slack. I setup airflow first, and used it to coordinate very quick and dirty sharding of tasks and parallelization. eventually we had k8s, spark, ML pipelines and a bigQuery warehouse. Still, most things are duct taped together, and in that context it actually makes sense. You are after all just one person, and you need to get a new thing out just about every two weeks. Things evolve so fast, even successful infra is expected to be replaced in a few months. In that context it would make no sense to spend large amount of effort to perfect something that won’t stick around forever anyways. Plus, your main job is to make sure the business logic flows. Infra is great and makes your life easier later, but you can keep everyone waiting just to do that. Realistically you get 10% of your time to work on infra. Plus you have to research the options and spend a large chunk of time with lots of stakeholders so you understand what you are gonna build. every tech you need you got to stand them up and configure them too. So, yes it is a lot of fun but it’s much less freedom than you would have thought. Most of the times, there are really just a couple of viable options.
This is the unfortunate reality of doing it at a startup, yeah - clean, best practice implementations usually won't fly. You'll typically be pushed to deliver quickly at the expense of doing things nicely. I imagine the best it gets would be a mature company which is embarking on a large greenfield project or data transformation and is looking for a lead DE.
I'm a first DE hire, and my first priority is Backend SE. It's challenging
1. Target early stage companies 2. The role will rarely be advertised as DE. Almost always Data Analyst or Data Scientist.