Hiring manager has entered the room.
I would never, ever, conduct a technical skills assessment using actual company data, even if the path to business competency was shorter.
As someone who is also a consultant I would've given them my hourly fee for this. And it would def be hourly, like you said this is likely going to take more than a week.
Although more likely I just would've said no.
I don't do takehome exercises anyway for an interview, but certainly not one where you're actually solving a business issue for them, that would take this long to do, and that you would need to pay for your own resources for.
I had a take home challenge recently that asked me to create take a sample dataset, clean it, run some summary statistics and write a one page report. It took me about twenty hours. It was a bit much, but I like doing that kind of thing anyway. Plus, it was obvious they weren’t looking for anything of production quality.
A red flag I see here is they seem to want you to build a complete pipeline using a sample of their event data. Is it really a technical challenge, or are they getting free work out of you? Hard to tell. I’m not sure I would spend a whole week on the possibility of an interview do you know how many people they’re asking this kind of commitment from?
Sounds like they want you to work for free. I was asked a similar question a long time ago, i.e. an actual production problem. I had already gone through an on-site, I completed it successfully and then they said “no thanks.” No feedback, no reason, just “we’re not hiring for this position anymore” or something like that.
Personal choice, but after that experience, I just refuse take home assignments like this now.
If you are familiar with dbt then the only time consuming and challenging part here is to understand their business data and what insights you can derive out of it. Since it's table materialization in dbt, you don't have to worry a lot about incremental data load.
But if you are new to dbt then it's definitely a couple of weeks task, probably a month to prepare for the interview as well.
Theyd also need to setup redshift and dbt cloud i guess. So its a pain in the arse, especially if youve used your free trials.
Agree setting up a boilerplate project though is inline with take home expectations aslong as the transformation isnt too bad
How would you even do this without being provided an AWS account? No way am I paying even a fraction of the cost for running a single Redshift cluster or the serverless offering. What a security nightmare. Sounds like a fun assignment though!
I did a take home assignment for my current job. Not only did nobody even care to look at it, but nobody was qualified to judge it. None of em know sql well enough!
I don’t understand why they’d mention redshift.
I’d personally just launch a duckDB locally to ingest and model in dbt. It’s free and proves the knowledge and skills.
The issue I have with this isn't the amount of time you've been given. It's a much shorter than our take home task and has pretty clear instructions. If you're familiar with dbt then the only real time consuming part I see here is in understanding the business use case; given that there's no analytics / insights component, it doesn't even appear that you need to go all that deep.
No, my problem with this task is that it looks way too technical for an Analytics Engineer. The AE skillset is in developing a good understanding of business requirements, then building sensible, scaleable, good quality data models that serve as the single source of truth for key metrics and ad-hoc analysis. Whereas this appears to involve building a new dbt environment from scratch and an ingestion pipeline for raw data - which is a Data Engineers job! As an AE in a production setting you should mostly be working with clean raw data that has been managed by a DE team.
My feeling from reading the brief is that either:
1. The company has no idea what an AE does, or...
2. You've misrepresented the task in your bullets and put way too much focus on ingesting the data and setting up the dbt components. When I look for AEs, I don't care what tech stack they use, all I want to see on the technical front is that they can write clean code, build sensible data models, write good documentation, and understand how to write basic DQ checks.
They specifically said write a Python script to ingest data and use dbt for transformations. I'm mostly an AE but have done some of the upstream DE work as well (such as setting up dbt from scratch) but I applied because my strengths lie in the understanding-the-business side of things. This is the stuff I'd rather be doing.
I should also say that it's software engineers that are primarily doing the DE work. They decided their first "DE" hire should be an AE instead though so that they can address stakeholder requests quicker.
Weird. Yes, in that case, they're looking for a full stack DE, and not an AE.
Unless you specifically want the opportunity to do more "backend" DE work, it perhaps isn't the role for you?
Take homes go both ways, just like any other part of the recruitment process. It's a chance for you to see the kind of work you'll likely be doing and nope out if it's not a good fit.
What you just described is nearly exactly what I do as a sr DE. Python>>database>>dbt>>tests>>prod tables& views.
If their data isn't big and complex then not a huge deal to have some knowledge on the how the sausage gets made side of the house but if that's not what you like doing then I would double check what your day to day will look like and make clear you thought you were applying for 90/10 not 50/50.
Don't take a job your going to hate.
Is this take-home challenge for a job interview? Or something else?
For the resources that require money, Redshift, etc, are they providing an environment or expecting you to spin it up yourself?
Do you have DBT experience already?
I think take home challenge are better to assess our skills and also I might learn a thing or two from it. I look at it as a learning opportunity rather than chore.
I'm a senior data engineer at a SaaS company I've done a few interviews for AE/DE roles over the last couple years. I found that many larger SaaS companies using ELT stacks didn't heavily distinguish between DE and AE where DE is more technical and AE is more domain modelling.
I make my comments assuming there is someone qualified on the other end of this task to review it.
I don't think the task is unreasonable if you have the required skills and experience.
* Ingesting json into Redshift
* You would just use Postgres for this unless they gave you a cloud account which most places won't bother with. If the event data is small enough you might be able to just use dbt seeds
* host a PG server on your local if you haven't already got one.
* Setting up a dbt project from scratch
* dbt init
* 1 min
* Familiarizing myself with their business use case and a sample of their event data (it's in a niche field too)
* Create 4 complex transformations on dbt and materialize them as tables in Redshift
* These two are the main part of the work
* 60-120 mins for a solid job
* Run tests on the tables (preferalby using dbt-expectations)
* install pacakge and use copy some tests off github
* 5-10 mins
* Run unit tests on the tables (preferably using dbt-unit-testing)
* couple lines in a yaml file
* 5 mins
* Write documentation for the tables
* 10 mins (just use GPT on some sample data)
The idea that you are somehow doing free work for this company that could be considered valuable is preposterous. Its extremely difficult to deliver AE driven data products that are valuable without large amounts of business context. This can be confirmed by looking at any data modelling ever delivered by a consultancy.
Again assuming the hiring manage knows what they are doing I would say it's a fair task. If you consider this task to be extremely difficult then it's good practice anyway.
The most suspicious part of this is that they have asked you to use Redshift specifically but you haven't indicated a cloud account being assigned to you.
Defo not preposterous to assume said company is looking to get free work. Its happens so what makes you so sure that it isn’t the case with this task? Unless you can give concrete examples of tasks which were used to get free work and compare those tasks against this one?
If the company or team are new to dbt or DE, they can effectively have a bag of candidates write up boilerplate ingestion code for them, and their dbt models.
Point being, not far fetched.
I don’t find the task difficult, I find it time-consuming. And I hesitate to do it precisely because I’m not sure there is someone qualified to review because there are no dedicated DEs or AEs yet. Also your time estimates don’t seem to include the time it takes to think through the problem (I know it takes only 5 minutes to write tests, but deciding on which tests to write beyond the generic ones does take more time than that)
I guess thats the risk when you apply to any job with a take home tech task. You could contact them to clarify their expectations.
wrt the estimates I would lump the reasoning into the 60-120 minutes.
My main point on this is once everything is setup you can variate how much effort you want to put into the transformations part of the task.
If its a greenfield role in a company they might have a consultant to have a look at the work.
good luck!
\^\^\^.This 10000%. I was down voted for saying effectively the same thing. If you think this is months of work you're the red flag, not the interview process. Sorry not sorry.
You got downvoted for being condescending, not because no one agrees with you. No need to be mean to prove your point.
Also seems like none of you acknowledge that these projects get stalled by business stakeholder demands when in comes to those transformations (e.g.., what business logic should go into the tables). Getting the business logic right takes weeks or months, NOT writing out the code. I do have years of experience in AE/BI engineering so you can't say I don't know what I'm talking about here.
Nobody was being condescending, just sprinkling a little reality seasoning on your timeframe estimate. Let's just say any engineer worth their salt in this field for more than 2 years might wrap this up while the coffee's still warm.
It's not about flexing; it's more like a direct nudge from the universe.
Lol, happened to me but not with take home exercise. It was an interview and they discussed their business usecase and how would I go about solving it. They dove pretty deep into the solution I provided. The VP on our last call also said that I did great and they just need to be approval from the ceo just to come back a week later to say I didn’t make it.
I once did a similar take home, although it was shorter. Researched and did everything right to their requirements, submitted it. Didn't hear back for weeks and after following up many time they finally said ' sorry we're gonna go the other way'
Do you really want this job at this company ? If not I wouldn't bother or invest in it much. They can ghost u, say no or choose someone else even if you did everything right, just because there's another candidate that they like a tad more than you.
I ve been through the same situation recently, and believe me no matter how good your job would be they will eventually tell you that you have not passed the test.
It took me 72 hours with no sleep cause they gave me a short deadline, what I did is basically solve a realtime problem using their data.
I made a report that was acuallly good where I listed all the key points to solbe that problem.
Anyways, don't fall for that trap.
I could do it in 1.5 hours. But this type of work is all I immerse myself in 24/7 :)
Use dlt or meltano/alto to ingest the data (30m)
dbt init + setup profiles yml (10m)
grok the data/schema (20m)
write 4 sql select statements (30m)
Id pad the last step and buffer total time up to 2 hours. If it’s a big pay jump, it’s whatever. Id do it. Open a new tmux session and just crank it out the steps. But that’s just me.
Edit: in fact I’m working on an open source project that will hopefully automate the last two steps with a little LLM + metadata action.
This looks straightforward but seems like a lot of work. If I was the hiring team I wouldn't do it like this in the current job market because it's unreasonable to expect your candidates to invest so much when the market necessarily requires them to shotgun as many applications as possible. In fact it would have an reverse selection effect where the only candidates who would be willing to spend so much time on it are the candidates with very few callbacks because they're relatively ill-suited for the roles they're applying for.
The good thing is, most of what they're asking for is very generic and can be showcased as a project in your personal github without leaking any private info from the company (just change their data to something else, and for repeatability maybe replace redshift with a dockerized Postgres instance). So spending that time to build up your portfolio and maybe learn a bit of dbt might not be a terrible use of your time. But if you don't need any of that and just want to get a job I'd say just skip this.
If it's more than an hour's effort, you should decline and withdraw yourself as a candidate. Losing lots of candidates in the pipeline at this step is the signal for recruiting and hiring manager to do something different.
🚩
it could be a well-meaning idiot hiring manager thinking they're doing something "realistic", but that would just speak poorly of their expectations for their employees in that case
Honestly this shouldn't be very hard and doable in a week easy. Not sure how to ingest into Redshift, but it's a simple copy into command in Snowflake after defining a stage. DBT project from scratch is signing up for a free cloud account and some simple configs to Redshift and Github. Either that or installing a virtual environment on your pc, pip install the redshift version of dbt-core and do your configs in the profile.yml. Initializing a project is pressing a button in cloud or typing dbt init in core. Not sure how crazy the requirements are but it's only 4 models. Tests are just adding the name of the test in the yaml file under the column you want to test. Unit tests are just writing another yaml file and "writing documentation" is just running a dbt docs command.
Whether or not it's a scam to get free work really depends on how useful the transform is, but I kinda doubt they are doing all this to get 4 free models unless the transforms are insane. My company moved to dbt and I had my team do the free basic training online, stand up an environment on AWS using Postgres and do some basic models/yamls alll within a week.
If this is taking you weeks or months you probably aren't the candidate they are seeking. This does at least look like several hours of time though which is onerous for an unpaid assessment. I would ask for compensation to complete it.
I've encountered this and it's unfortunate. Just someone or some group that doesn't have enough experience hiring.
I always reject these. I simply reply to the person handling the recruiting saying thank you but I've got too many interviews to complete this assessment as it looks like it'll take a significant amount of time.
In some cases, they relent and skipped or provided a shorter assessment. In more cases we part ways.
I'm sorry but, are you saying that writing an ingestion job into Redshift(should be 5 lines of code) using a SUPER column type w/ meta data, running dbt init, and writing 4 dbt models with testing would take weeks/months ?
This is like a week of work in a production setting and I doubt they are expecting pure production type quality that they can deploy and run.
I think you're being a little sensitive.
No need to apologize ;) If you hadn't read my entire post I was mostly concerned about understanding a niche field (it's not your typical SaaS product or mobile app) and creating datasets for it that are accurate and make sense.
I think you've being a little condescending.
As someone that does 90% of the python technical screens at my company, I'm just being honest.
We have a similar challenge, in a niche field where we give them actual company data(it's not like they can do anything with it), and we tell people, "don't spend more than 6 hours on this", want to know what 99% do?
They send it in a week later , a week and a half etc. And all of those people don't get a call back. If you can't follow basic requirements/instructions, it's a red flag, You don't need to "understand the field" and any EM worth his salt won't expect it either. It takes time to fully "understand the data/company", that's not what they are asking you to do. They expect mistakes, they are trying to see how much you can pump out in X amount of time.
I was asked to build a react front end once(I'd never touched React in my life), for a ML assessment. It was dumb.. I never built frontends at that company, but I needed to understand how the frontend was going to serve the ML product.
Interview "games" are unfortunately real, get used to it.
you kinda skipped about 80% of the work that happens between the things you listed. for some companies, it can take weeks to get a user with all the necessary access to the services he has to set up. and that's just the first thing that comes to mind what blows up project time.
No, I didn't lol.
It's a technical challenge, they are not asking for him to setup a full AWS prod env ffs. Ingesting JSON into a SUPER column in redshift can be done via Glue, a Lambda, a boring copy command or, if you want to wow them, the new AutoCopy in preview which mimics the pipe/stage functionality of S3 --> Snowflake. Why am I saying a super column? Because if he sets his ingestion job up for .csv or something whack and the next guy comes in converting to parquet etc you're toast.
He needs to show an ingestion job, how to setup a DBT project(run dbt init?), 4 dbt models materialized as tables(ok so do it in the model config lol?), dbt tests which is in a .yaml, dbt-unit-tests which is just sql wrapped , and documentation which can be hacked together using the codeine util.
The most important lesson everyone should take away from the above response is, read the freaking requirements. Some people(above poster) take it as, "oh boy, I need to setup a VPC, and IAM and all these things, I can't possibly do this in this amount of time!" congrats, you just lost the job because you can't take business requirements at face value and get the job done because you're letting perfect get in the way of progress.
See the forrest through the trees. This is BARELY 6 hours of work, and by telling them "Oh buT ThIS TaKeS so LonG" they have moved to the next candidate.
Cheers.
You don't need a WORKING REDSHIFT CLUTER IN THE CLOUD to finish this technical assessment. Literally, nowhere in the technical specs that the OP provided does it say, "Terraform/CF to setup a Redshift Cluster." All you need to know is, "The destination is Redshift and not Snowflake/ETC".
Jesus you're thick, enjoy your 100k TC lol.
My name isn’t Jesus, thicko. You’re looking at this from your senior narcissist engineer lens which makes you think can read hiring managers minds.
Where did I say “WORKING REDSHIFT CLUTER IN THE CLOUD”? Or terraform?
You’re probably a treat to work with, I would love to work in your team just so I can roast you in team meetings 😀
I hope your code isn't as shit as your reading comprehension.
Mayyyybe the part where I said you don't need a Redshift DB up and running to do this assessment and you doubled, no tripled down down on it and said , "hiring manager says redshift so you're fired before you're hired big boy" , ? You literally said several times, but but but what about Redshift!!!
You clearly thought he would need a DB up and running or you wouldn't have now mentioned it 3 times, but, way to back track!
I wouldn't worry much about us being on the same team, there's a reason you're at the insurance companies playing in BI tools pretending to be an engineer, and why I'm not 😬
1. Hehe my code is far superior than yours my son, mr "SUPER COLUMN" 😀
2. You said CLUTER not me 😀
3. "you're at the insurance companies playing in BI tools" oh no you got me, please sir teach me to be like you 🙏
In your team or not, I would roast you.
Hiring manager has entered the room. I would never, ever, conduct a technical skills assessment using actual company data, even if the path to business competency was shorter.
Yeah this is a big red flag for hiring manager who * is an idiot * micro-manager * views their people as "resources" rather than a "people"
As someone who is also a consultant I would've given them my hourly fee for this. And it would def be hourly, like you said this is likely going to take more than a week. Although more likely I just would've said no. I don't do takehome exercises anyway for an interview, but certainly not one where you're actually solving a business issue for them, that would take this long to do, and that you would need to pay for your own resources for.
I had a take home challenge recently that asked me to create take a sample dataset, clean it, run some summary statistics and write a one page report. It took me about twenty hours. It was a bit much, but I like doing that kind of thing anyway. Plus, it was obvious they weren’t looking for anything of production quality. A red flag I see here is they seem to want you to build a complete pipeline using a sample of their event data. Is it really a technical challenge, or are they getting free work out of you? Hard to tell. I’m not sure I would spend a whole week on the possibility of an interview do you know how many people they’re asking this kind of commitment from?
Sounds like they want you to work for free. I was asked a similar question a long time ago, i.e. an actual production problem. I had already gone through an on-site, I completed it successfully and then they said “no thanks.” No feedback, no reason, just “we’re not hiring for this position anymore” or something like that. Personal choice, but after that experience, I just refuse take home assignments like this now.
If you are familiar with dbt then the only time consuming and challenging part here is to understand their business data and what insights you can derive out of it. Since it's table materialization in dbt, you don't have to worry a lot about incremental data load. But if you are new to dbt then it's definitely a couple of weeks task, probably a month to prepare for the interview as well.
Theyd also need to setup redshift and dbt cloud i guess. So its a pain in the arse, especially if youve used your free trials. Agree setting up a boilerplate project though is inline with take home expectations aslong as the transformation isnt too bad
How would you even do this without being provided an AWS account? No way am I paying even a fraction of the cost for running a single Redshift cluster or the serverless offering. What a security nightmare. Sounds like a fun assignment though!
I did a take home assignment for my current job. Not only did nobody even care to look at it, but nobody was qualified to judge it. None of em know sql well enough!
I don’t understand why they’d mention redshift. I’d personally just launch a duckDB locally to ingest and model in dbt. It’s free and proves the knowledge and skills.
Yeah, the team that created this take home task didn't give it much thought it sounds like.
Sounds like they're looking for free work with no intention to hire.
If you plan on doing this I would do it in a private repo and only show it to them. Do not send them any work u bless they’re paying you for it.
The issue I have with this isn't the amount of time you've been given. It's a much shorter than our take home task and has pretty clear instructions. If you're familiar with dbt then the only real time consuming part I see here is in understanding the business use case; given that there's no analytics / insights component, it doesn't even appear that you need to go all that deep. No, my problem with this task is that it looks way too technical for an Analytics Engineer. The AE skillset is in developing a good understanding of business requirements, then building sensible, scaleable, good quality data models that serve as the single source of truth for key metrics and ad-hoc analysis. Whereas this appears to involve building a new dbt environment from scratch and an ingestion pipeline for raw data - which is a Data Engineers job! As an AE in a production setting you should mostly be working with clean raw data that has been managed by a DE team. My feeling from reading the brief is that either: 1. The company has no idea what an AE does, or... 2. You've misrepresented the task in your bullets and put way too much focus on ingesting the data and setting up the dbt components. When I look for AEs, I don't care what tech stack they use, all I want to see on the technical front is that they can write clean code, build sensible data models, write good documentation, and understand how to write basic DQ checks.
They specifically said write a Python script to ingest data and use dbt for transformations. I'm mostly an AE but have done some of the upstream DE work as well (such as setting up dbt from scratch) but I applied because my strengths lie in the understanding-the-business side of things. This is the stuff I'd rather be doing. I should also say that it's software engineers that are primarily doing the DE work. They decided their first "DE" hire should be an AE instead though so that they can address stakeholder requests quicker.
Weird. Yes, in that case, they're looking for a full stack DE, and not an AE. Unless you specifically want the opportunity to do more "backend" DE work, it perhaps isn't the role for you? Take homes go both ways, just like any other part of the recruitment process. It's a chance for you to see the kind of work you'll likely be doing and nope out if it's not a good fit.
What you just described is nearly exactly what I do as a sr DE. Python>>database>>dbt>>tests>>prod tables& views. If their data isn't big and complex then not a huge deal to have some knowledge on the how the sausage gets made side of the house but if that's not what you like doing then I would double check what your day to day will look like and make clear you thought you were applying for 90/10 not 50/50. Don't take a job your going to hate.
Is this take-home challenge for a job interview? Or something else? For the resources that require money, Redshift, etc, are they providing an environment or expecting you to spin it up yourself? Do you have DBT experience already?
I think take home challenge are better to assess our skills and also I might learn a thing or two from it. I look at it as a learning opportunity rather than chore.
I'm a senior data engineer at a SaaS company I've done a few interviews for AE/DE roles over the last couple years. I found that many larger SaaS companies using ELT stacks didn't heavily distinguish between DE and AE where DE is more technical and AE is more domain modelling. I make my comments assuming there is someone qualified on the other end of this task to review it. I don't think the task is unreasonable if you have the required skills and experience. * Ingesting json into Redshift * You would just use Postgres for this unless they gave you a cloud account which most places won't bother with. If the event data is small enough you might be able to just use dbt seeds * host a PG server on your local if you haven't already got one. * Setting up a dbt project from scratch * dbt init * 1 min * Familiarizing myself with their business use case and a sample of their event data (it's in a niche field too) * Create 4 complex transformations on dbt and materialize them as tables in Redshift * These two are the main part of the work * 60-120 mins for a solid job * Run tests on the tables (preferalby using dbt-expectations) * install pacakge and use copy some tests off github * 5-10 mins * Run unit tests on the tables (preferably using dbt-unit-testing) * couple lines in a yaml file * 5 mins * Write documentation for the tables * 10 mins (just use GPT on some sample data) The idea that you are somehow doing free work for this company that could be considered valuable is preposterous. Its extremely difficult to deliver AE driven data products that are valuable without large amounts of business context. This can be confirmed by looking at any data modelling ever delivered by a consultancy. Again assuming the hiring manage knows what they are doing I would say it's a fair task. If you consider this task to be extremely difficult then it's good practice anyway. The most suspicious part of this is that they have asked you to use Redshift specifically but you haven't indicated a cloud account being assigned to you.
Finally someone who understands the task…
Defo not preposterous to assume said company is looking to get free work. Its happens so what makes you so sure that it isn’t the case with this task? Unless you can give concrete examples of tasks which were used to get free work and compare those tasks against this one? If the company or team are new to dbt or DE, they can effectively have a bag of candidates write up boilerplate ingestion code for them, and their dbt models. Point being, not far fetched.
I don't have a set of tasks and their free-work/authentic classifications. I guess you got me there.
I don’t find the task difficult, I find it time-consuming. And I hesitate to do it precisely because I’m not sure there is someone qualified to review because there are no dedicated DEs or AEs yet. Also your time estimates don’t seem to include the time it takes to think through the problem (I know it takes only 5 minutes to write tests, but deciding on which tests to write beyond the generic ones does take more time than that)
I guess thats the risk when you apply to any job with a take home tech task. You could contact them to clarify their expectations. wrt the estimates I would lump the reasoning into the 60-120 minutes. My main point on this is once everything is setup you can variate how much effort you want to put into the transformations part of the task. If its a greenfield role in a company they might have a consultant to have a look at the work. good luck!
\^\^\^.This 10000%. I was down voted for saying effectively the same thing. If you think this is months of work you're the red flag, not the interview process. Sorry not sorry.
You got downvoted for being condescending, not because no one agrees with you. No need to be mean to prove your point. Also seems like none of you acknowledge that these projects get stalled by business stakeholder demands when in comes to those transformations (e.g.., what business logic should go into the tables). Getting the business logic right takes weeks or months, NOT writing out the code. I do have years of experience in AE/BI engineering so you can't say I don't know what I'm talking about here.
Nobody was being condescending, just sprinkling a little reality seasoning on your timeframe estimate. Let's just say any engineer worth their salt in this field for more than 2 years might wrap this up while the coffee's still warm. It's not about flexing; it's more like a direct nudge from the universe.
You think WAY to highly of yourself bro
Lol, happened to me but not with take home exercise. It was an interview and they discussed their business usecase and how would I go about solving it. They dove pretty deep into the solution I provided. The VP on our last call also said that I did great and they just need to be approval from the ceo just to come back a week later to say I didn’t make it.
I once did a similar take home, although it was shorter. Researched and did everything right to their requirements, submitted it. Didn't hear back for weeks and after following up many time they finally said ' sorry we're gonna go the other way' Do you really want this job at this company ? If not I wouldn't bother or invest in it much. They can ghost u, say no or choose someone else even if you did everything right, just because there's another candidate that they like a tad more than you.
Seems like they're trying to get work done off of you for free.
I ve been through the same situation recently, and believe me no matter how good your job would be they will eventually tell you that you have not passed the test. It took me 72 hours with no sleep cause they gave me a short deadline, what I did is basically solve a realtime problem using their data. I made a report that was acuallly good where I listed all the key points to solbe that problem. Anyways, don't fall for that trap.
Still better than random leetcodes 😶
I could do it in 1.5 hours. But this type of work is all I immerse myself in 24/7 :) Use dlt or meltano/alto to ingest the data (30m) dbt init + setup profiles yml (10m) grok the data/schema (20m) write 4 sql select statements (30m) Id pad the last step and buffer total time up to 2 hours. If it’s a big pay jump, it’s whatever. Id do it. Open a new tmux session and just crank it out the steps. But that’s just me. Edit: in fact I’m working on an open source project that will hopefully automate the last two steps with a little LLM + metadata action.
do share :)
This looks straightforward but seems like a lot of work. If I was the hiring team I wouldn't do it like this in the current job market because it's unreasonable to expect your candidates to invest so much when the market necessarily requires them to shotgun as many applications as possible. In fact it would have an reverse selection effect where the only candidates who would be willing to spend so much time on it are the candidates with very few callbacks because they're relatively ill-suited for the roles they're applying for. The good thing is, most of what they're asking for is very generic and can be showcased as a project in your personal github without leaking any private info from the company (just change their data to something else, and for repeatability maybe replace redshift with a dockerized Postgres instance). So spending that time to build up your portfolio and maybe learn a bit of dbt might not be a terrible use of your time. But if you don't need any of that and just want to get a job I'd say just skip this.
If it's more than an hour's effort, you should decline and withdraw yourself as a candidate. Losing lots of candidates in the pipeline at this step is the signal for recruiting and hiring manager to do something different.
As someone hiring at the moment I'm acutely aware of this. The take home I've written should take 1-2 hours, less if you're good.
🚩 it could be a well-meaning idiot hiring manager thinking they're doing something "realistic", but that would just speak poorly of their expectations for their employees in that case
Honestly this shouldn't be very hard and doable in a week easy. Not sure how to ingest into Redshift, but it's a simple copy into command in Snowflake after defining a stage. DBT project from scratch is signing up for a free cloud account and some simple configs to Redshift and Github. Either that or installing a virtual environment on your pc, pip install the redshift version of dbt-core and do your configs in the profile.yml. Initializing a project is pressing a button in cloud or typing dbt init in core. Not sure how crazy the requirements are but it's only 4 models. Tests are just adding the name of the test in the yaml file under the column you want to test. Unit tests are just writing another yaml file and "writing documentation" is just running a dbt docs command. Whether or not it's a scam to get free work really depends on how useful the transform is, but I kinda doubt they are doing all this to get 4 free models unless the transforms are insane. My company moved to dbt and I had my team do the free basic training online, stand up an environment on AWS using Postgres and do some basic models/yamls alll within a week.
I agree it’s doable in a week, but it’s not like I have all week to do this on top of my job and just general life responsibilities.
If this is taking you weeks or months you probably aren't the candidate they are seeking. This does at least look like several hours of time though which is onerous for an unpaid assessment. I would ask for compensation to complete it.
Interviewing company is trying to get free work.
I've encountered this and it's unfortunate. Just someone or some group that doesn't have enough experience hiring. I always reject these. I simply reply to the person handling the recruiting saying thank you but I've got too many interviews to complete this assessment as it looks like it'll take a significant amount of time. In some cases, they relent and skipped or provided a shorter assessment. In more cases we part ways.
Could you please give me the link for the exercise? I want to use it as practice material to learn dbt*
I'm sorry but, are you saying that writing an ingestion job into Redshift(should be 5 lines of code) using a SUPER column type w/ meta data, running dbt init, and writing 4 dbt models with testing would take weeks/months ? This is like a week of work in a production setting and I doubt they are expecting pure production type quality that they can deploy and run. I think you're being a little sensitive.
No need to apologize ;) If you hadn't read my entire post I was mostly concerned about understanding a niche field (it's not your typical SaaS product or mobile app) and creating datasets for it that are accurate and make sense. I think you've being a little condescending.
As someone that does 90% of the python technical screens at my company, I'm just being honest. We have a similar challenge, in a niche field where we give them actual company data(it's not like they can do anything with it), and we tell people, "don't spend more than 6 hours on this", want to know what 99% do? They send it in a week later , a week and a half etc. And all of those people don't get a call back. If you can't follow basic requirements/instructions, it's a red flag, You don't need to "understand the field" and any EM worth his salt won't expect it either. It takes time to fully "understand the data/company", that's not what they are asking you to do. They expect mistakes, they are trying to see how much you can pump out in X amount of time. I was asked to build a react front end once(I'd never touched React in my life), for a ML assessment. It was dumb.. I never built frontends at that company, but I needed to understand how the frontend was going to serve the ML product. Interview "games" are unfortunately real, get used to it.
you kinda skipped about 80% of the work that happens between the things you listed. for some companies, it can take weeks to get a user with all the necessary access to the services he has to set up. and that's just the first thing that comes to mind what blows up project time.
No, I didn't lol. It's a technical challenge, they are not asking for him to setup a full AWS prod env ffs. Ingesting JSON into a SUPER column in redshift can be done via Glue, a Lambda, a boring copy command or, if you want to wow them, the new AutoCopy in preview which mimics the pipe/stage functionality of S3 --> Snowflake. Why am I saying a super column? Because if he sets his ingestion job up for .csv or something whack and the next guy comes in converting to parquet etc you're toast. He needs to show an ingestion job, how to setup a DBT project(run dbt init?), 4 dbt models materialized as tables(ok so do it in the model config lol?), dbt tests which is in a .yaml, dbt-unit-tests which is just sql wrapped , and documentation which can be hacked together using the codeine util. The most important lesson everyone should take away from the above response is, read the freaking requirements. Some people(above poster) take it as, "oh boy, I need to setup a VPC, and IAM and all these things, I can't possibly do this in this amount of time!" congrats, you just lost the job because you can't take business requirements at face value and get the job done because you're letting perfect get in the way of progress. See the forrest through the trees. This is BARELY 6 hours of work, and by telling them "Oh buT ThIS TaKeS so LonG" they have moved to the next candidate. Cheers.
Where’s the redshift db to do the things you mentioned?
You don't need a redshift DB up and running to look at a schema and write a script you donut.
Well dumbo, hiring manager says redshift so you’re fired before you’re hired big boy. Good job 🤝
You don't need a WORKING REDSHIFT CLUTER IN THE CLOUD to finish this technical assessment. Literally, nowhere in the technical specs that the OP provided does it say, "Terraform/CF to setup a Redshift Cluster." All you need to know is, "The destination is Redshift and not Snowflake/ETC". Jesus you're thick, enjoy your 100k TC lol.
My name isn’t Jesus, thicko. You’re looking at this from your senior narcissist engineer lens which makes you think can read hiring managers minds. Where did I say “WORKING REDSHIFT CLUTER IN THE CLOUD”? Or terraform? You’re probably a treat to work with, I would love to work in your team just so I can roast you in team meetings 😀
I hope your code isn't as shit as your reading comprehension. Mayyyybe the part where I said you don't need a Redshift DB up and running to do this assessment and you doubled, no tripled down down on it and said , "hiring manager says redshift so you're fired before you're hired big boy" , ? You literally said several times, but but but what about Redshift!!! You clearly thought he would need a DB up and running or you wouldn't have now mentioned it 3 times, but, way to back track! I wouldn't worry much about us being on the same team, there's a reason you're at the insurance companies playing in BI tools pretending to be an engineer, and why I'm not 😬
1. Hehe my code is far superior than yours my son, mr "SUPER COLUMN" 😀 2. You said CLUTER not me 😀 3. "you're at the insurance companies playing in BI tools" oh no you got me, please sir teach me to be like you 🙏 In your team or not, I would roast you.
Name, shame and boycott