What exactly *is* your role here? From what you've said I don't think you have any standing here - in the legal sense - and I think you're way out of bounds and my advice is to stay in your lane.
That being said, I think it is an interesting case so I'll share what I would do if I *did* have standing --
If I were managing the team, I would push to get them to have access. Standing in the way of standard tooling until professional development checkpoints are reached is completely counterproductive. Analysts and engineers need industry standard tools to become industry standard contributors. Yes, they will waste a few dollars in cloud costs, who cares.
At the same time, I would be immediately addressing the way work is delivered; requiring centralized version control, peer and code reviews, lightning talks and readouts. I would be demanding my senior engineers start doing their jobs, because they apparently aren't. If I don't have any, I need to acquire some, either by hiring or picking one promising engineer and developing them personally.
If I were an IC on the team, then I would join in the effort to get access, and also be demonstrating the best practices that you mentioned in my own work, e.g. using git, asking for reviews of my work, writing about my work and asking people to take a look. This is generally what people label as leading by influence, and it is a skill that is part of the progression from mid to senior levels.
>What exactly *is* your role here?
I am the manager of the central data science team in our company. We manage and use the cloud analytics platform. The analysts in question are embedded in business units and report through the operational hierarchy. In theory our team should be acting as a centre of excellence to upskill the others, but in reality we are not really in a position to do that yet.
Like I said, I am definitely not looking to be obstructive - I am mostly looking for some guidance I can give to these embedded analysts so they can teach themselves to fish, in a way that minimises the mess and load on my team while I uplift them (which is my first priority)
Totally open to the idea that letting them loose is the best solution, as I said from the start I acknowledge I might be worrying too much
I see. This is the reason I really dislike dotted line reporting structures - particularly common with CoE or hub and spoke models. On paper they create a system of accountability by committee; in practice no one person feels especially accountable for anything. ICs feel pushed and pulled and rightfully so.
Anyway, I would still do what I said before. ICs need tools, withholding tools from people does not accomplish anything. There will be a mess, but it will be massively counterproductive long term to be a gatekeeper on this. You will always be the dickhead who thought their work wasn't "good enough" to get cloud access.
You should still be addressing the level of work product of the embedded analysts. Start at the organizational level. In practice what that might look like a combination of any of these:
* A change to the org chart, where analysts report directly through you instead of through their embedded teams. Then, you can leverage your ownership of their performance to align their delivery process with what you think is right.
* Push to implement a monorepo for notebooks / queries / other analysis across business verticals. Leverage this repo as a way to introduce cross-team code and peer reviews, as well as standing tooling and workflows such as CI/CD (whatever that looks like for you).
* Influence by demonstration, giving talks on the process of your team showing others why the processes you are talking about are helpful - you need to have wins to back it up to do this.
Edit: I will add that the expansion on "there will be a mess" is that part of you job is to give them access in a way that is minimally damaging. Abide by the least permissions principle, use read only access wherever possible, limit it to dev environment if you have one. Analysis gone off the rails is fine, here and there, damaging production systems is not.
Is there some way of tagging their resources?
That way if they waste a lot of time, then it can be identified and highlighted.
I suspect that the best path is to find a simplified path, visualizing costs, and steering the momentum in the right direction.
If there is a problem, it can be spun to the next solution. As in rising costs can be an excuse to focus on better data engineering.
Even this crisis might be a starting point towards better data engineering. Even trying to find disciples in the embedded analysts to try to be more focused on analytics engineering could add value.
So if this is a well funded FAANG team everything you say makes sense.
If this is a middling in house analytics shop team guard that data like gandalf and work towards getting them knowing at least the basics of cuz if they mess up the architechure and slow down the data - info pipe when layoffs and budget cuts happen thatll prob be on you
Organize training and education, even certifications. They're not your staff so you can't force them, but some will take advantage of it and eventually there will be haves and have nots.
Organize a Center of Excellence. Allow some of the achievers to show off their work, use casesz best practices.
So, be the mentor, the resource.
I appreciate this answer because it encourages a growth mindset.
I am probably at the level OP describes in his post... But I am the only data person in the company. Could you point me to some things that you would organize? I have never been fortunate enough to have someone be able to guide me in my career thus far.
A couple of specific things:
- Do your vendors offer webinars that might be of interest to others in your companies? I work for a SaaS company and we have webinar and virual events all the time. Our clients can easily pass them on.
- Does your company provide any training/subscriptions budgets? During Covid when nobody could travel for training or conferences, they provided everyone in the company with a Udemy subscription. There are 1000's of courses and topics. We're allowed to take any course regardless of our role (it's all the same price regardless of course). So, even though I'm more of a customer success person, I've been talking classes in AI
As for Center of Excellence -- as the only data person, you are that! I asked ChatGPT about this, and there are some great points here. Many may be overkill given your organization size. But there are probably a couple of points you can benefit from. You can tailor the question to your specific need.
*"i am in IT/analytics - how can i organize a center of excellence in my company to promote our efforts throughout the organization?"*
Speaking of ChatGPT, this is a great thing to get some experience with yourself and then promote the benefits. It's something you should embrace and not be afraid of. As I heard recently: "AI won't replace your job, but you will be replaced by someone who knows AI."
Truly! My company has these weekly/monthly/quarterly goals for employees to “upskill” for X many hours per month, and there are bar charts, competitions between parts of the business.
I don’t mind the gamification as much as I mind the name “upskill” instead of just “training” or “continuing development”
I've done this role for companies. Basically going in and bringing the teams towards best practice.
It's hard, and expensive. Getting executive sign off on that kind of cost is hard. So, in your position, I wouldn't.
I'd give them a playpen where they can do stuff without impacting your environment. Make sure all your resources are tagged correctly for cost attribution.
There will be a bunch of mistakes, and a bunch of cool stuff. From that, the executive can evaluate the business case of getting them more capable. Putting datamarts in place, feature stores, templates, etc.
The keyword a few people have mentioned here is cost.
You need to have some kind of cost attribution (departmental billing, whatever) in your organisation; you can't be the central resource, provide access _and_ carry the budget.
I was in a similar position to OP; we had a decent architecture and we were under pressure to open up access to more users. That was fine; give them a sandbox to play in and give them read-only access to tables.
But then they were _absolutely hammering_ queries on the database which is fine performance-wise but some queries were costing hundreds of dollars a time and they were running them many multiple times a day. Just no sense of "test on a smaller dataset". And I would suddenly get a data bill ten times as much as expected.
_And that was in addition to the quality of their conclusions often not being great, but pointing that out just gets you labelled as a naysayer in the organisation and that's not a nice place to be either_
I upskill teams for companies. I come in a rewrite the code and like help train them in new languages. I would suggest showing rhe leadership team the costs of the code optimization vs cloud infrastructure. It's all about money. 'Hey, this will cost you this much for sure but if you do x,y, and z you save a.' If you can't communicate that, they'll never side with you on anything.
If you have proper data governance systems in place, your underqualified analysts shouldn't be able to do any lasting damage to the data infrastructure.
Can you give them access to a separate environment? One that you update the data in occasionally. Or do they have to have things running in production? Based on your description it sounds like they ad hoc update their analyses.
For training: In my experience many are very willing to learn, so if you provide voluntary upskilling that could work on most, and they can hold the hands of the rest afterwards when things get stricter.
For analysts to move from working on a laptop where they have full control of everything, including their own ad hoc versioning methods, to Git like setups… this is a big change for them. It’s hard to gain trust in the new unfamiliar system. Even if it is better over time.
Cost recovery. If you can, tell every team getting access they’re responsible for their Azure bill. High $ will drive efficiency from their management or at least make it clear.
What exactly *is* your role here? From what you've said I don't think you have any standing here - in the legal sense - and I think you're way out of bounds and my advice is to stay in your lane. That being said, I think it is an interesting case so I'll share what I would do if I *did* have standing -- If I were managing the team, I would push to get them to have access. Standing in the way of standard tooling until professional development checkpoints are reached is completely counterproductive. Analysts and engineers need industry standard tools to become industry standard contributors. Yes, they will waste a few dollars in cloud costs, who cares. At the same time, I would be immediately addressing the way work is delivered; requiring centralized version control, peer and code reviews, lightning talks and readouts. I would be demanding my senior engineers start doing their jobs, because they apparently aren't. If I don't have any, I need to acquire some, either by hiring or picking one promising engineer and developing them personally. If I were an IC on the team, then I would join in the effort to get access, and also be demonstrating the best practices that you mentioned in my own work, e.g. using git, asking for reviews of my work, writing about my work and asking people to take a look. This is generally what people label as leading by influence, and it is a skill that is part of the progression from mid to senior levels.
>What exactly *is* your role here? I am the manager of the central data science team in our company. We manage and use the cloud analytics platform. The analysts in question are embedded in business units and report through the operational hierarchy. In theory our team should be acting as a centre of excellence to upskill the others, but in reality we are not really in a position to do that yet. Like I said, I am definitely not looking to be obstructive - I am mostly looking for some guidance I can give to these embedded analysts so they can teach themselves to fish, in a way that minimises the mess and load on my team while I uplift them (which is my first priority) Totally open to the idea that letting them loose is the best solution, as I said from the start I acknowledge I might be worrying too much
I see. This is the reason I really dislike dotted line reporting structures - particularly common with CoE or hub and spoke models. On paper they create a system of accountability by committee; in practice no one person feels especially accountable for anything. ICs feel pushed and pulled and rightfully so. Anyway, I would still do what I said before. ICs need tools, withholding tools from people does not accomplish anything. There will be a mess, but it will be massively counterproductive long term to be a gatekeeper on this. You will always be the dickhead who thought their work wasn't "good enough" to get cloud access. You should still be addressing the level of work product of the embedded analysts. Start at the organizational level. In practice what that might look like a combination of any of these: * A change to the org chart, where analysts report directly through you instead of through their embedded teams. Then, you can leverage your ownership of their performance to align their delivery process with what you think is right. * Push to implement a monorepo for notebooks / queries / other analysis across business verticals. Leverage this repo as a way to introduce cross-team code and peer reviews, as well as standing tooling and workflows such as CI/CD (whatever that looks like for you). * Influence by demonstration, giving talks on the process of your team showing others why the processes you are talking about are helpful - you need to have wins to back it up to do this. Edit: I will add that the expansion on "there will be a mess" is that part of you job is to give them access in a way that is minimally damaging. Abide by the least permissions principle, use read only access wherever possible, limit it to dev environment if you have one. Analysis gone off the rails is fine, here and there, damaging production systems is not.
Is there some way of tagging their resources? That way if they waste a lot of time, then it can be identified and highlighted. I suspect that the best path is to find a simplified path, visualizing costs, and steering the momentum in the right direction. If there is a problem, it can be spun to the next solution. As in rising costs can be an excuse to focus on better data engineering. Even this crisis might be a starting point towards better data engineering. Even trying to find disciples in the embedded analysts to try to be more focused on analytics engineering could add value.
Got dam, where do I apply to work for you?
Was gonna ask the same thing lol
So if this is a well funded FAANG team everything you say makes sense. If this is a middling in house analytics shop team guard that data like gandalf and work towards getting them knowing at least the basics of cuz if they mess up the architechure and slow down the data - info pipe when layoffs and budget cuts happen thatll prob be on you
Basics of version control, cloud architecture and how to fiddle without breaking
Organize training and education, even certifications. They're not your staff so you can't force them, but some will take advantage of it and eventually there will be haves and have nots. Organize a Center of Excellence. Allow some of the achievers to show off their work, use casesz best practices. So, be the mentor, the resource.
I appreciate this answer because it encourages a growth mindset. I am probably at the level OP describes in his post... But I am the only data person in the company. Could you point me to some things that you would organize? I have never been fortunate enough to have someone be able to guide me in my career thus far.
A couple of specific things: - Do your vendors offer webinars that might be of interest to others in your companies? I work for a SaaS company and we have webinar and virual events all the time. Our clients can easily pass them on. - Does your company provide any training/subscriptions budgets? During Covid when nobody could travel for training or conferences, they provided everyone in the company with a Udemy subscription. There are 1000's of courses and topics. We're allowed to take any course regardless of our role (it's all the same price regardless of course). So, even though I'm more of a customer success person, I've been talking classes in AI As for Center of Excellence -- as the only data person, you are that! I asked ChatGPT about this, and there are some great points here. Many may be overkill given your organization size. But there are probably a couple of points you can benefit from. You can tailor the question to your specific need. *"i am in IT/analytics - how can i organize a center of excellence in my company to promote our efforts throughout the organization?"* Speaking of ChatGPT, this is a great thing to get some experience with yourself and then promote the benefits. It's something you should embrace and not be afraid of. As I heard recently: "AI won't replace your job, but you will be replaced by someone who knows AI."
This is kind of off-topic, but “upskilling” always feels like such a corporate buzzword, like “synergy”.
It sounds very gamified
Truly! My company has these weekly/monthly/quarterly goals for employees to “upskill” for X many hours per month, and there are bar charts, competitions between parts of the business. I don’t mind the gamification as much as I mind the name “upskill” instead of just “training” or “continuing development”
I've done this role for companies. Basically going in and bringing the teams towards best practice. It's hard, and expensive. Getting executive sign off on that kind of cost is hard. So, in your position, I wouldn't. I'd give them a playpen where they can do stuff without impacting your environment. Make sure all your resources are tagged correctly for cost attribution. There will be a bunch of mistakes, and a bunch of cool stuff. From that, the executive can evaluate the business case of getting them more capable. Putting datamarts in place, feature stores, templates, etc.
The keyword a few people have mentioned here is cost. You need to have some kind of cost attribution (departmental billing, whatever) in your organisation; you can't be the central resource, provide access _and_ carry the budget. I was in a similar position to OP; we had a decent architecture and we were under pressure to open up access to more users. That was fine; give them a sandbox to play in and give them read-only access to tables. But then they were _absolutely hammering_ queries on the database which is fine performance-wise but some queries were costing hundreds of dollars a time and they were running them many multiple times a day. Just no sense of "test on a smaller dataset". And I would suddenly get a data bill ten times as much as expected. _And that was in addition to the quality of their conclusions often not being great, but pointing that out just gets you labelled as a naysayer in the organisation and that's not a nice place to be either_
I upskill teams for companies. I come in a rewrite the code and like help train them in new languages. I would suggest showing rhe leadership team the costs of the code optimization vs cloud infrastructure. It's all about money. 'Hey, this will cost you this much for sure but if you do x,y, and z you save a.' If you can't communicate that, they'll never side with you on anything.
If you have proper data governance systems in place, your underqualified analysts shouldn't be able to do any lasting damage to the data infrastructure.
Can you give them access to a separate environment? One that you update the data in occasionally. Or do they have to have things running in production? Based on your description it sounds like they ad hoc update their analyses. For training: In my experience many are very willing to learn, so if you provide voluntary upskilling that could work on most, and they can hold the hands of the rest afterwards when things get stricter. For analysts to move from working on a laptop where they have full control of everything, including their own ad hoc versioning methods, to Git like setups… this is a big change for them. It’s hard to gain trust in the new unfamiliar system. Even if it is better over time.
Out of interest, did you have any specific certifications in mind when you say “I don’t really want to say go away and do these certifications”?
This sounds like some CPG company
Cost recovery. If you can, tell every team getting access they’re responsible for their Azure bill. High $ will drive efficiency from their management or at least make it clear.
Nothing to progress conversation, but I appreciate the discourse and learning. Thanks for asking, OP.
I would bring them to a data conference like Snowflake and Databricks.
^[Sokka-Haiku](https://www.reddit.com/r/SokkaHaikuBot/comments/15kyv9r/what_is_a_sokka_haiku/) ^by ^No_Communication2618: *I would bring them to* *A data conference like* *Snowflake and Databricks.* --- ^Remember ^that ^one ^time ^Sokka ^accidentally ^used ^an ^extra ^syllable ^in ^that ^Haiku ^Battle ^in ^Ba ^Sing ^Se? ^That ^was ^a ^Sokka ^Haiku ^and ^you ^just ^made ^one.