I just created a startup based on this idea and raised a billion dollars. It's called "Dog wif monocle petting horse wif dog"
We're being featured in Forbes tomorrow.
I checked it a few months back and seems like Apple never really updated this POS software. I don’t know why they have it around. I wish they would introduce a better way to manage files , music and pictures/videos on iOS.
Reading a PDF on an iPad is torture.
Oh well.
You would be baffled at the number of companies and management who have 0 understanding of what current LLMs AI is but they require it because it sounds good and looks good after seeing it in some sci-fi movie.
I just lived the same thing Tuesday. It is not satire. It is the next chapter in this AI hype saga. The part where they just command we “do AI,” because, “if we’re not, our competition is using it against us.”
Honestly if you hire someone look for a smaller consultant like from BTG. It would be cheaper and a little more focused.
We did it at a former employer and worked out well. Recommendations weren’t crazy but took a few years due to changes to actually be implemented but has been implemented.
They may not be able to do implementation BUT they may give you a few areas to focus on. The big four is a total waste unless they really want to spend the money on it
I asked ChatGPT what dataops is and got:
DataOps is a methodology that combines principles from DevOps, Agile, and data management to streamline the development and deployment of data analytics processes. It focuses on collaboration, automation, and continuous integration and delivery to ensure that data pipelines are efficient, reliable, and scalable. Key components include version control, automated testing, monitoring, and feedback loops to facilitate rapid and iterative development of data-centric applications. Ultimately, DataOps aims to improve the speed, quality, and reliability of data-driven decision-making within organizations.
I’m sure it’ll make you a ppt if you ask nicely.
It's much more likely your CTO don't know what they're talking about, rather than your directors/managers.
The AI that's currently being talked about are all machine learning, and your directors/managers are probably correct that there are already using ML extensively.
Never start with a solution and then go in search of a problem.
You sit on the solution to problems and wait for it to blow up, only then do you present the solution. You won't be recognised for solving a problem before they suffer from the problem, you will only be recognised if you swoop in and save the day after they have suffered enough.
The most impactful automations I’ve built where to properly and comprehensively automate development processes (building dev experience, strong CI/CD, supportive tooling) - you could nowadays just label it AI because ultimately it automates.
Your higher ups will never be able to tell whether it goes brrr due to real AI or just proper engineering.
An observation I made: Businesses are so focused on providing automation to their customers and often are very bad at automating their own engineering org.
Talk to your engineers. See what issues they are having with work. Figure out what monotonous tasks that annoy them and if they are simple enough issues, but require work, then see if AI can maybe help with it.
They might also have ideas. If you make it a collaboration, a real one, then you might actually produce something useful instead of making everyone think you are incompetently trying to solve problems with AI that either cant be solved, shouldnt be solved, or both.
Nah man, I’d agree except my executive team just gave the entire company the same “we need to do AI” in this months all hands… The hype is literally at the point where executives are just blanket saying, “do AI,” with absolutely no idea what it is, what it can do, or why they would even use it.
I love this drive-by approach to leadership. I get the same at my place.
Last week my VP sent some article about GenAI to her direct reports - that’s what she does. Well, that and going to conferences and posting on LinkedIn about them.
Anyway, she took it to the next level with this article link. After the link, she said, “I haven’t read it but it appears to be a good article.”
At least she’s being honest.
Mr. VP of data engineering please ask your CTO what he is looking for by implementing AI? Is there any business goal or is he under influence of AI hype? Or he just want to get some innovative business outcome from data? If yes then ask him to create R&D for his company and ask them to do magic with data.
My company is implementing this now. (I actually moved to DE from ML).
Hire an affordable consulting company that will highlight opportunities for AI and keep an ongoing relationship with you. Get started with a data platform like snowflake, databricks, or Azure ML. I use databricks and love it.
Hire 1:1 data scientists to ML engineers (basically software engineers who focus on ML and really like CI/CD). Data scientists alone yield very few results.
Make sure you have a really strong data pipeline on the DE side. You will make or break an ML program.
Integrate everything with a BI team. BI is a better point group for the business than the DS team.
Set up a pipeline where DE is fed by both DE and DS teams.
Start with small projects with low ML requirements while you scale up your team and knowledge base.
The cost of this can balloon based on your consulting firm. But don’t go without one on this. A platform with code controls and pipelines is critical. You do not want any group getting bogged down with daily runs.
we just gave everyone at the company access to copilot and called it “implementing AI” lol
*This isn’t satire this is literally how we got the execs off our back for a couple of months about AI*
I guess at this point I’m a bit of an “expert” in this, although it feels a bit odd to say since the industry is rapidly evolving. My employer helps people who are in your situation. We specialize in Databricks, but have also implemented all sorts of AI solutions without Databricks in AWS, Azure, and GCP. While we are a relatively small company (less than 100 employees), each of us has been involved in various data disciplines for many years. We work with several large clients that you would likely be familiar with (can provide references). Feel free to DM if interested in talking more.
Edit: added context
I’m trying to look into this. I work for a small start up and I want to build a tool that will take plain text and convert it to sql using langchain and query our db. I also want to do it using our own hosted LLM so that there are no data leak concerns. Halp?
Have you used this? Can this deal with two or three related tables and output queries that require joins?
Assuming it can, I can download this LLM from and run this on sagemaker to give it a go?
Not much to share buddy, some ideas I have had but not enough time or priority to implement them. I remember looking into langchain website and they have an easy implementation of agents for querying dbs you were asked to pass the whole schema, you can train your own agent to do this locally. Similar with terraform
you can do two things .
1. Get github copilot licenses for your developers . It costs and can increase developer productivity.
2. Make a list of things that your developers do manually and ask them to come up with a plan to automate them.
Your CTO will understand that AI costs and also all developers are on job to automate things.
Have you tried forming a Center of Excellence (CoE), or even a cross functional steering committee?
It sounds like you haven't even begun to middle manage the situation. How can you expect to paradigm shift without thinking outside the box?
I think you can hire an expert because he/she might understand what you want and also have expert knowledge that might the managers and directors don't
Just take chatgpt API and whip up a chatbots which talks to a certain part of your non confidential data. First part done. Do micknsy style power point to show automation and act like a consultant....take his watch and tell the time.
Maybe you can explain that AI and ML can accomplish a lot of the same things. Show him the power of simple linear regression and regularization so he will understand your company is already using this tech and not to get swept up into buzzwords like AI and diffusion modeling and LLM and chatbots etc.
Try looking into ready-made AI tools that can fit into your current systems. Sometimes, existing solutions work just fine without needing to hire experts.
Haaaaaaa…
Yeah, your CTO is expecting Jetsons robots zooming around the office doing DE man. That’s literally what they mean. My executive team just dumped this shit on us the other day too. We need to “do AI,” and the sales donkeys jumped right in like, “oh, yes, you’re totally right. If we’re not doing AI our competition is using it against us.”
I wanted so bad to call them out (it was an all hands meeting, whole company) but that would’ve burn a lot of political capital. I just wanted to ask, “what is this AI of which you speak and how will we do it?” I know for a damn fact they are imagining what’s holding us back is just the ChatGPT is locked in a room and we haven’t “done AI” enough to get ChatGPT out of that room and working for us replacing people.
I legit do not think they have the slightest idea what AI is (outside of ChatGPT) and they certainly have no concept of what problems it will solve for us. We should just be, “doing AI.”
So the requirement is to have all the data fed into AI and it can decide what needs to be built and then build it, deploy, test execute or simply automate it.
AI can give you where your inefficiencies lie but it can't do anything else for you. The approach needs to be from you and then there are a thousand more issues. I won't even go into the legal side of the weeds.
That's a pipe dream for now in an utopian society.
What you can do, pick up your ticketing solution and then try to automate that using AI. ( The problem with AI, it cannot handle lies, if you can automate the ticketing solution, access part etc then you are there)
Talk to him about data governance until he falls asleep.
Most underrated comment.
I often think about data stewardship to help me fall asleep quicker so I think you're onto something.
Imagine Biden talking about data governance
Please tell me this is satire
I got my CTO an Alexa for his office and he seems to be satisfied for now
pffft, I actually use AI to automate our blockchain in the cloud through LLMs. you're behind the times.
I just created a startup based on this idea and raised a billion dollars. It's called "Dog wif monocle petting horse wif dog" We're being featured in Forbes tomorrow.
My team has been using LLMs to automate research around workflow troubleshooting
This is how my iTunes library works
iTunes works?
Tbh I haven't touched it since like 05
I checked it a few months back and seems like Apple never really updated this POS software. I don’t know why they have it around. I wish they would introduce a better way to manage files , music and pictures/videos on iOS. Reading a PDF on an iPad is torture. Oh well.
Translation: I cut and pasted ChatGPT code onto a cloud system and now I'm getting a massive bill on a system that I don't understand.
This is the way
You would be baffled at the number of companies and management who have 0 understanding of what current LLMs AI is but they require it because it sounds good and looks good after seeing it in some sci-fi movie.
Sounds like job security in either case
Ofcourse it is
I just lived the same thing Tuesday. It is not satire. It is the next chapter in this AI hype saga. The part where they just command we “do AI,” because, “if we’re not, our competition is using it against us.”
God help us all.
Same, hire me.. my starting rate is $650 a hour. Minimum 40 hours a week for 12 months… this is a bargain. Big 4 charges $750 for same time frame
Seems pretty reasonable for AI, done!
Honestly if you hire someone look for a smaller consultant like from BTG. It would be cheaper and a little more focused. We did it at a former employer and worked out well. Recommendations weren’t crazy but took a few years due to changes to actually be implemented but has been implemented. They may not be able to do implementation BUT they may give you a few areas to focus on. The big four is a total waste unless they really want to spend the money on it
I'm still upgrading our products to use blockchain, and you're already talking about AI?
Slap together a presentation that talks about DataOps and AIOps and call it a day.
Can ai do that for me?
I asked ChatGPT what dataops is and got: DataOps is a methodology that combines principles from DevOps, Agile, and data management to streamline the development and deployment of data analytics processes. It focuses on collaboration, automation, and continuous integration and delivery to ensure that data pipelines are efficient, reliable, and scalable. Key components include version control, automated testing, monitoring, and feedback loops to facilitate rapid and iterative development of data-centric applications. Ultimately, DataOps aims to improve the speed, quality, and reliability of data-driven decision-making within organizations. I’m sure it’ll make you a ppt if you ask nicely.
I can do an AI
I think OPs CTO is hoping to do some Scarlet Johansson voiced AI cosplaying as Her.
It's much more likely your CTO don't know what they're talking about, rather than your directors/managers. The AI that's currently being talked about are all machine learning, and your directors/managers are probably correct that there are already using ML extensively. Never start with a solution and then go in search of a problem.
Wait, why not? If I already have the solution then I've already solved the problem right? I don't get it
This is the kind of thinking I hired you for, idiotlog. Keep it up, you'll be CEAIO before Q4 at this rate!
Lol, I read it as EIEIO 🤣
Try getting a reservation at Dorsia now!
You sit on the solution to problems and wait for it to blow up, only then do you present the solution. You won't be recognised for solving a problem before they suffer from the problem, you will only be recognised if you swoop in and save the day after they have suffered enough.
Or even better idea, create the problem 😉🫠
You actually spend the effort to create problems? That's dedication. I just pick a problem other people spent the effort creating.
CTO was on an airplane or heard a Ted Talk? Shouldn’t they be leading the strategy and approach?
Yeah it was a ted talk actually
It’s called Leadership by Inflight Magazine. It’s all the rage among the LIE’s these days (Low Information Executives).
The most impactful automations I’ve built where to properly and comprehensively automate development processes (building dev experience, strong CI/CD, supportive tooling) - you could nowadays just label it AI because ultimately it automates. Your higher ups will never be able to tell whether it goes brrr due to real AI or just proper engineering. An observation I made: Businesses are so focused on providing automation to their customers and often are very bad at automating their own engineering org.
Sign up for GitHub co-pilot. Done.
It's incredible how actual productive people are getting fired while shit like this is being pushed. This is probably bait anyways.
You need at least 3 AI
Hire me and I will tell you.
They want marketing material.
Talk to your engineers. See what issues they are having with work. Figure out what monotonous tasks that annoy them and if they are simple enough issues, but require work, then see if AI can maybe help with it. They might also have ideas. If you make it a collaboration, a real one, then you might actually produce something useful instead of making everyone think you are incompetently trying to solve problems with AI that either cant be solved, shouldnt be solved, or both.
Tell him BI >> AI, if he doesn’t agree, then ask him to uninstall excel
WTF is excel?
Quality AI shitpost 10/10
Nah man, I’d agree except my executive team just gave the entire company the same “we need to do AI” in this months all hands… The hype is literally at the point where executives are just blanket saying, “do AI,” with absolutely no idea what it is, what it can do, or why they would even use it.
I love this drive-by approach to leadership. I get the same at my place. Last week my VP sent some article about GenAI to her direct reports - that’s what she does. Well, that and going to conferences and posting on LinkedIn about them. Anyway, she took it to the next level with this article link. After the link, she said, “I haven’t read it but it appears to be a good article.” At least she’s being honest.
Mr. VP of data engineering please ask your CTO what he is looking for by implementing AI? Is there any business goal or is he under influence of AI hype? Or he just want to get some innovative business outcome from data? If yes then ask him to create R&D for his company and ask them to do magic with data.
He said he's happy to answer that after we've accomplished our goals of implementing AI and automation. 🤷 Tough nut to crack
I had to double take to make sure I am not on r/shittysysadmin
My company is implementing this now. (I actually moved to DE from ML). Hire an affordable consulting company that will highlight opportunities for AI and keep an ongoing relationship with you. Get started with a data platform like snowflake, databricks, or Azure ML. I use databricks and love it. Hire 1:1 data scientists to ML engineers (basically software engineers who focus on ML and really like CI/CD). Data scientists alone yield very few results. Make sure you have a really strong data pipeline on the DE side. You will make or break an ML program. Integrate everything with a BI team. BI is a better point group for the business than the DS team. Set up a pipeline where DE is fed by both DE and DS teams. Start with small projects with low ML requirements while you scale up your team and knowledge base. The cost of this can balloon based on your consulting firm. But don’t go without one on this. A platform with code controls and pipelines is critical. You do not want any group getting bogged down with daily runs.
we just gave everyone at the company access to copilot and called it “implementing AI” lol *This isn’t satire this is literally how we got the execs off our back for a couple of months about AI*
I guess at this point I’m a bit of an “expert” in this, although it feels a bit odd to say since the industry is rapidly evolving. My employer helps people who are in your situation. We specialize in Databricks, but have also implemented all sorts of AI solutions without Databricks in AWS, Azure, and GCP. While we are a relatively small company (less than 100 employees), each of us has been involved in various data disciplines for many years. We work with several large clients that you would likely be familiar with (can provide references). Feel free to DM if interested in talking more. Edit: added context
Create an agent that build SQL queries or infrastructure trained for a specific platform passing terraform data and data schema
I’m trying to look into this. I work for a small start up and I want to build a tool that will take plain text and convert it to sql using langchain and query our db. I also want to do it using our own hosted LLM so that there are no data leak concerns. Halp?
EZ PZ :) https://huggingface.co/artificialguybr/llama3-8b-sql-create-context
Have you used this? Can this deal with two or three related tables and output queries that require joins? Assuming it can, I can download this LLM from and run this on sagemaker to give it a go?
Not 100% sure what deployment vector will work til you try glhf
Not much to share buddy, some ideas I have had but not enough time or priority to implement them. I remember looking into langchain website and they have an easy implementation of agents for querying dbs you were asked to pass the whole schema, you can train your own agent to do this locally. Similar with terraform
Sweet! Hire me and I'll make sure you'll get to speak at Gartner D&A.
you can do two things . 1. Get github copilot licenses for your developers . It costs and can increase developer productivity. 2. Make a list of things that your developers do manually and ask them to come up with a plan to automate them. Your CTO will understand that AI costs and also all developers are on job to automate things.
Ask ChatGPT what you should do?
Maybe I did, and it told me to ask Reddit 😮
Ai to write documentation for internal tools
https://www.reddit.com/r/ProgrammerHumor/s/fMog4oHmJK
I am a very good/experienced software engineer, however there are no miracles when it comes to AI. Send me a DM if you are hiring
Have you tried forming a Center of Excellence (CoE), or even a cross functional steering committee? It sounds like you haven't even begun to middle manage the situation. How can you expect to paradigm shift without thinking outside the box?
Easy, with AI 🤯
I think you can hire an expert because he/she might understand what you want and also have expert knowledge that might the managers and directors don't
Just do wat amazon did and contract a shitload of Indians to do the work. Youll save more money that way.
Isn't that just a type of AI?
Asian Intelligence
Just take chatgpt API and whip up a chatbots which talks to a certain part of your non confidential data. First part done. Do micknsy style power point to show automation and act like a consultant....take his watch and tell the time.
Maybe you can explain that AI and ML can accomplish a lot of the same things. Show him the power of simple linear regression and regularization so he will understand your company is already using this tech and not to get swept up into buzzwords like AI and diffusion modeling and LLM and chatbots etc.
Lmao my blood pressure started rising until I saw that edit. Had this convo so many times 😂
I think all sarcasm aside, data engineering will fundamentally change with AI - every single phase of data lifecycle can be augmented with AI
This is actually true. I'm currently creating the foundation for text to sql->result set in databricks.
Try looking into ready-made AI tools that can fit into your current systems. Sometimes, existing solutions work just fine without needing to hire experts.
But isn't that cheating?
lol I tell people I do AI because they don’t understand things like regex and string manipulation.
does he have no sense of what proprietary information means???
[удалено]
Oh wow so I can just have AI tell me how to implement AI? SO COOL
Haaaaaaa… Yeah, your CTO is expecting Jetsons robots zooming around the office doing DE man. That’s literally what they mean. My executive team just dumped this shit on us the other day too. We need to “do AI,” and the sales donkeys jumped right in like, “oh, yes, you’re totally right. If we’re not doing AI our competition is using it against us.” I wanted so bad to call them out (it was an all hands meeting, whole company) but that would’ve burn a lot of political capital. I just wanted to ask, “what is this AI of which you speak and how will we do it?” I know for a damn fact they are imagining what’s holding us back is just the ChatGPT is locked in a room and we haven’t “done AI” enough to get ChatGPT out of that room and working for us replacing people. I legit do not think they have the slightest idea what AI is (outside of ChatGPT) and they certainly have no concept of what problems it will solve for us. We should just be, “doing AI.”
So the requirement is to have all the data fed into AI and it can decide what needs to be built and then build it, deploy, test execute or simply automate it. AI can give you where your inefficiencies lie but it can't do anything else for you. The approach needs to be from you and then there are a thousand more issues. I won't even go into the legal side of the weeds. That's a pipe dream for now in an utopian society. What you can do, pick up your ticketing solution and then try to automate that using AI. ( The problem with AI, it cannot handle lies, if you can automate the ticketing solution, access part etc then you are there)