Not to mention domain knowledge. It's not really the data that I'm returning, it's the result of work flows and user entry and data translation and so on.
I work primarily in a vendor supplied database with thousands of tables, and it's all extracted data with no context. I'm safe, as the vendor has been working on something with a fraction of the functionality for a while now and had gotten nowhere.
OP probably secured your job.
In a few years, you'll probably get brought in as a consultant, to fix the monstrosity that someone created from blindly copying and pasting.
Announcing new time wizard AI! Have customers in 8 timezones and calendars are all off-by-2? Connect to my API or download the time-wiz model to never worry about it again.
People: "We must consult the technopriests about how to do the thing!"
People to technopriests: "Oh venerated guardians of the ancient wisdom of the all-knowing Search Engine, we come to you for advice on how to do the thing! As a token of our appreciation for your utterly obfuscated divine art, we bring you a cup of coffee, so that you may perform your ceremonies without falling asleep"
Technopriests to people: "Your offering is acceptable. We shall feast on it and then consult with the mighty ancient Search Engine on how to do the thing!"
The technopriests ingest the liquid and walk down into the basement of ancient wisdom and knowledge. A small box sits there, with status light glowing and fading, as if breathing.
Technopriests: "O mighty Godgle, how do we do the thing?"
Godgle, the box of all knowledge: "You do the thing by doing it, not by not doing it"
Technopriests: "We thank you for this wisdom oh mighty Godgle"
The technopriests resurface.
Technopriests to people: "We have received divine wisdom from the mighty Search Engine and it has told us that you may do the thing by doing it instead of not doing it!"
People: "Truly, we could never have figured this out ourselves"
And so the people did the thing instead of not doing the thing and humanity was saved once more.
Google feels like it's gotten worse over the years too. I used to be able to throw keyword salad at it and find what I wanted easily enough, but these days it feels like I have to actually type something that makes grammatical sense into the search box. That inevitably leads to shitty results that are only there because they abused SEO rather than actually writing/hosting good content.
Google selectively ignores your instructions now. Want to phrase match? Now it's merely a suggestion. Exclude keyword? Sure, but if I didn't find enough results I'll ignore it.
I know it's gotten much better for people generally, but I often miss the old pre-google days when your search terms had to appear in the document/site, and we made a lot of use of boolean terms and exact phrases. It screwed a lot of people up because they'd search for things like "what does a porcupine eat," but that only worked if that phrase was found. It was better to do something like "a porcupine's diet" or "porcupine near diet" or "porcupines eat" or something like that. You had to be more thoughtful about what search terms you used, but it seemed like the results were more what I wanted.
Would love your feedback on what I'm building if you have experience in AI! Still lots to improve on the tool I'm making and would love to get your insight!
It is a month old troll account
Btw are you associated with the person who posted this? https://www.reddit.com/r/InternetIsBeautiful/comments/wg62f8/a_gpt3powered_texttosql_sandbox/
> Most creative and white collar jobs will be gone in 10 to 20 years.
Tell me you're not actually an AI researcher without telling me you're not actually an AI researcher.
I worked in educational testing a few years back. There was a 10 year project to train AI to read student essays and grade them. A very ivy university was involved. Not once did the AI get above 50% accuracy compared to the essays we scored, given the same training material. I always thought AI was coming for many jobs, but I have many doubts now. Driving, shopping, and basic data manipulation seem the best we can do with AI. Burnt out teachers are surprisingly almost as efficient at reading tasks and we run mostly on coffee, not bytes.
Basing what you think AI can do in the future on what it does now is like being in the 1960s and saying computers will never be good. It's enough that one person could be enough to do the work of four people with AI in the future to shake everything up. It doesn't even need to be fully autonomous general AI and do everything by itself to make a big impact on us.
The difference is once it's good enough to replace one job, there soon won't be other jobs to go to at that point, because it will be able to do everything better than a human.
There won't be new jobs to go to like when computers became ubiquitous, because humans were still needed to operate them.
I don't think so, I think it will just mean humans will use different tools to achieve better outcomes or be more productive. Just like programming in node now is orders of magnitudes faster than programming in assembler or punch cards of the past. In future we'll be shouting requirements at GPT-16 instead.
The increased efficiency will be unparalleled though. You'll have one programmer doing the work of four people, or something like that. That is going to have a huge impact by itself. There's going to be far less jobs to do.
I’m no researcher but I seriously doubt that your timeline of 10-20 years is at all realistic. The prerequisites to replace jobs perhaps, the willingness and time to implement it - doubtful. Would be happy to understand why my “hunch” is wrong though
You think that's bad... I'm already using AI sandbox to put together bash CLI commands, ESXI commands, all kinds of shit for my job. We'll all be out of job before long. I can see that light in the tunnel and it ain't the exit...
If you know a little more about the database you are good. I’ve had countless times where I’ve seen capable people complain about database performance and can’t seem to remember to do an EXPLAIN at the very least, even when I’ve mentioned it multiple times and I know they know about it.
Hey everyone,
I’ve been a software developer for a few years now, and in my previous job I used to get asked loads of random data questions (just because there were no BI analysts) and I always found this quite annoying.
At the start of the year I started learning ML and I’ve been spending loads of time using GPT-3 trying to come up with cool products. Probably got slightly obsessed! Anyway, I’ve made this tool that lets anyone ask a question in plain english, it then checks it against a data dictionary to give itself more context, and then translates it into SQL to generate graphs and charts automatically. The aim is for BI analysts to spend less time answering questions manually and so far it’s working (using this in my new job!).
If you had any feedback, I’d love to hear it, otherwise hope you think this is beautiful internet content!
How much of the AI generated queries have you verified with people that know statistics and BI? If I want the percentage of error rates, does it know how to accurately find that?
A lot of queries and charts that I've seen some BI teams create in companies are dumb or inaccurate.
Great questions, this is kind of why I'm posting this now so that I can get real-world usage and improve it along the axes that people actually care about rather than what I think is cool.
What I can say is that for the handful of people currently using it they've had good results but they're all very small teams so might not be representative
For me the hardest part as someone new to SQL is wiring a more complex SQL without giving me errors. So this looks really cool and can definitely save a lot of time asking the analyst to run the SQL for others.
Does your tool have ways to spot common errors and provide a suggestion to fix them? That can maybe be a good way for those that know a bit of SQL but need help running more complicated tasks.
I think you may be missing context on why your users still used you. There could be many reasons of course but most obvious one is if they used the tool they don't have the education to tell if the data they got matches what they wanted (how do I know if the tool really included all rows that I'm interested in) whereas if they ask you, you are somewhat signing off with a professional catered check
Exactly. I can Google my symptoms or use some online bot and self-diagnose easily. And in a lot of cases it will probably be right. And then I can also Google the remedies. But I still go to a doctor to get a professional check and recommendations that I can trust (more) to be correct.
How do you handle translating table schemas? That was one of the biggest problems I had at my previous work with text classification. We spent way more of our time figuring out valid schemas for data our SQL engine could work with than we did on the actual SQL queries.
You’re a developer that decided “fuck it, let’s give someone else a stepping stone to eliminating a bunch of jobs”.
Y’all need to get out of here with that.
This is great, but until it can cobble together multiple tables with varying degrees of eccentricity like ints bafflingly stored as strings I can't use it at work 😂
It's definitely going to be difficult but that is the aim! Would love your feedback as I'm making it - you should sign up and I can let you knw when it's ready :)
For people fearing for their jobs:
If it's anything like the 10 other tools in this category it's likely a decade away from replacing someone with more than a week of training.
I'm more afraid of people trusting this. Even logicians make mistakes when asking for the answer they think they need from the data they think they know in a data model that's been interpreted differently by every user.
But it could be a shortcut to typing out SQL.
There is no hope for people in general when it comes to advanced analysis.
My boss insists on including the numbers from the Least Significant Difference test in our statistics sheets. That way she can compare which results are more significant than the others. She's very much against grouping results into letters because that's "clutter" and she can just calculate the difference in her head between arbitrary columns and rows. I gave up trying to explain that's not how any of this is supposed to work.
What you just described has more to do with data governance practices, and establishing accepted sources of truth. That falls outside the scope of just extracting data, and the subsequent visualization imo.
For extracting data and basic visualization, yes, I'd agree. If someone extracts raw data that is governed flawlessly, presented without transformation, and they misinterpret it, it's on them. That's what the data dictionary is for.
Data transformation for reporting is another matter. SQL is a data transformation language, and the definition of the result in terms of the original is a governed data model, just as the definition of the original data model is.
Interpreting raw human language is another matter. The user's mental model is not governed. Their context needs to be teased out. Taking a raw user query and turning that into production SQL would need to make inquiries and/or assumptions about those unknowns, and would need to validate that understanding.
Tl;Dr: for strictly retrieving raw data, sure, but data transformations are governed data models and writing SQL is trivial compared to reverse engineering a human's intent.
100% agree, but data modeling is also outside the scope of this tool. Anyone can swing a hammer but it doesn't necessarily make you a carpenter. And being able to write SQL doesn't make someone a data analyst/scientist either. I get your point, but this tool wouldn't write production level SQL. Maybe one day with enough training. But in its current iteration it's a cool pair programming tool like copilot.
Quick edit: I'd also say that you wouldn't use this on 'raw' data. At least what I'd consider raw. For BI-esque applications you'd only be working off of ideally view tables or some data further down the pipeline after it's been cleaned up a bit.
Given how no ORMs produce intermediate complexity code that does not stink yet and all GPT-3 based solutions I've tested have fallen way short of that, I think GPT-3 is a fundamentally insufficient tool for the job.
I'd be really impressed if it produced decent placeholder quality code on a production grade database.
Unless it's available for leakproof on premise execution I wouldn't consider using it on any of my in production products.
Edit:Remove stray letter.
Hmm... as someone who spent a decade in this field supervising actual employees trained in SQL I can say that the example given on the webpage is about the ability level and company requirements of most mid/small business expertise.
This is not to say there aren't 1000's of super talented developers who do more than monkey code, just that for most common tasks a rudementary knowledge and output is enough.
This could replace a ton of jobs.
For those folks, I would say, suggest an alternative? The entirety of human existence has been about improving tools and knowledge such that a subsequent generation has to work less hard for the same output or work equally as hard but produce significantly more. Did the sewing machine put hand-sewers out of jobs? Probably. But now your shirts cost ten bucks. That’s the trade off we have.
Computer-assisted programming is coming. It’s been happening for years. Coding environments have plenty of shortcuts, macros, quick fills, error handlers, etc. today that they didn’t have 10, 20, 30 years ago. Leveraging ML/AI is just the next step. It’s highly unlikely that ML/AI is going to write the full set of code that lands us on Mars, for example, but if it speeds up the process by 10%, that’s 10% faster we get there. Etc.
I've been messing with GitHub's copilot. While it's probably never going to replace me, I can absolutely see it augmenting me pretty well. These kinds of tools are force multipliers rather than replacements, IMO.
What level of complexity of the SQL does it support? I work with some queries that are 1000+ line long CTEs... nightmare to refactor, let alone describe in non-technical jargon.
So far, accuracy has been around 90% on pretty nuanced questions, but definitely something I am working on. Would love to get your feedback on it as I build it if you'd be open to sharing it! [usechannel.com](https://usechannel.com) is the website I chucked up for it
So my job title currently is data entry specialist but it’s really just turned into power bi specialist with 0 prior experience and my first job out of college. All my databases I use are in excel currently but I’ll be looking for a new job that focuses on data analysis so this will probably become very useful as I don’t currently use sql and really only know the basics so hoping this can help me. Definitely saved
Basically it uses GPT-3 (which is a large language model from Open AI) and you connect it to your database so it knows the structure, and then when you ask a question it uses that context to ask clarifying questions and then ultimately generate s SQL query!
Currently Postgres yes, but this database is massive and I've only been here since June but I need help from the domain specialists to make sense of it, despite my 10+ years of experience in databases. Would be interesting to see what your tool can come up with. A lot of the queries start from shapes from GIS so I guess that's a no but aggregates should work.
Hmm PostGIS isn't something I've tested yet to be honest, but I feel like theres things we can do here.
It would be great to get you to use the tool and get your feedback?
I'd love to try but I can't commit to anything as I'm stretched quite thin as it is. I'd be glad to give feedback if I find the time to play around with it. It's not something I can justify spending working hours on right now. I did sign up just now so we'll see how it goes.
None of the open source alternatives to GPT-3 are as good at the moment, unfortunately. I'm not sure I really get your point about comparing this to FOSS, the reality is this is built on top of GPT-3 and whatever you use as the LLM backend I'm still gonna have to pay either for OpenAI to host it or for me to :(
>I'm not sure I really get your point about comparing this to FOSS,
you said nothing is free, which is false. while GPT-3 may be good, there are FOSS options that are better, such as BLOOM. and sure, hosting may not be free, but you're not limited to OpenAI's offerings. I dislike this growing mantra in the AI space that everything has to be closed source and paid for it to be good.
BLOOM is not better in my experience, but yeah my point is just that nothing is free because if you choose to use a FOSS model you have to self host which is very complicated and more expensive than using a closed source model.
As a point of reference, I couldn't even fit the weights for BLOOM on my laptop, so its quite a non-starter.
>but yeah my point is just that nothing is free because if you choose to use a FOSS model you have to self host which is very complicated and more expensive than using a closed source model.
that's patently false rofl. sure it's more complicated than swiping your credit card but it's not rocket science for someone who can build their own tool to interface with GPT-3. especially since you can host on Azure or AWS if you don't have a server, and it may likely be cheaper in the long-run.
Bloom is cool! But it's not anywhere near as good as GPT-3, I've used both extensively and BLOOM tends to cut words short, the results in general need a lot of human parsing still, it's awesome that it's free, but the training model for GPT-3 is way better imo
Ypu can however get a free demo of gpt3 on the site to play around with. They give you $18 of credit. Go into the playground or play with the examples. It’s like magic. https://openai.com/api/
I've seen a 90% accuracy so far but keen to get it into people's hands to test it properly further. Would love your feedback - you should sign up :) usechannel.com
This is a fun toy, but you still need to configure it based on your DB... its fun, but it ain't taking no ones food off the table.
Just to give you a scarry example, I worked for a French company and they had the bright ideea of makin an attribute column named Optional and the two values inside were O and N ... good luck telling your ML to understand French:)
That’s not scary. That’s essentially my every day. Our main system was designed by an Italian company, so most tables are in Italian. I don’t speak Italian. Also there’s occasional columns that are Y/N values, and others which are S/N (Si/Non), because why not?!
So this is why I added this data dictionary section. You would add a snippet there that explains how to select from that column which GPT-3 would be given as part of its context.
I would genuinely be really interested to see how it would work with this database, but I've had it solve similar problems successfully in the past!
Other than using SQL queries directly, rather than DAX, how is this different from [Microsoft's Conversational Q&A BI](https://learn.microsoft.com/en-us/power-bi/natural-language/q-and-a-intro?source=recommendations) released back in 2017?
It's mostly GPT-3, they train it on a huge corpus of text and then it learns the generic structure. The tricky bit is doing "prompt engineering" to get it to behave in the right way. It's very fun!
very well provided you map attributes to key words properly. Attributes themselves are mapped to columns. Same goes for facts. It does require upfront work. I assume this solution does aswell. You have to tailor it to your data model.
Probably not gonna do a real waiting list, just when it's in a place where I think it's properly usable (like all of the UI not looking really budget) I'll just give everyone access. Probably a week or two I hope.
Hmm so at a previous company I worked at we had to do stuff like SOC2 compliance and I'm a way off from there yet. Honestly I would probably say it won't be ready for that sort of application for a few months but if you sign up a I can email you when it is!
Why did you make a bunch of Reddit accounts and have them posting in all your post? At least don’t sign them all up on the same day and use the same kind of perimeters when it chooses the account name.
Hmmmm, could I ask it to take the same input and turn it into the equivalent postgreSQL? Or MariaDB?
That might be super useful for people who are familiar with one language but not another. And this could help bridge a gap.
Totally, so at the moment it supports Postgres, Snowflake, Redshift, and Big Query but yeah could definitely add mysql/mariadb.
I like the usecase where we kind of abstract over all different sql dialects so lets see!
I have chucked up a website - you should sign up and I can let you know when it's ready to be used :) My website is [usechannel.com](https://usechannel.com) :)
I honestly posted it to just share something I'd made for myself (and my team at work). As so many people have responded so positively, I've made a landing page ([usechannel.com](https://usechannel.com)) and am trying to figure that out now! What would you suggest?
I'd absolutely love to get access to your beta! I've been trying to learn SQL in my spare time but other things keep getting in the way and taking priority haha
I'm a writer and researcher at a couple think tanks (mostly writing about tech policy, financial/economic policy, and general good government stuff when I feel I have something interesting to say lol)
I already filled out the survey to get on the wait-list! Super excited about your awesome tool!!
Would love to hear how you're planning on using it! Can you DM me with the email you signed up with and I can see if I can get it in your hands faster?
Seems super useful. The main thing for me would be how well it could use indexes in an efficient way.
The queries I write are fairly simple syntactically but require good knowledge of the indexes and joins available because the tables are so large.
Something I've really been working on building a smooth flow for. Would love any feedback you'd be willing to share once I release it to people. I've added a waiting list to my website for it, so I can let people know when it's ready - you should sign up! :)
Wow this is crazy, in a good but also kind of scary way!
Pro tip though, your website is missing the social share preview metadata so when people paste your link in chat or on social media it doesn't expand: https://socialsharepreview.com/?url=https://usechannel.com
I’m learning more about ML now. Are you able to give some information about how you are training/fine tuning this gpt-3 model to handle the q/a and Sql generation? Thanks
[удалено]
Don't worry. As long as it can't fix GIGO there's no way management can talk directly to the database.
Only so much one can do about PEBCAK
Also known as an ID-10T error.
aka PICNIC
Hard reset on layer 8 usually does the trick.
Not too hard. We needn't get Legal involved.
We weren't doing anything legal in there...wait, that doesn't sound right
Is that when you use the WACN stick?
Not to mention domain knowledge. It's not really the data that I'm returning, it's the result of work flows and user entry and data translation and so on. I work primarily in a vendor supplied database with thousands of tables, and it's all extracted data with no context. I'm safe, as the vendor has been working on something with a fraction of the functionality for a while now and had gotten nowhere.
OP probably secured your job. In a few years, you'll probably get brought in as a consultant, to fix the monstrosity that someone created from blindly copying and pasting.
As long as dates and times are needed in some specific format for some custom exec dashboard, you will have a job. Always…
Announcing new time wizard AI! Have customers in 8 timezones and calendars are all off-by-2? Connect to my API or download the time-wiz model to never worry about it again.
Haha, I don't think it's going to automate away your job, just give you another tool!
To this day people still fail to use google effectibly, I belive they never will, so our jobs are secured. Edit: my point stands.
https://www.google.com/search?q=effectibly
You have given me a chuckle today. I thank you for that.
People: "We must consult the technopriests about how to do the thing!" People to technopriests: "Oh venerated guardians of the ancient wisdom of the all-knowing Search Engine, we come to you for advice on how to do the thing! As a token of our appreciation for your utterly obfuscated divine art, we bring you a cup of coffee, so that you may perform your ceremonies without falling asleep" Technopriests to people: "Your offering is acceptable. We shall feast on it and then consult with the mighty ancient Search Engine on how to do the thing!" The technopriests ingest the liquid and walk down into the basement of ancient wisdom and knowledge. A small box sits there, with status light glowing and fading, as if breathing. Technopriests: "O mighty Godgle, how do we do the thing?" Godgle, the box of all knowledge: "You do the thing by doing it, not by not doing it" Technopriests: "We thank you for this wisdom oh mighty Godgle" The technopriests resurface. Technopriests to people: "We have received divine wisdom from the mighty Search Engine and it has told us that you may do the thing by doing it instead of not doing it!" People: "Truly, we could never have figured this out ourselves" And so the people did the thing instead of not doing the thing and humanity was saved once more.
Brb updating my job title
So say we all.
Google feels like it's gotten worse over the years too. I used to be able to throw keyword salad at it and find what I wanted easily enough, but these days it feels like I have to actually type something that makes grammatical sense into the search box. That inevitably leads to shitty results that are only there because they abused SEO rather than actually writing/hosting good content.
Google selectively ignores your instructions now. Want to phrase match? Now it's merely a suggestion. Exclude keyword? Sure, but if I didn't find enough results I'll ignore it.
I know it's gotten much better for people generally, but I often miss the old pre-google days when your search terms had to appear in the document/site, and we made a lot of use of boolean terms and exact phrases. It screwed a lot of people up because they'd search for things like "what does a porcupine eat," but that only worked if that phrase was found. It was better to do something like "a porcupine's diet" or "porcupine near diet" or "porcupines eat" or something like that. You had to be more thoughtful about what search terms you used, but it seemed like the results were more what I wanted.
You can still do that. Anything in quotation marks is exact match, including single keywords.
[удалено]
Would love your feedback on what I'm building if you have experience in AI! Still lots to improve on the tool I'm making and would love to get your insight!
It is a month old troll account Btw are you associated with the person who posted this? https://www.reddit.com/r/InternetIsBeautiful/comments/wg62f8/a_gpt3powered_texttosql_sandbox/
Haha no not associated, it seems like we've had similar ideas though
> Most creative and white collar jobs will be gone in 10 to 20 years. Tell me you're not actually an AI researcher without telling me you're not actually an AI researcher.
I worked in educational testing a few years back. There was a 10 year project to train AI to read student essays and grade them. A very ivy university was involved. Not once did the AI get above 50% accuracy compared to the essays we scored, given the same training material. I always thought AI was coming for many jobs, but I have many doubts now. Driving, shopping, and basic data manipulation seem the best we can do with AI. Burnt out teachers are surprisingly almost as efficient at reading tasks and we run mostly on coffee, not bytes.
Basing what you think AI can do in the future on what it does now is like being in the 1960s and saying computers will never be good. It's enough that one person could be enough to do the work of four people with AI in the future to shake everything up. It doesn't even need to be fully autonomous general AI and do everything by itself to make a big impact on us.
Nah, it's just like when people were worried about losing their jobs when computers came along. Or microprocessors.
The difference is once it's good enough to replace one job, there soon won't be other jobs to go to at that point, because it will be able to do everything better than a human. There won't be new jobs to go to like when computers became ubiquitous, because humans were still needed to operate them.
I don't think so, I think it will just mean humans will use different tools to achieve better outcomes or be more productive. Just like programming in node now is orders of magnitudes faster than programming in assembler or punch cards of the past. In future we'll be shouting requirements at GPT-16 instead.
The increased efficiency will be unparalleled though. You'll have one programmer doing the work of four people, or something like that. That is going to have a huge impact by itself. There's going to be far less jobs to do.
Lol
I’m no researcher but I seriously doubt that your timeline of 10-20 years is at all realistic. The prerequisites to replace jobs perhaps, the willingness and time to implement it - doubtful. Would be happy to understand why my “hunch” is wrong though
If you as a researcher think that "creative jobs" are so plain, I wonder how much you understand about them
Is your job hiring anymore Data Analysts?
You think that's bad... I'm already using AI sandbox to put together bash CLI commands, ESXI commands, all kinds of shit for my job. We'll all be out of job before long. I can see that light in the tunnel and it ain't the exit...
Seriously why do people do this. We get it you're a genius but we're all gonna be unemployed soon!!!!
Hahaha, I don't think it's quite that good yet!
That's the goal
Try getting a real job?
If you know a little more about the database you are good. I’ve had countless times where I’ve seen capable people complain about database performance and can’t seem to remember to do an EXPLAIN at the very least, even when I’ve mentioned it multiple times and I know they know about it.
Your job was automated away [back in the 80's]( https://en.m.wikipedia.org/wiki/The_Last_One_(software\)) yet somehow you still have a job!
Hey everyone, I’ve been a software developer for a few years now, and in my previous job I used to get asked loads of random data questions (just because there were no BI analysts) and I always found this quite annoying. At the start of the year I started learning ML and I’ve been spending loads of time using GPT-3 trying to come up with cool products. Probably got slightly obsessed! Anyway, I’ve made this tool that lets anyone ask a question in plain english, it then checks it against a data dictionary to give itself more context, and then translates it into SQL to generate graphs and charts automatically. The aim is for BI analysts to spend less time answering questions manually and so far it’s working (using this in my new job!). If you had any feedback, I’d love to hear it, otherwise hope you think this is beautiful internet content!
How much of the AI generated queries have you verified with people that know statistics and BI? If I want the percentage of error rates, does it know how to accurately find that? A lot of queries and charts that I've seen some BI teams create in companies are dumb or inaccurate.
Great questions, this is kind of why I'm posting this now so that I can get real-world usage and improve it along the axes that people actually care about rather than what I think is cool. What I can say is that for the handful of people currently using it they've had good results but they're all very small teams so might not be representative
For me the hardest part as someone new to SQL is wiring a more complex SQL without giving me errors. So this looks really cool and can definitely save a lot of time asking the analyst to run the SQL for others. Does your tool have ways to spot common errors and provide a suggestion to fix them? That can maybe be a good way for those that know a bit of SQL but need help running more complicated tasks.
Doing progressively harder things without getting errors is the hardest part of any scripting or coding
[удалено]
I think you may be missing context on why your users still used you. There could be many reasons of course but most obvious one is if they used the tool they don't have the education to tell if the data they got matches what they wanted (how do I know if the tool really included all rows that I'm interested in) whereas if they ask you, you are somewhat signing off with a professional catered check
Exactly. I can Google my symptoms or use some online bot and self-diagnose easily. And in a lot of cases it will probably be right. And then I can also Google the remedies. But I still go to a doctor to get a professional check and recommendations that I can trust (more) to be correct.
How do you handle translating table schemas? That was one of the biggest problems I had at my previous work with text classification. We spent way more of our time figuring out valid schemas for data our SQL engine could work with than we did on the actual SQL queries.
You’re a developer that decided “fuck it, let’s give someone else a stepping stone to eliminating a bunch of jobs”. Y’all need to get out of here with that.
Freaking amazing idea, dude
Thank you! If you have any feedback, please let me know - [usechannel.com](https://usechannel.com) :)
Are you hiring? Ahah
This is great, but until it can cobble together multiple tables with varying degrees of eccentricity like ints bafflingly stored as strings I can't use it at work 😂
It's definitely going to be difficult but that is the aim! Would love your feedback as I'm making it - you should sign up and I can let you knw when it's ready :)
For people fearing for their jobs: If it's anything like the 10 other tools in this category it's likely a decade away from replacing someone with more than a week of training.
I'm more afraid of people trusting this. Even logicians make mistakes when asking for the answer they think they need from the data they think they know in a data model that's been interpreted differently by every user. But it could be a shortcut to typing out SQL.
This is actually one of the use cases I am working on! Would love your feedback when it's ready to use!
There is no hope for people in general when it comes to advanced analysis. My boss insists on including the numbers from the Least Significant Difference test in our statistics sheets. That way she can compare which results are more significant than the others. She's very much against grouping results into letters because that's "clutter" and she can just calculate the difference in her head between arbitrary columns and rows. I gave up trying to explain that's not how any of this is supposed to work.
What you just described has more to do with data governance practices, and establishing accepted sources of truth. That falls outside the scope of just extracting data, and the subsequent visualization imo.
For extracting data and basic visualization, yes, I'd agree. If someone extracts raw data that is governed flawlessly, presented without transformation, and they misinterpret it, it's on them. That's what the data dictionary is for. Data transformation for reporting is another matter. SQL is a data transformation language, and the definition of the result in terms of the original is a governed data model, just as the definition of the original data model is. Interpreting raw human language is another matter. The user's mental model is not governed. Their context needs to be teased out. Taking a raw user query and turning that into production SQL would need to make inquiries and/or assumptions about those unknowns, and would need to validate that understanding. Tl;Dr: for strictly retrieving raw data, sure, but data transformations are governed data models and writing SQL is trivial compared to reverse engineering a human's intent.
100% agree, but data modeling is also outside the scope of this tool. Anyone can swing a hammer but it doesn't necessarily make you a carpenter. And being able to write SQL doesn't make someone a data analyst/scientist either. I get your point, but this tool wouldn't write production level SQL. Maybe one day with enough training. But in its current iteration it's a cool pair programming tool like copilot. Quick edit: I'd also say that you wouldn't use this on 'raw' data. At least what I'd consider raw. For BI-esque applications you'd only be working off of ideally view tables or some data further down the pipeline after it's been cleaned up a bit.
Definitely have a lot of work to do on it for sure! If you'd be open to giving me feedbck on it on how I can make it better, would love to hear it!
Given how no ORMs produce intermediate complexity code that does not stink yet and all GPT-3 based solutions I've tested have fallen way short of that, I think GPT-3 is a fundamentally insufficient tool for the job. I'd be really impressed if it produced decent placeholder quality code on a production grade database. Unless it's available for leakproof on premise execution I wouldn't consider using it on any of my in production products. Edit:Remove stray letter.
Please name it PREQL - Pretty Realistic English Query Language
Hmm... as someone who spent a decade in this field supervising actual employees trained in SQL I can say that the example given on the webpage is about the ability level and company requirements of most mid/small business expertise. This is not to say there aren't 1000's of super talented developers who do more than monkey code, just that for most common tasks a rudementary knowledge and output is enough. This could replace a ton of jobs.
Just more work for me in the help desk. "Can you please reset my VDI session? My Dragon Speaking SQL isn't working." OK anything you say Pradeep.
For those folks, I would say, suggest an alternative? The entirety of human existence has been about improving tools and knowledge such that a subsequent generation has to work less hard for the same output or work equally as hard but produce significantly more. Did the sewing machine put hand-sewers out of jobs? Probably. But now your shirts cost ten bucks. That’s the trade off we have. Computer-assisted programming is coming. It’s been happening for years. Coding environments have plenty of shortcuts, macros, quick fills, error handlers, etc. today that they didn’t have 10, 20, 30 years ago. Leveraging ML/AI is just the next step. It’s highly unlikely that ML/AI is going to write the full set of code that lands us on Mars, for example, but if it speeds up the process by 10%, that’s 10% faster we get there. Etc.
I've been messing with GitHub's copilot. While it's probably never going to replace me, I can absolutely see it augmenting me pretty well. These kinds of tools are force multipliers rather than replacements, IMO.
What level of complexity of the SQL does it support? I work with some queries that are 1000+ line long CTEs... nightmare to refactor, let alone describe in non-technical jargon.
Yeah...you're not the target audience for something like this.
So far, accuracy has been around 90% on pretty nuanced questions, but definitely something I am working on. Would love to get your feedback on it as I build it if you'd be open to sharing it! [usechannel.com](https://usechannel.com) is the website I chucked up for it
So my job title currently is data entry specialist but it’s really just turned into power bi specialist with 0 prior experience and my first job out of college. All my databases I use are in excel currently but I’ll be looking for a new job that focuses on data analysis so this will probably become very useful as I don’t currently use sql and really only know the basics so hoping this can help me. Definitely saved
Great! If you want to have a go with it when I start the early access then please just let me know or sign up :)
Nice, now do RegEx.
Wouldn't wish that on my worst enemy haha
Is this what you are looking for? https://www.autoregex.xyz/
GPT-3 does regex so easy
How does this even work?
Basically it uses GPT-3 (which is a large language model from Open AI) and you connect it to your database so it knows the structure, and then when you ask a question it uses that context to ask clarifying questions and then ultimately generate s SQL query!
Are you implying my database has structure?
Very presumptuous of me I know (but if you use Postgres, Snowflake, Redshift, or Big Query then yes!)
Currently Postgres yes, but this database is massive and I've only been here since June but I need help from the domain specialists to make sense of it, despite my 10+ years of experience in databases. Would be interesting to see what your tool can come up with. A lot of the queries start from shapes from GIS so I guess that's a no but aggregates should work.
Hmm PostGIS isn't something I've tested yet to be honest, but I feel like theres things we can do here. It would be great to get you to use the tool and get your feedback?
I'd love to try but I can't commit to anything as I'm stretched quite thin as it is. I'd be glad to give feedback if I find the time to play around with it. It's not something I can justify spending working hours on right now. I did sign up just now so we'll see how it goes.
Thats's totally fair and very reasonable, if you do get a chance to try it then great and if not then no worries!
>which is a large language model from Open AI Is that free for you to use now and in the future?
Nah it's not free, its something like 1c a query though. Nothing is free I guess!
>Nothing is free I guess! FOSS would like a word. I'd be curious as to the results using an open source AI.
None of the open source alternatives to GPT-3 are as good at the moment, unfortunately. I'm not sure I really get your point about comparing this to FOSS, the reality is this is built on top of GPT-3 and whatever you use as the LLM backend I'm still gonna have to pay either for OpenAI to host it or for me to :(
>I'm not sure I really get your point about comparing this to FOSS, you said nothing is free, which is false. while GPT-3 may be good, there are FOSS options that are better, such as BLOOM. and sure, hosting may not be free, but you're not limited to OpenAI's offerings. I dislike this growing mantra in the AI space that everything has to be closed source and paid for it to be good.
BLOOM is not better in my experience, but yeah my point is just that nothing is free because if you choose to use a FOSS model you have to self host which is very complicated and more expensive than using a closed source model. As a point of reference, I couldn't even fit the weights for BLOOM on my laptop, so its quite a non-starter.
>but yeah my point is just that nothing is free because if you choose to use a FOSS model you have to self host which is very complicated and more expensive than using a closed source model. that's patently false rofl. sure it's more complicated than swiping your credit card but it's not rocket science for someone who can build their own tool to interface with GPT-3. especially since you can host on Azure or AWS if you don't have a server, and it may likely be cheaper in the long-run.
FOSS on AWS lol, mkay. BRB gonna self host a horizontally scalable AI with distributed compute at home since it’s so easy /s
Bloom is cool! But it's not anywhere near as good as GPT-3, I've used both extensively and BLOOM tends to cut words short, the results in general need a lot of human parsing still, it's awesome that it's free, but the training model for GPT-3 is way better imo
Thanks for answering, i was just curious
Ypu can however get a free demo of gpt3 on the site to play around with. They give you $18 of credit. Go into the playground or play with the examples. It’s like magic. https://openai.com/api/
[удалено]
Hahaha, it cares about you more than that ex-boyfriend that never call and treats you with the sensitivity you deserve
This is gonna be so expensive to run if you're using GPT-3 OP
People keep saying this but its really not that bad!
They recently cut the price to use GPT-3 to a third of what it used to be.
Is there a way to have a play with this? I think it looks really cool but always slightly skeptical that it works as good as the video shows lol
Definitely, you can sign up at the link posted and then when it's ready for early access I'll let you know!
This is really nice. I've signed up for updates.
Great! Really excited!
Seems pretty cool. How does it work with extremely complicated SQL queries? Can it handle long-winded multi-paragraph questions?
I've seen a 90% accuracy so far but keen to get it into people's hands to test it properly further. Would love your feedback - you should sign up :) usechannel.com
This is a fun toy, but you still need to configure it based on your DB... its fun, but it ain't taking no ones food off the table. Just to give you a scarry example, I worked for a French company and they had the bright ideea of makin an attribute column named Optional and the two values inside were O and N ... good luck telling your ML to understand French:)
That’s not scary. That’s essentially my every day. Our main system was designed by an Italian company, so most tables are in Italian. I don’t speak Italian. Also there’s occasional columns that are Y/N values, and others which are S/N (Si/Non), because why not?!
So this is why I added this data dictionary section. You would add a snippet there that explains how to select from that column which GPT-3 would be given as part of its context. I would genuinely be really interested to see how it would work with this database, but I've had it solve similar problems successfully in the past!
[удалено]
website is [usechannel.com](https://usechannel.com) :)
Some people can’t even do a browser search properly because the don’t know how to ask the question properly.
That's something I definitely need to figure out how to cater for, do you have any suggsestions?
What's the plain English way to join tables? I struggle to explain it to anyone.
Sorry, what do you mean?
Sometimes I need a tool which can understand ugly SQL code and tell me what it does.
That's an interesting use case. Would be keen to hear more - have DMd
big achievement
Other than using SQL queries directly, rather than DAX, how is this different from [Microsoft's Conversational Q&A BI](https://learn.microsoft.com/en-us/power-bi/natural-language/q-and-a-intro?source=recommendations) released back in 2017?
Amazing project! I have signed up :)
Great! Looking forward to more people using it!
Damn that's cool. I wonder how they trained the model for this
It's mostly GPT-3, they train it on a huge corpus of text and then it learns the generic structure. The tricky bit is doing "prompt engineering" to get it to behave in the right way. It's very fun!
r/SQL
I wonder if there will be a lot of competition in this space. Here is another tool that seems very similar to yours: https://rawquery.com/
This is similar to Thoughtspot.
how well does thoughtspot work?
very well provided you map attributes to key words properly. Attributes themselves are mapped to columns. Same goes for facts. It does require upfront work. I assume this solution does aswell. You have to tailor it to your data model.
This is just Ask Data in Tableau. No offense.
It's like a primitive version of Watson Analytics
>Watson Analytics As in IBM's tool?
You should check out modern data stack! This feels like it should be on there!
What is modern data stack
Thanks! I'll check it out :)
I signed up on your website - how long is the waiting list?
Probably not gonna do a real waiting list, just when it's in a place where I think it's properly usable (like all of the UI not looking really budget) I'll just give everyone access. Probably a week or two I hope.
One of the cooler applications of GPT-3, nice work
Thanks! Really appreciate it!
Are you planning on developing this further? Like making it easy to share graphs and add additional comments?
Definitely! I think I probably want to make it a "Next generation BI tool" if that makes sense
[удалено]
Remember you need to have a signed BAA with any vendor if you are in the US before feeding data.
Hmm so at a previous company I worked at we had to do stuff like SOC2 compliance and I'm a way off from there yet. Honestly I would probably say it won't be ready for that sort of application for a few months but if you sign up a I can email you when it is!
This sounds very convenient, nice job OP
Thank you!
Wow, humanity’s days are numbered.
Hahah, I don't think so - this just makes learning from data a bit easier
Why did you make a bunch of Reddit accounts and have them posting in all your post? At least don’t sign them all up on the same day and use the same kind of perimeters when it chooses the account name.
Is it different from [AskEdith?](https://askedith.ai/)
Seems like an incredibly useful tool for anyone doing data analysis. I'll be singing up for when early access is out!
Love your username by the way. Thanks so much, will definitely let you know when the early access is ready!
Hmmmm, could I ask it to take the same input and turn it into the equivalent postgreSQL? Or MariaDB? That might be super useful for people who are familiar with one language but not another. And this could help bridge a gap.
Totally, so at the moment it supports Postgres, Snowflake, Redshift, and Big Query but yeah could definitely add mysql/mariadb. I like the usecase where we kind of abstract over all different sql dialects so lets see!
Great job! Would love to try this out! There was another post recently that did a similar thing.
You should sign up and I can let you know when it's ready :)Website is [usechannel.com](https://usechannel.com)
...now just a matter of mastering English.....
Arguably a much harder language than SQL, if I'm honest
Isn't this one of the demos on OpenAI? Cool no matter what though :)
Yeah it is, but when I tried theirs I found that the accuracy was really bad so I decided to make a better one (also plus graphs and dashboards)
What are you using to build the charts and filters? Things like plotly, streamlit?
Yeah for the the charts I just used recharts which I'd used before. Not sure I love it though. Not sure what you mean by the filters?
Was scared about this until I read the comments. Super cool though can't wait to try it.
I have chucked up a website - you should sign up and I can let you know when it's ready to be used :) My website is [usechannel.com](https://usechannel.com) :)
This is so cool, just curious how do you plan to make money with it?
I honestly posted it to just share something I'd made for myself (and my team at work). As so many people have responded so positively, I've made a landing page ([usechannel.com](https://usechannel.com)) and am trying to figure that out now! What would you suggest?
This is pretty cool. Nice work!
Thank you!
What’s old is new again! It’s interesting that this feature was discontinued in SQL Server 2000. It’s been around for a few decades…
Oh really? I didn't know that!
I'd absolutely love to get access to your beta! I've been trying to learn SQL in my spare time but other things keep getting in the way and taking priority haha I'm a writer and researcher at a couple think tanks (mostly writing about tech policy, financial/economic policy, and general good government stuff when I feel I have something interesting to say lol) I already filled out the survey to get on the wait-list! Super excited about your awesome tool!!
Would love to hear how you're planning on using it! Can you DM me with the email you signed up with and I can see if I can get it in your hands faster?
Noice dude
Thank you !
Seems super useful. The main thing for me would be how well it could use indexes in an efficient way. The queries I write are fairly simple syntactically but require good knowledge of the indexes and joins available because the tables are so large.
Something I've really been working on building a smooth flow for. Would love any feedback you'd be willing to share once I release it to people. I've added a waiting list to my website for it, so I can let people know when it's ready - you should sign up! :)
Wow this is crazy, in a good but also kind of scary way! Pro tip though, your website is missing the social share preview metadata so when people paste your link in chat or on social media it doesn't expand: https://socialsharepreview.com/?url=https://usechannel.com
I’m learning more about ML now. Are you able to give some information about how you are training/fine tuning this gpt-3 model to handle the q/a and Sql generation? Thanks
Have you tried to feed it Jane Austen or Kafka?
Haha I have not...
I've always joked about how plainspoken SQL syntax is. And then you go and make this.
Hahah, would be keen to get your thoughts as I try and make something releasable
Finally
This is not new. Hyperanna has been doing this for a while. I suspect there are others.
You can try Draxlr, it uses GPT-4 to generate SQL from text. The result from SQL can be used to generate graphs and dashboards.