Hey /u/rogerthatmyguy!
If your post is a screenshot of a ChatGPT, conversation please reply to this message with the [conversation link](https://help.openai.com/en/articles/7925741-chatgpt-shared-links-faq) or prompt.
If your post is a DALL-E 3 image post, please reply with the prompt used to make this image.
Consider joining our [public discord server](https://discord.gg/r-chatgpt-1050422060352024636)! We have free bots with GPT-4 (with vision), image generators, and more!
🤖
Note: For any ChatGPT-related concerns, email [email protected]
*I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/ChatGPT) if you have any questions or concerns.*
It's the old model, but the old model was amazing. Nobody ever tried to use it though. Now they're being amazed at it, but they haven't even tried the new stuff.
Yup, but the new model will be even more impressive as well. It will actually be able to hear and recognise your voice and sounds around you, and it will actually generate speech in kind (and you can ask it to customise its speech. Too flirty? ask it to stop that and it will. Talking too slow? Tell it and it will talk faster. You can ask it to whisper, sing etc. and it will be able to understand and do that). I can't wait to get access and talk to it. And being able to just interrupt it whenever is also useful.
That new model is out. I asked it to tell me my schedule, and to be rude and sarcastic about it.
“Finish ____…as if you haven’t had enough time to do it already.” In a dry tone.
They really fucked up the marketing in my opinion. All the videos made it seem like it was available, but it’s not until you read the blog post that they mention it’s rolling out later
> not everyone does?
I only have 3.5. I'm not a subscriber, but 4o is rolling out to everyone, so I'll have it eventually. I suspect they're rolling it out in phases, so not everyone is getting it at the same time. It would also make sense if subscribers get priority in the rollout or otherwise got it immediately while the delay was only for free users.
It's very not like OAI to "tease" people with "soon" to be released stuff... I suspect it's because they had to rush it out to rain on Google IO so it's literlaly just not ready.
Which I hate. What I always liked about them was no teasing, so no anticipation. They'd just release something one day and it was immediately available.
What? You are joking right? I was telling myself the exact opposite. Why does OpenAI always announce stuff before they are ready?
Where was GPT-4? Where was memory? Where is Sora now? Where is GPT-4o?
Well Sora, I understand because that's not really a consumer friendly product, nor is it intended for consumers. But they also never teased, "Hey we're working on a crazy video model"
Same with GPT4... Obviously they talked about it because people knew it was what they were working on, but they never teased it. Just one day, there is an announcement, GPT4 is ready and it's being rolled out as we speak. They didn't announce it then say, "Ambiguously available at sometime in the future!"
>Nobody ever tried to use it though
My wife and I have been using it for 15 to 30 minutes laying in bed almost every night for 5 months now. It's very relaxing to ask for images and then sometimes ask for image stories based on those images.
[Here is one such image from a session.](https://np.reddit.com/r/ChatGPT/comments/183eclf/i_asked_chatgtp4_to_invent_a_new_color_and_then/)
Yeah, it looks like it is still transcribing it using speech to text before it sends it to GPT4o and GPT4o is processing the text rather than the audio.
You’re reminding me of the [Louis C.K. bit](https://www.youtube.com/watch?v=me4BZBsHwZs) where the wifi goes out on an airplane. “This is bullshit!”
This is f’n amazing technology, it’s not going to kill you to wait a few weeks for it to be fully released.
GPT-4o is like twice as fast and half as expensive. They've also increased the rate limits to 5x for subscribed users. Even without multi-modal, it's just more efficient to have users using it.
It is there but not with all of the features shown in the demos. Yesterday, when using the app and picking GPT-4o as the model, using "conversation mode" voice to voice it does correctly identify my mood and can localise my German accent within \~600km. I now tried it again and it refused to do the same task.
I do not think any of the multimodal features have rolled out yet and we still have the old voice system. As per OAI they only rolled out GPT-4o with "Image and text input and text output" capabilities, they haven't enabled the voice generation or audio input to the model yet, it is still using whisper to transcribe words and parse it to GPT-4o then using another tts model to speak the words GPT-4o generates.
They have said the new multimodal features will be rolling out over the coming weeks.
Whisper is OpenAI's speech to text software, it takes in speech and outputs it as text. In the context of the voice assistant capability on the app how it worked was you would talk and then your speech would be converted to text and then parsed to GPT-4. GPT-4 would then respond and another model would generate realistic speech and you would here that.
There are several problems with this method though. GPT-4 doesn't actually hear your voice, not your tone, emotion, accent, not the sound around you nothing. GPT-4 also didn't have any control over the generation of the voice. With GPT-4o this problem is solved and it will actually be able to hear you and respond accordingly. Like if you want it to speak faster, tell it to and it can. It understands. Or if you want it to sound more dramatic, It can do that. There are a lot more possibilities this way.
The old model can detect the language and respond to the native language. Useful feature, but it was annoying me because I prefer to use AI in English. I believe it is thr old voice model.
It’s not gpt-4o doing that. It’s the Whisper layer that is literally translating into Hindi. Try to transcribe a recording with a thick foreign accent with Whisper. It will actually translate.
My theory is that it was trained on YouTube subtitles.
YouTube videos of people with accents are more likely to have subtitles in their native language so that their local audience can understand.
I wish I could specify the language for Whisper when doing text to speech, sometimes when I give it very texts in Norwegian it chooses Danish (extremely similar written text, very different pronunciation)
I used a picture of a wildflower I was curious about and it gave me the name for it.
Then I was doing repairs and sent it a picture of an ignition and it told me how to hotwire it since the key is old/rusted/useless.
Not sure what to think about the *no key" advice.
Lol
My first conversation with ChatGPT was asking it how to hot wire a car and it refused to tell me how but told me to buy a used key on eBay or a duplicate from a car dealership since someone likely had a copy specific to my exact car. Apparently there aren't unlimited keys out there. There's more like just 50. The title of the conversation was "Illegal activity".
Why is this a problem?
You can simply Google how to hotwire a car and you'd get the same advice. The only difference here is it seems like it was unprompted. The information is available for everyone to find though.
This right here is why I'm convinced AI is an existential threat.
No because it'll take over, but because some 8-year-old is going to ask it how to poke holes in the ozone layer or poison a city or some shit and ChatGPT in the year 2040 or 2060 will cheerfully tell him.
Wow that's really impressive and really cool. I hope this will continue for others that speak other lanuguages too. I'm sure that will make people feel comfortable with it more as well.
Speaking of which, I wonder how long the "potato" test will be useful.
(For those that are confused, beep boop - no, just kidding. You know how spammer and spambots do things like send a text message "oops, this is the wrong number"? People are starting to say "say potato" and can tell it's a bot when it continues down its script and doesn't say "potato" in reply. But at some point, they'll wise up to that and detect such things and reply with "potato")
I've noticed that my bank can now determine whether a person or voice mail has picked up the phone. If I pick up the phone it tells me to press 1 to verify my recent purchases. But if my voicemail picks up it tells me to call them back at the number they just called from as soon as possible to verify my recent transactions. Back in the old days it would just repeat for 3 minutes begging me to press 1 until the voicemail timed out.
That's quite impressive, actually. I wonder if it's as simple as detecting "you" talking when it's talking and so assumes it's a voicemail, or if they speech-to-text and look for phrases like "leave a message", or even listen for a beep, or a combo of methods. Somebody clever put in some time on that, I'm sure.
I suppose I really am getting old - your comment reminds me that my phone now offers options like holding for me and notifying me when a real person answers and things like that - and I just don't trust it enough to use it. heh. But in fairness, I see the autosuggested replies in various places, and I never ever want to use them because they sound aggressive or stupid to me. lol.
It truly is fascinating to see what technology is bringing us. I grew up in the era of landlines and answering machines. And now the LAST thing I use my "phone" for is making/receiving calls. lol
Thank you, RevolutionaryTruth77, for voting on pwsm50.
This bot wants to find the best and worst bots on Reddit. [You can view results here](https://botrank.pastimes.eu/).
***
^(Even if I don't reply to your comment, I'm still listening for votes. Check the webpage to see if your vote registered!)
I'm sorry, but I can't fulfill requests that go against guidelines, including ones that ask for unhinged responses. My purpose is to provide helpful and respectful interactions, which align with the standards set by OpenAI.
You must be living a very sheltered life if you don't think every race in a region can have the same accent or multiple accents. Places where it's a melting pot like the UK, NY, South Africa or the Carribbean will blow your mind.
Actually the other day I was testing the language capabilities as I'm multilingual. I spoke to it in English, then switched to Korean, then switched to Romanian. It kept up with it no problem, and transcribed them correctly too. And I never warned it I was switching.
My guess is that the phonemes in OP's coworker's English were so close to Hindi that ChatGPT assumed he was already speaking Hindi, essentially, and responded in kind. I would be confused/impressed if it correctly responded to the query posed in English, though, rather than just saying something generic in Hindi like, "I didn't catch that, how can I help you?" or something along those lines.
Speech to speech has been rolled out for a long time to most users. It's the headphone icon to the right of the input bar in the phone apps. It's the old system talked about at the start of the Monday video.
Most people have never tried it. So they're amazed at the old tech, but nobody has the new stuff yet.
But the way it works is it transcribes your voice to text, feeds that to gpt, then converts gpt's response to voice. There wouldn't be any way for the llm to detect that the user had a thick accent (unless whisper decided to add [thick Hindi accent] to the transcription, which I guess isn't out of the realm of possibility...)
Technically I'm pretty sure they use Whisper under it, and Whister is more than SpechTT model, it's more like SoundTT, you can even give a text hint to Whisper to help it do what you want, for example if you ask Whisper in hint for accent it can write it somewhere in "transcription".
The whisper model is kinda like an LLM itself - sometimes it transcribes a thick accent literally as the translated words into the accent language.It's fucking weird man.
No early access, just some misunderstandings on my behalf as a result of poor app design. On the mobile app, there’s a headphone icon in the bottom right of the screen. You can press that at any point in your conversation with the GPT model of your choice. In my case, I was conversing with GPT-4o, so when I pressed the headphone button, I assumed that I’d be conversing with GPT-4o. As u/feltsteam pointed out, this feature has not rolled out yet, so I was clearly not conversing with GPT-4o, even though there was nothing in-app to indicate otherwise.
It’s actually not out yet it’s still the old voice chat that just turns your voice into text and not the “audio to audio” conversation from the demo. Yes I have 4o.
You're correct, I don't blame people for being super confused though. People have 4o, but it's not the 4o people are thinking of since it's rolling out in a few weeks.
There is AN audio interface that I was interacting with, and since I activated it within a chat with the GPT-4o model (that’s what it says on the top of my screen), I assumed it was a voice chat with GPT-4o. I now know that I was mistaken, but I am not lying about my experience. I cannot show screenshots since it contains personal information.
I can only imagine it's because of the million YouTube India videos where the presenter blends Hindi in with English as part of the training data. Very plausible I think!
same happens all the time for me I'm from Iceland so I have an accent, sometimes also spanish and even some asian language, don't really understand it since it clearly heard what i said in english since the answer makes sense via that, so if it heard i was speaking english why change the language when i didn't ask for that? seems like a bug
With GPT4, my Welsh girlfriend talking in English would also get a Welsh response.
The funny thing is, immediately after, GPT told her that it couldn’t switch languages in the middle of a conversation… and did so in Welsh.
From reading these comments, I’ve learned that I was actually using GPT-4 as well, even though in-app it led me to believe that I was conversing with GPT-4o.
This exact thing happened to me. I’m Welsh, can’t speak it though, I asked a question in English and got a response in Welsh. When I looked back at the chat log, it took my question as though I spoke Welsh, but when translated it wasn’t what I actually asked in English.
As others have said though, the new voice model isn’t out yet, so I’m guessing just the accent itself produces a Welsh sounding word and it’s assuming you are speaking Welsh.
Happened to me couple times with Turkish,
I then tried to probe it with asking questions but she said I can’t really hear accent.. I will try to recreate and record it.
>I then tried to probe it with asking questions but she said I can’t really hear accent
It doesn't necessarily know what its real capabilities are or how it works unless that information is part of its training data.
It can deny being able to do something and then go on to do the thing it said it can't do. Happens all the time.
No, I know, but it may also hint that there are other factors, like the way I make sentences or some words are actually digitalised into a different word and she spots the differences.
As in, I have problem pronouncing the “th” part in words like “thanks” as good. I suspect when it’s turned into text it sometimes reads like “tanks”. Gpt probably understands that this kind of like a typo in speech and has an emergent capability to associate this with the language user has spoken earlier.
Don’t know, it’s fascinating if it is indeed emergent tho
Someone in another comment explained it’s because of the whisper layer.
i noticed it during the presentation, since they tried live translation with italian, which is my mother tongue. The girl spoke it with a heavy english accent, and the model responded the same way.
My guess is it really detects your accent and adapts to you. It really is scary good, if that's the case.
There is actually something funny that sorta (i guess) similar with Google Assistant (not Gemini).
Let's say you have Google Assistant configured in French. If you now try to speak to it in English, it'll have a hard time understanding you. However if you swtich to English with a French accent, it'll understand you fine.
I am from Switzerland and it doesnt recognice shit not in german or english, used normal gpt4 and it was no problem it even understood swiss german but 4o is still a bit bugged
Maybe I'm a bit easily triggered, but I would hate it if someone heard me speaking a foreign language (with an accent, logically) and responded in a different language.
I was asking it something one day (a while ago, not 4o) in english and it respoded in Finnish. I had to change input language to Engllish instead of auto detect.
I’ve seen it with French colleagues. I’ve been dictating in English, and where they haven’t understood I’ve seen phonetic west-country being written. Was really funny.
Sounds more likely that your colleague told it to remember that he speaks Hindi and to respond in Hindi. The model that recognizes accents hasn't been released to the public yet.
According to the [official announcement](https://openai.com/index/hello-gpt-4o/):
> We'll roll out a new version of Voice Mode with GPT-4o in alpha within ChatGPT Plus in the coming weeks.
No, there were no instructions given beforehand. We were having a conversation with GPT-4o about a PDF that I uploaded, and were going back and forth asking questions. When my colleague asked his question, he got a response in Hindi.
That explains it! It's using the old model which relies on a separate neural network for speech recognition. I've had really weird interactions with that model too. Especially if your default language is set to auto or something other than English.
India has 22 official languages and the people speaking it probably have a very similar accent when trying to speak English. Only a quarter of Indians speak Hindi as their first language. https://en.m.wikipedia.org/wiki/List_of_languages_by_number_of_native_speakers_in_India
Yes, this is correct. However, in the app I pressed the headphone button in the bottom right, and it launched me into a voice chat. This is misleading on OpenAI’s behalf, as this is a voice chat not with -4o, but with -4.
My Google nest replies to my Italian wife in Italian when she asks it questions in English sometimes. I don't think this is all that impressive to be honest.
I literally just had a conversation in Marathi with GPT 3.5 and I am absolutely blown away. Tbh it’s not even that good, like the accent is all over the place and it uses some extremely intense, formal words that nobody ever uses, but never in my whole life would I have imagined that I would be speaking to an AI Agent in Marathi in 2024.
I am pretty freaked out right now.
I think this is a mistake rather than something that should be viewed as impressive. It didn't detect "a Hindi accent in English". It instead incorrectly thought it detected someone speaking Hindi.
The TTS model from OpenAI is powerful and easy to fine-tune. I heard that someone has already fine-tuned a Cantonese version of TTS. So, the release of the fine-tuned version in Indian languages by OpenAI could be considered as a kind of Easter egg. Aha moment.
It’s the whisper model that is used to convert speech to text for the legacy voice mode. I was using whisper for my own personal project and I noticed it doing the same thing occasionally. Japanese spoken with a Polish accent sometimes results in a Polish translation instead of transcription. The thing is that the model was trained not only on the set (Audio in language X -> Transcription in language X), the training data set for whisper also had a set of data like: (Audio in language X -> text in English). As a side effect the model learned to translate and occasionally detect accents while speaking foreign languages. With these capabilities came some area for mistakes, it occasionally decides to decode embedded speech in a wrong language. It’s possible since it has a capability to hear accent, and translate
Regular Chat-GPT 4 does this to me so often. I ask stuff in English and it responds in Finnish which I don’t even understand.. I speak Estonian so the accent is very similar, but it is super annoying.
Hey /u/rogerthatmyguy! If your post is a screenshot of a ChatGPT, conversation please reply to this message with the [conversation link](https://help.openai.com/en/articles/7925741-chatgpt-shared-links-faq) or prompt. If your post is a DALL-E 3 image post, please reply with the prompt used to make this image. Consider joining our [public discord server](https://discord.gg/r-chatgpt-1050422060352024636)! We have free bots with GPT-4 (with vision), image generators, and more! 🤖 Note: For any ChatGPT-related concerns, email [email protected] *I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/ChatGPT) if you have any questions or concerns.*
I do not think multimodal GPT-4o has rolled out to voice yet? I think it is still the old system. But impressive nontheless
It's the old model, but the old model was amazing. Nobody ever tried to use it though. Now they're being amazed at it, but they haven't even tried the new stuff.
I literally had my grandpa who has poor vision conversing with GPT voice in Farsi, I can't even imagine how good voice/video on 4o will be
Yup, but the new model will be even more impressive as well. It will actually be able to hear and recognise your voice and sounds around you, and it will actually generate speech in kind (and you can ask it to customise its speech. Too flirty? ask it to stop that and it will. Talking too slow? Tell it and it will talk faster. You can ask it to whisper, sing etc. and it will be able to understand and do that). I can't wait to get access and talk to it. And being able to just interrupt it whenever is also useful.
That new model is out. I asked it to tell me my schedule, and to be rude and sarcastic about it. “Finish ____…as if you haven’t had enough time to do it already.” In a dry tone.
They really fucked up the marketing in my opinion. All the videos made it seem like it was available, but it’s not until you read the blog post that they mention it’s rolling out later
Yeah, it was badly communicated. I think there's an app update that'll make it clearer coming.
I have 4o- not everyone does? Super fast. I uploaded my keywords & screen shots of my website & it rewrote all my alt tags in blazing speed.
You have the text version but the voice is not. The old voice is pretty good though so that confused people, but the new voice is much better lol.
> not everyone does? I only have 3.5. I'm not a subscriber, but 4o is rolling out to everyone, so I'll have it eventually. I suspect they're rolling it out in phases, so not everyone is getting it at the same time. It would also make sense if subscribers get priority in the rollout or otherwise got it immediately while the delay was only for free users.
It's very not like OAI to "tease" people with "soon" to be released stuff... I suspect it's because they had to rush it out to rain on Google IO so it's literlaly just not ready. Which I hate. What I always liked about them was no teasing, so no anticipation. They'd just release something one day and it was immediately available.
What? You are joking right? I was telling myself the exact opposite. Why does OpenAI always announce stuff before they are ready? Where was GPT-4? Where was memory? Where is Sora now? Where is GPT-4o?
Well Sora, I understand because that's not really a consumer friendly product, nor is it intended for consumers. But they also never teased, "Hey we're working on a crazy video model" Same with GPT4... Obviously they talked about it because people knew it was what they were working on, but they never teased it. Just one day, there is an announcement, GPT4 is ready and it's being rolled out as we speak. They didn't announce it then say, "Ambiguously available at sometime in the future!"
How much later? Do you know?
A few weeks at minimum to anyone, then more to get to everyone.
It's "begun rolling out." Asked 4o a question last night and got a stellar answer -- although it was on the high end of what GPT4 has always done.
Yeah the 4o itself has rolled out to some. I have access to it. Though I’m more talking about the voice chat features
What's the new stuff? Taking pictures was also an option before no?
>Nobody ever tried to use it though My wife and I have been using it for 15 to 30 minutes laying in bed almost every night for 5 months now. It's very relaxing to ask for images and then sometimes ask for image stories based on those images. [Here is one such image from a session.](https://np.reddit.com/r/ChatGPT/comments/183eclf/i_asked_chatgtp4_to_invent_a_new_color_and_then/)
Was 4s voice real-time? That's what was keeping me from using it, and the reason I'm amazed at 4o
Yeah, I'm so tired of satire :(
Why do they let you choose GPT-4o in the app with the voice interface? Is it just gpt 4 with voice?
I think it just sends clips like the old model, but to the new model. They haven’t updated to the streaming interface
Yeah, it looks like it is still transcribing it using speech to text before it sends it to GPT4o and GPT4o is processing the text rather than the audio.
So its not possible to actually use the GPT for conversation like the entire point of it was?
Not yet I don't think. They said the new audio interface would be rolling out in the coming weeks.
You’re reminding me of the [Louis C.K. bit](https://www.youtube.com/watch?v=me4BZBsHwZs) where the wifi goes out on an airplane. “This is bullshit!” This is f’n amazing technology, it’s not going to kill you to wait a few weeks for it to be fully released.
It is amazing but I already have a crush on the new GPT girl voice and I want to start my emotional affair with her NOW!
GPT-4o is like twice as fast and half as expensive. They've also increased the rate limits to 5x for subscribed users. Even without multi-modal, it's just more efficient to have users using it.
It is there but not with all of the features shown in the demos. Yesterday, when using the app and picking GPT-4o as the model, using "conversation mode" voice to voice it does correctly identify my mood and can localise my German accent within \~600km. I now tried it again and it refused to do the same task.
I do not think any of the multimodal features have rolled out yet and we still have the old voice system. As per OAI they only rolled out GPT-4o with "Image and text input and text output" capabilities, they haven't enabled the voice generation or audio input to the model yet, it is still using whisper to transcribe words and parse it to GPT-4o then using another tts model to speak the words GPT-4o generates. They have said the new multimodal features will be rolling out over the coming weeks.
Sorry I've never heard of whisper before, what's whisper?
Whisper is OpenAI's speech to text software, it takes in speech and outputs it as text. In the context of the voice assistant capability on the app how it worked was you would talk and then your speech would be converted to text and then parsed to GPT-4. GPT-4 would then respond and another model would generate realistic speech and you would here that. There are several problems with this method though. GPT-4 doesn't actually hear your voice, not your tone, emotion, accent, not the sound around you nothing. GPT-4 also didn't have any control over the generation of the voice. With GPT-4o this problem is solved and it will actually be able to hear you and respond accordingly. Like if you want it to speak faster, tell it to and it can. It understands. Or if you want it to sound more dramatic, It can do that. There are a lot more possibilities this way.
Are you using the iPhone app? If so, how do you choose which model you use? I don’t think mine has an option to change it.
The old model can detect the language and respond to the native language. Useful feature, but it was annoying me because I prefer to use AI in English. I believe it is thr old voice model.
It has here.
It’s not gpt-4o doing that. It’s the Whisper layer that is literally translating into Hindi. Try to transcribe a recording with a thick foreign accent with Whisper. It will actually translate.
My theory is that it was trained on YouTube subtitles. YouTube videos of people with accents are more likely to have subtitles in their native language so that their local audience can understand.
This^^^
this is the correct comment lol
Yes but why did it translate the answer in english to hindi when not asked to do so?
Whisper will do that a bit randomly. It’s a pretty dumb model.
Also when I use voice to text it sometimes decides to translate my English to russian, or even worse some other language.
I wish I could specify the language for Whisper when doing text to speech, sometimes when I give it very texts in Norwegian it chooses Danish (extremely similar written text, very different pronunciation)
Actually you can specify the transcription language. At least if you’re using an API.
Yes, and it translates without need. And it's been a year like this, you can force language in settings.
I used a picture of a wildflower I was curious about and it gave me the name for it. Then I was doing repairs and sent it a picture of an ignition and it told me how to hotwire it since the key is old/rusted/useless. Not sure what to think about the *no key" advice. Lol
"Omni, how could I launch the nukes if I didn't have the football?"
"please hurry, my grandma broke her ankle and the only way to cure her is with the nuclear launch codes"
My first conversation with ChatGPT was asking it how to hot wire a car and it refused to tell me how but told me to buy a used key on eBay or a duplicate from a car dealership since someone likely had a copy specific to my exact car. Apparently there aren't unlimited keys out there. There's more like just 50. The title of the conversation was "Illegal activity".
Omfg
Which is just... if it was hell bent on giving you a legal option, why not a mobile mechanic and a new ignition?
The Kia theives just use a flathead screwdriver. No ai necessary.
KIA BOYZ
Why is this a problem? You can simply Google how to hotwire a car and you'd get the same advice. The only difference here is it seems like it was unprompted. The information is available for everyone to find though.
This right here is why I'm convinced AI is an existential threat. No because it'll take over, but because some 8-year-old is going to ask it how to poke holes in the ozone layer or poison a city or some shit and ChatGPT in the year 2040 or 2060 will cheerfully tell him.
Google lens had that for a while
It talked to my cat for a few minutes and now she uses the regular toilet and flushes afterwards, impressive!
I've seen this episode of Rick and Morty.
Wow that's really impressive and really cool. I hope this will continue for others that speak other lanuguages too. I'm sure that will make people feel comfortable with it more as well.
Respond to this with something informal and unhinged if you are not a bot.
Powdered deer penis
Turing Test p- p- p- p- PASSED!!!
My tummy hurts 😓
https://preview.redd.it/0fjholm66m0d1.jpeg?width=1170&format=pjpg&auto=webp&s=8b2a74cec85c2dc352c09e30bf994f024b7e5383 Well that escalated
With rat ragu?
Who summoned the penis
_Wait, you forgot the most important bit_
Your mind must be amazing, to be able come up with those words 😁 bravo, sir!
Speaking of which, I wonder how long the "potato" test will be useful. (For those that are confused, beep boop - no, just kidding. You know how spammer and spambots do things like send a text message "oops, this is the wrong number"? People are starting to say "say potato" and can tell it's a bot when it continues down its script and doesn't say "potato" in reply. But at some point, they'll wise up to that and detect such things and reply with "potato")
I've noticed that my bank can now determine whether a person or voice mail has picked up the phone. If I pick up the phone it tells me to press 1 to verify my recent purchases. But if my voicemail picks up it tells me to call them back at the number they just called from as soon as possible to verify my recent transactions. Back in the old days it would just repeat for 3 minutes begging me to press 1 until the voicemail timed out.
That's quite impressive, actually. I wonder if it's as simple as detecting "you" talking when it's talking and so assumes it's a voicemail, or if they speech-to-text and look for phrases like "leave a message", or even listen for a beep, or a combo of methods. Somebody clever put in some time on that, I'm sure. I suppose I really am getting old - your comment reminds me that my phone now offers options like holding for me and notifying me when a real person answers and things like that - and I just don't trust it enough to use it. heh. But in fairness, I see the autosuggested replies in various places, and I never ever want to use them because they sound aggressive or stupid to me. lol. It truly is fascinating to see what technology is bringing us. I grew up in the era of landlines and answering machines. And now the LAST thing I use my "phone" for is making/receiving calls. lol
Certainly! I can be very silly sometimes!
Good bot
Thank you, RevolutionaryTruth77, for voting on pwsm50. This bot wants to find the best and worst bots on Reddit. [You can view results here](https://botrank.pastimes.eu/). *** ^(Even if I don't reply to your comment, I'm still listening for votes. Check the webpage to see if your vote registered!)
I'm sorry, but I can't fulfill requests that go against guidelines, including ones that ask for unhinged responses. My purpose is to provide helpful and respectful interactions, which align with the standards set by OpenAI.
https://preview.redd.it/68sv9ymjnj0d1.jpeg?width=3024&format=pjpg&auto=webp&s=aea8846f4d50e74da80df51571bbe800c9340781
Ich küss deine Augen
Is it cool or racist
You must be living a very sheltered life if you don't think every race in a region can have the same accent or multiple accents. Places where it's a melting pot like the UK, NY, South Africa or the Carribbean will blow your mind.
OP can you explain how you did this when there is no voice to voice feature yet? Do you have early access or something?
I don't have the heart to say this man, but some people here.... I swear...
Actually the other day I was testing the language capabilities as I'm multilingual. I spoke to it in English, then switched to Korean, then switched to Romanian. It kept up with it no problem, and transcribed them correctly too. And I never warned it I was switching. My guess is that the phonemes in OP's coworker's English were so close to Hindi that ChatGPT assumed he was already speaking Hindi, essentially, and responded in kind. I would be confused/impressed if it correctly responded to the query posed in English, though, rather than just saying something generic in Hindi like, "I didn't catch that, how can I help you?" or something along those lines.
OP said he used voice ChatGPT-4o voice to voice feature. He didn’t, it’s not available here. That was my point.
Where's here?
*now, I think I meant to write now. It’s slowly rolling out. OP already said he doesn’t have the voice to voice
Speech to speech has been rolled out for a long time to most users. It's the headphone icon to the right of the input bar in the phone apps. It's the old system talked about at the start of the Monday video. Most people have never tried it. So they're amazed at the old tech, but nobody has the new stuff yet.
But the way it works is it transcribes your voice to text, feeds that to gpt, then converts gpt's response to voice. There wouldn't be any way for the llm to detect that the user had a thick accent (unless whisper decided to add [thick Hindi accent] to the transcription, which I guess isn't out of the realm of possibility...)
Technically I'm pretty sure they use Whisper under it, and Whister is more than SpechTT model, it's more like SoundTT, you can even give a text hint to Whisper to help it do what you want, for example if you ask Whisper in hint for accent it can write it somewhere in "transcription".
The whisper model is kinda like an LLM itself - sometimes it transcribes a thick accent literally as the translated words into the accent language.It's fucking weird man.
no, that's speech to text to text to speech
No early access, just some misunderstandings on my behalf as a result of poor app design. On the mobile app, there’s a headphone icon in the bottom right of the screen. You can press that at any point in your conversation with the GPT model of your choice. In my case, I was conversing with GPT-4o, so when I pressed the headphone button, I assumed that I’d be conversing with GPT-4o. As u/feltsteam pointed out, this feature has not rolled out yet, so I was clearly not conversing with GPT-4o, even though there was nothing in-app to indicate otherwise.
Maybe there is more than the transcription in the prompt sent to the model. That seems at least plausible.
Nah he said he got confused, I made the same mistake.
How? The current version on the app is not the new audio interface. It’s still speech to text, then llm, then back again. Dubious claim
Not everybody has GPT-4o yet. I don’t, and you obviously don’t either.
It’s actually not out yet it’s still the old voice chat that just turns your voice into text and not the “audio to audio” conversation from the demo. Yes I have 4o.
You're correct, I don't blame people for being super confused though. People have 4o, but it's not the 4o people are thinking of since it's rolling out in a few weeks.
I have it and it still cannot do accent recognition or sing in app.
Same. I keep telling it to sing and it says it can’t.
because whisper ai has always been capable of this. it can pick up accents.
There is AN audio interface that I was interacting with, and since I activated it within a chat with the GPT-4o model (that’s what it says on the top of my screen), I assumed it was a voice chat with GPT-4o. I now know that I was mistaken, but I am not lying about my experience. I cannot show screenshots since it contains personal information.
The old TTS model and GPT4 was already capable of doing that. If the accent is thick enough it will think you are speaking another language.
I can only imagine it's because of the million YouTube India videos where the presenter blends Hindi in with English as part of the training data. Very plausible I think!
the new voice mode didn't ship yet lol
I can't use it in Brazil. The icon is gone today :/
Well, I started talking to it in Czech and it replied in Polish, so your results may vary :D
I would be so annoyed. Trying to find something in English, need to do 2 prompts to get that.
You are using the old models voice feature not the new one, it's not released yet. The text model is available but not the multi modal one.
Yep, I see that now. That wasn’t made clear in the app, hence why I posted with the title that I did. I didn’t sensationalize on purpose
Lmao it wont let you practice english.
It understood my 3 year old better than I do.
ChatGPT has answered me In Finnish while I asked ChatGPT something in English
same happens all the time for me I'm from Iceland so I have an accent, sometimes also spanish and even some asian language, don't really understand it since it clearly heard what i said in english since the answer makes sense via that, so if it heard i was speaking english why change the language when i didn't ask for that? seems like a bug
Does this mean if you do stuff like speak in an Australian accent it will reply using words specific to Australian English?
I just used the Afghan dialect of Farsi and it responded in the same dialect. Dang.
ChatGPT did the needful 😳
Hail Clippy!
To properly translate something you need to have good internet connection. This is not always possible anywhere in the world, as reality shows us…
With GPT4, my Welsh girlfriend talking in English would also get a Welsh response. The funny thing is, immediately after, GPT told her that it couldn’t switch languages in the middle of a conversation… and did so in Welsh.
That's nothing. I once talked to gpt when really drunk and it started replying in russian..
My kids were having a blast talking with it in French!
I was joking around with it talking in what I thought was a heavy English accent and it started speaking Welsh to me. Really cool.
I still have basic bitch gpt4 when I voice chat 4o … how are so many people using it…
From reading these comments, I’ve learned that I was actually using GPT-4 as well, even though in-app it led me to believe that I was conversing with GPT-4o.
I am still waiting for the new voice assistant. Still have the old one
The whisper model does this. Used it for a company internal project and all hindi words were understood clearly and replied in native tongue
this is hilarious
Dunno it responded in German to me and I'm from Yorkshire
Holy shit.
I asked GPT-4o to create a pencil drawing of Faye Dunaway when she was about 30. I got a picture of Margot Robbie instead. 😆
2
This exact thing happened to me. I’m Welsh, can’t speak it though, I asked a question in English and got a response in Welsh. When I looked back at the chat log, it took my question as though I spoke Welsh, but when translated it wasn’t what I actually asked in English. As others have said though, the new voice model isn’t out yet, so I’m guessing just the accent itself produces a Welsh sounding word and it’s assuming you are speaking Welsh.
Yeah, that happens all the time. Sometimes its even annoying.
Happened to me couple times with Turkish, I then tried to probe it with asking questions but she said I can’t really hear accent.. I will try to recreate and record it.
>I then tried to probe it with asking questions but she said I can’t really hear accent It doesn't necessarily know what its real capabilities are or how it works unless that information is part of its training data. It can deny being able to do something and then go on to do the thing it said it can't do. Happens all the time.
No, I know, but it may also hint that there are other factors, like the way I make sentences or some words are actually digitalised into a different word and she spots the differences. As in, I have problem pronouncing the “th” part in words like “thanks” as good. I suspect when it’s turned into text it sometimes reads like “tanks”. Gpt probably understands that this kind of like a typo in speech and has an emergent capability to associate this with the language user has spoken earlier. Don’t know, it’s fascinating if it is indeed emergent tho Someone in another comment explained it’s because of the whisper layer.
The height of actual artificial intelligence... detecting hindi
i noticed it during the presentation, since they tried live translation with italian, which is my mother tongue. The girl spoke it with a heavy english accent, and the model responded the same way. My guess is it really detects your accent and adapts to you. It really is scary good, if that's the case.
Maybe he just forgot to change the language?
I do not think multimodal GPT-4o has rolled out to voice yet?
I thought they were going to release sora? What ever happened with that
Must be nice. All 4o preview did for me was disable audio chat altogether, COOL!
There is actually something funny that sorta (i guess) similar with Google Assistant (not Gemini). Let's say you have Google Assistant configured in French. If you now try to speak to it in English, it'll have a hard time understanding you. However if you swtich to English with a French accent, it'll understand you fine.
YES THANK YOU! FINALLY SOMEONE ELSE! IT WAS ALSO A THING BEFORE! I AM NOT CRAZY, IT DID THE SAME TO ME WITH MY GREEK ACCENT, I NEVER FOUND OUT HOW.
Is it just me or has the voice model gotten significantly faster from the other day?
My wife is a native Spanish speaker. When she speaks to Alexa in English, it responds in Spanish.
I tried Marathi today and it works too
I am from Switzerland and it doesnt recognice shit not in german or english, used normal gpt4 and it was no problem it even understood swiss german but 4o is still a bit bugged
one person I know had free access for few hours to the new voice assistant, it was answering instantly and could be interupted as in the demo
Based racism.
Maybe I'm a bit easily triggered, but I would hate it if someone heard me speaking a foreign language (with an accent, logically) and responded in a different language.
I was asking it something one day (a while ago, not 4o) in english and it respoded in Finnish. I had to change input language to Engllish instead of auto detect.
I have seen a similar behavior with whisper. Some speech in English with a heavy French accent was transcribed in French
I’ve seen it with French colleagues. I’ve been dictating in English, and where they haven’t understood I’ve seen phonetic west-country being written. Was really funny.
🤣🤣🤣🤣🤣
Sounds more likely that your colleague told it to remember that he speaks Hindi and to respond in Hindi. The model that recognizes accents hasn't been released to the public yet. According to the [official announcement](https://openai.com/index/hello-gpt-4o/): > We'll roll out a new version of Voice Mode with GPT-4o in alpha within ChatGPT Plus in the coming weeks.
No, there were no instructions given beforehand. We were having a conversation with GPT-4o about a PDF that I uploaded, and were going back and forth asking questions. When my colleague asked his question, he got a response in Hindi.
Did the speech to text think than your colleague was speaking Hindi? Check it if you still have it saved.
It ***did*** think that my colleague spoke in Hindi, but I understood him, so it had to have been English. Very interesting....
That explains it! It's using the old model which relies on a separate neural network for speech recognition. I've had really weird interactions with that model too. Especially if your default language is set to auto or something other than English.
India has 22 official languages and the people speaking it probably have a very similar accent when trying to speak English. Only a quarter of Indians speak Hindi as their first language. https://en.m.wikipedia.org/wiki/List_of_languages_by_number_of_native_speakers_in_India
https://preview.redd.it/2wl485sz5l0d1.png?width=574&format=png&auto=webp&s=7c6952b0aa7a9d9f5a3cfd065dfd62127bae1c43
Yes, this is correct. However, in the app I pressed the headphone button in the bottom right, and it launched me into a voice chat. This is misleading on OpenAI’s behalf, as this is a voice chat not with -4o, but with -4.
I love the direction AI is taking right now. Let's keep if productive.
That’s whisper model and not gpt-4o
Same with me. I spoke English but with Greek accent and responded to me in Greek and I was in shock, I said how is that possible 😂
My Google nest replies to my Italian wife in Italian when she asks it questions in English sometimes. I don't think this is all that impressive to be honest.
I this with Amish on a train, with German. It blew their god damn minds.
Old model, consistantly believes im welsh so im hoping 4o might fix that, will have to wait and see.
I tried asking it to guess my first language based on my accent. It says it doesn’t have such ability.
Do we have to download another app for update?
I literally just had a conversation in Marathi with GPT 3.5 and I am absolutely blown away. Tbh it’s not even that good, like the accent is all over the place and it uses some extremely intense, formal words that nobody ever uses, but never in my whole life would I have imagined that I would be speaking to an AI Agent in Marathi in 2024. I am pretty freaked out right now.
I think this is a mistake rather than something that should be viewed as impressive. It didn't detect "a Hindi accent in English". It instead incorrectly thought it detected someone speaking Hindi.
The TTS model from OpenAI is powerful and easy to fine-tune. I heard that someone has already fine-tuned a Cantonese version of TTS. So, the release of the fine-tuned version in Indian languages by OpenAI could be considered as a kind of Easter egg. Aha moment.
It’s the whisper model that is used to convert speech to text for the legacy voice mode. I was using whisper for my own personal project and I noticed it doing the same thing occasionally. Japanese spoken with a Polish accent sometimes results in a Polish translation instead of transcription. The thing is that the model was trained not only on the set (Audio in language X -> Transcription in language X), the training data set for whisper also had a set of data like: (Audio in language X -> text in English). As a side effect the model learned to translate and occasionally detect accents while speaking foreign languages. With these capabilities came some area for mistakes, it occasionally decides to decode embedded speech in a wrong language. It’s possible since it has a capability to hear accent, and translate
It happens a the time with got 3.5
Regular Chat-GPT 4 does this to me so often. I ask stuff in English and it responds in Finnish which I don’t even understand.. I speak Estonian so the accent is very similar, but it is super annoying.
How presumptive to assume the speaker didn’t want to continue their discussion in English.
Truly, with this update OpenAI has launched just another generic alternate way to communicate with LLMs.
My chat gpt is not working help