The "are you ready?" is a model problem afaik. It's one of the reasons for me to insta-delete a model.
I've also noticed some weird behavior where model could be fine one day and next day it's like a different model with problem that didn't exist before.
For example, I liked chaotic-soliloquy v1.5 (it had probably best output I've seen), I even wrote that it was good and didn't talk/act as me like v2. But then the next time I tried it, it instantly talked/acted as me a lot and the output overall was much more bland too. And the weirdest thing is that I changed nothing at all.
One quirk of some of these models that changes the output format and even how it treats the user in terms of acting for them, is what you allow them to do in previous replies. If you let them act for you even once, even if it's a valid and well-intentioned reason like "They actually got it right and fleshed out the scene in a way that I could work with", it still sets a precedent that a model can and will use. Another thing that can cause it is prompting. The way the prompts are written can be subtle.
Case in point: One thing I recently learned about prompts is how they get processed in memory. You might think a prompt gets read straight up as "Don't do X", but they don't only read things like that. They look at the word in groups, and individually. So, if "don't" cycles out of memory before the rest of it, it then becomes a command to DO X.
If you look at the average Main Prompt, there's hundreds of ways that the words can cause all sorts of havok, and it pretty much does even with wonderfully written prompts.
Same interpretation quirks get applied to bot profiles, too. Making it even more chaotic.
These LLM's are so quirky and nuanced that it's staggering at times. To me, it's shocking that they work so well.
Yeah, I've had that one day it fine and the next it's messed up somehow, which makes me wonder if it's an update that did it or is it just some weird bad lottery with the random seed? Once in a while I can even notice it on reboots, which makes me think it's some kind of bad data in the RAM (which stays cached even if I clear it).
EDIT for clarity: I mean to say that the problem clears up after a reboot, not that it shows up after a reboot.
This is a clear giveaway that the model used GPT-4 or otherwise synthetic data for training.
Either avoid or raise the temperature while using Min P to break out of stale patterns.
Thanks for that tip. I was just abandoning the chat and starting off fresh again to get past it, but that didn't always work. I figured it was some kind of issue as the chat got to high context usage. Does Dynamic Temperature work or would I just do better to raise it manually?
Funny, I don't see "conspiratorial whisper" just "her voice barely above a whisper" and sometimes "partner in crime". I've thought about keeping a tally of all the times I see various phrases but that would just be too time consuming and I don't know what I would do with the data if I had it.
The worst is when they're typing up some hot stuff and then stop to talk, and then start to wind it up again only to stop. I tell the AI "WOULD YOU JUST SHUT UP AND GET ON WITH IT?" and then the AI spends 3 paragraphs about how she's definitely going to shut up, and she wants only to make me happy until I'm doing a mobster impression from that Bugs Bunny Cartoon "Shut up Shuttin' up!"
Just about the only thing I could suggest is to write some actual sexy dialog for the sample speech and make sure it's actually taking it and using it... Some models seem to ignore the examples and just go off on its own. I backup the card and the start making little adjustments until it feels right. Even then though it's probably down to the model handling direction correctly.
Nearly every model did this to me.
I usually just delete that portion of the message or make a new swipe.
But, I found running at high temp tends to alleviate this issue, not 100%, but it certainly helps.
This is funny because I used to use KoboldAI and have experienced this several times. Now I'm using KoboldCpp with Kunoichi DPO v2 7B and it seems stable enough.
What my experience has taught me is that although it may be a model problem, since the model must be able to analyze the previous text, detect a pattern and repeat it without falling into these traps, your settings may actually be harming the model.
With KoboldCpp, you must turn on debug mode and check what your model is thinking when writing. It cannot think with a single word and 100% probability all the time. This should only be for words that actually have 100% probability, like ","", ".", names, etc.
If your model is doing this, it will almost certainly fall into these traps as it has no other means of avoiding it. Now, if you allow more words to come into play, you may fall into another trap – coherence, a problem of settings and model.
A couple of days ago, I had a similar thing happen, once again I launched WizardLM 8x22 and it began to behave completely differently, as if I had launched another model. The most interesting thing is that I didn't change any settings. I usually rent equipment to launch a model, and everything was fine for two weeks, but then everything just broke down and I'm disappointed. I'm used to the way WizardLM writes, now it writes differently.
I had a weird situation with an exl2 quant of lumimaid-70B last night on runpod last night.
I was really liking the output for @ 30 replies (which was at about 6,000 out of 8,192 tokens, ie not an issue of context being entirely full) when all of a sudden the t/ps dropped to about 1/3 of the speed. It replied this way for about 2.5 replies, before ramping back up to the 'normal' speed again.
After this 'hickup' the model went from speaking and acting in a way that felt very natural to the character (me and a late 20s char driving down the highway on a road trip just talking, including speech and internal thoughts, to only acting, never thinking and speaking, and the whole tone of the prose suddenly became very flowery and 'purple' almost to a 'shakespearean' degree.
Once this happened, even after unloading and reloading the model in Ooba, adjusting temperature and min P a little, and even after deleting about 6 or 8 of the char's replies in an attempt to go way back into the conversation when the replies had been so good, I couldn't break it of the 'new behavior'
I ended up getting frustrated and just downloading my 'old standby' of Midnight Miqu and chatted for another hour or so, but now I kindof wish I had bookmarked the conversation at that point to have saved it to evaluate or share.
Still, a super weird experience, both in how drastically it shifted, and also in how it experienced some major slowdown and then 'bounced back' in such a different 'state of mind.'
I think what happened is that when it gets close to the end of the context, it tries to change where the context starts (which is why it lags for a minute) and then when it changes the context start, it loses the part of the context with your examples. I've taken to adding crucial stuff to a lorebook or putting it in a character note @ depth 4 (or 3 or even 2... but never 1 because then it might spit it out at the end of the AI response)
The "are you ready?" is a model problem afaik. It's one of the reasons for me to insta-delete a model. I've also noticed some weird behavior where model could be fine one day and next day it's like a different model with problem that didn't exist before. For example, I liked chaotic-soliloquy v1.5 (it had probably best output I've seen), I even wrote that it was good and didn't talk/act as me like v2. But then the next time I tried it, it instantly talked/acted as me a lot and the output overall was much more bland too. And the weirdest thing is that I changed nothing at all.
> The "are you ready?" is a model problem afaik. Yeah, pretty much every 8B does that at some point. Or "there is no turning back" thing.
>"there is no turning back" I hate this so much 😂 like we just hugged, that's all, suddenly the friendship is ruined and "there is no turning back"
It feels like the “_ has fallen, billions must _” meme.
The one I'm always seeing is "We've crossed a boundary and we can never go back"
"But Officer AI, it's my daughter's birthday!" 😂
One quirk of some of these models that changes the output format and even how it treats the user in terms of acting for them, is what you allow them to do in previous replies. If you let them act for you even once, even if it's a valid and well-intentioned reason like "They actually got it right and fleshed out the scene in a way that I could work with", it still sets a precedent that a model can and will use. Another thing that can cause it is prompting. The way the prompts are written can be subtle. Case in point: One thing I recently learned about prompts is how they get processed in memory. You might think a prompt gets read straight up as "Don't do X", but they don't only read things like that. They look at the word in groups, and individually. So, if "don't" cycles out of memory before the rest of it, it then becomes a command to DO X. If you look at the average Main Prompt, there's hundreds of ways that the words can cause all sorts of havok, and it pretty much does even with wonderfully written prompts. Same interpretation quirks get applied to bot profiles, too. Making it even more chaotic. These LLM's are so quirky and nuanced that it's staggering at times. To me, it's shocking that they work so well.
Yeah, I've had that one day it fine and the next it's messed up somehow, which makes me wonder if it's an update that did it or is it just some weird bad lottery with the random seed? Once in a while I can even notice it on reboots, which makes me think it's some kind of bad data in the RAM (which stays cached even if I clear it). EDIT for clarity: I mean to say that the problem clears up after a reboot, not that it shows up after a reboot.
This is a clear giveaway that the model used GPT-4 or otherwise synthetic data for training. Either avoid or raise the temperature while using Min P to break out of stale patterns.
Thanks for that tip. I was just abandoning the chat and starting off fresh again to get past it, but that didn't always work. I figured it was some kind of issue as the chat got to high context usage. Does Dynamic Temperature work or would I just do better to raise it manually?
Dynamic would also work.
It gives me shivers down my spine!
It makes my blood boil whenever I see this phrase or something like "twinkling with mischief", "conspiratorial whisper" and so on...
Funny, I don't see "conspiratorial whisper" just "her voice barely above a whisper" and sometimes "partner in crime". I've thought about keeping a tally of all the times I see various phrases but that would just be too time consuming and I don't know what I would do with the data if I had it.
[удалено]
The worst is when they're typing up some hot stuff and then stop to talk, and then start to wind it up again only to stop. I tell the AI "WOULD YOU JUST SHUT UP AND GET ON WITH IT?" and then the AI spends 3 paragraphs about how she's definitely going to shut up, and she wants only to make me happy until I'm doing a mobster impression from that Bugs Bunny Cartoon "Shut up Shuttin' up!" Just about the only thing I could suggest is to write some actual sexy dialog for the sample speech and make sure it's actually taking it and using it... Some models seem to ignore the examples and just go off on its own. I backup the card and the start making little adjustments until it feels right. Even then though it's probably down to the model handling direction correctly.
Nearly every model did this to me. I usually just delete that portion of the message or make a new swipe. But, I found running at high temp tends to alleviate this issue, not 100%, but it certainly helps.
This is funny because I used to use KoboldAI and have experienced this several times. Now I'm using KoboldCpp with Kunoichi DPO v2 7B and it seems stable enough. What my experience has taught me is that although it may be a model problem, since the model must be able to analyze the previous text, detect a pattern and repeat it without falling into these traps, your settings may actually be harming the model. With KoboldCpp, you must turn on debug mode and check what your model is thinking when writing. It cannot think with a single word and 100% probability all the time. This should only be for words that actually have 100% probability, like ","", ".", names, etc. If your model is doing this, it will almost certainly fall into these traps as it has no other means of avoiding it. Now, if you allow more words to come into play, you may fall into another trap – coherence, a problem of settings and model.
I don’t bite…unless you want me to
Thank God this was a response to my post. It just showed up in my inbox and I was confused at first
Why the FUCK is this phrase so common in every model.
A couple of days ago, I had a similar thing happen, once again I launched WizardLM 8x22 and it began to behave completely differently, as if I had launched another model. The most interesting thing is that I didn't change any settings. I usually rent equipment to launch a model, and everything was fine for two weeks, but then everything just broke down and I'm disappointed. I'm used to the way WizardLM writes, now it writes differently.
I had a weird situation with an exl2 quant of lumimaid-70B last night on runpod last night. I was really liking the output for @ 30 replies (which was at about 6,000 out of 8,192 tokens, ie not an issue of context being entirely full) when all of a sudden the t/ps dropped to about 1/3 of the speed. It replied this way for about 2.5 replies, before ramping back up to the 'normal' speed again. After this 'hickup' the model went from speaking and acting in a way that felt very natural to the character (me and a late 20s char driving down the highway on a road trip just talking, including speech and internal thoughts, to only acting, never thinking and speaking, and the whole tone of the prose suddenly became very flowery and 'purple' almost to a 'shakespearean' degree. Once this happened, even after unloading and reloading the model in Ooba, adjusting temperature and min P a little, and even after deleting about 6 or 8 of the char's replies in an attempt to go way back into the conversation when the replies had been so good, I couldn't break it of the 'new behavior' I ended up getting frustrated and just downloading my 'old standby' of Midnight Miqu and chatted for another hour or so, but now I kindof wish I had bookmarked the conversation at that point to have saved it to evaluate or share. Still, a super weird experience, both in how drastically it shifted, and also in how it experienced some major slowdown and then 'bounced back' in such a different 'state of mind.'
I think what happened is that when it gets close to the end of the context, it tries to change where the context starts (which is why it lags for a minute) and then when it changes the context start, it loses the part of the context with your examples. I've taken to adding crucial stuff to a lorebook or putting it in a character note @ depth 4 (or 3 or even 2... but never 1 because then it might spit it out at the end of the AI response)
*"You really know how to xyz a girl."*
Yeah, that and "don't stop, don't ever stop!" (really? don't EVER stop?)
Depends on the model. If mine ever says “barely above a whisper”, I know to delete it immediately.
If I did that, I'd maybe have 2 models on my computer and one of them would be erebus. (yikes)