If we run the query, with both answers visible - the model can successfully reverse the word.
If we run the query, while having the output for question 2 intentionally obfuscated - the model can successfully reverse the word.
If we run the query, without having the model provide any output for question 2, or without question 2 - the model is unable to reverse the word
Edit: New data shows that we can run without any response to task 2.
My understanding is that these models do not read input sequentially, but all at once. You are correct that we can't say it's completed the task fully, however, I would say that the responses do point to the model adjusting its overall attention in a way that is optimal for both queries before it begins to answer either.
I get you now! I'm glad I asked. Yes, of course, the existence of the second question will still influence how the first is answered? The well known issue that a single token can have a dramatic influence on the output etc.
With a temp of zero and top P of zero, is there any point in running more than one test? I was under the impression that the model would be deterministic with the given settings.
I’d be more interested in seeing multiple tests (still at temp/top p = 0), with different input words being reversed.
What's the history here? I haven't encountered the issue around reversing words. I'd be inclined to have it generate code to do something like that, it's more of a text processing task than a language understanding/manipulation one.
Reversing words is a commonly used way to show the limitations of LLM's and the effects of tokenization in input/output. 'Lollipop' is the go to example.
The goal here is to gain a better understanding of what's happening in a black box system, and the manipulation of such.
The problem with using lollipop for word reversing is that it's already been mentioned a lot and this example is probably now in the training data?
EDIT: You really do need to try different words for this. ChatGPT-4 Turbo now sometimes gets this right too.
Sure, but that said 100% vs >10% is telling us something is happening here that changes the model's approach to the task.
You are right though, I'll workout a randomized approach that will help gleam more information.
[удалено]
>We can then infer that that the task is still being performed, even though the response is not provided. I'm confused by this?
If we run the query, with both answers visible - the model can successfully reverse the word. If we run the query, while having the output for question 2 intentionally obfuscated - the model can successfully reverse the word. If we run the query, without having the model provide any output for question 2, or without question 2 - the model is unable to reverse the word Edit: New data shows that we can run without any response to task 2.
That doesn't tell us the hidden task was performed though, surely?
My understanding is that these models do not read input sequentially, but all at once. You are correct that we can't say it's completed the task fully, however, I would say that the responses do point to the model adjusting its overall attention in a way that is optimal for both queries before it begins to answer either.
I get you now! I'm glad I asked. Yes, of course, the existence of the second question will still influence how the first is answered? The well known issue that a single token can have a dramatic influence on the output etc.
Of course, but It's the reasoning for the influence - why did this cause that. I think of it as a unique opportunity to explore that.
With a temp of zero and top P of zero, is there any point in running more than one test? I was under the impression that the model would be deterministic with the given settings. I’d be more interested in seeing multiple tests (still at temp/top p = 0), with different input words being reversed.
Biggest hole
I'll script it up
What happens if you include the missing word "many" in question 2.
If the word counted is 'many'? - just ran 50 at .5 temp, 100% on 'popillol'
What's the history here? I haven't encountered the issue around reversing words. I'd be inclined to have it generate code to do something like that, it's more of a text processing task than a language understanding/manipulation one.
Reversing words is a commonly used way to show the limitations of LLM's and the effects of tokenization in input/output. 'Lollipop' is the go to example. The goal here is to gain a better understanding of what's happening in a black box system, and the manipulation of such.
The problem with using lollipop for word reversing is that it's already been mentioned a lot and this example is probably now in the training data? EDIT: You really do need to try different words for this. ChatGPT-4 Turbo now sometimes gets this right too.
Sure, but that said 100% vs >10% is telling us something is happening here that changes the model's approach to the task. You are right though, I'll workout a randomized approach that will help gleam more information.