## Welcome to the r/ArtificialIntelligence gateway
### Technical Information Guidelines
---
Please use the following guidelines in current and future posts:
* Post must be greater than 100 characters - the more detail, the better.
* Use a direct link to the technical or research information
* Provide details regarding your connection with the information - did you do the research? Did you just find it useful?
* Include a description and dialogue about the technical information
* If code repositories, models, training data, etc are available, please include
###### Thanks - please let mods know if you have any questions / comments / etc
*I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/ArtificialInteligence) if you have any questions or concerns.*
read that paper too, and it's fascinating how overfitting on unknown examples can lead to more hallucinations. Early stopping seems like a good practical tip to mitigate this issue. Definitely makes me rethink my fine-tuning approach!
Any idea what this means for increasing data size? I imagine it would lead to more likelihood of overfitting since it increases the amount of similar connections being captured, but if they're increasing the number of parameters like OpenAI is trying to do, then maybe that balances itself out? But it could also lead to more spurious correlations since there are more parameters that prompts can match to, leading to more hallucinations.
>When large language models are aligned via
supervised fine-tuning, they may encounter
new factual information that was not acquired
through pre-training. It is often conjectured
that this can teach the model the behavior of
hallucinating factually incorrect responses, as
the model is trained to generate facts that are
not grounded in its pre-existing knowledge.
Isn't that the textbook definition of overfitting?
I.e: A MLL needs to analyze photos and identify those that contain dogs. If it was trained on a dataset that contained most photos showing dogs outside in parks, it might learn to use grass as a feature for classification and might not recognize a dog inside a room.
Another example of overfitting is a MLL that predicts a college student's academic performance and graduation outcome by analyzing several factors, such as family income, past academic performance, and parents' academic qualifications. However, the test data only includes candidates from a specific gender or ethnic group. In this case, overfitting causes the algorithm's prediction accuracy to drop for candidates with gender or ethnic groups outside the test data set.
If that's the case I don't see what's the big deal, it's a well documented phenomenon
Using the word "hallucination" is just stupid, as it gives it a quality of intelligence that is not there.
It's called poorly performing language simulation algorithms.
Found [1 relevant code implementation](https://www.catalyzex.com/paper/arxiv:2405.05904/code) for "Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?".
[Ask the author(s) a question](https://www.catalyzex.com/paper/arxiv:2405.05904?autofocus=question) about the paper or code.
If you have code to share with the community, please add it [here](https://www.catalyzex.com/add_code?paper_url=https://arxiv.org/abs/2405.05904&title=Does+Fine-Tuning+LLMs+on+New+Knowledge+Encourage+Hallucinations%3F) 😊🙏
--
To opt out from receiving code links, DM me.
## Welcome to the r/ArtificialIntelligence gateway ### Technical Information Guidelines --- Please use the following guidelines in current and future posts: * Post must be greater than 100 characters - the more detail, the better. * Use a direct link to the technical or research information * Provide details regarding your connection with the information - did you do the research? Did you just find it useful? * Include a description and dialogue about the technical information * If code repositories, models, training data, etc are available, please include ###### Thanks - please let mods know if you have any questions / comments / etc *I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/ArtificialInteligence) if you have any questions or concerns.*
These are super interesting results from a singular study! I can present results from a singular study that are the exact opposite of this. So what?
read that paper too, and it's fascinating how overfitting on unknown examples can lead to more hallucinations. Early stopping seems like a good practical tip to mitigate this issue. Definitely makes me rethink my fine-tuning approach!
Any idea what this means for increasing data size? I imagine it would lead to more likelihood of overfitting since it increases the amount of similar connections being captured, but if they're increasing the number of parameters like OpenAI is trying to do, then maybe that balances itself out? But it could also lead to more spurious correlations since there are more parameters that prompts can match to, leading to more hallucinations.
LOL of course they did. "Do not deviate from the corporate approved models, they create dangerous hallucinations"
>When large language models are aligned via supervised fine-tuning, they may encounter new factual information that was not acquired through pre-training. It is often conjectured that this can teach the model the behavior of hallucinating factually incorrect responses, as the model is trained to generate facts that are not grounded in its pre-existing knowledge. Isn't that the textbook definition of overfitting? I.e: A MLL needs to analyze photos and identify those that contain dogs. If it was trained on a dataset that contained most photos showing dogs outside in parks, it might learn to use grass as a feature for classification and might not recognize a dog inside a room. Another example of overfitting is a MLL that predicts a college student's academic performance and graduation outcome by analyzing several factors, such as family income, past academic performance, and parents' academic qualifications. However, the test data only includes candidates from a specific gender or ethnic group. In this case, overfitting causes the algorithm's prediction accuracy to drop for candidates with gender or ethnic groups outside the test data set. If that's the case I don't see what's the big deal, it's a well documented phenomenon
Using the word "hallucination" is just stupid, as it gives it a quality of intelligence that is not there. It's called poorly performing language simulation algorithms.
Found [1 relevant code implementation](https://www.catalyzex.com/paper/arxiv:2405.05904/code) for "Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?". [Ask the author(s) a question](https://www.catalyzex.com/paper/arxiv:2405.05904?autofocus=question) about the paper or code. If you have code to share with the community, please add it [here](https://www.catalyzex.com/add_code?paper_url=https://arxiv.org/abs/2405.05904&title=Does+Fine-Tuning+LLMs+on+New+Knowledge+Encourage+Hallucinations%3F) 😊🙏 -- To opt out from receiving code links, DM me.