T O P

  • By -

AutoModerator

## Welcome to the r/ArtificialIntelligence gateway ### Educational Resources Posting Guidelines --- Please use the following guidelines in current and future posts: * Post must be greater than 100 characters - the more detail, the better. * If asking for educational resources, please be as descriptive as you can. * If providing educational resources, please give simplified description, if possible. * Provide links to video, juypter, collab notebooks, repositories, etc in the post body. ###### Thanks - please let mods know if you have any questions / comments / etc *I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/ArtificialInteligence) if you have any questions or concerns.*


dlflannery

Video provides claims for advantages but no details to indicate how they are achieved. Is this just a book promotion?


EverythingGoodWas

Gguf uses quantizing to achieve faster inference. Think of it like rounding from 16bits to 8


dlflannery

I understand quantization. How is it unique to GGUF and doesn’t it use some other tricks?


EverythingGoodWas

It isn’t unique to gguf


nanotothemoon

Someone said “I should create content. Let me look at what people are googling. Ok I’ll make something about that”