Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Yeah, figure the 'nearly lossless' claim is the most controversial thing. But in my defense, ~97% recovery in benchmarks is what I consider 'nearly lossless'. When quantized with calibration data for a specialized domain, the difference in my internal benchmark is pretty much indistinguishable. But for agentic work, 4-bit quants can indeed fall a bit short in long-context usecase, especially if you quantize the attention layers.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: