I think the biggest net new recent technique is P/D disaggregation. And that spe... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

philipkiely 24 days ago | parent | context | favorite | on: The efficient frontier of LLM inference

I think the biggest net new recent technique is P/D disaggregation. And that spec dec is very different now especially post DSpark/DFlash.

But overall yes the fundamentals of LLM performance optimization have been remarkably stable over the last few years.

brrrrrm 24 days ago [–]

perhaps its unfair to say this in hindsight, but it's a fairly straightforward application of little's law that's been around for some time

https://arxiv.org/html/2401.09670v2

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact