Please share what operating system and model runtime you use? I have two and don... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

SomeHacker44 15 days ago | parent | context | favorite | on: Speculative Decoding in vLLM on AMD GPUs

Please share what operating system and model runtime you use? I have two and don't get close to that with AMD's own Lemonade. Thanks!

lrvick 15 days ago [–]

Lemonade is one of the worst performing options. Run any modern Linux distro and ask your current LLM to setup llama.cpp with dflash2 for you as an unprivileged container running from a systemd user unit.

Obviously only on a system you do not trust at all.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact