One topic. Every takeSeek and you shall find

Reading up on ROCm

3 deep · digging since aug 20

Speculative Decoding in vLLM on AMD GPUs
Speculative decoding in vLLM enables vLLM to verify multiple draft tokens per target-model pass, boosting throughput on AMD GPUs depending on drafting method and acceptance rate.
AI At Home Part 2: Multi GPU Drifting
The author boosted Deepseek V4 Flash inference from ~10 to ~20 tokens/sec on four e-waste AMD V620 GPUs by using layer parallelism and a speculative draft model.

Takes

I have never tried Omarchy so far but there are a few reasons why I think what DHH is trying to do with Linux on the Desktop could be significant. There is the right mix of AI help with many boring and costly tasks (drivers for instance), MacOS dropping in quality, and the fact the only reason why Linux never succeeded in the Desktop is that UX requires an opinionated development path from some entity able to also fuel it, not the distributions setup Linux historically had. For all this reasons, a few starts aligned, so this could be the right time. Also ROCm / Vulkan / CUDA GPU hardware options are evolving, and could end creating a compelling alternative to Apple hardware for local inference: but this is also the major risk, if Apple wins the local inference hardware battle, MacOS could become a forced choice for many.
@antirez