AI At Home Part 2: Multi GPU Drifting

kept by elaramirez aug 20

The author boosted Deepseek V4 Flash inference from ~10 to ~20 tokens/sec on four e-waste AMD V620 GPUs by using layer parallelism and a speculative draft model.

I Tried to Make AI Writing Sound Human by Banning AI Words Through logit_bias - Vincent Schmalbach
The author applied logit_bias to penalize AI‑common tokens, observing a roughly one‑third reduction in those words while sacrificing one acceptable rewrite out of eight.
also on DeepSeek V4 Flash, #ai-coding, #developer-tools
The machine never raises its voice
LLMs consistently prefer their own machine-generated literary passages over human-authored classics, valuing precision and restraint over voice and strangeness when evaluating literary quality.
also on DeepSeek V4 Flash, #developer-tools
Qwen 3.6 27B is the sweet spot for local development - Quesma Blog
Qwen 3.6 27B with llama.cpp runs at usable speeds on Apple Silicon and Nvidia RTX hardware, making it the first local model practical for coding development.
also on Llama.cpp, #hardware, #open-source, #ai-coding, #developer-tools
Can I run AI locally?
The site canirun.ai estimates which local AI models a given machine can run based on hardware specs, but commenters found its RAM detection and speed predictions inaccurate for modern hardware.
also on Llama.cpp, #hardware, #open-source, #developer-tools