tokens too cheap to meter

kept by paperleaf sep 23

The price of running machine‑learning intelligence is falling by orders of magnitude yearly, making AI tokens as cheap as electricity and enabling pervasive local and hosted AI use.

Jev introduces a new shape of LLM—System One, aka Decision Models
The piece presents Jev, a TypeSafe AI decision‑model that turns text inputs into probabilistic scores for yes/no, choice, and rating tasks, explores its low cost, bias risks, and community hacks.
also on Typesafe.ai, Jev, #open-source, #developer-tools
Jev, Prolog, Pi, and the dream of probabilistic logic programming
The integration of Jev with DeepClause and its Pi extension enables scalable probabilistic logic programming for expert‑system applications like SOP2AGENT.
also on Typesafe.ai, Jev, #open-source, #developer-tools
Speculative Decoding in vLLM on AMD GPUs
Speculative decoding in vLLM enables vLLM to verify multiple draft tokens per target-model pass, boosting throughput on AMD GPUs depending on drafting method and acceptance rate.
also on vLLM, #open-source, #ai, #developer-tools
The 6-Stage AI Infrastructure Journey: Navigating the Three FinOps & Hardware Crises with ACE Gateway — ACE Blog
The piece outlines a six‑stage AI infrastructure journey, maps three predictable cost/latency crises, and demonstrates how ACE Gateway’s control‑plane features cut spend and prevent hardware failures at each stage.
also on vLLM, #ai, #developer-tools
What every programmer should know about memory, Part 1 [LWN.net]
Ulrich Drepper’s LWN series introduces programmers to modern memory hardware, explaining DRAM, caches, NUMA, and how hardware limits affect software performance.
also on Intel