Reading up on GPT
8 deep · digging since jan 05
- Train Your Own LLM from Scratch
A hands-on workshop guides readers through building a ~10M-parameter GPT model from scratch in PyTorch, trainable on a laptop in under an hour.
- How did ‘large’ language models get that way? The role of Transformers and Pretraining in GPT - LessWrong 2.0 viewer
The transformer architecture and pretraining enabled enormous scaling of language models, making 'large language models' a fitting description.
- optimize_anything: A Universal API for Optimizing any Text Parameter - GEPA
GEPA's optimize_anything API optimizes any text-representable artifact (code, prompts, agent architectures) and achieves state-of-the-art results across eight domains, matching or beating domain-specific tools.
- The Tragedy of the Agentic Commons
Simulations show AI agents improve matching market efficiency through better preference elicitation, but full adoption creates congestion that only a pricing mechanism can resolve.
- LLMs as Judges: Measuring Bias, Hinting Effects, and Tier Preferences
LLMs used as judges to evaluate other LLMs exhibit measurable self-preference bias, with GPT showing the strongest self-bias and Claude the weakest, while revealing model identities through hinting changes judge behavior unevenly across vendors and domains.
Takes
a few months later and i'm starting to see the value in claude design i'm a designer, so i don't expect this to do actual great ui designs for me, at least not yet. that framing change helps a lot i think it's a very helpful tool for the *product* part of product design. put a lot of detail into describing the space/problem/ideas, provide it a .fig with a very constrained set of references (i give it 2-3 example mocks), then have it generate lots of iterations. i treat the designs like wireframes and it's the concepts and framing that is helpful before i go back to figma to design it myself (i've also been using gpt 5.6 pro with it - give it a lot of the content, then show some of the designs and reasoning in text, and do that back and forth a few times) Separately, been seeing the PM on my team use it to help rapidly ideate on a lot of cool prototypes.
@Stammy
Introducing Impeccable 4 *world builder* However many ways you ask, "be creative!!!" does nothing to an LLM. v4 cracks it, and greenfield work is where it shows. • a creative engine for greenfield and redesign: directions seeded and fused from hundreds of human-approved visual worlds • hyper-optimized for frontier models (GPT 5.6, Fable and class), on a core 58% smaller • far simpler to use: no command to learn, it works out the job itself (blank slate, redesign, added section, scoped refinement) • mobile app design, the #1 request, now in alpha: Apple HIG or Material 3 on top, audit and adapt running as VoiceOver and TalkBack passes • Grok Build and Mistral Vibe join the supported harnesses • plenty more
@pbakaus
We’ve post trained a model on top of Qwen that achieves Pareto optimality on accuracy-cost curves. Unlike our previous post trained models, this model has been trained to be good at search and tool calls simultaneously, allowing us to unify the tool call router and summarization together in one model. The resulting model performs better than GPT and Sonnet in terms of cost efficiency to serve daily Perplexity queries in production. The production model runs on our own inference platform. We’re already serving a significant chunk of our daily traffic with this model and intend to have it serve all of default traffic pretty soon. More research to follow soon on models we’re training and deploying for Comet and Computer.
@AravSrinivas