Reading up on infrastructure
100 deep · digging since nov 26, 25
- So long, Spokes: GitHub rewrites storage to restore reliability, just in time for agentic hordes
GitHub is rewriting its Git storage architecture, replacing the Spokes system with Azure Blob Storage to achieve a 35× write performance boost for agent-driven traffic.
- The Ethiopian economic miracle - TheBorys’s Substack
Ethiopia has sustained over 8% annual real GDP growth since 2008 despite ongoing wars, driven by state-led infrastructure, hydropower, agricultural mechanization, and mining expansion.
- The history of the Hetzner Cloud network stack
The article explains how Hetzner Cloud’s network stack evolved from basic Linux bridges to Open vSwitch, enabling Private Networks and Cloud Firewalls for scalable, secure tenant isolation.
- Why Texas Is Making Data Centers Wait
Texas halted new data‑center permits amid a flood of speculative filings, grid‑reliability worries, and disputes over who should pay for costly transmission upgrades needed to support AI‑driven load.
- What the Data Center Boom Looks Like From the Sky
Hundreds of new data‑center projects are being launched nationwide to meet AI computing demand, and these facilities are rapidly expanding in size and scale.
- celld: self-hosted, distributed Durable Objects
celld is a self‑hosted, distributed implementation of Cloudflare Durable Objects that lets Workers apps run unchanged while storing state in a user‑owned bucket at dramatically lower cost.
- HBM: High-Bandwidth Mistake - Irrational Analysis
The piece claims HBM is a costly mistake that will be replaced by optically linked LPDDR memory pools, forecasting DRAM stocks to double or triple before a eventual 70% drawdown.
- Inside Muse’s browser stack
Muse’s browser uses near‑stock Chromium driven by an 18 MB Rust broker that provisions warm VM pools, residential IP egress, and accessibility‑tree clicks for reliable automated browsing.
- How the Internet got its addresses (and why they weren't enough)
Vint Cerf chose a 32‑bit IPv4 address space in 1977, assuming 4.3 billion addresses would suffice for an experiment, but the protocol’s longevity exhausted the supply decades later.
- Elon Musk's SpaceXAI to add another 660,000 AI GPUs this year, nearing a total of 1.44 million in operation — firm is building 1.2-gigawatt power plant to bring systems fully online
Elon Musk said SpaceXAI will add 660,000 AI GPUs this year, reaching about 1.44 million total, while building a 1.2‑gigawatt power plant to run them.
- Agent (Muse) Compute Demand - by FD - Robonomics
Estimating Meta’s Muse agent for 100 M DAU, the analysis finds ~0.1 GW CPU/VM power, ~1‑2 GW inference power, requiring ~$800 M CPUs, ~$2 B DRAM, and up to 3‑4 GW total.
- S3 Is the Future, S3 Is the Past
S3's architecture is built for disk-era limits, but with cheap SSDs and fast networks, a new SSD-based disaggregated storage service could replace it — cloud providers lack incentive.
- Why strict memory overcommit matters for Postgres
ClickHouse Managed Postgres enables strict memory overcommit so that out‑of‑memory conditions fail a single query instead of triggering an instance‑wide OOM restart, with no measurable throughput loss.
- Saving another 100TB of RAM
This piece describes a technique or system change that freed an additional 100TB of RAM in data-center operations.
- Why AI Clouds Need a Tiered Storage Architecture
Neoclouds require a tiered storage architecture—flash for active training, cost-effective capacity for other stages—to avoid hyperscaler lock-in via data gravity.
- Short & Long Tail
Short & Long Tail presents a series of essays that examine everyday topics—cars, hurricane names, maps, water transport, and WWE moves—by tracing their common origins into obscure, fascinating details.
- There Is No Now
The article argues that in distributed systems, timestamps alone cannot capture causality, showing how approvals, messages, and actions can refer to outdated states despite correct delivery.
- GitHub - hieunc229/mailflare: Email for professionals and teams
Mailflare is a self-hosted email inbox for custom domains, built on Cloudflare, that handles sending, receiving, and routing with its own D1 database and R2 storage.
- Saturation at GitHub: the saga continues – Surfing Complexity
A GitHub cleanup job saturated a database primary because its safeguard monitored only replica lag, not primary load, causing site-wide errors.
- When Intelligence Becomes Abundant, Architecture Becomes the Constraint
When machine intelligence becomes abundant, the bottleneck shifts from AI capability to enterprise architecture, requiring systems designed for AI as a first‑class participant to unlock lasting value.
- Opinion | Please Stand Clear of the Accidental Art Museum
The article argues that New York subway’s constant decay and patchwork repairs unintentionally create an ever‑changing underground art exhibit dubbed the MTA Modern.
- Protocol-Aware Deterministic Simulation Testing
TigerBeetle’s protocol-aware deterministic simulation testing validates safety and liveness invariants at each replica, enabling deep verification of consensus and storage beyond black‑box methods.
- Backups Aren't Simple
The article argues that creating reliable backups is deceptively complex, requiring careful planning, testing, and ongoing maintenance beyond simple copy‑paste solutions.
- Connection reuse is the money
By reusing TLS connections with Undici keep‑alive, the webhook sender eliminated most handshakes, cutting latency and saving over $2,000 monthly.
- statichost.eu - 100% European static site hosting
Statichost.eu offers fully European-owned static site hosting, from git deploy to CDN, targeting developers who want infrastructure located and owned in Europe.
- Just bury your trash - Works in Progress Magazine
Landfills, when properly managed, are cleaner, cheaper, and more environmentally sensible than recycling most everyday materials, except metals, according to lifecycle analyses.
- How we rebuilt Cloudflare Workers’ module registry for Node.js compatibility
Cloudflare rebuilt the Workers module registry to use URL-based resolution, lazy compilation, shared caches, and Node.js-compatible module handling.
- T.S.A. May Ease Liquid Rules for Passengers With PreCheck
The TSA is considering allowing passengers with TSA PreCheck to bring bottled water through security, ending a two‑decade‑old liquid ban.
- A Response to Bill Gates's Essay - by X.PIN and CT Zhao
The essay contends that the real AI risk lies in industry shifting costs to the public, urging accountability for today’s AI companies rather than focusing on hypothetical superintelligence.
- New York’s Subways Weren’t Built for This
New York’s subway system, with 47,000 openings, is increasingly prone to flooding as heavy rains overwhelm its aging infrastructure across the city.
- There Is Still a Void at Ground Zero. Here’s How We Fill It.
After 25 years, Ground Zero remains underdeveloped despite the World Trade Center's success, and we should transform it into a vibrant neighborhood.
- ovlive — live kaart van het openbaar vervoer in België
ovlive provides a real-time map showing all buses, trams, metros and trains across Belgium, operated by De Lijn, STIB-MIVB, TEC and NMBS/SNCB, with regional access links.
- Everyone is right about AI
AI datacentre expansion will consume massive resources, trigger societal collapse, and render money obsolete as shared values shift beyond capitalist constructs.
- AI Waste – Ubergeek Kelly's World- Life, technology, science, rants
The author argues that massive AI investment yields minimal profit while consuming excessive power, water, and hardware, calling the boom a toxic, self‑reinforcing aberration.
- Launching Route 53 Files
Route 53 Files exposes AWS Route 53 hosted zones as an NFS file system, allowing standard Unix tools to edit DNS records with near real‑time synchronization.
- tailcat
Tailscale lets you securely connect any internet resource using WireGuard‑based, zero‑trust, end‑to‑end encrypted tunnels that are fully configurable across any infrastructure.
- Compute that takes any shape - Vercel
Vercel's Fluid compute architecture dynamically assembles tailored machines for builds, sandboxes, and functions, enabling instant provisioning and efficient resource use.
- Grand Canyon flooding: 1 dead, 15 people may be missing and dozens are evacuated
One person died, about 15 may be missing and dozens evacuated after a flash flood destroyed bridges and knocked out the Grand Canyon's South Rim water pipeline.
- Accidental Aesthetics and Romance of Power Wires
The article traces how power and telegraph wires have been depicted in Japanese and global art, showing their evolution from symbols of progress to romantic, aesthetic subjects.
- A Network That Can Wait
Store and Forward enables networks to treat waiting as an operation, allowing communication across time-varying links by storing data until future connectivity appears, extending connectivity beyond constant paths.
- Starbase, LA | Hacker News
SpaceX's Starbase expansion in Los Angeles aims to boost local aerospace industry but faces regulatory and community hurdles.
- Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache
Cloudflare cut over 100 terabytes of RAM used by its 1.1.1.1 DNS cache by redesigning cache layout and eviction policies without hurting query latency.
- GitHub - tobi/walgit
walgit is a git server implemented as a single binary that runs in front of an object store (S3/GCS), requiring no local state beyond a cache.
- The American People Really Hate Data Centers
American opposition to data centers is driven less by their physical impacts and more by distrust of AI, big tech, and elites, fueled by bandwagon effects and rising AI visibility.
- Vedansh — backend & AI engineer
Vedansh, a backend and AI engineer, builds storage engines, network stacks, and real-time media systems from scratch, grounding his work in academic papers and AI/ML research.
- Data centers become "killer application" for new power transformer tech - Ars Technica
Solid-state transformers, spurred by AI data center demand, offer mass-manufacturable, efficient power conversion that could also benefit EV charging and household grids.
- The Teaser Period: Why the AI Boom Is Hitting a Reset Wall
The AI boom mirrors 2008's housing crisis as take-or-pay compute contracts create a 2027-2028 'reset wall' where payments start despite labs' revenue being insufficient to cover obligations.
- How Kubernetes Probes Work
The article explains how Kubernetes uses liveness, readiness, and startup probes to monitor container health and manage pod lifecycle effectively.
- GitHub Outages Show the Limits of Reactive Scaling — Rahmi Pruitt
GitHub’s eight‑hour outage shows that reactive scaling fails when hidden concurrency limits are hit and client retries amplify demand, proving that agents need proactive traffic pacing.
- A County Got Rich From Data Centers. Some Question ‘At What Cost?’
Loudoun County, Virginia, earns substantial tax revenue from over 250 data centers, but officials and residents warn that reliance on this income may be risky.
- Git at any scale
Cursor argues Git’s packfile‑based design hinders scalable hosting and presents Continuity, a S3‑backed write‑ahead log system that enables linear, consistent replication.
- The 6-Stage AI Infrastructure Journey: Navigating the Three FinOps & Hardware Crises with ACE Gateway — ACE Blog
The piece outlines a six‑stage AI infrastructure journey, maps three predictable cost/latency crises, and demonstrates how ACE Gateway’s control‑plane features cut spend and prevent hardware failures at each stage.
- Git at any scale
Cursor explains why Git's packfile design hinders scalable hosting, reviews past solutions like Spokes, and introduces Continuity, a S3‑based write‑ahead log system that provides linearizable, elastically scalable Git storage.
- Miami’s Skyline Is Getting Super Tall
At least 10 new skyscrapers, including five supertalls, are planned for Miami over the next five years, raising concerns about whether such rapid vertical growth is sustainable or wise.
- Power 2026 by Neel Somani - Electricity Pricing in the Age of AI
The primer explains how data center power demand from AI is straining the US grid, details power plant economics and marginal pricing, and outlines the development process using Homer City as a case study.
- Airport Lines Are Grueling. New Programs Aim to Change That.
U.S. airlines and airports are testing new passenger and baggage screening programs designed to significantly cut airport security wait times.
- Is Kentucky About to Give Billions in Tax Breaks to Data Centers? - Kentucky Center for Economic Policy
Kentucky’s data‑center sales‑tax exemption could cost the state over $1 billion in forgone revenue if awarded to just a few large, AI‑optimized facilities.
- Choose DuckDB rather than SQLite
The article argues DuckDB outperforms SQLite for analytical queries due to its columnar storage and vectorized execution, recommending it for data analysis tasks over SQLite's transactional strengths.
- Elevators | Hacker News
The article explores the often-overlooked complexity of elevator systems, from scheduling algorithms to user experience.
- Five Takeaways From the Times Investigation Into Larry Ellison’s A.I. Gamble
The Times investigation reveals Oracle incurred massive debt to build global AI data centers, with repercussions extending beyond the company itself.
- How we set up our cloud agent environment
Cursor rebuilt its monorepo to support cloud agents by matching local dev, simplifying the interface, and adding a self‑healing environment, boosting agent‑authored PRs from 10% to over half.
- A Deluge of A.I. Computing Power Is About to Come Online, Fueling Major Leaps - The New York Times
AI chip count will double roughly every nine months, reaching about 200 million H100‑equivalent units by 2028 as massive data‑center construction ramps up worldwide.
- Networking and the Internet, from First Principles
The article explains the foundational concepts of networking and the Internet, breaking down protocols, layers, and principles from first principles to build intuitive understanding.
- Claude Is Not a Compiler - exe.dev blog
Claude surpasses a compiler by enabling engineers to make cross‑stack decisions—from strategy to machine code—through iterative agent loops, turning vibe‑engineering into real engineering.
- Inside Roblox’s Bet on World Models - ByteByteGo Newsletter
Roblox is pairing its game engine with a video world model (Super Upsampler) to deliver photorealistic, scalable multiplayer games while solving latency, consistency, and creator‑control challenges.
- Introducing EmDash — the spiritual successor to WordPress that solves plugin security
Cloudflare launches EmDash, a serverless TypeScript CMS built on Astro that sandboxed plugins via Workers isolates to solve WordPress security issues while offering MIT licensing and x402 payments.
- Job queues are deceptively tricky
Designing reliable job queues requires careful handling of ordering, duplicate processing, failure retries, backpressure, and visibility, revealing many hidden pitfalls that developers often underestimate.
- Introducing Meerkat: an experiment in global consensus
Cloudflare Research introduces Meerkat, a global consensus service using the QuePaxa algorithm to provide strong consistency and high availability for control‑plane data across its 330+ data centers.
- Current AI – Open Source AI Gap Map
Mozilla's Current AI project maps the open source AI stack, evaluating 24,626 projects to identify gaps and seeking collaborators to close them.
- ‘It’s whack-a-mole’: how Europe’s smart border melted down
Europe's new smart border system (EES) failed at launch due to poor planning, technical glitches, and inadequate testing, causing major delays.
- OpenAI unveils its first custom chip, built by Broadcom
OpenAI unveiled its first custom inference chip, Jalapeño, designed with Broadcom and assisted by OpenAI's own models, claiming better performance-per-watt.
- Postgres transactions are a distributed systems superpower
Co-locating workflow state and application data in Postgres enables atomic transactions that provide exactly-once execution without application-level idempotency logic.
- Run any Dockerfile on Vercel - Vercel
Vercel now supports deploying any HTTP server from a Dockerfile on Fluid compute with autoscaling and CPU-based pricing.
- Why the West stopped making land - Works in Progress Magazine
1970s environmental regulations, not geography or transportation, halted U.S. urban land reclamation despite high land values and housing needs.
- Where Are the Most Data Centers Planned?
Rural areas in the southern United States are expected to host the most planned data centers, despite their noise and power demands.
- SpaceX's newest Starmind will make earth data centers obsolete
SpaceX plans to deploy Starmind, a constellation of one million AI compute satellites in orbit, aiming to make terrestrial data centers obsolete.
- Self-hosted dev sandboxes with preview URLs (Docker, Go, no K8s)
sandboxd is an open-source backend that creates isolated Linux containers with AI coding agents and live preview URLs, all self-hosted on one machine using Docker and Go.
- Buildings May Soon Have ‘Immune Systems’ That Fight Airborne Disease
The U.S. government is investing $150 million in technologies that give buildings immune-like systems to fight airborne diseases, following the pandemic.
- How SpaceX Humiliated Wall Street
SpaceX's Starlink generated more revenue in 2023 than all other commercial space companies combined, demonstrating the financial power of vertical integration and rapid iteration over traditional aerospace contracting.
- What job interviews taught me about Kubernetes
Kubernetes is overly complex for small teams, requiring numerous add-ons and frequent upgrades that outweigh its benefits.
- The Cloud Has Sound: The Unrelenting and Unseen Cost of A.I. Data Centers
Residents near AI data centers report health problems from constant low-frequency vibrations, highlighting an overlooked cost of infrastructure buildout.
Takes
Server monitoring
@sinequanonh
new serverless horrors story 😱 https://serverlesshorrors.com/all/cloudflare-108k/
@heyandras
We made an interactive explainer of the AI compute stack: from grid power and data centers to chips, racks, clusters, and workloads, plus the public companies at each layer. Explore the infrastructure behind what @JensenHuang calls the five layer cake: https://intelligence.mts.now/compute
@MTSlive
It's time to go bare metal. On Hetzner, similar budget: ~$80/mo VPS → 8 shared vCPU, 16 GB shared RAM, 320 GB SSD ~$68/mo Dedicated → 6c/12t, 64 GB RAM, 1 TB NVMe Yes, I have to set everything up by myself, but that's only once. I'd rather have 4x the RAM for less money.
@DeepakNesss
✨ Okay with lots of help from @javilopen and his Spanish scraping friends I've managed to vibe code my own @Scrapingbee alternative and replace my $249/mo Scrapingbee plan with my own $1/mo scraper running on my VPS! First I tried OpenSERP, an open source project that seems freshly maintained but it didn't work on my VPS IP or laptop IP and got blocked fast Then I tried to use residential IP proxies from IPRoyal but they were already quite tainted IPs meaning Google already knew they were scrapers and just blocked about 60% of them already, not so great, also didn't work! Then I discovered OpenSERP uses a vanilla headless Chrome, Google instantly detects that and blocks that, so literally the entire repo is useless cause it won't even work? I don't get why they don't say that in the docs? OpenSERP is even worse than just basic CURL, as my AI said: "plain curl got 7% blocked but OpenSERP's headless browser got 100% blocked" So then I just built my own with Playwright and a real Google Chrome on my VPS, headful and persistent context, and slow pacing like every 30 seconds or so And I got some better residential IP recommendations (which I can't tell you because thenhey t stop working!) Now I'm at about 90% success rate, which is similar or better than Scrapingbee And it costs me ~$1/mo (measured by GBs, obviously for text scraping I barely use a lot) instead of the $249/mo I paid Scrapingbee! My only remaining SaaS now are: - Cloudflare Domains - Cloudflare Email Sending - Cloudflare R2 Storage - Backblaze B2 for backups - Hetzner for all my servers for all my sites (VPS) - xAI for all LLMs for all my sites (so many things like moderation, writing emails, anything)
@levelsio
A video explaining a handful of the tricks we use to optimize for memory within the Superlogical server: in-memory compression, moving file descriptors between event-driven and blocked threads dynamically, full terminal state binary snapshot/restore in microseconds, and more!
@mitchellh
@waynesutton @Tailscale Tailscale is a lot of the magic behind https://agentcookie.dev
@mvanhorn
Calling it now, @Tailscale will be one of the most important parts of the agentic tech stack. Once a non-tech workflow is embedded into existing applications, it's a wrap.
@waynesutton
the models have no moat (OpenAI, Anthropic, XAI) the IDEs have no moat (Cursor, Windsurf) the harnesses have no moat (Cognition, Factory, LangChain) the app builders have no moat (Replit, Lovable, Bolt) the wrappers have no moat (Harvey, Abridge, OpenEvidence) the inference providers have no moat (Together, Fireworks, Groq) the voice layer has no moat (Sierra, Decagon, ElevenLabs) the data labeling companies have no moat (Scale, Surge, Mercor) the AI infrastructure has no moat (Baseten, Modal, Railway) the neoclouds have no moat (CoreWeave, Lambda, Crusoe) the generative media companies have no moat (Runway, Higgsfield, Suno) apparently nobody in AI has a moat except the venture firm ☠️
@nikunj
We bought 19 PB of flash storage for our AWS S3 exit in 2025 for about $1.5m. The list price for this setup today would be $19 MILLION 🤑. Don't know if I've ever timed a trade this well in my life. https://world.hey.com/dhh/it-s-five-grand-a-day-to-miss-our-s3-exit-b8293563
@dhh
we moved a lot of our data pipeline to cloudflare and i made a quick video showing it off yes this video is NOT ABOUT AI incredible
@thdxr
if I had to start a new startup from scratch, I'd pick a very boring infra problem. boring problems are owned by lazy giants. and lazy giants have legacy SDKs and confusing docs. in an agent's eyes, that's the whole product.
@zenorocha
for those of you that have very serious applications deployed on cloudflare, do you really manage it all with a wrangler config?
@thdxr
What it actually takes to build agent infrastructure yourself
@harsehaj
i am evaluating all sandbox providers, our requirements are high - fast disk, not networked - fast CPU, should be virtualizing on bare metal - can pause/resume
@thdxr
Dumb question: I’ve been running most of my cron jobs locally on my Mac Mini because it’s already authenticated with Google Workspace and the other apps I use. But should I run these jobs in the cloud instead with these apps OAuth’d to my Claude or ChatGPT account? How should I think about what jobs should stay local vs. move to the cloud?
@petergyang
☁️ I made my own little Cloudflare called Pietflare, it's a DDOS and probe detector with AI and with a central IP / ASN / country block list Each server (VPS) sends suspicious probes, or DDOS attempts etc, from the access logs to the central admin and each server pulls a central blocklist every minute and blocks it in Nginx It has a central dashboard where I can see any threats and then instantly block them but preferably the AI blocks it by itself
@levelsio
A few months ago my kids started vibecoding little web games with Cursor and wanted their friends to play them. GitHub Pages was fine until the games needed real backends, so I hacked together a setup where each game was a folder in one repo that deployed to a Hetzner box on every push. That held up until we shipped FULL SEND for Vibe Jam 2026 and it took off with 38,000+ players. The duct tape needed to become something real, so I rebuilt it properly and pulled it out into its own project. It turns one Linux server into a push-to-deploy host for many apps. The whole thing is a single Go binary that installs and drives Docker, Kamal, Cloudflare, Tailscale, and GitHub for you. After that: - Each app is a GitHub repo. - A git push is live in <5 seconds. - Deploys are zero-downtime. - Each app runs in its own container. - Automatic Cloudflare DNS and TLS tunnels. - SQLite-aware backup and restore. It's deliberately single server using convention over configuration, so for a typical app there's no YAML or Dockerfile to write. The idea is that one decent VPS can reliably run all your projects without per-app bills or piles of infra config. It's built on top of Kamal, so it's basically a Kamal wrapper for the "lots of apps on one server" case, with the Cloudflare, Tailscale, DNS, and backup glue wired up by convention. Setup is one interactive command on a fresh Linux box, which walks you through connecting everything. If you also have a bunch of projects you want to run on a single server, tell your Claude Code, Codex, Cursor, or favorite AI agent to grab a VPS and try it for you. It's fully open source and you can customize it to your liking:
@dvassallo
We're launching code storage and git hosting. Origin gives teams and agents a place to host, review, and collaborate on code. Available this fall. Join the waitlist. https://cursor.com/origin-waitlist
@cursor_ai