TLDR Data 2026-10-26
ChatGPT’s Scaled Storage 🗄️, Query Hugging Face Directly 🤗, Can Agents Really Analyze? 🕵️
Single-tenant Titus: VM-level isolation on a multi-tenant container platform (26 minute read)
Netflix is adding a single-tenant mode to Titus so sensitive workloads can get a dedicated VM without leaving the existing container platform. The motivation is noisy-neighbor interference at the hardware level, which can trigger severe tail-latency spikes for stateful systems. Dedicated nodes improved stability and even recovered capacity lost to anti-affinity, though with slower launches and more warm-capacity planning.
A file format claims to be 100x faster than Parquet (7 minute read)
Parquet's scan-oriented design is showing strain under AI workloads that need fast random access to embeddings and multimodal data. Lance tackles that mismatch with a layout built for direct page access and finer-grained reads, reducing metadata overhead and memory pressure. The broader takeaway is that AI-era storage may need to optimize for point retrieval, not just analytical scans.
Rebuilding LinkedIn's Tracking Infrastructure (13 minute read)
LinkedIn's tracking pipeline handles hundreds of billions of events a day, feeding experiments and product analytics. The team moved it off Java 8 and the Play Framework onto gRPC and a modern Java runtime. Before shifting traffic, they sent 10% of events through both pipelines and compared the output. Inbound network usage fell about 89%.
Rapidly scaling online storage to serve over 1 billion ChatGPT users (23 minute read)
OpenAI's storage platform, Habitat, serves over 500 petabytes to over 1 billion users. Its Azure Cosmos DB setup went from one shared container to one per feature, then back to shared after teams forgot to pre-scale for launches. With hot key throttling, request priorities, and a new serverless mode, it has had no noisy-neighbour incidents since.
We built three memory systems. The most expressive one wasn't the most useful (9 minute read)
Tiger Data gave its Slack agent memory three ways: a list of facts, a knowledge graph, and a tree. Agents struggled to decide what graph edges meant. A tree built on Postgres's ltree extension worked better, and filing a thread summary dropped from dozens of turns to one tool call. Its search layer became searchgres.js.
Can data agents do real analysis? (51 minute podcast)
Data agents still struggle on realistic analytics tasks, especially when work depends on joins across systems, undocumented business context, and messy text transformations. Benchmark results suggest prompt engineering alone will not close that gap. The best agent passed 61% of benchmark queries on its first attempt, with a semantic layer significantly improving accuracy. The episode also explores query planning and custom inference engines to reduce AI SQL costs.
Agents should query your metrics, not your tables (Sponsor)
One metric, twenty different queries, nobody sure which is right. An agent produces the disagreement faster.
Cube is the semantic layer for AI: define a metric once, and dashboards, apps and agents use that definition — LLM accuracy up 17–23 points.
Get the guide or try free → No card required
bbtree: fastest ordered map (6 minute read)
bbtree is a Rust ordered map whose benchmarks report up to 5x faster single-threaded performance than Rust's BTreeMap and 30% lower memory use. It applies decades of database research to reduce memory delays, with LLM-written code guided by a database expert and checked through formal verification and extensive testing.
Sutura (GitHub Repo)
Sutura is an Apache-2.0 Rust runtime that exposes certified metrics and semantic definitions to AI agents over HTTP or MCP, rather than letting them improvise directly against warehouse tables. It can federate queries across systems while verifying the caller and preserving data-source policies. The useful idea is a governed agent-access boundary where metrics, identities, and query execution travel together.
DuckDB hf extension — read and write Hugging Face assets (GitHub Repo)
The DuckDB hf extension lets you read, query, and write files on Hugging Face directly with SQL using hf:// URLs. It supports datasets, models, and storage buckets, with caching to speed up reads and deduplication to avoid uploading data already stored.
Curated deep dives, tools and trends in big data, data science and data engineering 📊
Join 590,000 readers for
one daily email