TLDR Data 2026-10-19
How Cloudflare Scales SQLite ๐, Mistral Introduces Timestral โณ, Which LLM for Analytics? ๐
Under the hood, off the hot path: How D1 scales to millions of databases (16 minute read)
Cloudflare's D1 serverless SQLite platform keeps query traffic at the edge while its control plane manages millions of tenant databases. Lifecycle operations use idempotent, recoverable flows, with Postgres for transactional metadata, edge KV for routing, and queues for eventually consistent work. The transferable design is to isolate central coordination from the data path so provisioning failures can be repaired without slowing queries.
A Graph Has No Centre (12 minute read)
Graph tools for MCP should return focused, bounded views instead of dumping broad subgraphs and hoping prompts impose structure later. The key design move is to make scope explicit in the server through named views, validation rules, and controlled navigation points. For changing entities like avatars, modeling location over time preserves auditability and keeps behavior consistent across clients.
The Self-Organizing Data Stack (10 minute read)
Autonomous data platforms need stable logical assets, not fragile table-and-DAG definitions. This proposed architecture keeps consumer contracts and semantic definitions persistent while a runtime synthesizes, consolidates, verifies, and atomically rewires physical pipelines as needs change. Its strongest safeguard is independent verification against business invariants, plus escalation whenever meaning is ambiguous. Language models may draft changes, but cannot authorize them.
Which LLM should run your agentic analytics? Accuracy vs. cost of Claude, Gemini and GPT (6 minute read)
Opus delivered the best analytics reports but cost more. Routing by task balances quality and cost: Sonnet for metrics and trends, Gemini Flash for guided analysis with human review, and Opus for root cause investigations.
Data Engineering Is the Cache Invalidation Problem Ft. Josh Wills (12 minute read)
Large Spark and Ray jobs offer the wrong feedback loop for autonomous coding: errors and cost often appear only after hours of distributed execution. Data engineers remain responsible for the materialization choices and failure cases agents cannot evaluate cheaply. The practical response is not blind generation, but representative tests, cost-aware guardrails, and human review where pipelines lack fast feedback.
Apple, AWS, Coinbase and Intuit engineers on what actually worked in production (Sponsor)
One free virtual day, Nov 4, to skip months of trial and error. NEXT QUERY 2026: 20+ sessions on the analytics systems behind people, apps and agents, where teams running them share architecture decisions, tradeoffs and fixes, with live Q&A. By PhoenixAI (the original creator of StarRocks).
Register free ยท
Agendaprompt_jev() now works on images (5 minute read)
MotherDuck's prompt_jev() now analyses images directly in SQL, returning structured answers and confidence scores that can be queried alongside other data. In MotherDuck's food-image benchmark, it processed 1,010 photos for about $0.06 at 90.2% accuracy, rising to 97.7% when keeping only the 81% of results with high confidence.
Differential storage for DuckDB (7 minute read)
duckdb-differential brings version-control-style workflows to DuckDB, adding snapshots, time travel, branches, clones, and restore through SQL. The core idea is to preserve known-good database states so teams can test imports, run experiments, and reproduce reports without copying entire databases. Shared immutable chunks keep versions efficient, while optional S3 publishing extends those states for remote access and recovery.
Introducing Timestral (7 minute read)
Mistral's Timestral models forecast multiple related time series, estimate uncertainty, and incorporate known future inputs such as production schedules. They run on CPUs without task-specific training, and Mistral reports that its largest model, with 209 million parameters, outperforms larger rivals across three public forecasting benchmarks.
Tempo 3.1 Release: Kafka, Trace Redaction, and TraceQL Metrics (16 minute read)
Grafana Tempo 3.1 makes Kafka-backed tracing easier to run with TLS, additional authentication methods, rack-aware reads, and configurable producer compression. Query-based redaction can permanently remove sensitive trace data without enumerating trace IDs, while sampling-aware TraceQL metrics improve analysis of incomplete telemetry. Operators should use redaction dry-runs and complete the upgrade before applying time-bounded deletions.
AI in Production: Panellists Highlight Silent Failures, Harness Engineering, and Agent Governance (3 minute read)
Agentic systems can fail while conventional telemetry stays green: semantic drift can corrupt business meaning, and an embedding-model update can abruptly change policy decisions. Production controls therefore need continuous evals fed by live edge cases, separate agent identities with tiered permissions, and human approval of business risk. Treating agents as ordinary application code or relying on prompts for security leaves blind spots.
Reliable GPU compute for large training & inference jobs (10 minute read)
Large GPU jobs fail for many small infrastructure reasons, so reliability has to be continuous rather than reactive. Netflix built a layered system that validates nodes before jobs start, monitors hardware during runs, and automatically drains, reboots, resets, or replaces bad machines. The result is more usable GPU capacity and fewer long training or inference runs lost to platform faults.
Neon now has realtime (13 minute read)
Neon's Realtime public beta turns SQL queries, including joins and aggregates, into live subscriptions that automatically update apps as Postgres data changes, without polling or a separate realtime service.
TabPFN (GitHub Repo)
TabPFN is a foundation model trained on synthetic datasets that performs classification and regression on new tabular data without training a separate model for each dataset.
Curated deep dives, tools and trends in big data, data science and data engineering ๐
Join 590,000 readers for
one daily email