TLDR Data 2026-08-24

Netflix is moving 30,000+ Flink jobs from a homegrown autoscaler toward the Apache Flink Autoscaler. The old system saved 25–45% by scaling whole clusters from container metrics, but missed operator-level state and suffered from telemetry gaps. The OSS autoscaler reasons from inside each job and produced 58% annualized compute savings for one team.

Model maintenance is not just retraining. When COVID-era demand patterns faded, Airbnb's forecasting team separated parameter drift from structural change, choosing among refit, respecify, or hold decisions instead of blindly updating thousands of markets. The result was a useful operating model for production forecasting under regime shifts.

Good SQL indexes come from real query patterns, not the table schema. Composite indexes need the right column order, usually equality filters first and sort or range columns after, with EXPLAIN ANALYZE used to verify the result. The right indexes cut example queries from 17ms to 0.04ms and 436ms to 0.5ms.

Uber's Gulfstream payments platform now supports $217B in annualized gross bookings, nearly twice that in money movement, and ledger balances for 1.2B+ entities. The design relies on immutable zero-sum money orders, double-entry accounting, DynamoDB-backed balances, Kafka pipelines, Cadence for synchronous flows, and generic payment-instrument interfaces that let new business lines plug in with minimal core changes.

Dashboards are not neutral: the metrics and visuals they prioritize shape what leaders see, fund, and act on, while harder-to-measure realities disappear. In African public-sector and NGO settings, donor-facing dashboards can obscure frontline needs and should be balanced with local knowledge.

Bulk export APIs are giving way to shared table references. Iceberg REST catalogs, object storage, Parquet, and credential vending let producers publish metadata while consumers bring their own compute, avoiding pagination, rate limits, and duplicate pipelines. The hard parts move to contracts, compaction, semantics, privacy, and table maintenance promises.

Production AI systems work best when each component earns its complexity. Recommenders, moderation, warehouse queries, and search cold starts need different mixes of deterministic logic, classic ML, semantic layers, and LLMs. The strongest pattern is often hybrid: use agents for reasoning, but keep business logic, metrics, routing, and guardrails deterministic where possible.

Two projects push DataFusion beyond standard analytics: one combines RDF graphs with time-series and temporal logic, while Cloudflare's AutoSQL uses it for distributed serverless SQL over Iceberg and R2. Both rely heavily on custom planning, pruning, joins, and execution optimizations.

TrueForge is a self-hostable agent harness focused on reducing task cost through context compaction, delayed tool-schema loading, subagents, and sandbox-as-a-tool execution. The release matters for data teams experimenting with governed agents because it separates orchestration from model choice and pairs naturally with gateways for identity, access control, observability, and budgets.

DuckDB v2.0 replaces its PostgreSQL-derived SQL parser with a PEG-based parser while preserving the downstream binder and AST pipeline. The change removes Bison conflict pain, avoids exponential backtracking on malformed input, and enables runtime grammar extensions that can add syntax without reimplementing SQL parsing.

Agent scaffolding mattered far more to cost than whether tools used MCP or CLI. CLI-only agent scaffolds were 5x to 28x cheaper, while MCP-vs-CLI cost ratios ranged from 0.43x to 29x, making interface choice alone a weak predictor of efficiency.

Peter Sobot used 1,000+ photos, Claude, and Gaussian Splatting to build a 3D model of the New York skyline. Running 74+ subagents and roughly 280 experiments, the project showed that AI can handle much of the technical workflow, but still struggles with visual judgement, messy data, and knowing when the result is genuinely good.

WarpStream's Orbit automates Kafka migration planning and cutover checks so teams can move clusters faster without silently breaking producers, consumers, offsets, or ordering assumptions.

Andrew Ng's AI engineering skills map centers on six areas: LLM foundations, data grounding, agents, evals, production operations, and ML fundamentals.