TLDR Data 2026-07-13

sqlsure found that text-to-SQL benchmarks such as BIRD and Spider contain incorrect “gold” queries, meaning execution accuracy can reward models for reproducing the benchmark's mistakes and penalise correct answers. It argues that evaluation should also check whether SQL respects declared data semantics such as join cardinality, grain, and additivity.

Affirm rebuilt its pre-checkout messaging service from a Python monolith into a Kotlin microservice with a rule engine, cutting P99 latency by 50% and reducing experiment setup time from 2 months to 4 days since adding a new experiment means writing a rule against existing attributes instead of engineering a new code path and waiting for deployment.

Arcesium migrated a 170TB, 15-trillion-record P&L warehouse from an RDBMS to Apache Iceberg on S3 with DuckDB. The new stack cut ingestion runtime by 80% for larger portfolios, reduced egress timeouts, and lowered infrastructure spend by about 40%. The key enablers were a homegrown, open-sourced DuckDB-Iceberg execution layer called Swiftlake, KEDA for Kubernetes autoscaling, and Iceberg time-travel recovery.

Organizations without dedicated data product owners spend 45% of their week on reactive work versus 27% for teams with clear ownership, a gap driven by the companies with no assigned owners for dashboards, datasets, or models. AI tools widen the gap: teams with clear ownership see an 18-point sentiment gain from AI, while teams without see a 26-point drop.

The updated Shift Left Manifesto sharpens its original software-engineering thesis around provenance: lineage must begin in producer code, not warehouses that only capture data after its meaning and logic have been defined. Release-linked, code-level lineage exposes ownership, sensitive-data flows, semantic dependencies, reuse opportunities, and change blast radius directly in pull requests. This makes governance, model-input traceability, impact analysis, and audit evidence continuous artifacts, a growing requirement as coding agents accelerate cross-system change.

PostgreSQL can prune partitions even when queries filter on a non-partition key, provided that column follows predictable ranges correlated with the partition key. By adding CHECK constraints for those ranges, including separate ranges for outliers, the optimizer can skip irrelevant partitions and avoid scanning every local index.

Privacy engineering requires more than redaction: it needs continuous classification, purpose-based access, fit-for-use controls, enforcement, and auditable policy tracking. Deterministic tokenization supports internal joins but remains pseudonymous because recipients may still link records.

nao is the first open-source framework to build and deploy analytics agents. Data teams own context engineering: data profiling, repositories, rules, semantics, skills. nao brings the harness: a CLI to build your context layer like a file system and a UI to deploy to business users.Try the 1-minute quickstart →

Ship That Code is a free learning platform where users build real systems from scratch, including databases, Redis, SQL tools, and distributed data stores.

Lakekeeper's Generic Table API makes non-Iceberg assets like Lance datasets governable catalog objects without converting them to Iceberg. It stores core metadata while reusing Iceberg catalog controls for access, lifecycle, and scoped credentials.

Apache Airflow is adding three AI-driven controls to reduce pipeline firefighting: semantic schema validation, persistent state for reconnecting failed jobs, and LLM-based retry decisions. Together, they catch schema drift, preserve external job IDs, and classify failures as retry, delay, or escalate.

GitHub added better shared tools to Copilot code review, but its benchmark results got worse: token costs rose and fewer issues were caught because generic tool instructions caused the agent to browse the full repository instead of starting from the diff. Rewriting the instructions to "start from the diff, narrow with grep and glob, read exact evidence with view" cut average review cost by 20% with no quality loss.

ClickHouse now ships hardened Docker images built on a minimal base with no shell, package manager, or network tools, dropping from 8 medium-severity CVEs in the standard Ubuntu image to zero while keeping database behavior identical. Migration is a one-line change to the docker run command, and a debug variant exists that temporarily attaches tooling without modifying the production image.

Databricks found that several models, including open-source options like GLM 5.2, now perform well on real internal coding tasks.

Awesome DuckDB is a curated directory of DuckDB tools, extensions, integrations, clients, tutorials, and real-world projects.