TLDR Data 2026-08-27

They're all showing up to OpenSearchCon North America to share how they run AI search, observability, and data infrastructure at (massive) scale.

Come for real production patterns and two hands-on workshops:

Build an OpenTelemetry observability stack

Build an agentic AI search app

Additional speakers come from Airbnb, Intel, IBM and more. Join in San Jose from September 22 to 24, to build the future of open source search. Come for the talks, leave with something you can actually use on Monday.

Check out the schedule →

DoorDash's SafeChat handles millions of chats, calls, and images per day with a hybrid moderation pipeline. Fast internal classifiers clear obvious safe cases, LLMs score the hardest cases across severity axes, and no-code workflows plus backtesting let teams tune actions before production. The system cut verbal-abuse incidents roughly in half.

In Kafka-style streaming systems, the shared log is the ordered event backbone that brokers and consumers build on. LogDrive separates that log into an AtomicLog sequencing layer and a durability layer built from flat single-value registers plus weakTail, making striping and quorum replication reusable across S3, DynamoDB, S3 Express, or other backends.

RocksDB amplification depends heavily on workload shape, compaction capacity, and key order. Random inserts produced 4.56x write amplification while sequential keys stayed the same, and delete-heavy workloads temporarily pushed space amplification to 1.58x. Bloom filters kept L0 read latency nearly flat as file counts grew.

LinkedIn built a cognitive memory agent that personalizes its hiring assistant using four layers: conversational, episodic, semantic, and procedural memory. It replaced GraphRAG with a hierarchical tree that supports cheaper incremental updates, while dedicated ingestion, consolidation, and retrieval services keep preferences fresh, traceable, access-controlled, and available within tight latency limits.

Complex tasks can exceed a single AI agent's organizational capacity, regardless of better context or tools. Graph Engineering addresses this by mapping tasks, specialized agents, and system states to coordinate parallel work, verification, and persistent execution.

AI creates more pipelines and dependencies without matching growth in data teams, making engineers the operational bottleneck. Companies should prevent burnout with scalable infrastructure, governance, and automated operations matched to their actual needs and capacity.

GitHub's secret-scanning work shows why LLM evaluation should optimize the production decision, not generic benchmark scores. Prompts, models, input construction, and eval configs were versioned like code, then tested against production-like ambiguity. That discipline cut false positives by 95% offline while preserving recall within the safety guardrail.

Missed RFS dates, Phantom Clusters, and layers of vendors compound challenges that start with a volatile supply chain. Michael Francisco, Vice President of WhiteFiber Cloud US, breaks down what's driving these dynamics from the seller's side, and how to simplify your sourcing process.

Read the breakdown

Talk to WhiteFiber's engineers

Khaos is an open-source Kafka load-testing and chaos-simulation tool that was recently rewritten in Go. It generates realistic producers and consumers, exercises broker failures, hot partitions, consumer lag, rebalances, and backpressure, and gives platform teams a repeatable way to test Kafka behavior before they encounter production incidents.

DuckLabs is joining AWS to gain the infrastructure, resources, and reach needed to scale DuckDB and the wider Duck Stack. DuckDB, DuckLake, and Quack will remain MIT-licensed open-source projects stewarded by the independent DuckDB Foundation, while AWS supports faster development and broader adoption.

PostgreSQL regex search can vary wildly by index and extension. In a 1.6M-row, 33 GB plan table, pg_trgm cut a simple LIKE scan from about 40 seconds to 1.6 seconds, pg_tre added fuzzy matching but had painful index build time, and pg_re2 delivered faster regex execution with fewer language features.

PagedAttention brings virtual-memory-style paging to LLM KV caches by replacing contiguous per-request allocation with fixed-size blocks and a block table. That design eliminates most cache fragmentation, raises utilization to roughly 96%, supports copy-on-write for shared prompts, and can serve 2–4x more users per GPU without changing attention outputs.

Enterprise application software is still growing, but aggregate market numbers hide where AI is actually changing buyer behavior. Workflow-heavy and automation-friendly categories are pulling ahead while customer service and HCM lag. For buyers, AI packaging and pricing now reveal more about value capture than broad market-growth headlines.

Analyst Gym uses daily two-minute workplace drills to sharpen analytical judgment rather than coding skills.

Neocarta builds a Neo4j semantic layer that unifies database schemas, business definitions, metrics, and query history, then exposes this context to AI agents through MCP.