xqlsystems/xarray-sql: Causal attention benchmark and SQL implementation

github.com9 min read

This benchmark script demonstrates causal attention implemented as standard SQL queries running on multiple engines, including DuckDB and DataFusion. The repository benchmark checks the forward pass, gradient calculation via ddx, mask semantics, and a control query, while verifying that the einfold query optimizer successfully fuses the queries into array-like execution plans. Open article →

More on this topic