TLDR AI 2024-04-30

GitHub Copilot Workspace πŸ’», ChatGPT Memory Feature 🧠, OpenAI & Financial Times 🀝

Headlines & Launches

Copilot Workspace (4 minute read)

GitHub has announced a new project to allow language models to somewhat autonomously finish features and fix bugs.

Effort Engine (3 minute read)

Effort enables real-time adjustment of computations during LLM model inference on Apple Silicon chips, striking a balance between speed and quality. The method, which requires no retraining but requires conversion and precomputation, allows models to operate faster by loading fewer weights. The developers are seeking assistance from Swift/Metal engineers to optimize the implementation, which is available for download on GitHub.

OpenAI Introduces β€œMemory” Feature For ChatGPT Plus Users (2 minute read)

OpenAI has launched a new "Memory" feature for ChatGPT Plus users, allowing the AI to retain information across conversations, which can be managed via settings.
Research & Innovation

FP6-LLM (27 minute read)

Reducing floating point precision speeds up training but can be unstable. This paper shows that you can get full tensor core utilization in a new packing format without common instabilities or slowdowns from naive attempts.

Star-Coder 2 15B instruct (17 minute read)

This model is trained on synthetic and human data. Permissively licensed, it achieves 72.6 on humaneval. Its developers give great insights into how to replicate their data pipeline and apply principles to other problems where synthetic data might be useful.

Open Gato Replication: JAT (28 minute read)

GATO by DeepMind was touted to be a generalist agent. A team associated with Hugging Face has trained and evaluated JAT, which is a Jack-of-All-Trades model that performs reasonably on a wide variety of tasks.
Engineering & Resources

Whisper cpp cli (GitHub Repo)

A fully self-contained speech-to-text system built on top of Whisper by the folks who built Ruff. (GitHub Repo)

A fast LLM inference platform that supports inference on a variety of devices, quantization, and easy-to-use applications with an OpenAI API-compatible HTTP server and Python bindings.

Memary (GitHub Repo)

Memary allows your agents to store a large corpus of information in knowledge graphs, infer user knowledge through our memory modules, and only retrieve relevant information for meaningful responses.

Jax internals (30 minute read)

Autodidax tutorial that shows the inner ideas in the growing Jax framework. It also introduces helpful jargon often used in the community.

Binding Public Sector AI Diffusion (6 minute read)

The OMB's new AI executive order policy aimed at the public sector could severely impede AI innovation due to stringent safety provisions and bureaucratic hurdles. Implemented amidst shrinking IT budgets, the regulations may not only slow AI deployment but also potentially halt projects crucial for modernizing government services. Critics worry that these constraints, coupled with funding cuts, could prevent agencies from keeping pace with technological advances in fields like healthcare.

AI isn't useless. But is it worth it? (16 minute read)

This article provides a critical examination of AI and machine learning, arguing that while AI tools can be useful for certain tasks, they often fail to live up to the grand promises made by AI companies. It highlights the significant gap between AI's practical use cases, like coding assistance and language generation, and the inflated expectations set by industry narratives. The article raises concerns about the ethical and environmental costs of AI development, along with the potential for misuse and overreliance on the technology.
Quick Links

Moondream llamafile (Hugging Face Hub)

A fully self-contained VLM executable that can run on edge devices based on the Moondream 2 model, which boasts powerful performance for its size.

OpenAI Will Train Its Models On Financial Times' Content (2 minute read)

The Financial Times has entered into a partnership with OpenAI where OpenAI will use the publication's journalism to train its AI models and develop new AI-driven features for its readers.

01 (GitHub Repo)

An attempt to build an open-source ecosystem for AI devices.
The most important AI, ML, and data science news in a free daily email.
Join 500,000 readers for