TLDR AI 2024-06-07

OpenAI security architecture 🔐, Qwen 2 2️⃣, DragonFly Vision Model 👓

Headlines & Launches

Qwen 2 Launch (8 minute read)

Qwen 2 is a new flagship language model from Alibaba Cloud. It slightly surpasses Llama 3 70B on benchmark performance in English while being a better multilingual model.

Securing Research Infrastructure For Advanced AI (5 minute read)

OpenAI has outlined the security architecture of its AI training supercomputers, emphasizing the protection of sensitive model weights and other assets using Azure-based infrastructure and Kubernetes for orchestration.

Nvidia emails: Elon Musk diverting Tesla GPUs to his other companies (3 minute read)

Elon Musk redirected 12,000 H100 GPUs ordered by Tesla to X. He had told investors in April that Tesla had spent $1 billion on GPUs in the three months of the year. Tesla has been developing its own in-house supercomputer for AI, but Musk has previously said that it would be redundant if the company could source more H100s. X's order of 12,000 H100s will be redirected to Tesla.
Research & Innovation

DragonFly Vision Model (11 minute read)

The Together AI team has a novel VLM that excels at extremely high resolution images due to its efficient architecture.

Extracting Concepts from GPT-4 (8 minute read)

The team at OpenAI has discovered 16 million interpretable features in GPT-4 including price increases, algebraic rings, and who/what correspondence. This is a great step forward for SAE interpretability at scale. They shared the code in a companion GitHub repository.

Self-Attention with Convolutions (16 minute read)

Researchers have developed a novel approach called FFNification that transforms self-attention mechanisms into more efficient token mixers using only convolutions while keeping the query-key-value framework.
Engineering & Resources

Mesop: Gradio Competition (3 minute read)

Google has released a competitor to Gradio, a popular AI prototyping framework. Mesop is pure Python and has slightly more composability but is less mature than Gradio.

Visual Geo-localization with Multi-Modal Learning (24 minute read)

Researchers have developed a new two-stage training method to improve Visual Geo-localization (VG), enhancing its performance in applications like autonomous driving, augmented reality, and SLAM.

Enhancing Long-Range Dependencies in State Space Models (GitHub Repo)

GrootVL is a network that improves state space models by dynamically generating a tree topology based on spatial relationships and input features.

AI Voice Agents (10 minute read)

Andreessen Horowitz put together a blog post and deck outlining the state of play for AI voice agents and where it sees opportunities for innovation and investment. It covers the currently leading infrastructure stack and maps out the current B2B and B2C application layer landscape.

Advancing Implicit Neural Representations with ReLU (26 minute read)

Researchers have revisited the use of ReLU activation functions in learning implicit neural representations (INRs). Inspired by second-order B-spline wavelets, they introduced simple constraints to ReLU neurons to counter spectral bias.

A Grand Unified Theory of the AI Hype Cycle (8 minute read)

The AI industry has seen several hype cycles throughout the decades, each producing genuinely useful technology and lasting longer than the last. Every cycle follows the process of a sigmoid curve rather than an exponential one. All strategies for developing the technology inevitably have a limit that doesn't take too long to find. While the current AI hype cycle is unlike any that have come before it, it will likely follow a similar trajectory as previous cycles.
Quick Links

Nvidia is now more valuable than Apple at $3.01 trillion (1 minute read)

Nvidia became the second most valuable company in the world on Wednesday afternoon as its market capitalization hit $3.01 trillion.

MMLU Pro (Hugging Face Hub)

MMLU is the standard measure of reasoning and knowledge for language models.

Synthflow AI (Product)

Automate customer interactions with AI voice assistants.
The most important AI, ML, and data science news in a free daily email.
Join 500,000 readers for