TLDR AI 2024-05-01

Meta ExecuTorch Framework πŸ“š, Apple’s Secretive European AI Lab πŸ§ͺ, CUDA Checkpointing πŸ’»

Headlines & Launches

Meta Releases ExecuTorch Framework for LLM on Edge Devices (4 minute read)

Meta's ExecuTorch Framework is a post-training quantization toolkit that enables running Llama models on various iPhone and Galaxy devices. It is able to get as much as 11 tokens per second on mobile phones running 7B sized language models.

Apple Poaches AI Experts From Google, Creates Secretive European AI Lab (3 minute read)

Apple has aggressively expanded its artificial intelligence capabilities by hiring numerous AI experts from Google and establishing a secretive AI research lab in Zurich, Switzerland. The lab, known as the "Vision Lab," focuses on developing advanced AI models that integrate text and visual inputs.
Research & Innovation

Exploring Mamba: A Visual Foundation Model for Advanced Computer Vision (25 minute read)

The Mamba model is a state-of-the-art approach that excels in handling long sequences without the computational drawbacks of traditional Transformers.

Text Retrieval with Faster Embedding Searches (16 minute read)

Seismic is an innovative approach to organizing inverted indexes that significantly enhances the speed and accuracy of text retrieval using learned sparse embeddings.

Adapting Graph Neural Networks to Changing Structures (17 minute read)

The Cluster Information Transfer (CIT) mechanism is a new method designed to enhance the adaptability of Graph Neural Networks (GNNs) to different and changing graph structures.
Engineering & Resources

Meta-Prompting in LLMs (GitHub Repo)

Meta-prompting is a technique that turns a single language model into a multi-skilled team. This method breaks down complex tasks into simpler parts, handled by specialized instances of the same model, leading to significantly improved performance across various tasks.

CUDA Checkpointing (GitHub Repo)

A new toolkit from Nvidia that allows checkpointing the CUDA state for transfer and restarting. It is useful for distributed training of very large AI models.

Friends don't let friends make bad visualizations (GitHub Repo)

Visualizing eval scores for models is key in today's AI research world. However, many charts are bad at conveying the intended information. This repository shows some great charts and comes with dos and don'ts for visualizing results.

Speeding Up Large Language Models with QUICK (GitHub Repo)

QUICK is a new set of optimized CUDA kernels that boosts the efficiency of large language models by solving memory issues. This leads to a notable speed increase on NVIDIA GPUs, making AI models faster and more efficient.

Diddo's new funding will bring its shoppable TV AI API to streaming platforms (3 minute read)

Diddo is a CV/AI API for streaming and media companies that can make their content instantly shoppable without the use of QR codes or second screens, opening up new revenue streams for them. Diddo's API keeps the purchase capabilities on platform and 100% native so it is truly an end-to-end commerce solution for video owners.

Cognition Seeks $2 Billion Valuation for AI Code-Writing Tool (2 minute read)

Cognition Labs, which is transitioning from crypto to AI, is seeking a $2 billion valuation for its code-writing AI tool, Devin. Amidst a trend of soaring valuations in AI startups, Cognition's success reflects the sector's growth and the importance of substantial investments in data and computational resources. Key players like Google and Microsoft continue to advance aggressively in AI with large language models.
Quick Links

FlowTestAI (GitHub Repo)

FlowTestAI stands as the world's first GenAI-powered OpenSource Integrated Development Environment (IDE) specifically designed to craft, visualize, and manage API-first workflows.

Phospho (GitHub Repo)

A text analytics platform for LLM apps.

Apple To Unveil AI-Enabled Safari Browser (2 minute read)

Safari 18 will introduce UI enhancements, an AI-powered Intelligent Search for content summarization, and a 'Web Eraser' for advanced content blocking.
The most important AI, ML, and data science news in a free daily email.
Join 500,000 readers for