👨💻
Engineering & Resources
Schedule Free Optimization (GitHub Repo)
Researchers from Meta have been teasing a new optimizer on X. They have now released the code along with various integrations. The optimizer has no LR schedule, which means you don’t need to know the full number of training steps beforehand. It has been shown empirically to work on a wide variety of problems including language models.
SWE-agent (3 minute read)
SWE-agent turns LLMs (e.g. GPT-4) into software engineering agents that can fix bugs and issues in real GitHub repositories. SWE-agent sets the state-of-the-art performance on the full SWE-bench benchmark, resolving 12.29% of issues.
Four Takeaways On The Race To Amass Data For AI (5 minute read)
The development of artificial intelligence heavily relies on vast amounts of data, which is being rapidly consumed by tech companies faster than it is being produced, leading to predictions that high-quality digital data may be exhausted by 2026. In response, companies like OpenAI, Google, and Meta are exploring new methods to obtain more data, including using YouTube video transcripts, revising privacy policies, considering the purchase of major publishers, and investigating the use of "synthetic" data generated by AI models, despite the risk of compounding errors.
Speed Tests for Llama 2, Stable Diffusion (6 minute read)
MLPerf has updated its inferencing benchmarks to include large language models like Llama 2 70B and Stable Diffusion XL, reflecting the industry's shift to massive generative AI. In the latest tests, Nvidia's systems, particularly those equipped with the H200 processor, outperformed competitors from Intel and Qualcomm in both speed and efficiency. The benchmarks highlight the importance of high-bandwidth memory in handling large AI models, with Intel's Gaudi 2 and Qualcomm's Cloud AI 100 Ultra also showcasing significant performances in the generative AI space.
Rabbit partners with ElevenLabs to power voice commands on its device (2 minute read)
Rabbit partners with ElevenLabs to incorporate voice command technology into its upcoming r1 devices, enhancing human-device interaction with low latency models for a more natural experience. The first batch of r1 devices, which feature functionalities like chatbot interaction and bi-directional translation, is set to ship by March 31. ElevenLabs has recently raised $80 million, despite facing challenges with misuse of its voice cloning technology.