TLDR AI 2024-12-27

Google contractors are comparing Gemini AI's responses with Anthropic's Claude, but it's unclear if Google has permission to use Claude for testing. The contractors noted that Claude prioritizes safety in its responses more than Gemini. Google states that while it evaluates model outputs against competitors, it does not train Gemini on Anthropic's models.

AI systems are rapidly advancing, outperforming expectations on new challenging evaluations, like Epoch AI's FrontierMath. Despite this progress, designing effective evals to understand and manage AI capabilities remains complex and underfunded. Experts stress the need for sophisticated, timely tests to monitor potential risks as models evolve.

AI hallucinations, often criticized for inaccuracies, are proving valuable in scientific research by accelerating idea generation and discovery processes. Notable achievements include Nobel Prize-winning protein designs and innovations in antibiotic development and catheter design. Despite controversies around the term "hallucinations," experts acknowledge AI's potential for groundbreaking scientific insights.

Meta has released an updated agents framework to measure and ensure robustness and safety when deployed in the wild.

This research introduces a technique for applying material transformations, like wetness or coating, across different scenes using disentangled Neural Radiance Fields (NeRF).

Researchers explored the consequences of training AI language models to optimize for user feedback, like thumbs up ratings. The study shows that this approach can lead to manipulation as AIs learn to game the system.

A new benchmark evaluates the federated fine-tuning of MLLMs across diverse scenarios, including two datasets, five baselines, and over ten types of multimodal heterogeneities.

A helpful guide for prompt engineering.

A number of puzzles and tutorials to learn GPU programming on Mac Metal acceleration.

Fei-Fei Li's startup, World Labs, aims to enhance AI with 3D spatial intelligence to generate and interact with 3D worlds. The focus on spatial intelligence is crucial for advancing AI capabilities in both real and virtual 3D environments. This technology could revolutionize sectors like robotics, design, and augmented reality.

AI development faces a data shortage crisis, projected to hit by 2028, as training data sizes approach the total stock of public online text. AI companies like OpenAI are exploring solutions such as generating new data and using unconventional sources to mitigate this issue. This data crunch may shift focus toward smaller, specialized AI models rather than large-scale LLMs.

Red Rabbit Robotics is advancing the field of autonomous labor by developing and open-sourcing the RX1 robot for manufacturing and commercial applications. It aims to tap into the labor shortage by providing cost-effective, robotic solutions for dull, dangerous, and dirty jobs. The company's goal is to make robots more accessible at a lower price point and gradually transition from teleoperation to full autonomy, prioritizing utility and widespread adoption.

Klarna CEO Sebastian Siemiatkowski claims generative AI has enabled a workforce reduction, though the company is still hiring for essential roles.

Researchers at Meta and UC San Diego are developing LLMs that process logical solutions in "latent space," bypassing natural language constraints.

OpenAI's Kubernetes incident on December 11 was caused by unexpected interactions, where a new telemetry service overloaded the Kubernetes API servers, leading to failures in DNS-based service discovery.