The Falcon models from the UAE have long been the best open models available. The newest 180B parameter model slightly outperforms Llama 2 70B and has a 2k context window. The models have historically been quite tunable. However, given the resource requirements, it's not clear whether the community will adopt this newest model.
Apple has boosted its budget for developing artificial intelligence, emphasizing creating conversational chatbot features for Siri — allegedly spending millions of dollars daily on research and development.
OpenAI will hold its inaugural DevDay conference in San Francisco on November 6, 2023. This event will preview new tools and foster idea exchange, attracting hundreds globally. Currently, over 2 million developers use tools like GPT-4 and DALL·E through OpenAI's continuously updated API.
PPO is a pain in RLHF because you need to have 3 models floating around. However, they all stay close and only require small updates from one another. Enter LoRA. If you instead use adapters you can dramatically reduce memory costs without losing performance. Beautiful idea in its simplicity.
This study delves into the complexities of 3D face animation driven by speech, highlighting two important aspects: global factors that change how the face moves over time (composite nature) and how different parts of the face move independently based on local muscles (regional nature).
Financial institutions use something called a limit order book to maintain all of the transaction information for trades on their platforms. This is useful, but it usually runs on the CPU, which makes running RL on them hard. This paper talks through creating an order book that runs on the GPU written in JAX.
Using a pre-trained text-to-image model to edit videos produces dream-like and strange results. TokenFlow is much smoother and maintains many semantic and structural features of the original video. It almost seems to outperform Runway Gen-2.
ReliTalk is a cool tech breakthrough that lets us make video avatars look natural even when you change the lighting or background. It uses a single video and the sound of your voice to create a 3D face model.
Doppelgangers is a new tech tool that helps figure out if two pictures that look almost the same are actually showing the same 3D object or not. It's smart enough to avoid mistakes that even people might make.
It appears that AI models were able to rapidly memorize examples from the dataset after seeing them just once. This astonishing feat contradicts most prior wisdom about neural network sample efficiency.