Effort Engine (3 minute read)
Effort enables real-time adjustment of computations during LLM model inference on Apple Silicon chips, striking a balance between speed and quality. The method, which requires no retraining but requires conversion and precomputation, allows models to operate faster by loading fewer weights. The developers are seeking assistance from Swift/Metal engineers to optimize the implementation, which is available for download on GitHub.
FP6-LLM (27 minute read)
Reducing floating point precision speeds up training but can be unstable. This paper shows that you can get full tensor core utilization in a new packing format without common instabilities or slowdowns from naive attempts.
Star-Coder 2 15B instruct (17 minute read)
This model is trained on synthetic and human data. Permissively licensed, it achieves 72.6 on humaneval. Its developers give great insights into how to replicate their data pipeline and apply principles to other problems where synthetic data might be useful.
Open Gato Replication: JAT (28 minute read)
GATO by DeepMind was touted to be a generalist agent. A team associated with Hugging Face has trained and evaluated JAT, which is a Jack-of-All-Trades model that performs reasonably on a wide variety of tasks.
Jax internals (30 minute read)
Autodidax tutorial that shows the inner ideas in the growing Jax framework. It also introduces helpful jargon often used in the community.
Binding Public Sector AI Diffusion (6 minute read)
The OMB's new AI executive order policy aimed at the public sector could severely impede AI innovation due to stringent safety provisions and bureaucratic hurdles. Implemented amidst shrinking IT budgets, the regulations may not only slow AI deployment but also potentially halt projects crucial for modernizing government services. Critics worry that these constraints, coupled with funding cuts, could prevent agencies from keeping pace with technological advances in fields like healthcare.
AI isn't useless. But is it worth it? (16 minute read)
This article provides a critical examination of AI and machine learning, arguing that while AI tools can be useful for certain tasks, they often fail to live up to the grand promises made by AI companies. It highlights the significant gap between AI's practical use cases, like coding assistance and language generation, and the inflated expectations set by industry narratives. The article raises concerns about the ethical and environmental costs of AI development, along with the potential for misuse and overreliance on the technology.