[AINews] Not much happened today. • ButtondownTwitterTwitter

The AI Twitter Recap section provides updates on AI models, research papers, techniques, and frameworks/tools discussed on Twitter. It covers the introduction of Meta 3D Gen for end-to-end generation of 3D assets, updated versions of Perplexity Pro Search and Phi-3 Mini, the launch of GPT4All 3.0, and Yi-Large. It also mentions research papers like Reinforcement Learning from Human Feedback, Persona-Driven Data Synthesis, Meta-tuning for Few-shot Generalization, and Steering Vectors. Additionally, new tools like LangSmith are highlighted.

Gain of Function Research with AI

• @JvNixon expressed concern about 'gain of function research' with AI, drawing parallels to bioweapons research and the potential dangers of creating teams trying to generate novel, dangerous outputs to prove whether models are safe or not.

Probability of Doom vs. Probability of Life

• @JvNixon argued that framing AI risk in terms of p(doom) is a deep collective psychological mistake, forcing people to imagine abstract superintelligence. They prefer p(life) - the probability of you and your loved ones surviving into the far future - as it brings in more of life and progress, and forces a balance of risks against benefits.

Idle Compute in AI Labs

• @far__el noted that many AI labs have lots of idle compute sitting around, as they need compute in bursts. This leads to things like heavily subsidized inference, redefining compute cost as a marketing expense.

CUDA Conclave Convenes

A CUDA-only hackathon hosted by Ash Vardanian features Chris Lattner and is scheduled for July 13th at the AGI House in San Francisco, offering hands-on experience with H100 accelerators. Details available here, courtesy of Nebius.ai.

Matrix Multiplication Mastery

Mobicham surfaces a guide to achieving over 1 TFLOPS performance on matrix multiplication on a CPU platform, specifically tuned for the AMD Ryzen 7700, which surpasses NumPy's offering. Tutorial can be found here.

Integrator Ins and Outs

Conversations unfold about compiling functions in Pytorch using the Inductor backend for Nvidia, mentioning John Carmack's commendations for the PyTorch team while delving into buffer loading and dequantization processes with torchao.

Model Memory Marvel

Cutting-edge memory efficiency strategies put the limelight on this channel's models which comfortably manage batch sizes that would see PyTorch balking, emphasizing on models' memory savings.

Optimizer Odyssey

A wave of optimism is evident with Facebook Research's schedule-free optimizers, claimed to demonstrate accelerated convergence across a spectrum of tasks, potentially reshaping optimization methodologies.

This section highlights various advancements and integrations in AI projects such as real-time text-to-speech operations, tool calling in vLLM, instructional ingenuity from Genstruct 7B, advancements in CommandR by Huggingface, GraphRAG by Microsoft, and more. Discussions range from optimizing models and algorithms to enhancing tools and frameworks, showcasing a collaborative and innovative landscape in AI development.

Datasette to transform data-driven stories, underscoring the importance of accessible and interpretable public data in the digital age.

• Phi-3 Mini update: A significant update to the Phi-3 Mini model is expected, with Gemma 2 support. The update aims to enhance performance and processing speed.
• Moshi's real-time voice model: Kyutai Labs introduced Moshi, a multimodal language model with a 160ms response time and plans for open-source support.
• Gemma 2 support: Unsloth now supports Gemma 2 for advanced AI tasks, with initial positive feedback from users.
• SEQ_CLS support discussion: Users discuss the functionality of SEQ_CLS support in unsloth for fine-tuning tasks, highlighting significant improvements.
• Graph-based RAG integration: Interest in integrating Microsoft's graph-based Retrieval-Augmented Generation system into unsloth to enhance capabilities and optimize workflows.

GPU offload value discussion led to specifics on configuring bot issues with TypeScript and Discord.js. Aquora encountered an invalid token error configuring a Discord bot, resolved by enabling disallowed MessageContent intents. Adjusting temperature and predicted tokens addressed bot hallucinations, fixing bot loops and 'thinking' state issues.

• DJ suggests adding message history and direct message handling in a future article to improve bot functionality. Aquora is eager to contribute to further improvements.
• Transformers 4.42 release introduces Gemma 2, RT-DETR, InstructBlip, and more.
• KerasNLP bridges fine-tuning for any Transformers model.
• AWS releases Chronos datasets on HF.
• Local Gemma offers 100% private and secure generation.
• Vision language models introduction released for tasks like image captioning and OCR.

Members of the Eleuther Discord channel focused on various research topics related to language models and training objectives. Discussions included the slow industry adoption of UL2 training objectives, resolution of discrepancies in scaling laws, efficiency concerns with PrefixLM, comparisons between FIM and UL2 objectives, and adaptations of models to different attention masks. These discussions shed light on the advancement and challenges in the field of language modeling.

The discussion in this section covers various topics related to PyTorch's Inductor, including steps for compiling Pytorch functions with Inductor on Nvidia devices and an issue with Triton kernels generation. Additionally, a tutorial on achieving high-performance matrix multiplication on CPUs, optimized for AMD Ryzen 7700, outperforming NumPy by achieving over 1 TFLOPS using 3 lines of OpenMP directives is shared. The section also includes insights on the impact of 3D V-Cache on AMD's performance and comparisons between 3D and non-3D Ryzen chips.

The discussion highlighted the differences between 3D and non-3D Ryzen chips, emphasizing that 3D versions have double L3 cache but operate at lower clocks to prevent damage to the extra cache silicon layer. Additionally, it was noted that the specialization in 3D V-Cache chips primarily involves more cache and lower clocks. Members also discussed interpreting packed int4 arrays efficiently in Python, dequantizing tensors using Python techniques, and torchao's buffer loading and unexpected keyword handling. Lastly, various links were mentioned throughout the conversation.

• Runtime Error and AV Issues: Users in the tinygrad community faced runtime errors and discussed potential solutions, like using a workaround for the UOps.UNMUL issue.
• Frontend Fuzzer Proposal: A suggestion was made for a frontend fuzzer to catch edge cases more effectively when developing in tinygrad.
• Bug Optimization and Dev Improvements: Discussions centered around improving error messages, dev tooling, and dealing with bugs before the 1.0 release in tinygrad.

Facing installation issue on Ubuntu 24.04: A user reported an installation issue on Ubuntu 24.04/Python 3.12.3, receiving errors for max-engine due to version mismatches. Another user shared a step-by-step guide to resolve this by installing Python 3.11 and adjusting alternatives.

Mojo's odd implicit conversion: A user noticed that multiplying integers by np.pi in Mojo produces an unexpected negative integer result due to an implicit conversion bug. The discussion pointed out that this is related to casting bugs already tracked as #3065 and #3167.

This section highlights a variety of posts and discussions related to legal and technical aspects within the Interconnects category. The topics include potential First Amendment challenges for SB 1047, debates on model weights as protected speech, admiration for Claude 3.5 release, and suggestions to promote Claude TM. The discussions offer insights on legal protections for code, comparisons with human languages, and community excitement regarding specific technology releases.

johnlenflure asked about integrating O1 into glasses. In the OpenAccess AI Collective general channel, weighted cross entropy in the trainer was discussed. Differences between LoRA and QLoRA quantization were explained, with QLoRA facilitating efficient fine-tuning. Members in the general-help channel discussed LoRA and QLoRA configurations, the efficiency of QLoRA in fine-tuning large models, and the impact of 8-bit quantization. They also shared a link for QLoRA: Efficient Finetuning of Quantized LLMs. Additionally, issues with CUDA memory allocation on Google Colab were addressed in the axolotl-help-bot channel, as well as tips for handling VRAM requirements. DeepSpeed configurations for data and model sharding were discussed in the LLM Finetuning (Hamel + Dan) axolotl channel. Lastly, in the LLM Perf Enthusiasts AI eval channel, Screens' evaluation report on LLM accuracy in legal contract review was mentioned, along with methodologies for LLM assessment in legal tasks. They also shared a link for the Screens Accuracy Evaluation Report.