Issue #66: A new, better AI training method

Google DeepMind introduces JEST

Welcome to Issue #66 of One Minute AI, your daily AI news companion. This issue discusses a recent update from Google DeepMind.

---

Introducing JEST, a new AI training method from DeepMind

Google DeepMind has introduced JEST, an innovative AI training method that significantly accelerates and optimizes large-scale training. JEST selects data batches jointly rather than independently, utilizing multimodal contrastive objectives to choose the most relevant sub-batches from larger super-batches. This approach reduces computational overhead and enhances training efficiency by leveraging pre-trained reference models. Additionally, Flexi-JEST, a variant of JEST, further minimizes costs using variable patch sizing, outperforming state-of-the-art models in both speed and computational efficiency.

The JEST method addresses the limitations of traditional data curation methods by focusing on joint batch selection, which improves learning signals and reduces dependency on individual data points. Offline and online data curation methods, including hard negative mining and model approximation, enhance the efficacy of JEST. The method prioritizes learnability scoring, combining high-loss and low-loss data selection to form the most learnable batches. JEST demonstrates superior performance in multimodal learning tasks, accelerating training and achieving high efficiency with fewer iterations and computational resources. This advancement highlights the potential for dynamic, high-quality data curation in AI training.

---

Want to help?

If you liked this issue, help spread the word and share One Minute AI with your peers and community.

You can also share feedback with us, as well as news from the AI world that you’d like to see featured by joining our chat on Substack.