AI Daily — May 10, 2026

huggingface.co

EMO: Pretraining Mixture of Experts for Emergent Modularity

Allen AI released EMO, a pretraining approach for Mixture-of-Experts models that encourages emergent modularity — where experts spontaneously specialize during training rather than through explicit routing constraints. The method aims to improve expert utilization and model efficiency without requiring hand-designed specialization objectives. This is a research contribution from AllenAI targeting a known weakness in standard MoE training dynamics.

#research#llms#open-source