Meta’s new open source model Muse Spark Open just dropped, and it uses this foundational architecture called the Mixtur…
From @harpercarrollai on Instagram — 662.0K followers · See full profile →
Meta’s new open source model Muse Spark Open just dropped, and it uses this foundational architecture called the Mixture of Experts architecture. Pretty much every AI model released today uses it, and it’s founded on this premise: a huge AI model doesn’t have to activate all its parameters for every input. With mixture of experts, or MoE, a router decides which expert networks to activate for each token. Instead of using the entire model every time, it selects specific experts, saving compute…
Tags: Model Launch · AI Infra · Reel
Topics in this post
AI Infra · Meta · MoE · NVIDIA
More from @harpercarrollai
- It was a joy to chat with Dierdre Bosa on Nebius Live today during NVIDIA GTC in Berlin! We talk AI capability, adoptio…
- In honor of @NVIDIA GTC in Berlin, here’s a video of me riding around San Francisco in the first production car to ship…
- Golden trio: @Apple Mac Studio + @Meta’s @Muse Glimmer + Hermes Agent I can do a whole walk through tutorial! Follow to…
- Interesting fact: insurance cover breast screenings AFTER 40, but lots of diagnoses come before then; in this clinic, 3…
- Here’s a step-by-step guide to use @ChatGPT to redesign all your YouTube thumbnails, and run the A/B tests for you. Cha…
- Last week! Interviewed head of science at @biohub Alex Rives. Biohub’s mission is to cure or prevent all disease. They …