Nov 15, 2023 · In this work, we seek to aggressively decouple learning capacity and FLOPs through Mixture-of-Experts (MoE) style models with large knowledge-rich vocabulary ...

Jun 16, 2024 · We demonstrate that MoWE performs significantly better than the T5 family of models with similar number of FLOPs in a variety of NLP tasks.

Our proposed approach, dubbed Mixture of Word Experts (MoWE), can be seen as a memory augmented model, where a large set of word-specific experts play the role ...

Mar 13, 2024 · Our paper on "Memory Augmented Language Models through Mixture of Word Experts (MoWE)" just got accepted at NAACL 2024.

Jun 13, 2024 · On-demand video platform giving you access to lectures from conferences worldwide.

Duration: 2:46
Posted: Nov 23, 2023

Nov 21, 2023 · Memory Augmented Language Models through Mixture of Word Experts abs: https://arxiv.org/abs/2311.10768 pdf: https://arxiv.org/pdf/2311.10768 ...

Nov 15, 2023 · Our proposed approach, dubbed Mixture of Word Experts (MoWE), can be seen as a memory augmented model, where a large set of word-specific ...

Duration: 23:02
Posted: Nov 21, 2023

Nov 21, 2023 · MoWE can be seen as a memory augmented model where the word experts act as a sparse memory. - MoWE uses tens or hundreds of thousands of experts ...