Module ep_dispatch

Module ep_dispatch 

Source
Expand description

EP=2 token dispatch/combine routing for MoE expert parallelism.

Instead of dense all-reduce after local expert compute, this module partitions tokens by expert ownership and dispatches only the tokens that need remote experts. Communication is O(dispatched_tokens * hidden) rather than O(total_tokens * hidden).

See tasks/ep2-token-dispatch-design.md for the full design.

Structs§

EpRoutingTable
Routing table for EP token dispatch.

Functions§

build_ep_routing_table
Build EP routing table from flattened gate indices and weights.