CoLT5 processes every input token with lightweight layers and gives selected tokens additional, higher-capacity computation. This guide explains its learned routing, light and heavy branches, faster decoding, and experiments with inputs up to 64k tokens.

View comments.

more ...