GitHub - google-deepmind/recurrentgemma: Open weights language model from Google DeepMind, based on Griffin.
RecurrentGemma
RecurrentGemma is a family of open-weights Language Models by Google DeepMind, based on the novel Griffin architecture. This architecture achieves fast inference when generating long sequences by replacing global attention with a mixture of local attention and linear recurrences.
This repository contains the model implementation and examples for sampling and fine-tuning. We recommend most users adopt the Flax implementation, which is highly optimized. We also provide an un-optimiz...
Read more at github.com