12,885
stars
+0+-1+-4
986
forks
255
watchers
57
prs
40
issues
0
releases
–
discussions
README
Open LLMs These LLMs (Large Language Models) are all licensed for commercial use (e.g., Apache 2.0, MIT, OpenRAIL-M). Contributions welcome! | Language Model | Release Date | Checkpoints | Paper/Blog | Params (B) | Context Length | Licence | Try it | | --- | --- | --- | --- | --- | --- | --- |-----------------------------------------------------------------------------------------------------------------------| | T5 | 2019/10 |T5 & Flan-T5, Flan-T5-xxl (HF) | Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer | 0.06 - 11 | 512 | Apache 2.0 | T5-Large | | RWKV 4 | 2021/08| RWKV, ChatRWKV | The RWKV Language Model (and my LM tricks) | 0.1 - 14 | infinity (RNN) | Apache 2.0 | | | GPT-NeoX-20B | 2022/04 | GPT-NEOX-20B | GPT-NeoX-20B: An Open-Source Autoregressive Language Model | 20 | 2048 | Apache 2.0 | | | YaLM-100B | 2022/06 | yalm-100b | Yandex publishes YaLM 100B, the largest GPT-like neural network in open source | 100 | 1024 | Apache 2.0 | | | UL2 | 2022/10 | UL2 & Flan-UL2, Flan-UL2 (HF) | UL2 20B: An Open Source Unified Language Learner | 20 | 512, 2048 | Apache 2.0 | | | Bloom | 2022/11 | Bloom | BLOOM: A 176B-Parameter Open-Access Multilingual Language Model | 176 | 2048 | OpenRAIL-M v1 | | | ChatGLM | 2023/03 | chatglm-6b | ChatGLM, Github | 6 | 2048 | Custom Free with some usage restriction (might require registration) | | | Cerebras-GPT | 2023/03 | Cerebras-GPT | Cerebras-GPT: A Family of Open, Compute-efficient, Large Language Models (Paper) | 0.111 - 13 | 2048 | Apache 2.0 | Cerebras-GPT-1.3B | | Open Assistant (Pythia family) | 2023/03 | OA-Pythia-12B-SFT-8, OA-Pythia-12B-SFT-4, OA-Pythia-12B-SFT-1 | Democratizing Large Language Model Alignment | 12 | 2048 | Apache 2.0 | Pythia-2.8B | | Pythia | 2023/04 | pythia 70M - 12B | Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling | 0.07 - 12 | 2048 | Apache 2.0 |