README

Mistral Inference

 ⚠️ Archived — no longer maintained

This repository is archived and no longer actively maintained. We may launch a new library or a major update in the future if the community needs it or we believe we can bring value to the inference ecosystem.

---


  



This repository contains minimal code to run Mistral models.

Blog 7B: https://mistral.ai/news/announcing-mistral-7b/\
Blog 8x7B: https://mistral.ai/news/mixtral-of-experts/\
Blog 8x22B: https://mistral.ai/news/mixtral-8x22b/\
Blog Codestral 22B: https://mistral.ai/news/codestral \
Blog Codestral Mamba 7B: https://mistral.ai/news/codestral-mamba/ \
Blog Mathstral 7B: https://mistral.ai/news/mathstral/ \
Blog Nemo: https://mistral.ai/news/mistral-nemo/ \
Blog Mistral Large 2: https://mistral.ai/news/mistral-large-2407/ \
Blog Pixtral 12B: https://mistral.ai/news/pixtral-12b/
Blog Mistral Small 3.1: https://mistral.ai/news/mistral-small-3-1/

Discord: https://discord.com/invite/mistralai\
Documentation: https://docs.mistral.ai/\
Guardrailing: https://docs.mistral.ai/usage/guardrailing

 Installation

Note: You will use a GPU to install mistral-inference, as it currently requires xformers to be installed and xformers itself needs a GPU for installation.

 PyPI


pip install mistral-inference


 Local


cd $HOME && git clone https://github.com/mistralai/mistral-inference
cd $HOME/mistral-inference && poetry install .


 Model download

 Direct links

| Name        | Download | md5sum |
|-------------|-------|-------|
| 7B Instruct | https://models.mistralcdn.com/mistral-7b-v0-3/mistral-7B-Instruct-v0.3.tar | 80b71fcb6416085bcb4efad86dfb4d52 |
| 8x7B Instruct | https://models.mistralcdn.com/mixtral-8x7b-v0-1/Mixtral-8x7B-v0.1-Instruct.tar (Updated model coming soon!) | 8e2d3930145dc43d3084396f49d38a3f |
| 8x22 Instruct | https://models.mistralcdn.com/mixtral-8x22b-v0-3/mixtral-8x22B-Instruct-v0.3.tar | 471a02a6902706a2f1e44a693813855b |
| 7B Base | https://models.mistralcdn.com/mistral-7b-v0-3/mistral-7B-v0.3.tar | 0663b293810d7571dad25dae2f2a5806 |
| 8x7B |     Updated model coming soon!       | - |
| 8x22B | https://models.mistralcdn.com/mixtral-8x22b-v0-3/mixtral-8x22B-v0.3.tar | a2fa75117174f87d1197e3a4eb50371a |
| Codestral 22B | https://models.mistralcdn.com/codestral-22b-v0-1/codestral-22B-v0.1.tar | 1ea95d474a1d374b1d1b20a8e0159de3 |
| Mathstral 7B | https://models.mistralcdn.com/mathstral-7b-v0-1/mathstral-7B-v0.1.tar | 5f05443e94489c261462794b1016f10b |
| Codestral-Mamba 7B | https://models.mistralcdn.com/codestral-mamba-7b-v0-1/codestral-mamba-7B-v0.1.tar | d3993e4024d1395910c55db0d11db163 |
| Nemo Base | https://models.mistralcdn.com/mistral-nemo-2407/mistral-nemo-base-2407.tar | c5d079ac4b55fc1ae35f51f0a3c0eb83 |
| Nemo Instruct | https://models.mistralcdn.com/mistral-nemo-2407/mistral-nemo-instruct-2407.tar | 296fbdf911cb88e6f0be74cd04827fe7 |
| Mistral Large 2 | https://models.mistralcdn.com/mistral-large-2407/mistral-large-instruct-2407.tar | fc602155f9e39151fba