| AI model | Parameters | Active parameters | Layers | KV heads | Context |
|---|---|---|---|---|---|
| Mistral AI Mistral 7B Instruct v0.3 | 7.25 B | – | 32 layers | 8 heads | 32,768 tokens |
| Mistral AI Mistral Small 3 24B Instruct (2501) | 24 B | – | 40 layers | 8 heads | 32,768 tokens |
| Mistral AI Mistral Small 3.2 24B Instruct (2506) | 24 B | – | 40 layers | 8 heads | 131,072 tokens |
| Mistral AI Mistral-7B-Instruct-v0.1 | 7.24 B | – | 32 layers | 8 heads | 32,768 tokens |
| Mistral AI Mistral-7B-Instruct-v0.2 | 7.24 B | – | 32 layers | 8 heads | 32,768 tokens |
| Mistral AI Mistral-7B-v0.1 | 7.24 B | – | 32 layers | 8 heads | 32,768 tokens |
| Mistral AI Mixtral 8x7B Instruct v0.1 | 46.7 B | 12.9 B | 32 layers | 8 heads | 32,768 tokens |
Other makers
- Qwen (Alibaba Cloud)73
- DeepSeek23
- Meta21
- ibm-granite17
- nvidia15
- Microsoft14
- allenai13
- Google13
- zai-org13
- EleutherAI8
- NousResearch8
- LiquidAI6
- HuggingFaceTB5
- moonshotai5
- h2oai4
- MiniMaxAI4
- openbmb4
- poolside4
- bigscience3
- internlm3
- LGAI-EXAONE3
- openai-community3
- tencent3
- TinyLlama3
- XiaomiMiMo3
- 01-ai2
- ai21labs2
- bigcode2
- CohereLabs2
- inclusionAI2
- OpenAI2
- PrimeIntellect2
- tiiuae2
- upstage2
- aisingapore1
- baidu1
- BSC-LT1
- ByteDance-Seed1
- codellama1
- facebook1
- Salesforce1
- stepfun-ai1
- swiss-ai1