Meta
Llama 3.1 70B Instruct
Check price at Amazon for Meta Llama 3.1 70B Instruct (affiliate link, opens in new tab)Affiliate link
- Weights at 16-bit
- 140 GB
- Weights at 8-bit (Q8_0)
- 74 GB
- Weights at 4-bit (Q4_K_M)
- 42 GB
- Smallest card at 4-bit
- RTX 6000 Ada Generation · 48 GB
- Parameters
- Layers
- KV heads
- Licence
Specs
- Attention heads
- Downloads (30 days)
- Family
- GGUF downloads
-
repo bytes files quant bartowski/Meta-Llama-3.1-70B-Instruct-GGUF 16,751,200,832 1 entries IQ1_M bartowski/Meta-Llama-3.1-70B-Instruct-GGUF 19,097,389,632 1 entries IQ2_XXS bartowski/Meta-Llama-3.1-70B-Instruct-GGUF 21,142,112,832 1 entries IQ2_XS bartowski/Meta-Llama-3.1-70B-Instruct-GGUF 22,242,347,584 1 entries IQ2_S bartowski/Meta-Llama-3.1-70B-Instruct-GGUF 24,119,298,624 1 entries IQ2_M bartowski/Meta-Llama-3.1-70B-Instruct-GGUF 26,375,113,280 1 entries Q2_K bartowski/Meta-Llama-3.1-70B-Instruct-GGUF 27,401,161,280 1 entries Q2_K_L bartowski/Meta-Llama-3.1-70B-Instruct-GGUF 29,307,734,592 1 entries IQ3_XS bartowski/Meta-Llama-3.1-70B-Instruct-GGUF 30,912,055,872 1 entries Q3_K_S bartowski/Meta-Llama-3.1-70B-Instruct-GGUF 31,937,038,912 1 entries IQ3_M bartowski/Meta-Llama-3.1-70B-Instruct-GGUF 34,267,499,072 1 entries Q3_K_M bartowski/Meta-Llama-3.1-70B-Instruct-GGUF 37,140,597,312 1 entries Q3_K_L bartowski/Meta-Llama-3.1-70B-Instruct-GGUF 37,902,666,304 1 entries IQ4_XS bartowski/Meta-Llama-3.1-70B-Instruct-GGUF 38,059,936,320 1 entries Q3_K_XL bartowski/Meta-Llama-3.1-70B-Instruct-GGUF 40,347,224,640 1 entries Q4_K_S bartowski/Meta-Llama-3.1-70B-Instruct-GGUF 42,520,398,400 1 entries Q4_K_M bartowski/Meta-Llama-3.1-70B-Instruct-GGUF 43,300,194,880 1 entries Q4_K_L bartowski/Meta-Llama-3.1-70B-Instruct-GGUF 48,657,451,584 1 entries Q5_K_S bartowski/Meta-Llama-3.1-70B-Instruct-GGUF 49,949,821,696 2 entries Q5_K_M bartowski/Meta-Llama-3.1-70B-Instruct-GGUF 50,598,284,032 2 entries Q5_K_L bartowski/Meta-Llama-3.1-70B-Instruct-GGUF 57,888,148,224 2 entries Q6_K bartowski/Meta-Llama-3.1-70B-Instruct-GGUF 58,397,068,032 2 entries Q6_K_L bartowski/Meta-Llama-3.1-70B-Instruct-GGUF 74,975,054,592 2 entries Q8_0 - Hidden size
- KV heads
- Layers
- Licence
- Parameters
- Parameters by type
- Publisher
- Released
- Task