Skip to content
Meta

Llama 3.1 70B Instruct

Maker’s page
Check price at Amazon for Meta Llama 3.1 70B Instruct (affiliate link, opens in new tab)
Weights at 16-bit
140 GB
Weights at 8-bit (Q8_0)
74 GB
Weights at 4-bit (Q4_K_M)
42 GB
Smallest card at 4-bit
RTX 6000 Ada Generation · 48 GB
Parameters
Layers
KV heads
Licence

Specs

Attention heads
Downloads (30 days)
Family
GGUF downloads
repobytesfilesquant
bartowski/Meta-Llama-3.1-70B-Instruct-GGUF16,751,200,8321 entriesIQ1_M
bartowski/Meta-Llama-3.1-70B-Instruct-GGUF19,097,389,6321 entriesIQ2_XXS
bartowski/Meta-Llama-3.1-70B-Instruct-GGUF21,142,112,8321 entriesIQ2_XS
bartowski/Meta-Llama-3.1-70B-Instruct-GGUF22,242,347,5841 entriesIQ2_S
bartowski/Meta-Llama-3.1-70B-Instruct-GGUF24,119,298,6241 entriesIQ2_M
bartowski/Meta-Llama-3.1-70B-Instruct-GGUF26,375,113,2801 entriesQ2_K
bartowski/Meta-Llama-3.1-70B-Instruct-GGUF27,401,161,2801 entriesQ2_K_L
bartowski/Meta-Llama-3.1-70B-Instruct-GGUF29,307,734,5921 entriesIQ3_XS
bartowski/Meta-Llama-3.1-70B-Instruct-GGUF30,912,055,8721 entriesQ3_K_S
bartowski/Meta-Llama-3.1-70B-Instruct-GGUF31,937,038,9121 entriesIQ3_M
bartowski/Meta-Llama-3.1-70B-Instruct-GGUF34,267,499,0721 entriesQ3_K_M
bartowski/Meta-Llama-3.1-70B-Instruct-GGUF37,140,597,3121 entriesQ3_K_L
bartowski/Meta-Llama-3.1-70B-Instruct-GGUF37,902,666,3041 entriesIQ4_XS
bartowski/Meta-Llama-3.1-70B-Instruct-GGUF38,059,936,3201 entriesQ3_K_XL
bartowski/Meta-Llama-3.1-70B-Instruct-GGUF40,347,224,6401 entriesQ4_K_S
bartowski/Meta-Llama-3.1-70B-Instruct-GGUF42,520,398,4001 entriesQ4_K_M
bartowski/Meta-Llama-3.1-70B-Instruct-GGUF43,300,194,8801 entriesQ4_K_L
bartowski/Meta-Llama-3.1-70B-Instruct-GGUF48,657,451,5841 entriesQ5_K_S
bartowski/Meta-Llama-3.1-70B-Instruct-GGUF49,949,821,6962 entriesQ5_K_M
bartowski/Meta-Llama-3.1-70B-Instruct-GGUF50,598,284,0322 entriesQ5_K_L
bartowski/Meta-Llama-3.1-70B-Instruct-GGUF57,888,148,2242 entriesQ6_K
bartowski/Meta-Llama-3.1-70B-Instruct-GGUF58,397,068,0322 entriesQ6_K_L
bartowski/Meta-Llama-3.1-70B-Instruct-GGUF74,975,054,5922 entriesQ8_0
Hidden size
KV heads
Layers
Licence
Parameters
Parameters by type
Publisher
Released
Task

More from Meta

Across the range 5 models by family