SCB 10X · 8B · Dense
Thai/English bilingual chat model built on Llama 3.1
Use Cases
| Quant | Bits | VRAM | Quality | Status |
|---|---|---|---|---|
| Q2_K | 2 | 3.1 GB | low | — |
| Q3_K_M | 3 | 4.1 GB | moderate | — |
| Q4_K_M | 4 | 4.6 GB | good | — |
| Q5_K_M | 5 | 5.6 GB | good | — |
| Q6_K | 6 | 6.6 GB | excellent | — |
| Q8_0 | 8 | 8.7 GB | excellent | — |
| F16 | 16 | 16.9 GB | lossless | — |
GPUs that run Typhoon 2 8B
Typhoon 2 8B needs about 4.6 GB of VRAM at Q4_K_M, so it runs comfortably on a GPU with 6 GB of VRAM or more (157 of the GPUs we track). Common cards, smallest first:
| VRAM | GPUs |
|---|---|
| 6 GB | RTX 2060 , Arc A380 , RX 5600 XT |
| 8 GB | RTX 5060 Ti , RX 7600 , Arc A750 |
| 10 GB | RTX 3080 , RX 6700M |
| 11 GB | RTX 2080 Ti , GTX 1080 Ti |
About this model
Llama3.1-Typhoon2-8B: Thai Large Language Model (Instruct) - Q4_K_M quantized
Llama3.1-Typhoon2-8B-instruct is a instruct Thai 🇹🇭 large language model with 8 billion parameters, and it is based on Llama3.1-8B.
By using this model, you agree to the OpenTyphoon Terms and Conditions and acknowledge the Privacy Notice: https://opentyphoon.ai/tac · https://opentyphoon.ai/privacy
*To acknowledge Meta’s effort in creating the foundation model and to comply with the license, we explicitly include “llama-3.1” in the model name.
Run
ollama run scb10x/llama3.1-typhoon2-8b-instruct
Performance
Instruction-Following & Function Call Performance
Specific Domain Performance (Math & Coding)
Long Context Performance
Detail Performance
| Model | IFEval - TH | IFEval - EN | MT-Bench TH | MT-Bench EN | Thai Code-Switching(t=0.7) | Thai Code-Switching(t=1.0) | FunctionCall-TH | FunctionCall-EN | GSM8K-TH | GSM8K-EN | MATH-TH | MATH-EN | HumanEval-TH | HumanEval-EN | MBPP-TH | MBPP-EN |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Llama3.1 8B Instruct | 58.04% | 77.64% | 5.109 | 8.118 | 93% | 11.2% | 36.92% | 66.06% | 45.18% | 62.4% | 24.42% | 48% | 51.8% | 67.7% | 64.6% | 66.9% |
| Typhoon2 Llama3 8B Instruct | 72.60% | 76.43% | 5.7417 | 7.584 | 98.8% | 98% | 75.12% | 79.08% | 71.72% | 81.0% | 38.48% | 49.04% | 58.5% | 68.9% | 60.8% | 63.0% |