wizardlm

wizardlm

80.1K Downloads Updated 1 year ago

General use model based on Llama 2.

Models

Name

73 models

Size

Context

Input

wizardlm:7b-q2_K

2.8GB · 2K context window · Text · 1 year ago

wizardlm:7b-q2_K

2.8GB

2K

Text

wizardlm:7b-q3_K_S

2.9GB · 2K context window · Text · 1 year ago

wizardlm:7b-q3_K_S

2.9GB

2K

Text

wizardlm:7b-q3_K_M

3.3GB · 2K context window · Text · 1 year ago

wizardlm:7b-q3_K_M

3.3GB

2K

Text

wizardlm:7b-q3_K_L

3.6GB · 2K context window · Text · 1 year ago

wizardlm:7b-q3_K_L

3.6GB

2K

Text

wizardlm:7b-q4_0

3.8GB · 2K context window · Text · 1 year ago

wizardlm:7b-q4_0

3.8GB

2K

Text

wizardlm:7b-q4_1

4.2GB · 2K context window · Text · 1 year ago

wizardlm:7b-q4_1

4.2GB

2K

Text

wizardlm:7b-q4_K_S

3.9GB · 2K context window · Text · 1 year ago

wizardlm:7b-q4_K_S

3.9GB

2K

Text

wizardlm:7b-q4_K_M

4.1GB · 2K context window · Text · 1 year ago

wizardlm:7b-q4_K_M

4.1GB

2K

Text

wizardlm:7b-q5_0

4.7GB · 2K context window · Text · 1 year ago

wizardlm:7b-q5_0

4.7GB

2K

Text

wizardlm:7b-q5_1

5.1GB · 2K context window · Text · 1 year ago

wizardlm:7b-q5_1

5.1GB

2K

Text

wizardlm:7b-q5_K_S

4.7GB · 2K context window · Text · 1 year ago

wizardlm:7b-q5_K_S

4.7GB

2K

Text

wizardlm:7b-q5_K_M

4.8GB · 2K context window · Text · 1 year ago

wizardlm:7b-q5_K_M

4.8GB

2K

Text

wizardlm:7b-q6_K

5.5GB · 2K context window · Text · 1 year ago

wizardlm:7b-q6_K

5.5GB

2K

Text

wizardlm:7b-q8_0

7.2GB · 2K context window · Text · 1 year ago

wizardlm:7b-q8_0

7.2GB

2K

Text

wizardlm:7b-fp16

13GB · 2K context window · Text · 1 year ago

wizardlm:7b-fp16

13GB

2K

Text

wizardlm:13b-llama2-q2_K

5.4GB · 4K context window · Text · 1 year ago

wizardlm:13b-llama2-q2_K

5.4GB

4K

Text

wizardlm:13b-llama2-q3_K_S

5.7GB · 4K context window · Text · 1 year ago

wizardlm:13b-llama2-q3_K_S

5.7GB

4K

Text

wizardlm:13b-llama2-q3_K_M

6.3GB · 4K context window · Text · 1 year ago

wizardlm:13b-llama2-q3_K_M

6.3GB

4K

Text

wizardlm:13b-llama2-q3_K_L

6.9GB · 4K context window · Text · 1 year ago

wizardlm:13b-llama2-q3_K_L

6.9GB

4K

Text

wizardlm:13b-llama2-q4_0

7.4GB · 4K context window · Text · 1 year ago

wizardlm:13b-llama2-q4_0

7.4GB

4K

Text

wizardlm:13b-llama2-q4_1

8.2GB · 4K context window · Text · 1 year ago

wizardlm:13b-llama2-q4_1

8.2GB

4K

Text

wizardlm:13b-llama2-q4_K_S

7.4GB · 4K context window · Text · 1 year ago

wizardlm:13b-llama2-q4_K_S

7.4GB

4K

Text

wizardlm:13b-llama2-q4_K_M

7.9GB · 4K context window · Text · 1 year ago

wizardlm:13b-llama2-q4_K_M

7.9GB

4K

Text

wizardlm:13b-llama2-q5_0

9.0GB · 4K context window · Text · 1 year ago

wizardlm:13b-llama2-q5_0

9.0GB

4K

Text

wizardlm:13b-llama2-q5_1

9.8GB · 4K context window · Text · 1 year ago

wizardlm:13b-llama2-q5_1

9.8GB

4K

Text

wizardlm:13b-llama2-q5_K_S

9.0GB · 4K context window · Text · 1 year ago

wizardlm:13b-llama2-q5_K_S

9.0GB

4K

Text

wizardlm:13b-llama2-q5_K_M

9.2GB · 4K context window · Text · 1 year ago

wizardlm:13b-llama2-q5_K_M

9.2GB

4K

Text

wizardlm:13b-llama2-q6_K

11GB · 4K context window · Text · 1 year ago

wizardlm:13b-llama2-q6_K

11GB

4K

Text

wizardlm:13b-llama2-q8_0

14GB · 4K context window · Text · 1 year ago

wizardlm:13b-llama2-q8_0

14GB

4K

Text

wizardlm:13b-llama2-fp16

26GB · 4K context window · Text · 1 year ago

wizardlm:13b-llama2-fp16

26GB

4K

Text

wizardlm:13b-q2_K

5.4GB · 2K context window · Text · 1 year ago

wizardlm:13b-q2_K

5.4GB

2K

Text

wizardlm:13b-q3_K_S

5.7GB · 2K context window · Text · 1 year ago

wizardlm:13b-q3_K_S

5.7GB

2K

Text

wizardlm:13b-q3_K_M

6.3GB · 2K context window · Text · 1 year ago

wizardlm:13b-q3_K_M

6.3GB

2K

Text

wizardlm:13b-q3_K_L

6.9GB · 2K context window · Text · 1 year ago

wizardlm:13b-q3_K_L

6.9GB

2K

Text

wizardlm:13b-q4_0

7.4GB · 2K context window · Text · 1 year ago

wizardlm:13b-q4_0

7.4GB

2K

Text

wizardlm:13b-q4_1

8.2GB · 2K context window · Text · 1 year ago

wizardlm:13b-q4_1

8.2GB

2K

Text

wizardlm:13b-q4_K_S

7.4GB · 2K context window · Text · 1 year ago

wizardlm:13b-q4_K_S

7.4GB

2K

Text

wizardlm:13b-q4_K_M

7.9GB · 2K context window · Text · 1 year ago

wizardlm:13b-q4_K_M

7.9GB

2K

Text

wizardlm:13b-q5_0

9.0GB · 2K context window · Text · 1 year ago

wizardlm:13b-q5_0

9.0GB

2K

Text

wizardlm:13b-q5_1

9.8GB · 2K context window · Text · 1 year ago

wizardlm:13b-q5_1

9.8GB

2K

Text

wizardlm:13b-q5_K_S

9.0GB · 2K context window · Text · 1 year ago

wizardlm:13b-q5_K_S

9.0GB

2K

Text

wizardlm:13b-q5_K_M

9.2GB · 2K context window · Text · 1 year ago

wizardlm:13b-q5_K_M

9.2GB

2K

Text

wizardlm:13b-q6_K

11GB · 2K context window · Text · 1 year ago

wizardlm:13b-q6_K

11GB

2K

Text

wizardlm:13b-q8_0

14GB · 2K context window · Text · 1 year ago

wizardlm:13b-q8_0

14GB

2K

Text

wizardlm:13b-fp16

26GB · 2K context window · Text · 1 year ago

wizardlm:13b-fp16

26GB

2K

Text

wizardlm:30b-q2_K

14GB · 2K context window · Text · 1 year ago

wizardlm:30b-q2_K

14GB

2K

Text

wizardlm:30b-q3_K_S

14GB · 2K context window · Text · 1 year ago

wizardlm:30b-q3_K_S

14GB

2K

Text

wizardlm:30b-q3_K_M

16GB · 2K context window · Text · 1 year ago

wizardlm:30b-q3_K_M

16GB

2K

Text

wizardlm:30b-q3_K_L

17GB · 2K context window · Text · 1 year ago

wizardlm:30b-q3_K_L

17GB

2K

Text

wizardlm:30b-q4_0

18GB · 2K context window · Text · 1 year ago

wizardlm:30b-q4_0

18GB

2K

Text

wizardlm:30b-q4_1

20GB · 2K context window · Text · 1 year ago

wizardlm:30b-q4_1

20GB

2K

Text

wizardlm:30b-q4_K_S

18GB · 2K context window · Text · 1 year ago

wizardlm:30b-q4_K_S

18GB

2K

Text

wizardlm:30b-q4_K_M

20GB · 2K context window · Text · 1 year ago

wizardlm:30b-q4_K_M

20GB

2K

Text

wizardlm:30b-q5_0

22GB · 2K context window · Text · 1 year ago

wizardlm:30b-q5_0

22GB

2K

Text

wizardlm:30b-q5_1

24GB · 2K context window · Text · 1 year ago

wizardlm:30b-q5_1

24GB

2K

Text

wizardlm:30b-q5_K_S

22GB · 2K context window · Text · 1 year ago

wizardlm:30b-q5_K_S

22GB

2K

Text

wizardlm:30b-q5_K_M

23GB · 2K context window · Text · 1 year ago

wizardlm:30b-q5_K_M

23GB

2K

Text

wizardlm:30b-q6_K

27GB · 2K context window · Text · 1 year ago

wizardlm:30b-q6_K

27GB

2K

Text

wizardlm:30b-q8_0

35GB · 2K context window · Text · 1 year ago

wizardlm:30b-q8_0

35GB

2K

Text

wizardlm:30b-fp16

65GB · 2K context window · Text · 1 year ago

wizardlm:30b-fp16

65GB

2K

Text

wizardlm:70b-llama2-q2_K

29GB · 4K context window · Text · 1 year ago

wizardlm:70b-llama2-q2_K

29GB

4K

Text

wizardlm:70b-llama2-q3_K_S

30GB · 4K context window · Text · 1 year ago

wizardlm:70b-llama2-q3_K_S

30GB

4K

Text

wizardlm:70b-llama2-q3_K_M

33GB · 4K context window · Text · 1 year ago

wizardlm:70b-llama2-q3_K_M

33GB

4K

Text

wizardlm:70b-llama2-q3_K_L

36GB · 4K context window · Text · 1 year ago

wizardlm:70b-llama2-q3_K_L

36GB

4K

Text

wizardlm:70b-llama2-q4_0

39GB · 4K context window · Text · 1 year ago

wizardlm:70b-llama2-q4_0

39GB

4K

Text

wizardlm:70b-llama2-q4_1

43GB · 4K context window · Text · 1 year ago

wizardlm:70b-llama2-q4_1

43GB

4K

Text

wizardlm:70b-llama2-q4_K_S

39GB · 4K context window · Text · 1 year ago

wizardlm:70b-llama2-q4_K_S

39GB

4K

Text

wizardlm:70b-llama2-q4_K_M

41GB · 4K context window · Text · 1 year ago

wizardlm:70b-llama2-q4_K_M

41GB

4K

Text

wizardlm:70b-llama2-q5_0

47GB · 4K context window · Text · 1 year ago

wizardlm:70b-llama2-q5_0

47GB

4K

Text

wizardlm:70b-llama2-q5_K_S

47GB · 4K context window · Text · 1 year ago

wizardlm:70b-llama2-q5_K_S

47GB

4K

Text

wizardlm:70b-llama2-q5_K_M

49GB · 4K context window · Text · 1 year ago

wizardlm:70b-llama2-q5_K_M

49GB

4K

Text

wizardlm:70b-llama2-q6_K

57GB · 4K context window · Text · 1 year ago

wizardlm:70b-llama2-q6_K

57GB

4K

Text

wizardlm:70b-llama2-q8_0

73GB · 4K context window · Text · 1 year ago

wizardlm:70b-llama2-q8_0

73GB

4K

Text

Readme

WizardLM is a 70B parameter model based on Llama 2 trained by WizardLM.

Get started with WizardLM

The model used in the example below is the WizardLM model, with 70b parameters, which is a general-use model.

API

Start Ollama server (Run ollama serve)
Run the model

curl -X POST http://localhost:11434/api/generate -d '{
  "model": "wizardlm:70b-llama2-q4_0",
  "prompt":"Why is the sky blue?"
 }'

CLI

Install Ollama
Open the terminal and run ollama run wizardlm:70b-llama2-q4_0

Note: The ollama run command performs an ollama pull if the model is not already downloaded. To download the model without running it, use ollama pull wizardlm:70b-llama2-q4_0

Memory requirements

70b models generally require at least 64GB of RAM

If you run into issues with higher quantization levels, try using the q4 model or shut down any other programs that are using a lot of memory.

Model variants

By default, Ollama uses 4-bit quantization. To try other quantization levels, please try the other tags. The number after the q represents the number of bits used for quantization (i.e. q4 means 4-bit quantization). The higher the number, the more accurate the model is, but the slower it runs, and the more memory it requires.

Model source

WizardLM source on Ollama

70b parameters source: The Bloke

70b parameters original source: WizardLM

<img src="https://user-images.githubusercontent.com/633681/263121166-8b16834a-1da2-4a68-8aac-13d19775e7d7.png" style="max-width:60%" />

WizardLM is a 70B parameter model based on Llama 2 trained by WizardLM.

## Get started with WizardLM

The model used in the example below is the WizardLM model, with 70b parameters, which is a general-use model.

### API

1. Start Ollama server (Run `ollama serve`)
2. Run the model

```bash
curl -X POST http://localhost:11434/api/generate -d '{
  "model": "wizardlm:70b-llama2-q4_0",
  "prompt":"Why is the sky blue?"
 }'
  ```

### CLI

1. Install Ollama
2. Open the terminal and run `ollama run wizardlm:70b-llama2-q4_0`

Note: The `ollama run` command performs an `ollama pull` if the model is not already downloaded. To download the model without running it, use `ollama pull wizardlm:70b-llama2-q4_0`

## Memory requirements

- 70b models generally require at least 64GB of RAM

If you run into issues with higher quantization levels, try using the q4 model or shut down any other programs that are using a lot of memory.

## Model variants

By default, Ollama uses 4-bit quantization. To try other quantization levels, please try the other tags. The number after the q represents the number of bits used for quantization (i.e. q4 means 4-bit quantization). The higher the number, the more accurate the model is, but the slower it runs, and the more memory it requires.

## Model source

**WizardLM source on Ollama**

70b parameters source:
 [The Bloke](https://huggingface.co/TheBloke/WizardLM-70B-V1.0-GGML)

70b parameters original source:
 [WizardLM](https://huggingface.co/WizardLM/WizardLM-70B-V1.0)

Paste, drop or click to upload images (.png, .jpeg, .jpg, .svg, .gif)