Instructions to use Envoid/Llama-3-8B-Instruct-DADA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Envoid/Llama-3-8B-Instruct-DADA with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Envoid/Llama-3-8B-Instruct-DADA")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Envoid/Llama-3-8B-Instruct-DADA")
model = AutoModelForCausalLM.from_pretrained("Envoid/Llama-3-8B-Instruct-DADA")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Envoid/Llama-3-8B-Instruct-DADA with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Envoid/Llama-3-8B-Instruct-DADA"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Envoid/Llama-3-8B-Instruct-DADA",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Envoid/Llama-3-8B-Instruct-DADA

SGLang

How to use Envoid/Llama-3-8B-Instruct-DADA with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Envoid/Llama-3-8B-Instruct-DADA" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Envoid/Llama-3-8B-Instruct-DADA",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Envoid/Llama-3-8B-Instruct-DADA" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Envoid/Llama-3-8B-Instruct-DADA",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Envoid/Llama-3-8B-Instruct-DADA with Docker Model Runner:
```
docker model run hf.co/Envoid/Llama-3-8B-Instruct-DADA
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Llama-3-8B-Instruct-DADA

Warning: This model is experimental and thus potentially unpredictable.

This model employs the same strategy as Mixtral Instruct ITR DADA

I trained Llama-3-8B-Instruct on the Alpaca-DADA dataset for 10 epochs at 1e-6 learning rate. I then did a 50/50 SLERP merge of the resulting model back onto Llama-3-8B-Instruct

This model may require custom stopping strings to tame due to current issues surrounding Llama-3 EOS tokens and various back-ends. It certainly gives some interesting answers using an assistant template/card in SillyTavern, though.

The below answer is one of the more interesting answers I've gotten out of an LLM on the same query, although there was an indentiation error (indicated by the red circle)

Training was done using qlora-pipe

GGUFs care of Quant Cartel

exl2 RPCAL care of Qaunt Cartel

Downloads last month: 2

Safetensors

Model size

8B params

Tensor type

F16

Model tree for Envoid/Llama-3-8B-Instruct-DADA

Quantizations

2 models