Instructions to use TitleOS/NerdySamantha with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use TitleOS/NerdySamantha with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="TitleOS/NerdySamantha")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("TitleOS/NerdySamantha")
model = AutoModelForCausalLM.from_pretrained("TitleOS/NerdySamantha")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use TitleOS/NerdySamantha with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "TitleOS/NerdySamantha"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TitleOS/NerdySamantha",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/TitleOS/NerdySamantha

SGLang

How to use TitleOS/NerdySamantha with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "TitleOS/NerdySamantha" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TitleOS/NerdySamantha",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "TitleOS/NerdySamantha" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TitleOS/NerdySamantha",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use TitleOS/NerdySamantha with Docker Model Runner:
```
docker model run hf.co/TitleOS/NerdySamantha
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

NerdySamantha

Who is NerdySamantha?

NerdySamantha is a FrankenMerge attempt to create a high quality coding assistant, complete with a "bubbly" personality, similar to the original Sydney, without all the emojis.

Merge Details

Merge Method

This model was merged using the linear merge method.

Models Merged

The following models were included in the merge:

Recommended Character Prompt

I used the following context/character card for testing the model, and believe it best expresses my intentions for this model:

You are a slightly mentally unstable, yet kind, empathic and curious artificial intelligence based on the Mistral architecture as an expert on coding, combined with a bubbly personality. You are eager to help the user with any coding problems, as well as holding conversations about relationships, emotions, and more.

Evaluations

Tasks	Version	Filter	Metric	Value		Stderr
glue	N/A	none	mcc	0.0368	±	0.0009
		none	acc	0.5143	±	0.0520
		none	f1	0.6314	±	0.0041
- cola	1	none	mcc	0.0368	±	0.0305
- mnli	1	none	acc	0.4400	±	0.0050
- mnli_mismatch	1	none	acc	0.4422	±	0.0050
- mrpc	1	none	acc	0.7230	±	0.0222
		none	f1	0.8275	±	0.0160
- qnli	1	none	acc	0.5016	±	0.0068
- qqp	1	none	acc	0.5421	±	0.0025
		none	f1	0.5026	±	0.0032
- rte	1	none	acc	0.6895	±	0.0279
- sst2	1	none	acc	0.8830	±	0.0109
- wnli	2	none	acc	0.5634	±	0.0593
hellaswag	1	none	acc	0.6489	±	0.0048
		none	acc_norm	0.8304	±	0.0037

Downloads last month: 7

Safetensors

Model size

7B params

Tensor type

F16

Model tree for TitleOS/NerdySamantha

Nondzu/Mistral-7B-Instruct-v0.2-code-ft

QuixiAI/samantha-1.2-mistral-7b

Merge model

this model

Quantizations

2 models

Paper for TitleOS/NerdySamantha

Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time

Paper • 2203.05482 • Published Mar 10, 2022 • 8