Instructions to use GreatCaptainNemo/ProLLaMA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use GreatCaptainNemo/ProLLaMA with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="GreatCaptainNemo/ProLLaMA")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("GreatCaptainNemo/ProLLaMA")
model = AutoModelForCausalLM.from_pretrained("GreatCaptainNemo/ProLLaMA")

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use GreatCaptainNemo/ProLLaMA with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "GreatCaptainNemo/ProLLaMA"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "GreatCaptainNemo/ProLLaMA",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/GreatCaptainNemo/ProLLaMA

SGLang

How to use GreatCaptainNemo/ProLLaMA with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "GreatCaptainNemo/ProLLaMA" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "GreatCaptainNemo/ProLLaMA",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "GreatCaptainNemo/ProLLaMA" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "GreatCaptainNemo/ProLLaMA",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use GreatCaptainNemo/ProLLaMA with Docker Model Runner:
```
docker model run hf.co/GreatCaptainNemo/ProLLaMA
```

Open sourcing the instruction fine-tuning dataset and help on tokenizer

by aayush14 - opened Jun 7, 2024

Discussion

aayush14

Jun 7, 2024

Hello,

Firstly, thank you very much for sharing this work. I wanted to ask if you plan on open-sourcing the dataset used in stage 2, for the instruction fine-tuning? I think it would be very helpful for the community.

Also, did you create a tokenizer from scratch or built upon the base tokenizer? What corpus did you use for it and how did you choose the parameters and which special tokens should be added?

Best Regards

GreatCaptainNemo

Owner Jun 8, 2024

•

edited Jun 8, 2024

Thanks for your recognition!
We have open-sourced the instruction dataset.
As for tokenizer, we use the original tokenizer of Llama2 without any modification, at least in this version.
Any notification of updates will be posted on my github repo.

Best Regards

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment