Instructions to use GreatCaptainNemo/ProLLaMA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use GreatCaptainNemo/ProLLaMA with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="GreatCaptainNemo/ProLLaMA")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("GreatCaptainNemo/ProLLaMA")
model = AutoModelForCausalLM.from_pretrained("GreatCaptainNemo/ProLLaMA")

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use GreatCaptainNemo/ProLLaMA with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "GreatCaptainNemo/ProLLaMA"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "GreatCaptainNemo/ProLLaMA",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/GreatCaptainNemo/ProLLaMA

SGLang

How to use GreatCaptainNemo/ProLLaMA with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "GreatCaptainNemo/ProLLaMA" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "GreatCaptainNemo/ProLLaMA",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "GreatCaptainNemo/ProLLaMA" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "GreatCaptainNemo/ProLLaMA",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use GreatCaptainNemo/ProLLaMA with Docker Model Runner:
```
docker model run hf.co/GreatCaptainNemo/ProLLaMA
```

Need Help on Dataset preparation for Continual Learning

by surajk02 - opened Jun 21, 2024

Discussion

surajk02

Jun 21, 2024

Trying a similar approach from prollama where you had implemented a two step training and tuning an llm.

step 1 - continual learning on top of a pretrained base model on your dataset using the protein sequences
step 2 - finetuning using instructions for the domain specific adaption

how can we prepare data X/Y for the continual leraning in case of sequences ?
is it like two parts of a same sequence will be considered X and Y but if we are doing these the sequence can be split into two at multiple places ?

Please help with how you prepared data for doing this continual learning on those sequences

GreatCaptainNemo

Owner Jun 23, 2024

Hello, you can refer to "next token prediction" or "causal language modeling" for answers.
Briefly, one sequence, which consists of serveral tokens, serves as both X and Y. We predict the next token for each token. So we don't have to care about how to split one sequence.
Related codes have been integrated into various open-source libraries, such as huggingface.transformers.

GreatCaptainNemo

Owner Jun 27, 2024

We have released our training codes here. Hope it is helpful.

surajk02

Jul 22, 2024

Thanks a lot for this

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment