HebArabNlpProject/Hebatron · AWQ or GPTQ (INT4) Quantization Planned?

AWQ or GPTQ (INT4) Quantization Planned?

by alonsoko - opened 1 day ago

Are you planning on releasing an AWQ or GPTQ INT4 quantizations of the model?
So I can serve it on VLLM.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment