problem while loading fintuned startcoder2 model with base model
I have finetuned bigcode/starcoder2-15b and generated adapter weights. Now while trying to load adapter model with base model I'm getting the following error.
RuntimeError: Error(s) in loading state_dict for PeftModelForCausalLM:
size mismatch for base_model.model.model.embed_tokens.modules_to_save.adapter_model.weight: copying a param with shape torch.Size([49156, 6144]) from checkpoint, the shape in current model is torch.Size([49152, 6144]).
size mismatch for base_model.model.lm_head.modules_to_save.adapter_model.weight: copying a param with shape torch.Size([49156, 6144]) from checkpoint, the shape in current model is torch.Size([49152, 6144]).
I have used https://github.com/OpenAccess-AI-Collective/axolotl/tree/main for training and i have put following params in .yml
lora_modules_to_save:
- embed_tokens
- lm_head
using the following code to load the model:
def load_model_and_tokenizer(adapter_path, bnb_config=None):
base_model = AutoModelForCausalLM.from_pretrained(
"bigcode/starcoder2-15b",
# load_in_4bit=True,
torch_dtype=torch.bfloat16,
# quantization_config=bnb_config,
device_map='auto'
)
tokenizer = AutoTokenizer.from_pretrained("bigcode/starcoder2-15b")
model = PeftModel.from_pretrained(
model=base_model,
model_id=adapter_path,
adapter_name="adapter_model",
torch_dtype=torch.bfloat16,
is_trainable= False)
tokenizer.pad_token = tokenizer.eos_token
model.eval()
return model, tokenizer