mallow-p1

datasets:

• nvidia/OpenCodeInstruct language:
• en base_model:
• Salesforce/codegen-350M-mono library_name: peft tags:
• python
• code
• instruction-tuning
• lora
• transformers

---

mallow-p1: Instruction-Tuned CodeGen-350M for Python

This repository contains a LoRA (Low-Rank Adaptation) adapter for the Salesforce/codegen-350M-mono model. It has been fine-tuned on the high-quality nvidia/OpenCodeInstruct dataset to enhance its ability to generate Python code based on natural language instructions.

Model Details

Model Description

mallow-p1 is not a standalone model but a lightweight adapter that modifies the behavior of the Salesforce/codegen-350M-mono base model. By fine-tuning with LoRA on the OpenCodeInstruct dataset, this model is specifically optimized to understand and execute coding tasks described in English, acting as a capable instruction-following code assistant.

The use of LoRA makes the adapter highly efficient, with a very small file size, making it easy to share and use.

• Developed by: Maaz Waheed
• Model type: Causal language model (Decoder-only Transformer)
• Language(s) (NLP): English, Python
• License: Apache-2.0 (Inherited from the base model)
• Finetuned from model: Salesforce/codegen-350M-mono
• Finetuning Dataset: nvidia/OpenCodeInstruct

Model Sources

• Repository: https://github.com/42Wor
• Base Model: Salesforce/codegen-350M-mono
• Dataset: nvidia/OpenCodeInstruct

Uses

Direct Use

This model is intended for generating Python code snippets from natural language prompts. It excels at:

• Function & Class Generation: Create boilerplate for functions and classes based on a description.
• Code Scaffolding: Quickly generate scripts or project file structures.
• Algorithmic Tasks: Provide a description of an algorithm, and the model will attempt to implement it.
• Learning and Prototyping: Use it as a tool to quickly explore different ways to write a piece of code.

Out-of-Scope Use

This is an assistive tool and is not intended for production-critical systems without human oversight.

• Security Vulnerabilities: The generated code is not guaranteed to be secure and must be audited.
• Bugs and Errors: The model can generate code that is syntactically correct but logically flawed or inefficient. Always test the generated code.
• General Chat: This is not a conversational chatbot. It is optimized for instruction-following related to code.

Bias, Risks, and Limitations

The model inherits biases from its training data, which includes a vast amount of public code. It may generate code that reflects suboptimal patterns or vulnerabilities present in that data. The model does not have an understanding of the consequences of the code it writes.

Recommendations

Always have a human developer review, test, and validate the code generated by this model before any production use. Be specific and clear in your instructions to get the best results.

How to Get Started with the Model

First, install the necessary libraries:

pip install torch transformers accelerate peft bitsandbytes

Then, use the following Python script to load the base model and apply the LoRA adapter for inference.

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

# Define the model identifiers
base_model_id = "Salesforce/codegen-350M-mono"
adapter_id = "Maazwaheed/mallow-p1" # <-- Replace with your Hugging Face repo ID
adapter_subfolder = "mallow_codegen_lora_adapter_3M" # <-- Add the subfolder name

# --- 1. Load the base model and tokenizer ---
print(f"Loading base model: {base_model_id}")
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.float16, # Use float16 for memory efficiency
    device_map="auto"         # Automatically map model to GPU if available
)

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
# Set a padding token if one doesn't exist
if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token

# --- 2. Load and merge the LoRA adapter ---
print(f"Loading and merging LoRA adapter: {adapter_id}/{adapter_subfolder}") # Update print message
model = PeftModel.from_pretrained(base_model, adapter_id, subfolder=adapter_subfolder) # Add subfolder argument
model = model.merge_and_unload() # Merge weights for faster inference
print("✅ Adapter merged.")

model.eval()

# --- 3. Generate a response ---
prompt = "### Instruction:\nWrite a Python function to find the factorial of a number using recursion.\n\n### Response:\n"
input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to(model.device)

print("\nGenerating response...")
with torch.no_grad():
    outputs = model.generate(
        input_ids,
        max_length=250,
        num_return_sequences=1,
        do_sample=True,
        top_k=50,
        top_p=0.95,
        temperature=0.7,
        pad_token_id=tokenizer.eos_token_id
    )

generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)

print("\n--- Generated Response ---")
print(generated_text)
print("--------------------------")

Training Details

Training Data

The model was fine-tuned on the nvidia/OpenCodeInstruct dataset. This dataset contains high-quality instruction-response pairs for code generation, making it ideal for teaching a model to follow programming instructions.

Training Procedure

The model was trained using the peft library from Hugging Face, employing the Low-Rank Adaptation (LoRA) technique.

Preprocessing

The training data was formatted into a prompt template to facilitate instruction-following. The template used is:

### Instruction:
{instruction}

### Response:
{response}

Citation

If you use this model, please consider citing the original works that made it possible.

CodeGen:

@inproceedings{Nijkamp2022A,
  author    = {Erik Nijkamp and
               Bo Pang and
               Hiroaki Hayashi and
               Lifu Tu and
               Huan Wang and
               Yingbo Zhou and
               Silvio Savarese and
               Caiming Xiong},
  title     = {A Conversational Paradigm for Program Synthesis},
  booktitle = {Arxiv},
  year      = {2022}
}

LoRA:

@article{hu2021lora,
  title={LoRA: Low-Rank Adaptation of Large Language Models},
  author={Hu, Edward J and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu},
  journal={arXiv preprint arXiv:2106.09685},
  year={2021}
}

Model Card Contact

Maaz Waheed (GitHub: 42Wor)