FDA-guidance-zephyr-7b-beta-PEFT

PEFT Model Trained on FDA Guidances

This is a PEFT adapter model trained on a synthetic dataset derived from over 700 pharma-related FDA guidances.

Usage

%pip install accelerate peft

from transformers import AutoModelForCausalLM, AutoTokenizer

model_path = "HuggingFaceH4/zephyr-7b-beta"
peft_model_path = "yitzashapiro/FDA-guidance-zephyr-7b-beta-PEFT"

tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(
    model_path,
    device_map="auto",
    torch_dtype='auto'
).eval()
model.load_adapter(peft_model_path)

messages = [
    {"role": "user", "content": "What's the best way to submit an NDA to the FDA?"}
]

input_ids = tokenizer.apply_chat_template(conversation=messages, max_length=45, tokenize=True, add_generation_prompt=True, return_tensors='pt')
pad_token_id = tokenizer.pad_token_id if tokenizer.pad_token_id is not None else 0
attention_mask = (input_ids != pad_token_id).long()

output_ids = model.generate(
    input_ids.to(model.device),
    max_length=2048,
    max_new_tokens=500,
    attention_mask=attention_mask.to(model.device)
)


response = tokenizer.decode(output_ids[0][input_ids.shape[1]:], skip_special_tokens=True)

print(response)

Example Response:

User:

User: How does the FDA approach the regulation of biosimilars compared to original biologics?

Model Response

Model: The FDA approaches the regulation of biosimilars, which are biological products that are highly similar to and have no clinically meaningful differences from an already approved original biologic product, with a regulatory pathway that balances the need to provide approval for safe and effective biosimilar products with the need to maintain the safety and effectiveness of the original biologic. Specifically, the FDA has established a biosimilar development program that includes a risk evaluation mitigation strategy (REMS) for certain biosimilars, as well as a pathway for demonstrating biosimilarity to an FDA-approved reference product. \n\nTo demonstrate biosimilarity, a biosimilar applicant must provide data and information that demonstrate that the biosimilar is highly similar to the reference product notwithstanding minor differences in clinically inactive components, and there are no clinically meaningful differences between the biosimilar and the reference product in terms of safety, purity, and potency. This includes data from analytical studies, nonclinical studies, and clinical studies, as appropriate. The FDA also requires a risk evaluation mitigation strategy (REMS) for certain biosimilars, such as those approved under the 351(k) biosimilar pathway for certain products that have a REMS for the reference product. The REMS for the biosimilar is designed to mitigate risks identified in the REMS for the reference product that are relevant to the biosimilar. \n\nOverall, the FDA's approach to regulating biosimilars is designed to ensure that biosimilars meet the same rigorous standards for safety, purity, and potency as their reference products, while also providing a pathway for the approval of biosimilars that can help to increase competition and potentially lower costs for certain biologic products.