Poetic_Llama-3.2-1B

Poetic_Llama-3.2-1B

Model Description

Poetic_Llama-3.2-1B is a specialized language model fine-tuned for creative poetry generation. Built upon Meta's Llama-3.2-1B foundation model, this adapter has been optimized using supervised fine-tuning (SFT) with Low-Rank Adaptation (LoRA) techniques to generate coherent, stylistically diverse, and contextually appropriate poetic content in English. The model is designed to understand poetic instructions and generate verses across various styles, forms, and themes, making it suitable for creative writing applications, educational tools, and literary experimentation.

How to Get Started with the Model

Use the code below to get started with the model:

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
# Load the model and tokenizer
model_name = "Exquisique/Poetic_Llama-3.2-1B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
# Example poetry generation
prompt = "Human: Write a poem about hope and dawn.\nAssistant:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
# Generate
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=100,
        temperature=0.7,
        top_p=0.9,
        do_sample=True
    )
poem = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(poem)

Important: Prompt Formatting for Best Results

For the most consistent poetic outputs, format your prompts to match the model’s training style:

• Use the dialog-style prefix: Human: <your instruction or poetic request>\nAssistant:
• After Assistant:, let the model continue the poem.

Recommended examples:

• Human: Write a poem about hope and dawn.\nAssistant:
• Human: Compose a haiku about autumn leaves and wind.\nAssistant:
• Human: Write a Shakespearean sonnet on time and memory.\nAssistant:

Why this matters: The model was fine-tuned on instruction-following data that uses this Human/Assistant schema. Adhering to this format helps the model align to the intended poetic style and structure.

Usage Tips

• Use clear, specific instructions in your prompts (e.g., "Write a sonnet about...", "Create a free verse poem describing...")
• Adjust temperature (0.6-0.9) for creativity vs. coherence tradeoff
• Use top_p sampling for diverse outputs
• Keep the Human:/Assistant: format for consistent results

Model Details

Model Description

• Developed by: Exquisique
• Model type: Causal Language Model (Fine-tuned Adapter)
• Language(s): English
• License: Apache 2.0
• Finetuned from model: meta-llama/Llama-3.2-1B
• Architecture: Llama 3.2 with LoRA adapter
• Parameters: 1.24B (base model) + LoRA adapter weights
• Precision: BF16

Model Sources

• Repository: Exquisique/Poetic_Llama-3.2-1B
• Base Model: meta-llama/Llama-3.2-1B
• Training Framework: TRL (Transformer Reinforcement Learning)
• Adapter Method: PEFT 0.17.1 (LoRA)

Uses

Direct Use

This model can be directly used for:

• Poetry Generation: Creating original poems based on prompts, themes, or specific instructions
• Creative Writing Assistance: Helping writers with poetic language, metaphors, and verse structure
• Educational Applications: Teaching poetic forms, literary devices, and creative expression
• Content Creation: Generating poetic content for social media, marketing, or artistic projects
• Literary Exploration: Experimenting with different poetic styles, forms (sonnets, haiku, free verse, etc.), and themes

Downstream Use

The model can be integrated into:

• Creative writing applications and platforms
• Educational software for literature and creative writing courses
• Chatbots and virtual assistants with creative capabilities
• Content generation pipelines for marketing and social media
• Research tools for computational creativity and natural language generation studies

Out-of-Scope Use

• Factual Information Retrieval: This model is optimized for creative poetry generation, not factual accuracy or information retrieval
• Professional Medical, Legal, or Financial Advice: Not suitable for generating content requiring professional expertise
• Harmful Content Generation: Should not be used to create content that promotes violence, hatred, discrimination, or illegal activities
• Impersonation: Not intended for generating content that impersonates specific authors or public figures without disclosure
• Academic Dishonesty: Should not be used to generate content for plagiarism or academic fraud

Bias, Risks, and Limitations

Known Limitations

• Creative Scope: While fine-tuned on poetry, the model's output quality may vary across different poetic styles and forms
• Cultural Representation: The training data (poem-instructions-dataset) may have limited representation of non-Western poetic traditions
• Context Length: As a 1B parameter model, it has limitations in maintaining coherence over very long compositions
• Factual Accuracy: Creative outputs may contain factually incorrect information, as the model prioritizes poetic expression over accuracy
• Originality: Generated poems may occasionally reflect patterns or themes from training data

Potential Biases

• The model may exhibit biases present in the base Llama-3.2-1B model and the poem-instructions-dataset
• Potential over-representation of certain poetic styles, themes, or cultural perspectives in the training data
• Language biases toward formal or conventional English poetic traditions

Risks

• Misattribution: Generated content might be mistaken for human-authored work without proper disclosure
• Content Appropriateness: May occasionally generate content with mature themes or sensitive subject matter
• Over-reliance: Users may become overly dependent on AI-generated content, potentially limiting human creativity

Recommendations

Users should:

• Clearly disclose when content is AI-generated, especially in published or commercial contexts
• Review and edit generated content for appropriateness, quality, and alignment with intended use
• Use the model as a creative tool to augment, not replace, human creativity and judgment
• Be aware of potential biases and work to ensure diverse and inclusive creative outputs
• Not use generated poetry for academic submissions without proper attribution and instructor approval
• Implement content filtering mechanisms when deploying in user-facing applications

Training Details

Training Data

The model was fine-tuned on the iamketan25/poem-instructions-dataset, which contains instruction-following examples for poetry generation. This dataset includes:

• Diverse poetic prompts and instructions
• Examples covering various poetic forms and styles
• Instruction-output pairs designed for supervised fine-tuning
• Approximately 2,100 examples (based on dataset statistics) The dataset focuses on teaching the model to understand and execute creative instructions for generating contextually appropriate poetic content.

Training Procedure

Training Method

• Fine-tuning Approach: Supervised Fine-Tuning (SFT) using the TRL library
• Parameter-Efficient Method: LoRA (Low-Rank Adaptation) for efficient training and deployment
• Base Model: meta-llama/Llama-3.2-1B (1.24B parameters)
• Adapter Framework: PEFT (Parameter-Efficient Fine-Tuning) version 0.17.1
• Training Framework: Hugging Face Transformers with TRL

Training Configuration

The model was trained using parameter-efficient fine-tuning techniques:

• LoRA adapters were applied to enable efficient training without modifying all base model parameters
• Training focused on instruction-following capabilities for creative poetry generation
• BF16 precision was used for efficient computation and memory usage Note: Specific hyperparameters (learning rate, batch size, number of epochs, LoRA rank, alpha values, etc.) were not provided in the model repository.

Evaluation

Testing Data and Metrics

Testing Approach

Formal quantitative evaluation metrics for creative text generation (especially poetry) are challenging, as quality is often subjective and context-dependent. The model's performance should be evaluated based on:

• Coherence: Logical flow and consistency within generated poems
• Instruction Following: Adherence to specified themes, styles, or constraints
• Creativity: Originality and interesting use of language and literary devices
• Grammaticality: Proper syntax and language structure
• Stylistic Appropriateness: Alignment with requested poetic forms or styles

Qualitative Assessment

Users are encouraged to evaluate the model through:

• Human evaluation of generated poetry quality
• Comparison with baseline models (unmodified Llama-3.2-1B or other poetry-generation models)
• User feedback in creative writing applications Note: Specific quantitative evaluation results, test sets, or benchmark scores were not provided in the model repository.

Environmental Impact

Carbon emissions and environmental impact can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Training Infrastructure

• Hardware Type: GPU (specific model not specified)
• Training Time: Not specified
• Cloud Provider: Not specified
• Compute Region: Not specified
• Estimated Carbon Emitted: Not measured Note: Using LoRA and parameter-efficient fine-tuning significantly reduces computational requirements and environmental impact compared to full model training.

Inference Efficiency

The model uses:

• BF16 precision for efficient inference
• LoRA adapters for reduced memory footprint
• 1.24B parameters (relatively small model size) enabling deployment on consumer hardware These characteristics make the model more environmentally friendly and accessible for inference compared to larger language models.

Technical Specifications

Model Architecture and Objective

• Base Architecture: Llama 3.2 (Causal Language Model)
• Model Size: 1.24B parameters
• Adapter Type: LoRA (Low-Rank Adaptation)
• Tensor Format: BF16 (Brain Floating Point 16-bit)
• Objective: Supervised instruction-following fine-tuning for poetry generation
• Tokenizer: Llama tokenizer (from base model)

Compute Infrastructure

Hardware

Training was conducted using GPU acceleration (specific hardware details not provided in repository).

Software

• Framework: Hugging Face Transformers
• Fine-tuning Library: TRL (Transformer Reinforcement Learning)
• Adapter Framework: PEFT 0.17.1
• Precision: BF16
• Model Format: Safetensors

Citation

BibTeX

If you use this model in your research or applications, please cite:

@misc{poetic_llama_3_2_1b,
  author = {Exquisique},
  title = {Poetic_Llama-3.2-1B: A Fine-tuned Language Model for Poetry Generation},
  year = {2024},
  publisher = {HuggingFace},
  howpublished = {\url{https://huggingface.co/Exquisique/Poetic_Llama-3.2-1B}}
}

Please also cite the base model:

@article{dubey2024llama,
  title={The Llama 3 Herd of Models},
  author={Dubey, Abhimanyu and others},
  journal={arXiv preprint arXiv:2407.21783},
  year={2024}
}

And the training dataset:

@misc{poem_instructions_dataset,
  author = {iamketan25},
  title = {Poem Instructions Dataset},
  year = {2023},
  publisher = {HuggingFace},
  howpublished = {\url{https://huggingface.co/datasets/iamketan25/poem-instructions-dataset}}
}

APA

Exquisique. (2024). Poetic_Llama-3.2-1B: A Fine-tuned Language Model for Poetry Generation. HuggingFace. https://huggingface.co/Exquisique/Poetic_Llama-3.2-1B

Model Card Authors

This model card was created by Exquisique with comprehensive updates to meet Hugging Face model card standards.

Model Card Contact

For questions, issues, or collaboration opportunities regarding this model, please:

• Open an issue in the model repository discussions
• Visit the Exquisique profile for contact information

---

Acknowledgments

• Meta AI for developing and releasing the Llama 3.2 base model
• iamketan25 for creating and sharing the poem-instructions-dataset
• Hugging Face for the Transformers, PEFT, and TRL libraries
• The open-source AI community for tools and frameworks enabling this work