syntax-analysis-llama-3.1

Fine-Tuned LLaMA 3.1 on Dependency Parsing

This model is a fine-tuned version of LLaMA 3.1 specifically designed to automate dependency parsing of simple sentences, categorizing words into their syntactic roles according to Universal Dependency Parsing tags.

Model Details

Model Description

The model has been fine-tuned to accurately parse simple sentences by classifying each word into its respective dependency category, such as nsubj, obj, and root, following the Universal Dependency framework. This fine-tuning enhances the LLaMA 3.1 model's ability to understand and analyze sentence structures, making it a valuable tool for linguistic analysis and natural language processing tasks.

• Developed by: Emanuel Pinasco
• Model type: NLP, Dependency Parsing
• Language(s) (NLP): English

Direct Use

The model can be used directly for syntactic analysis and linguistic research, where dependency parsing is required to understand sentence structures. It’s particularly suited for tasks involving simple sentence parsing.

Recommended Prompt

"Below is a sentence. Please, provide the dependency category for each word." {sentence}

Downstream Use [optional]

Ideal for integration into larger NLP systems that require detailed sentence parsing, such as grammar checking tools, machine translation systems, and educational software.

Out-of-Scope Use

The model is not designed for complex sentence structures, idiomatic expressions, or languages other than English. Misuse may involve attempts to apply it to tasks beyond simple dependency parsing, leading to inaccurate results.

Recommendations

Users (both direct and downstream) should be aware that the model's accuracy may decline with more complex or less conventional sentence structures. It's recommended to use this model in conjunction with other tools for more comprehensive linguistic analysis.

Training Details

The model was trained on a curated dataset of simple English sentences annotated with Universal Dependency Parsing tags. The dataset was sourced from the "manupinasco/syntax_analysis" dataset available on Hugging Face's Datasets Hub. The training data focused on ensuring high accuracy in syntactic role assignment, aiming to improve the model's ability to understand and generate syntactically correct responses.

Training Data

The model was trained on a curated dataset of simple English sentences annotated with Universal Dependency Parsing tags. The training data focused on ensuring high accuracy in syntactic role assignment.

Training Procedure

The training procedure involved fine-tuning the unsloth/Meta-Llama-3.1-8B-Instruct model using a custom prompt format inspired by the Alpaca prompt template. The procedure included quantization to 4-bit to reduce memory usage, and mixed precision training to leverage GPU capabilities effectively.

Key components of the training process:

Model Quantization: 4-bit quantization was applied to the model to reduce VRAM usage while maintaining performance. Gradient Checkpointing: Enabled using "unsloth" mode to save memory during training, which allowed handling longer sequences effectively. Prompt Template: The model was trained using a structured prompt that provided instructions and expected responses, ensuring consistency and clarity in the tasks presented to the model.

Training Hyperparameters

• Batch Size: 2 per device
• Gradient Accumulation: 4 steps
• Warmup Steps: 5
• Max Training Steps: 60
• Learning Rate: 2e-4
• Optimizer: AdamW with 8-bit quantization
• Weight Decay: 0.01
• LR Scheduler: Linear
• Mixed Precision: fp16 (or bf16 if supported)

Evaluation

The model's performance was evaluated using the test split from the same dataset. The evaluation focused on syntactic role assignment accuracy.

Testing Data

The model was evaluated using a separate dataset of simple sentences annotated with Universal Dependency tags.

Factors

Evaluation focused on sentence simplicity, vocabulary diversity, and syntactic structure variations.

Metrics

Accuracy in word classification into dependency categories was the primary metric.

Summary

The fine-tuned model demonstrates high accuracy in dependency parsing of simple English sentences, making it a robust tool for basic syntactic analysis.

Model Card Authors

Emanuel Pinasco

Model Card Contact

Emanuel Pinasco

Framework versions

• PEFT 0.12.0