chatdoc-llama3-q4

LLaMA3 8B Fine-Tuned on Medical QA (GGUF Q4)

Model Summary

This model is a fine-tuned version of the LLaMA3 8B model optimized for answering medical questions. It has been quantized to the GGUF Q4 format for efficient inference. Fine-tuning was performed on a dataset of 1,000 examples of medical questions and answers, leveraging Unsloth for streamlined model training and deployment.

Intended Use

• Primary Use: Assisting with medical-related queries in a controlled environment.
• Applications:
  • Medical education and training
  • Information retrieval for medical professionals
  • Question-answering systems for medical domains

Limitations

• Not a Replacement for Medical Advice: This model should not be used as a substitute for professional medical consultation or diagnosis.
• Bias and Errors: The model may reflect biases in the training data and can produce incorrect or harmful answers if used improperly.

Training Details

• Base Model: LLaMA3 8B
• Fine-tuning Dataset: Custom dataset of 1,000 medical question-answer pairs.
• Fine-tuning Framework: Hugging Face's transformers library and PyTorch.
• Fine-tuning Workflow: The fine-tuning process was managed using Unsloth, which facilitated:
  • Automated data preprocessing
  • Distributed training across GPUs
  • Hyperparameter tuning
  • Monitoring and logging for improved reproducibility
• Quantization: GGUF Q4 format for optimized inference on resource-constrained environments.
• Epochs/Steps: 100
• Learning Rate: 2e-4
• Hardware Used: T4 (Google Colab)