Legal_TAIDE_LLM

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

This is a large language model developed by the Taiwan AI Dialog Engine (TAIDE) project. It is based on Meta's Llama 3.1 architecture and has been specifically finetuned for Traditional Chinese language tasks and cultural context.

Model Details

Model Description

The Llama-3.1-TAIDE-LX-8B-Chat model is an 8.5 billion parameter model designed for chat and general office tasks. It excels at tasks requiring deep Chinese language understanding, including document analysis, creative writing, and technical discussions. The model's context length has been expanded to 131,584 tokens, significantly improving its long-form content handling.

<!-- Provide a longer summary of what this model is. -->

• Developed by: Phil Liao
• Model type: Causal Language Model
• Language(s) (NLP): Traditional Chinese (zh) and English (en)
• License: MIT License
• Finetuned from model: Llama-3.1-TAIDE-LX-8B-Chat

Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

Direct Use

The model is intended for direct use in chat-based applications, content creation, document summarization, and bilingual communication (Traditional Chinese and English). Its long context window makes it suitable for complex, multi-turn conversations and long-form document analysis.

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

Downstream Use

The model can be further finetuned for specific domain applications, such as educational assistance, healthcare, and technical writing. Its foundation and cultural-specific training make it a strong base for building new applications tailored to the Taiwanese market.

<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->

Out-of-Scope Use

The model should not be used for generating misinformation, hate speech, or content that violates legal or ethical guidelines. Due to its cultural focus, it may not perform as well in other languages or on topics with limited Traditional Chinese-language training data.

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->

Bias, Risks, and Limitations

The model is trained on a curated dataset to promote trustworthiness, but like all language models, it may still reflect biases present in its training data. The model may generate incorrect, nonsensical, or unhelpful responses, especially when dealing with ambiguous or highly specialized queries. It is a tool and its output should always be reviewed by a human.

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Content generated by the model should be fact-checked and verified, especially for critical applications.

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

How to Get Started with the Model

Use the code below to get started with the model. Please note that PEFT is used for parameter-efficient finetuning, while the base model can be loaded and used directly with the transformers library.

Training Details

Training Data

https://github.com/NCHU-NLP-Lab/LawFactsQA-TW

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

• PEFT 0.17.2.dev0