Llama-3-8B-Instruct-pgfplots-finetune-q4f16_1-MLC

Usage

This model is saved as MLC LLM format. View the installation guide of MLC LLM for how to install the library. Then use the following command to try the model:

mlc_llm chat .

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

The model is finetuned from Llama 3 LLM to provide more accurate results on generating LaTeX code of pgfplots package, which is based on the dataset LogCreative/latex-pgfplots-instruct extracted from the documentation of pgfplots LaTeX package.

• Developed by: LogCreative
• Model type: Text Generation
• Language(s) (NLP): English
• License: Llama 3
• Finetuned from model: unsloth/llama-3-8b-Instruct

Model Sources

<!-- Provide the basic links for the model. -->

• Repository: LogCreative/llama-pgfplots-finetune

Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

This model is intended to generate the pgfplots LaTeX code according to the user's prompt. It is suitable for users who are not familiar with the API provided in the pgfplots package or does not want to consult the documentation for achieving the intention.

Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

PGFPlotsEdt: A PGFPlots Statistic Graph Interactive Editor.

Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->

Any use outside the pgfplots package could only be of the performance of the base Llama 3 model.

Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

This model could not provide sufficient information on other LaTeX packages and could not guarantee the absolute correctness of the generated result.

Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. If you can not get the correct result from this model, you may need to consult the original pgfplots documentation for more information.

Training Details

Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

LogCreative/latex-pgfplots-instruct: a datasets contains the instruction and corresponding output related to pgfplots and pgfplotstable LaTeX packages.

Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

This model is finetuned based on the dataset based on unsloth library.

Training Hyperparameters

• Training regime: bf16 mixed precision <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->

Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

The evaluation is based on the success compilation rate of the output LaTeX code in the test dataset.

Testing Data, Factors & Metrics

Testing Data

<!-- This should link to a Dataset Card if possible. -->

LogCreative/latex-pgfplots-instruct: the test part of this dataset only contains instructions only related to the pgfplots package.

Factors

<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->

When testing, the prompt prefix is added to tell the model what role it is and what the requested response format is to only output the code without any explanation.

Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

Success compilation rate: $$\frac{\text{#Success compilation}}{\text{#Total compilation}}\times 100%$$ The uncessful compilation is rather LaTeX failure or the timeout case (compilation time > 20s).

Results

The test is based upon unquantized model which is in fp16 precision.

• Llama 3: 34%
• This model: 52% (+18%)

Summary

This model is expected to output the LaTeX code output related to the pgfplots package with less error compared to the baseline Llama 3 model.

Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the Machine Learning Impact calculator.

• Hardware Type: Nvidia A100 80G
• Hours used: 1h = 10min training + 50min testing
• Cloud Provider: Private infrastructure
• Carbon Emitted: 0.11kg CO2 eq.

Framework versions

• PEFT 0.11.1
• MLC LLM nightly_cu122-0.1.dev1404
• MLC AI nightly_cu122-0.15.dev404
• Unsloth 2024.6