BAAI_-_OPI-Llama-3.1-8B-Instruct-gguf

Quantization made by Richard Erkhov.

Github

Discord

Request more models

OPI-Llama-3.1-8B-Instruct - GGUF

• Model creator: https://huggingface.co/BAAI/
• Original model: https://huggingface.co/BAAI/OPI-Llama-3.1-8B-Instruct/

Original model description:

license: llama3.1 datasets:

• BAAI/OPI language:
• en pipeline_tag: text-generation tags:
• Life Science
• AI4Science
• Biology
• Protein
• LLM
• Instruction base_model: meta-llama/Meta-Llama-3.1-8B-Instruct

---

Model Card of OPI-Llama-3.1-8B-Instruct

OPI-Llama-3.1-8B-Instruct was fine-tuned from the Meta-Llama-3.1-8B-Instruct model using the complete OPI training set (i.e.,OPI_full_1.61M_train.json). For more details of training and testing, please visit https://github.com/baaihealth/opi.

Evaluation of OPI-Llama-3.1-8B-Instruct on 9 tasks

Each testing result is derived from the Meta-Llama-3.1-8B-Instruct model that has been fine-tuned using OPI_full_1.61M.json and subsequently evaluated on the respective testing set for each specific task.

<table border="1" style="text-align:center; border-collapse:collapse; width: 100%;"> <thead> <tr> <th style="text-align:center;">Task Type</th> <th style="text-align:center;">Task Name</th> <th style="text-align:center;">Testing file</th> <th style="text-align:center;">Accuracy</th> <th style="text-align:center;">Precision</th> <th style="text-align:center;">Recall</th> <th style="text-align:center;">F1</th> <th style="text-align:center;">Rouge-L</th> </tr> </thead> <tbody> <tr> <td rowspan="6">Sequence Understanding</td> <td rowspan="2">EC Number Prediction (split100)</td> <td>CLEAN_EC_number_new_test</td> <td>-</td> <td>0.3724</td> <td>0.3374</td> <td>0.3468</td> <td>-</td> </tr> <tr> <td>CLEAN_EC_number_price_test</td> <td>-</td> <td>0.0738</td> <td>0.0738</td> <td>0.0738</td> <td>-</td> </tr> <tr> <td rowspan="3">Fold Type Prediction</td> <td>fold_type_test_Fold_Holdout</td> <td>0.1045</td> <td>-</td> <td>-</td> <td>-</td> <td>-</td> </tr> <tr> <td>fold_type_test_Superfamily_Holdout</td> <td>0.1507</td> <td>-</td> <td>-</td> <td>-</td> <td>-</td> </tr> <tr> <td>fold_type_test_Family_Holdout</td> <td>0.6145</td> <td>-</td> <td>-</td> <td>-</td> <td>-</td> </tr> <tr> <td>Subcellular Localization Prediction</td> <td>subcell_loc_test</td> <td>0.4214</td> <td>-</td> <td>-</td> <td>-</td> <td>-</td> </tr> <tr> <td rowspan="9">Annotation Prediction</td> <td rowspan="3">Function Keywords Prediction</td> <td>CASPSimilarSeq_keywords_test</td> <td>-</td> <td>0.4202</td> <td>0.5057</td> <td>0.4385</td> <td>-</td> </tr> <tr> <td>IDFilterSeq_keywords_test</td> <td>-</td> <td>0.6762</td> <td>0.6905</td> <td>0.6650</td> <td>-</td> </tr> <tr> <td>UniProtSeq_keywords_test</td> <td>-</td> <td>0.7606</td> <td>0.7489</td> <td>0.7374</td> <td>-</td> </tr> <tr> <td rowspan="3">Gene Ontology(GO) Terms Prediction</td> <td>CASPSimilarSeq_go_terms_test</td> <td>-</td> <td>0.1113</td> <td>0.0936</td> <td>0.099</td> <td>-</td> </tr> <tr> <td>IDFilterSeq_go_terms_test</td> <td>-</td> <td>0.6686</td> <td>0.6287</td> <td>0.6304</td> <td>-</td> </tr> <tr> <td>UniProtSeq_go_terms_test</td> <td>-</td> <td>0.7150</td> <td>0.6897</td> <td>0.6849</td> <td>-</td> </tr> <tr> <td rowspan="3">Function Description Prediction</td> <td>CASPSimilarSeq_function_test</td> <td>-</td> <td>-</td> <td>-</td> <td>-</td> <td>0.7524</td> </tr> <tr> <td>IDFilterSeq_function_test</td> <td>-</td> <td>-</td> <td>-</td> <td>-</td> <td>0.4786</td> </tr> <tr> <td>UniProtSeq_function_test</td> <td>-</td> <td>-</td> <td>-</td> <td>-</td> <td>0.5144</td> </tr> <tr> <td rowspan="3">Knowledge Mining</td> <td>Tissue Location Prediction from Gene Symbol</td> <td>gene_symbol_to_tissue_test</td> <td>-</td> <td>0.4002</td> <td>0.9356</td> <td>0.5466</td> <td>-</td> </tr> <tr> <td>Cancer Prediction from Gene Symbol</td> <td>gene_symbol_to_cancer_test</td> <td>-</td> <td>0.2890</td> <td>0.2701</td> <td>0.2664</td> <td>-</td> </tr> <tr> <td>Cancer Prediction from Gene Name</td> <td>gene_name_to_cancer_test</td> <td>-</td> <td>0.2786</td> <td>0.2707</td> <td>0.2659</td> <td>-</td> </tr> </tbody> </table>

Prediction comparison with SOTA mdoels