i2b2-querybuilder-34b-merged

i2b2 QueryBuilder - 34b Merged

<!-- TODO: Add a link here N: DONE-->

Model Description

This model will generate queries for your i2b2 query builder trained on this dataset for 10 epochs . For evaluation use.

• Do not use as a final research query builder.
• Results may be incorrect or mal-formatted.
• The onus of research accuracy is on the researcher, not the AI model.

Prompt Format

If you are using text-generation-webui, you can download the instruction template i2b2.yaml

Below is an instruction that describes a task.

### Instruction:
{input}

### Response:
```xml

Architecture

nmitchko/i2b2-querybuilder-34b-merged is a large language model LoRa specifically fine-tuned for generating queries in the i2b2 query builder. It is based on codellama-34b-hf at 34 billion parameters.

The primary goal of this model is to improve research accuracy with the i2b2 tool. It was trained using LoRA, specifically QLora Multi GPU, to reduce memory footprint.

See Training Parameters for more info This Lora supports 4-bit and 8-bit modes.

Requirements

bitsandbytes>=0.41.0
peft@main
transformers@main

Steps to load this model:

1. Load base model (codellama-34b-hf) using transformers

Training Parameters

The model was trained for or 10 epochs on i2b2-query-data-1.0 i2b2-query-data-1.0 contains only tasks and outputs for i2b2 queries xsd schemas.

Training procedure

The following bitsandbytes quantization config was used during training:

• quant_method: QuantizationMethod.BITS_AND_BYTES
• load_in_8bit: False
• load_in_4bit: True
• llm_int8_threshold: 6.0
• llm_int8_skip_modules: None
• llm_int8_enable_fp32_cpu_offload: False
• llm_int8_has_fp16_weight: False
• bnb_4bit_quant_type: nf4
• bnb_4bit_use_double_quant: True
• bnb_4bit_compute_dtype: bfloat16

Framework versions

• PEFT 0.6.0.dev0