llama3_8b_taiwan_stock_qlora

Model Card for Model ID

This PEFT model is designed for predicting the prices of these five Taiwan stocks:

Disclaimer: This model is for a time series problem on LLM performance, and it's not for investment advice; any prediction results are not a basis for investment reference.

Model Details

The training data source is from the 臺灣證券交易所 / Taiwan Stock Exchange (TWSE), covering the period from January 1, 2019, to July 31, 2024 (5 years).

Model Description

This repo contains QLoRA format model files for Meta's Llama 3 8B Instruct.

Uses

import torch
from peft import LoraConfig, PeftModel

from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    BitsAndBytesConfig,
    HfArgumentParser,
    TrainingArguments,
    TextStreamer,
    pipeline,
    logging,
)

device_map = {"": 0}
use_4bit = True
bnb_4bit_compute_dtype = "float16"
bnb_4bit_quant_type = "nf4"
use_nested_quant = False
compute_dtype = getattr(torch, bnb_4bit_compute_dtype)

bnb_config = BitsAndBytesConfig(
    load_in_4bit=use_4bit,
    bnb_4bit_quant_type=bnb_4bit_quant_type,
    bnb_4bit_compute_dtype=compute_dtype,
    bnb_4bit_use_double_quant=use_nested_quant,
)

based_model_path = "DavidLanz/Llama3-tw-8B-Instruct"
adapter_path = "DavidLanz/llama3_8b_taiwan_stock_qlora"

base_model = AutoModelForCausalLM.from_pretrained(
    based_model_path,
    low_cpu_mem_usage=True,
    return_dict=True,
    quantization_config=bnb_config,
    torch_dtype=torch.float16,
    device_map=device_map,
)
model = PeftModel.from_pretrained(base_model, adapter_path)

tokenizer = AutoTokenizer.from_pretrained(based_model_path, trust_remote_code=True)

import torch
from transformers import pipeline, TextStreamer

text_gen_pipeline = pipeline(
    "text-generation",
    model=model,
    model_kwargs={"torch_dtype": torch.bfloat16},
    tokenizer=tokenizer,
)

messages = [
    {
        "role": "system",
        "content": "你是一位專業的台灣股市交易分析師",
    },
    {"role": "user", "content": "鴻海上週五的表現，開盤價是211.00，當日最高價是217.50，當日最低價是211.00，收盤價是212.00，與前一天相比下跌了5.50，成交股數為174,076,905，交易金額為37,127,696,834。請預測今天的收盤價?"},
]

prompt = text_gen_pipeline.tokenizer.apply_chat_template(
        messages, 
        tokenize=False, 
        add_generation_prompt=True
)

terminators = [
    text_gen_pipeline.tokenizer.eos_token_id,
    text_gen_pipeline.tokenizer.convert_tokens_to_ids("<|eot_id|>")
]

outputs = text_gen_pipeline(
    prompt,
    max_new_tokens=256,
    eos_token_id=terminators,
    do_sample=True,
    temperature=0.6,
    top_p=0.9,
)
print(outputs[0]["generated_text"][len(prompt):])

Framework versions

• PEFT 0.11.1