plamo-3-610m-fin-instruct

PLaMo-3-610M-Fin-Instruct

Model Description

PLaMo-3-610M-Fin-Instruct is a Japanese language model for the financial domain, developed by Preferred Networks, Inc. It is fine-tuned from a PLaMo-3 base model through continued pre-training on financial corpora followed by supervised fine-tuning and direct preference optimization. This is a small instruction-tuned model for the financial domain. The model follows the PLaMo-3 chat format.

Usage

Requirements

python>=3.10.0
numpy>=1.26.4
numba>=0.60.0
torch>=2.6.0,<=2.9.0
transformers>=5.0.0
accelerate

Direct loading

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "pfnet/plamo-3-610m-fin-instruct"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_name, trust_remote_code=True, dtype=torch.bfloat16, device_map="auto"
)

messages = [
    {"role": "user", "content": "PERとは何ですか?一文で答えて。"},
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_dict=True, return_tensors="pt"
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

The response contains a reasoning block delimited by <|plamo:begin_think:plamo|> and <|plamo:end_think:plamo|>, followed by the answer, terminated by <|plamo:tag|>.

Model Details

• Developed by: Preferred Networks, Inc.
• Model type: Causal decoder-only
• Context length: 262,144 tokens
• Language(s): Japanese
• Tensor type: BF16
• License: PLaMo community license

Tokenizer

The tokenizer is the PLaMo-3 tokenizer, distributed with this repository as tokenizer.jsonl and tokenization_plamo.py.

Bias, Risks, and Limitations

PLaMo-3-610M-Fin-Instruct is a new technology that carries risks with use. Testing conducted to date has been in Japanese, and has not covered, nor could it cover all scenarios. For these reasons, as with all LLMs, PLaMo-3-610M-Fin-Instruct's potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate, biased or other objectionable responses to user prompts. Therefore, before deploying any applications of PLaMo-3-610M-Fin-Instruct, developers should perform safety testing and tuning tailored to their specific applications of the model.

Domain specialization

This model has been additionally trained on financial-domain data and is intended for financial use cases. It is not a general-purpose assistant. If you intend to use it outside the financial domain, evaluate it on your own task first.

Model size and hallucination

This is a small model. It holds less factual knowledge than larger models and is more prone to hallucination: fluent, confident text that is factually wrong. Verify factual and numerical claims against an authoritative source before acting on them, and keep a human reviewer in the loop for consequential decisions.

Harmful content

Safety alignment is less reliable at this model size, and the model may generate harmful content, including offensive or violent expressions. Developers should apply content filtering appropriate to their deployment.

Not professional advice

The outputs of this model do not constitute legal, financial, tax, or investment advice, and must not be presented to end users as such. Applications built on this model should not be used to make investment decisions on a user's behalf.

License

PLaMo-3-610M-Fin-Instruct is released under the PLaMo Community License.

• (EN) https://plamo.preferredai.jp/info/plamo-community-license-en
• (JA) https://plamo.preferredai.jp/info/plamo-community-license-ja

Please review the PLaMo Community License and contact us via this form for commercial use: https://forms.gle/mTL8tBLrMYXKNZD56

Contributors

Preferred Networks, Inc.

• Ryo Wakai
• Ryota Ozaki
• Kentaro Imajo
• Masanori Hirano
• Lihang Hong
• Yoshiki Sato

AI policies for Preferred Networks, Inc. group

• (EN) https://www.preferred.jp/en/company/aipolicy/
• (JA) https://www.preferred.jp/ja/company/aipolicy/