Plant-Based-LLM

Worlds first Whole Foods Plant Based (WFPB) LLM. A Qwen3 1.7B model fine tuned with a LoRA adapter trained on 1255 synthetically generated training pairs from blog posts and video transcripts from Nutritionfacts.org. May output Chinese characters (Qwen3 is from Alibaba and multilingual) and long responses unless properly prompted. Created by Toby Miller. Linkedin: https://www.linkedin.com/in/robertmilleree/

note: Chat template needs adjustments, but works with the following python script:

from transformers import AutoTokenizer, AutoModelForCausalLM

repo = "tobymiller2/Plant-Based-LLM" tokenizer = AutoTokenizer.from_pretrained(repo) model = AutoModelForCausalLM.from_pretrained(repo, dtype="auto")

prompt = "What is the best diet for human health and longevity? what about meat?" inputs = tokenizer(prompt, return_tensors="pt") outputs = model.generate( **inputs, max_new_tokens=500, do_sample=False, no_repeat_ngram_size=3, repetition_penalty=1.3, pad_token_id=tokenizer.eos_token_id, eos_token_id=tokenizer.eos_token_id ) print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Example output: Q: ¨What is the best diet for human health and longevity?¨ A. ¨Plant-based diets rich in vegetables, fruits, legumes, grains, and seeds support long life with minimal chronic disease. Animal products increase inflammation and risk of cancer, while plants reduce it; this illustrates how whole food plant foods can prevent age-related diseases naturally. 植物性饮食是维持健康和延长寿命的最佳选择，它通过减少慢性病风险提供最大保护。¨

Note: The Chinese output translates to: “A plant-based diet is the best choice for maintaining health and extending lifespan, as it provides maximum protection by reducing the risk of chronic diseases.”

The base model previously talked favoribly about meat. Qwen3-1.7B-base Q: ´´Does that include meat? is meat consumption ideal for optimal health? A: [long response, so here is the conclusion only] ¨Conclusion Meat can be ideal for optimal health when consumed in moderation and as part of a balanced diet. The key is to prioritize lean sources, limit processed options, and ensure variety in protein intake. For those with specific dietary needs (e.g., vegetarianism or low-sodium diets), adjustments are possible while maintaining overall health.¨

Training data 1255 synthetically generated training pairs from blog posts and video transcripts from Nutritionfacts.org.

Fine tuning setting for Llama Factory:

create_new_adapter: true cutoff_len: 512 dataset: - dataset_dir: - ddp_timeout: 180000000 do_train: true double_quantization: true enable_thinking: false eval_steps: 100 eval_strategy: steps finetuning_type: lora flash_attn: auto fp16: true gradient_accumulation_steps: 4 include_num_input_tokens_seen: true learning_rate: 0.0002 logging_steps: 5 lora_alpha: 128 lora_dropout: 0.1 lora_rank: 128 lora_target: all lr_scheduler_type: cosine max_grad_norm: 1.0 max_samples: 100000 model_name_or_path: - num_train_epochs: 4.0 optim: adamw_torch output_dir: - packing: true per_device_eval_batch_size: 1 per_device_train_batch_size: 1 plot_loss: true preprocessing_num_workers: 16 quantization_bit: 4 quantization_method: bnb report_to: none save_steps: 100 stage: sft template: default trust_remote_code: true use_rslora: true val_size: 0.15 warmup_steps: 0