adaption_llama_3_3_70b_instru_mrketing_data_30k_126723ff

Adaption Llama 3.3 70B Instruct Marketing LoRA

This repository contains LoRA adapter weights trained for the Marketing category of the Adaption AutoScientist Challenge.

It is not a standalone merged 70B checkpoint. To use it, load the base model meta-llama/Llama-3.3-70B-Instruct, extract the adapter archive, then apply or merge the LoRA adapter.

The model was trained from prathmeshadsod/mrketing-data-30k, a public marketing instruction dataset created from a 30k synthetic source corpus and a 20k selected Adaption training subset.

Evaluation Summary

Adaption's evaluation dashboard shows a strong preference for the adapted model over the base model.

The Marketing-category win-rate values are dashboard-rounded values. They show that the adapted LoRA was preferred not only on the submitted dataset, but also across broader Marketing tasks.

Training And Evaluation Metrics

The training run used the AutoScientist-selected configuration unchanged. The run completed with the following identifiers:

Training Configuration

{
  "lora": true,
  "lora_r": 16,
  "n_evals": 5,
  "n_epochs": 2,
  "batch_size": "max",
  "lora_alpha": 32,
  "lora_dropout": 0,
  "min_lr_ratio": 0.1,
  "warmup_ratio": 0.05,
  "weight_decay": 0.05,
  "learning_rate": 0.0001,
  "max_grad_norm": 1,
  "base_model_size": "70B",
  "train_on_inputs": false,
  "training_method": "sft",
  "lr_scheduler_type": "cosine",
  "scheduler_num_cycles": 0.5,
  "lora_trainable_modules": "all-linear"
}

Dataset

Training data:

prathmeshadsod/mrketing-data-30k

The dataset was built for marketing decision intelligence rather than generic copywriting. It covers basic, intermediate, advanced, and expert marketing tasks across strategy, performance marketing, lifecycle, retention, ABM, partnerships, events, brand, product marketing, SEO, social/content/community, analytics, experiments, compliance-aware marketing, and operations.

The source corpus was generated with the earlier MarketScientist pipeline using GPT-OSS-120B as the primary generation model through an external LLM API provider. This was not the later v4/multi-model exploration. The generation process used coverage planning across category, difficulty, persona, market, industry, business model, prompt style, and required response mode. Responses were designed to be practical and answer-first, with assumptions, caveats, watch-outs, calculations, or next steps added only when helpful.

Adaption remastering improved the dataset quality score from 9.0/10 to 9.7/10, with grade A before and after, and percentile movement from 33.0 to 57.7.

Adapter Files

Current repository contents include:

• adaption_llama_3_3_70b_instru_mrketing_data_30k_126723ff.tgz

This archive contains the exported adapter artifact. Extract it before loading. The extracted directory should contain PEFT/LoRA adapter files such as an adapter config and adapter weights.

Usage

1. Download and extract the adapter

hf download prathmeshadsod/adaption_llama_3_3_70b_instru_mrketing_data_30k_126723ff \
  adaption_llama_3_3_70b_instru_mrketing_data_30k_126723ff.tgz \
  --local-dir ./adaption_marketing_lora

mkdir -p ./adaption_marketing_lora/extracted
tar -xzf ./adaption_marketing_lora/adaption_llama_3_3_70b_instru_mrketing_data_30k_126723ff.tgz \
  -C ./adaption_marketing_lora/extracted

On Windows PowerShell, tar -xzf also works in recent Windows versions.

2. Load with PEFT

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "meta-llama/Llama-3.3-70B-Instruct"
adapter_dir = "./adaption_marketing_lora/extracted"

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

model = PeftModel.from_pretrained(base_model, adapter_dir)

messages = [
    {
        "role": "user",
        "content": (
            "Our paid search CPA rose 28% this month while conversion volume is flat. "
            "What should I check first, and what decision would you make before adding more budget?"
        ),
    }
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=700,
        temperature=0.3,
        top_p=0.9,
    )

print(tokenizer.decode(output[0], skip_special_tokens=True))

3. Optional: merge the adapter

merged_model = model.merge_and_unload()
merged_model.save_pretrained("./merged_adaption_marketing_llama_70b")
tokenizer.save_pretrained("./merged_adaption_marketing_llama_70b")

Running or merging a 70B model requires substantial GPU memory. Quantization or multi-GPU inference may be needed.

Intended Behavior

The adapter is intended to improve a Llama 3.3 70B Instruct model for marketing decision support. Good responses should:

• Answer the user's marketing question directly.
• Use the provided data and avoid inventing missing metrics.
• Ask targeted clarification questions when required data is absent.
• Include assumptions, caveats, or watch-outs only when they improve the decision.
• Prefer practical next steps over generic marketing advice.
• Handle both simple beginner requests and more complex executive/analyst prompts.

Limitations

• This is a LoRA adapter, not a merged model.
• The base model license and access requirements still apply.
• The model is specialized for marketing decision support and should be evaluated before production use.
• Synthetic and remastered data can still contain blind spots. Human review remains important for legal, financial, regulated, or brand-sensitive decisions.

Competition Note

This run was completed late in the competition cycle with a small number of full experiments. The result suggests that high-quality dataset design, Adaption remastering, and a strong AutoScientist-selected LoRA configuration can produce meaningful gains even without a large hyperparameter sweep.

Citation

@misc{adsod2026adaptionmarketinglora,
  title = {Adaption Llama 3.3 70B Instruct Marketing LoRA},
  author = {Prathmesh Adsod},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/prathmeshadsod/adaption_llama_3_3_70b_instru_mrketing_data_30k_126723ff}}
}