fqpegaqmsmbd

Private LLM Hugging Face Wrapper

This repository wraps private_LLM_model.py as a custom Hugging Face Transformers model. The private script is loaded only at runtime.

Loading this model requires trust_remote_code=True because it uses custom model and tokenizer code.

Install

pip install -r requirements.txt

Standard Text Generation Pipeline

from transformers import pipeline

generator = pipeline(
    "text-generation",
    model="YOUR_USERNAME/YOUR_REPO",
    trust_remote_code=True,
)

print(generator("Write a short greeting.", max_new_tokens=64))

Direct Model Loading

from transformers import AutoModelForCausalLM
from transformers import AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "YOUR_USERNAME/YOUR_REPO",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(
    "YOUR_USERNAME/YOUR_REPO",
    trust_remote_code=True,
)

print(model.generate_text("Write a short greeting."))

Optional Pipeline

from transformers import pipeline

pipe = pipeline(
    "private-llm",
    model=".",
    trust_remote_code=True,
)

print(pipe("Write a short greeting."))

Publish To The Hub

Authenticate first:

hf auth login

Then upload the current folder:

python publish_to_hub.py YOUR_USERNAME/YOUR_REPO

The publish script creates a private model repository by default. Use --public only if you want the Hub repo to publicly expose private_LLM_model.py.

Private Script Entrypoints

The wrapper auto-detects these common patterns:

• Loader functions: load_model, create_model, build_model, get_model, load_llm
• Model objects: model, llm, MODEL, LLM_MODEL
• Model classes: PrivateLLM, LLM, Model
• Generation methods/functions: generate_text, generate, complete, predict, chat, __call__

If the script is meant to be run directly instead of imported, set this in config.json:

{
  "execution_mode": "subprocess"
}

Subprocess mode sends the prompt to stdin by default. It also sets PRIVATE_LLM_PROMPT and PRIVATE_LLM_KWARGS environment variables.

To pin exact names without changing your private script, set fields like:

{
  "loader_function": "load_model",
  "generate_function": "generate"
}

If your private script returns the full prompt plus completion instead of only the completion text, set:

{
  "private_output_includes_prompt": true
}