DocsGPT-7B-imatrix-GGUF

Quantizations of https://huggingface.co/Arc53/DocsGPT-7B

From original readme

This model is fine tuned on top of llama-2-7b

DocsGPT is optimized for Documentation: Specifically fine-tuned for providing answers that are based on documentation provided in context, making it particularly useful for developers and technical support teams.

We used 50k high quality examples to finetune it over 1.5 days on A10G GPU. We used lora fine tuning process.

Its an apache-2.0 license so you can use it for commercial purposes too.

How to run it

from transformers import AutoTokenizer, AutoModelForCausalLM
import transformers
import torch

model = "Arc53/docsgpt-7b"

tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
    device_map="auto",
)
sequences = pipeline(
   "Girafatron is obsessed with giraffes, the most glorious animal on the face of this Earth. Giraftron believes all other animals are irrelevant when compared to the glorious majesty of the giraffe.\nDaniel: Hello, Girafatron!\nGirafatron:",
    max_length=200,
    do_sample=True,
    top_k=10,
    num_return_sequences=1,
    eos_token_id=tokenizer.eos_token_id,
)
for seq in sequences:
    print(f"Result: {seq['generated_text']}")