text-to-rdb-queries

Model Card for shaheerzk/text-to-rdb-queries

Inference with hugging face transformers

from transformers import AutoModelForCausalLM
 
model = AutoModelForCausalLM.from_pretrained("shaheerzk/text-to-rdb-queries")
model.to("cuda")
 
generated_ids = model.generate(tokens, max_new_tokens=1000, do_sample=True)

# decode with mistral tokenizer
result = tokenizer.decode(generated_ids[0].tolist())
print(result)

[!TIP] PRs to correct the transformers tokenizer so that it gives 1-to-1 the same results as the mistral_common reference implementation are very welcome!

---

The shaheerzk/text-to-rdb-queries Large Language Model (LLM) is an instruct fine-tuned version of the Mistral-7B-v0.2.

Instruction format

This format is available as a chat template via the apply_chat_template() method:

from transformers import AutoModelForCausalLM, AutoTokenizer

device = "cuda" # the device to load the model onto

model = AutoModelForCausalLM.from_pretrained("shaheerzk/text-to-rdb-queries")
tokenizer = AutoTokenizer.from_pretrained("shaheerzk/text-to-rdb-queries")

messages = [
    {"role": "user", "content": ""},
    {"role": "assistant", "content": ""},
    {"role": "user", "content": ""}
]

encodeds = tokenizer.apply_chat_template(messages, return_tensors="pt")

model_inputs = encodeds.to(device)
model.to(device)

generated_ids = model.generate(model_inputs, max_new_tokens=1000, do_sample=True)
decoded = tokenizer.batch_decode(generated_ids)
print(decoded[0])