42dot_LLM-SFT-1.3B_GGUF

42dot_LLM-SFT-1.3B_GGUF

• Model Creator: 42dot
• original Model: 42dot_LLM-SFT-1.3B

Description

This repository contains the GGUF conversion and the most relevant quantizations of 42dot's 42dot_LLM-SFT-1.3B model - ready to be used with llama.cpp and similar applications.

Files

In order to allow for fine-tuning (the model has the required LLaMA architecture) the original GGUF conversion has been made available

• 42dot_LLM-SFT-1.3B.gguf

From this file, the following quantizations were derived:

• 42dot_LLM-SFT-1.3B-Q4_K_M
• 42dot_LLM-SFT-1.3B-Q5_K_M
• 42dot_LLM-SFT-1.3B-Q6_K
• 42dot_LLM-SFT-1.3B-Q8_K

(tell me if you need more)

Usage Details

Any technical details can be found on the original model card The most important ones for using this model are

• context length is 4096
• there does not seem to be a specific prompt structure - just provide the text you want to be completed

Text Completion with LLaMA.cpp

For simple inferencing, use a command similar to

./main -m 42dot_LLM-SFT-1.3B-Q8_K.gguf --temp 0 --top-k 4 --prompt "who was Joseph Weizenbaum?"

Text Tokenization with LLaMA.cpp

To get a list of tokens, use a command similar to

./tokenization -m 42dot_LLM-SFT-1.3B-Q8_K.gguf --prompt "who was Joseph Weizenbaum?"

Embeddings Calculation with LLaMA.cpp

Text embeddings are calculated with a command similar to

./embedding -m 42dot_LLM-SFT-1.3B-Q8_K.gguf --prompt "who was Joseph Weizenbaum?"

License

The original model "is licensed under the Creative Commons Attribution-NonCommercial 4.0 (CC BY-NC 4.0)" - for that reason, the same license was also chosen for the conversions found in this repository.

So, in order to be fair and give credits to whom they belong:

• the original model was created and published by 42dot
• besides quantization, no changes were applied to the model itself