BanBTP-comparison

BanBTP Model Family

All BanBTP models are trained on the BanBTP dataset. this is internet data. V5 however is trained on the BanBTP distilled dataset (from various models), so model will be slightly better than V4,3,2 and 1 but V6 will be slightly better than V5. So a edited response could be: "All models from V1-4 are based on a closed source BanBTP dataset" while V5 is based on distilled dataset from all these models." V6 will be based on V5 distilled, but will not use the BanBTP dataset. instead, we will use V5 and finetune it on BanBTP 124M as that one was the highest quality as well as GPT2-large and tinyllama 1.1b and a updated higher quality BanBTP dataset.

Please note: V3-V4 are trained on same BanBTP dataset, however 124M was finetuned a lot better than the V3-4. V3-4 are more of test models than anything, to experiment to see if BanBTP can be well on smaller sizes. Turns out not.

V5 is a mini BanBTP 124M. V6 will improve upon V5's training data and be more like BanBTP 124M in the sense it aims to beat it while being a tad bit smaller!

V7 completely rewrites BanBTP. It is trained on BanRAG dataset. This model focuses BanBTP on RAG forcing the model to think logically in RAG making the RAG the data. It is a experimental model more than anything.

Though, BanBTP models are pretty experimental anyways. :P

V8 will aim to improve upon V7 and V6's core arch. by merging them together. V7 distilled into V8 and model given 76M parameters.

V9 is the last BanBTP model. (It will have tool calling, RAG and conversational.)> {EDIT: it will not have tool calling or RAG. it will just be conversational.}

EDIT for V7: As it turns out, V7 is unable to do RAG properly, so V8 will be completely rebuilt not on RAG but just on being a chatbot.