uzbek-operator-rag

Uzbek Operator RAG — Retrieval Encoder

A BERT-style transformer encoder (42.1M params, 8 layers, 512 hidden, trained from a custom BPE tokenizer) pretrained with Masked Language Modeling and fine-tuned with SimCSE for dense retrieval. Built for a retrieval-augmented operator chatbot — no LangChain or LlamaIndex. Architecture, pretraining, and full RAG pipeline code: uzbtrust/uzbek-operator-rag on GitHub.

Pipeline this encoder feeds into

Dense embeddings from this encoder are fused with TF-IDF sparse retrieval (score fusion, 0.5/0.5) before generation. See the GitHub repo for the retriever and generator code.

Files

• model.pt — {"epoch", "loss", "config", "model": state_dict}
• config.json — architecture and training summary

Usage

The encoder class and BPE tokenizer live in the GitHub repo, not in this repository:

git clone https://github.com/uzbtrust/uzbek-operator-rag
cd uzbek-operator-rag

import torch
from model.transformer import Encoder  # see repo for exact constructor args

ckpt = torch.load("model.pt", map_location="cpu")
model = Encoder(**ckpt["config"]["model"])
model.load_state_dict(ckpt["model"])
model.eval()

The tokenizer is trained with tokenizer/train_tokenizer.py in the GitHub repo; it is not bundled in this model repository.

License

MIT, matching the GitHub repository.