Downloads · 30 days
0
hazemfreePalestine/haz-gpt-x1-33m
haz-gpt-x1-33m is a machine learning model from hazemfreePalestine. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Model Description This is a custom sequence-to-sequence (Seq2Seq) model built with TensorFlow, featuring stacked LSTM layers for both encoding and decoding. It uses the T5-base tokenizer to preprocess text inputs and…
Downloads · 30 days
0
Access
Public
Updated Aug 2, 2025
Repo size
—
Likes
0
Public
Click a slice to open those files.
.md2 KB · 57%
From the Hugging Face model README
Model Description This is a custom sequence-to-sequence (Seq2Seq) model built with TensorFlow, featuring stacked LSTM layers for both encoding and decoding. It uses the T5-base tokenizer to preprocess text inputs and outputs. The model is trained on a JSONL dataset containing instruction-input-output triples, enabling it to generate relevant textual responses given instructions and optional inputs. Intended Use Text generation based on instructions. Instruction-following conversational AI. Generating structured responses for NLP tasks involving input-output mappings. Research and experimentation with Seq2Seq LSTM architectures. Training Data Dataset format: JSON lines file (data.jsonl), with fields instruction, optional input, and output. Data shuffled and split into training (90%) and validation (10%). Tokenized with T5 tokenizer using padding and truncation to max lengths 50 tokens. Model Architecture Embedding layer: 128-dim embeddings. Encoder: Four stacked LSTM layers (512 units each). Decoder: Single LSTM layer with 512 units. Output: Dense layer with softmax activation over vocabulary. Custom masked loss ignoring padding tokens. Trained with Adam optimizer (learning rate 5e-5), gradient clipping. Usage Feed tokenized instruction and optional input sequences. Use beam search decoding (beam width 3) for better generation quality. Can be fine-tuned further or adapted to similar instruction-following tasks. Limitations Limited by size and diversity of training data. May not generalize well outside domain of training instructions. Beam search decoding parameters may need tuning. No explicit attention mechanism — might underperform on very long or complex input sequences. How to Use Load the model weights. Tokenize inputs using the provided tokenizer setup. Generate output sequences using beam search or greedy decoding. Citation If you use this model in your work, please cite appropriately and mention it is a custom TensorFlow Seq2Seq LSTM model trained on instruction-output JSONL data with T5 tokenizer.