inlp-a3-task2-ssm

gautambhetanabhotla/inlp-a3-task2-ssm

inlp-a3-task2-ssm is a machine learning model from gautambhetanabhotla. Use it for the machine learning task on the model card, and read the license before you ship it in a product.

Mamba-inspired S6 language model for next-word prediction.

Downloads · 30 days

Access

Public

Updated Mar 30, 2026

Repo size

109 MB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.pt109 MB · 100%

At a glance

Access: Public
Created: Mar 28, 2026
Updated: Mar 30, 2026
SHA: 042f3bcd

Model card

From the Hugging Face model README

Task-2 SSM Language Model

Mamba-inspired S6 language model for next-word prediction.

Training Summary

Best epoch: 6
Best validation loss: 6.396149
Best validation perplexity: 599.531929
Local checkpoint path: models/task2_ssm.pt

Training Configuration

seed: 2498374213
data:
  plain_path: data/plain.txt
  append_eos: true
  val_fraction: 0.1
  test_fraction: 0.1
  min_freq: 1
  max_vocab_size: null
train:
  seq_len: 256
  stride: 32
  batch_size: 128
  epochs: 8
  lr: 0.001
  weight_decay: 0.0
  clip_grad_norm: 1.0
model:
  d_model: 48
  d_state: 12
  n_layers: 12
  conv_kernel: 6
  dropout: 0.2
checkpoint:
  path: models/task2_ssm.pt
eval:
  batch_size: 128
  output_path: outputs/task2_ssm_eval.txt
  sample_prompts:
  - the dominion
  - we can
  - language modeling
device: auto
wandb:
  enabled: true
  project: inlp-a3-task2-ssm
  run_name: task2-ssm
huggingface:
  enabled: true
  repo_id: gautambhetanabhotla/inlp-a3-task2-ssm
  token_env: HF_TOKEN
  checkpoint_path_in_repo: task2_ssm.pt
  config_path_in_repo: training_config.yaml
  readme_path_in_repo: README.md

Created: Mar 28, 2026
Updated: Mar 30, 2026