Iamam2

Iamam2 overview

This is Iamam2, a successor to Iamam1, it aims to be a better base model for other LSTM Ai's.

Model Details

Model Description

Iamam2 (Which stands for I am a model), is family number 2 of the Iamam 'brand' of models, it is a BiLSTM with attention, it aims to be a good base model for other LSTMs and a good model to finetune into something useful, Iamam models are distrobued with no RHLF, finetuneing or alignment layers.

• Developed by: [DJF-on-arm]
• Model type: [BiLSTM with attention]
• Language(s) (NLP): [English]
• License: [AGPL-v3]

Uses

As a base model for finetunes and for people to train it more!

Direct Use

Hopefully, I can get IAMAM in to a consistant state of being ok

Downstream Use

This model (will, hopefully) have ok english skills and some general knowlage, which can be used as a base for other LSTM models, becuase of its Attention mechanism, it can pick out words and focus on them, useful if you need a finetuned version to look carfully at specific words and evaluate it, The 'Bi' part of the BiLSTM means it looks at a sentense forward and then backwards, meaning it misses less infomation and can technically proform more complex math, although, it currently has not been trained on any math data sets.

Out-of-Scope Use

This will probally be a terriable chatbot, mathbot and/or englishbot

Bias, Risks, and Limitations

Biases:

• I think it has none, if it does, please report it!

Risks:

• Its fairly stupid, so please don't use it for critical infomation

Limitations:

• Its a 12M param BiLSTM, so, it probally has a very low limit of capibility compared to bigger models.

Recommendations

Please fact check it if it says something obviously wrong.

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

How to Get Started with the Model

Use the code below to get started with the model.

[Insert procrastination]

Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

Training Hyperparameters

• Training regime: Mixed Bfloat 16
• Vocab size: [Varies per version]
• Optimiser: [Varies per version]
• Max output length: [Varies per version]
• Embed dim: [Varies per version]
• Latent dim: [Varies per version]
• Max LR: [Varies per version]
• Init LR: [Varies per version]

Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: CPU, AMD, AM4
• Hours used: Total will be calculated when Iamam 3 starts
• Cloud Provider: N/A
• Compute Region: England
• Carbon Emitted: Total will be calculated when Iamam 3 starts

Hardware

For training a Rysan 5 5600, paired with 16GB of DDR4 ram was used.

Software

Tensorflow, python and VS:code was used (windows 11 Pro (IAMAM2 started after windows build 25H2) is the OS).

Model Card Contact

To be done at a later date.