LLM_is_LinearRegression

LLM_is_LinearRegression model is a Single Shot, Non-Autoregressive, Text-Generation model without even using Neural Networks, GPUs, Attention Mechanism and it only uses basic multivariate linear regression with decision boundaries and inspired from the facts that classification is a form of regression having decision boundaries. This is ahead of time model as when models like Jev seems like trying to do non autoregressive for classification task, This model does Text Generation with non autoregressive way. Large Language Model (LLM) is Linear Regression. This statement is not shared here without any evidence as This repository contains a reproducible Notebook named as LLM_is_Linear_Regression.ipynb , therefore, anyone can run this notebook and verify that statement.

Benchmark (on Train set of main split of GSM8K dataset):

BLEU Score is 93%
Exact Match (EM) metrics is 100%

Artificial Intelligence (AI), Machine Learning, Deep Learning means Neural Network are evolved from Linear Regression so it may possible that achieving LLM is possible by using Linear Regression as Logistic Regression which is backbone of classification is also evolved from Linear Regression itself, therefore, This model is the attempt to find the better and simplest solution using the backbone of modern AI means Linear Regression.

Solving Large Language model using Linear Regression in Machine Learning using library like scikit-learn do these five things:-

1. It changes the timeline to before Transformer Era or precisely before attention Era like who needs expensive GPU to run the LLMs if they can do it using machine learning only in the personal computer.

2. It's Lightening fast so no comparison with existing Transformer based methods like Next word prediction and if one wants to compare then this method would be atleast 1000% times faster as believed.

3. Linear Regression is determinstic, Like we need AI which can be debugged and get fixed rather than probablistic for critical task like robotic surgery and code generation.

4. Generalisation happens when embedding comes in like previously also Word2Vec or Glove embeddings used for transfer learning and generalisation perspective so it can be done here also if required.

5. No hallucination and everytime same outcome because its deterministic.

Use below code to see the predictions.

from transformers import AutoModel

model = AutoModel.from_pretrained("Kratim-Budhimata/LLM_is_LinearRegression", trust_remote_code=True, device_map="auto")

prompt=['Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?']

model.predict(prompt)

### Outcome should be: ['1/3rd circled = = <<48/2=24>>24 clips in may. natalia sold 48+24 = <<48+24=72>>72 clips altogether in april and may. #### 72']

This is true democratisation of AI and Only magic available in this notebook is considered Y axis means dependent variable as array of floats which gives luxury to generate prediction and later decision boundary logic did everything remaining.

So Text Generation is solved using this way, but it provides vision to solve image and video generation also by the same way because Fundamentally Generation is happening at Y axis which is already a dependent variable so for example if its Text-to-Text or Text-to-Image or Text-to-video generation, In all three scenarios, Y axis has text or image or video so its already dependent variable, means you map Text or Image or Video as dependent variable, model does not discriminate to generate the outcome,

And if its Text-Image-to-Image Generation or Text-Video-to-Video generation then in that case also X axis would have two features (Text and Image, OR, Text and Video) which will get concatenated before going into the model and Y axis already have Image or Video so these usecases also can be solved this way, therefore, Most Generative AI applications can be solved using this way.

As believed, people will say that Generative AI is Linear Regression application or use cases in near future.

Common Question:

1. If it is machine learning based then why integrated with Transformer

Answer: Modern AI Community has tech stack in transformer so if it can be integrated with existing tech then it will have less friction for developers, otherwise reproducible notebook is here to implement from scratch for everyone.

Feedback and Queries: If you have any queries or feedback do reach out to us at-

Email- [email protected]

Linkedin: https://in.linkedin.com/in/aviral-vijay-299b5886