text-classification-with-transformer

keras-io/text-classification-with-transformer

text-classification-with-transformer is a text classification model from keras-io. Use it when you need a label for a piece of text. It is set up for tf-keras.

Implement a Transformer block as a Keras layer and use it for text classification.

Downloads · 30 days

4% of all-time downloads

All-time downloads

455

Public

Repo size

3.3 MB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.data-00000-of-000012.6 MB · 79%

At a glance

Task: Text Classification
Library: tf-keras
Access: Public
Created: Jun 8, 2022
Updated: Jul 5, 2024
SHA: c90864c3

Try a prompt

Spaces

Model card

From the Hugging Face model README

Model description

Implement a Transformer block as a Keras layer and use it for text classification.

For details on the implementation, please see the original link on keras

Full credits to: Apoorv Nandan

Training and evaluation data

The model is trained and evaluated on the IMDB dataset for sentiment analysis.

Details on the dataset can be found on keras

Note that the keras dataset is already tokenized, so the model doesn't have an associated tokenizer. Since the hosted text classification pipeline only accepts text as input, it is currently broken.

For a live demo of the model using my own tokenizer, please use the space

Training hyperparameters

The following hyperparameters were used during training:

name	learning_rate	decay	beta_1	beta_2	epsilon	amsgrad	training_precision
Adam	0.0010000000474974513	0.0	0.8999999761581421	0.9990000128746033	1e-07	False	float32

Training Metrics

Epochs	Train Loss	Train Accuracy	Validation Loss	Validation Accuracy
1	0.385	0.819	0.298	0.868
2	0.198	0.925	0.333	0.863

Model Plot

<details> <summary>View Model Plot</summary>

Model Image

</details>

Task: Text Classification
Library: tf-keras
Created: Jun 8, 2022
Updated: Jul 5, 2024