ann-x-squared-approximation

ANN Approximation of y = x² — random_600_hidden6_mae_sigmoid

A single-hidden-layer feedforward neural network trained to approximate y = x² on the interval [-1, 1].

Model Details

• Architecture: Linear(1 → 6) → Sigmoid → Linear(6 → 1)
• Hidden units: 6
• Activation function: sigmoid
• Loss function used in training: mae
• Training dataset: random sampling, 600 points, 80/20 train/val split
• Optimizer: Adam, lr=0.01, full-batch gradient descent

Performance (validation set)

This model was selected as the best-performing configuration out of a 1350-run grid search sweeping dataset (10 variants), hidden-layer width (2–10), loss function (MSE/MAE/Huber), and activation function (ReLU/Tanh/Sigmoid/Leaky ReLU/ELU).

Usage

import torch
import torch.nn as nn

class SingleHiddenLayerANN(nn.Module):
    def __init__(self, hidden_units=6, activation="sigmoid"):
        super().__init__()
        self.hidden = nn.Linear(1, hidden_units)
        self.output = nn.Linear(hidden_units, 1)
        activations = {"relu": nn.ReLU(), "tanh": nn.Tanh(), "sigmoid": nn.Sigmoid(),
                        "leaky_relu": nn.LeakyReLU(0.01), "elu": nn.ELU()}
        self.activation = activations[activation]

    def forward(self, x):
        return self.output(self.activation(self.hidden(x)))

model = SingleHiddenLayerANN()
model.load_state_dict(torch.load("pytorch_model.bin", map_location="cpu"))
model.eval()

x = torch.tensor([[0.5]])
y_pred = model(x)
print(y_pred)  # should be close to 0.25 (0.5^2)

Visualizing this model

The model.onnx file in this repo can be opened directly at netron.app (drag-and-drop, no install) to see the network graph, layer shapes, and weights interactively.

Source

Full training pipeline, all 1350 experiment results, and analysis tools: see the linked GitHub repository.