pi-predicter

pi-predicter

A neural network model that memorizes and predicts digits of π (pi) using Fourier feature encoding. This model demonstrates the memorization capabilities of MLPs with positional encodings by learning to predict specific digits of π at given positions.

Model Description

pi-predicter is a multi-layer perceptron (MLP) that takes a position index as input and predicts the corresponding digit of π at that position. The model uses Fourier feature encoding to transform integer positions into high-dimensional representations, enabling the network to memorize up to 100,000 digits of π.

Key Features:

• Fourier Feature Encoding: 16 frequency components for positional encoding
• Deep Architecture: 3 hidden layers with 512 dimensions each
• Large Capacity: Trained on 100,000 digits of π
• Efficient Inference: Instant digit prediction at any position within training range

Model Architecture

PiPredictor(
  ├── Input: position (integer)
  ├── Fourier Encoding: 16 frequencies → 32-dim vector
  ├── MLP: 32 → 512 → 512 → 512 → 10
  └── Output: digit probability distribution (0-9)
)

Intended Use

Primary Use Case

Memorization and retrieval of π digits for positions 1-100,000 (after decimal point). The model serves as a demonstration of neural network memorization capabilities and positional encoding techniques.

Limitations

• Position Range: Only valid for positions 1-100,000 (training range)
• No Generalization: Cannot predict digits beyond training range
• Memorization Only: Not designed for mathematical computation or pattern discovery
• Position 0: Integer part (3) not included in training

How to Use

Installation

pip install torch

Quick Start

import torch
from model import PiPredictor

# Load model
checkpoint = torch.load('model.pt', map_location='cpu')
config = checkpoint['config']

# Initialize model
model = PiPredictor(
    max_pos=checkpoint['model_max_pos'],
    num_frequencies=config['num_frequencies'],
    hidden_dims=config['hidden_dims'],
    dropout=config['dropout'],
    encoding=config['encoding'],
    embedding_dim=config['embedding_dim']
)

model.load_state_dict(checkpoint['model_state'])
model.eval()

# Predict digit at position 2 (should be 4: π = 3.14159...)
position = torch.tensor([2], dtype=torch.long)
with torch.no_grad():
    logits = model(position)
    predicted_digit = torch.argmax(logits, dim=-1).item()
    
print(f"Digit at position 2: {predicted_digit}")  # Output: 4

Batch Inference

# Predict multiple positions at once
positions = torch.tensor([1, 2, 3, 4, 5, 10, 100], dtype=torch.long)
with torch.no_grad():
    logits = model(positions)
    predictions = torch.argmax(logits, dim=-1)
    
# π = 3.1415926535...
# positions 1-5: [1, 4, 1, 5, 9]
print(predictions.tolist())  # [1, 4, 1, 5, 9, 5, 9]

Technical Notes

Fourier Feature Encoding

The model uses Fourier features to transform scalar positions into high-dimensional vectors:

• 16 frequency components create a 32-dimensional embedding
• Frequencies are likely sampled from a Gaussian distribution
• This encoding enables the MLP to learn high-frequency functions of position

Memory Efficiency

Despite memorizing 100,000 digits, the model achieves this with only ~550k parameters, demonstrating the efficiency of neural networks as lookup tables for structured data.

License

This model is released under the MIT License.

---

Note: This model is intended for educational and demonstration purposes, showcasing neural network memorization capabilities rather than mathematical computation. The digits of π are deterministic and can be computed exactly using algorithms; this model demonstrates an alternative approach using machine learning techniques.