reformer-character-domain-generator

Domain Name Generator - Reformer Character-Level Model

A character-level Reformer model trained to generate domain names based on descriptive tags. The model takes a set of content and style tags as input and generates appropriate, creative domain names.

Model Description

This model is a fine-tuned version of google/reformer-enwik8 specifically adapted for domain name generation. It uses a pure tag-based approach where both content descriptors (e.g., "tech", "health") and style descriptors (e.g., "modern", "minimal") are treated as equal tags.

Key Features

• Character-level generation: Generates domains character by character for maximum flexibility
• Tag-based prompting: Uses 3-4 descriptive tags to guide generation
• Style-aware: Understands style tags like "modern", "minimal", "playful"
• Position-independent: Tag order doesn't matter due to training-time shuffling

Model Details

• Architecture: Reformer with LSH attention
• Base Model: google/reformer-enwik8
• Model Size: ~597M parameters
• Vocabulary Size: 258 (byte-level encoding)
• Max Sequence Length: 256 characters
• Hidden Size: 1024
• Layers: 12
• Attention Heads: 8

Training Details

Training Data

• Primary Dataset: 250k real domains from BrandBucket
• Synthetic Dataset: 1.75M AI-generated domains
• Total Examples: ~2M domains
• Data Split: 80% synthetic, 20% real

Training Configuration

• Epochs: 5
• Batch Size: 256 (128 × 2 gradient accumulation)
• Learning Rate: 5e-05
• Tag Dropout: 10%
• Style Tag Probability: 30%
• Hardware: NVIDIA H100 GPU
• Training Time: 17.6 hours

Training Results

• Final Training Loss: 1.1113
• Best Validation Loss: 0.9716
• Loss Reduction: 75%
• Training Stability: std=0.0014 (very stable)

Intended Use

Primary Use Cases

• Generate domain names for startups and businesses
• Brainstorm creative domain ideas based on keywords
• Explore domain variations with different styles

Input Format

tags: tag1;tag2;tag3 domain:

Supported Tags

Content Tags (examples):

• tech, ai, startup, app, software
• health, wellness, fitness, medical
• eco, green, sustainable, organic
• fashion, beauty, style, boutique
• food, restaurant, cafe, delivery

Style Tags:

• modern - Clean, contemporary
• classic - Traditional, timeless
• playful - Fun, casual
• bold - Strong, impactful
• elegant - Sophisticated, refined
• techy - Technical, digital
• eco - Environmental, green
• luxury - Premium, high-end
• minimal - Simple, short
• creative - Artistic, unique
• professional - Business-oriented
• casual - Relaxed, informal
• trendy - Current, fashionable
• simple - Straightforward
• unique - Distinctive

Usage

With Transformers Library

from transformers import ReformerModelWithLMHead, AutoTokenizer
import torch

# Load model
model = ReformerModelWithLMHead.from_pretrained("humbleworth/reformer-character-domain-generator")
model.eval()

# Character encoding (Reformer standard)
def encode_text(text):
    return [c + 2 for c in text.encode('utf-8')]

def decode_ids(ids):
    return bytes([max(0, id - 2) for id in ids if id > 2]).decode('utf-8', errors='ignore')

# Generate domain
prompt = "tags: tech;startup;modern domain:"
input_ids = torch.tensor([encode_text(prompt)])

with torch.no_grad():
    output = model.generate(
        input_ids,
        max_new_tokens=50,
        temperature=1.2,
        top_p=0.95,
        do_sample=True,
        pad_token_id=0,
        eos_token_id=2
    )

generated = decode_ids(output[0].tolist())
domain = generated.split("domain:")[-1].strip()
print(f"Generated: {domain}")

Generation Parameters

• Temperature: 1.2 (recommended for creativity)
• Top-p: 0.95
• Max Length: 50 tokens after prompt

Examples

Input → Output Examples

tags: tech;startup;ai → techflow.ai
tags: eco;sustainable;modern → greenleaf.eco
tags: health;wellness;minimal → purelife.health
tags: fashion;luxury;elegant → velvetrose.com
tags: food;delivery;playful → snackdash.io

Limitations

• Best results with 3-4 tags (trained range)
• May occasionally generate non-standard TLDs
• Domain availability not guaranteed
• Works best with English keywords

Ethical Considerations

• Generated domains should be checked for trademark conflicts
• May reflect biases present in training data
• Should not be used to generate misleading or deceptive domains

Model Card Contact

For questions or issues, please open an issue in the repository.

Citation

If you use this model, please cite:

@software{domain_generator_reformer,
  title = {Domain Generator - Character-Level Reformer},
  year = {2025},
  publisher = {HuggingFace},
  url = {https://huggingface.co/humbleworth/reformer-character-domain-generator}
}

Changelog

• v1.0 (2024-01): Initial release
  • 5 epochs training on combined dataset
  • 0.9716 validation loss
  • Stable generation quality