PHPko-31M

PHPko-31M

A 31M-parameter PHP code-completion model, trained from scratch (random initialisation — not a fine-tune of any existing model).

It writes the next lines of PHP given the code you have typed so far. It is small enough to run comfortably on a laptop CPU.

---

Architecture

A Qwen3-style decoder-only transformer, implemented from scratch in PyTorch. It loads as a stock Qwen3ForCausalLM, so no trust_remote_code is required (needs transformers>=4.51).

Size

The tokenizer was trained on this PHP corpus rather than reused, so it packs PHP efficiently (~4.1 characters per token) and can represent any byte sequence.

Training

What it does well

• Continues PHP you have started: class bodies, method signatures, property declarations, docblocks.
• Produces syntactically valid, idiomatic modern PHP — typed signatures (: string, ?int), constructor property promotion, fluent return $this;, PSR-style formatting.
• Knows conventions of major frameworks it trained on (Symfony, Laravel, Doctrine, PHPUnit style).
• Fast: roughly 0.3 s for a short completion on a CPU — usable as live editor autocomplete.

What it does NOT do

Please read this before using it — it is a small model and these limits are real:

• It is not a chatbot. It does not follow instructions. Prompting it with "write me a function that sorts users" will not work — give it the start of code and it continues.
• It does not understand your program's logic. It writes code that looks right more reliably than code that is right. Example: asked to complete add(Money $other), it may return 0 instead of summing.
• It repeats itself. It sometimes emits the same method twice. Use a repetition penalty (~1.15) and keep max_new_tokens modest.
• No fill-in-the-middle (FIM). It only sees code before the cursor. Do not send a suffix; use prefix-only completion.
• It does not know your codebase, private APIs, or anything outside its training data.
• Not for security-sensitive code. Always review output. It can produce insecure or non-functional code.
• PHP only. Other languages will be poor.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("AnandRago/PHPko-31M")
tok = AutoTokenizer.from_pretrained("AnandRago/PHPko-31M")

prompt = "<?php\n\nclass Invoice\n{\n    private array $lines = [];\n\n    public function getTotal(): float\n    {\n        return "
ids = tok(prompt, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=64, do_sample=True,
                     temperature=0.2, top_p=0.9, repetition_penalty=1.15)
print(tok.decode(out[0], skip_special_tokens=True))

Recommended settings: temperature 0.2-0.5, top_p 0.9, repetition_penalty 1.15, max_new_tokens 48-96. Very low temperature (<0.1) makes it loop.

Training data & provenance

Trained only on permissively licensed (MIT / BSD-3) open-source PHP. GPL projects (e.g. WordPress, Drupal) were deliberately excluded so the corpus carries no copyleft obligations.

Files were filtered to remove vendor directories, tests, generated code, minified files, translation/lookup tables, and exact duplicates. License header comments were stripped.

Sources (each retains its original license)

Corpus sources. Each retains its original license.

Carbon                   MIT      https://github.com/briannesbitt/Carbon.git
FastRoute                BSD-3    https://github.com/nikic/FastRoute.git
PHP-Parser               BSD-3    https://github.com/nikic/PHP-Parser.git
Slim                     MIT      https://github.com/slimphp/Slim.git
Sylius                   MIT      https://github.com/Sylius/Sylius.git
Twig                     BSD-3    https://github.com/twigphp/Twig.git
bagisto                  MIT      https://github.com/bagisto/bagisto.git
cakephp                  MIT      https://github.com/cakephp/cakephp.git
collections              MIT      https://github.com/doctrine/collections.git
collision                MIT      https://github.com/nunomaduro/collision.git
commonmark               BSD-3    https://github.com/thephpleague/commonmark.git
composer                 MIT      https://github.com/composer/composer.git
core                     MIT      https://github.com/api-platform/core.git
csv                      MIT      https://github.com/thephpleague/csv.git
dbal                     MIT      https://github.com/doctrine/dbal.git
filament                 MIT      https://github.com/filamentphp/filament.git
flysystem                MIT      https://github.com/thephpleague/flysystem.git
framework                MIT      https://github.com/laravel/framework.git
guzzle                   MIT      https://github.com/guzzle/guzzle.git
horizon                  MIT      https://github.com/laravel/horizon.git
laminas-mvc              BSD-3    https://github.com/laminas/laminas-mvc.git
laravel                  MIT      https://github.com/laravel/laravel.git
laravel-medialibrary     MIT      https://github.com/spatie/laravel-medialibrary.git
laravel-permission       MIT      https://github.com/spatie/laravel-permission.git
livewire                 MIT      https://github.com/livewire/livewire.git
log                      MIT      https://github.com/php-fig/log.git
migrations               MIT      https://github.com/doctrine/migrations.git
monolog                  MIT      https://github.com/Seldaek/monolog.git
oauth2-server            MIT      https://github.com/thephpleague/oauth2-server.git
orm                      MIT      https://github.com/doctrine/orm.git
phpdotenv                BSD-3    https://github.com/vlucas/phpdotenv.git
phpstan-src              MIT      https://github.com/phpstan/phpstan-src.git
phpunit                  BSD-3    https://github.com/sebastianbergmann/phpunit.git
promises                 MIT      https://github.com/guzzle/promises.git
psr7                     MIT      https://github.com/guzzle/psr7.git
rector-src               MIT      https://github.com/rectorphp/rector-src.git
symfony                  MIT      https://github.com/symfony/symfony.git
symplify                 MIT      https://github.com/symplify/symplify.git
telescope                MIT      https://github.com/laravel/telescope.git
uuid                     MIT      https://github.com/ramsey/uuid.git
yii2                     BSD-3    https://github.com/yiisoft/yii2.git

Files kept per source repository

Total: 30,044 files, ~25.6M BPE tokens.

License

Apache-2.0 for the model weights. Each training source retains its own license (listed above); please respect them.