h3

h3

This model is a fine-tuned version of distilgpt2 on hearthstone dataset. GitHub repo. It achieves the following results on the evaluation set:

• Loss: 0.2782
• Exact Match: 0.2879
• Bleu: 0.9121
• Codebleu: 0.7482
• Ngram Match Score: 0.7504
• Weighted Ngram Match Score: 0.7583
• Syntax Match Score: 0.7673
• Dataflow Match Score: 0.7169
• Chrf: 93.1064

Model description

DistilGPT2 fine-tuned on HearthStone dataset for 200 epochs.
Related to dvitel/h0 but with preprocessing which anonymizes classes and function variables (Local renaming).
dvitel/h2 implements global renaming where all names are removed. Global renaming showed worse results compared to local renaming.

Example of generated code with mistake on last eval iteration (EV L - gold labels, EV P - prediction):

EV L class CLS0(MinionCard):
        def __init__(self):
                super().__init__('Darkscale Healer', 5, CHARACTER_CLASS.ALL, CARD_RARITY.COMMON, battlecry=Battlecry(Heal(2), CharacterSelector()))
        def create_minion(self, v0):
                return Minion(4, 5)
EV P class CLS0(MinionCard):
        def __init__(self):
                super().__init__('Darkscale Healer', 5, CHARACTER_CLASS.ALL, CARD_RARITY.COMMON, battlecry=Battlecry(Heal(2), CharacterSelector())
        def create_minion(self, v0):
                return Minion(4, 5)

EV L class CLS0(WeaponCard):
        def __init__(self):
                super().__init__('Fiery War Axe', 2, CHARACTER_CLASS.WARRIOR, CARD_RARITY.FREE)
        def create_weapon(self, v0):
                return Weapon(3, 2)
EV P class CLS0(WeaponCard):
        def __init__(self):
                super().__init__('Fiery War Axe', 2, CHARACTER_CLASS.WARRIOR, CARD_RARITY.FREE,
        def create_weapon(self, v0):
                return Weapon(3, 2)

EV L class CLS0(MinionCard):
        def __init__(self):
                super().__init__('Frostwolf Warlord', 5, CHARACTER_CLASS.ALL, CARD_RARITY.COMMON, battlecry=Battlecry(Give([Buff(ChangeAttack(Count(MinionSelector()))), Buff(ChangeHealth(Count(MinionSelector())))]), SelfSelector()))
        def create_minion(self, v0):
                return Minion(4, 4)
EV P class CLS0(MinionCard):
        def __init__(self):
                super().__init__('Frostwolf Warlord', 5, CHARACTER_CLASS.ALL, CARD_RARITY.COMMON, battlecry=Battlecry(Give([Buff(ChangeAttack(Count(MinionSelector(),), Buff(ChangeHealth(Count(MinionSelector()))))]),), SelfSelector()))
        def create_minion(self, v0):
                return Minion(4, 4)

EV L class CLS0(SpellCard):
        def __init__(self):
                super().__init__('Hellfire', 4, CHARACTER_CLASS.WARLOCK, CARD_RARITY.FREE)
        def use(self, v0, v1):
                super().use(v0, v1)
                v2 = copy.copy(v1.other_player.minions)
                v2.extend(v1.current_player.minions)
                v2.append(v1.other_player.hero)
                v2.append(v1.current_player.hero)
                for v3 in v2:
                        v3.damage(v0.effective_spell_damage(3), self)
EV P class CLS0(SpellCard):
        def __init__(self):
                super().__init__('Hellfire', 4, CHARACTER_CLASS.WARLOCK, CARD_RARITY.FREE,
        def use(self, v0, v1):
                super().use(v0, v1)
                v2 = copy.copy(v1.other_player.minions)
                v2.extend(v1.current_player.minions)
                for.append(v1.other_player.hero)
                for.append(v1.other_player.hero)
                for v3 in v2:
                                .damage(v0.effective_spell_damage(3), self)

Intended uses & limitations

HearthStone card code synthesis.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 17
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• num_epochs: 200
• mixed_precision_training: Native AMP

Training results

Framework versions

• Transformers 4.24.0
• Pytorch 1.13.0
• Datasets 2.6.1
• Tokenizers 0.13.1