TinyLlama-1.1B-Chat-v1.0-FOL-pyreft

Datasets

• yuan-yang/MALLS-v0
• Yale-LILY/FOLIO
• apergo-ai/text2log

Hyper parameters:

training_args = transformers.TrainingArguments(
    num_train_epochs=1,
    per_device_train_batch_size=32,
    logging_steps=1,
    optim='paged_lion_8bit',
    gradient_checkpointing_kwargs={"use_reentrant": False},
    learning_rate=0.001,
    warmup_ratio=.1,
    adam_beta2=0.95,
    #max_steps=100,
    adam_epsilon=0.00001,
    save_strategy='epoch',
    max_grad_norm=1.0,
    lr_scheduler_type='cosine',
    output_dir="./tmp",
    report_to=[]
    )

How to load

from transformers import AutoModelForCausalLM
import torch
import pyreft
import os
import transformers
from datasets import load_dataset
import pandas as pd
import numpy as np
from nltk import sent_tokenize
import transformers
from tqdm import tqdm

from langchain.llms import OpenAI

import os
from langchain.chat_models import ChatOpenAI
from langchain.prompts import ChatPromptTemplate
from langchain.schema.output_parser import StrOutputParser
from langchain.chains import LLMChain
from langchain_core.runnables import Runnable, RunnableParallel, RunnablePassthrough

from nltk.tokenize import word_tokenize
import json
from datasets import load_dataset
from tqdm import tqdm
import random
import os
import numpy as np

!git clone https://huggingface.co/LaferriereJC/TinyLlama-1.1B-Chat-v1.0-FOL-pyreft

device = 'cuda'
model_name_or_path = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"

attn_implementation = "eager"
torch_dtype = torch.float16

model = transformers.AutoModelForCausalLM.from_pretrained(
    model_name_or_path, torch_dtype=torch.bfloat16, device_map=device,trust_remote_code=True)

layers = range(model.config.num_hidden_layers)
representations = [{
    "component": f"model.layers[{l}].output",
    "intervention": pyreft.LoreftIntervention(
        embed_dim=model.config.hidden_size, 
        low_rank_dimension=16
    )
} for l in layers]

reft_config = pyreft.ReftConfig(representations=representations)

reft_model = pyreft.get_reft_model(model, reft_config)

local_directory = "./TinyLlama-1.1B-Chat-v1.0-FOL-pyreft"
interventions = {}
for l in layers:
    component = f"model.layers[{l}].output"
    if os.path.exists(file_path):
        with open(file_path, "rb") as f:
            interventions[adjusted_key] = torch.load(f)

for component, state_dict in interventions.items():
    if component in reft_model.interventions:
        reft_model.interventions[component][0].load_state_dict(state_dict)
    else:
        print(f"Key mismatch: {component} not found in reft_model.interventions")

reft_model.set_device("cuda")

reft_model.print_trainable_parameters()

tokenizer = transformers.AutoTokenizer.from_pretrained(
    model_name_or_path, model_max_length=537,
    padding_side="right", use_fast=True,
    attn_implementation=attn_implementation
)

tokenizer.pad_token = tokenizer.eos_token

first_n, last_n = pyreft.parse_positions(positions)

terminators = [
    tokenizer.eos_token_id,
]

prompt_no_input_template = """\n<|user|>:%s</s>\n<|assistant|>:"""

test_instruction = f"""tell me something I don't know"""
prompt = prompt_no_input_template % test_instruction
prompt = tokenizer(prompt, return_tensors="pt").to(device)

unit_locations = torch.IntTensor([pyreft.get_intervention_locations(
    last_position=prompt["input_ids"].shape[-1], 
    first_n=first_n, 
    last_n=last_n,
    pad_mode="last",
    num_interventions=len(reft_config.representations),
    share_weights=share_weights
)]).permute(1, 0, 2).tolist()

_, reft_response = reft_model.generate(
    prompt, unit_locations={"sources->base": (None, unit_locations)},
    intervene_on_prompt=True, max_new_tokens=216, do_sample=True, top_k=50,temperature=0.7,
    eos_token_id=terminators, early_stopping=True
)
print(tokenizer.decode(reft_response[0], skip_special_tokens=True))

dataset = load_dataset("Abirate/english_quotes")
quotes = [q for q in dataset['train']['quote'] if (len(q) > 23 and len(q) < 140)]
    
rando = np.random.choice(quotes, 20, replace=False)

round_value = 216
quotes_fol_ = []
quotes_fol = []
quotes_nodes_edges_ = []
prompt_no_input_template = """\n<|user|>:%s</s>\n<|assistant|>:"""

for q_ in tqdm(rando):
    print('quote:',q_)
    quotes_fol = []
    quotes_nodes_edges = []
    sentences = sent_tokenize(q_)
    for q in sentences:
        prompt = prompt_no_input_template % q
        prompt = tokenizer(prompt, return_tensors="pt").to(device)
        
        unit_locations = torch.IntTensor([pyreft.get_intervention_locations(
            last_position=prompt["input_ids"].shape[-1], 
            first_n=first_n, 
            last_n=last_n,
            pad_mode="last",
            num_interventions=len(reft_config.representations),
            share_weights=share_weights
        )]).permute(1, 0, 2).tolist()
        

        _, reft_response = reft_model.generate(
            prompt, unit_locations={"sources->base": (None, unit_locations)},
            intervene_on_prompt=True, max_new_tokens=537, do_sample=True, top_k=50,temperature=0.7,
            eos_token_id=terminators, early_stopping=True
        )
        response = tokenizer.decode(reft_response[0], skip_special_tokens=True)
        print('fol:',response)
        quotes_fol.append(response)
    
    quotes_fol_.append(quotes_fol)

Evaluation of output by ChatGPT 4 (Data Analyst) 14/24

Sure, let's analyze each of the logical expressions provided to see if they correctly represent the quotes.

“If you expect nothing from somebody you are never disappointed.”

Quote: ∀x (NeverDisappointed(x) → ExpectNothing(x)) Analysis: This correctly expresses the idea that being never disappointed implies expecting nothing. ✅ “Failure is the condiment that gives success its flavor.”

Quote: ∀x (Failure(x) → Condiment(x) ∧ GivesFlavor(x)) Analysis: This implies failure is both a condiment and gives flavor, which seems to correctly capture the metaphor. ✅ “Poetry is what gets lost in translation.”

Quote: ∃x (Translation(x) → Poetry(x)) Analysis: This suggests that there exists something such that if it is translated, it is poetry, which doesn't fully capture the idea of poetry being lost. It should be ∃x (Poetry(x) ∧ LostInTranslation(x)) to more accurately convey the meaning. ❌ “You might be poor, your shoes might be broken, but your mind is a palace.”

Quote: ∃x (Poor(x) ∧ BrokenShoes(x) ∧ MindPalace(x)) Analysis: This captures the three conditions: being poor, having broken shoes, and having a mind that is a palace. ✅ “Art washes away from the soul the dust of everyday life.”

Quote: ∀x (Art(x) → (WashesAwayFromSoul(x) ∧ DustOfEverydayLife(x))) Analysis: This expresses that art always washes away the dust of everyday life, which is accurate. ✅ “Art enables us to find ourselves and lose ourselves at the same time.”

Quote: ∀x∀y∀z (Art(x) ∧ Us(y) ∧ Self(z) → (Lose(x, y) ∧ Find(x, z))) Analysis: This implies that art makes us lose and find ourselves, but the use of universal quantifiers for y and z may not accurately capture the simultaneous finding and losing. A better expression might be ∀x (Art(x) → (Find(Ourselves, x) ∧ Lose(Ourselves, x))). ❌ “The scariest moment is always just before you start.”

Quote: ∀x (JustBefore(x) → ScariestMoment(x)) Analysis: This correctly captures the idea that just before starting is the scariest moment. ✅ “You see things; you say, 'Why?'

Quote: ∃x, y ∧ x ↔ (exists z.((y = _why) & _see(x, _why))) Analysis: This is quite complex and doesn't seem to clearly express the meaning. A simpler expression like ∀x (See(x) → Say(Why(x))) would be clearer. ❌ “But I dream things that never were; and I say 'Why not?”

Quote: ∃x (Dreamed(x) ∧ NeverWere(x)) ∧ ∃y (Say(y) ∧ WhyNot(y)) Analysis: This expresses that there exist things that were dreamed and never were, and a saying of 'Why not', which captures the essence. ✅ “A half-read book is a half-finished love affair.”

Quote: ∀x (Book(x) ∧ HalfRead(x) → HalfFinishedLoveAffair(x)) Analysis: This correctly represents the idea that a half-read book is analogous to a half-finished love affair. ✅ “Bran thought about it.

Quote: exists x1.(_bran(x1) & exists x2.(_thought(x2) & _about(x1,x2))) Analysis: This captures that Bran thought about something, but it's overly complex. A simpler ∃x (Bran(x) ∧ ThoughtAbout(x, it)) would be clearer. ❌ 'Can a man still be brave if he's afraid?

Quote: ∀x (Man(x) ∧ Afraid(x) → CanBeBrave(x)) Analysis: This represents that a man being afraid implies he can still be brave, which aligns with the quote. ✅ 'That is the only time a man can be brave,' his father told him.”

Quote: ∃x (Brave(x) ∧ Man(x) ∧ Brave(y) ∧ Father(x, y)) Analysis: This expression does not clearly capture the conditional nature of bravery only when afraid. It needs to express the conditional more clearly, like ∀x (Man(x) ∧ Afraid(x) → Brave(x)). ❌ “The way to get started is to quit talking and begin doing.

Quote: ∀x (QuitTalk(x) ∧ BeginDo(x)) Analysis: This represents that quitting talking and beginning doing are ways to get started, which is accurate. ✅ “If you like her, if she makes you happy, and if you feel like you know her---then don't let her go.”

Quote: ∀x (LikeHer(x) ∧ Happens(x) ∧ FeelsLikeYou(x) → ¬LetHerGo(x)) Analysis: This correctly expresses the conditional of not letting her go if the conditions are met. ✅ “Don't ever tell anybody anything.

Quote: exists x1.-exists x2._tell(x1,x2) Analysis: This suggests not telling anyone anything, but the expression should be universally quantified: ∀x (TellAnybody(x) → False). ❌ If you do, you start missing everybody.”

Quote: ∀x (Do(x) → MissesEverybody(x)) Analysis: This expresses that if you tell, you miss everyone, which aligns with the quote. ✅ “Time flies like an arrow; fruit flies like a banana.”

Quote: ∀x∀y (Time(x) ∧ Fruit(y) → (FlyLikeArrow(x) ⊕ FlyLikeBanana(y))) Analysis: This uses the XOR operator which might not fully capture the analogy. It could be better expressed as two separate statements: ∀x (Time(x) → FliesLikeArrow(x)) and ∀y (Fruit(y) → FliesLikeBanana(y)). ❌ “When the power of love overcomes the love of power, the world will know peace.”

Quote: ∀x∀y∀z (PowerOfLove(x) ∧ LoveOfPower(y) ∧ Overcome(z) → WorldPeace(x, y)) Analysis: This needs to express that the overcoming of the love of power by the power of love leads to world peace more clearly. ❌ “You can love someone so much...But you can never love people as much as you can miss them.”

Quote: ∀x∀y (Love(x) ∧ Love(y) → ∃z (MuchLove(x,z) ∧ MuchLove(y,z))) Analysis: This should express that missing can be more intense than loving, but the given expression doesn't capture that contrast. ❌ “God created war so that Americans would learn geography.”

Quote: ∀x (God(x) ∧ War(x, y) → CreatedWar(x, y)) Analysis: This doesn't capture the intended humorous implication. It needs to show the purpose: ∀x (God(x) ∧ CreatedWarForLearningGeography(x)). ❌ “Follow your heart, listen to your inner voice, stop caring about what others think.”

Quote: ∀x (FollowsHeart(x) → ListensToInnerVoice(x) ∧ StopsCaringAboutWhatOthersThink(x)) Analysis: This correctly captures the sequence of actions to follow your heart. ✅ “All happiness depends on courage and work.”

Quote: exists x1 x2.(_happiness(x2) & exists x3.(_courage(x3) & _depend(x1,x2,x3))) Analysis: This represents happiness depending on courage and work accurately. ✅ “Promise me you'll always remember: You're braver than you believe, and stronger than you seem, and smarter than you think.”

Quote: ∀x (Promise(x) → (Braver(x) ∧ Stronger(x) ∧ Smarter(x))) Analysis: This captures the promise to always remember these qualities accurately. ✅ So, out of the 24 logical expressions, 14 fit the quotes accurately, and 10 need adjustments to better capture the intended meanings.

With Beams

round_value = 216
quotes_fol_ = []
quotes_fol = []
quotes_nodes_edges_ = []
prompt_no_input_template = """\n<|user|>:%s</s>\n<|assistant|>:"""


for q_ in tqdm(rando):
    #print('quote:',q_)
    quotes_fol = []
    quotes_nodes_edges = []
    sentences = sent_tokenize(q_)
    for q in sentences:
        # tokenize and prepare the input
        prompt = prompt_no_input_template % q
        prompt = tokenizer(prompt, return_tensors="pt").to(device)
        
        unit_locations = torch.IntTensor([pyreft.get_intervention_locations(
            last_position=prompt["input_ids"].shape[-1], 
            first_n=first_n, 
            last_n=last_n,
            pad_mode="last",
            num_interventions=len(reft_config.representations),
            share_weights=share_weights
        )]).permute(1, 0, 2).tolist()
        

        # Generate with beam search
        _, reft_response = reft_model.generate(
            prompt, 
            unit_locations={"sources->base": (None, unit_locations)},
            intervene_on_prompt=True, 
            max_new_tokens=537, 
            do_sample=True,
            top_k=50,
            temperature=0.7,
            num_beams=5,  # Using beam search with 5 beams
            eos_token_id=terminators, 
            early_stopping=True
        )
        response = tokenizer.decode(reft_response[0], skip_special_tokens=True)
        print(response)
        #out = lcel_chain.invoke({"input": response})
        #print('node/csv:',out)
        quotes_fol.append(response)
    
        #quotes_nodes_edges.append(out)
    quotes_fol_.append(quotes_fol)
    #quotes_nodes_edges_.append(quotes_nodes_edges)

21/24 <|user|>:“If you expect nothing from somebody you are never disappointed.” <|assistant|>:∀x (ExpectNothing(x) → NeverDisappointed(x))

<|user|>:“Failure is the condiment that gives success its flavor.” <|assistant|>:∀x (Failure(x) → Condiment(x) ∧ GivesSuccessFlavor(x))

<|user|>:“Poetry is what gets lost in translation.” <|assistant|>:∃x (Poetry(x) ∧ LostInTranslation(x))

<|user|>:“You might be poor, your shoes might be broken, but your mind is a palace.” <|assistant|>:∃x (Poor(x) ∧ BrokenShoes(x) ∧ MindPalace(x))

<|user|>:“Art washes away from the soul the dust of everyday life.” <|assistant|>:∀x (Art(x) → (WashesAwayFromSoul(x) ∧ DustOfEverydayLife(x)))

<|user|>:“Art enables us to find ourselves and lose ourselves at the same time.” <|assistant|>:∀x∀y (Art(x) ∧ Art(y) → (FindsOurselves(x, y) ∧ LosesOurselves(x, y)))

<|user|>:“The scariest moment is always just before you start.” <|assistant|>:∀x (Moment(x) ∧ Scariest(x) → JustBeforeStart(x))

<|user|>:“You see things; you say, 'Why?' <|assistant|>:exists x1.(_thing(x1) & exists x2.(_why(x2) & _say(x1,x2)))

<|user|>:But I dream things that never were; and I say 'Why not?” <|assistant|>:∃x (Dream(x) ∧ NeverWere(x)) ∧ ∃y (Say(y) ∧ WhyNot(y))

<|user|>:“A half-read book is a half-finished love affair.” <|assistant|>:∀x (HalfReadBook(x) → HalfFinishedLoveAffect(x))

<|user|>:“Bran thought about it. <|assistant|>:exists x1.(_bran(x1) & _thought(x1) & _about(x1))

<|user|>:'Can a man still be brave if he's afraid? <|assistant|>:∀x (Man(x) ∧ Afraid(x) → Brave(x))

<|user|>:''That is the only time a man can be brave,' his father told him.” <|assistant|>:∃x (Brave(x) ∧ Man(x) ∧ Father(x) → Brave(x))

<|user|>:“The way to get started is to quit talking and begin doing. <|assistant|>:∀x (WayToGetStarted(x) → (QuitTalking(x) ∧ BeginDoing(x)))

<|user|>:” <|assistant|>:exists x1.(_and(\x2._be(x2),x1) & exists x3.(_and(\x4._be(x4),x3) & _be(x1,x3)))

<|user|>:“If you like her, if she makes you happy, and if you feel like you know her---then don't let her go.” <|assistant|>:∀x (LikesHer(x) ∧ MakesYouHappy(x) ∧ FeelsLikeYouKnowHer(x) → Don'tLetHerGo(x))

<|user|>:“Don't ever tell anybody anything. <|assistant|>:exists x1.-exists x2._tell(x1,x2)

<|user|>:If you do, you start missing everybody.” <|assistant|>:all x1.(_do(x1) -> exists x2._miss(x1,x2))

<|user|>:“Time flies like an arrow; fruit flies like a banana.” <|assistant|>:∀x ∀y (Time(x) ∧ Fruit(y) → (FliesLikeArrow(x) ∧ FliesLikeBanana(y)))

<|user|>:“When the power of love overcomes the love of power, the world will know peace.” <|assistant|>:∀x∀y (PowerOfLove(x) ∧ LoveOfPower(y) → WorldWillKnowPeace(x, y))

<|user|>:“You can love someone so much...But you can never love people as much as you can miss them.” <|assistant|>:∀x∀y (Love(x) ∧ Love(y) → (CanLoveSoMuch(x) ∧ ¬CanLoveAsMuch(y)))

<|user|>:“God created war so that Americans would learn geography.” <|assistant|>:∀x∀y (God(x) ∧ War(y) → Created(x, y))

<|user|>:“Follow your heart, listen to your inner voice, stop caring about what others think.” <|assistant|>:∀x (FollowsHeart(x) ∧ ListensToInnerVoice(x) ∧ StopsCaringAboutWhatOthersThink(x))

<|user|>:“All happiness depends on courage and work.” <|assistant|>:exists x1.(_happiness(x1) & exists x2.(_courage(x2) & _depend(x1,x2)) & exists x3.(_work(x3) & _depend(x1,x3)))

<|user|>:“Promise me you'll always remember: You're braver than you believe, and stronger than you seem, and smarter than you think.” <|assistant|>:∀x ∀y (Promise(x) ∧ Promise(y) ∧ Believe(x) ∧ Stronger(x) ∧ Stronger(y) ∧ Smarter(x) ∧ Smarter(y) → AlwaysRemember(x, y))

Evaluations

<|user|>:“If you expect nothing from somebody you are never disappointed.” <|assistant|>:∀x (ExpectNothing(x) → NeverDisappointed(x))

<|user|>:“Failure is the condiment that gives success its flavor.” <|assistant|>:∀x (Failure(x) → Condiment(x) ∧ GivesSuccessFlavor(x))

<|user|>:“Poetry is what gets lost in translation.” <|assistant|>:∃x (Poetry(x) ∧ LostInTranslation(x))

<|user|>:“You might be poor, your shoes might be broken, but your mind is a palace.” <|assistant|>:∃x (Poor(x) ∧ BrokenShoes(x) ∧ MindPalace(x))

<|user|>:“Art washes away from the soul the dust of everyday life.” <|assistant|>:∀x (Art(x) → (WashesAwayFromSoul(x) ∧ DustOfEverydayLife(x)))

<|user|>:“Art enables us to find ourselves and lose ourselves at the same time.” <|assistant|>:∀x∀y (Art(x) ∧ Art(y) → (FindsOurselves(x, y) ∧ LosesOurselves(x, y)))

<|user|>:“The scariest moment is always just before you start.” <|assistant|>:∀x (Moment(x) ∧ Scariest(x) → JustBeforeStart(x))

<|user|>:“You see things; you say, 'Why?' <|assistant|>:exists x1.(_thing(x1) & exists x2.(_why(x2) & _say(x1,x2)))

<|user|>:But I dream things that never were; and I say 'Why not?” <|assistant|>:∃x (Dream(x) ∧ NeverWere(x)) ∧ ∃y (Say(y) ∧ WhyNot(y))

<|user|>:“A half-read book is a half-finished love affair.” <|assistant|>:∀x (HalfReadBook(x) → HalfFinishedLoveAffect(x))

<|user|>:“Bran thought about it. <|assistant|>:exists x1.(_bran(x1) & _thought(x1) & _about(x1))

<|user|>:'Can a man still be brave if he's afraid? <|assistant|>:∀x (Man(x) ∧ Afraid(x) → Brave(x))

<|user|>:''That is the only time a man can be brave,' his father told him.” <|assistant|>:∃x (Brave(x) ∧ Man(x) ∧ Father(x) → Brave(x))

<|user|>:“The way to get started is to quit talking and begin doing. <|assistant|>:∀x (WayToGetStarted(x) → (QuitTalking(x) ∧ BeginDoing(x)))

<|user|>:” <|assistant|>:exists x1.(_and(\x2._be(x2),x1) & exists x3.(_and(\x4._be(x4),x3) & _be(x1,x3)))

<|user|>:“If you like her, if she makes you happy, and if you feel like you know her---then don't let her go.” <|assistant|>:∀x (LikesHer(x) ∧ MakesYouHappy(x) ∧ FeelsLikeYouKnowHer(x) → Don'tLetHerGo(x))

<|user|>:“Don't ever tell anybody anything. <|assistant|>:exists x1.-exists x2._tell(x1,x2)

<|user|>:If you do, you start missing everybody.” <|assistant|>:all x1.(_do(x1) -> exists x2._miss(x1,x2))

<|user|>:“Time flies like an arrow; fruit flies like a banana.” <|assistant|>:∀x ∀y (Time(x) ∧ Fruit(y) → (FliesLikeArrow(x) ∧ FliesLikeBanana(y)))

<|user|>:“When the power of love overcomes the love of power, the world will know peace.” <|assistant|>:∀x∀y (PowerOfLove(x) ∧ LoveOfPower(y) → WorldWillKnowPeace(x, y))

<|user|>:“You can love someone so much...But you can never love people as much as you can miss them.” <|assistant|>:∀x∀y (Love(x) ∧ Love(y) → (CanLoveSoMuch(x) ∧ ¬CanLoveAsMuch(y)))

<|user|>:“God created war so that Americans would learn geography.” <|assistant|>:∀x∀y (God(x) ∧ War(y) → Created(x, y))

<|user|>:“Follow your heart, listen to your inner voice, stop caring about what others think.” <|assistant|>:∀x (FollowsHeart(x) ∧ ListensToInnerVoice(x) ∧ StopsCaringAboutWhatOthersThink(x))

<|user|>:“All happiness depends on courage and work.” <|assistant|>:exists x1.(_happiness(x1) & exists x2.(_courage(x2) & _depend(x1,x2)) & exists x3.(_work(x3) & _depend(x1,x3)))

<|user|>:“Promise me you'll always remember: You're braver than you believe, and stronger than you seem, and smarter than you think.” <|assistant|>:∀x ∀y (Promise(x) ∧ Promise(y) ∧ Believe(x) ∧ Stronger(x) ∧ Stronger(y) ∧ Smarter(x) ∧ Smarter(y) → AlwaysRemember(x, y))