pgdyn-plan

pgdyn-plan

This is a pretrained model for the planning component of the PG_Dyn system, described in the EACL 2023 paper "Document-Level Planning for Text Simplification". It is the be used in conjunction with the simplification component to form the full pipeline. The code in this repo should be used.

How to use

Here is how to load this model in PyTorch:

from plan_simp.models.classifier import load_planner
from plan_simp.models.bart import load_simplifier

# contextual simplification planner
planner, p_tokenizer, p_hparams = load_planner("liamcripwell/pgdyn-plan")

# simplification model
simplifier, tokenizer, hparams = load_simplifier("liamcripwell/pgdyn-simp")

To perform end-to-end planning+simplification with dynamic document context, use the commands below. This assumed data is in a .csv format and context representations have been generated for each input document.

# using planner
python plan_simp/scripts/generate.py dynamic 
  --clf_model_ckpt=<planner_model> # e.g. liamcripwell/pgdyn-plan
  --model_ckpt=<simplification_model> # e.g. liamcripwell/pgdyn-simp
  --test_file=<test_sentences>
  --doc_id_col=pair_id # document identifier for each sentence
  --context_doc_id=c_id
  --context_dir=<context_dir>
  --reading_lvl=s_level 
  --out_file=<output_csv>

# manual specification of operations (no planner)
python plan_simp/scripts/generate.py inference 
  --model_ckpt=<simplification_model> # e.g. liamcripwell/pgdyn-simp
  --test_file=<test_sentences> 
  --op_col=label
  --reading_lvl=s_level 
  --out_file=<output_csv>