FIM-SE-SC-15B

Empowering Character-level Text Infilling by Eliminating Sub-Tokens

<p align="center"> <a href="https://arxiv.org/abs/2405.17103">📄 Paper</a> • <a href="https://github.com/SenseLLM/FIM-SE">🏠 Repo</a> • <a href="https://huggingface.co/SenseLLM/FIM-SE-CL-13B">🤖 Models</a> </p>

Introduction

FIM-SE stands for Fill-In-the-Middle with both Starting and Ending character constraints. The proposed method addresses character-level infilling tasks by utilizing a line-level format to avoid predicting any sub-token in inference.

<hr>

Models

How to Use

Prompt Format

As shown in the figure, the prompt is organized as

<PRE>R-Prefix<SUF>R-Suffix<START>L-Prefix<END>F-Suffix<MID>

Inference Code

Please refer to our GitHub Repo for more technical details.

Citation

If you find this repo useful for your research, please kindly cite our paper:

@misc{ren2024empowering,
    title={Empowering Character-level Text Infilling by Eliminating Sub-Tokens}, 
    author={Houxing Ren and Mingjie Zhan and Zhongyuan Wu and Hongsheng Li},
    year={2024},
    eprint={2405.17103},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

Acknowledgments

We thank the following amazing projects that truly inspired us:

• FIM