Fine Tuning On Completions only using Unsloth
I want to fine tune a LLM using Unsloth's framework and I am interested in doing that by using only the output ids and not the input ids. There are two ways described and I am confused which one is the most appropriate:
The first way is to use a DataCollatorForCompletionOnlyLM with UnslothTrainer as described here: https://huggingface.co/docs/trl/sft_trainer#train-on-completions-only
The other way is to use DataCollatorForSeq2Seq with UnslothTrainer and then use the train_on_responses_only to create a new instance of the trainer as documented here:
https://github.com/unslothai/unsloth/wiki#train-on-completions--responses-only-do-not-train-on-inputs
Which one should I use?