I’m thinking of using Llama 2 to detect spam messages:

1. The model will first be fine tuned using LoRa/PEFT with some public dataset.

2. Then, when given a block of text, it will decide if it’s spam and provide reasons for the user.

3. However, there can be false positives etc., so I figured a way to combat this would be to let the user tell the model if the response is correct or wrong (thumbs up/down).

Based on my requirements, is it better to use RLHF or DPO? Am I over complicating this, will fine tuning it based on user feedback work too?