My main usecase for LLMs is literally as an auto-complete, mainly via coding, so I was wondering whether anyone has played with/had any luck using the base model for use cases that are close to simply auto completing? I could imagine the instruction tuning adding a sycophancy bias in those areas
Just to be clear, you aren’t doing fine tuning here as in gradient updates, you are using the base model + ICL?