My main usecase for LLMs is literally as an auto-complete, mainly via coding, so I was wondering whether anyone has played with/had any luck using the base model for use cases that are close to simply auto completing? I could imagine the instruction tuning adding a sycophancy bias in those areas
Get a base model of your choice, finetune it with plain text book of a style you want it to talk. Done.
Fine tune as in gradient updates or as in ICL?