My main usecase for LLMs is literally as an auto-complete, mainly via coding, so I was wondering whether anyone has played with/had any luck using the base model for use cases that are close to simply auto completing? I could imagine the instruction tuning adding a sycophancy bias in those areas

  • wojcechOPB
    link
    fedilink
    English
    arrow-up
    1
    ·
    10 months ago

    Just to be clear, you aren’t doing fine tuning here as in gradient updates, you are using the base model + ICL?