My main usecase for LLMs is literally as an auto-complete, mainly via coding, so I was wondering whether anyone has played with/had any luck using the base model for use cases that are close to simply auto completing? I could imagine the instruction tuning adding a sycophancy bias in those areas

  • FPhamB
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 year ago

    Get a base model of your choice, finetune it with plain text book of a style you want it to talk. Done.

    • wojcechOPB
      link
      fedilink
      English
      arrow-up
      1
      ·
      1 year ago

      Fine tune as in gradient updates or as in ICL?