I was recently trying to work on a ML project and realized that the I need very specific type of data for what I am trying to achieve which makes me wonder how do people and companies deal with data scarcity, I am pretty sure they at some point need very specific type of data which isn’t easily available

  • Ok_Reality2341BEnglish
    1·
    10 months ago

    Your problem is that you are surrounded with other ML engineers

    Every bit of data is quickly fed to a neural network - new problems require new data

    The best way I advise is join an industry where ML engineers are rare, you’ll have first pick on all the data

  • Delicious-Farmer-234BEnglish
    1·
    10 months ago

    Fine tuning doesnt take much, I’ve used like 100 human-made Q&A pairs, and worked just fine