I was recently trying to work on a ML project and realized that the I need very specific type of data for what I am trying to achieve which makes me wonder how do people and companies deal with data scarcity, I am pretty sure they at some point need very specific type of data which isn’t easily available
You must log in or # to comment.
Your problem is that you are surrounded with other ML engineers
Every bit of data is quickly fed to a neural network - new problems require new data
The best way I advise is join an industry where ML engineers are rare, you’ll have first pick on all the data
Fine tuning doesnt take much, I’ve used like 100 human-made Q&A pairs, and worked just fine