Say you are a potato chips company. The goal is to have consumers upload images of the product they are having issues with and be able to identify the product by brand/variant using machine learning. Consumers can upload real product photos that they have taken, or upload bogus images from the internet, or even upload completely irrelevant/inappropriate photos (like that of a dog or cat).
In this example, for the legitimate image, the goal is to classify it as “Lays Classic”. There might be products that are not in bag form, such as those in tubes. Furthermore, the images taken can be in different lighting conditions/orientations. Some images might have other products as well.
I have been out of the ML field for the past 4 years so I’m not up to date on the most state of the art methods for this problem. I have studied CNNs 4 years ago, but there has been advances like transformer based methods. Someone has tried ResNet-50 and YOLOv5, and I’m thinking about using a pretrained model like CLIP and just train the final classification layer.
But I would appreciate to hear from someone more well versed what recommended approach to take as far as model/labeling/number of images needed per class, etc. It might be that I would need multiple models, such as one to identify the legitimate images from the rest, and then another one to identify the product/variants.
Any advice would be welcome. Thanks
Thanks.
Once you get the embeddings from the pretrained model, what classification method should one use for the final classification? Random forest? SVM?
I will also look into the average method you mentioned. Are you saying taking the averages of the embeddings for each class, and then to classify an embedding, see which class average is closest to the embedding (by closest you mean something like the L2 norm)?
It’s encouraging that one can do this in a day, but I haven’t done any ML work for a few years. Should I use Pytorch or Tensorflow?
Thanks