Fine-Grained Grocery Product Recognition by One-Shot Learning

Abstract

Fine-grained grocery product recognition via camera is a challenging task to identify the visually similar products with subtle differences by using single-shot training examples. To address this issue, we present a novel hybrid classification approach that combines feature-based matching and one-shot deep learning with a coarse-to-fine strategy. The candidate regions of product instances are first detected and coarsely labeled by recurring features in product images without any training. Then, attention maps are generated to guide the classifier to focus on fine discriminative details by magnifying the influences of the features in the candidate regions of interest (ROI) and suppressing the interferences of the features outside, improving the accuracy of fine-grained grocery products recognition effectively. Our framework also performs a good adaptability which allows existing classifier to be refined without retraining for new coming product classes. As an additional contribution, we collect a new grocery product database with 102 classes from 2 stores. Extensive experiments demonstrate that our approach outperforms the state-of-the-art methods.

Illustration of fine-grained grocery product recognition, which can detect and recognize grocery products captured via mobile devices by using one-shot learning.

BibTex

@article{Geng2018FineGrainedGP,
    title={Fine-Grained Grocery Product Recognition by One-Shot Learning},
    author={Weidong Geng and Feilin Han and Jiangke Lin and Liuyi Zhu and J. Bai and Suzhen Wang and L. He and Qiang Xiao and Zhangjiong Lai},
    journal={Proceedings of the 26th ACM international conference on Multimedia},
    year={2018}
}

Acknowledgements

This work is supported by a grant from the National Key Research and Development Program of China (No. 2017YFB1002800) and the National Natural Science Foundation of China (No. 61379067).