Semi-Supervised Learning for Surface EMG-based Gesture Recognition

Abstract

Conventionally, gesture recognition based on non-intrusive muscle-computer interfaces required a strongly-supervised learning algorithm and a large amount of labeled training signals of surface electromyography (sEMG). In this work, we show that temporal relationship of sEMG signals and data glove provides implicit supervisory signal for learning the gesture recognition model. To demonstrate this, we present a semi-supervised learning framework with a novel Siamese architecture for sEMG-based gesture recognition. Specifically, we employ auxiliary tasks to learn visual representation; predicting the temporal order of two consecutive sEMG frames; and, optionally, predicting the statistics of 3D hand pose with a sEMG frame. Experiments on the NinaPro, CapgMyo and csl-hdemg datasets validate the efficacy of our proposed approach, especially when the labeled samples are very scarce.

Illustration of the proposed semi-supervised ConvNet for sEMG-based gesture recognition.

Conventionally, gesture recognition based on non-intrusive muscle-computer interfaces required a strongly-supervised learning algorithm and a large amount of labeled training signals of surface electromyography (sEMG). In this work, we show that temporal relationship of sEMG signals and data glove provides implicit supervisory signal for learning the gesture recognition model. To demonstrate this, we present a semi-supervised learning framework with a novel Siamese architecture for sEMG-based gesture recognition. Specifically, we employ auxiliary tasks to learn visual representation; predicting the temporal order of two consecutive sEMG frames; and, optionally, predicting the statistics of 3D hand pose with a sEMG frame. Experiments on the NinaPro, CapgMyo and csl-hdemg datasets validate the efficacy of our proposed approach, especially when the labeled samples are very scarce.

This is a part of our sEMG-based gesture recognition project.

BibTex

@inproceedings{Du_IJCAI_2017,
    author    = {Yu Du, Yongkang Wong, Wenguang Jin, Wentao Wei, Yu Hu, Mohan Kankanhalli, Weidong Geng},
    title     = {Semi-Supervised Learning for Surface EMG-based Gesture Recognition},
    booktitle = {Proceedings of the Twenty-Sixth International Joint Conference on
                    Artificial Intelligence, {IJCAI-17}},
    pages     = {1624--1630},
    year      = {2017},
    doi       = {10.24963/ijcai.2017/225},
    url       = {https://doi.org/10.24963/ijcai.2017/225},
}

Acknowledgements

This work is supported by the National Key Research and Development Program of China (No.2016YFB1001300), the National Natural Science Foundation of China (No.61379067), and the National Research Foundation, Prime Ministers Office, Singapore under its International Research Centre in Singapore Funding Initiative.