Phone: (+39) 0813995453

Cross-Modal Contrastive Hashing Retrieval for Infrared Video and EEG

HumanInsight Cross-Modal Contrastive Hashing Retrieval for Infrared Video and EEG

Sensors (Basel). 2022 Nov 14;22(22):8804. doi: 10.3390/s22228804.


It is essential to estimate the sleep quality and diagnose the clinical stages in time and at home, because they are closely related to and important causes of chronic diseases and daily life dysfunctions. However, the existing "gold-standard" sensing machine for diagnosis (Polysomnography (PSG) with Electroencephalogram (EEG) measurements) is almost infeasible to deploy at home in a "ubiquitous" manner. In addition, it is costly to train clinicians for the diagnosis of sleep conditions. In this paper, we proposed a novel technical and systematic attempt to tackle the previous barriers: first, we proposed to monitor and sense the sleep conditions using the infrared (IR) camera videos synchronized with the EEG signal; second, we proposed a novel cross-modal retrieval system termed as Cross-modal Contrastive Hashing Retrieval (CCHR) to build the relationship between EEG and IR videos, retrieving the most relevant EEG signal given an infrared video. Specifically, the CCHR is novel in the following two perspectives. Firstly, to eliminate the large cross-modal semantic gap between EEG and IR data, we designed a novel joint cross-modal representation learning strategy using a memory-enhanced hard-negative mining design under the framework of contrastive learning. Secondly, as the sleep monitoring data are large-scale (8 h long for each subject), a novel contrastive hashing module is proposed to transform the joint cross-modal features to the discriminative binary hash codes, enabling the efficient storage and inference. Extensive experiments on our collected cross-modal sleep condition dataset validated that the proposed CCHR achieves superior performances compared with existing cross-modal hashing methods.

PMID:36433399 | DOI:10.3390/s22228804

Powered by WPeMatico

P.IVA 08738511214
Privacy Policy
Cookie Policy

Sede Legale
Viale Campi Flegrei 55
80124 - Napoli

Sede Operativa
Via G.Porzio 4
Centro Direzionale G1
80143 - Napoli

AI 4394
© Copyright 2022 - Humaninsight Srls - All Rights Reserved
Privacy Policy | Cookie Policy
envelopephone-handsetmap-marker linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram