Most micro- and macro-expression spotting methods in untrimmed videos suffer from the burden of video-wise collection and frame-wise annotation. Weakly supervised expression spotting (WES) based on video-level labels can potentially mitigate the complexity of frame-level annotation while achieving fine-grained frame-level spotting. However, we argue that existing weakly supervised methods are based on multiple instance learning (MIL) involving inter-modality, inter-sample, and inter-task gaps. The inter-sample gap is primarily from the sample distribution and duration. Therefore, we propose a novel and simple WES framework, MC-WES, using multi-consistency collaborative mechanisms that include modal-level saliency, video-level distribution, label-level duration and segment-level feature consistency strategies to implement fine frame-level spotting with only video-level labels to alleviate the above gaps and merge prior knowledge. The modal-level saliency consistency strategy focuses on capturing key correlations between raw images and optical flow. The video-level distribution consistency strategy utilizes the difference of sparsity in temporal distribution. The label-level duration consistency strategy exploits the difference in the duration of facial muscles. The segment-level feature consistency strategy emphasizes that features under the same labels maintain similarity. Experimental results on three challenging datasets-CAS(ME)(2), CAS(ME)(3), and SAMM-LV-demonstrate that MC-WES is comparable to state-of-the-art fully supervised methods.
基金:
STI2030-Major Projects [2022ZD0204600]; Huzhou Science and Technology Program [2023GZ13]; Natural Science Foundation of Sichuan Province [2023NSFSC0640]
第一作者机构:[1]Univ Elect Sci & Technol China, Sichuan Canc Hosp & Inst, Sch Life Sci & Technol, Chengdu 610054, Peoples R China
通讯作者:
推荐引用方式(GB/T 7714):
Yu Wang-Wang,Yang Kai-Fu,Yan Hong-Mei,et al.Weakly Supervised Micro- and Macro-Expression Spotting Based on Multi-Level Consistency[J].IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE.2025,47(8):6912-6928.doi:10.1109/TPAMI.2025.3564951.
APA:
Yu, Wang-Wang,Yang, Kai-Fu,Yan, Hong-Mei&Li, Yong-Jie.(2025).Weakly Supervised Micro- and Macro-Expression Spotting Based on Multi-Level Consistency.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,47,(8)
MLA:
Yu, Wang-Wang,et al."Weakly Supervised Micro- and Macro-Expression Spotting Based on Multi-Level Consistency".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 47..8(2025):6912-6928