WatchHAR: Real-time On-device Human Activity Recognition System for Smartwatches
Taeyoung Yeon, Vasco Xu, Hank Hoffman, and 1 more author
In Proceedings of the 27th ACM International Conference on Multimodal Interaction, 2025
Despite advances in practical and multimodal human activity recognition (HAR), a system that runs entirely on smartwatches in unconstrained environments remains elusive. We present WatchHAR, an audio and inertial-based HAR system that operates fully on smartwatches, addressing privacy and latency issues associated with external data processing. By optimizing each component of the pipeline, WatchHAR achieves compounding performance gains. We introduce a novel architecture that unifies sensor data pre-processing and inference into an end-to-end trainable module, significantly accelerating performance by 25× while maintaining over 90% accuracy on 25+ activity classes. WatchHAR outperforms state-of-the-art models in terms of accuracy while running on the smartwatch directly at 30 Hz for multimodal activity classification and 40 Hz for activity event detection. This research advances edge-based activity recognition, realizing smartwatches’ potential as standalone, minimally-invasive activity tracking devices.
