Streaming Neural Networks with Memory for Efficient Human Activity Recognition

Human Activity Recognition (HAR) uses motion sensors, such as a smartwatch’s accelerometer, to recognize activities like walking or cooking, often with neural networks running directly on the device. The standard approach cuts the sensor signal into short, overlapping windows and classifies each window on its own. Because the windows overlap, every sensor sample is processed several times, which wastes computation. And because each window is classified in isolation, the network can’t use what happened just before, so predictions often jump between activities. Recent work shows that a memory of previous windows makes predictions more accurate and more consistent [1].

This thesis explores running HAR neural networks on the continuous sensor stream instead, processing each sample only once. This works for purely convolutional networks, which are among the most accurate and efficient HAR models [2, 3, 4]. Their filters compute the same values on the overlapping parts of windows, so one pass over the stream can replace per-window processing [5]. A small recurrent neural network (GRU [6]) on top adds memory across windows. The approach will be applied to existing networks such as CNN-HAR [3] and LiteWay [4], and evaluated on several HAR datasets for both accuracy on complete recordings [1] and computational cost. The goal is a simple, general method that makes HAR neural networks cheaper to run on wearables while giving smoother, more reliable predictions.

References
[1] Bock, M., Gall, J., Moeller, M., & Van Laerhoven, K. “DeepConvContext: A Multi-Scale Approach to Timeseries Classification in Human Activity Recognition.” ISWC, 2026.
[2] HARena: A Living Benchmark for Sensor-Based Human Activity Recognition on Resource-Constrained Edge Devices. Under review.
[3] Yang, J. B., et al. “Deep Convolutional Neural Networks on Multichannel Time Series for Human Activity Recognition.” IJCAI, 2015.
[4] Nshimyimana, D., Fortes Rey, V., Liu, M., Zhou, B., & Lukowicz, P. “LiteWay: Lightweight HAR via Temporal Efficient highWAY.” ISWC, 2026.
[5] Long, J., Shelhamer, E., & Darrell, T. “Fully Convolutional Networks for Semantic Segmentation.” CVPR, 2015.
[6] Cho, K., et al. “Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation.” EMNLP, 2014.

• Review sliding-window HAR and convolutional and recurrent neural networks for sensor data.
• Convert existing convolutional HAR networks into streaming versions and verify they match the window-based results.
• Add and compare small recurrent networks (e.g. GRU) for memory across windows.
• Evaluate accuracy and computational cost against the standard sliding-window approach on several datasets.

• Basic knowledge of machine learning and neural networks.
• Good Python skills; PyTorch experience is a plus.
• Interest in sensor data, wearables and efficient neural networks.