Leveraging Deep Learning to Assess Upper Limb Kinematics after Stroke with Off-the-shelf Webcams
Project state
closed
Project start
October 2021
Funding duration
33 months
Universities involved
ZHAW, UZH
Practice partners
Cereneo
Funding amount DIZH
CHF 204'600
Using the labeled video data, a deep learning-based AI model was trained. This model utilizes visually extracted body joint coordinates to predict whether an individual exhibits compensatory movements while performing the drinking task. The result is a sequence of computer-assisted processing steps that detect movement patterns based on recordings from commercially available cameras.
To develop this approach into a measurement tool suitable for everyday therapeutic practice, further research and development are necessary. A subsequent study would need to focus, for example, on quantifying the compensatory movements, and on creating a userfriendly interface for both clinicians as well as patients.
The project was presented at several conferences and resulted in the publication of multiple peer-reviewed papers:
- T. Unger, B. Kühnis, L. Sauerzopf, M.R Spiess, A. de Spindler,
R.Gassert, O. Lambercy, A. Luft, C. Awai Easthope , J.G. Schönhammer,
E. Gavagnin, in preparation. A Deep Learning Approach to Detect Upper
Limb Compensation in Individuals Post-Stroke Using Consumer-Grade
Webcams; to be submitted to Frontiers in 2025 - Sauerzopf, Lena; Chavez Panduro, Celina G.; Luft, Andreas; Kühnis,
Benjamin; Gavagnin, Elena; Unger, Tim; Easthope Awai, Christopher;
Schönhammer, Josef G.; Degenfellner, Jürgen; Spiess, Martina, 2024.
Evaluating inter- and intra-rater reliability in assessing upper limb
compensatory movements post-stroke : creating a ground truth through
video analysis?. Journal of NeuroEngineering and Rehabilitation.
21(217). Available from: https://doi.org/10.1186/s12984-024-01506-7 - Unger, Tim; Weikert, Thomas; Mokni, Marwen; Luft, Andreas; Gassert,
Roger; Lambercy, Olivier; Sauerzopf, Lena; Gavagnin, Elena; Kühnis,
Benjamin; Spiess, Martina; Schönhammer, Josef; Awai Easthope, Chris,
2024. Quantifying upper limb movement quality after stroke with a
webcam [Poster]. In: 13th World Congress for Neurorehabilitation,
Vancouver, Canada, 22-25 May 2024. - Unger, T.; de Sousa Ribeiro, R.; Mokni, M.; Weikert, T.; Pohl, J.;
Schwarz, A.; Held, J.P.O.; Sauerzopf, L.; Kühnis, B.; Gavagnin, E.; Luft,
A.R.; Gassert, R.; Lambercy, O.; Awai Easthope, C.; Schönhammer,
J.G., 2024. Upper limb movement quality measures : comparing IMUs
and optical motion capture in stroke patients performing a drinking task.
Frontiers in Digital Health. 6(1359776). Verfügbar unter:
https://doi.org/10.3389/fdgth.2024.1359776 - Sauerzopf, Lena, 2022. Promoting telerehabilitation after stroke :
applications of assessments and intervention in occupational therapy. In:
26th European Network of Occupational Therapy in Higher Education
Annual Meeting (ENOTHE), Tbilisi, Georgia,14-16 October 2022. - Sauerzopf, Lena; Luft, Andreas; Spiess, Martina, 2022. Implementation
of tools for technology-based teleassessment of sensorimotor recovery
after stroke. In: 19th International Medical Ph.D. Conference, Hradec
Králové, Czech Republic, 24-25 November 2022.
Team
Dr. Martina Spiess, ZHAW Gesundheit, Institut für Ergotherapie
Lena Sauerzopf, ZHAW Gesundheit, Institut für Ergotherapie
Prof. Dr. med. Andreas Luft, UniversitätsSpital Zürich, Klinik für Neurologie
Dr. Elena Gavagnin, ZHAW School of Management and Law, Institut für Wirtschaftsinformatik
Practice partner
Dr. Josef Schönhammer, Lake Lucerne Institute & UniversitätsSpital Zürich, Klinik für Neurologie