Body Monitoring
Introduction
Body monitoring is the measurement and analysis of overtly observable and covert data that a person produces during behavior. Behaving can also mean not moving because, for example, startle reactions or spasms for various reasons can be an important sign of a person's situation or disease state. In most cases, however, behavior refers to a person's movements, which can be well studied using data that are overtly observable, such as a person's arm movements, gait, or facial expressions. For example, gait analysis can tell a lot about a person's situation, whether it's their training condition, their health condition, or their degree of a physical constraint [1, 2]. To take advantage of this overtly observable behavior, technical devices are needed that allow human behavior to be recorded and analyzed systematically and with high accuracy. On the other hand, technical devices can also be used to collect and analyze covert human data, i.e., physiological data that cannot otherwise be observed by a human [3].
Covert human data is a valuable resource for not only identifying and tracking specific health conditions, but also for understanding their causes. In addition, physiological data can be used to control assistive or rehabilitation devices such as prostheses [4, 5], exoskeletons [6], or wheelchairs [7]. It is obvious that body monitoring is of great importance for many medical fields such as diagnosis, therapy or support. The challenge is to integrate the required technical devices into the human environment or, better, to place them on the person in such a way that these devices do not restrict him or her and enable complete monitoring in all living conditions. The latter is especially important so that a physician or therapist or trainer gets a more comprehensive picture of the actual training condition or training progress, the disease and accompanying limitations in everyday life, or to enable cross-sector treatment.
State of the Art
To record and analyze overt and covered human data different devices and signal processing and other algorithmic approaches or machine learning solutions are researched on.
Inertial sensors are commonly used to record overt data for analyzing gait or gross body movements. They are wearable and allow recordings in the wild. However, they must cope with drifts and are often still bulky [8, 9]. Alternatively, camera data can be used for skeleton tracking [10]. Here two main limitations exist. One is that cameras must be installed in the environment of the observe person and will therefore only allow analysis in dedicated environments such as gait laboratories. Second, video data in particular can easily be (mis)used to identify individuals, which is a difficult problem to solve and hinders large-scale applications or solutions built on that data. Indirectly, recordings of muscle activity through, for example, the electromyogram (EMG) can be used with some processing effort to also analyze observable behavior such as gait [11]. More commonly, EMG is used to analyze muscle strength in patients or athletes, to better understand disease, or to control assistive devices [12-14].
Recording and analyzing human EMG requires various signal processing and machine learning solutions, not only for interpretation but also to cope with drifts in the data caused, for example, by sweating or the sensitivity of EMG to changes in electrode placement [15]. Therefore, there are many approaches to develop different devices to record muscle activity, such as the aforementioned EMG, which has been extended to include array measurements that provide deeper insight into muscle function or better cope with shifts in measurement points [16-19]. Other sometimes indirect measurements of muscle activity are also being pursued with, e.g., flexible capacitive touch electrodes [20-22], force myography (FMG) [4-6] or electrical impedance tomography (EIT) [23]. All of these approaches have advantages and disadvantages. While some of them are portable and quite robust, such as the use of flexible capacitive touch electrodes, not all of them allow deep insights into the function of the muscle, and more importantly, they are still bulky and interfere with the user's normal behavior [24, 25] or may cause problems with blood circulation when they encircle the limb and require a certain pressure, as is the case with FMG measurements on the forearm.
Own research
Our research is going in different directions regarding body monitoring and its use in medicine, rehabilitation, training and therapy. From a technological point of view, we focus on building the smallest devices that allow monitoring the body and behavior unnoticed and possibly without contact with the body, with high resolution that surpasses high-density EMG grid measurements. This very precise and transparent monitoring is used for diagnosis and treatment not only in the laboratory but also in the wild, i.e., in the patient's everyday life. We also use body monitoring - here in particular based on covert data - to optimize the control of assistive and therapeutic devices such as exoskeletons [26,27].
Here, our approach to body monitoring allows us to draw conclusions about the patient's intention for the intentional control of robotic and therapeutic devices, as well as to enable continuous detection of the degree of support needed, i.e., assist-as-needed [28]. Our research on the application of THz-driven tiny passive and active sensors and receivers for non-contact measurements [29] or the development of new chip less epidermal electronics that can be attached to humans almost like a second skin requires research in engineering, signal processing and machine learning in close collaboration with researchers and users in the medical field.
Vision
Our vision is to seamlessly monitor human movements and underlying physiological processes for novel approaches to understand the sensorimotor system, its diseases, and their impact on daily life.
With our THz-based solutions, we aim to develop novel cross-sector approaches, especially for diagnosis and therapy, and to explore and develop interfaces to robotic assistance systems that enable imperceptible and inherent robotic assistance and therapy.
Figures
References
(1) Baker, R., Esquenazi, A., Benedetti, M. G., & Desloovere, K. (2016). Gait analysis: clinical facts. Eur. J. Phys. Rehabil. Med, 52(4), 560-574.
(2) Baker, R. (2006). Gait analysis methods in rehabilitation. Journal of neuroengineering and rehabilitation, 3, 1-10.
(3) Zheng, Z., Wu, Z., Zhao, R., Ni, Y., Jing, X., & Gao, S. (2022). A review of EMG-, FMG-, and EIT-based biosensors and relevant human–machine interactivities and biomedical applications. Biosensors, 12(7), 516.
(4) Prakash, A., Sahi, A. K., Sharma, N., & Sharma, S. (2020). Force myography controlled multifunctional hand prosthesis for upper-limb amputees. Biomedical Signal Processing and Control, 62, 102122.
(5) Ha, N., Withanachchi, G. P., & Yihun, Y. (2019). Performance of forearm FMG for estimating hand gestures and prosthetic hand control. Journal of Bionic Engineering, 16, 88-98.
(6) Xiao, Z. G., Elnady, A. M., & Menon, C. (2014, August). Control an exoskeleton for forearm rotation using FMG. In 5th IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics (pp. 591-596). IEEE.
(7) Lahane, P., Adavadkar, S. P., Tendulkar, S. V., Shah, B. V., & Singhal, S. (2018, April). Innovative approach to control wheelchair for disabled people using BCI. In 2018 3rd International Conference for Convergence in Technology (I2CT) (pp. 1-5). IEEE.
(8) Fan, B., Li, Q., & Liu, T. (2017). How magnetic disturbance influences the attitude and heading in magnetic and inertial sensor-based orientation estimation. Sensors, 18(1), 76.
(9) De Vries, W. H. K., Veeger, H. E. J., Baten, C. T. M., & Van Der Helm, F. C. T. (2009). Magnetic distortion in motion labs, implications for validating inertial magnetic sensors. Gait & posture, 29(4), 535-541.
(10) D’Antonio, E., Taborri, J., Palermo, E., Rossi, S., & Patane, F. (2020, May). A markerless system for gait analysis based on OpenPose library. In 2020 IEEE International Instrumentation and Measurement Technology Conference (I2MTC) (pp. 1-6). IEEE.
(11) Singh, R. E., Iqbal, K., White, G., & Holtz, J. K. (2019). A review of EMG techniques for detection of gait disorders. Artificial Intelligence-Applications in Medicine and Biology, 19-40.
(12) Lee, M. H., Siewiorek, D. P., Smailagic, A., Bernardino, A., & Badia, S. B. I. (2022). Enabling AI and robotic coaches for physical rehabilitation therapy: iterative design and evaluation with therapists and post-stroke survivors. International Journal of Social Robotics, 1-22.
(13) Li, L., Tyson, S., & Weightman, A. (2021). Professionals' Views and Experiences of Using Rehabilitation Robotics With Stroke Survivors: A Mixed Methods Survey. Frontiers in Medical Technology, 3, 780090.
(14) Akbari, A., Haghverd, F., & Behbahani, S. (2021). Robotic home-based rehabilitation systems design: from a literature review to a conceptual framework for community-based remote therapy during COVID-19 pandemic. Frontiers in Robotics and AI, 8, 612331.
(15) Barbero, M., Merletti, R., & Rainoldi, A. (2012). Atlas of muscle innervation zones: understanding surface electromyography and its applications. Springer Science & Business Media.
(16) Merletti, R., Holobar, A., & Farina, D. (2008). Analysis of motor units with high-density surface electromyography. Journal of electromyography and kinesiology, 18(6), 879-890.
(17) Negro, F., Muceli, S., Castronovo, A. M., Holobar, A., & Farina, D. (2016). Multi-channel intramuscular and surface EMG decomposition by convolutive blind source separation. Journal of neural engineering, 13(2), 026027.
(18) Del Vecchio, A., & Farina, D. (2019). Interfacing the neural output of the spinal cord: robust and reliable longitudinal identification of motor neurons in humans. Journal of neural engineering, 17(1), 016003.
(19) Barsotti, M., Dupan, S., Vujaklija, I., Došen, S., Frisoli, A., & Farina, D. (2018). Online finger control using high-density EMG and minimal training data for robotic applications. IEEE Robotics and Automation Letters, 4(2), 217-223.
(20) Dankovich, L. J., & Bergbreiter, S. (2019, May). Gesture recognition via flexible capacitive touch electrodes. In 2019 International Conference on Robotics and Automation (ICRA) (pp. 9028-9034). IEEE.
(21) Dankovich IV, L. J., Vaughn-Cooke, M., & Bergbreiter, S. (2022). Can You Do That Again? Time Series Consolidation as a Robust Method of Tailoring Gesture Recognition to Individual Users. Sensors, 22(19), 7512.
(22) Pearl, O., Rokhmanova, N., Dankovich, L., Faille, S., Bergbreiter, S., & Halilaj, E. (2022). Capacitive Sensing for Natural Environment Rehabilitation Monitoring.
(23) Liu, X., Zheng, E., & Wang, Q. (2022). Real-Time Wrist Motion Decoding With High Framerate Electrical Impedance Tomography (EIT). IEEE Transactions on Neural Systems and Rehabilitation Engineering, 31, 690-699.
(24) Bawa, A., & Banitsas, K. (2022). Design validation of a low-cost EMG sensor compared to a commercial-based system for measuring muscle activity and fatigue. Sensors, 22(15), 5799.
(25) Song MS, Kang SG, Lee KT, Kim J. Wireless, Skin-Mountable EMG Sensor for
Human-Machine Interface Application. Micromachines (Basel). 2019 Dec;
10(12): 879.
(26) Kirchner, E.A., Seeland, A., and Tabie, M. (2014). Multimodal Movement Prediction - Towards an Individual Assistance of Patients. In PLoS ONE, Public Library of Science, volume 9, number 1, pages e85060, 2014.
(27) Kumar, S., Wöhrle, H., Trampler, M., Simnosfke, M. Peters, H., Mallwitz, M., Kirchner, E. A., and Kirchner, F. (2019). Modular Design and Decentralized Control of the Recupera Exoskeleton for Stroke Rehabilitation. In Applied Sciences, MDPI, volume 9, number 4 (626), pages 1-23, Feb/2019.
(28) Kirchner E. A. and Bütefür, J. (2022). Bidirectional and Coadaptive Robotic Exoskeletons for Neuromotor Rehabilitation and Assisted Daily Living: a Review. In Current Robotics Reports, Springer Nature, volume N./A., pages 2662-4087, Apr/2022.
(29) Batra, A., Prokscha, A., Santhakumaran, S., Fabricius, J., Kirchner, E. A., & Kaiser, T. (2023). Towards Skin and Muscle Sensing from GHz to sub-THz Spectrum.