On Upper Limb Use

What is it? How do we measure it?

Summary

1. Upper limb use is the most fundamental construct of upper limb functioning.

2. Wrist-worn IMUs are the preferred approach for quantifying UL use.

3. Machine learning methods do better than traditional ones at detecting UL use.

4. Machine learning approaches may be able to do better

5. The way forward

6. Suggested reading

Upper limb use is the most fundamental construct of upper limb functioning.

The ultimate goal of upper limb (UL) neurorehabilitation is to enable patients to successfully incorporate their ULs to carry out their daily activities, i.e., improve UL functioning¹. The use of wearable sensors to objectively assess UL functioning is an active area of research in neurorehabilitation. Four questions are of interest in UL functioning:

1. How much is the upper limb used?

2. What is the relative use of the two upper limbs?

3. What tasks are carried out with the upper limbs?

4. What is the quality of movements performed with the upper limbs?

This specific order for the four questions allows UL functioning information to be presented hierarchically — from coarse to fine-grained level of detail. A clinician is probably interested in first knowing how actively a patient has been using his/her UL, before wanting to know what specific tasks were accomplished with that limb or how well these movements were made.

The starting point for answering these questions is upper limb use, arguably the most fundamental construct in UL functioning.

Upper limb use is any voluntary, goal-directed movement² or posture³ performed with an upper limb.

It is a binary construct indicating the presence or absence of a voluntary, goal-directed movement at any given instant. We will also use the terms “functional” versus “non-functional” movements to indicate the presence or absence of UL use, respectively.

What counts as UL use?

The majority of activities of daily living will count as UL use or functional movements when they are performed voluntarily, e.g., feeding oneself, manipulating objects, reaching for a switch, riding a bicycle/motorbike, writing, etc. These are movements that a clinician is interested in picking up. Non-purposeful resting of the UL, movements of the UL due to external forces, arm swing during walking, and movements from involuntary muscle activations are all examples of non-functional movements; movements that a clinician generally does want to monitor. However, not all postures are non-functional. For example, using a UL to support oneself from falling, resting the forearm/hand on a book to prevent it from closing, holding a cup steadily in one’s hand, etc. are purposeful uses of the UL.

Wrist-worn IMUs are the preferred approach for quantifying UL use

Cartoon depicting a wrist-worn IMU while the subject performs a reaching movement.

The most common approach for quantifying UL use is through a wrist-worn MEMS inertial measurement unit (IMU); these are the sensors commonly used in activity trackers, pedometers, mobile phones, etc. What do the accelerometer and the gyroscope of a wrist-worn IMU pick up? The following equations provide a compact mathematical answer to this question.

Accelerometer measurement:

\(\mathbf{a}_s = \mathbf{R}_{ES}^\top \left( \frac{d}{dt}\left( \mathbf{r}_{sh} \times {\omega}_{sh} + \mathbf{r}_{el} \times {\omega}_{el} \right) + \mathbf{g} + \mathbf{a}_{oth}\right)\)

Gyroscope measurement:

\({\omega}_{S} = \mathbf{R}_{ES}^\top \left( {\omega}_{sh} + {\omega}_{el} + {\omega}_{oth}\right)\)

These measured signals:

1. have a complex relationship with shoulder and elbow movements.

2. are affected by linear acceleration and angular velocity components from other sources (e.g. trunk).

3. will not pick up movements of the wrist and fingers that do not impact the state of the IMU sensor, and

4. do not provide any contextual information.

This implies that mapping wrist-worn inertial measurements to a binary UL use will be a complex process prone to errors due to: (a) movements other than voluntary shoulder/elbow movements and (b) the lack of the appropriate context. However, at present, wrist-worn measurements seem to be the most viable approach, and we need to make the most of these measurements. Research in UL use detecting is aimed at identifying the optimal algorithms for extracting the most information from these measurements.

Machine learning methods do better than traditional ones at detecting UL use

Methods based on some human intuition behind UL use are referred to as traditional methods, while machine learning methods are data-driven. Activity counting (AC) is the most popular traditional method that classifies movements with large (gravity-subtracted) accelerations as functional movements, while low ones are deemed non-functional. As you might have guessed, this is a terrible approach. AC has very high sensitivity and poor specificity. A cleverer approach is the gross movement (GM) score which classifies movements where the forearm is roughly parallel to the ground as functional movements and everything else as non-functional. The GM score, however, is highly specific but has poor sensitivity.

On the other hand, machine learning (ML) methods have shown more promising results when trained using data collected from healthy subjects and patients performing various activities of daily living. Being data-driven methods, both inter- and intra-subject ML models have been explored. An inter-subject ML model is one where a single model is trained for all subjects, while intra-subject models are ML models trained individually for each subject. Intra-subject models have better performance than inter-subject models, owing to the large variability between subjects.

Why do ML methods do better than traditional methods?

I think there are several reasons for this:

1. ML methods (e.g. SVM, random forests, neural networks, etc.) allow more complex and flexible structures allowing more sophisticated decision boundaries for detecting UL use; traditional methods have relatively simple structures.

2. The parameters of traditional methods are chosen based on human intuitions which are likely to be suboptimal. ML methods optimize the parameters using real data.

Note: We recently observed that a simple algorithm combining the two traditional algorithms — AC and GM — with optimized parameters has a performance similar to that of the best ML algorithm for inter-subject models.

Machine learning approaches may be able to do better

Although ML approaches do better, their overall accuracy remains around 70/80% in detecting UL use from wrist-worn inertial measurements. Can we do better than this?

We don’t know.

Wrist-worn inertial measurements are upper-bounded in the amount of information they carry about UL use; see the point about these measurements above. We don’t know if we’ve already reached this limit with our existing ML approaches.

My gut feeling is that there is still some juice we can extract from the wrist-worn measurements if we have:

1. more data, i.e. more subjects, a more diverse set of tasks, and better real-life contexts.

2. more sophisticated algorithms, i.e. deeper networks working on the raw data stream, and models with memory (e.g. RNN, NARX, etc.). Current approaches are purely feedforward in nature; feeding back the past output will allow the model to capture the autocorrelation structure of the stochastic UL use signal. For instance, the probability that I am performing a functional movement at this time instant will depend on whether I have been performing a functional movement in my immediate past.

The way forward

There is only one way forward if we are to bring modern ML/AI developments to solve problems in neurorehabilitation: more data and more advanced algorithms.

This is one of the active areas we are exploring in my group at the moment. Building this large, high-quality open dataset is an ongoing project which we believe will be an important step in solving the UL use assessment problem.

Several open questions remain in UL use detection, and we hope to help answer some of these in the coming years.

Suggested reading

If you find this problem interesting, you should read the following for more details of this fascinating problem:

1. Bailey et al., PloS One (2014)

2. Bailey et al., Neurorehabilitation and Neural Repair (2015)

3. Leuenberger et al., Med. Biol. Eng. Comput. (2017)

4. Lum et al., Neurorehabilitation and Neural Repair (2020)

5. David et al., J. Rehabil. Assist. Technol. Eng. (2021)

6. David et al., Front. Hum. Neurosci. (2021)

7. Subash et al., Front. Physiol. (2022)

8. Balasubramanian, medRxiv (2023)

1. Upper limb function is the upper limb's movement behaviour in natural settings.

2. Movements are events where the state of a limb changes with time.

3. Postures are events where the state of the limb is fixed or constant over some time.