Appendix HAssessments and measures

Publication Details

These guidelines do not recommend or even mention specific measures or assessments. A brief explanation is given in the introduction. This appendix gives a little more information. However, it must be emphasised that the measures mentioned here are not being recommended in any way.

First, the reader is reminded of the second paragraph in section 3.2.1:

The process of assessment refers both to the collection and to the interpretation of data needed to identify problems (screening) and to inform the solutions. The process of assessment may or may not include measurement, which is the quantification of data against some metric. In other words, an assessment procedure may be considered to have two purposes which may be distinct or combined:

  • the detection of a phenomenon (ie diagnosis); and,
  • the measurement of a phenomenon.

The questions faced are whether we are able to give recommendations that cover:

  • how specific impairments or activity limitations should be detected
  • how the severity of any impairment or activity limitation should be quantified, for example when measuring the outcome of an intervention
  • what protocols should be used when first seeing a person with MS in order to detect most common or important difficulties.

In all three cases the first step is to collect data from or about the person with MS, and so the discussion is primarily concerned with whether or not the guidelines should mention or recommend specific data collection tools. These are referred to as ‘measures’ or ‘assessments’ interchangeably and loosely. A measure is a way of quantifying something; an assessment is a process of making a rehabilitation diagnosis through identifying and sometimes quantifying the presence of various abnormalities. The recommendations have emphasised the need for the process of assessment, and implied that measurement should be considered. The question is whether specific tools should be recommended.

The problems

There are several problems, especially in the context of a document that is emphasising the need to base recommendations on evidence.

The evidence searches undertaken have not been appropriate for making decisions on which data collection tools to recommend. To give reasonable evidence we would need to search for evidence on what purposes each tool might be suitable for, how reliable it is, how sensitive to change if used for that purpose, its diagnostic sensitivity and specificity if used for that purpose, its feasibility, and its comparative utility with other tools.

For almost all tools only minimal published data are available. Specifically there are very little comparative data to guide any choice between tools that collect data on the same problem.

However, the number of tools used in research is vast. This appendix includes a list of measures mentioned in the studies tabulated in the evidence tables. This list is only a small proportion of the measures used in the totality of MS research. Moreover, in clinical practice a different set of measures is used (in as far as measures are used at all). The utility, validity etc of most of these measures is unknown.

It also needs to be recognised that many people hold strong views on what to use and will anyway disagree with and probably ignore any recommendations. The comments by stakeholders illustrated the wide variety of opinions on appropriate measures vividly – there was no agreement by stakeholders on any single measure.

At the same time those without strong views might not use any tools, whatever is recommended.

In favour of making recommendations

In the guidelines we have suggested repeatedly that specific problems (such as swallowing difficulties) are identified, but not how. This omission lessens the likelihood of the recommendations being followed. Many stakeholders suggested that we should include recommendations, but usually recommended ones that others did not.

Against making recommendations

Although we have not searched systematically for evidence on measures, there is in fact little useful evidence available and we have not searched for or found the relatively limited available evidence. Consequently any recommendations made will be arbitrary and will not be agreed. Therefore it is unlikely that any specific recommendations will be followed, making it unwise to make them.

We do not have the time or resources to undertake a proper systematic search for evidence even in one domain, let alone for the whole of neurological rehabilitation. It is wiser to make no recommendations than to make recommendations that are personal preferences, rather than based on firm evidence, and likely to be disputed by more people than agree on them.

What is available?

In the process of formulating these guidelines and collating the evidence we have, however, produced some information.

First, there is a simple list of all the measures used in the research studies referred to in this document. Many of these measures are untested and few will be useful.

List of measures

The tabulated list of measures is simply set out in order of frequency, with a brief note on some measures where known. The names given are those abstracted from the original papers, and some of the measures may have no further data available. To find further details the reader should identify the study that used the test and read the original paper.

Conclusions

Several conclusions can be drawn from this appendix.

Firstly, future search strategies should focus on investigating:

  • validity: what purposes the measure can fulfil (or what measures can fulfil a specific purpose)
  • reliability: how consistent the tool is when used in different circumstances by different people or in different ways
  • sensitivity: what change or difference can it detect
  • utility: this is a review of such factors as the time and effort needed to use the measure, the amount of equipment needed, the training needed by the assessor, and how much the information gained alters clinical decisions.

Second, a large number of measures have been used, but many only once.

A research programme should be set up to investigate simple measures for routine use, preferably not limited to MS but covering all neurological conditions.

Table H1. Assessment tools used as outcome measures in multiple sclerosis RCTs and CCTs in evidence tables.

Table H1

Assessment tools used as outcome measures in multiple sclerosis RCTs and CCTs in evidence tables.