EDITORIAL NOTE : RESEARCH INSTRUMENTS:
A principal
function of the Bulletin is to promote and disseminate research and empirical study in massage. Such activity requires measurement of various kinds, with instruments, which can be physical - such as a clock for time or a ruler for distances, or, non-physical, based on use of a questionnaire.
Instruments need to be valid and reliable to produce credible research results.
Suppose
a 30 cm ruler is actually 31cm long. If used it will give a result that is wrong
i.e.. not valid. But when used tomorrow it should give, reliably the same (wrong)
answer. It is reliable but not valid.
Now
suppose the ruler is made of a material that is sensitive to temperature and humidity so expands or contracts. The results it gives today are probably different from the lengths its user reads tomorrow it is unreliable. It is of course also not valid, because it cannot give the right answer.
Suppose
it remains made of this material that contracts and expands depending on environmental conditions but this 30cm ruler is actually
30cm long in laboratory controlled conditions. But of course each day it can
give a different reading it is unreliable, though valid (in the laboratory).
Finally
suppose it really is set up at 30cm and is of a material which never changes its length.
So measurement on any day is the same as on any other day it is reliable. And
it is always correct so it is valid.
These
concepts of validity and reliability carry over into questionnaire design when the questionnaire is intended to measure something
such as pain or quality or satisfaction obviously things which are rather more complicated to deal with than simple distance
calculations.
A questionnaire
(instrument) to measure variables such as stress, pain, quality, satisfaction etc. when used in research has to be valid and
reliable.
VALIDITY
By valid
is meant simply does it really measure stress/pain/quality/satisfaction etc. Validity
is concerned with what the questionnaire actually measures. Perhaps the most
well known arguments about this relate to whether an IQ test really measures intelligence, i.e. whether it is a valid measure
of intelligence.
When
we assess the results of a massage treatment which is meant to reduce stress or pain or improve the quality of life, or create
patient satisfaction we need to have some assurance that somehow we can measure these things using a questionnaire. Research is often discarded because the researcher has not used a questionnaire of proven validity. There are several aspects to validity.
A questionnaire
must have content validity are the questions in it representative of the content of the matter (e.g. pain) being measured. This usually has to be decided on expert judgement perhaps by a panel of experts after
discussion.
Validity
can also be looked at predictively. For example if a questionnaire purporting
to measure pain is accurate, then we might expect that where it measures pain as reduced after treatment, we could predict
lower use of pain relief medication (which is directly observable) or greater movements or range of motion (also observable). So a questionnaire would be regarded as valid predictively if such data had been collected
and checked.
Another
aspect of validity is concurrent validity. If we already had an existing acceptable
measure of pain, does our new questionnaire agree (i.e. concur) with it? It could
be asserted that if there were already such a measure, why bother to create a new measure?
This would be done to create perhaps a quicker, cheaper more user friendly instrument than the existing one, which
might be onerous and cumbersome to use.
A questionnaire
to measure something such as pain or patient satisfaction must intuitively appear on the face of it to actually deal with
the right criteria. If users thought the items in the questionnaire had, on the
face of it, nothing to do with the real items to be measured, the questionnaire would lack face validity It has to look right.
Finally
and perhaps most difficult of all is construct validity. Construct validity is the degree to which an instrument (questionnaire)
measures the construct under investigation. This is difficult and challenging
and deals with the question: What construct is the instrument actually measuring? A
construct could be a grief, anxiety, stress, conflict, intelligence. We cannot
observe a construct directly. Pain would not be a construct it is felt directly
and can increase or decrease. Mobility is not a construct it can be measured
as an angle or a distance.; but sensitivity or satisfaction are not so specific and easy to isolate or observe they are made
up of ideas from numerous components, not necessarily with universal agreement about what they are. So to validate an instrument dealing with patient satisfaction which is
a construct likely to have many components we need to use several kinds of indirect evidence including correlations
with other evidence in associated activity such as returning for more treatment, or giving a large tip as an indicator of
satisfaction, or making recommendations to other possible clients. Validity in
the construct sense is complex to establish and requires a variety of correlational evidence.
(A good discussion appears in Burns, 2000 4th ed)
RELIABILITY
Questionnaires
which measure should also possess reliability or consistency. That is different
users of the questionnaire should be able to get the same result. And if the
situation in which it is being used is unchanging, then measuring tomorrow should produce the same result as measuring today.
Reliability is about dependability and consistency.
