Standardisation  |  How to Interpret a standard process

Psychometry is the science of psychological assessment and forms the basis of psychological measurement.

Within psychometry there are four fundamental principles which determine the quality of an assessment tool. They are reliability, validity, standardisation, and freedom from bias. These principles inform not only the use of the assessment tool but also the assessment development process.

In previous newsletters, we discussed the principles of reliability and validity.  To recap, reliability is the extent to which an assessment tool is free from error while validity is the extent to which a tool measures what it intends to assess.

Now, let us discuss the principle of standardisation. Standardisation gives us information on how the results of an assessment should be used or interpreted. It is also an agreement about the assessment administration process, as a standardised assessment is administered and processed in a consistent or “standard” manner.

One aspect of how the results should be utilised is the use of norms (or a comparison group). The number of correct responses on an ability assessment is meaningless unless we know how it compares to others who were assessed. We need to know what the “normal” or the “average” performance on the assessment is, and the norm group or comparison groups assists us with this. Comparing scores to a reference group therefore allows us to standardise the interpretation of how a candidate’s result compares to an appropriate benchmark.

Standardisation of the assessment administration ensures that the assessment is conducted under specific conditions which can be repeated.  For ethical reasons, as well as comparison requirements, it is important that candidates are treated the same way.  For example, if one candidate was administered an ability assessment, in a quiet room, and another candidate undertook the assessment next to a noisy construction site, the candidate who undertook the assessment in noisy conditions could be affected by the noise. When making a comparison, the confidence in the test results would therefore decrease. The candidate’s performance could be affected because of distractions, which is why the standardisation of assessment administration is important. 

There are four classified assessment administration modes, namely:

1. Open mode – This is the most accessible and from a security aspect, the least reliable. The candidate has direct access to the test materials and there is no briefing from the psychometrist. Internet-based assessments without any requirement for registration are also considered an example of this mode of administration.
2. Controlled mode – Offers more security in this mode but limited supervision. Although no direct supervision of the assessment session is involved, security is enhanced because the assessment is only made available to known candidates. Access is typically limited to those who received some sort of briefing with an individualised username/password. These often are designed to operate on a one-time-only basis.
3. Supervised mode – Where there is a level of direct supervision over assessment conditions. The candidate’s identity can be authenticated, which addresses security concerns. For Internet testing, this typically requires the psychometrist to log-in a candidate and take steps to ensure that the assessment is correctly administered and completed.
4. Managed mode – Where there is a high level of human supervision and control over the assessment environment including the equipment being used. In computer based testing this is normally achieved by the use of dedicated assessment centers, where there is a high level of control over access, security, qualification of test administration staff and the quality and technical specifications of the test equipment.

Increased accessibility to the internet resulted in an increase in the availability of psychometric assessments. This had a considerable impact and has changed the standardisation practices, which were previously carefully controlled. Therefore, in 2005 the International Test Commission issued guidelines highlighting the need for caution given the various challenges associated with the newly classified assessment modes.

The various modes all have their own benefits therefore consideration needs to be given to what is most appropriate in specific scenarios. For example, in organisational settings, concerns arise over security and authentication when using psychometric assessments for selection purposes. Many companies therefore use an open or controlled mode for pre-screening recruitment purposes only. The assessment of shortlisted candidates can then be authenticated more strictly. This typically requires supervised or managed modes of delivery.

Psychometry focuses on how to properly measure certain psychological concepts such as cognition, personality and values or motives. A Psychometrist is an individual who is highly trained in administering and scoring various psychometric assessments, as well as report writing.  To ensure that assessment results are meaningful and utilised correctly, it needs to be interpreted by a trained psychometrist or psychologist and adherence to all four fundamental principles are non-negotiable.

Next, we will look at the final principle of a psychometric assessment, namely freedom of bias.