validity  |  what is valid?  #validmeasures

Validity is an important concept and works in tandem with Reliability. In our previous Psychometric Vibes, we focussed on Reliability.  We will now look at Validity.

Psychometric assessment methods and tests should have valid and reliable research, as well as data to support the statement that it is a sound measure. These two properties are the foundation of psychometric assessments. Therefore, one can have confidence that valid and reliable psychometric assessments can assist in identifying and selecting the most suitable candidate for a role.

What is psychometric assessment validity? Psychometric assessment methods (including ability assessments, personality questionnaires, structured interviews, or any other method) are valid to the extent that the assessment measures what it was designed to measure – it therefore refers to relevance. For example, a measure of intelligence should measure intelligence, and not something else, such as retention or memory.

Why does psychometric assessment validity matter? Validity is important in psychometric testing as well as the recruitment process, as it can give employers a good indication of how candidates would perform in the position they applied for. The various criteria fulfilled by the assessments should encourage that the most suitable candidates are selected. In addition, if the tests were not valid it would be a waste of the organisation’s time and money to use them.

Why is psychometric assessment validity necessary? While reliability is necessary, alone, it is not sufficient.  For a test to be reliable, it also needs to be valid.  For example, if a scale is off by 5 kg’s, it reads your weight every day with an excess of 5 kg’s.  The scale is reliable because it consistently reports the same weight every day, but it is not valid because it adds 5 kg’s to your true weight.  It is therefore not a valid measure of your weight.

Validity assumes that a test is reliable. In a test with high validity, the items will be closely linked to the test’s intended purpose. This means that the items will be highly related to a specific job and its requirements. A test that does not measure job-related content or competencies has poor validity and therefore the results should not be used.

Predictive validity is the extent to which a test or questionnaire predicts some future behaviour or desired outcome, for example work behaviour or on-the-job performance. This validity has importance in both selection/recruitment processes, as well as in the development of candidates. 

Predictive validity is usually measured by the correlation between the test score (result) and an appropriate criterion (or measure or standard).  The criterion could be on the job performance, counter-productive behaviours, manager ratings on competencies or any other outcome that can be measured.

Combined assessments (referred to as an assessment battery) can increase the assessment validity. Assessment validity can therefore be increased by using more than one assessment. This allows one to draw conclusions from the results of the (current) assessments completed to a meaningful outcome of what one wants to know or predict about the future behaviour of a candidate. Therefore, valid, and reliable psychometric assessments can assist in identifying and selecting the most suitable candidate for a role.

In summary – Reliability is getting the test right. Validity is getting the right test.

“Wide acceptance of an idea is not proof of its validity” – Dan Brown.