18 Construct validity

18.1 Definition

As noted above, validity focuses on the extent to which our construct is in fact what we think it is. If our construct is what we think it is, it should relate in known ways with other measures of the same or different constructs. On the other hand, if it is not what we think it is, relationships that should exist with other constructs will not be found.

Construct validity is established when relationships between our test and other variables confirm what is predicted by theory. For example, theory might indicate that the personality traits of conscientiousness and neuroticism should be negatively related. If we develop a test of conscientiousness and then demonstrate that scores on our test correlate negatively with scores on a test of neuroticism, we’ve established construct validity evidence for our test. Furthermore, theory might indicate that conscientiousness contains three specific dimensions. Statistical analysis of the items within our test could show that the items tend to cluster, or perform similarly, in three specific groups. This too would establish construct validity evidence for our test.

Module 21 presents EFA and CFA as tools for exploring and confirming the factor structure of a test. Results from these analysis, particularly CFA, are key to establishing construct validity evidence.

18.2 Examples

The entire set of relationships between our construct and other available constructs is sometimes referred to as a nomological network. This network outlines what the construct is, based on what it relates positively with, and conversely what it is not, based on what it relates negatively with. For example, what variables would you expect to relate positively with depression? As a person gets more depressed, what else tends to increase? What variables would you not expect to correlate with depression? Finally, what variables would you expect to relate negatively with depression?

Table 18.1 contains an example of a correlation matrix that describes a nomological network for a hypothetical new depression scale. The BDI would be considered a well known criterion measure of depression. The remaining labels in this table refer to other related or unrelated variables. “Fake bad” is a measure of a person’s tendency to pretend to be “bad” or associate themselves with negative behaviors or characteristics. Positive correlations in this table represent what is referred to as convergence. Our hypothetical new scale converges with the BDI, a measure of anxiety, and a measure of faking bad. Negative correlations represent divergence. Our new scale diverges with measures of happiness and health. Both types of correlation should be predicted by a theory of depression.

Table 18.1: Nomological Network for a Hypothetical Depression Inventory

	New Scale	BDI	Anxiety	Happy	Health	Fake Bad
New Scale	1.00
BDI	0.80	1.00
Anxiety	0.65	0.50	1.00
Happy	-0.59	-0.61	-0.40	1.00
Health	-0.35	-0.10	-0.35	0.32	1.00
Fake Bad	0.10	0.14	0.07	-0.05	0.07	1.00