Questionnaires are among the most common tools used in psychological research. To avoid the so-called agreement bias, where people automatically agree with statements regardless of their content, inversely coded items are often included in the scales – that is, statements formulated in the opposite direction ("I don't like being the center of attention" instead of "I like being the center of attention"). A mixture of positively and negatively worded items also better covers the "content" of the measured attribute. In other words, it allows for more accurate measurement of not only high but also low levels of the measured attribute, trait, or attitude. However, it is precisely these items that often seem to cause more problems than benefits. Why is this the case?
A new study led by David Elek, in collaboration with Hynek Cígler and Stanislav Ježek from INPSY, provides answers – the problem may not only lie in the cognitive load on respondents, but also in the language, logic, and way we formulate questions.
Language as a source of (mis)understanding
Responses to inversely formulated questions are often labeled as "inconsistent." Typically, for example, when a person disagrees with the statement "I am happy" and its counterpart "I am unhappy." At first glance, this may seem like a mistake, but according to the authors, it is completely logical. Language also allows for a middle, neutral position – neither happy nor unhappy. If neither applies, the respondent will answer both questions in the negative. In terms of meaning, this is a perfectly understandable response, even though the statistical model evaluates it as a contradiction. One must also understand that "disagreeing" with a negative statement is actually agreeing with it. This process of reversing the logic of a statement increases cognitive demands and also increases the risk of unintentional errors or the aforementioned ambiguous responses.
A new framework for measurement in psychology
In psychometrics, it is commonly assumed that positive and inversely worded items are just two sides of the same coin – they measure the same thing, just "in reverse." But language doesn't work that way. Words, negations, and antonyms do not behave like exact mirror counterparts. Moreover, people differ in how "sharply" they perceive negations. Some people tend to perceive positively and negatively worded items as exact opposites, while others perceive them as non-mutually exclusive, overlapping statements. However, these differences in perception are not related to the measured constructs, such as attitudes or personality characteristics.
The traditional consequence of using a mixture of agreeable and disagreeable items is a distortion of the psychometric parameters of the questionnaire. In particular, researchers may feel that instead of a single attitude, they are suddenly measuring two attitudes. An example of this is the measurement of extraversion and introversion. In reality, it is a single construct, with both terms merely denoting opposite ends of a "scale" (introversion is a term for a low level of extraversion and vice versa). However, if the questionnaire contains contradictory items ("I like being around people" vs. "I like being alone"), it may seem that we actually have two similar but different constructs: introversion and extroversion. The same effect can also create false correlations in research studies across different questionnaires. This can have a far-reaching impact on the quality of research studies not only in psychology, but in the social sciences in general, as well as in medicine and other fields.
There are a number of ways to control for these effects. However, no matter how sophisticated the statistical model we use, if we overlook the linguistic aspects of the questions, we risk distorting the results.
What does this mean in practice?
Instead of mechanically eliminating "problematic" items, the authors recommend classifying different types of inversion more precisely. According to them, it is necessary to use existing knowledge from linguistics about the perception of antonyms and negations in general. Another option is to examine how disaggregating items manifest themselves in different languages. When translating questionnaires in particular, researchers should be aware that the same negation can have a different effect in another language. As the authors themselves summarize: "The path to more accurate measurement may not lie in more complex models, but in a deeper understanding of how people understand words."
Recommended citation:
Elek, D., Cígler, H., Grüning, D. J., & Ježek, S. (2025). Advancing the psychometrics of reverse-keyed items: enriching cognitive theory by a logical and linguistic perspective. Frontiers in Psychology, 16, 1684612. https://doi.org/10.3389/fpsyg.2025.1684612
