Writing a meta-analysis on political orientation and MFT, I’ve learned that there are (self-report) measures of political orientation that have completely different approaches to the question, and they seem to be favored by different kinds of users. I’m sure there are interesting philosophical writings on the subject, because topics like “what is political orientation” and “what is ideology” (not to mention “what is measurement”) are probably inexhaustible. This is just my quick observation on the subject.

Four (and a half?) different types of self-report measures:

Self-identification measure uses broad labels (such as “liberal vs. conservative”, “left vs. right”), asking people where they would place themselves (practically always on a bipolar) scale. A pragmatic choice – the focus is somewhere else, but something about the PO needs to be found out, so let’s use the easiest and simplest one. Views PO as a matter of identification and so skips the difficult questions of how to measure PO by settling for the idea that PO is whatever people think it is. However, because the scale predefined but not by the respondents themselves, this may lead to problems when using the measure outside the population from which it emerged – e.g. the use of “lib-cons” and “left-right” are different in the US and Europe. Also psychometrically problematic.

Issue-based measures ask an array of questions about specific political issues. They seem to be favored by researchers in politics – most likely they know about the inconsistencies in people’s identification vs. political behavior (also, their focus is probably more often specifically in political behavior) and the problems with minor groups that do not fit into the bipolar scale, so a single-item scale is viewed as inadequate. Issue-based measures are often combined with self-identification measures to empirically label self-identifications to sets of issue patterns. The approach is that one’s PO is determined by the similarity to other people (a data-driven approach). A general problem with an issue-based measure arises when the issues in the items are not relevant for the respondents (e.g. too old or from a foreign political culture: abortion may be a hot topic in the US and catholic countries, but it is a non-issue in protestant Western/Northern Europe). Another problem emerges when the identifications and response patterns diverge.

Theory-based measure is based on the a priori definitions from particular theoretical approach, and in practice considers the ways people identify themselves as more or less irrelevant. One’s PO is determined by the correspondence to the theoretically important factors that may be issue-based or use items that are about more abstract principles, or both. If you answer in a particular pattern, you can be labeled with a name related to the theoretical thought behind this (such as “liberal vs. authoritarian/statist”), whether you like it or not. I’ve especially seen it in libertarian opinion writings with no empirical research, but it is also used in research. The problem with the former is often that even though a label opposite to their own favored position may seem objective to the writer (like “authoritarian”), people who are not already on the same side surprisingly do not appreciate being  called that, so the writing achieves more to flame than discuss. The tradeoff in the theory-based research approach (in addition to those already related to the issue-based approach) is naturally that often you find what you look for, so the measure can be only as strong as the theory – and I’m not aware of particularly good psychological theories that would take a realistic view of human mind and political behavior into account.

Proxy measures use a measure not intended specifically for political orientation – such as values, particular personality traits like SDO, openness, or moral foundations – but that have earlier established reliable relationships the user can rely on. Psychologists seem to be especially fond of this, probably because they are not interested in political issues per se. Obviously, the problem is that this measure can only capture the facets of PO that happen to correlate with the proxy, so it may miss important information.

In regard to affective psychological view, the different types are not just different ways to answer the same question, but (to some extent) reflect fundamentally different processing. My assumption is that there is a (probably largish) number of nonconscious processes of different levels that produce the range of phenomena that may go under “political orientation”, and that some of the differences in the operating parameters of the low-level processes produce an important portion of the stable differences between political orientations. These trait differences behind political orientation are what I’m mainly interested in. It is not necessarily that the high-level differences are not important, but I assume that their operation in interaction with cultural influences are more complex, and therefore more difficult to study, so it’s better to at least start with the more easier ones.

Self-identification is a result of the conscious processing of the automatic construction of identification categories and those linking oneself to one of them. As such, it is a high-level process, and although that will have some relationship to the trait differences in the lower levels, the heavy processing of the higher levels mostly serve to confound that relationship. So self-identification tells us something about how people self-identify, but less about how this identification fundamentally works – although of course mapping the identification patterns may provide good information for further study. Proxy measures may target the low-level processes better, but they leave the relationship to PO itself unclear, if we assume that PO is something more than just the proxy (and we do).

Issue- and theory-based measures require judgments instead of self-reflection, which is in principle a better way to probe nonconscious processes. The issue-based approach cannot escape the problem that you should have a pretty good theoretical idea of what matters when you choose the items, or else your measure is just a collection of random items that are related to each other on the surface, but that don’t necessarily tell much about the processes underneath. Of course, if the research is on the surface level, that’s completely fine. A theory-based measure with abstract principles comes with the disadvantage that asking people how they would make judgments may give different answers than making them actually make the judgments. So if my assumption about the underlying structure is correct and we want to study the low-level processes with self-report measures, judgment items about issues, guided by a good theory, seems to be the way to go (without spending too much time on analyzing this).