A lot of contemporary psychology research is based on the assumption that there are five basic dimensions of personality that define people around the world.
It’s called the “Big Five” personality model, and it assumes that each of our personalities are a unique blend of a handful of traits: extraversion, agreeableness, openness, conscientiousness and neuroticism. The thinking is that we’re all pretty much made up of the same five core ingredients, even if the measurements differ from person to person. Insights gleaned from personality tests based on the Big Five framework are frequently used to understand trends related to issues like crime, education, employment, wages and more.
Whether these personality traits are truly universal is still a matter of debate. But a new analysis by an international research team highlights an important caveat when it comes to administering cookie-cutter personality tests across cultures: these tests aren’t one-size-fits-all. And this is problematic, the researchers say, because most personality tests used in developing countries are modeled on the ones used in white, educated, industrialized, rich and democratic societies (or, more concisely, WEIRD, an acronym used in the research community).
In their work, the researchers found a number of factors — some cultural, some not — that can impact the reliability of results gleaned from Big Five personality assessments administered in developing countries. Some factors maybe aren’t surprising — such how a question’s true meaning might be lost during translation because of language differences. But even the very notion of what personality means within a culture can make administering personality assessments more of a challenge in some parts of the world.
Put to the Test
In their study, published in Science Advances on Wednesday, the researchers analyzed more than 90,000 survey responses, including data gathered from face-to-face testing and online databases, that came from people in 23 countries, such as Bolivia, Kenya, Sri Lanka and others.
According to study author Karen Macours, a researcher at the Paris School of Economics, the team soon noticed that these tests often failed in capturing the personality traits of people in developing countries. And oftentimes, the results from these countries didn’t pass standard validity measures that researchers use to ensure the data is accurate and reliable.
Most strikingly, an odd contradiction related to personality and job success was a key observation that cast doubt on the cross-country applicability of these tests.
Generally, a person’s cognitive ability (a cringe-inducing term in the research community describing intelligence) has been found to be a predictor of success in complex jobs — such as those requiring literacy or math skills. But when it comes to less-complex jobs, personality is a greater predictor of success – particularly how conscientiousness (being achievement-oriented and organized) and emotionally stable (less neurotic) a person is.
But oddly, when digging into the country data, the researchers found that the opposite was true in developing countries. In countries where more of the population is engaged in less complex lines of work, the data suggested that cognition mattered more than personality for job success. This finding just doesn’t add up, as previous studies have shown that soft skills tend matter more in less complex work, the researchers said.
There were other giveaways in their analysis that suggested these assessments don’t work as well in developing countries. For example, in questions aimed at assessing conscientiousness, people would answer “strongly agree” to statements on the opposite end of the behavioral spectrum (for instance, strongly agreeing both to statements about being highly organized and being careless). Contradictory answers like these were peppered throughout the data, suggesting that the tests don’t work well outside of WEIRD countries.
Lost in Translation
So, why doesn’t the Big Five test hold up around the world? Lead author Rachid Laajaj, an economics researcher at the University of Los Andes in Columbia, said many of the reasons are rooted in literacy and education barriers. Many personality tests used in WEIRD countries are intended to be self-administered, designed for people who can read and write. But because of lower literacy rates in developing countries, tests may need to be given verbally. This introduces the possibility of translation or phrasing differences that could skew results.
Researchers also think that face-to-face questioning allows social desirability bias to creep into the process. This means that respondents may try to interpret social cues for a “right answer” or give answers they think would be viewed more favorably by others.
“Yea-saying,” or the tendency to agree with a statement even if it’s untrue, is also more common in developing countries, where there’s less access to education, the researchers say.
“People may have a harder time understanding abstract questions. Acquiescence bias may be accentuated when people do not fully understand, in which case it feels safer to just agree,” Laajaj said.
Additionally, the idea of personality tests — or personality itself — may not be a natural concept everywhere. Understandably, people who aren’t familiar with the idea of personality testing might be a bit wary of revealing personal details about themselves.
“Imagine that you live in a poor area and someone comes to you to ask you a bunch of questions, such as how hardworking you are, whether you get stressed easily or whether you are a polite person. If it is not common for you to fill out surveys, or if it’s not clear what will be done with it, you may, for example, care more about giving a good impression than being completely truthful,” Laajaj said.
Tailor the Test
Data from standard personality tests are increasingly used by researchers, employers and even policymakers. Because of this, the researchers hope that their study will serve as a cautionary tale in revealing some of the limits of personality testing — that the tests we know are still stuck in a white, western world.
Taking measures to reduce survey bias and creating some innovative methods to capture personality traits of people in developing countries could go a long way toward making sure these tests are relevant to people of all walks of life. To see an example of a Big Five personality test for yourself, try the Big Five Project Personality Test that researchers gathered data from. It’s an open access test that collects responses for psychology research.
