Tim Simpson via www.flickr.com creative commons
Nobel-prize-winning psychologist Daniel Kahneman realized how true this was while working as a psychological evaluator for the Israeli Defense Forces in the 1950s, when he saw that an extensive test for officer candidates boiled down to intuitive judgments that were close to worthless and that, due to a bias he called the illusion of validity, evaluators continued to trust in their method even when it was found to be ineffective.
In order to counteract biases, Kahneman designed a new method for evaluating officer candidates, which involved coming up with multiple quantifiable criteria, evaluating choices based on those criteria, and trusting in the result.
In effect, he was forcing evaluators to think like a computer. Although the new approach nearly caused mutiny, it was soon shown to be more effective.
The psychologist recommends using a similar method for various hard decisions, as he writes in his 2011 book, "Thinking, Fast And Slow":
Suppose that you need to hire a sales representative for your firm. If you are serious about hiring the best possible person for the job, this is what you should do.
First, select a few traits that are prerequisites for success in this position (technical proficiency, engaging personality, reliability, and so on. Don't overdo it - six dimensions is a good number. The traits you choose should be as independent as possible from each other, and you should feel that you can assess them reliably by asking a few factual questions.
Next, make a list of those questions for each trait and think about how you will score it, say on a 1-5 scale. You should have an idea of what you will call "very weak" or "very strong."
These preparations should take you half an hour or so, a small investment that can make a significant difference in the quality of the people you hire. To avoid halo effects, you must collect the information on one trait at a time, scoring each before you move on to the next one. Do not skip around.
To evaluate each candidate add up the six scores ... Firmly resolve that you will hire the candidate whose final score is the highest, even if there is another one whom you like better - try to resist your wish to invent broken legs to change the ranking.
A vast amount of research offers a promise: you are much more likely to find the best candidate if you use this procedure than if you do what people normally do in such situations, which is to go into the interview unprepared and to make choices by an overall intuitive judgment such as "I looked into his eyes and liked what I saw."
Although Kahneman does not explicitly discuss using this process outside of applicant evaluations, there's no reason it could not be applied widely.
Indeed, Business Insider used it when evaluating the best employers in America, and I used it when choosing between New York City apartments. It could also be used to choose between investments, video game consoles, or suitors. It could even be used in binary decisions, especially if decision-makers also adopt the wise strategy of converting those binary decisions into non-binary decisions.
Broadly speaking, the insight is to use coolly logical (and often quantifiable) rules to guide any decisions that may be subject to significant bias: In other words, force yourself to think like a computer.