Evaluate a Candidate’s Critical Thinking Skills

Authors: Christopher Frank, Paul Magnone, and Oded Netzer

Hiring is one of the most challenging competencies to master, yet it is one of the most strategic and impactful managerial functions. A McKinsey study quantified that superior talent is up to eight times more productive, showing that the relationship between talent quality and business performance is dramatic. Organizations ­seeking growth or simply survival during difficult times must successfully recruit A-list talent, thought leaders, and subject matter experts. This is often done under time constraints, as they must quickly fill a key position. Essentially, you are committing to a long-term relationship after a few very short dates.

Now let’s consider the typical process of hiring talent. The primary tool we use to assess talent is a set of job interviews. The typical interview process is a Q&A format where some version of the common questions are asked of the candidate: Tell us about your background. Why do you want to work for us? Share a challenge you faced at work and how you dealt with it. What are you most proud of? What impact have you had? Why are you the best person for the job? What is your greatest achievement? What are your weaknesses?

While much has been said in favor and against the value of interviews themselves, the typical Q&A job interview process fails to assess one of the most important, if not the most important, skills you should be looking for from people you hire—critical thinking. Critical thinking is seeking information from various sources, assessing its credibility, and determining its relevance and veracity. Often classified as a higher-order skill, critical thinking is not a single skill but a collection of skills involving reasoning, constructing sound arguments, and identifying a situation’s flaws, biases, logic, or inconsistencies. Critical thinking is different from creative thinking. Creative thinking is the ability to generate new, innovative ideas. Critical thinking requires the candidate to seek information by asking questions, carefully and logically analyze facts, and form a judgment.

Critical Thinking Is Fundamentally a Process of Questioning

A key insight from the research for our book Decisions Over Decimals, based on surveying and teaching thousands of executives, is that the smartest person in the room is not the one with the answer but the person asking the question. They are curious, engaged, unafraid, inquisitive, and ready to explore a new domain that may not have answers yet. By the nature of their questions, they demonstrate observation, analysis, inference, interrogation, interpretation, and explanation. Critical thinkers are curious. Innate curiosity has been shown to be associated with the following eight traits—avid learners, problem solvers, active listeners, self-driven, high productivity, growth mindset, overachievers, and strong at stakeholder management. These traits read like a wish list of qualifications for people you want to be part of your team.

Consulting firms and technology companies have pushed the standard battery of interview questions by using scenario-based or behavioral questions: How much should you charge to mow a lawn in Atlanta? Why are utility holes round? How would you design a wine rack for people who are blind? If you received $5 million to solve a global problem, what issue would you address and how?

These are a good step forward, but while answers to these questions will give you a good idea of candidates’ problem-solving skills, ability to deal with ambiguity, and creative thinking, they will not indicate if the candidates are curious, self-starters, or passionate about your company, products, culture, or any of the eight traits referenced earlier.

How do you identify individuals with this versatile and compelling mix of critical thinking and curiosity? Boldly flip the interview process.

The oldest and still the most powerful tactic for fostering critical thinking is the Socratic method, developed over 2,400 years ago by the Greek philosopher Socrates. The Socratic method uses thought-provoking question-and-answer probing to promote learning. It focuses on generating more questions than answers, where the answers are not a stopping point but the beginning of further analysis. Hiring managers can apply this model to create a different dialogue with candidates in a modern organization.

The Flip Interview

Let the candidate interview you. The flip interview is an alternate method to uncover a candidate’s intrinsic strengths, preferred ways of working, and how they think. The interview will showcase the candidate’s thinking and decision-making process and indicate if the candidate exhibits leadership traits.

The flip interview goes beyond letting the candidate ask you open-ended questions like, “What is the company culture?” or “What is it like to work for the company?” The interviewer comes to the discussion with a business scenario, an understanding of relevant information, and an invitation to the interviewee to guide discovery through a series of questions.

The interviewee may use four types of questions, listed below by increasing level of complexity and involvement.

• Factual questions: Questions that have straightforward answers based on facts or awareness.

• Convergent questions: Close-ended questions with finite answers. Typically, these questions have one correct answer.

• Divergent questions: Open-ended questions that encourage many answers. These questions are a means for analyzing a situation, problem, or complexity in greater detail and stimulate creative thought.

• Evaluative questions: Questions that require deeper levels of thinking. They can be open or closed. Evaluative questions elicit analysis at multiple levels and from different perspectives to arrive at newly synthesized information or conclusions.

Conducting the Flip Interview: A Four-Step Process

Use the following framework to maximize the value of your conversations with recruits:

1.Frame

Briefly describe a scenario. State up front that you are the source of information for the scenario and ask the ­interviewee to drive the next eight to 12 minutes with a straightforward “How would you start this discovery?” When they inevitably get stuck, prompt them with a branch of the decision tree that opens the discovery further. Invite them to ask questions.

Interviewer evaluation:

In the framing step, you evaluate how they frame the problem. Do they take the situation at face value or probe to get at the essence of the situation? The quality of the questions they ask should lead to determining the information they need, the effort required, and uncover the essential decision to be made. This first step is all about problem-finding more than problem-solving.

2.Link

Once they define the problem, invite them to ask questions about context: “Given how you framed the scenario, what other information would you like to know to work toward a recommendation?”

Interviewer evaluation:

You are looking to see if their questions lead them to put the scenario in context. Understanding the context is crucial. To truly put a situation in context, their questions should enable them to triangulate it by looking at it in (1) absolute terms, (2) over time, and (3) relative to what’s going on across the market, with customers or actions from competitors.

3.Interrogate

Based on the original scenario, combined with what they learned, ask, “What is the essential decision that is needed?” or “How has their understanding of the situation shifted?”

Interviewer evaluation:

Have they interrogated the situation properly? Are they able to start to formulate the narrative? You are assessing if their narrative is a summary or a synthesis: summary is statement of the data; synthesis is data plus judgment. The critical thinker will demonstrate how they can consume and synthesize different pieces of information in parallel to arrive at a deeper understanding of the scenario or decision needed.

4.Perform

In this final step, ask, “What are the immediate next steps you would take?”

Interviewer evaluation:

If they start to ask questions about key stakeholders, shadow influencers, advocates, or swing voters who need to be convinced, the candidate is thinking critically. Do they seek to learn about headwinds and tailwinds to enable them to move forward? Note: This is not about solving the problem or assessing if they develop a recommendation, but simply how they approach problem-solving and decisions.

Using a flip interview, you can evaluate the candidate’s logic and passion for the role based on their questions. Are the questions superfluous or consequential? Are the questions generic or specific? Do they ask not only factual or convergent but also divergent and evaluative questions? Does the candidate pivot, dive deep, and revisit a topic from different angles? Are the questions grounded in the context of the problem and its environment? As they ask you questions, it will enable you to determine if they are actively listening by adjusting their questions in real-time, pivoting, and probing.

A skilled questioner creates a cooperative dialogue to elicit new learning through a series of questions. They engage the other person. Their questions should lead to inferences and connections and open up viewpoints that are not apparent. This exploration mindset encourages trial and iteration; unexpected learning may originate from the discussion.

Successful candidates in the flip interview will demonstrate a critical thinking mindset. This is different from knowing analytic tools and methods. A critical thinking philosophy is a skill that almost every leader seeks, and many teams have only in short supply. Thinking analytically and critically includes being clear about the purpose of the essential question rather than wandering in the forest of data, being inquisitive not from a statistical perspective but questioning the initial face value, being able to connect the dots via synthesis, and eventually being able to tell an informed story that is based on deeper truths, judgment, and context, not just restating the initial facts.

Reactivity, insight, and ingenuity are needed for companies to thrive. You seek agile thinkers who can be growth champions, truth-tellers, customer stewards, and insight creators. The candidate who can conceptualize the problem, frame the situation, and ask more thoughtful questions will outperform those relying on textbook answers. Hiring talent capable of asking thoughtful questions is the key to building successful teams.

Finally, once you successfully recruit amazing people, you must create an environment where employees are motivated to work collectively yet feel valued and recognized for being curious and inquisitive. An entire organization with a growth mindset that embraces questions and curiosity can reframe challenges as opportunities and move more freely to adjust to business conditions.

Case Study: Microsoft Changes the Interview Process

While their interview was not labeled as a flip interview, a candidate applied for a job at Microsoft in 1998 and found himself in one. During the interview, the recruiter prompted: “Sell me a toaster.” The candidate paused. Then they started to ask questions: “How many people are in the family?” “Are there young children in the house?” “What is the age range of toaster users in the family?” “Do they live in a small space, such as an apartment, or a large house?” “Are they big breakfast eaters, or is their main meal dinner?” “How often do they eat out?” For three minutes, it was a stream of questions about demographics, psychographics, and physical space. The recruiter asked the candidate to stop and asked what this had to do with selling a toaster. The candidate replied, “Everything.”

It is important to discover and understand the context of the “toaster environment” to ensure I sell you what you need. If I match the toaster features and functions to your needs, the success of the sales increases, and you will be a satisfied customer. Do you need a toaster oven, a pop-up two-slice toaster, a four-slice, a wide-slice toaster, a conveyor toaster, a commercial toaster, or a convection oven? Maybe a panini press would serve your needs best.

The exercise was not about the toaster: The recruiter was looking to see how the candidate handled ambiguity. Were they a critical thinker? Did they dive right in with a sales pitch or pause, learn, and pivot? In case you are wondering, the candidate got the job.

Christopher Frank is the coauthor of Decisions Over Decimals: Striking the Balance between Intuition and Information and Drinking from the Fire Hose: Making Smarter Decisions Without Drowning in Information. He has 25 years of experience with Microsoft and American Express. He is the chief client officer at PSB Insights, a WPP company.

Paul Magnone is the coauthor of Decisions Over Decimals: Striking the Balance between Intuition and Information and Drinking from the Fire Hose: Making Smarter Decisions Without Drowning in Information. He currently serves as the head of global strategic alliances for Google.

Oded Netzer is the coauthor of Decisions Over Decimals: Striking the Balance between Intuition and Information. He is the Vice Dean for Research and the Arthur J. Samberg Professor of Business at Columbia Business School, an affiliate of the Columbia Data Science Institute, and an Amazon Scholar.