Developing Structured Medical School Interviews: Content and Evaluation Related Components

Overview and Intended Audience

This section includes guidance regarding:

• Developing structured interview questions.
• Developing a structured interview evaluation process.

This content is best suited for individuals who serve on admissions committees and/or those who contribute to developing the interview content and establishing the interview processes at your medical school. Those who serve as interviewers may benefit from some of this content as well.

Behavioral and Situational Questions

There are many types of interview questions. Two of them — behavioral and situational questions — have been widely studied and are considered relatively structured.

• Behavioral questions are based on the premise that past behavior predicts future behavior. They ask applicants to describe what they did in a previous context (typically, in previous jobs, at school, or in volunteer experiences) that provide evidence of important competencies. Past-behavior questions often ask an applicant to describe a specific situation, the behavior or action they took, and the outcome or consequence of that behavior.

Example: Tell me about a time when you were forced to make an unpopular decision. What actions did you take before and after making the decision?

• Situational questions are based on the premise that intentions predict future behavior. They pose hypothetical situations that might occur at medical school or as a physician and ask applicants to describe how they would respond in the situations.

Example: I’d like you to imagine that you are in one of your clinical courses. The professor describes a difficult project that you and another M-1 worked on earlier in the week and compliments your handling a difficult situation. In doing so, she gives you sole credit and fails to mention that your colleague played a major role. What would you do?

Refer to the Sample Structured Interview Guide for example behavioral and situational interview questions for select AAMC Core Competencies.

Maintain a balance between the number of competencies you want to assess, the number of questions needed to assess them, and the amount of time you have available for each interview.

Evaluation-Related Components

A best practice for structured interviews is to use rating scales to evaluate applicants’ responses. In addition to enhancing reliability and validity, rating scales will also increase interviewers’ ability to compare applicants because they are evaluated on a common scale.

There are five key points to note about rating scales:

1. A best practice is to develop rating scales for each competency that the interview was designed to assess.
2. They can be developed to work with behavioral and situational questions.
3. Rating scales often vary in numbers of scale points, but typically range from three to five. The chosen number of points corresponds to the range of ability levels or proficiency levels commonly observed among applicants, with wider ranges accommodating more levels within the rating scale.
4. Ideally, each point on the rating scale is described with some specific behavioral examples that reflect that particular level of proficiency. The behavioral examples on the rating scales should reflect real expectations of each level of performance (in this case, for first-year medical students), providing raters with common definitions for each point on the scale. This will both make the rating task easier for raters and help ensure that applicants are being evaluated in a consistent manner.
5. Interviewers should be instructed to use the behavioral examples on the rating scale as a general guide for evaluating applicants’ responses.

Refer to the Sample Structured Interview Guide for an example of a behaviorally anchored rating scale for select AAMC Core Competencies.

Key Steps for Developing Interview Content

Below we’ve outlined key steps for developing structured interview content:

1. Identify key requirements of the position.
2. Determine which three to five competencies to target in the interview.
3. Develop behavioral or situational questions for each competency (refer to the Sample Structured Interview Guide for examples).
4. Invite all parties who interact with medical students to review questions and discuss how applicants would respond.
5. Use responses to create draft behavioral examples for each point on the scale (refer to the Sample Structured Interview Guide for examples).
6. Review content for cultural appropriateness and understanding, as well as alignment with program goals.
7. Document the process and explain how the target competencies and questions were selected.
8. Train a pilot group of three to five interviewers on how to use the structured content and rating scale.
9. Train all interviewers on how to use the guide and rating scale.

Sample Structured Interview Guide

Sample interview guide overview

Below is a sample interview guide. The guide can be adapted to meet your medical school’s specific needs. Example structured interview questions and evaluation content are included for reference for two sample competencies: Service Orientation and Teamwork. Note that while only two competencies are included in the example guide, your medical school may evaluate different and/or more competencies in your interviews.

During the interview: opening, body, and closing

A note on probing questions

You can use the following probing questions to help you better frame your interview questions. The questions below are rooted in the situation, task, action, result (STAR) method — a technique for answering behavioral and situational questions in an organized and effective manner. As the interviewer, you will play a key role in ensuring a full STAR answer is given for all competency-based questions.

Applicants should focus on one specific scenario in their response for each question rather than describing what they have done in general or describing multiple situations, and part of your role is to keep them focused on one strong situation that addresses the question. You will need to ask as many of these probing questions as necessary to ensure you get a full STAR response from the applicant. For example, the applicant might provide an answer with the situation, task, and action, but fail to elaborate on the result. In that case, you will need to ask the follow-up questions about the outcome or result.

Probes for Situational Interview Questions*

Probes for Behavioral Interview Questions

Situation or Task: What do you consider the most critical issue in this situation? What other issues are of concern? Action: What would you say? What is the first thing you would do? What factors would affect your course of action? What other actions could you take? Results: How do you think your action would be received? What would you do if your action was not received well? What do you consider benefits of your action?

Situation or Task: What factors led up to the situation or task? Could you or anyone else have done something to prevent the situation or task? What did you determine as the most critical issue to address in this situation or task? Action: How did you respond? What was the most important factor you considered in taking action? What is the first thing you did? Results: What was the outcome? Is there anything you would have said and/or done differently? Were there any benefits from the situation?

* Note that for situational questions, the majority of the applicant’s response will be comprised of Action and Result since the Situation/Task is provided to them in the question.

Interview opening script

Hello, my name is [NAME], and I am the [ROLE] at [INSTITUTION]. Our interview today will take approximately [LENGTH OF TIME] and will include questions that ask you about specific past experiences or how you would handle hypothetical situations. When responding to a question, it is recommended that you tell us a story about what happened; specifically, I would like to hear a short description of the situation or task, the actions you took, and the result of your actions. It is most helpful to focus on one specific example when responding to questions rather than speaking about your experiences in general. You may also discuss group- or team-based examples, but please focus on your specific individual contributions and actions when doing so.

Please let me know if you would like me to repeat any questions during the interview. You may choose to take notes to help collect your thoughts before responding. I will also be taking notes throughout the interview. We will have a few minutes at the end where I can respond to specific questions you have about our medical school.

Do you have any questions for me before we get started?

Interview questions

Interpersonal and Communication Skills — effective information exchange and teaming with patients, their families, and other health professionals

#	Choose 1 Question (Ask follow-up probing questions as needed
1	Imagine that you need to communicate difficult news to a patient’s family. Walk me through your approach.
2	Tell me about a time when communicating effectively with other medical professionals helped you improve patient care.
3	Tell me about a time when it was difficult to communicate with a patient. What made communication difficult, and how did you navigate the situation?

Professionalism — a commitment to carrying out professional responsibilities, adherence to ethical principles, and sensitivity to a diverse patient population

#	Choose 1 Question (Ask follow-up probing questions as needed
1	Imagine that you are part of a medical team caring for a transgender patient. One of your fellow residents keeps addressing the patient with different pronouns than the patient prefers. What would you do?
2	Please provide an example of a time when you had to adjust a treatment for a patient due to their culture or background.
3	Imagine that you are part of a medical team caring for a young patient. You learn that one of your fellow residents inadvertently provided a higher dose of a medication than intended to the patient. It will not have any substantial health consequences. What would you do in this situation?

Teamwork — works collaboratively with others to achieve shared goals, shares information and knowledge with others and provides feedback, and puts team goals ahead of individual goals

#	Choose 1 Question (Ask follow-up probing questions as needed
1	Tell me about a time you worked well as part of a team.
2	Tell me about a time you faced an obstacle while working on a group project. What was the obstacle, and how did you overcome it?
3	What would you do if you were assigned to work on an important group project with someone whose personality was very different from yours?

Interview closing

This concludes our interview. Thank you so much for your time and for providing information that allows us to get to know more about you and experiences. [NEXT STEPS TO EXPECT IN SELECTION PROCESS AND GENERAL TIMELINE].