Ultrasound Student Assessment: Statistical Analysis

Research on How to Best Select Students for Ultrasound Programs (Part 4)

So if the title did not scare you off, then I know you are ready for this article. In this fourth in the series (part 1, part 2, part 3) of blogs related to a better selection process for ultrasound students, we discuss the statistical analysis resulting from phase 1 data.

The Metric:

As mentioned earlier, this comparison weighs the various schools’ admission practices and student characteristics against stated graduation rates.  A positive correlation between a parameter and the graduation rate implies that an increase in that specific parameter generally would produce a higher rate of graduation. Conversely, a negative correlation implies that an increase in that specific parameter would result in a decrease in graduation rates. A correlation close to zero implies that an increase in a specific parameter can result in an increase, a decrease, or no change in the graduation rates. Also as mentioned previously, although not perfect, graduation rates represented the best non-subjective metric I could create, given no ability to perform specific testing during this first phase.

Does it make Sense?

Before analyzing a bunch of different parameters, I wanted to get a feeling for whether or not the data, at a top level, made sense. The first two parameters I compared were selectivity into the ultrasound programs and graduation rates. Since the survey asked the average number of applicants and the number of students accepted, determining selectivity numerically was relatively simple. Selectivity was defined as the number of applicants rejected divided by the total number of applicants, or:

.

Therefore, a school that accepted all students who applied (Rejected 0) would have a selectivity of 0 whereas a school that had many more applicants than accepted registrants would have a high selectivity percentage. When a correlation was run between selectivity and graduation rates, the positive correlation was quite high, 0.385. To be completely certain that this result was not random, I also checked the correlation between the number of students accepted vs. graduation and the number of students applied vs. graduation, both of which gave correlations which rounded off to 0. This result indicates that selectivity clearly relates to higher graduation rates, and that graduation rate is probably a fairly good metric.

Greatest Negative Correlation:

The survey asked questions regarding the entrance requirements of schools including if the school: offered an entrance exam, required a high school diploma, completion of (some) college courses, a college degree, a specific GPA, and recommendations and evaluations. The greatest negative correlation (-0.223) resulted with programs that required a high school diploma only. In other words, programs that did not have higher requirements had lower graduation rates.

The Surprise Result:

There was a negative correlation (-0.184) between schools which offered an entrance exam and graduation rates. More on this in a bit (but yes, I was initially shocked too).

What attribute (as ranked important by instructors) correlated best with graduation?

Instructors were asked which attribute they considered most important for students to be successful in ultrasound: Choices were:

• GPA
• Good recommendations
• Communication skills
• Work experience
• Advanced education
• Persistence
• Work ethic
• Reading skills
• Problem solving
• Study habits
• Social Skills
• Patience
• Innate ability
• Spatial ability

Which category do you think had the greatest correlation with graduation rates?

Click to show answer:

Summaries based on Statistics against the metric:

• There was a strong positive correlation with selectivity (more selective programs had higher graduation percentages)
• In terms of selection criteria, requiring a high GPA had the highest correlation with graduation rate
• Requiring a high school diploma (only) had the highest negative correlation with graduation rates
• The ability to problem solve showed a very high correlation with graduation based on both instructor opinion and when programs specifically tested students for problem-solving skills as part of the admission process.
• There was a negative correlation between schools which used an entrance exam and graduation rates.

More about the Surprise:

As mentioned earlier, I was very surprised at the negative correlation between schools using an entrance exam and graduation rates. With further analysis, I found three reasons which help to explain these surprise findings:

• Schools which offered an exam, (from the statistics) often relied too heavily on the exam and not enough on other admission practices
• The entrance exams were likely not geared (or ideally geared) toward what really matters in terms of ultrasound student selection.
• There are clear indications that there are many characteristics, if tested for, could help determine a priori who would more likely succeed.

So What Next:

When I delivered the presentation last year, I gave the following recommendations for follow-up.

1. Develop a testing rubric
   1. Specify methodology
   2. Create a “better” metric
   3. Include existing standardized tests?
   4. Create specialty tests?
   5. Create data collection system (anonymous and rigorous)
2. Get willing schools to participate in the research
3. Collect individual students data to test against
4. Use results to determine what aspects of testing are most beneficial
5. “Publish” new testing paradigm