The trouble with holistic admissions

Through the looking glass, darkly

Welcome back! Yesterday, we talked about why some people benefit so much from an Ivy-plus education. Today I want to talk about holistic admissions.

Some people think that elite private colleges are poisoning America. Others want to abolish the Ivy League colleges or tax them into oblivion. In a New Republic piece a few years ago, William Deresiewicz delivered a fiery diatribe against selective colleges, provocatively titled “Don’t send your kid to the Ivy League.” His article – and the accompanying book “Excellent Sheep” – criticizes Ivy League schools for putting kids through a punishing application process that rewards meritocracy above all else, and produces “young people who are smart and talented and driven, yes, but also anxious, timid, and lost, with little intellectual curiosity and a stunted sense of purpose: trapped in a bubble of privilege, heading meekly in the same direction, great at what they’re doing but with no idea why they’re doing it.”

That’s not how I see it. The Harvard college students I know are smart and talented of course, but also ambitious, purposeful, and interested in the world around them. They are brilliant, well-rounded, kind and generous. Not excellent sheep, just excellent.

(I have an inside view on this question because I live on Harvard’s campus with my family. My wife and I are the Faculty Deans of Kirkland House, one of the twelve undergraduate houses at Harvard.¹ There are about 400 students in Kirkland, and we get to know many of them really well. We eat meals in the dining hall and host regular gatherings in our home, which is attached to the student dorms. We hand out diplomas at graduation. I see them come in as anxious sophomores and leave as mature, thoughtful seniors, ready to make their marks on the world.)

Holistic admissions is really hard

Fixing selective college admissions would be much easier if we were choosing the wrong kids, as Deresiewicz argues. The problem is not that we pick the wrong people, but rather that there are far too many to choose from.

About 2 million high school seniors take the SAT each year (a bit less in recent years). Another 1.5 million take the ACT. By definition, 35 thousand of them score in the top 1 percent (e.g an SAT score of 1550 or higher and an ACT score of 34 or higher - there may be some overlap across high scorers, so let’s call it 30 thousand conservatively).  There are less than 20 thousand students each year in the entire freshman class of all 12 Ivy-Plus colleges. Even if we filled these classes straight down the list of the highest-scoring students in the country, not everyone in the top 1 percent would get a spot.

As one of the most desirable colleges, Harvard could fill its incoming class several times over taking only students with perfect SAT or ACT scores. Harvard students deserve to be here, but so do many, many others. There is so much talent in the world that any admissions process at a place attracting the best applicants will end up throwing darts at a board at some point. It’s better for everyone if we internalize this reality and respond accordingly.²

Selective colleges have responded to their incredible surplus of applicants with holistic admissions policies, which explicitly look beyond applicants’ grades and test scores at factors like extracurricular talents, moral character, and diversity.

Holistic admission has a sordid history at Harvard. In 1926, Harvard President Abbott Lawrence Lowell switched from exam-based admissions to holistic review, allegedly to limit the number of Jewish students at the college.³

Happily, Harvard ended that particular practice. More recently, the stated goal of holistic admissions has been to admit a class with diversity along a variety of dimensions, including race but also religion, national origin, and political beliefs. And, since explicit racial quotas were outlawed by the Supreme Court in the Bakke decision in 1978, it’s fair to think of holistic admissions as achieving racial and ethnic diversity goals through implicit means.

I think Harvard and other selective colleges should be racially and socioeconomically diverse. But leaving aside the legal issues, I believe quotas or more flexible - but explicit - targets would be better than holistic admissions.

The premise of holistic admission is that if we stare long and hard enough at a 17-year-old kid’s self-presentation on their college application, we can discern who is most worthy. I think that is folly, and our paper’s results back me up.

Non-academic factors don’t predict success

In Section 5 of the paper, we ask which applicant characteristics are associated with better post-college outcomes. In other words, who benefits most from an Ivy-Plus college education? Doing this right is tricky, because students differ not only in overall potential, but also in the next best alternative college they would have attended. For example, suppose the impact of attending a particular college is much greater for legacies. This could be because legacies derive more benefit from attending an Ivy-Plus college than non-legacies, or because the legacies who are not admitted fall much further down in the college quality distribution.

Without getting technical, we develop a procedure to account for these different channels. The intuition is in Figure 17. Panel A shows the impact of attending an Ivy-Plus college relative to other students – for example the value of -0.7 for legacy students in the solid bar (“raw comparison”) means that legacies are 0.7 percentage points less likely to earn high incomes compared to non-legacies. The shaded bar labeled “VA comparison” shows the difference in college “value-added” for the same outcome between admitted and rejected students. A value of 0.3 means that Harvard is 0.3 percentage points better at producing high earners than the average of other schools attended by rejected Harvard legacies. In other words, the legacies don’t fall very far. Subtracting the first from the second gets you a value of -1.0 (panel B), which tells us that legacies who attend Ivy-Plus colleges are 1 percentage point less likely to earn in the top 1% than non-legacies.

Figure 17A: Post-College Outcomes by Application Credentials Among Ivy-Plus Applicants; Predicted Top 1% Outcomes vs. College Value-Added

Panel B shows the same comparison for athletes and for students with high non-academic and academic ratings respectively. Panels C and D (shown below in the footnote) repeat the exercise for non-monetary outcomes – attending a top graduate school and working in a prestigious firm.⁴

Figure 17B: Post-College Outcomes by Application Credentials Among Ivy-Plus Applicants; Differences in Predicted Chance of Reaching Top 1%

All else equal, applicants with high non-academic ratings don’t do any better in terms of earnings or graduate school attendance. However, academic ratings strongly predict later life success. The implication is clear, at least to me – the “extras” in a s student’s application privileged by holistic admissions policies don’t do a very good job of telling us who is going to succeed.

However, this doesn’t mean that classroom achievement is the only thing that matters. Personal qualities like leadership and character are important in school, and in life. If we could measure those attributes well, we would. The problem is that the college application is a bad way to evaluate someone - it asks too much. Holistic admission is well-intended, but at some point you really aren’t learning anything by asking applicants to write multiple essays about their life perspective or to demonstrate their deep commitment to 17 different extracurricular activities.

Money buys distinctiveness

Even worse than not learning anything, you are probably biasing your process in favor of the rich. Wealthy families understand that you must “stand out” in some way to be selected from an extraordinarily crowded and talented applicant pool. So they choose the right schools who know just how to market their child to selective colleges. They hire private counselors who help them carefully curate a persona that seems appealing to admissions officers. The list goes on.

Money buys distinctiveness; the thousands of high achieving middle-class kids just start to look the same after a while. Again, not because they are the same, but because they couldn’t play the game as effectively. They may not even know the rules.

Our paper shows how rating applicants on non-academic factors favors the wealthy, even though it doesn’t predict success. But an even bigger problem with holistic admissions is its purposeful ambiguity. Nothing is guaranteed, and you never quite know where you stand. For achievement-oriented kids (and parents), that ambiguity means that no college preparation is ever good enough. We’ve created an unhealthy college admission culture centered around gaming the system and figuring out “who gets in, and why?” That’s bad, and it’s likely to get worse as more schools go test-optional.

Tomorrow I’ll close out the week with some thoughts about where we go from here.

1. Our most famous alum is Mark Zuckerberg. The Social Network was a good movie, but it did violence to the truth with its sumptuous portrayal of Kirkland House dormitories.

2. The extreme selectivity of a few schools is a relatively recent phenomenon, as Caroline Hoxby shows in her excellent 2009 paper “The Changing Selectivity of American Colleges”. Half a century ago, many fewer people attended college, and those that did typically stayed close to home. Today, selective colleges attract talent from all around the world.

3. In a 1922 letter to Alfred A. Benesch ‘00, Lowell wrote “If [the] number [of Jews] should become 40 percent of the student body, the race feeling would become intense. If every college in the country would take a limited proportion of Jews, I suspect we should go a long way toward eliminating race feeling among students.”

4. Figures 17C and 17D, shown below, describe the same pattern as described 17A and 17B, but through measurements of different outcomes.