That the SAT is "an imperfect measurement" is true. It is also true that everything else in a college application is an imperfect measurement of the qualities being looked for, and the SAT is relatively reliable by comparison, particularly considering it is a three hour test. It does not primarily measure "a young person’s preparation for college-level work." Academic preparation is better measured by AP test scores, the old SAT II achievement tests, and, for high schools that have rigorous required courses and honest grades (are there any left?), High School GPA. The SAT is what for much of its history it was called, a "scholastic aptitude test." It is useful in identifying (1) students with the potential to do well in college who have relatively unimpressive high school grades because of lack of diligence, unchallenging schools, or for other reasons, and (2) students who have done well in high school due to exceptional diligence or easy grading but will probably be unable to duplicate that success at the college level. It measures, imperfectly, academic potential, not preparation. Sending students to a year of prep school after graduation might make them better prepared for college, but it would not cure low SAT scores.
It is inevitable that selective colleges will return to the SAT or something equivalent. These colleges are prepared to make significant academic quality tradeoffs to achieve their diversity objectives, but within each applicant category they want the smartest kids they can find, and the SAT is the most useful tool that is readily available to identify those kids. Colleges which use SAT scores in admissions will have a significant advantage in terms of student body academic quality over those which do not.
The recent SAT studies cited simply confirm the findings of the truly enormous body of previous SAT studies and technical literature which the College Board and independent researchers have amassed over the last century.
“In response to a civil rights lawsuit in 2019, the University of California system banned consideration of SAT and ACT in admissions altogether (test-blind, not test-optional). More on the UC system later.”
The reporting on this has been bad so I don’t blame you for this frame. This really should be seen as a collusive use of the Court system to evade democratic accountability. The UC Regents had been looking to ban testing, and had in fact commissioned an internal academic study that they expected would show it was racist and not useful to predict student outcomes. The faculty study did not in fact find that, and recommended keeping the SAT. https://senate.universityofcalifornia.edu/_files/underreview/sttf-report.pdf#page66
They then turned to this case as a reason to do what they already wanted to do. It essentially was not an adversary court proceeding at all.
Yale has just announced that it is abandoning its test-optional policy. The admissions office says that "our researchers and readers found that when admissions officers reviewed applications with no scores, they placed greater weight on other parts of the application. But this shift frequently worked to the disadvantage of applicants from lower socio-economic backgrounds." Of course Yale could offset this disadvantage simply by applying a heavier thumb to the scale in favor of lower socio-economic applicants. And the disadvantage presumably was limited to those applicants who had relatively high SAT scores but did not submit them, which is probably rare. Yale's problem with the test-optional policy is not that applicants from lower socio-economic backgrounds are disadvantaged, but that applicants from all backgrounds who test poorly are advantaged. And "Yale’s research from before and after the pandemic has consistently demonstrated that, among all application components, test scores are the single greatest predictor of a student’s future Yale grades. This is true even after controlling for family income and other demographic variables, and it is true for subject-based exams such as AP and IB, in addition to the ACT and SAT."
It strikes me that, philosophically and politically, the problem with the framing of this argument is that it imagines a just outcome in terms of the correct distribution of winners - and therefore losers. If only we can show that a principle of selection is fairer, then we can comfort ourselves that all those future low earners will have deserved it this time. That is: Being poor and exploited is implicitly okay, providing the reason you’re poor and exploited isn’t because you’ve been discriminated against. Remove the discrimination from the system and - yay - you’ll get the correct people on the top and bottom.
That’s what’s a little distasteful, to my mind, about all the talk of those future high earners. Not only does it assume that winning the income race (with the correct group of low earners wheezing in last) is a central goal of education, but it assumes that all those high earners are worth hosting at all in an equitable and desirable society. That’s to say: if the goal of college is yet more consultants for McKinsey, corporate lawyers, and hedge fund managers, then I can’t see why we should be worrying so much about getting the “right” people in. I’d be more inclined to regard the kids who become, say, public school teachers as the successes here. And until teachers are paid more than hedge fund managers, adequately matching test scores with future high earnings is never going to be an indicator of a more just system.
In sum, while your arguments here are obviously valuable, and presented sensibly and in good faith, I think we should remain concerned about any vision of education founded on the free market principle that you have to select the right winners and losers. Thinking about who might flourish in a class on the novel, a design lab, or a theoretical physics seminar is nothing like allocating young people to the income bracket that best matches their quantifiable capacities or achievements.
Besides, the most reliable and persistent correlation amid all of this is well known: that between elite college admission and family wealth (NB: not income). In that context, any overlay of tests that makes the system appear more just surely counts as a mystification. The best argument against SATs is not that they’re discriminatory but that they propagate an impoverished view of education and what it is for.
"Jesse Rothstein argues that the findings above only hold at highly selective colleges where almost every applicant has a high GPA, and thus the poor prediction is due to range restriction."
But doesn't the range restriction issue also apply to test scores as well? And haven't broader studies (e.g. the Johns Hopkins CTY stuff) shows that test scores do indeed have significant predictive power into adulthood?
Given the range restriction in SAT, another measure (perhaps narrow and self-selecting) might be AMC-12 scores. These are much better in sampling the tail (see this nice article by Ellison and Swanson - https://faculty.wharton.upenn.edu/wp-content/uploads/2016/02/manuscript_and_appendix.pdf). I wonder if the AMC-12 scores have similar correlations in their college success.
That the SAT is "an imperfect measurement" is true. It is also true that everything else in a college application is an imperfect measurement of the qualities being looked for, and the SAT is relatively reliable by comparison, particularly considering it is a three hour test. It does not primarily measure "a young person’s preparation for college-level work." Academic preparation is better measured by AP test scores, the old SAT II achievement tests, and, for high schools that have rigorous required courses and honest grades (are there any left?), High School GPA. The SAT is what for much of its history it was called, a "scholastic aptitude test." It is useful in identifying (1) students with the potential to do well in college who have relatively unimpressive high school grades because of lack of diligence, unchallenging schools, or for other reasons, and (2) students who have done well in high school due to exceptional diligence or easy grading but will probably be unable to duplicate that success at the college level. It measures, imperfectly, academic potential, not preparation. Sending students to a year of prep school after graduation might make them better prepared for college, but it would not cure low SAT scores.
It is inevitable that selective colleges will return to the SAT or something equivalent. These colleges are prepared to make significant academic quality tradeoffs to achieve their diversity objectives, but within each applicant category they want the smartest kids they can find, and the SAT is the most useful tool that is readily available to identify those kids. Colleges which use SAT scores in admissions will have a significant advantage in terms of student body academic quality over those which do not.
The recent SAT studies cited simply confirm the findings of the truly enormous body of previous SAT studies and technical literature which the College Board and independent researchers have amassed over the last century.
As a lazy high school student with a 1580 SAT I would caution that turning in your homework in high school is definitely a skill which carries over.
“In response to a civil rights lawsuit in 2019, the University of California system banned consideration of SAT and ACT in admissions altogether (test-blind, not test-optional). More on the UC system later.”
The reporting on this has been bad so I don’t blame you for this frame. This really should be seen as a collusive use of the Court system to evade democratic accountability. The UC Regents had been looking to ban testing, and had in fact commissioned an internal academic study that they expected would show it was racist and not useful to predict student outcomes. The faculty study did not in fact find that, and recommended keeping the SAT. https://senate.universityofcalifornia.edu/_files/underreview/sttf-report.pdf#page66
They then turned to this case as a reason to do what they already wanted to do. It essentially was not an adversary court proceeding at all.
Yale has just announced that it is abandoning its test-optional policy. The admissions office says that "our researchers and readers found that when admissions officers reviewed applications with no scores, they placed greater weight on other parts of the application. But this shift frequently worked to the disadvantage of applicants from lower socio-economic backgrounds." Of course Yale could offset this disadvantage simply by applying a heavier thumb to the scale in favor of lower socio-economic applicants. And the disadvantage presumably was limited to those applicants who had relatively high SAT scores but did not submit them, which is probably rare. Yale's problem with the test-optional policy is not that applicants from lower socio-economic backgrounds are disadvantaged, but that applicants from all backgrounds who test poorly are advantaged. And "Yale’s research from before and after the pandemic has consistently demonstrated that, among all application components, test scores are the single greatest predictor of a student’s future Yale grades. This is true even after controlling for family income and other demographic variables, and it is true for subject-based exams such as AP and IB, in addition to the ACT and SAT."
It strikes me that, philosophically and politically, the problem with the framing of this argument is that it imagines a just outcome in terms of the correct distribution of winners - and therefore losers. If only we can show that a principle of selection is fairer, then we can comfort ourselves that all those future low earners will have deserved it this time. That is: Being poor and exploited is implicitly okay, providing the reason you’re poor and exploited isn’t because you’ve been discriminated against. Remove the discrimination from the system and - yay - you’ll get the correct people on the top and bottom.
That’s what’s a little distasteful, to my mind, about all the talk of those future high earners. Not only does it assume that winning the income race (with the correct group of low earners wheezing in last) is a central goal of education, but it assumes that all those high earners are worth hosting at all in an equitable and desirable society. That’s to say: if the goal of college is yet more consultants for McKinsey, corporate lawyers, and hedge fund managers, then I can’t see why we should be worrying so much about getting the “right” people in. I’d be more inclined to regard the kids who become, say, public school teachers as the successes here. And until teachers are paid more than hedge fund managers, adequately matching test scores with future high earnings is never going to be an indicator of a more just system.
In sum, while your arguments here are obviously valuable, and presented sensibly and in good faith, I think we should remain concerned about any vision of education founded on the free market principle that you have to select the right winners and losers. Thinking about who might flourish in a class on the novel, a design lab, or a theoretical physics seminar is nothing like allocating young people to the income bracket that best matches their quantifiable capacities or achievements.
Besides, the most reliable and persistent correlation amid all of this is well known: that between elite college admission and family wealth (NB: not income). In that context, any overlay of tests that makes the system appear more just surely counts as a mystification. The best argument against SATs is not that they’re discriminatory but that they propagate an impoverished view of education and what it is for.
An interesting take on the problem (from Gail) …. https://gadfly4.wordpress.com/2024/02/20/standardized-tests-what-is-the-problem/
"Jesse Rothstein argues that the findings above only hold at highly selective colleges where almost every applicant has a high GPA, and thus the poor prediction is due to range restriction."
But doesn't the range restriction issue also apply to test scores as well? And haven't broader studies (e.g. the Johns Hopkins CTY stuff) shows that test scores do indeed have significant predictive power into adulthood?
Given the range restriction in SAT, another measure (perhaps narrow and self-selecting) might be AMC-12 scores. These are much better in sampling the tail (see this nice article by Ellison and Swanson - https://faculty.wharton.upenn.edu/wp-content/uploads/2016/02/manuscript_and_appendix.pdf). I wonder if the AMC-12 scores have similar correlations in their college success.