ANALYSIS: When Metrics Are the Endgame, Data Fails Us - Ask Youngkin
The new CNBC ranking of the best states to do business is out. Virginia dropped; Youngkin's screaming 'foul.' If he'd only do that with school data.
By Martin Davis
EDITOR-IN-CHIEF
Email Martin
“Virginia is the best place to live, work, and raise a family.”
If you’ve heard Gov. Glenn Youngkin speak, you’ve likely heard this quote, which he has used repeatedly throughout his four years in office.
He will also frequently cite CNBC’s “America’s Top States for Business” to back up his quote. And it was an effective piece of evidence … until Thursday.
Virginia ranked No. 1 in 2019, 2021, and 2024 (there was no poll in 2020 due to COVID). The commonwealth ranked 3rd in 2022, and 2nd in 2023.
This morning, however, the Old Dominion State fell to fourth.
That drop is due largely to a new metric introduced for this year’s ranking.
Writing in the Virginia Scope, Brandon Jarvis described it this way: “With Virginia receiving its lowest ranking in years, it is worth noting that CNBC introduced a new metric to evaluate states this year: tariffs and federal jobs cuts — two priorities of the Trump administration that Youngkin and Republican gubernatorial nominee Winsome Earle-Sears have defended.” (To read why CNBC added this metric, see the ranking’s website.)
In a lengthy statement on X, Youngkin wrote: “CNBC’s new methodology this year is thrown off by a new subjective metric that mistakenly ascribes substantial risk to Virginia from the federal government’s presence in the Commonwealth.”
Fair enough. Every metric, to one degree or another, can be pegged with the label “subjective.”
The question becomes, which subjective metrics do you embrace, and which ones do you quietly ignore?
How people answer that question tells us a lot about how they approach data. Is it a tool to deepen our knowledge and learn how to improve, or is it just one more thing we reach for to tell ourselves we’re right?
Youngkin is clearly choosing the latter. How do we know?
In 2024, when Virginia regained the top spot, Youngkin crowed. But as we pointed out, he didn’t bother to discuss the “subjective” categories that were responsible for boosting Virginia to No. 1 — the No. 1 ranking CNBC gave the commonwealth’s public schools, which Youngkin has consistently accused of failing; and Virginia’s No. 19 Quality of Life ranking, which was higher than for any other Southern state because Virginia protects the right to an abortion; a right Youngkin tried, and failed, to limit.
In the high-minded world of academia, what Youngkin is guilty of is cherry-picking. The only data that’s legitimate is the data a person chooses to agree with.
He’s hardly alone in so acting.
And if Youngkin and others would stop for just a moment to think about it, they would realize how they’re approaching data renders it useless.
If we celebrate only data that supports our opinion, then data is simply an endgame to a self-serving end.
To wrestle with data when it challenges our thinking is to appreciate data as a tool that can help us resolve problems and improve.
Youngkin’s disingenuous reaction to CNBC’s new ranking helps us see this clearly.
It should help us to better appreciate how we abuse data in other areas of our public discussion.
Like education.
Data Abuse
As such studies go, the CNBC business rankings are fairly straight-forward. Their complexity pales in comparison to the vast data sets that are picked over when discussing education.
When it comes to cherry-picking “facts,” the mind-numbingly complex data sets that capture student achievement — statewide Standards of Learning tests; the National Assessment of Educational Progress (NAEP), oftentimes called the Nation’s Report Card; and the Programme for International Student Assessment (PISA) — provide plenty of fodder.
And there are groups aplenty cherry-picking data to support their approach to public education.
Indeed, there’s an entire commercial industry worth billions of dollars that exists solely to exploit this data.
The National Education Association approaches educational data in the same way Youngkin approaches this year’s CNBC ranking. Rather than accept data that indicates schools’ shortcomings, they attack the data itself, calling it “inequitable,” “ineffective,” and not “fair.”
At the American Enterprise Institute, researchers latch onto educational data that demonstrates shortcomings, while ignoring data that shows how innovative and successful many public schools are, and how happy most people are with their local schools.
This is not to suggest that researchers at the NEA or AEI don’t do outstanding work. They surely do. Frederick Hess at AEI, for example, may well be one of the most-impressive education researchers in the country.
It is to suggest that NEA and AEI have a financial stake in perpetuating their narratives.
In fact, these organizations are just two in Washington that collectively rake in billions of dollars to parse data and package it for consumption by those who carry the political messages of success or failure across the country.
By the time these analyses reach the local level, the data used has frequently become so twisted that it’s lost any real meaning or validity. It’s simply a cudgel to beat the opposition with.
Here’s just one local example. Nationally, there are longstanding efforts to use the Nation’s Report Card as a check on state tests and standards. This research is useful, but complex, nuanced, and far from definitive.
But look how both Youngkin and former Spotsylvania Schools Superintendent Mark Taylor manipulated NAEP and state testing data to argue — incorrectly — that our schools were among the worst in the nation.
Their error? From the Washington Post analysis we quoted at the time:
“What Youngkin’s team got wrong is that the two tests have different meanings of what it means to be ‘proficient’ in a subject, so comparisons aren’t valid. NAEP has long been the subject of criticism over its achievement levels — and for too many years those levels have been equated to grade-level skills. They aren’t.”
This is no minor oversight. That they made it suggests they: 1. Don’t understand the data they’re working with; 2. Intentionally are misrepresenting the data to score political points; or 3. Are cherry-picking data at a superficial level to reinforce their opinions.
All three are concerning.
Gaining Perspective
Let’s return to the CNBC ranking for a moment. A drop from one to four is hardly catastrophic. It would be irresponsible to conclude, for example, that this decline represents a collapse of the Virginia economy. But that’s precisely what Democrats and Republicans are doing right now.
Youngkin complains on X, and not without cause, that the CNBC ranking ignores all the positive economic news in the state.
Earle-Sears, who is running for governor as a Republican, doesn’t criticize the study or point to the state’s strengths, but rather suggests the state is marching toward economic trouble because of “liberals.”
Referencing the CNBC rankings, her press secretary, Peyton Vogel, said: “This frighteningly is a harbinger of things to come if Democrats continue to stand in the way of smart economic policies that foster growth and opportunity. Our competitive advantage nationally has already been weakened with liberal control in the General Assembly. It’ll be hard-working Virginians who pay the price if Abigail Spanberger is governor.”
Abigail Spanberger, who is running for governor as a Democrat, responds by embracing the results and suggesting they represent a significant weakening of the state’s economy: “Today’s rankings make clear that Virginia needs a Governor who will build a more resilient economy, leverage our Commonwealth’s strengths to attract new investment, prepare our workforce for 21st-century jobs, and stand up for our workers and small businesses in the face of ongoing threats.”
She then shifts gears and sounds like Youngkin: “As the next Governor of Virginia, I will make it a top priority of my administration to make sure Virginia is the best place in the country to start or grow a business, and I will put a plan in place on day one that is clear-eyed about the challenges and opportunities ahead.”
See the problem?
Tests, Rankings Have Their Place
This is not to suggest that testing doesn’t have a place in education, and that rankings aren’t useful for understanding how one state is performing relative to another, or how one school district is performing relative to another.
It is to say that our over-the-top reactions and weaponization of data to score political points and suggest that problems are easier to solve than they are, is a major problem.
Data is a powerful tool that reveals levels of complexity that prior to computer technology took decades to uncover, or were missed altogether. Artificial Intelligence has the potential to greatly increase our ability to use that data.
Today’s outsized reaction across the aisle to CNBC’s ranking shows on a scale that is easy to grasp how unhelpful the careless use of data in public discourse can be, and too often is.
There is a far healthier approach to using this data.
One example of this is evidenced by Matt Hurt of the Comprehensive Instructional Program. His research has led to significant positive improvement for many school districts across the state.
Why? Hurt goes where the data takes him, no matter how uncomfortable it is. And he balances that data with the softer art of comparing it with “intangibles” — those things that are harder to quantify, but nonetheless play a powerful role in how effective organizations are.
This type of responsible handling of data — not blaming it when it doesn’t show what you want, and not weaponizing it to advance a cause — is what we need.
The best way to get there? Stop making data the endgame. When it is, then the data is invariably misunderstood, misused, and manipulated.
If Youngkin and others could grasp that and bring it to bear on education, we could dump high-stakes testing and divert the millions wasted on its development and implementation, and instead invest in quality education that puts the focus on mastering skills and understanding the human condition.
Do that — use data to help us understand where things are working and where we need to improve — and the rankings and test scores will take care of themselves.
Support the Advance with an Annual Subscription or Make a One-time Donation
The Advance has developed a reputation for fearless journalism. Our team delivers well-researched local stories, detailed analysis of the events that are shaping our region, and a forum for robust, informed discussion about current issues.
We need your help to do this work, and there are two ways you can support this work.
Sign up for annual, renewable subscription.
Make a one-time donation of any amount.
Local Obituaries
To view local obituaries or to send a note to family and loved ones, please visit the link that follows.
This article is published under Creative Commons license CC BY-NC-ND. It can be distributed for noncommercial purposes and must include the following: “Published with permission by FXBG Advance.”