Garbage in, garbage out at the college and university level
Written by Michael Lewis on June 18, 2014
It was exciting to read in Miami Today that a county study found annual economic impact of $7.3 billion for Miami-Dade’s colleges and universities. What a sales tool for our higher education institutions!
Then I read the study itself. All I could think of is GIGO – garbage in, garbage out.
No, not the institutions or the students, but the results. If government is using that report to make policy, it’s better to go without any figures at all.
It would be logical when dealing with higher education to refer to a college text, but it’s pretty safe to pull the definition of “garbage in, garbage out” from lowbrow Wikipedia:
“Garbage in, garbage out in the field of computer science or information and communications technology refers to the fact that computers, since they operate by logical processes, will unquestioningly process unintended, even nonsensical, input data (‘garbage in’) and produce undesired, often nonsensical, output (‘garbage out’).”
That, unfortunately, is what appears to have happened to this study, which at three key points misses its aim.
Let’s take a step back: we questioned the purpose and speed of the study last June when Commissioner Lynda Bell introduced a measure that later passed calling on county staff to total within 90 days the direct and indirect economic impact of institutions of higher learning in Miami-Dade. The only reason she gave to undertake the study was that no measurement existed.
Subsequently unveiled in February, the study tallied impact of the nine largest institutions at $7.3 billion, with county researchers saying they had taken numbers only from materials available to the public – they did no original research.
The total they found was barely half of what five of those institutions had individually reported earlier. Last June, Miami Today tallied the five reports at $14.275 billion. The University of Miami alone had reported $6.1 billion.
But those five reports all differed in method and date, making that figure unscientific.
A county study, for whatever purpose Ms. Bell had in mind, could be expected to produce different results. That’s no surprise.
It’s also no surprise that in 90 days the county did no original research. We were sure it couldn’t be done that fast.
What concerned us was three vital elements of the study:
1. Anything that any student spent other than for tuition was counted as a contribution of our local colleges and universities to the local economy.
That would be true for anyone who came to Miami just to study. But it’s illogical to say that our universities added to the economy all funds that local students were spending here the day before they enrolled and would still be spending the day after they left school. Why count their rent, their car, their clothing or their food as additions caused by our colleges? The same people would have spent the same – maybe more – if they had jobs instead of classes.
The study included $820 million a year for that student spending – more than 30% of higher education’s total direct economic impact, according to the study.
2. The study didn’t include any construction spending by any college or university. Building a $40 million campus addition would count for zero economic impact.
The researchers explained that capital spending varies widely from year to year and all schools don’t report it, so it was “not included in this analysis for lack of common information across the nine institutions.” Because they didn’t have some capital spending figures they didn’t count any of them. Excluding capital spending leaves a huge gap in any economic impact study.
3. The study totaled the area’s college and university graduates and computed how much each added degree would increase labor productivity, which seems logical. But it listed in a study headed “The Contribution of Local Colleges and Universities to Miami-Dade Economy” every degree from schools outside of the area, the state or even the nation.
The study then says “The economic contribution of post-secondary degrees in the workforce includes $80.7 billion in total sales revenues of local business establishments, and nearly 450,700 employment positions in the county.” Huge numbers, but the study didn’t find what percentage of that is attributable to local colleges and universities. Those figures count degrees from everywhere.
When we asked Ms. Bell about the study, though not its methods, she applauded the findings and said the study shows the need to keep talented young graduates here.
Her conclusion is absolutely on target. We vitally need to keep talented young graduates here.
But we didn’t need the study to tell us that – and indeed it does not tell us that. It’s hard to say what it does tell us, except that you can look at anything and produce economic impact figures. They might not reflect reality, but they’re a good sales tool – unless you analyze the work.
Especially when studying higher education, we’d like something more credible.