5/02/2014

Discover Quality Change of U.S Postsecondary Institutions from 1990 to 2010 (conclusion and further consideration)



      In this paper I mainly used expenditure records of four types of postsecondary institutions to discover where schools have been investing in. I used fixed effect (FE) model regression to find out contributing effects of investment on instruction and research and inferred quality change of higher education. The results are that private institutions and public research institutions all experienced some levels of quality improvement while community colleges are left behind.
      There are several advantages of this approach. Firstly, expenditure data are relatively complete and accessible to general public[1]. In addition, using expenditure per FTE student as a measure of quality avoids particular assumptions about the best, or preferred, way for schools to allocate their total resources among specific inputs. (Ladd and Loeb, 2012) For example, an institution can use a certain amount of instruction per FTE student either for smaller classes with less experienced instructors or larger classes with more experienced instructors. With no evidence showing that certain categories of resources are preferred to others in all postsecondary institutions, it would be inappropriate to conclude that one way of spending is absolutely better than the other. Finally, this measure allows for straightforward comparisons across schools, and by using regression tests, researchers can also study how one category of spending affects total expenditure.
      However, there are three main limitations with this method. Firstly, different schools report data under various accounting standards, which creates debate upon what data reports to use. Besides, we can only draw an inference of how quality changes by discovering where money has gone and whether institutions are putting more money in core missions like instruction. Lastly, this method takes no account of differences in effectiveness or efficiency with which dollars are spent. The key assumption of input measure is that high quality inputs tend to produce better education provided that market is competitive. We don’t know if students will fully utilize inputs deemed to be of high quality, such as whether students will have more contact with professors or do more critical thinking. Even though higher education market is generally competitive (Hoxby 1997), there would still be concern about inefficiency given the unique governance of institutions.[2]
      Since data on mean SAT scores are incomplete, I couldn't create a quality index to give numerical assessment of quality, which is another popular input measure methodology. Two papers can exemplify and reveal questions about this approach. Black and Smith (2006) used index as proxy of college quality to discover its impact on graduates’ income. They first summarized previous related researches and then compared four measures with multiple proxies. They found that Generalized Method of Moments (GMM) estimation is the most efficient econometrics model and proxies that are powerful are mean SAT scores, mean faculty salaries, student-faculty ratio, application rejection rate and first year retention rate. Cohodes and Goodman (2013) criticized that Black and Smith’s choice of proxies is partly flawed because of certain level of perfect correlation. Instead, they introduced another proxy, namely college completion rate, into their index. One obvious problem with this approach is that quality index is created rather arbitrarily: there is no consensus about what proxies should be chosen and what weights should be assigned. What’s more, most researchers use index to function as a regressor for graduates’ income. In other words, quality of education is not their major concern.
      Recently there are some new approaches trying to do a better job. For example, Porter (2012) argued that detailed student self-report can help discover what they have obtained in college. This attempt is a value-added measure and is theoretically desirable.  However, the validity of this approach depends on whether reports are well-designed. 

      Another attempt is more promising. With the advent of online course platforms like Massive Open Online Course (MOOC) system, people are able to learn college materials without actually sitting in a classroom. I think it possible to design a value-added like measure. We can do comparison tests between two groups of students with similar abilities and motivations and have them take several courses. One group learns materials in class, while the other takes online versions. The main control is that the former group is accessible to all those services related to instruction while the group that takes online versions is not. At the end of the semester both groups take the same tests and we compare their test results. If students who learn materials in class do perform better, then we might conclude that sitting in class and enjoying services like academic support can help improve students’ performance; then we can step further and test to what extent does these services help and find out possible inefficiency in terms of expenditure. This method is not impeccable, however, since we might also count peer effect as positive effect of instruction related spending.
      As Robert Pirsig (1974) wisely wrote, “Some things are better than others; that is, they have more quality. But when you try to say what the quality is, apart from the things that have it, it all goes poof. ” Quality measure in terms of higher education is hard because many aspects of college life cannot be quantified, and there are a lot of factors like student motivation that affects student performance. We might never know how college undergraduate education really bestows on us, and all we can do is come up with some ways to get a better approximation. 

references:

1.      Ladd, Helen and Susanna Loeb. “The Challenge of Measuring School Quality: Implications for Educational Equity,” Education, Justice and Democracy, The University of Chicago Press, pp. 22-55, forthcoming
2.      Black, Dan and Jeffrey Smith. “Estimating the Returns to College Quality with Multiple Proxies for Quality.” Journal of Labor Economics, Vol. 24, No. 3, 2006
3.      Cohodes, Sarah and Joshua Goodman. “Merit Aid, College Quality and College Completion: Massachusetts’ Adams Scholarship as an In-kind Subsidy.” American Economic Journal: Applied Economics, forthcoming, March 2013
4.      Pirsig, Robert. Zen and the Art of Motorcycle Maintenance: An Inquiry into Values. Harper Torch; Reprint edition (April 25, 2006)  ISBN-10: 0060589469
 




[1] Still, some specific information like investment in each department over years is unavailable.

[2] Ronald Ehrenberg elaborated possible reasons why American postsecondary institutions might suffer from inefficiency in Rising Tuition. ( 978-0674009882)

Discover Quality Change of U.S Postsecondary Institutions from 1990 to 2010 (testing result)



     In this section I discuss testing results of four different postsecondary institutions. In terms of the testing, dependent variable is total expenditure, while independent variables are function expenditure like instruction and student services.
      For private research institutions, table 17 tabulates test results for private research institutions. Column 1 shows that holding other variables unchanged, increasing instruction expenditure by one dollar tends to increase expenditure by 1.37 dollar. As for research expenditure, the influence is 0.97 dollar. The coefficient of instruction indicates that instruction spending has an economically significant impact upon total expenditure. If spending on instruction is efficient, then it means these institutions have experienced a certain amount of quality increase. One thing interesting in this testing is that coefficient of grant expenditure is negative, indicating that spending more on scholarships and fellowships alone actually decrease total expenditure. The sign of the coefficient depends on how institutions do their budgeting. If institutions do zero-based budgeting, namely that they come up with a number to spend on each category and implement it, then spending more on each subcategory is bound to increase total expenditure. However, if institutions first set an overall expenditure and then allocate to each category based on revenue sources and donors’ specific requirements, it is reasonable to see negative coefficient.
      Considering that many private research institutions have affiliated hospitals, I further decomposed samples into two subgroups and tested difference with respect to instruction and research expenditure. Column 2 and 3 tabulate the testing results. The results show that institutions without hospitals have more quality increase in terms of instruction while institutions with affiliated hospitals have more quality improvement in research.
      As for public research institutions (Table 19), since coefficients of instruction and research are both bigger than 1, it implies that these institutions experienced quality increase from both instruction investment and research effort. Column 2 and 3 illustrate results for institutions with and without hospitals. We can see that hospital-affiliated institutions, compared to peers without one, have more quality return from research and less from instruction. I think that existence of affiliated hospitals tends to let schools have different spending decisions, and the possible reason is that: schools with hospital have more research projects related to medicine, and thus they are more likely to get results and explore frontier of knowledge in terms of this field of researches. As a result, they attract more talented students and faculties interested in this field and their quality contribution in terms of research is more significant than peers without hospitals.


      As for private bachelor institutions, their coefficient of instruction is larger than 1, which implies that they also experienced certain level of quality increase. One thing to notice is that, even though their coefficient of research is larger than 1 (though not much), this doesn’t mean that they gain extra quality increase from this field because their investments in research are very small.
      For community colleges, holding other variables constant, increasing instruction expenditure by 1 dollar will cause total expenditure to rise by 0.97 dollar, and since these institutions are not research oriented, their research coefficient doesn’t tell much about their quality change. Direct expenditure on instruction doesn't lead to economically significant increase of total expenditure, and this implies that with respect to public associate institutions, they didn’t experience much quality improvement over years.