A new evaluation approach for teacher preparation programs using labor market competitiveness of teacher applicants

From Section:
Programs & Practicum
Aug. 02, 2021
August 2021

Source: Teaching and Teacher Education, Volume 104

(Reviewed by the Portal Team)

This paper argues that the of teacher preparation programs (TPPs) that produce graduates who are more likely to be employed, compared to graduates of other programs vying for the same position, are more effective.
The authors’ main research question is:
What is the labor market competitiveness of the TPP, and how can it be used for better TPP evaluation systems?
To answer this question, they
(a) consider whether a uniform ranking of TPP is valid;
(b) assess whether specialization affects the labor market competitiveness of TPP graduates; and
(c) consider the impact of proximity and “localness” of the labor market.
To demonstrate the validity of this novel approach, this study uses statewide applicant data for three academic years and adopts vacancy fixed effects models with additional statistical controls.
As a primary analytic tool, the vacancy fixed effects model measures the probability of being hired among candidates who applied to the same teaching vacancy.
In addition, their inclusion of a distance variable between TPPs and the school posting the vacancy illustrates how TPPs are embedded in their local teacher labor markets.
Finally, this study investigates the characteristics of TPPs related to the effectiveness of policy discussions.


Data and sample
The authors’ primary source of data is statewide teacher vacancy-application information provided by the Wisconsin Education Career Access Network (WECAN) for the 2014-2015 through 2016-2017 school years.
WECAN is an online job portal system for both administrators and teachers in Wisconsin K-12 public schools that includes a wide array of information on teacher applicants (e.g., academic background, credentials, and the number of applications each applicant submitted) and vacancies (e.g., when and what types of positions are posted and the number of applicants who applied).
WECAN is the only statewide data source in the U.S. that captures detailed labor market activities from both supply and demand sides, allowing unique and valuable opportunities for researchers to pursue various issues in educator labor markets.
These unique data make it possible to examine the extent to which individual or organizational factors are associated with the probability of being hired by using applicant pools, which is distinct from previous studies that utilize teachers' final placement only.
To understand teacher candidates' job placement, state agencies and TPPs typically rely on descriptive surveys that only present hiring results.
Although this traditional approach allows states and TPPs to understand how many graduates are employed in a given year, it does not provide any information about the extent to which a TPP produces competitive candidate teachers.
WECAN enables the authors to measure competitiveness among TPPs by examining vacancies teacher candidates applied for and applicants’ hiring outcomes in the application pool.
In addition, the data include the number of job positions to which each applicant applied, which can be used to control higher chances caused by the greater number of applications submitted.
Despite such uniqueness, the vacancy-application data only capture those who applied for a teaching position in Wisconsin public schools and do not include those who applied for private or out-of-state schools.
Therefore, the estimates should be interpreted as specific labor market competitiveness rather than general TPP competitiveness.
The analytic sample captures 119,742 job applications submitted by 8,317 candidates who graduated from the Wisconsin TPP to 6,731 vacancies posted by 376 districts.
The authors limited the sample to novice candidate teachers who graduated from TPPs within three school years.
Using the licensure information provided by the Wisconsin Department of Public Instruction (DPI), they identified applicants who had received their first teaching license.
They also restricted the sample to vacancies that hired novice teachers.
The outcome variable in our model is whether the job candidates were hired by schools or school districts.
The predictor of interest is the TPP in which an applicant completed their training and obtained their teaching license.
There are 14 public, 21 private, and eight alternative-route TPPs in Wisconsin (Wisconsin Department of Public Instruction, 2017).
The complete sample included 12 public and 14 private TPPs, which accounted for 93.4% of all novice applicants.
Those who graduated from TPPs outside Wisconsin (e.g., Minnesota and Illinois) within the three school years were included in the sample and categorized into one group named “out-of-state” that was used for comparison.
They also created a series of control variables that capture the demographics of applicants in their sample, including gender, race/ ethnicity, years of professional experience, grade-point average (GPA), and the number of applications that an applicant sends out to vacancies in a given year.
To explore how geographic proximity affects local teacher labor markets, the authors utilized the distance measure between the TPP and the school districts to which applicants applied. Geographical proximity matters because prospective teachers prefer to attend TPPs and work at schools close to home; likewise, employers also prefer local applicants (Boyd et al., 2013; Engel et al., 2014; Goldhaber et al., 2019), which might significantly affect the hiring probability.
To examine the impact of TPPs on teacher hiring, the authors estimated linear probability models.

Results and discussion
In this study, the authors examine TPPs in a novel way by employing the concept of organizational learning in school-level hiring for novice teachers who have recently graduated from their preparation program, essentially making their education a major component of their marketability.
They posit that the hiring process is informed by organizational learning, which means that schools rely on previous experiences, sharing and dissemination of information, and organizational memory to select new teachers and sustain partnerships (Huber, 1991; Tippins & Sohi, 2003).
A school's prior experiences with graduates from a particular TPP play a role in the local teacher labor market for new teachers, and these continued partnerships are likely part of the organizational learning in which schools engage.
They used application and staffing data for novice teachers in Wisconsin to understand whether a specific TPP's alumni are more competitive than those graduating from another program and examine the role of program heterogeneity and the local teacher labor market.
This study has three main implications.
First, the findings suggest a new way of evaluating TPPs that supplement current absolute ranking systems.
At a time when the efficacy of TPPs is called into question with the rise of alternative pathways to teaching (Darling-Hammond et al., 2005), policymakers should be thoughtful about the evaluation and measurement of TPPs.
The current evaluation systems might mislead policy efforts, such as Title II of the Higher Education Act and overall CAEP quality assessments, to improve the overall quality of TPPs by inaccurately identifying good and bad programs.
The labor market competitiveness approach can be a useful measure for states, accreditors, and TPPs, suggesting the establishment of systems to track application data.
Second, the authors’ analyses demonstrate the value of taking TPP contexts into account in the evaluation system and suggest that it is worthwhile to generalize this approach to other TPP evaluation measures.
Participation in a particular TPP can make a difference when novice teachers apply for jobs, and the local market and program characteristics must be considered as factors in this process.
When graduates apply for jobs close to their TPP, their hiring probability typically increases relative to their chances of being hired by a school farther from their preparation program.
When comparing the overall ranking of TPPs across Wisconsin, the authors find that schools toward the bottom of the list are almost as competitive as the top schools when looking at their marginal probability of being hired by a school in close proximity to their TPP.
These findings call into question whether scholars, policymakers, or administrators should place too much emphasis on the overall absolute orderings of TPPs, and this result implies that a single typical criterion cannot meaningfully rank TPPs.
Along with the authors’ suggestions, evaluators of TPPs can improve their evaluation systems if they incorporate the relative strength of TPPs’ local labor market competitiveness into their current criteria.
Similarly, media such as U.S. News and World Report and institutions, such as the National Council of Teacher Quality, may add metrics related to labor market competitiveness to provide a better sense of localized TPP quality.
If there is a meaningful, differential signal obtained from this approach, ratings could be used to provide TPPs, prospective students, and government agencies important information to shape policy, programs, and practice (Feuer et al., 2013).
The authors investigated whether particular programs within TPPs affected the competitiveness of graduates by comparing TPP alumni hiring for elementary and non-elementary school vacancies.
They found no significant difference in TPPs between vacancy levels, but the available amount of data limited their ability to investigate differences among programs such as special education or specific disciplines (e.g., secondary mathematics), which may have produced more significant findings.
The similarity in TPP graduate competitiveness between elementary applicants and all other applicants may indicate that the central practices used and values imparted by TPPs are universal across all their programs.
When a common approach for teaching literacy, for example, is used across all programs and deployed by a common group of faculty, such a possibility becomes more plausible.
In larger institutions, programs may be more balkanized and isolated from each other, making such commonalities less likely.
In addition to the actual similarities among programs within a TPP, the authors also speculate that organizational learning may be blunt and shallow, which means schools and districts are not really learning about specific programs at TPPs and instead have a more holistic impression of the TPP.
If so, the findings may imply potential limitations of organizational learning theory, questioning the extent to which organizational knowledge is more useful for practitioners than “metaphorical implications” (Prange, 1999) and concerns about a tendency of knowledge oversimplification in organizations (Rashman et al., 2009).
However, all the inquiries are beyond the scope of this paper and remain as questions for future research when more data are available to investigate programmatic differences.
Finally, the findings of the impact of distance on TPP graduate competitiveness further suggest the benefits of partnerships between TPPs and K-12 schools, given that close relationships likely promote organizational learning.
To confirm the findings, they reviewed websites for the TPPs at the top and at the bottom of the list according to the predictive margins of hiring when the distance from the TPP to the school was included; the primary similarity across the top five TPPs (1, 5, 12, 2, and 8, respectively) was that each touted their extensive network of community and school partners and the 100-plus hours that preservice teachers spent in educational settings before commencing student teaching.
The authors suggest that schools’ organizational learning and memory are shaped by these extensive partnerships, making surrounding schools more likely to hire new teachers from these institutions.
The TPPs toward the bottom of the list (22, 25, and 27) also illustrate this inference.
Of the TPPs whose alumni were found to be the least competitive, the authors’ website search revealed that each had a prominent online program component.
They conjecture that if students are enrolling in an online TPP, they are unlikely to live near the university and thus may not be able to take advantage of local partnerships.
With a large number of online students, the TPP likely does not place as high a value on developing networks with school partners and/or may lack opportunities to cultivate substantive relationships.
Second, the schools at the bottom of our list were often located close to the state border.
The websites stated that the programs prepared students to work in the neighboring state and highlighted their proximity to larger cities outside of Wisconsin.
The study data only capture Wisconsin public school vacancies, and it should be noted that the graduates of these schools may be searching for jobs in other states.
The authors conclude that TPPs with strong school and community partnerships are more competitive in the local market because school leaders are more willing to hire the candidates they know, as a result of organizational learning.
Additional qualitative research is needed to further investigate the role of partnerships and organizational learning in TPP competitiveness.

Boyd, D., Lankford, H., Loeb, S., & Wyckoff, J. (2013). Analyzing the determinants of the matching of public school teachers to jobs: Disentangling the preferences of teachers and employers. Journal of Labor Economics, 31, 83-117.
Darling-Hammond, L., Holtzman, D. J., Gatlin, S. J., & Heilig, J. V. (2005). Does teacher preparation matter? Evidence about teacher certification, teach for America, and teacher effectiveness. Education Policy Analysis Archives, 13
Engel, M., Jacob, B., & Curran, F. C. (2014). New evidence on teacher labor supply. American Educational Research Journal, 51, 36-72.
Feuer, M. J., Floden, R. E., Chudowsky, N., & Ahn, J. (2013). Evaluation of teacher preparation programs: Purposes, methods, and policy options. National Academy of Education.
Goldhaber, D., Krieg, J., Naito, N., & Theobald, R. (2019). Student teaching and the geography of teacher shortages (CEDR Working Paper No. 10172019-1-1). University of Washington.
Huber, G. P. (1991). Organizational learning: The contributing processes and the literatures. Organization Science, 2(1), 88-115
Prange, C. (1999). Organizational learning - desperately seeking theory?. In M. Easterby-Smith, L. Araujo, & J. Burgoyne (Eds.), Organizational learning and the learning organization: Developments in theory and practice (pp. 23e43) Sage Publications, Inc.
Rashman, L., Withers, E., & Hartley, J. (2009). Organizational learning and knowledge in public service organizations: A systematic review of the literature. International Journal of Management Reviews, 11(4)
Tippins, M. J., & Sohi, R. S. (2003). IT competency and firm performance: Is organizational learning a missing link? Strategic Management Journal, 24(8) 

Updated: Jan. 20, 2022
Evaluation | Teacher education programs | Teachers’ work