edTPA studies

I will keep updating this post as it seems that more and more edTPA news comes in – on 2/26/2020 – The Pearson-scored test is meant to assess whether prospective educators have the skills to teach, but critics question the time and cost involved, its reliability and Pearson’s “corporate profits.”

The new study by Drew Gitomer from Rutgers (a very solid researcher) and others entitles “Assessing the Assessment: Evidence of the Reliability and Validity in the edTPA” is pretty clear.  Their conclusion:

“We argue that, in light of the evidence available, the current proposed and actual uses of edTPA in evaluating PSTs and programs are not sufficiently supported on technical and empircal groups. We recommend that serious consideration be given to a moratorium on using edTPA scores for consequential decisions at the individual level, pending provision of appropriate evidence of the reliability, precision and validity of the scores produced by the assessments and given the stakes involved, an independent technical review of this evidence by an expert panel.”

A podcast that covers the recent AERA study on the edTPA problems actually got me to listen to the whole thing.  This is pretty amazing stuff and you should listen to the researchers talk about the MAJOR issues with the edTPA.

• Candidates who meet or exceed the cut score can go on to teach – those who don’t are not permitted to seek their license and either have to retake the exam or give up on teaching.
• What we do know is that edTPA does not offer appropriate estimates of reliability precision or decision consistency and overall the information that is offered is highly misleading.
• The area is heavily understudied.
• Reliability of assessments like this is usually much lower – approaches and formulas they (edTPA) use are erroneous.   .6 to .8 reliabilities when you use normal formulas – edTPA is using their own formulas in order to show higher reliabilities.
• “It is critical to examine the implications of an assessment on historically marginalized groups of examinees particularly those groups that are underrepresented in the teaching force. Now in our paper we point out that African American candidates are much less likely to pass the assessment and more likely to be misclassified in terms of passing status. And so that creates disparate access to the teaching profession. It’s a critical issue and those results are discounted by edTPA in their reports.”
• “We discovered another issue after we published that is quite troubling – decision makers rely on the technical abilities of the assessment – and edTPA in its technical reports claims that all analysis have been reviewed by national recognized psychometricians and they meet the technical standards for licensure assessments set forth by AERA and NCME standards – but based on confidential statements they have received from multiple sources they believe this statement is highly misleading and that technical reviews by experts has not occurred as described.”

costs $300 per test – if you fail you are now up to $600 in testing. The number 1 reason people do not enter teaching is because of the cost and this is a HUGE barrier.

edTPA scorers only receive $75 per portfolio – they are supposed to grade it “holistically” – ie not spend that much time on something it took an entire year to create and is the sole determinant if a person gets to stay teacher or not.

California –

• Required test a roadblock for teachers – https://edsource.org/2019/required-tests-roadblock-for-many-california-teacher-candidates/607239
• This article calls attention to the overreliance on research about the Performance Assessment for California Teachers (PACT)—often labeled edTPA’s predecessor—as justification for the edTPA. The article argues that the distinctions between the assessments are too vast to rely on PACT data to support the edTPA, given the localized nature of PACT and the way in which it is scored. https://www.tandfonline.com/doi/abs/10.1080/00131725.2016.1242680?src=recsys&journalCode=utef20

Connecticut –

• New bill just launched in CT legislature – HB05376 to end the use of edTPA in Connecticut.
• Based on final report – Working Group to Study Issues Relating to the
  Implementation of the Pre-Service Assessment, edTPA, as Adopted by the Connecticut State Board of Education – does not quite recommend repealing but admits the financial burden is HUGE and EPPs really struggle to implement.
• https://thedoddcenter.uconn.edu/wp-content/uploads/sites/319/2019/03/Confronting-the-edTPA-in-Connecticut.pdf
  • Based on a program evaluation conducted with students, faculty, and cooperating teachers in the University of Connecticut’s teacher education program, the paper documents the experiences of participants and impacts of the implementation of edTPA. It finds: • Diminishment of Candidate Learning, through o the substantial time required to complete assessment tasks; o narrowed range of teacher practices considered by edTPA; o tensions created with various stakeholders; and o the emphasis on decontextualized, regimented standards. • Perpetuation of Inequitable Systems, through o bias against teacher candidates of color in the standardized assessment format; o the financial burden it places on low-income teacher candidates; and o the privacy concerns raised by the appropriation of video-recorded lessons by a for-profit company. • Application of Developmentally Inappropriate Standards, through o the evaluation of candidates’ “readiness to teach” without accounting for their development across the years of the program; o the requirement to demonstrate “readiness to teach” in ways that lead to pedagogically inappropriate abstraction and isolation of certain teaching practices; and o the necessity of candidates in some academic areas to apply conventions that are inappropriate within their fields. Given these concerns and impacts, this report recommends policymakers suspend the current planned implementation of the edTPA in Connecticut and work collaboratively with stakeholders in the state to develop a teacher candidate assessment that: • can flexibly evaluate candidate in the range of teacher education programs in Connecticut; • is developmentally appropriate for teacher candidates; and • supports the state’s goal of recruiting more minority teachers.
• Public Act 19-139(“An Act Concerning Education Issues”) repeals an expedited teacher tenure provision for teachers or administrators who were previously tenured in one district and subsequently transfer into a priority school district. The bill further establishes a working group to study issues related to implementing the pre-service teacher performance assessment known as “edTPA,” which was adopted by the State Board of Education. Finally, the bill will continue to permit “non-Sheff“ magnet schools that are not in compliance with the state’s minority student enrollment requirements to continue to be eligible for magnet school operating grants for the 2019-2020 and 2020-2021 fiscal years if such a school submits a compliance plan to the Commissioner of Education (and the Commissioner approves it).

Georgia –

• https://scholarworks.gsu.edu/cgi/viewcontent.cgi?article=1066&context=mse_diss

Illinois  – held hearings on edTPA this spring – concerned that cost and other issues with the edTPA is making their teacher shortage worse. https://www.rrstar.com/news/20190320/teacher-licensing-tests-come-under-fire-in-illinois

Oregon – https://pilotscholars.up.edu/cgi/viewcontent.cgi?article=1074&context=etd

National –   Stop using edTPA for certification due to the many issues with reliability  http://blogs.edweek.org/teachers/teaching_now/2019/12/edtpa_scores_teacher_certification_decisions.html

Lesson Six: The edTPA Privileges Candidates from Certain Linguistic and Cultural Backgrounds

A teaching candidate may have carefully planned and successfully taught an effective “learning segment” but unless the candidate has also learned the language of the edTPA exam, followed particular directives, understood rubric objectives, and crafted their commentary by thoroughly reflecting the terminology designed and used by SCALE, she or he risks a failing score. This goal of learning the language of a teacher “bar exam” specifically disadvantages teacher candidates of color, as noted by Christine D. Clayton, Department chair of the Pace University School of Education:

Recent data from other states (where the assessment is not a certification exam) indicate that some groups, including teaching candidates of color and those from linguistic minority groups, were failing edTPA at disproportionate rates. Other reports showed evidence of candidates who performed well on edTPA, but did not earn fair supervision reports when actually teaching students.26

Wayne Au, associate professor in the education program at the University of Washington, expresses a legitimate concern, “Given the severe lack of teachers of color and teachers from working-class backgrounds, I wonder if the edTPA will systematically reproduce race and class inequalities, like every other high-stakes standardized test.”²⁷

https://blogs.edweek.org/teachers/teaching_now/2018/09/edtpa_study.html

https://journals.sagepub.com/doi/abs/10.1177/0042085918795020 – Carrie if your person could access –

NOTE  – lack of cultural competency in the edTPA is a big criticism as well – another issue that our urban leaders should be concerned with.

Assessment Is Repetitive and Misaligned, Students Say

But many students surveyed said they didn’t think the assessment would help them become better teachers. In response to the question about how edTPA helped them grow, 40 percent of the teacher-candidates either explicitly said that they couldn’t identify any value in the process or left the question blank. (Findings on the test’s reliability at predicting good teaching are mixed. A 2016 study from the American Institutes of Research found that the students of teachers who passed edTPA on their first try scored higher in reading than students whose teachers didn’t. But passing the test didn’t have any effect on students’ math scores.)

“Trying to satisfy all of the instruction rubrics’ requirements forced me to (unsuccessfully) coerce steps in the natural learning process,” wrote one of the participants. The assessment requires teachers to record themselves conducting one of the lessons that they submit as part of their portfolio. To demonstrate all of the skills edTPA requires demonstrated in the video, this participant wrote that they had to artificially speed up the pace of the lesson, “prematurely assigning group work” and “using ineffective written assessments.”

About one-fifth of students also said that the test was repetitive. In open-ended responses, students wrote that they felt they were asked to justify and analyze the same parts of their lesson over and over, without moving onto broader instructional applications, said Dover.

And while about 90 percent of candidates said using culturally responsive pedagogy was an important part of their teaching practice, only 47 percent said that edTPA prepared them to use it. “My program focused on cultural/social justice issues, but that did not help much when it came down to writing commentaries for the [edTPA],” one participant wrote, referring to the section of the assessment that requires written self-analysis of the teacher’s lesson planning

NPR – http://www.nprillinois.org/post/teacher-testing-1-2-3#stream/0

So if you want a critical teacher shortage and an all white teacher workforce – go edTPA! (just kidding JP – I know it is not causal)

Critics point to these requirements as one big reason that 85 percent of superintendents say they have a teacher shortage. And that shortage is compounded by a lack of diversity. More than 80 percent of Illinois teachers white, while fewer than half of students are white

I did purchase the report and it is very thorough.  They bring into question the way SCALE is analyzing the data and that until they better analyzing the subgroups, there should be a moratorium on using edTPA as a test to determine if someone should become a teacher.  Essentially the data is not good enough to make this call.

They have a great section on resourced and under-resourced institutions that edTPA will hurt those under-resourced. With the attention of the rich buying slots in universities – this is a big part of what the NEA has issue with as well -more wealthy teachers pay for better quality videos to send in to ensure they pass.

Some specific examples from institutions in North Carolina help illustrate this point. Large, comparatively resourced institutions such as UNC-Charlotte, UNCWilmington, and North Carolina State University each contain web-housed edTPresources for students and faculty. NC State has half-credit courses specifically targeted toward edTPA, such as how to write the extensive commentaries necessary for passing. UNC-Charlotte has an edTPA lesson plan template to assists interns aligning their planning to edTPA along with extensive literature for students, faculty, and staff. The state’s HBCUs, much smaller institutions by comparison, have fewer of these resources available to candidates. In our brief search, we found some HBCUs such as North Carolina A&T provide links to edTPA handbooks created by SCALE but few resources and supports beyond those. Winston-Salem State University is an important exception to this trend, which we discuss in the recommendations section below. Overall, the resources and labor necessary for edTPA implementation and candidate support disadvantage small HBCUs that prepare a disproportional percentage of Black teachers.

Overall, the small subsamples of Black candidates and candidates of color more broadly limit the administrative reports from providing meaningful information about the exam’s potential adverse impact on candidates of color. Thus, while confirmatory analyses reported in the technical manuals demonstrate reasonable validity and reliability evidence for the constructs, rubrics, and scores, the evidence is based on the large group only. To appropriately make group comparisons on derived scores, it is necessary to establish measurement equivalence.

Otherwise, one cannot make claims that inferences are valid (Raju et al. 2002; Stark et al. 2006). To date, there is no evidence that the constructs examined demonstrate measurement equivalency for different racial groups. This absence of evidence calls into question high-stakes and consequential decisions made about candidates. While regular reports such as these initiate an important conversation about bias and adverse impact, the reports—from the body that owns edTPA— leave much to be desired

Winter is Coming edTPA on African American Teachers – https://link.springer.com/article/10.1007/s11256-018-0453-1

Calder Center Study – https://caldercenter.org/publications/evaluating-prospective-teachers-testing-predictive-validity-edtpa-update

edTPA Disproportionately Impacts Minority Teachers.

•  In Washington, where the edTPA is required, Hispanics are up to three times more likely to fail edTPA. [1]
• “This difference in passing rates strongly implies that the high-stakes use of the edTPA in Washington may have an adverse impact on the diversity of the state’s teacher candidate pool.”
• In New York, black test takers are nearly twice as likely to fail the edTPA as white or Hispanic candidates, according to the state. [2]
• [An educator] said from her past experience at Stanford with one of the forerunners of edTPA, scorers looked for certain traits or behaviors “that ran counter to what good teachers of African-American students typically did.”[3]
• Another way certification exams may deter potential teachers of color — and others from low-income backgrounds — is through their cost.
• With some exceptions, Historically Black Colleges and Universities and Minority Serving Institutions are more likely to be under-resourced for the heavy implementation and support that research indicates is necessary for edTPA.[4]

edTPA also Disadvantages Teachers in Minority or Low-Income Communities.

• In one case study, candidates indicated they faced difficult choices between addressing student differences and presenting themselves positively on the edTPA. [5]
  • For example, one candidate felt the need to bypass family and community issues on her edTPA, and others who struggled to connect contextual information within a limited snapshot of teaching and learning, “may have actually reinforced existing inequities rather than acting to challenge them.“

 edTPA Takes Time Away from the Classroom.

• “edTPA remains a taxing process for preservice teachers, potentially impacting their ability to focus on and learn from their coursework and clinical experiences.”[6]
• Most respondents found the edTPA to be an unfair (76%) and excessively time consuming (96%) assessment of their teaching practices.

edTPA is not transparent and is not a good tool for high-stakes decision making.

• edTPA’s brevity of content, variability in school climate, and video recording limitations render the test problematic for high-stakes decision-making.[7]
• Research in Washington found that edTPA is not significantly predictive of teacher effectiveness in mathematics. [8]
• edTPA over relies on research on California teacher assessments to justify itself.[9]
  • Given the localized nature of the California assessment, the distinctions disbetween assessments are too vast to rely on its data.
• “To date, there is no evidence that the constructs examined demonstrate measurement equivalency for different racial groups. This absence of evidence calls into question high-stakes and consequential decisions made about candidates. While regular reports such as these initiate an important conversation about bias and adverse impact, the reports—from the body that owns edTPA— leave much to be desired.”[10]
• When participating in Pearson workshops, trainings, or test scoring, faculty must sign non-disclosure agreements. Faculty are not allowed to share materials with colleagues or their students. [11]
  • Score reports do not include the qualifications of their scorer nor is data about edTPA current scorers readily available online.
• “…Much more transparency is needed on behalf of SCALE and AACTE with respect to highlighting these differences and limitations and offering additional research that focuses explicitly on the edTPA.”[12]

[1] Goldhaber, Cowan, Throbald: Evaluating Prospective Teachers: Testing the Predictive Validity of the edTPA.

[2] Barnum, Matt: Certification Rules and Tests are Keeping Would-be Teachers of Color out of America’s Classrooms. 

[3] Barmore, Peggy: Will Controversial New Tests for Teachers Make the Profession Even More Overwhelmingly White?

[4] Petchauer, Bowe, Wilson: Winter is Coming: Forecasting the Impact of edTPA on Black Teachers and Teachers of Color.

[5] Paugh, Wendell, Power, Gilbert: ‘It’s not that easy to solve’: edTPA and preservice teacher learning. 

[6] Meuwissen, Choppin, Cloonan, Shang-Butler: Teaching Candidates’ Experiences with the edTPA as an Initial Certification Test in New York and Washington States: Survey Results from the Second Year of a Two-Year Study.

[7] Meuwissen, Choppin, Cloonan, Shang-Butler: Teaching Candidates’ Experiences with the edTPA as an Initial Certification Test in New York and Washington States: Survey Results from the Second Year of a Two-Year Study.

[8] Goldhaber, Cowan, Throbald: Evaluating Prospective Teachers: Testing the Predictive Validity of the edTPA.

[9] Hebert, Cristyne: What Do We Really Know About the edTPA? Research, PACT, and Packaging a Local Teacher Performance Assessment for National Use.

[10] Petchauer, Bowe, Wilson: Winter is Coming: Forecasting the Impact of edTPA on Black Teachers and Teachers of Color.

[11] Greenblatt, O’Hara: Buyer Beware: Lessons Learned from edTPA Implementation in New York State.

[12] Hebert, Cristyne: What Do We Really Know About the edTPA? Research, PACT, and Packaging a Local Teacher Performance Assessment for National Use.

15 Year Anniversary, 15 Years, alabama teacher certification, alabama teacher shortage, alabama teacher shortage task force, alabama teacher shrotage task force, alternative teacher certification, Anniversary, Arizona Teachers, Become a Teacher, Become a Teachers, Celebrate, center for american progress, chalkbeat, continuing education teachers, continuing education units for teachers, covid, covid-19, dave saba, distance learning, diverse teachers, diversity in teaching, Economy, edt, edtpa, edtpa georgia, edtpa issues, edtpa pearson, edtpa problems, edtpa study, edtpa teacher certification, edtpa texas, education, education school enrollment, educator awards, educator exams, educator preparation, engineers week, engineers week 2020, ferpa, florida teacher pay, Florida Teachers, Growth, H-E-B excellence in education, heb awards, heb excellence in education, helpful tips, how long does it take to be a teacher, how to become a high school teacher, how to become a substitute teacher, how to become a teacher, how to become an elementary school teacher, Illinois teacher certification, Illinois teacher shortage, Indiana teachers, Indiana teachers of tomorrow, Job Security, Living The Vision Award, marketing webinar, math teacher shortage, Michigan Teachers, Michigan Teachers of Tomorrow, micro-credentials, Nevada Teachers, new teachers, non-degree programs, north carolina teachers, north carolina teachers of tomorrow, online learning, online teacher recruiting, online teaching, pandemic, paraprofessional, paraprofessionals, PD, physics teacher shortage, portfolio, praxis, principal award, pro dev, professional development, Program, Reasons to Teach, Recession-proof, residency license, scholarship for teaching, Second Career, SK12, South Carolina Teachers, Stable Career, starting teacher salary, Strada, substitute, substitutes, Success, TASA, teach 313, teach in detroit, teach in michigan, teacher attrition, teacher certification exam, teacher certification exams, teacher certification test, teacher certification tests, teacher diversity, teacher micro-credential, teacher pay raise, teacher profession, teacher professional development, teacher recruiting, teacher resources, teacher salary, teacher scholarship, teacher tesst, teacher test, teacher tests, teacher tips, teachers of color, teaching interview, teaching interview questions, test, texas edtpa, The Learning Liaisons, Trent Beekman, update, virtual conferences, virtual job fair, virtual job interview, virtual teacher recruiting, virtual teaching, virtual teaching job interview, Why Teach, zoom, zoom virtual learning, zoombombing