The "NH TCAP"

The NH TCAP, Teacher Candidate Assessment of Performance, is a capstone performance-based project of teacher candidates and is a primary initiative of Network institutions.

Background

In a November 21st, 2012 letter to the NH Board of Education, the NH IHE Network committed themselves to “Craft, calibrate, implement and analyze a common assessment of teacher efficacy that can be used by all EPPs, regardless of size or specialization.” This is one of four core initiatives developed by the IHE Network.

The result of this initiative is the New Hampshire Teacher Candidate Assessment of Performance (NHTCAP), a complex, subject-specific, portfolio-style, multiple measures performance assessment designed to assess and provide formative learning experiences for beginning teachers. The design and implementation of NHTCAP is grounded in the following principles and premises:

• An authentic assessment of a teacher’s performance should acknowledge and maintain the complexity of teaching.
• The assessment should examine student learning in relation to teaching practice.
• The assessment should incorporate a formative thread that provides analytic feedback and support for the candidate’s continuing professional development.
• The assessment must focus on content and pedagogy within disciplines embedded in the educator preparation program.
• The assessment should be adaptive, responsive to local contexts (integrating key institution-specific values and goals), and transferable to broader contexts.
• The assessment should be viewed as a necessary albeit insufficient step toward initial licensure (i.e., as one of multiple measures leading to licensure recommendation).

From this conceptual framework, the members of the NH IHE Network built upon established teacher candidate performance assessments, most notably the Performance Assessment for California Teachers (PACT), and tailored specific strands to align with the New Hampshire’s 610.02 Professional Standards and the InTASC Model Core Teaching Standards. The strands outlined in the NH TCAP include:

• Strand 1: Contextualizing Learners and Learning
• Strand 2: Planning and Preparing
• Strand 3: Instructing Students and Supporting Student Learning
• Strand 4: Assessing Student Learning
• Strand 5: Reflecting and Growing Professionally
• Strand 6: Using Academic Language 

In 2013 – 2014, five institutions in the NH IHE Network participated in a first round of NH TCAP piloting. The purpose of this small-scale pilot was to explore the feasibility of implementing the NH TCAP across various institutions in the state, to begin preliminary conversations around criteria for evaluating and scoring work samples, and to analyze preliminary teacher candidate data. The NH State Board of Education publicly endorsed these efforts at a state board meeting in December 2013. In the summer 2014, the NH IHE network analyzed the candidate work samples and revised the NH TCAP across all subject areas. 

During the 2014 - 2015 academic year, the NH IHE Network has initiated a second round of piloting the TCAP across 12 higher education institutions, with approximately 270 teacher candidates participating across grade levels and certification areas.

In 2015-2016, all but one IHE in the Network implemented the TCAP. 

Statement on the NH TCAP 

The IHE Network recognizes the importance of standard setting, training, and calibration as critical components of the implementation of the NH TCAP – New Hampshire Teacher Candidate Assessment of Performance.

The Spencer Research subcommittee of the IHE Network took the lead in holding sessions and facilitating the process for examining and scoring for TCAP samples. The Spencer Research group is an inter-institutional collaboration initiated by faculty at UNH who secured a $50,000 research grant. Led by Emilie Reagan and Tom Schram, this research also includes faculty from Plymouth State University, St. Anselm, Southern New Hampshire University, and Upper Valley Educators Institute. Across their institutions, this team surveyed student teachers who completed the NH TCAP in the 2014-15 pilot year and then secured a sub-group of recent graduates to follow into their first year of teaching in 2015-16 to more deeply analyze the connections between teacher preparation and actual implementation in the first year of teaching.

Since the 2014-15 time frame a series of sessions have been held within the Spencer group and with the larger IHE Network that engaged faculty in discussions around TCAP samples. An official protocol was utilized and shared among the IHE Network for use in their own departments. The protocol was adapted from “Describing Student Work Protocol,” received at the National Writing Project in New Hampshire (NWPNH) Summer Institute 2011. This process enables colleagues from the New Hampshire IHEs to learn about the TCAP instrument itself, how our students approach the writing prompts, and the ways in which IHEs are responding to their learning around teacher candidate samples. This work is ongoing into the 2016-2017 academic year.

To guide its work in scoring, the Spencer Research group draws heavily from the Performance Assessment of California Teachers (PACT) Technical Report that described their process of standard setting in the following way:

“Standard Setting. Based on standard setting models described by Haertel (2002) and Haertel & Lorie (2000), as well as the process used by the National Board for Professional Teaching Standards (Phillips, 1986), we utilized a three-stage process that first convened a panel of teacher educators familiar with the PACT scoring process to formulate initial recommendations for the passing standard on the Teaching Event. The second stage called for a confirmatory group from the PACT programs to review those initial recommendations and to decide on a set of passing standards (along with a cutscore model) that would be submitted to all participating programs for review and approval. Based on consensus agreement from all PACT consortium members, the standard setting process resulted in a passing standard that valued all five components (Planning, Instruction, Assessment, Reflection, Academic Language) equally. The final step involved the gathering of policymakers (deans of education and directors of teacher education) across the PACT Consortium to review and approve a final set of passing standards. This step was completed in January 2007. The Final Passing Standard that was ultimately selected by program directors and deans in January 2007 is below.

Candidates pass the Teaching Event if they pass ALL FIVE rubric categories (Planning, Instruction, Assessment, Reflection, and Academic Language) AND have no more than 3 scores of “1” across the tasks.”

Source: Pecheone, R. L. & Chung Wei, R. R. (2007). PACT Technical Report: Summary of Validity and Reliability Studies. Stanford University

Scoring guidance distributed to TCAP piloters in 2015-2016 indicated that each IHE determines how they will implement the TCAP and suggested to pass the NH TCAP, candidates should earn at least a TOTAL score of 24/48 (across 12 rubrics), with a maximum of one “1” per strand. Institutions can implement higher passing scores, can determine within institution double-scoring. For inter-rater reliability purposes across New Hampshire institutional double-scoring will eventually take place.

With the increased number of IHEs implementing the TCAP and the increased number of samples, calibration efforts to score TCAP submissions are underway. The process that the Spencer group has followed has been to distribute samples across pairs or groups who score separately and come together to compare scores and individual rationales.

It is important to note that each IHE Network determines their level of involvement in the larger calibration process and in defining the minimum performance required for a particular level of achievement on the NH TCAP.

Questions regarding discussion protocols, scoring, and calibration should be directed to Megan Birch (PSU) of the Spencer Research Team: Spencer Group Principal Investigator: Emilie Reagan; and Co Principal Investigator Tom Schram (UNH). Supporting Institutions include: Megan Birch, PSU; Kathryn McCurdy, UNH; Dianna Terrell, St. Anselm; Page Tompkins, UVEI; Audrey Rogers, SNHU; and Chris Ward, UVEI.

Given that that we are in the early stages of implementation, discussion, and calibration, scores should be interpreted in that context: IHE faculty, and their teacher candidates, are in a learning process and are working toward deeper understanding of TCAP samples and scoring both within and between preparation programs.

o Graduates of NH IHEs are encouraged to present multiple artifacts that provide evidence of their preparation when applying for jobs. We suggest our new graduates use a lot of examples from their student teaching and program to demonstrate their ability to take over their own classroom. This would likely include discussions and potentially examples of their learning from the NH TCAP.

However, we do not rank students based on their scores or release their scores. As stated above, the IHE Network is working toward calibration of scoring both within and between institutions. While this process is time consuming and slow, it will ultimately give us greater confidence in the reliability of scores in the future.

Completion of the TCAP is supported by New Hampshire IHE educator preparation programs. The IHE Network takes the position that the NH TCAP is one of a multitude of measures in a candidate’s preparation program. One of the distinguishing features of the NH TCAP is that scores remain at the Institutional level and while their performance contributes to being recommended for certification, IHEs do not send the scores to the state or otherwise make them public. Furthermore, federal law protects the privacy of student educational records for coursework they are fulfilling while enrolled in our IHEs. Our intent is that the NH TCAP is an assessment for learning as well as of learning.

While we are building toward calibration to ensure reliability of scores, we are not intending for this to set up a ranking system.

If you have any questions about TCAP, please consult your student teacher’s Preparation Program.

Statement updated December 1, 2016