It Depends: Understanding Psychometrics for Testing + Assessment, by Dr. Amanda Dainis + Tim Burnett22 Mar 2024

Ep3: Cut Scores Explained

Top Episode Takeaways: ✂️ Cut scores are important decision points in criterion-referenced tests that categorize test takers as passing or failing. They should be set thoughtfully using a defensible process, not arbitrarily (e.g. always using 70%). Setting the cut score too high or low can have significant consequences.

📏 The Angoff method is a common way to set cut scores. It involves having subject matter experts review test questions and estimate what percentage of minimally competent test takers would answer each question correctly. This provides a way to link the cut score to the test content and the minimum performance standard for the job or domain being tested.

⚖️ Cut scores may need to be revisited and adjusted over time if the test content changes substantially. Small tweaks may not require changing the cut score, but replacing many items with ones that perform differently in terms of difficulty may necessitate an adjustment to keep the cut score fair and aligned with the intended minimum competency standard. Ongoing evaluation is important.

Episode Keywords:Validity, Reliability, Construct, Cut score, Angoff method, Item analysis, Fairness, Rubric, Feedback, Blueprint.

Connect with us:⁠⁠⁠

Amanda Dainis, PhD, MPA⁠⁠⁠ https://www.linkedin.com/in/amanda-dainis-phd-mpa-012b94119/⁠⁠⁠

Tim Burnett⁠⁠⁠ https://www.linkedin.com/in/tburnett/

A proud supporter of the ⁠⁠⁠https://www.testcommunity.network

Denne episoden er hentet fra en åpen RSS-feed og er ikke publisert av Podme. Den kan derfor inneholde annonser.

Episoder(12)

It Depends Episode 10: Criterion, Norm, CAT & LOFT

This episode of the It Depends Podcast examines criterion referenced testing, norm referenced testing, computer adaptive testing (CAT), and linear on-the-fly testing (LOFT). Amanda and Tim explore the...

29 Nov 202459min

Ep9 - Test Centre V Online Proctoring

This episode of the It Depends Podcast compares online proctoring and test centres for assessment delivery. Amanda and Tim explore the advantages and challenges of both methods, discussing candidate e...

20 Sep 202449min

Ep8 - Testing on a Budget

This episode of the It Depends Podcast explores testing on a budget, from test development to secure delivery. Amanda and Tim discuss testing on a budget, exploring various ways organisations can impl...

14 Aug 202438min

Ep 7 - Performance Based Testing

This episode of the It Depends Podcast explores performance-based testing in professional certification and licensure. Amanda and Tim discuss the benefits and challenges of performance tests compared ...

16 Jul 202439min

Episode 6 - Exam Maintenance

Top Episode Takeaways: 📈 Test maintenance involves regularly monitoring test and item performance statistics to ensure content remains current and secure, and should be conducted annually at minimum...

20 Jun 202438min

Ep5: Test Blueprints

Top Episode Takeaways: 📐 Test blueprints provide the overall design and structure for an assessment. They define what content areas the test will cover and what proportion of the test each content ...

12 Mai 202427min

Ep4: Exploring Item Types

Top Episode Takeaways: 📝 Multiple choice questions are the most common item type, but they can be done well or poorly. A good multiple choice question takes effort to create but can measure a wide r...

18 Apr 202437min