Why Interrater Reliability Matters
Why Interrater Reliability Matters
As an observation-based tool, the Assessment, Evaluation, and Programming System for Infants and Children, Third Edition (AEPS®-3) provides an authentic assessment of a child’s skills and abilities and generates useful information for instruction and intervention. However, one challenge in using observation-based measures is the need to trust the judgments of professionals conducting the assessments. That’s why monitoring interrater reliability is so important.
What is interrater reliability?
Interrater reliability (IRR) refers to whether different people administering the same 1 assessment tool can do so in a consistent way. Because a child’s behavior can vary from one day to the next, interrater reliability is determined by having two or more individuals observe the child at the exact same time, using the same metric. Once the observation is complete, the ratings of the observers are compared. The more similar the ratings, the higher the reliability is between the observers.
Why check interrater reliability?
An assessment can’t be considered authentic and accurate if it can’t be conducted and 2 scored as intended. The purpose of checking interrater reliability is to ensure that the assessment score is an accurate representation of the child. Higher interrater reliability indicates that the assessment is a more reliable gauge of the child’s behavior. Checking IRR, then, contributes to more meaningful assessments for young children.
How does the AEPS-3 IRR Certification help?
Now updated for AEPS-3, this field-tested online certification module helps your
3program ensure that AEPS-3 is used accurately to promote real progress for every child.
The module includes a narrated training presentation with video examples, so that users can practice and prepare for the certification test. Users who complete the test and achieve 80% overall agreement with the gold standard scoring receive an AEPS Interrater Reliability Certification that lasts for three years. When AEPS-3 users are certified, you can be confident that your assessment information is reliable: for every child, across observers, across different circumstances, and on different days.
This handout includes information adapted from Assessing Young Children
Learn More
in Inclusive Settings, Second Edition, by Kristie Pretti-Frontczak, Ph.D., & Jennifer Grisham, Ed.D., with Lynn D. Sullivan, M.Ed.
AEPS® is a registered trademark of and is owned by Paul H. Brookes Publishing Co., Inc.