Interobserver agreement (IOA) tells you whether you can trust your data. But there is no single IOA formula. The right one depends on how you collected the data: by counting, timing, or recording intervals. This guide walks through every type of IOA, matched to its data collection method, with formulas, worked examples, and tips for choosing the right one on the BCBA exam and in practice.
Key takeaways
- What IOA is: the degree to which two independent observers record the same values for the same events.
- Match the IOA to the data: event recording, timing, and interval or time sampling data each have their own IOA formulas.
- Stricter is better: total count and total duration IOA are easy but can hide disagreements. Per-interval and per-occurrence versions are more conservative.
- Watch the base rate: use scored-interval IOA for low-rate behavior and unscored-interval IOA for high-rate behavior.
- Standards: aim for at least 80% agreement (90% preferred), checked in at least 20% of sessions across every phase.
In this guide
- What is interobserver agreement (IOA)?
- Which IOA goes with which data collection method?
- IOA for event recording (count data)
- IOA for timing data (duration, latency, IRT)
- IOA for interval recording and time sampling
- IOA formulas at a glance
- IOA standards: how much and how often
- What to do when IOA is low
- FAQ
What is interobserver agreement (IOA)?
Interobserver agreement is the degree to which two or more independent observers report the same values after measuring the same events. When two trained observers watching the same session record nearly the same data, you have more confidence that a change on the graph reflects a real change in behavior, not a change in how someone was measuring it.
For IOA to mean anything, the observers must:
- Use the same measurement system and operational definition
- Observe the same person, behavior, and time period
- Record independently, without seeing or hearing each other’s data
On the BCBA exam, IOA falls under Domain C, Measurement, Data Display, and Interpretation (21 scored questions), especially task C.8 on evaluating the validity and reliability of measurement procedures.
Which IOA goes with which data collection method?
Start here. Find the way you collected your data, then use one of the IOA types in that box.
Event recording
Count or frequency
- Total count
- Mean count-per-interval
- Exact count-per-interval
- Trial-by-trial
Timing
Duration, latency, IRT
- Total duration
- Mean duration-per-occurrence
- Mean latency-per-response
- Mean IRT-per-response
Interval recording
Partial, whole interval
- Interval-by-interval
- Scored-interval
- Unscored-interval
Time sampling
Momentary time sampling
- Interval-by-interval
- Scored-interval
- Unscored-interval
IOA for event recording (count data)
Use these when observers count how many times a behavior occurs. We will use the same example for all three count-based formulas: two observers count hand-raising across five 2-minute intervals.
| Interval | 1 | 2 | 3 | 4 | 5 | Total |
|---|---|---|---|---|---|---|
| Observer A | 3 | 2 | 0 | 4 | 1 | 10 |
| Observer B | 3 | 1 | 0 | 4 | 2 | 10 |
Total count IOA
Formula(Smaller total ÷ Larger total) × 100
Example: 10 ÷ 10 × 100 = 100%.
Total count IOA is the simplest measure, but also the least strict. Here the observers agree perfectly on the total even though they disagreed in two of the five intervals. It tells you the totals match, not that the observers counted the same responses.
Mean count-per-interval IOA
Formula(Sum of each interval’s IOA ÷ Number of intervals) × 100
Example: calculate smaller ÷ larger for each interval: 100%, 50%, 100%, 100%, 50% (an interval where both recorded 0 counts as 100%). The mean is 80%.
Breaking the session into intervals makes it much harder for disagreements to cancel each other out.
Exact count-per-interval IOA
Formula(Intervals with an exact match ÷ Total intervals) × 100
Example: the observers recorded exactly the same count in 3 of 5 intervals, so exact count-per-interval IOA = 60%.
This is the most stringent count-based IOA. Notice how the same data produce 100%, 80%, or 60% depending on the formula.
Trial-by-trial IOA
Formula(Trials with agreement ÷ Total trials) × 100
Use trial-by-trial IOA for discrete trial data, where each trial is scored correct or incorrect. For example:
| Trial | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
|---|---|---|---|---|---|---|---|---|---|---|
| Observer A | + | + | − | + | + | − | + | + | + | − |
| Observer B | + | + | − | + | − | − | + | + | + | + |
Example: the observers agreed on 8 of 10 trials, so trial-by-trial IOA = 80%.
IOA for timing data (duration, latency, IRT)
Use these when observers time behavior. In this example, two observers time three tantrums, in seconds:
| Occurrence | 1 | 2 | 3 | Total |
|---|---|---|---|---|
| Observer A | 40 s | 50 s | 30 s | 120 s |
| Observer B | 36 s | 50 s | 22 s | 108 s |
Total duration IOA
Formula(Shorter total duration ÷ Longer total duration) × 100
Example: 108 ÷ 120 × 100 = 90%. Like total count IOA, this can hide disagreements on individual occurrences.
Mean duration-per-occurrence IOA
Formula(Sum of each occurrence’s IOA ÷ Number of occurrences) × 100
Example: calculate shorter ÷ longer for each tantrum: 90%, 100%, 73.3%. The mean is 87.8%.
This is the more conservative timing measure. The same formula works for mean latency-per-response IOA (time from cue to response) and mean IRT-per-response IOA (time between responses): just compare each latency or IRT instead of each duration.
IOA for interval recording and time sampling
Use these for partial-interval, whole-interval, and momentary time sampling data, where each interval is scored as an occurrence (+) or nonoccurrence (−). In this example, two observers record a low-rate behavior across ten intervals:
| Interval | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
|---|---|---|---|---|---|---|---|---|---|---|
| Observer A | + | + | − | − | − | − | + | − | − | − |
| Observer B | + | − | − | − | − | − | + | − | + | − |
Interval-by-interval IOA
Formula(Intervals agreed ÷ (Intervals agreed + Intervals disagreed)) × 100
Example: the observers agreed on 8 of 10 intervals (both + or both −), so interval-by-interval IOA = 80%.
This is the standard interval measure, also called point-by-point IOA. Its weakness: when behavior is very rare or very frequent, most agreements come from intervals that were easy to agree on.
Scored-interval IOA (low-rate behavior)
Formula(Intervals both scored + ÷ Intervals either observer scored +) × 100
Example: at least one observer scored the behavior in 4 intervals, and both did in 2. Scored-interval IOA = 2 ÷ 4 × 100 = 50%.
That is a big drop from 80%. Interval-by-interval IOA looked acceptable only because the observers agreed on all the empty intervals. For low-rate behavior, scored-interval IOA gives the more honest picture.
Unscored-interval IOA (high-rate behavior)
Formula(Intervals both scored − ÷ Intervals either observer scored −) × 100
Example: at least one observer scored no behavior in 8 intervals, and both did in 6. Unscored-interval IOA = 6 ÷ 8 × 100 = 75%.
Use unscored-interval IOA for high-rate behavior, where interval-by-interval IOA would be inflated by all the intervals in which both observers easily agreed the behavior occurred.
IOA formulas at a glance
| Data collection method | IOA type | Formula | Strictness |
|---|---|---|---|
| Event recording | Total count | Smaller ÷ larger total × 100 | Least strict |
| Event recording | Mean count-per-interval | Mean of each interval’s IOA × 100 | Moderate |
| Event recording | Exact count-per-interval | Exact-match intervals ÷ total intervals × 100 | Most strict |
| Discrete trials | Trial-by-trial | Agreed trials ÷ total trials × 100 | Strict |
| Timing | Total duration | Shorter ÷ longer total × 100 | Least strict |
| Timing | Mean duration-per-occurrence | Mean of each occurrence’s IOA × 100 | More strict |
| Interval / time sampling | Interval-by-interval | Agreed intervals ÷ total intervals × 100 | Standard |
| Interval / time sampling | Scored-interval | Both + ÷ either + × 100 | Best for low rates |
| Interval / time sampling | Unscored-interval | Both − ÷ either − × 100 | Best for high rates |
IOA standards: how much and how often
- Acceptable agreement: by convention, at least 80%, with 90% or higher preferred.
- How often: in at least 20% of sessions, preferably 25% to 33%.
- Where: in every phase or condition, not just baseline or just treatment.
- Spread it out: across days of the week, times of day, settings, and observers.
- Report it clearly: most often as a narrative in the method section, with the mean and range for each condition.
What to do when IOA is low
- Check the definition. Vague operational definitions are the most common cause. Add examples and non-examples.
- Retrain observers. Use behavioral skills training: instructions, modeling, rehearsal, and feedback, with practice videos until they reach criterion.
- Look for observer drift. Observers can gradually change how they apply a definition over time. Schedule periodic recalibration.
- Reduce reactivity. Observers who know they are being checked may record more carefully. Make IOA checks routine and unannounced where possible.
- Check the IOA formula itself. Make sure the formula fits the data and the base rate of the behavior.
Practice question
Two observers use partial-interval recording to measure a student’s self-injury, which occurs in fewer than 10% of intervals. Which IOA method gives the most accurate estimate of agreement?
- Total count IOA
- Interval-by-interval IOA
- Scored-interval IOA
- Unscored-interval IOA
Show the answer
C. Scored-interval IOA. The data come from interval recording, which rules out total count IOA. Because the behavior is low-rate, most intervals are empty, so interval-by-interval IOA would be inflated by easy agreements on nonoccurrence. Scored-interval IOA only looks at intervals where at least one observer scored the behavior. Unscored-interval IOA is the right choice for high-rate behavior, not low-rate.
Master Domain C, one task at a time
The free BCBA Study Checklist lists every task in the 6th edition Test Content Outline, including all 12 Measurement tasks, so you can track exactly what you have mastered.
IOA FAQ
What is IOA in ABA?
Interobserver agreement (IOA) is the degree to which two or more independent observers report the same values after measuring the same events. It is the most common way behavior analysts check the reliability and believability of their data.
What are the types of IOA?
For event recording: total count, mean count-per-interval, exact count-per-interval, and trial-by-trial IOA. For timing data: total duration and mean duration-per-occurrence IOA (and the matching latency and IRT versions). For interval recording and time sampling: interval-by-interval, scored-interval, and unscored-interval IOA.
What is an acceptable IOA percentage?
By convention, 80% agreement is the usual minimum, and 90% or higher is preferred, especially in research.
How often should IOA be collected?
By convention, in at least 20% of sessions, and preferably 25% to 33%, spread across every phase or condition, and across days, times, settings, and observers.
When should I use scored-interval vs. unscored-interval IOA?
Use scored-interval IOA when the behavior occurs at a low rate, because interval-by-interval IOA is inflated by all the intervals where both observers agreed nothing happened. Use unscored-interval IOA when the behavior occurs at a high rate, for the opposite reason.
Which IOA is most conservative for event recording?
Exact count-per-interval IOA is the most stringent event-recording measure, because an interval only counts as an agreement when both observers recorded exactly the same number of responses.
Related reading: Understanding Multiple Baseline Design · BCBA 6th Edition Task List Explained · How to Pass the BCBA Exam in 2027
References
- Behavior Analyst Certification Board. (2022). BCBA test content outline (6th ed.).
- Cooper, J. O., Heron, T. E., & Heward, W. L. (2020). Applied behavior analysis (3rd ed.), Chapter 5: Improving and assessing the quality of behavioral measurement. Pearson.
- Reed, D. D., & Azulay, R. L. (2011). A Microsoft Excel 2010 based tool for calculating interobserver agreement. Behavior Analysis in Practice, 4(2), 45–52.
- Repp, A. C., Deitz, D. E. D., Boles, S. M., Deitz, S. M., & Repp, C. F. (1976). Differences among common methods for calculating interobserver agreement. Journal of Applied Behavior Analysis, 9(1), 109–113.