Inter-Observer Agreement

Inter-observer agreement (IOA) measures the consistency between two independent observers recording the same behavior at the same time. sight·line includes built-in IOA calculation and reporting, so you can document reliability without a separate spreadsheet.

When to use IOA

IOA is required or strongly recommended in two situations:

IDEA evaluations. Many district policies and state guidelines require documented inter-observer reliability for behavioral observation data used in eligibility determinations or FBA-based intervention planning. Having a printed IOA report protects the evaluation and demonstrates methodological rigor.

Research-grade data collection. If observation data will be used in a study, program evaluation, or published report, ≥80% agreement across all behaviors is the generally accepted minimum threshold in educational and applied behavior analysis settings. sight·line highlights any metric that falls below this threshold so you know immediately where reliability is insufficient.

Even when not strictly required, running IOA at least once per student — especially early in an evaluation — is good practice. It catches disagreements in how behaviors are defined before they affect your conclusions.

Session setup

IOA requires both observers to configure their sessions identically before they begin.

Agree on the following before starting:

• The same student
• The same observation method (e.g., partial interval, whole interval, frequency)
• The same behavior definitions and behavior labels (exact spelling matters for matching)
• The same interval length (for interval-based methods)
• The same phase label, if applicable

Assign a shared session code. On the Setup screen, both observers enter the same IOA session code in the IOA field. Use something specific enough to be unique, such as ioa-2024-03-15-room204. Both sessions must have an identical code for sight·line to pair them later.

Both observers then record independently and simultaneously, without consulting each other during the session.

Linking sessions after recording

Once both sessions are complete:

1. Open either session in the Results screen.
2. Tap Link IOA session.
3. Enter the shared session code.
4. sight·line searches your local database for a matching session with the same code and pairs the two recordings.

You only need to link once — the pairing is stored with both sessions.

Agreement metrics

sight·line calculates three standard agreement metrics for each behavior:

Total agreement — the percentage of scoring opportunities (intervals, blocks, or duration windows) where both observers recorded the same result, regardless of whether the behavior occurred or not. This is the most commonly reported metric.

Occurrence agreement — calculated only for intervals where at least one observer scored the behavior as occurring. This metric is more sensitive when behavior rates are low, since rare behaviors inflate total agreement even when observers often disagree on occurrence.

Non-occurrence agreement — calculated only for intervals where neither observer scored the behavior. This metric is useful when behavior rates are high and you want a conservative reliability estimate.

How agreement is calculated by method:

Method	Agreement unit
Partial interval, whole interval, momentary time sampling	Point-by-point interval comparison
Frequency (event recording)	Block-by-block comparison across equal time windows
Duration recording	Duration agreement across the session

sight·line calculates all three metrics for each behavior and highlights any result below 80% — the generally accepted minimum threshold for educational settings per IDEA guidance. If a metric is highlighted, review your operational definitions and re-train before relying on that data.

IOA reports

From the linked session’s Results screen, tap Export IOA Report to generate a PDF that includes:

• Per-behavior agreement table — total, occurrence, and non-occurrence agreement percentages for each behavior, with values below 80% flagged.
• Interval-by-interval comparison table — a side-by-side view of what each observer recorded at each interval, making it easy to identify where disagreements occurred.
• Summary statement — a plain-language reliability statement formatted for inclusion in evaluation reports, such as: “Inter-observer agreement was calculated for [behavior] using point-by-point interval agreement. Total agreement was 87%, occurrence agreement was 84%, and non-occurrence agreement was 91%, indicating adequate reliability.”

The PDF is exported locally to your device and never transmitted anywhere.