A single study can show that an intervention worked. It cannot show that it will work for your next client. That second question is about external validity, and in behavior analysis the way to answer it is replication. This guide explains what external validity means in single-case research, how direct and systematic replication build it, how it differs from internal validity, and how BCBA exam questions test it.
Key takeaways
- External validity is the degree to which a study’s results hold for other people, settings, behaviors, implementers and times.
- Internal validity asks whether the intervention caused the change. External validity asks how far that finding reaches. A study needs internal validity first.
- Single-case designs use few participants and no random sampling, so they build external validity through replication, not large samples.
- Direct replication repeats the same procedure (with the same or new participants). Systematic replication changes something on purpose, such as the setting, behavior or implementer.
- Each successful systematic replication extends the generality of a finding. Failed replications are useful too: they show the limits of an intervention.
In this guide
What is external validity?
External validity is the degree to which the results of a study apply beyond the specific conditions in which they were found. A finding has strong external validity when the same functional relation shows up with other participants, in other settings, with other target behaviors, with other people delivering the intervention, and over time.
Behavior analysts often call this the generality of a finding. Be careful not to confuse it with generality as one of Baer, Wolf and Risley’s seven dimensions of ABA. That dimension is about the client’s behavior change lasting over time, appearing in other settings or spreading to other behaviors. External validity is about the research finding holding up for other clients and situations.
Internal vs. external validity
| Internal validity | External validity | |
|---|---|---|
| Question it answers | Did the intervention cause the change in behavior? | Will the same effect happen for other people, settings, behaviors and times? |
| How it is shown | Experimental control within a study: prediction, verification and replication across phases or tiers | Replication across studies, participants, settings and implementers |
| Main threats | Confounding variables: history, maturation, testing, instrumentation, poor treatment integrity | Unusual participants or settings, specially trained implementers, effects tied to one study’s conditions |
| Order | Comes first. Without it there is no finding to generalize. | Builds over time as the finding is repeated in new conditions. |
A study can have strong internal validity and limited external validity. A tightly controlled reversal design with one child in a clinic room can clearly show that a token system reduced that child’s disruptions. Whether it will work for a group of teenagers in a classroom is a separate question that only more studies can answer.
Why single-case research depends on replication
Group-design research tries to establish external validity with large, randomly selected samples: if the sample represents a population, the average result should apply to that population. Single-case research works differently. Each participant serves as their own control, and studies usually include only a few participants who were not randomly selected. So the generality of a single-case finding is not assumed from the sample. It is earned by repeating the effect.
Replication happens at two levels:
- Within a study, replication is part of baseline logic. Reintroducing the intervention in the second B phase of an ABAB design, or seeing the effect again in each tier of a multiple baseline, shows the effect is reliable. This mainly supports internal validity.
- Across participants and studies, replication shows the effect is not limited to one person or one set of conditions. This is what builds external validity.
Behavior analysts also value an individual-level view of generality: instead of asking whether an intervention works “on average,” they ask which individuals and conditions it works for, and why. Replications that fail are part of that answer.
Direct vs. systematic replication
The two main types of replication were described by Murray Sidman in Tactics of Scientific Research (1960), and they are still the terms used on the BCBA exam.
Direct replication
In a direct replication, the researcher repeats the experiment with the same procedures. It can be intrasubject (the same participant again) or intersubject (new participants with similar characteristics, under the same conditions). Direct replication mainly shows that the effect is reliable: it was not a fluke.
Systematic replication
In a systematic replication, the researcher deliberately changes one or more parts of the original study, such as the participants, setting, target behavior, implementer or a feature of the procedure, and checks whether the effect still occurs. A successful systematic replication extends the generality of the finding. An unsuccessful one shows where the limits are.
| Direct replication | Systematic replication | |
|---|---|---|
| What changes | Nothing on purpose; same procedures (same or similar participants) | One or more variables on purpose: people, setting, behavior, implementer or procedure |
| Main purpose | Shows the effect is reliable | Shows how far the effect generalizes |
| Example | A second child with the same profile gets the same DRA procedure in the same clinic | The same DRA procedure is run by a parent at home, with a different problem behavior |
Threats to external validity
A finding may not generalize when something about the original study was unusual. Common threats include:
- Participant characteristics: the participants had a skill, history or diagnosis that most clients do not share.
- Setting characteristics: the effect depended on a specialized setting, such as a one-to-one clinic room with no distractions.
- Implementer characteristics: the intervention was delivered by highly trained researchers and may not hold up with typical staff or caregivers.
- Measurement effects: participants behaved differently because they knew they were being observed (reactivity).
- Multiple-treatment interference: the effect of one intervention was shaped by earlier interventions in the same study, so it may differ when used alone.
- Poorly described procedures: if the method is not written clearly enough to repeat (the technological dimension), no one can replicate it, and its generality can never be tested.
From replications to evidence-based practice
Because single-case findings generalize through replication, groups that rate evidence-based practices set replication thresholds. A widely cited standard (Horner et al., 2005) treats a practice as evidence-based when it is supported by at least 5 single-case studies that meet quality standards, conducted by at least 3 different research teams in at least 3 different locations, with a combined total of at least 20 participants. That is external validity in numbers: the same effect, again and again, across different people and places.
How this shows up on the BCBA exam
External validity and replication fall in the Experimental Design domain of the BCBA 6th edition Test Content Outline, alongside internal validity and the features of single-case designs. Expect questions that:
- Describe a study and ask whether a change to it is a direct or systematic replication.
- Ask whether a concern is about internal or external validity (cause versus generality).
- Ask how single-case researchers establish generality (answer: replication, not large random samples).
- Mix up the two meanings of generality: the research finding (external validity) versus the client’s behavior change (the dimension of ABA).
Test yourself
Practice question 1
A researcher shows that a high-p request sequence increases compliance for three boys in a clinic. A second team then tests the same procedure with teenage girls in a group home, delivered by direct-care staff. What is the second study?
- A direct replication
- A systematic replication
- A reversal design
- A test of internal validity
Show the answer
2. A systematic replication. The second team deliberately changed the participants, setting and implementers to see whether the effect would still occur. That extends the generality of the original finding. A direct replication would repeat the same procedures under the same conditions.
Practice question 2
A BCBA reads a study with strong experimental control and asks, “Would this work for my clients, who are older and in a public school?” Which concern is she raising?
- Internal validity
- Treatment integrity
- External validity
- Social validity
Show the answer
3. External validity. She is not doubting that the intervention caused the change in the study (internal validity). She is asking whether the finding applies to different people and settings.
Practice question 3
Single-case research designs usually have few participants and no random sampling. How do behavior analysts establish the generality of their findings?
- By using inferential statistics
- By replicating the effect across participants, settings and studies
- By averaging data across participants
- By increasing the length of baseline
Show the answer
2. By replicating the effect. Generality in single-case research is built through direct and systematic replication, not through large samples or group averages.
Related reading: What Are Experimental Designs? · Independent Variables, Dependent Variables and Experimental Control · Understanding Reversal Designs · Understanding Multiple Baseline Design · Mastering Generalization in ABA
External validity FAQ
What is external validity in ABA?
External validity is the degree to which a study’s results hold for other participants, settings, behaviors, implementers and times. In behavior analysis it is often called the generality of a finding.
What is the difference between direct and systematic replication?
Direct replication repeats a study with the same procedures, with the same or similar participants, to show the effect is reliable. Systematic replication deliberately changes something, such as the setting, participants, behavior or implementer, to show how far the effect generalizes.
Can a single-case design have external validity?
Yes, but not from one study alone. A single study with a few participants shows a reliable effect for those participants. External validity builds as the effect is replicated with other people, in other settings and by other researchers.
Is external validity the same as the generality dimension of ABA?
No. The generality dimension (Baer, Wolf and Risley, 1968) is about a client’s behavior change lasting over time, appearing in other settings or spreading to other behaviors. External validity is about whether a research finding applies to other people and conditions.
Why are failed replications useful?
A replication that does not produce the same effect shows the limits of an intervention: the people, settings or conditions where it does not work. That tells practitioners when to choose something else and tells researchers what to study next.
References
- Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some current dimensions of applied behavior analysis. Journal of Applied Behavior Analysis, 1(1), 91–97.
- Cooper, J. O., Heron, T. E., & Heward, W. L. (2020). Applied behavior analysis (3rd ed.). Pearson.
- Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71(2), 165–179.
- Kazdin, A. E. (2011). Single-case research designs: Methods for clinical and applied settings (2nd ed.). Oxford University Press.
- Sidman, M. (1960). Tactics of scientific research: Evaluating experimental data in psychology. Basic Books.