Review of the Efficacy of the Picture Exchange Communication

J Autism Dev Disord (2009) 39:1471–1486 DOI 10. 1007/s10803-009-0763-y ORIGINAL PAPER A Review of the Ef? cacy of the Picture Exchange Communication System Intervention Deborah Preston ? Mark Carter Published online: 3 June 2009 O Springer Science+Business Media, LLC 2009 Abstract The Picture Exchange Communication System (PECS) is a communication program that has become widely used, especially with children with autism. This paper reports the results of a review of the empirical literature on PECS. A descriptive review is provided of the 27 studies identi? d, which included randomized controlled trials (RCTs), other group designs and single subject studies. For 10 appropriate single subject designs the percentage of nonoverlapping data (PND) and percentage exceeding median (PEM) metrics were examined. While there are few RCTs, on balance, available research provides preliminary evidence that PECS is readily learned by most participants and provides a means of communication fo r individuals with little or no functional speech. Very limited data suggest some positive effect on both socialcommunicative and challenging behaviors, while effects on speech development remain unclear.Directions for future research are discussed including the priority need for further well-conducted RCTs. Keywords Picture exchange communication system A Augmentative and alternative communication A Autism Introduction Serious de? cits in communication form part of the primary diagnostic criteria for autism (American Psychiatric Association 2000). It has been estimated that around D. Preston A M. Carter (&) Macquarie University Special Education Centre, Macquarie University, Sydney, NSW 2109, Australia e-mail: mark. [email  protected] edu. au one-third to one-half of children and adults with autism do not have unctional speech (Mirenda 2003). Such individuals may show only pre-intentional communication, such as reaching for a desired item, or communication may demonstrate intent through behaviors such as alternating eye gaze, and conventional gestures such as pointing (Yoder et al. 2001). Communication may also take the form of challenging behaviors (Mirenda 1997). When speech does develop it may be limited mainly to unusual or echolalic verbalizations (Paul 2005). Individuals with serious developmental disabilities other than autism may also fail to develop speech and language skills (Westling and Fox 2004).In order to help develop communication skills, various forms of augmentative and alternative communication (AAC) have been developed. These include the use of manual signs (e. g. , Layton 1988; Yoder and Layton 1988), voice output communication devices (VOCAs) (reviewed by Lancioni et al. 2001), and various picture-based systems (Keen et al. 2001; Sigafoos et al. 1996). The Picture Exchange Communication System (PECS) is a picturebased system developed by Bondy and Frost (1993, 1994) to help young children with autism acquire functional communication sk ills. PECS appears promising for several reasons.First, it avoids dif? culties inherent in other systems by requiring very few prerequisites; in fact the only prerequisite is that the individual can clearly indicate (e. g. , by reaching for an item) what he or she wants, in a way that can be shaped into exchanging a physical symbol such as a picture (Bondy and Frost 2002). Other skills such as eye contact, motor or verbal imitation skills, the ability to sit quietly in a chair, match-to-sample skills, picture discrimination, or the ability to follow verbal prompts are not necessary (Bondy and Frost 1994, 2002), at least at the earliest program stage. 23 1472 J Autism Dev Disord (2009) 39:1471–1486 Second, the ? rst skill taught in PECS is requesting. Requesting has often been targeted in early instruction of individuals with developmental disabilities due to motivational considerations (Reichle and Sigafoos 1991). In relation to PECS, it is argued that individuals with autism in particular are less likely to be motivated by the social consequences of labeling or commenting (Bondy and Frost 1995) and more likely to be motivated by requesting and immediately obtaining a speci? c, typically concrete, desired item (Bondy and Frost 1994).Third, PECS systematically addresses the issue of spontaneity, which has often been reported as problematic in individuals with autism spectrum disorders (ASD) (Chiang and Carter 2008; Koegel 2000). Rather than being dependent on a partner to establish a communicative exchange, or requiring a partner to watch for the learner to point to a picture board or generate a manual sign, which could easily be missed, PECS speci? cally teaches the individual to approach the partner and gain their attention by putting a picture symbol into their hand.Fourth, picture symbols can be highly iconic, closely resembling their referents (Ganz and Simpson 2004; Mirenda 2003). Consequently, they may be easily recognized by the learner (Ganz and Simpson 2004) and are more recognizable by communicative partners than some other systems, such as manual signs (Lancioni et al. 2007). The PECS protocol begins with a reinforcer assessment through which the trainer determines an ordered list of reinforcers for the individual (Bondy and Frost 1998). This is followed by six phases, which are brie? overviewed in Table 1. For each phase, the criterion for successful completion is 80% or more trials successful without prompting in a 10-trial block (Charlop-Christy and Jones 2006). The randomized control trial (RCT) is recognized as providing the gold standard for evaluating clinical interventions in areas such as medicine and education (Evidence-Based Medicine Working Group 1992; What Works Clearinghouse 2006) and ideally evaluations would be limited to such evidence (Carter and Wheldall 2008).Such designs, however, are relatively rare in educationrelated areas (Carter and Wheldall 2008) for a variety of reasons and clinicians must oft en look to a second line of evidence to inform decision-making. Single subject quasiexperimental designs employ repeated measures of the dependent variable over time with a single or small number of participants. Although not offering the standard of evidence of RCTs, the best of these designs are considered capable of effectively controlling major threats to internal validity and strong conclusions about causal inference can be drawn (Campbell and Stanley 1963; Horner et al. 005). Similarly, high quality quasi-experimental group designs, speci? cally those involving non-equivalent groups with pre-test matching, are generally considered to be interpretable (Flay et al. 2004). The weakest level of evidence is provided by pre-experimental designs, such as single group pre-test—post-test studies, where few threats to internal validity are controlled and causal inference cannot be inferred with any degree of con? dence (Campbell and Stanley 1963). Early papers on PECS were largel y descriptive with limited data (e. . , Bondy and Frost 1993, 1994, 1995, 1998), or presented outcome data without control (e. g. , Schwartz et al. 1998). In recent years, however, a number of interpretable group studies (e. g. , Yoder and Stone 2006b) and single subject studies (e. g. , Charlop-Christy et al. 2002; Tincani 2004) have been published. Lancioni et al. (2007) reviewed the use of PECS and VOCA (voice output communication aids) for request making in individuals with developmental disabilities. In addition to studies following the PECS protocols of Frost and BondyTable 1 Summary PECS stages Phase Teaching target I II Make requests through picture exchange Persistence in initiating communication Discrimination between symbols Introduction of sentence structure Answering question with a request Commenting Description Second person acts as a prompter from behind learner; when learner reaches for the desired item, physically prompts to exchange picture; prompts faded as quick ly as possible Communicative partner moves gradually further away; picture is also gradually moved further away; number of communicative partners increased; ‘‘Communication book’’ is introduced; range of items requested is increased, but only one picture and item is available at a time Initially, two pictures are presented (highly preferred and neutral or disliked); more pictures are added; later, more than one preferred item is offered at a time; periodic correspondence checks are carried out to check accuracy at discrimination Taught to use a sentence strip, placing an ‘‘I want’’ as well as the symbol; requests can also be expanded with attributes such as color or size, e. g. ‘‘I want red playdough’’ Taught to answer question ‘‘What do you want? ’’ Taught to respond to other simple questions such as ‘‘What do you see? ’’; gradually, more spontaneous com menting is developed III IV V VI 123 J Autism Dev Disord (2009) 39:1471–1486 1473 (1994, 2002), they also included studies employing ‘‘any conventional use of pictorial material as a way of making requests for preferred items’’ (p. 4).Thus, studies that did not follow PECS protocols, and in which the participants pointed to a picture rather than actually exchanging it, were included (e. g. , Dyches et al. 2002; Keen et al. 2001). In addition, no attempt was made to quantify the data obtained from the studies to evaluate either the overall ef? cacy or effectiveness of the approach or the effect of relevant variables on outcomes. To date, no comprehensive review of empirical literature speci? cally examining the PECS approach of Frost and Bondy (1994, 2002) appears to have been published. The present paper looks speci? cally at studies of PECS intervention as described by Frost and Bondy (1994, 2002).In the absence of a substantial number of gold-stand ard RCT studies that would allow a conventional meta-analysis, a broader approach to evaluation of the research was undertaken. This review is intended to examine the extant empirical research on PECS, with speci? c consideration of the research designs employed and, consequently, the strength of conclusions that can be drawn. excluded (e. g. , Son et al. 2006). One article in which previously taught communication using the PECS program was compared to facilitated communication (Simon et al. 1996) was excluded because there was no PECS intervention during the study. The study of Rosales and Rehfeldt (2007), in which the ? rst three phases of PECS was taught prior to the experiment, was lso excluded because no data on the results of the PECS training was provided. Analysis A summary of each study was prepared including participants, research design, treatment and duration of study, outcomes investigated, setting, PECS version and phases implemented, and a summary of the outcomes, as well as inter-observer and procedural reliability, social validity, maintenance and generalization data. Ages of participants were coded into 5, 5–8, 9–17, or over 18 years. Four categories of dependent variable were identi? ed: PECS exchanges (number or percentage of independent exchanges), speech or vocalization, social-communicative behaviors, and undesirable behaviors.As the majority of studies employed single subject designs, they were coded for quality using an adaptation of the guidelines for single subject research presented by Horner et al. (2005). These indicators addressed several areas: adequacy of participant and setting descriptions; dependent variables; independent variables; baseline; experimental control/internal validity; external validity; and social validity. A total of 10 points were allocated to each area with the exception of external validity, which was allocated 5 points in recognition of the inherent limitations of single subject designs in th is regard. Thus, studies were rated on a scale from 0 to 65, with higher scores indicating greater quality.Details of the criteria are included in the Appendix. Traditionally, single subject studies have been interpreted by visual inspection of graphed data (Reynhout and Carter 2006). More recently, attempts have been made to quantify results of these studies objectively, and to provide reliable data summaries for evaluating evidence-based interventions (Parker et al. 2007). The most commonly used of the resulting statistical indices is the percentage of non-overlapping data (PND) (Scruggs et al. 1987). The PND is the percentage of treatment data points that are above (or below when behavior decrease is targeted) the highest (or lowest) baseline data point.Scruggs and Mastropieri (1998) provided guidelines for the interpretation of PND: scores between 91 and 100 indicate highly effective interventions, between 71 and 90 effective interventions, between 51 and 70 questionable interve ntions, and 50 or below ineffective interventions. Whilst PND has been criticized on a number of grounds (e. g. , Allison and Method Search Strategy Empirical studies using PECS were identi? ed through computerized searches of A? Education, British Education Index, ERIC, Expanded Academic ASAP, Linguistic and Language Behavior Abstracts, PsycINFO, PubMed and ScienceDirect, using the descriptors ‘‘PECS’’ or ‘‘Picture Exchange Communication System’’. In addition, manual searches of the reference lists of articles identi? ed were carried out to locate further studies.Studies were included if they: (1) (2) were journal articles in English from 1992 to July 2007; used PECS (Bondy and Frost 1994; Frost and Bondy 1994, 2002) as whole or part of an intervention strategy as indicated by reference to program documentation and description of implementation; presented group or individual data on the results of the intervention. (3) Articles th at referred to PECS but did not follow Frost and Bondy’s protocol were excluded. For example, Dooley et al. (2001) used a ‘‘PECS-based schedule board’’ (p. 58) but no actual picture exchange. In addition, articles that used a picture exchange system but did not speci? cally stated that the PECS protocols (Bondy and Frost 1994; Frost and Bondy 1994, 2002) were employed were 123 1474 J Autism Dev Disord (2009) 39:1471–1486 Gorman 1993; Salzberg et al. 1987; White 1987), it is nevertheless the most widely used statistic for quantifying data from single subject studies (e. g. Bellini and Akullian 2007; Erion 2006; Lee et al. 2007; Reynhout and Carter 2006; Xin et al. 2005). A particular disadvantage of PND is that if any baseline data point has reached the ceiling or ? oor level of the measurement scale, the calculated PND is 0%, even if visual inspection indicates a treatment effect (Ma 2006). Ma (2006) has suggested an alternative, the percentag e of data points exceeding the median (PEM). The PEM is the percentage of treatment data points that are above (or below when behavior decrease is targeted) the median baseline data point. There is also evidence that PEM may correlate better with author judgments of program ef? cacy than PND (Ma 2006).Nevertheless, PND is by far the most widely used metric for summarizing single subject studies and comparative data are available on a range of interventions. The application of the PEM statistic is very limited to date but, given its potential advantages, it was decided to calculate both PEM and PND values in the current review. It has also been argued that con? dence in ? ndings from analysis of single subject studies may be strengthened if multiple approaches to synthesis converge on similar conclusions (Smoot et al. 1990). PND and PEM statistics were calculated for all single subject studies with graphed data including a baseline and intervention phase. Changing conditions (i. e. , PECS phase changes), were coded as part of the ‘‘intervention’’ phase.Metrics were initially calculated for treatment data only and then for all intervention data, including treatment, maintenance and generalization phases. The PND statistic was calculated for each study using the pooled number of non-overlapping data points across all subjects and categories of dependent variable (PECS exchanges, speech/ vocalization, social-communicative behaviors, undesirable behaviors). In addition, a PND statistic was calculated for each participant and for each category of dependent variable within relevant studies. Similarly, PEM statistics were calculated using the pooled number of data points exceeding, or below when appropriate, (i. e. , for undesirable behavior) the baseline median.In cases where the exact value of data points on a graph was dif? cult to determine, a copy of the graph was obtained from a Portable Document Format copy of the article or a good qua lity digital scan. Subsequently, numeric data were extracted using the Digitizelt (Bormann 2003) computer software. Inter-Rater Reliability PND and PEM values were independently calculated by the ? rst and second authors for ? ve randomly selected single subject studies (50% of studies for which calculation was possible). Values were calculated for each graph that included a baseline and time-series data. Where more than one panel was shown on the same graph (e. g. multiple baseline, alternating treatments), a value was calculated for each panel. For each panel, reliability was calculated by dividing the lower percentage value by the higher percentage value and multiplying by 100 (i. e. , if both raters agreed on the percentage value, the reliability was 100%). The same ? ve studies were independently rated for quality indicators by the ? rst and second authors. Inter-rater reliability was calculated by dividing the number of agreements by the total of agreements and disagreements, and multiplying by 100. Inter-rater reliability for both PND and PEM was 100% for 54 panels and over 90% for the remaining three panels with an overall mean agreement of 99. 8%.There were discrepancies in only three graphs; the majority of these related to determining how many data points were present in very small ? gures. Inter-rater reliability for quality indicators was 97. 5%. Results A summary of the participants, research design, inclusion of maintenance or generalization data, PECS phases taught, and outcomes examined is presented in Table 2. Research Design The early studies (Bondy and Frost 1993, 1994, 1998; Schwartz et al. 1998) were all reports or program evaluation data without adequate experimental control, as were two later studies (Liddle 2001; Webb 2000). Malandraki and Okalidou (2007) used a case study.Magiati and Howlin (2003), in their pilot study, used a pre-PECS treatment measure plus three measures over time, with data mainly from teacher ratings. All of these studies can be considered pre-experimental. Of the 14 single subject studies, 4 used alternating treatments. Adkins and Axelrod (2001), Chambers and Rehfeldt (2003) and Tincani (2004) compared PECS and manual signing, while Bock et al. (2005) compared PECS and VOCA (voice output communication aid). Four studies (Charlop-Christy et al. 2002; Rehfeldt and Root 2005; Tincani et al. 2006, Study 1; Yokoyama et al. 2006) used a multiple baseline across participants, while two (Frea et al. 2001; Kravits et al. 2002) used a multiple baseline across settings, one (Marckel et al. 006) used a multiple baseline across descriptors taught, and one (Cummings and Williams 2000) used a multiple baseline across activities. Two studies (Stoner et al. 2006; Tincani et al. 2006, Study 2) used an ABAB design, while one (Ganz and Simpson 123 Table 2 Summary of Studies Ages Dependent variable Research design Maintenance (M) PECS or generalization (G) Phases data I–III I–III I–III Iâ⠂¬â€œIV I–III I–III I–III I–VI III Picture exchange, sign Picture exchange, VOCA Picture exchange Picture exchange, speech Picture exchange, speech Speech Social/communicative Picture exchange, sign language Speech, social, behavior (variation) Authors Participants DiagnosisAdkins and Axelrod (2001) – 35 Autism Autism Autism Autism Autism 1 ‘‘autistic characteristics’’ Autism 2 autism, 3 PDD 3–5 years Single-subject (multiple baseline) – 3–12 years Single-subject (multiple baseline) M,G 19–40 years Single-subject (alternating treatment) G 3–7 years 3–7 years Non-equivalent control group Non-equivalent control group – G 32 months Program evaluation – 6 years Program evaluation – 6 years– adult Program evaluation – 4 years Single-subject (alternating treatment) G 1 PDD 7 years Single-subject (alternating treatment) G Bock et al. (2005) 6 Bondy and F rost (1993) 74 Bondy and Frost (1994) 85 Bondy and Frost (1998) 1 J Autism Dev Disord (2009) 39:1471–1486 Carr and Felce (2007a) Carr and Felce (2007b) 10 41 Chambers and Rehfeldt (2003) 4 Charlop-Christy et al. (2002) 3Cummings and Williams (2000) Autism Autism Autism Autism 14 ASD Autism or ASD Autism Autism – 16 autism/PDD-NOS – 3–6 years 22–31 years 20–34 years 4–5 years 10 years Case study Single-subject (multiple baseline) Single-subject (multiple probe) Program evaluation Single-subject (ABAB) Single-subject (alternating treatment) 9–11 years Single-subject (multiple baseline, ABAB) 5–12 years Single group School Program evaluation 6 years Single-subject (multiple baseline) 3–7 years 4–11 years Single-subject (changing criterion RCT G M – – G M G – G G G G 4 years Single-subject (multiple baseline) – 5 Picture exchange, other I–III I–IV I–VI I–I II I–VI I–VI I–VI Extension I–III I–IV I–IV I–III I–IV Picture exchange, behavior Picture exchange, speech Picture exchange, speech, ADOS-G Picture exchange, speech, social Picture exchange Picture exchange, speech, other Picture exchange Picture exchange (improvised requests) Picture exchange, other Picture exchange, speech, other Picture exchange Picture exchange Picture exchange, sign, speech PECS, speech 1475 Frea et al. (2001) 1 Ganz and Simpson (2004) Howlin et al. (2007) 3 84 Kravits et al. (2002) 1 Liddle (2001) 21 Magiati and Howlin (2003) 34Malandraki and Okalidou (2007) 1 Marckel et al. (2006) 2 Rehfeldt and Root (2005) 3 Schwartz et al. (1998) 31 Stoner et al. (2006) 5 Tincani (2004) Autism 2 1 autism, 1 PDD-NOS 5–6 years 123 Tincani et al. (2006) 3 1476 J Autism Dev Disord (2009) 39:1471–1486 Maintenance (M) or generalization (G) data M,G 2004) used a within subjects changing criterion design. In seve ral studies, a changing criterion was included, re? ecting the PECS phase changes but it was secondary to the main design (Bock et al. 2005; Chambers and Rehfeldt 2003; Cummings and Williams 2000; Rehfeldt and Root 2005; Stoner et al. 2006; Tincani 2004; Tincani et al. 2006; Yokoyama et al. 2006).Comparative group designs were employed in ? ve papers. Yoder and Stone (2006a, b) used random assignment to PECS or Responsive Education and Prelinguistic Milieu Teaching (RPMT) intervention groups, while Howlin et al. (2007) used random assignment of classes to immediate treatment, delayed treatment or no treatment with PECS groups. Carr and Felce (2007a, b) used a quasiexperimental group design whereby PECS intervention and control groups were chosen by geographical location, and included both within subjects and between group measures. Pre-test equivalence of the groups was established. Participants Picture exchange, speech PECS Phases I–VI I–VI G I–VI G –Sin gle-subject (multiple baseline)) I–IV Picture exchange, speech Dependent variable Speech Social In total, there were 456 participants in the 27 studies; of these, 394 (86%) received PECS intervention and 62 (14%) were in non- or alternative-intervention groups. Of the total, 377 (83%) were described as having ASD. Ages of participants ranged from 20 months to 40 years and there were 198 males (43%) and 38 (8%) females with the gender of 220 (48%) of participants unstated. Where the same or a subgroup of participants were reported in multiple studies (Carr and Felce 2007a, b; Yoder and Stone 2006a, b), they were counted only once. The group experimental (Howlin et al. 007; Yoder and Stone 2006a, b) or quasi-experimental (Carr and Felce 2007a, b) studies involved a total of 161 participants (35% of the total sample): 98 in PECS intervention groups and 92 in control or other treatment groups. The Delayed Treatment Group in the Howlin et al. (2007) study was used as both control and intervention at different times. All these children were described as having autism or PDD-NOS and little or no speech. They ranged in age from 20 months to 11 years at study commencement. These studies all provided information on the initial abilities of the participants based on standardized tests. The single subject studies involved a total of 42 participants (9% of the total sample) and all provided information on diagnosis, age and gender.Only a minority provided information on the diagnostic instrument or protocol used to identify ASD (Ganz and Simpson 2004; Marckel et al. 2006; Yokoyama et al. 2006), described the degree of autism or provided standardized assessment data or a description of general ability for all participants (Chambers and Rehfeldt 2003; Frea et al. 2001; Kravits et al. 2002; Rehfeldt and Root 2005; Stoner et al. 2006; Research design Program evaluation 55–70 months RCT 21–54 months Autism/PDD 36 Yoder and Stone (2006a) 20–53 months Autism/PDD Yoder and Stone (2006b) 36 RCT Diagnosis Participants 6 Table 2 Summary of Studies 123 Yokoyama et al. (2006) Authors Webb (2000) 3 Autism 5 ASD 5–7 years Ages J Autism Dev Disord (2009) 39:1471–1486 1477 Yokoyama et al. 2006).Most researchers did document initial communication skills, either using standardized test results or a description of functional skills, although some descriptions were minimal. Participants were almost entirely described as non-verbal or having little or no functional speech, or in some cases no functional communication. The participants in the Marckel et al. (2006) study were able to use PECS independently to make requests at the start of the research. Participants in three studies were explicitly identi? ed by researchers as having challenging behavior (Adkins and Axelrod 2001; CharlopChristy et al. 2002; Frea et al. 2001). Interobserver and Procedural Reliability Interobserver reliability was reported for 20 of the 27 papers revie wed. Papers in which interobserver reliability was not reported included ? e earlier program evaluations (Bondy and Frost 1993, 1994, 1998; Liddle 2001; Webb 2000) and one single subject study (Adkins and Axelrod 2001). Reliability ranged from 80. 3 to 100% calculated on between 10 and 100% of data. Three studies (Howlin et al. 2007; Kravits et al. 2002; Malandraki and Okalidou 2007) estimated reliability on less than a minimum standard of 20% of total sessions. In contrast, procedural reliability was reported for only 7 of the 27 studies (Bock et al. 2005; Cummings and Williams 2000; Marckel et al. 2006; Tincani 2004; Tincani et al. 2006; Yoder and Stone 2006a, b) and discussed but not formally calculated in one other (Stoner et al. 2006). Where reported, procedural reliability ranged from 96 to 100%.In two papers (Yoder and Stone 2006a, b) less than 20% of sessions were used for the estimate. Social Validity Formal measures of social validity were reported in only four papers (Mag iati and Howlin 2003; Marckel et al. 2006; Tincani 2004; Yoder and Stone 2006a). Settings Fourteen studies were conducted in a special school, special preschool or special classroom setting. Remaining studies were conducted in a variety of settings including an integrated preschool, regular classroom, homes, clinics, day treatment facilities, and combinations of these settings. Ef? cacy and Effectiveness of PECS Of the total group of 394 individuals who received PECS intervention, only one child was reported as being nsuccessful at mastering at least phase I (Liddle 2001), and one adult had dif? culty with the motor and cognitive demands of the training and failed to progress past phase I (‘‘Mike’’, Stoner et al. 2006). ‘‘Carl’’, from Tincani’s (2004) study, was more successful with manual signs than PECS, but, the great majority successfully mastered at least some phases of PECS. Outcome data will now be considered further, initially focusing on pre-experimental designs, then single subject designs, quasi-experimental group designs and ? nally RCTs. This will be followed by a more detailed consideration of maintenance and generalization. Pre-Experimental Studies Several studies used pre-experimental designs.Bondy and Frost (1993) reported data on the implementation of PECS and found increased communicative initiations and use of pictures. Bondy and Frost (1994, 1998), Schwartz et al. (1998), Webb (2000), and Liddle (2001) also presented data on PECS implementation and reported increases in spoken language following PECS training. Schwartz et al. found that children were able to acquire communication with PECS training and there was evidence of generalization across pragmatic function. These studies, however, lacked adequate experimental control, and especially given the young age of the children involved in at least four studies, it is unknown how communication would have developed without the interve ntion.In their pilot study, Magiati and Howlin (2003) used a pre-treatment measure and three teacher ratings over time. They found signi? cant increases in PECS level (d = 2. 91),1 frequency of spontaneous use (d = 1. 75), and number of symbols used (d = 3. 01) over the 6 months following teacher training in PECS and its subsequent introduction. These are very large effect sizes by educational standards. They also found smaller but still statistically signi? cant increases in the number of signs (d = 0. 31), words (d = 0. 32) and phrases (d = 0. 30) used, and in the overall level of spontaneous communication (d = 0. 83). Outcomes were, however, measured mainly through teacher rating scales.The results must be treated with caution as they are likely to have been in? uenced by expectations and the research design was very weak. 1 For pre-test post-test designs, effect sizes were calculated by subtracting the pretest mean from the post-test mean and dividing by the pooled standard devi ation. For studies involving a comparison group, effect sizes were calculated by subtracting the mean of the control or alternate treatment group from the mean of the PECS intervention group and dividing by the pooled standard deviation. 123 1478 J Autism Dev Disord (2009) 39:1471–1486 Single Subject Studies PND and PEM statistics were calculated for the 10 single subject studies that provided baseline and intervention data.Initially, calculations were conducted on treatment data alone and then on all intervention data, including treatment, maintenance and generalization. When compared, the overall mean differences in favor of the treatment alone data were very small, only 0. 4% in the case of PND and 0. 8% for PEM. It was considered that the inclusion of all intervention data provided the best indicator of the ef? cacy of the overall package and these data were used for the remaining analysis. Results are provided in Table 3. Calculations were not possible for the four addit ional single subject studies (Adkins and Axelrod 2001; Cummings and Williams 2000; Ganz and Simpson 2004; Rehfeldt and Root 2005). These studies either lacked baseline data (e. g. alternating treatment design without baseline) or lacked baseline data that corresponded directly to that collected in intervention. The overall mean PND was 78. 5% (range 50– 100), placing the PECS intervention in the effective range (Scruggs and Mastropieri 1998). The overall mean PEM was 89. 1% (range 72. 3–100). Quality indicator scores are also presented in Table 3, and ranged from 30. 6 to 55. 7 out of a possible 65 points. Correlation between Quality Indicator scores for each study and their associated study PND was not signi? cant (rs = -0. 05, p = 0. 87). For PEM there was a trend toward weaker studies producing higher effect sizes but this did not reach signi? cance (rs = -0. 44, p = 0. 19).Mann–Whitney U tests or Kruskal–Wallis one-way ANOVAs were used to compare PND and PEM values across participant and study characteristics and these data are presented in Tables 4 and 5. No signi? cant difference in PND was found for age, gender, setting, inclusion of maintenance or generalization data, or research design. A signi? cant difference was found for PND scores for outcome variables, with studies addressing picture exchange only having a higher mean PND than those that included other dependent variables (i. e. , speech, social, behavioral, with or without picture exchange). A signi? cant difference was also found between PND values for participant diagnosis.Post hoc comparison showed that PND for children identi? ed with autism (i. e. , autistic disorder) were signi? cantly lower than for the other two groups, but these groups were not signi? cantly different from each other. No signi? cant difference was found between PEM values for any of the study or participant characteristics although participant diagnosis approached signi? cance (p = . 06). Fo ur of the single subject studies included data speci? cally relating to speech development from which PND and PEM values could be calculated (Charlop-Christy et al. 2002; Tincani 2004; Tincani et al. 2006; Yokoyama et al. 2006). The mean calculated PND was 49. % (range 19. 5– 100) and PEM 54. 2% (range 25. 0–100). These values are in the non-effective or at best very mildly effective range but with wide variation. Charlop-Christy et al. (2004) found increases in speech during PECS training. Tincani (2004) examined independent word vocalizations during PECS and sign language training. The addition of a Table 3 Single subject studies: PND and PEM results; study quality results Study PND PEM Study quality (Maximum 65) Picture Speech Social Behavior Overall Picture Speech Social Behavior Overall exchange exchange Adkins and Axelrod (2001) Bock et al. (2005) Chambers and Rehfeldt (2003) Charlop-Christy et al. 2002) Cummings and Williams (2000) Frea et al. (2001) Ganz and Si mpson (2004) Kravits et al. (2002) Marckel et al. (2006) Rehfeldt and Root (2005) Stoner et al. (2006) Tincani (2004) Tincani et al. (2006) Yokoyama et al. (2006) Mean SD – 92. 1 100. 0 – – 100. 0 – 87. 7 97. 3 – 77. 5 90. 6 98. 6 68. 7 90. 0 10. 9 – – – 59. 8 – – – – – – – 100. 0 20. 0 19. 5 49. 8 38. 4 – – – 86. 8 – – – – – – – – – – 86. 8 n/a – – – 26. 0 – 0 – – – – – – – – 13. 0 18. 4 – 92. 1 100. 0 55. 6 – 50. 0 – 87. 7 97. 3 – 77. 5 95. 3 70. 5 58. 6 78. 5 18. 8 – 92. 1 100. 0 – – 100. 0 – 87. 7 100. 0 – 90. 1 90. 6 98. 6 89. 9 94. 3 5. 2 – – – 65. 7 – – – – – – – 100. 0 25. 0 26. 0 54. 2 36. 0 – – – 95. – – – – – – – – – – 95. 6 n/a – – – 85. 0 – 100. 0 – – – – – – – – 92. 5 10. 6 90. 1 95. 3 72. 3 76. 7 89. 1 10. 6 – 92. 1 100. 0 76. 3 – 100. 0 – 87. 7 100. 0 30. 6 55. 7 43. 8 52. 4 32. 9 42. 4 35. 3 50. 4 49. 6 43. 8 50. 3 48. 2 45. 7 50. 3 45. 1 7. 6 123 J Autism Dev Disord (2009) 39:1471–1486 Table 4 Means, standard deviations and Mann–Whitney U test results for PND and PEM scores of study and participant characteristics Variable N PND M (SD) Quality indicators C50 50 PECS only Includes other Yes No 5 74. 3 (16. 6) 5 82. 6 (21. 7) 5 90. 9 (8. 9) 5 66. 0 (18. 0) 3. 0 0. 94 84. 6 (7. 5) 93. 5 (12. 0) 3. 0 1. 98* 94. 0 (5. 7) 84. 1 (12. 6) 7. 0 1. 14 5. 0 1. 6 U z PEM M (SD) U z Research design Multiple baseline Alternating treatments ABAB A ge Under 5 5–8 2. 0 1. 56 76. 5 (0. 3) 92. 2 (9. 4) 4. 0 1. 04 87. 9 (11. 2) 93. 9 (8. 7) 6. 0 1. 27 89. 9 (12. 2) 11. 0 0. 21 88. 5 (10. 5) 88. 5 (12. 5) 52. 0 0. 46 92. 5 (11. 2) 6. 0 0. 52 2. 0 1. 56 9–17 18? Diagnosis Autism PDD-NOS/autistic characteristics Other Setting Special school/ preschool Clinic Integrated preschool Home Combination 10 1 1 9 8 90. 1 (12. 5) 3. 87 79. 3 (n/a) 50. 0 (n/a) 74. 3 (30. 4) 82. 3 (21. 8) 2 57. 1 (2. 1) 8 83. 8 (17. 0) 9 8 3 9 13 85. 0 (17. 0) 2. 58 73. 1 (31. 2) 72. 2 (21. 4) 87. 9 (18. 5) 69. 8 (25. 9) 7. 68* 93. 8 (7. 3) 84. 4 (11. 4) 75. 4 (18. 8) 93. 5 (11. 1) 6 3 1 70. 0 (19. 0) 3. 82 95. 8 (4. 0) 77. 5 (n/a) 479 Table 5 Descriptive statistics and Kruskal–Wallis One-Way ANOVA results of PND and PEM scores of study and participant characteristics Variable N PND M (SD) H PEM M (SD) H 85. 5 (12. 3) 1. 62 95. 8 (4. 0) 90. 1 (n/a) 6. 74 Outcome variables Maintenance data included Generalization data included Yes 8 80. 9 (17 . 8) No Yes No Gender Male Female 2 68. 9 (26. 7) 4 88. 8 (12. 4) 6 71. 6 (20. 0) Procedural reliability data 83. 7 (13. 2) 5. 59 100. 0 (0) 92. 8 (9. 8) 90. 4 (11. 8) 2. 29 79. 3 (n/a) 100. 0 (n/a) 87. 8 (13. 1) 89. 2 (13. 4) 2 100. 0 (0) 14 89. 2 (15. 2) 25 78. 9 (23. 9) 40. 5 1. 12 5 92. 5 (11. 2) Note: * Indicates signi? cant result at 0. 05 level for two-tailed test reinforcer delay in phase IIIb resulted in increased in word vocalizations. Tincani et al. 2006) examined word vocalizations and vocal approximations during PECS training, and found a decrease during phases I-III before dramatic increases in phase IV. In a second experiment, looking at phase IV only, a higher percentage of word vocalizations was found with the reinforcement delay procedure than without. Yokoyama et al. (2006) examined frequency and intelligibility of vocalization during PECS training in phases I-IV; these authors also found an increase with the time delay procedure. Several other studies provided da ta on speech development, which was not suitable for calculation of PND or PEM values. Kravits et al. (2002) found an increase in frequency of intelligible speech but not in range of spoken vocabulary.Ganz and Simpson (2004) found that words per trial increased noticeably during phase IV or phases III and IV of PECS training, in particular, simultaneously with delayed word modeling. Charlop-Christy et al. (2002) provided the only appropriate data for calculation of PND and PEM values for social outcomes. From this very small amount of data, the PND of 86. 8% and PEM of 95. 6% suggest an effective or highly effective intervention. Variables that increased in this study were eye contact, joint attention, cooperative play, and frequency of initiations and requests including but not limited to PECS requests. Initiations and requests Note: * Indicates signi? cant result at 0. 05 level for two-tailed test ncreased the most, and joint attention also increased in all three children. It has been suggested that a direct positive relationship exists between joint attention and communication in children with autism, with improvement in one potentially stimulating an increase in the other (CharlopChristy et al. 2002). Kravits and colleagues (2002) reported some increase in duration of social interaction with peers although these data were not suitable for calculation of PND or PEM as only the mean level in each phase was presented. PND and PEM scores were calculated for data from only two studies for behavioral variables (CharlopChristy et al. 2002; Frea et al. 2001). The mean PND was 13. 0% while the mean PEM was 92. %, but, examination of graphed data showed treatment effects, indicating that decreased problem behaviors occurred in conjunction with increased communication skills through PECS training. Two studies compared sign language to PECS interventions (Chambers and Rehfeldt 2003; Tincani 2004) and one compared a VOCA to PECS (Bock et al. 2005). For each of these st udies PND and PEM were equal, and a higher value was found for PECS than for the alternative intervention. For Tincani (2004) calculated values were 95. 3% for PECS and 92. 3% for sign, for Chambers and 123 1480 J Autism Dev Disord (2009) 39:1471–1486 Rehfeldt (2003) values were 100% for PECS, and 65. 7% for sign, and for Bock et al. 2005) values were 92. 1% for PECS and 79. 7% for VOCA. Quasi-Experimental Group Studies Carr and Felce (2007b) found signi? cant improvement in several aspects of communicative interaction between children and staff following 15 h of PECS training (Phases I-III). Signi? cant increases were found for total child-toadult initiations, linguistic initiations, the percentage of adult response, the percentage of child response, and signi? cant decrease in adult-to-child interactions with no opportunity for child response. These differences were found in comparison to both a pre-intervention measure over time and to a non-intervention and non-equivalent control group.Examining a subset of this group, who used at least one word during observations, Carr and Felce (2007a) reported that over 6 weeks training in PECS phases I-III, 3 of the 24 children in the PECS group increased their spoken words. A further 2 who did not speak at pretesting did so at post-testing, while there was a marginal increase in speech for one child in the control group. RCTs Only three RCT studies were located. Yoder and Stone (2006a) conducted an experimental study of 36 children with autism, aged 21–54 months, who were randomly assigned to PECS or RPMT intervention groups. They found that the PECS group showed a signi? cantly greater increase in frequency of speech (d = 0. 3) and in number of different words used (d = 0. 50) after 6 months of intervention, but by 6 months post-intervention the difference was no longer evident. Interestingly, they also found differing effects according to pretreatment characteristics: children who were low in initial object exploration bene? ted more from the RPMT intervention, while those who were higher bene? ted more from PECS, these effects being evident 6 months post-intervention. Overall, there was a signi? cant increase in non-imitative spoken acts over 1 year. The actual increases were from a mean of 0. 25 nonimitative spoken acts in a 15-min session to a mean of 5. 5, and from a mean of 0. 7 different non-imitative words to a mean of 3. Given the young age of the children, the fact that their initial verbal mental age averaged 11. 9 months (range 7–19 months), just at the stage when verbal language is likely to develop naturally, it seems quite possible that this increase could be attributed to maturation. In a second article, Yoder and Stone (2006b) examined the effect of the interventions on the three major types of intentional communication used prior to speech development, (i. e. initiating joint attention, requesting, and turn-taking). They found that, overall all three comm unicative functions increased signi? cantly, but RPMT increased turn-taking more than PECS.Children who were higher in initiating joint attention before treatment had greater increases in both initiating joint attention and requesting following RPMT intervention, while those who were initially lower in initiating joint attention had greater increases following PECS intervention. Howlin et al. (2007) conducted a group RCT of 84 children with autism, examining the effect of teacher training and consultancy in PECS. It should be stressed that this study examined the effectiveness of a consultancy model to deliver PECS, rather than the ef? cacy of PECS per se. Thus, the study was noteworthy in that it appears to be the only research to examine effectiveness (i. e. , outcomes under clinical rather than optimal conditions). Howlin et al. ound that rates of communicative initiations and PECS usage were signi? cantly increased immediately following intervention, but that these effects were not maintained once the intervention ceased. They found no signi? cant increase in frequency of speech. Howlin et al. also examined ADOS-G (Lord et al. 2000) domain scores for communication and reciprocal social interaction. They found no increase in most ADOS-G ratings, with the exception of a decrease in the severity score for the Reciprocal Social Interaction domain at the 10 month followup. Unfortunately, no data was provided on the ? delity of implementation of the PECS program, or indeed on the ? delity of the teacher training.Maintenance and Generalization Only ? ve studies provided data on maintenance. Two of the RCT studies included long-term follow-up. Yoder and Stone (2006a) found that differences in speech variables were not maintained 6 months post-intervention, while Howlin et al. (2007) found that for the 26 children assessed at a 10-month follow-up, the increased rate of communicative initiations and PECS usage found immediately post-intervention was not maintained. Two single subject studies and one case study measured maintenance of skills 6–10 months postintervention (Charlop-Christy et al. 2002; Malandraki and Okalidou 2007; Yokoyama et al. 2006). Charlop-Christy et al. ound that speech and socio-communicative behaviors had been maintained or continued to increase for one participant followed up 10 months post-training. Yokoyama et al. found maintenance of PECS skills both in the training room and at home, 6–8 months after training for the three participants in their study. Malandraki and Okalidou in their study of one child found maintenance of skills 6 months after the main intervention. While the difference was not signi? cant and the number of studies was low, for the single 123 J Autism Dev Disord (2009) 39:1471–1486 1481 subject studies both PND and PEM were lower for studies that included maintenance data (Table 4).Fifteen of the 27 studies included some data on generalization of PECS skills. The great majority o f these were positive, with skills generalizing to different settings, people and stimuli. For some studies, generalization was an integral part of the way data were collected (CharlopChristy et al. 2002; Yoder and Stone 2006a, b). For others, generalization to untrained situations was speci? cally probed (e. g. , participants in Stoner et al. ’s (2006) study generalized their skills to use in fast food restaurants). Several studies demonstrated generalization to the classroom teacher or to home. In a small number of instances, generalization was unimpressive or absent.For example, in the Adkins and Axelrod (2001) study, tests for ‘‘generalization’’ simply required the child to mand for an object without immediately prior prompted trials. ‘‘Carl’’, from Tincani’s (2004) study, failed to generalize PECS skills to classroom teachers, preferring to use sign language. Discussion The PECS program was originally designed to provide a method of communication for children with autism, particularly those who do not use functional speech. PECS appears to be a popular intervention (Howlin et al. 2007) but, unfortunately, popularity of a given treatment does not necessarily re? ect actual ef? cacy (Green et al. 2006; Reynhout and Carter 2006). Only three RCTs have been reported to date. The studies of Yoder and Stone (2006a, b) compared PECS to RMPT.PECS was superior for some children but the study was designed to compare two treatments and, consequently, did not include a control arm. Thus, no conclusions can be drawn about the relative superiority of either intervention to a non-treatment control. Howlin et al. (2007) provided the only effectiveness study conducted. They found signi? cant effects on communicative initiations but this was not maintained once the intervention ceased. Thus, further examination of approaches to the delivery of PECS in clinical settings is needed. Con? dence in the Howlin et a l. study is somewhat weakened by the lack of any data on treatment ? delity, which is a critical feature in study quality (Gersten et al. 2005).The nature and quantity of data arising from RCTs at this point in time is insuf? cient to draw ? rm conclusions regarding the PECS interventions. Thus, probably the highest priority for research in this area is the conduct of further RCTs examining both ef? cacy and effectiveness in applied settings. In the absence of an adequate body of RCTs, clinicians still need to make informed decisions regarding interventions and may need to look to the second line of evidence. Evidence supporting the PECS intervention was provided by the well-designed quasi-experimental studies of Carr and Felce (2007a, b), which incorporated a non-equivalent control group with demonstration of pre-test equivalence between groups.Arguably, the bulk of interpretable data on PECS comes from single subject studies. For the relevant studies, the overall, mean PND (78. 5% ) and PEM (89. 1%) ? gures support the preliminary conclusion that PECS may be an effective intervention, at least when implemented under research conditions. There was a signi? cant difference between the PND results for studies that only looked at picture exchange outcome variables and those that included other collateral variables, such as speech, social, or challenging behavior. This indicates that, unsurprisingly, PECS training appears to be most effective in providing a successful means of communication through picture exchange.Nevertheless, it should be acknowledged that the number of studies remains relatively low and single subject designs have several limitations, including low external validity. While these studies contribute to our knowledge and give us a preliminary indication of the ef? cacy of PECS, they are not a substitute for well-conducted large scale RCTs. A substantial number of the extant studies were preexperimental in nature, particularly the early research. As such, they are not able to provide convincing demonstrations of experimental control. Hence, these studies offer no interpretable evidence on the ef? cacy of PECS. The effect of PECS training on speech development remains unclear.Research into various forms of AAC suggests they may have the potential to enhance speech development (Cress and Marvin 2003; Millar et al. 2006; Romski and Sevcik 2005) although results have sometimes been inconsistent (Carter 1999; Millar et al. 2006). Several of the studies reviewed in this paper reported increases in speech following PECS training, but others, including Howlin et al. (2007), reported little or no effect. Where speech increased, this has often occurred concurrently with phase III or IV of PECS, and in particular when a time delay was introduced. A related question, for which there is as yet no empirical evidence, is whether PECS training affects comprehension.Brady (2000) found increased comprehension skills with the use of VOCAs and it would be worth investigating whether PECS would have a similar effect. In comparison with other AAC systems, better overall results were obtained with PECS in the studies reviewed here (Adkins and Axelrod 2001; Bock et al. 2005; Chambers and Rehfeldt 2003; Tincani 2004). Nevertheless, there was variability in the results depending on initial imitation skills and, possibly, participant preference. It has been argued that individuals with ASD may bene? t from visually cued instruction (Quill 1997) and further examination of this issue would seem warranted. In addition, existing 123 1482 J Autism Dev Disord (2009) 39:1471–1486 application of PECS appears to have been exclusively limited to graphic symbols.There are distinct advantages to the use of three-dimensional tangible symbols, including decreased cognitive load and high iconicity (Rowland and Schweigert 1989, 1990; Turnell and Carter 1994). The exploration of the use of PECS with tangible symbols, especially with indiv iduals who are low functioning, would seem warranted. Only 5 of the 27 studies provided data on maintenance of PECS skills or other dependent variables. It is worthy of note that maintenance was problematic in both RCTs (Howlin et al. 2007; Yoder and Stone 2006a) that examined the issue. Overall, available evidence is mixed but there is certainly suf? cient doubt to indicate that maintenance should be formally and systematically monitored in the clinical application of PECS programs.Fifteen studies provided data on generalization of skills, the vast majority of these found that generalization did occur, but what was described as ‘‘generalization’’ varied greatly. There were several methodological limitations and issues in the research examined that warrant comment. In general, participant descriptions were poor, making it dif? cult to assess whether the intervention is best suited to individuals with particular characteristics. While nearly all studies prov ided a diagnosis, few speci? ed the diagnostic protocol or criteria. Further, when participants were diagnosed with ASD, few researches attempted to quantify the degree of autism.Noting the range of behaviors and symptom severity possible within individuals presenting with autistic disorder, and even greater variation in the broader autism spectrum, this would seem to be relevant, if not critical, information. Few researchers provided standardized assessment data or a detailed functional description of general ability, but probably re? ecting the aim of the intervention, most did provide some description of initial communication skills. While the number of studies was clearly insuf? cient to reach ? rm conclusions, PND data suggest that individuals with PDD-NOS or showing autistic traits made more progress with PECS than those with autistic disorder.It is unclear whether this is because the PECS protocol is better suited to them, or because they would do better with any treatment. W ithout further clear and consistent quanti? cation of the degree of autistic symptomatology, it is impossible to evaluate further this variable in relation to the ef? cacy of the PECS intervention. In addition, there was insuf? cient data on intellectual functioning to enable analysis of any relationship to PND or PEM. It is recommended that, in future studies, standardized psychometric data, standardized functional assessment of adaptive behavior, and clear information on initial communicative abilities should be provided.In addition, where a diagnosis of autism is provided the level of autistic symptomatology should be quanti? ed. Procedural reliability data were very limited, with data only meeting the conventional minimum standard in 5 of the 27 studies. Because of the absence of this data, it is not possible to determine in many cases whether what was being implemented was in fact the PECS program as designed. PECS is a complex and multi-component intervention making the veri? cation of treatment integrity even more critical. The absence of such information in research studies is somewhat dif? cult to understand given that the PECS manual (Frost and Bondy 2002) gives explicit and speci? criteria for assessing the integrity of training during each phase. Nevertheless, research on PECS is not alone in this regard and lack of procedural reliability data has been reported as a problem in other recent intervention reviews in the area of autism (e. g. , Bellini and Akullian 2007; Reynhout and Carter 2006). The calculated PND (88. 8%) and PEM (89. 9%) ? gures for studies that did meet the standard for reporting procedural reliability, are at the high end of the effective intervention range (Ma 2006; Scruggs and Mastropieri 1998), suggesting that monitoring of procedural integrity should be a key feature in research as well as clinical applications of PECS. A signi? ant component of the present review was the application of PND and PEM metrics to the relevant dat a. PND and PEM values were in most cases very similar, with lower variability for PEM (see Table 4). An exception was found for data relating to behavioral variables (CharlopChristy et al. 2002; Frea et al. 2001), where high baseline variability and ‘‘? oor’’ effects occurred, often causing the calculated PND for affected graphs to be low, while the PEM was high. For example, visual inspection of the graphed data in Frea et al. (2001) shows a clear treatment effect of the PECS intervention on disruptive behavior. Nevertheless, the calculated PND for these data was 0%, while the PEM was 100%.The discrepancy between the clear treatment effect seen in the graphed data and the PND value indicates that PEM may be a more appropriate metric for challenging behavior, where variability is likely to be high. Further, the advantages of using multiple methods of calculating effect sizes for single subject research are highlighted. As previously noted, PECS is a complex multi-component intervention program. Consequently, the question arises as to which of the components are most critical to its ef? cacy. For example, reinforcer assessment is formally and systematically incorporated into PECS and this may well be a salient factor in program ef? cacy. The use of picture exchange with a partner (rather than touching or pointing to a symbol) is a key distinguishing feature of PECS, but it is unclear whether exchange per se is essential to ef? cacy.The issue of developing spontaneity is addressed in an unusually systematic way in the PECS program (Chiang and Carter 2008) but extant research 123 J Autism Dev Disord (2009) 39:1471–1486 1483 provides only limited information on the circumstances under which communication occurs. Thus, there would appear to be considerable scope for examination of how speci? c components contribute to the overall ef? cacy of PECS. In addition, there has been only limited comparison of PECS to alternative interventio ns and this stands as a priority for future research. Several limitations of the current review must be acknowledged. Many of the earlier studies were descriptive and clear experimental control was not established.While later studies were of higher quality, only a limited number of RCTs have been conducted and much of the available interpretable data comes from second line of evidence single subject studies. Analysis of PND and PEM was only possible for a subset of the relevant single subject studies examined and analysis of speci? c study and participant characteristics were based on low numbers. In addition, very few studies provided adequate procedural reliability data so the extent to which PECS was appropriately implemented often remained unknown. Conclusion On balance, the studies reviewed provide preliminary evidence that PECS may be ef? cacious for children and adults with ASD and other developmental disabilities, who have little or no speech. Primary bene? s appear to be ev ident in communication by picture exchange. Identi? cation of the core aspects of the program that are important to its success, the individuals to whom it is best suited, and its relationship to other interventions remain to be substantively investigated. PECS stands as a promising intervention with some empirical support but many questions remain. The conduct of further RCTs into the ef? cacy and effectiveness of PECS stands as a high research priority. Appendix See Table 6. Table 6 Quality criteria for single subject research adapted from Horner et al. (2005) Area Indicator Description of participants Participants are described with suf? ient detail to allow others to select individuals with similar characteristics and settings (e. g. , age, gender, disability, diagnosis). One point awarded for each of the following (maximum of 5): 1. Statement of diagnosis such as autism, ASD, Asperger syndrome, intellectual disability (with or without indicating diagnostic source), age and gend er 2. Diagnostic instrument speci? ed (e. g. , WISC, AAMR diagnostic criteria, DSM-IV criteria, ADOS). Must provide if ASD or 0 awarded 3. If ASD, degree of autism speci? ed either with reference to symptoms (DSM-IV) or instrument like CARS. If not ASD, award point 4. Standardized assessment data (e. g. IQ, developmental scale, adaptive behavior) OR detailed functional description of general ability. Disability range (e. g. , moderate) acceptable for intellectual disability 5. Communication skills documented by means of standardized test results OR description of functional skills The process for selecting participants is described with replicable precision. MUST describe the process used to select participants, not just describe the participants or their needs. This would generally include the criteria the participants must meet (e. g. , 3–5 years, less than 5 spoken words, diagnosis of autistic disorder) and or the process of selecting participants (e. g. , the ? rst 5 chil dren on the waiting list).Essentially, authors must explicitly state HOW/WHY participants were selected Critical features of the physical setting are described with suf? cient precision to allow replication Dependent variables All dependent variables are described with operational precision Each dependent variable is measured with a procedure that generates a quanti? able index The measurement process is described with replicable precision Dependent variables are measured repeatedly over time Data are collected on the reliability or inter-observer agreement (IOA) associated with each dependent variable, and IOA levels meet minimal standards (i. e. , IOA = 80%; Kappa = 0. 60).Must be on minimum of 20% of sessions to be acceptable Independent variables Independent variable is described with replicable precision Independent variable is systematically manipulated and under the control of the experimenter Overt measurement of the ? delity of implementation for the independent variable. M UST be measured on a minimum of 20% of sessions to be acceptable 123 1484 Table 6 continued Area Baseline Indicator J Autism Dev Disord (2009) 39:1471–1486 A baseline phase provides repeated measurement of a dependent variable and establishes a pattern of responding that can be used to predict the pattern of future performance if introduction or manipulation of the independent variable did not occur. Should include a minimum of 3 stable data points.High variability is acceptable if intervention effects are unambiguous The procedural characteristics of the baseline conditions should be described operationally Experimental control/internal The design provides at least three demonstrations of experimental effect at different points in time. Effects of validity alternating treatments may be added, as main comparison is not with baseline. AB designs may not be added as they do not demonstrate intervention at different times when comparing to baseline The design controls for common threats to internal validity (e. g. , permits elimination of rival hypotheses). Acceptable designs include multiple baseline, ABAB and alternating treatment with counterbalancing.Unacceptable designs include: AB, ABA, and changing criterion External validity Social validity Experimental effects are replicated across participants, settings, or materials to establish external validity. At least three participants, settings or materials must be apparent The dependent variable is socially important Implementation of the independent variable is practical and cost effective (must be measured) Social validity is enhanced by implementation of the independent variable over extended time periods, by typical intervention agents, in typical ph