Identification of studies

Papers were eligible for inclusion in the review if (a) participants were diagnosed on the autism spectrum using established diagnostic criteria (ICD-11, DSM-V or previous versions); (b) they were aged under 18 years; (c) the study included an intervention based on any robotic platform; (d) data were reported for at least one outcome against which to measure intervention gains. We only included primary studies and all designs were considered (e.g., randomised, case controlled, case reports). Lateral search techniques were also used to identify additional papers for inclusion in the review.

Criteria for exclusion in the review were the following: (1) lack of separate presentation of study outcomes for children on the autism spectrum; (2) individual aged over the age of 19; (3) lack of recording study procedures including number, duration and frequency of robot-mediated sessions; (4) no reference to the robot type/model; (5) commentary papers, protocols, surveys and reviews; (6) qualitative studies; (7) qualitative elements in mixed-method designs; (8) studies that were not published in English.

We searched the following databases: PubMed, Scopus, EBSCOhost, Google Scholar, Cochrane Library, ACM Digital Library, and IEEE Xplore. Grey literature was also searched using PsycExtra, OpenGrey, British Library EThOS, and the British Library Catalogue.

Search terms aligned to the following core domains: ‘autism’ AND ‘robot’ AND outcome-specific terms aligned to the concepts of social, emotions, communication, education, academic attainment, behaviour, or health (S2 Table). Databases were searched from inception until April (6th) 2021. Additional studies were identified through a manual search of the references in relevant studies.

Study selection

Identified records were exported into Mendeley (v1.19.8) and duplicates removed. The first author (AK) screened all the titles and abstracts, and a second reviewer (AB) independently screened a random sample of 20% of the originally identified records, both using pre-determined inclusion criteria, to establish reliability for study selection. The full texts of potentially relevant papers were retrieved and independently assessed for eligibility by the first author (AK). Twenty percent of full text of the eligible studies were independently screened for eligibility by the second reviewer (AB). There was 100% agreement between reviewers in the selection of studies which met the inclusion/exclusion criteria.

Data extraction

A pre-piloted form was used to extract data, including the following items: authors and year of publication, the study aims, design, methods, and a summary of the outcomes. We extracted characteristics of study participants such as mean age of all study participants, gender distribution, diagnostic tool, and intelligent quotient (IQ), where available. Sample ethnicity was also recorded, where available. The socioeconomic status was not recorded in any of the included studies and so these data could not be extracted.

Characteristics of the interventions were extracted across the articles and included: robot type, duration of intervention, frequency and length of sessions, location where intervention was delivered, and outcome measured. Other intervention characteristics such as the type of robot used, and intervention location were also extracted. Authors of 10 separate studies were contacted to extract information which was not clearly reported in published work (e.g., confirm diagnosis, session location, number and/or duration of a session, moderator of session delivery, mean and standard deviation per participant). Three authors shared information about their studies. Findings were summarized narratively, and where relevant, using means, standard deviations, and percentages.

Meta-analysis

Meta-analyses were conducted using Comprehensive Meta-Analysis Version 3.0 for Windows [34]. We calculated Hedge’s g effect sizes (and 95% confidence intervals) for end-of-trial data comparing robot-mediated interventions and control groups in RCTs. Hedge’s g adjusts effect sizes according to sample size. Comparisons were made for intervention and control group at end-of-trial on the primary outcomes of: social, emotional, and motor benefits (which emerged as the main clusters of outcomes in RCTs). All meta-analyses used a random effects approach. We classified effect sizes as small (0.2) medium (0.5) and large (0.8) according to Cohen’s nomenclature. Heterogeneity was assessed using the I2 statistic, and for interpretation we followed Cochrane guidance (Higgins et al., 2019) where I2 values were identified: 0%-40% as might not be important; 30%-60% as may represent moderate heterogeneity; 50%-90% may represent substantial heterogeneity; 75%-100% representing considerable heterogeneity.

Selection and inclusion of studies

The search generated 2145 records. After removing duplicates, 1646 records were screened, with 151 deemed relevant and full texts reviewed, of which 44 articles reporting 40 studies were deemed eligible for inclusion into the systematic review. The most common reason for exclusion was the lack of information concerning diagnostic method for autism (n = 57) followed by studies which did not meet the inclusion criteria such as reviews, protocols, surveys, feasibility trials, opinion letters (n = 23). A smaller number of studies were excluded because of the following reasons: 1. a new robotic platform/ intervention was developed (n = 9); 2. adults/ children with diagnoses other than autism were examined (n = 7); 3. the robot name/type was missing (n = 4); 4. a qualitative study had been conducted (n = 2) (Fig 1).

Open in a separate window

Fig 1

PRISMA flow diagram of the study selection process.

Characteristics of included studies

The description of the study characteristics is based on 40 studies. Four articles [21, 36, 37] had overlapping samples and were not included in the average sample size. The majority of the studies were non-randomized (n = 23, 57%), followed by RCTs (n = 17, 43%) (Table 1). Thirty-four studies utilised video data to analyse the study findings. The average sample size of children and young people on the autism spectrum across the studies was 10 (range 1–30), (M = 10.22, SD = 6.58, k = 40).

Most studies recruited more males than females, with percentage of males ranging from 67% to 100%. Ethnicity of participants was reported in nine studies (22%), of which five studies included Chinese participants [38–42], three comprised Caucasian, African American, Asian Hispanic, Mixed African American and Caucasian, Mixed Caucasian and Hispanic and Latino participants [20, 43, 44] and one study included Italian children [45].

Comparing RCT and non-RCT designs, the former had more complete reporting about the autism diagnosis process (for example, from which type of healthcare professional). Only sixteen studies (40%) measured the cognitive capacity (IQ) of children. Further characterisation of participants (e.g., school type, parent’s demographics) was generally poor. Child/adolescent participants ranged in age from 2 years to 16 years (M = 7.4; SD = 3.08). Overall, the studies were published between 2008 and 2020, in Europe (k = 18, 41%) [i.e., Romania (k = 7), Portugal (k = 3), France (k = 2), Italy (k = 2), Netherlands (k = 2), Belgium (k = 1) and Luxembourg (k = 1)] followed by the United States of America (k = 11, 27%). Some studies were based in East Asia (k = 11, 25%) including in Hong Kong (k = 7), Korea (k = 1) and Japan (k = 3). Only one study was conducted in Canada. Most studies (k = 33, 82%) had received funding to conduct their work.

Quality assessment and risk of bias in included studies

The quality assessments revealed that few studies were rated as strong (k = 7), with most rates as moderate (k = 16) or weak (k = 17) (S2 and S3 Tables). Most studies did not adequately report on participant selection, resulting in three-quarters being rated as moderate in selection bias (n = 30, 75%). This trend was comparable across both randomised and non-randomised studies. The poor quality of many studies can also be partly attributed to the confounding variable rating. Reporting on ethnicity, age, family or socioeconomic status of families was often poorly described with limited matching again across most study groups (n = 29, 72%); and again, this was as much a feature of RCTs as non-RCTs. Effort was made to contact authors for further information, and five author responses were received, which resulted in information being classified as missing and thus reducing study quality. Finally, outcome assessors (e.g., researchers) were not blinded in (n = 38, 95%) of the studies, including often in RCTs as well as non-RCTs. Promisingly, most studies (n = 36, 90%) used appropriate methods to collect data including video data. Studies provided details about the position of the cameras and the use of at least two coders including inter-rater reliability between coders. Most studies (n = 28, 70%) also reported the number of participants approached, screened, and completed the intervention.

Robot types

Four robot types were used that can be characterised according to their appearance in the following categories: humanoid, animaloid, and other. A humanoid robot is distinguished by its resemblance to the human body. In general, a humanoid robot has a head, torso, two arms and legs. Some humanoid robots may have facial characteristics including eyes, nose and mouth whereas other humanoids may model part of the body from the waist up. Humanoid robots were used in 67% (27 out of 40) of the included studies. The robot platforms that facilitated a session with children and young people on the autism spectrum were the following: NAO, QTrobot, CommU, ACTROID-F, Isobot, Tito, iRobiQ, Caro, Keepon and HUMANE. The most frequently used robot was NAO which was used in 17 studies [20, 38–42, 44–54]. Studies used humanoid robots to examine a range of skills including eye-contact, imitation, joint attention, social skills and emotional regulation.

The use of animaloid (or animal-like) robots, such as an elephant, chicken, dinosaur and dogs, was examined in 11 out of 40 studies (27%). In the review, the Probo robot (elephant-like) was referenced in six studies [55–60]. Other animaloid robots were the robot, Pleo, [36], the dog robot Zoomer, [17, 18, 61] and POL (chicken-like) [62]. These robots facilitated sessions focusing on eye contact, imitation, joint attention and social skills.

The remaining ‘other’ robot category included a robotic arm and a plant robot. The robotic arm was used with children and young people on the autism spectrum to examine imitation and eye-contact [63]. The plant robot, called ‘Pekoppa’, was fully programmable with integrated sensors that allowed the robot to model a range of functions. Pekoppa was used with neurotypical children and young people and children and young people on the autism spectrum to compare the differences in heart rate, verbal fluency, and emotional response [64].

When exploring trends across study type, it appeared that humanoid robots were used in both RCTs (n = 12) and non-RCTs (n = 14). In particular, the robot NAO was utilised in 10 RCTs compared to six non-RCTs (Table 1). Non-RCTs were therefore more likely to include a broader range of robot platforms.

Settings

Intervention sessions with robots took place in five different settings. The most common location was autism centres/clinic rooms (n = 15) followed by home (n = 7), school (n = 7) and laboratories (n = 7). RCTs showed a trend to be more likely to take place in autism centres/clinics (n = 7 out of 17; 41% versus n = 8 or in a familiar environment such as school (n = 6 out of 17; 35% versus n = 1 out of 23; 4%). Sessions at home were more common in non-RCTs (n = 5; 22%) compared to RCTs (n = 2; 12%). Similarly, sessions in a laboratory were more common in non-RCTs (n = 6; 26%) than RCTs (n = 1; 5%) (S5 Table).

Sixty-five percent (n = 11) of RCTs and 78% (n = 18) of non-RCTs indicated a positive benefit of intervention. The mean duration of robot-intervention was 8.45 sessions and each session lasting an average of 33.5 minutes—most likely occurring once (22%) or twice weekly (28%) over the intervention period, though there was considerable variability (Table 2). The first session was usually a familiarisation meeting with the child and the robot/play partner. Intervention sessions tended to be longer in RCTs, with a mean of 35 (range: 6–120) minutes versus 27 (range: 3–180) minutes in non-RCTs. Similarly, the number of sessions was greater in RCTs with a mean of 9 sessions (range: 1–32) compared to 7 (range: 1–30) in non-RCTs. Notably, play partners across studies were more often ‘professionals’ for example, researchers or healthcare workforce (90%). Children and young people on the autism spectrum had individual sessions apart from in three studies [49, 50, 65].

Robot’s role in intervention

During intervention, robots took on the role of a social interface. Hence the robot moved its head and eyes to express emotions via facial expressions (e.g., happy, sad) or verbally, became a storyteller, an imitation agent, an intermediate to attract the eye gaze of the child on the autism spectrum or facilitated collaboration within a small group of two children and young people or an object where the child on the autism spectrum engaged in free play. In most studies (n = 38, 95%), children and young people engaged in a triadic relationship with an adult/therapist where the robot acted as a mediator. A typical session involved the play partner controlling the robot via a laptop/computer. Two studies used a fully autonomous robot to play independently without the guidance of an adult partner. The control group in RCTs was often a human therapist engaging the child with the same or similar activities apart from five studies that used a waitlist and so, received the robot session after study completion. It was unclear whether the children in waitlist studies were receiving no treatment interventions as part of their educational and community settings.

Targeted skills and outcomes

Studies included in this review targeted a number of skills that can be clustered into 3 main categories: (1) social and communication skills [narrative skills (n = 1), self-initiated questions (n = 4), engagement (n = 2), self-disclosure (n = 1), collaborative play (n = 1), level of prompting (n = 3), joint attention (n = 6), eye-contact (n = 6), imitation (n = 7)]; (2) emotional development [recognition and/or understanding (n = 2), emotional regulation (n = 4)]; and (3) motor skills [stereotyped or repetitive behaviour (n = 1)]. Over two-thirds (n = 29; 72%) of the articles reported a positive impact of a robot-mediated intervention in children and young people on the autism spectrum. Less than a quarter of the included articles (n = 10, 25%) reported no difference in targeted skills development. Finally, one article reported a decline in attention skills during the intervention period [66].

The majority of the targeted skills were measured through the examination of video recordings and coding procedures completed by researchers whereas only two studies [45, 50] used standardised assessment tools to examine social skills and emotional comprehension (Table 1). Nine studies [20, 38–42, 45, 55, 67] used blinded researchers to administer the questionnaires and one study [50] relied on parent-reported outcomes. Another three studies [55, 57, 58] utilised child self-report methods, three used qualitative methods (e.g., audio recording, notes by researchers) [56, 68, 69] and two used eye-tracking [70] or a sensory integration and praxis test [71] in conjunction with video recordings. Two studies made explicit reference to the benefits of the robot-mediated intervention at two weeks [48] and four weeks [40] following the end of the intervention. Finally, Marino and colleagues [45] reported that children on the autism spectrum in both groups (robot and human) spontaneously practiced the trained skills addressing generalisation issues. No studies included evaluation of health economics related to intervention delivery.

Strengths and limitations

As far as we are aware, the current review is both the first to be preregistered and the first to meta-analyse some evidence on robot-mediated intervention for autistic children and young people. In doing so, we have provided novel insights absent from other reviews [7, 32]. Our search of the grey literature generated articles that are included in this review and so, offers a more balanced picture of the current literature, minimising potential for publication bias. Evidence from the meta-analysis did not point to any obvious publication bias amongst RCTs. Nonetheless, the number of RCTs that provided sufficient data for meta-analysis were small and the findings should be interpreted in that context. Nonetheless, we believe this new evidence will be helpful to researchers, trialists, clinicians, educators, and parents, especially as the field of technology-assisted learning in autism is seeing an expansion [78–81]. Other limitations might be the large number of articles (k = 102) we excluded owing to their poor reporting on autism diagnosis. Systematically excluding these studies was in line with our preregistered inclusion criteria but may have excluded data from the true population of interest. Studies might however benefit from using clear diagnostic criteria. Further, the evidence from meta-analysis should be considered with the caveat that we could derive data from only 12 RCTs, although we did manage to capture data from almost three quarters of the trials (12/17). We also note that our examination of moderator variables was limited to analyses that were pooled across all three outcomes (social, emotional and motor) in order to obtain sufficient data points. Despite such limitations, the findings for social outcomes look especially promising, while those relating to emotional and motor abilities require further studies. We were unable to report on adverse events as these data have been poorly reported in studies. Only four studies reported adverse events [49, 54, 67, 82]. Finally, as commented on earlier, the studies included in this review have reported on the outcome of brief exposure and its impact at best in the immediate post intervention or short follow-up period. As such, though robot mediated interventions appear to be promising, the extent to which skills are generalised into everyday life and sustained is unclear.