Italy

The evaluation study by Cremaschini (2015) is a controlled, non-randomised, quasi-experimental evaluation of this intervention package. The intervention group are workers of the companies having selected to implement a specific sub-programme (i.e. on tobacco). The control group are workers of the companies involved in the WHP programme, but without having selected that specific programme (i.e. companies having selected a sub-programme for alcohol and other substances, physical activity etc). Exclusion criteria: companies with a low response rate among workers. Analysed were only data from workplaces,
1) where >70% of workers filled in the risk factor questionnaire at T0,
2) and >70% at T1,
3) which participated for 12 months and
4) which are health promoting workplaces. 

There were significant results for smoking cessation and for healthier food intake, i.e., 4 fold increase in smoking cessation rates among exposed workers (and no among controls); 10% increase in consuming 5 portions of fruit/vegetable a day among exposed (and 4% among controls). 

There are positive, but non-significant effects for alcohol use and physical activity. No effects on personal relationship and road injury risk were detected.
There are some concerns about validity, e.g. the appropriateness of the outcomes that were measured and the overall quality of the study, such as: a) unlinked pre-post questionnaires (it is not a cohort design but two prevalence surveys on the same population), b) no randomisation (selection bias), c) contamination of the control group, given that they are involved by an intervention, even if with different objective (this could underestimate association). 

Nevertheless, this intervention is a crucial piece in the puzzle of the health promotion strategies in the workplace, and the impact shown on tobacco cessation and alcohol abuse are very promising. 

The programme has been evaluated in randomized controlled trials in Finland, United Kingdom (2020), the Netherlands (2020), and in Italy (2016). Evaluations are underway in Estonia, Greece and UK (Clarkson et al., 2022). The intervention is also implemented in Spain (Lopez-Catalan et al. 2022) and Belgium. The latter two evaluations are not considered in the Xchange rating because of methodological issues.

KiVa was developed and first evaluated in Finland. Subsequently, two thirds of all Finnish comprehensive schools started implementing KiVa IN 2011. For this first evaluation, 78 schools were randomly assigned to intervention and control conditions. The first phase (2007-2008 with 8237 pupils, aged 10-12 years in Grades 4-6 in 78 schools) demonstrated significant reductions in pupil reported bullying and victimisation after one academic year (Kärnä et al., 2011). KiVa was found to significantly reduce (by 17-30% in comparison to control schools) both peer- and self-reported bullying and victimization. The odds of being a victim were about 1.5-1.8 times higher and being a bully 1.2-1.3 times higher for control school’s students than for an intervention school students.

Reductions occurred in all nine forms of bullying examined (including physical, verbal, and cyber-victimisation; Salmivalli, Kärnä, & Poskiparta, 2011). In phase two (2008-2009 with children aged 7-15 years, Grades 1-9) victimisation and bullying reduced by approximately a third for intervention schools. Increased empathy and self-efficacy in supporting and defending victims, and reductions in bully reinforcing behaviour were also reported (Salmivalli & Poskiparta, 2012). Furthermore, anxiety and depression decreased, peer perceptions improved (Williford et al., 2012) and school liking, academic motivation, and performance increased (Salmivalli, Garandeau, & Veenestra, 2012) (Clarkson et al. 2019).

A secondary analysis of this Finnish data (Garandeau et al. 2022) applied multilevel structural equation modelling analyses in pre-test and post-test (1 year later) data in the same sample (n=15,103, 399 control and 462 intervention classrooms from 140 schools). The study showed that KiVa had a positive effect after nine months on affective empathy, but not cognitive empathy, independent of students’ gender, initial levels of empathy, bullying, or popularity, nor of school type or classroom bullying norms.

The age group 10-12 years showed the best result in Finland (Unit 2 lesson curriculum) (Kärnä et al., 2011). Indeed, KiVa’s anti-bullying work has been found to be more challenging in Finnish secondary than in its primary schools (Kärnä et al., 2013).

The Italian RCT (Nocentini & Menesini, 2016) involved 2042 students in grades 4 and 6 (approx. 8 to 11 years old) in 13 randomly assigned school to intervention group (KiVa) and to control group (usual school provision). The study focused on different outcomes, such as bullying, victimisation, pro-bullying attitudes, pro-victim attitudes, and empathy towards victims. Multilevel models showed that KiVa reduced bullying and victimisation and increased pro-victim attitudes and empathy toward the victim in grade 4, with ES = 0.24 to -.40. In grade 6, KiVa reduced bullying, victimisation and pro-bullying attitudes, the effect was smaller, but also significant (d>= 0.20). The study showed that the odds of being a victim were 1.93 times higher for a control group than for intervention group.

The Dutch RCT (Huitsing, 2020) evaluated KiVa and Kiva+ among 4383 students in grades 3 – 4 (aged 8-9) from 98 schools who volunteered to participate in the research. The study collected outcome data at five time points over a period of three years. At the baseline, two-third of the participating schools were randomly assigned to the intervention group (KiVa or KiVa+, with an additional intervention component of network feedback to teachers) and one-third to the control group (waiting list, case as usual). The study showed that self-reported victimisation and bullying reduced more strongly in KiVa-schools compared with control schools, and with stronger effects after two school years of implementation. Moreover, it showed that the odds of being a victim were 1.29 – 1.63 times higher for control group, and the odds of being a bully were 1.19 – 1.66 higher than for KiVa students. There were no significant differences between KiVa and KiVa+.

The British two-arm waitlist control cluster RCT (Axford et al., 2020) involved 3214 students (aged 7-11) in 22 primary schools. The schools were randomly allocated to the intervention group and waitlist control group (usual school provision) with a 1:1 ratio. The outcomes targeted were student-reported victimisation and bullying perpetration, teacher reported child behaviour and emotional well-being, and school absenteeism (administrative reports). There was no statistically significant effect on the primary outcome of child-reported victimisation or on the secondary outcomes. The impact on victimisation was not moderated by gender, age or victimisation status at baseline. The trial found insufficient evidence to conclude that KiVa had an effect on the primary outcome. The programme has been rated as Promising by Blueprints for Healthy Youth Development based on the review of studies conducted worldwide.

The effectiveness of the programme was evaluated by two separate trials published in two papers. The first trial (Allara 2019), although non randomised, is methodologically sound, but had some negative findings (reductions in subjective wellbeing) with outcomes in favour of the control group. This increased  perception  of  psychosomatic  complaints might be due to an increased  emotional  competence. However, the evaluation study concludes that “the programme in its present form should not be disseminated due to the possibility of adverse unintended effects”.

In a second randomized controlled trial (Ribaglietti et al., 2021), the program was not found to have significant effects on the primary outcome variables and most of the secondary variables. However, for the mediators, the association was stronger for the girls in the intervention group, and there was a statistically significant difference in the empathic skills shown by girls, who reported higher levels than boys.

The programme has been evaluated in one quasi-experimental study and one four-year follow-up study in Spain, and one quasi-experimental design in Italy.

The Spanish quasi experimental study was conducted with an intervention group (n=2,567) and a control group (n=2328) enrolled in the last year of Primary Education in Spain (1999) at two time points. Intervention effects were measured by means of chi square, t-tests and tests of covariance. Last month and weekly tobacco use were not significantly affected by the intervention. Initiation of tobacco use among those that had initiated first use was significantly lower in the intervention group. Among those that had already tried alcohol the increase of use was significantly lower in the intervention group. A higher increase of alcohol use in the control group was established. A significantly higher number of participants reduced monthly wine use in the intervention group. There was a significant difference between first time alcohol use between IG and CG. These results should be interpreted cautiously considering that they were studied in a time-frame of only 6 months (November 1997 - January 1998). Effects on 'anti-social behaviour' were analysed but not analysed in relation to the substance use outcomes. 

The Spanish four-year (1995-1999) follow-up study (Gomez Frágüela 2003) involved one control group (n=485) and two intervention groups. The first intervention group (n=235) got the intervention from teachers, the other from professional prevention workers (n=309). A validated questionnaire was administered at four time points and analysed by means of ANOVA and pairwise multiple comparisons. The 15 and 27 months follow-up demonstrated some intervention effects. The 39 month follow up demonstrated similar monthly consumption frequency of tobacco, beer and spirits for all three groups but significantly lower general consumption of cannabis, tranquillizers and amphetamines in the intervention group. Consumption in the two intervention groups was significantly lower compared to the control group.

The Italian quasi-experimental design was conducted among 31 intervention group schools and 24 comparison group schools. The pre-test measurement was conducted prior to the start of the programme, and the post-test eight months after the first year, follow-up after the booster sessions in two subsequent years. The study showed significant effects at post-test in smoking initiation during the first year, weekly drunkenness initiation, and smoking initiation during third years. With regards to alcohol use, there was a lower normative expectation about adults’ drinking and fewer students reported weekly drunkenness. It appears that substance use related differences are less significant at two-year follow-up.

The programme has been rated as Model Plus by Blueprints for Healthy Youth Development based on a review of studies conducted world-wide

The programme has been evaluated in a cluster randomised controlled trial (RCT) involving children aged 12-14 years in several European countries: Austria, Belgium, Germany, Greece, Italy, Spain and Sweden (Caria et al., 2010; Faggiano et al., 2007, 2008, 2010; Giannotta et al., 2014; Vigna-Taglianti et al., 2009, 2014). A further RCT was conducted in Slovakia (Orosová et al., 2020). There were also two cluster RCT’s in the Czech Republic (Miovsky et al., 2012; Jandáč et al., 2021) involving children with a mean age of respectively 11.8 years and 15 years.

For the cross-country study at post-test, exposure to Unplugged was associated with a statistically significant lower prevalence of self-reported daily use of cigarettes, episodes of drunkenness and cannabis use in the past 30 days in the intervention condition compared with the control condition. Young people receiving the programme were less likely than those in the control condition to move from non-smoking or sporadic smoking to daily smoking. Similar patterns emerged in the use of other substances. An analysis by gender found that delayed progression and enhanced regression were higher in the intervention condition among boys, whereas no, minimal or reverse differences were observed among girls.

At 18-month follow-up (Faggiano et al., 2010; Vigna-Taglianti et al., 2014), the use of tobacco and frequency of drunkenness were lower among students in the intervention condition compared to those in the control condition. Students in the intervention condition showed higher tendencies to remain non-users of tobacco or to regress from occasional to no use. The number of students reporting no drunkenness in the past 30 days was higher among students in the intervention condition compared to those in the control condition. Intervention condition participants also reported fewer alcohol-related behaviour problems compared to controls. Further, participants who reported not drinking at baseline were more likely to retain this status at follow-up after participating in the intervention, and those who reported drinking only occasionally at baseline showed a slower progression towards frequent drinking by follow-up if they participated in the intervention. When considering cannabis use, the proportion of persistent non-users was higher among the intervention condition than the control condition. All of these differences were statistically significant.

The RCT conducted in Slovakia (Orosová et al., 2020) had a sample of 1283 schoolchildren with a mean age of 11.5 years from 63 schools. Assignment to either the control group or the experimental group was conducted randomly at school-level. According to the Cochran Q test, the study showed a statistically significant difference between the experimental group and the control group in an increase of the prevalence rates of alcohol consumption during follow-up (3-month, 12-month and 18-month follow-u However, in this study the quality of the randomisation is low, while the outcome measurement took place shortly after implementation in a target group that is quite young (11)

The first Czech study (Miovsky et al., 2012) found a statistically significant effect favouring the intervention, with intervention participants less likely than those in the control condition to have smoked cigarettes in the last 30 days at 3-, 15- and 24-months post-intervention. At the other two time periods (1 and 12 months), differences between conditions in 30-day cigarette use were not statistically significant. There were no statistically significant differences between intervention and control conditions on lifetime cigarette prevalence rates.

The second Czech study (Jandáč et al., 2021) consisted of 70 schools randomly selected with stratification according to their affiliation with a region and size, assigned to one of three groups (the control group, the intervention group 1 and the intervention group 2 exposed to the Unplugged intervention and n-Prevention. The n-Prevention programme is a follow-up (12 months) programme and consists of four lessons providing a general background addressing social norms, social beliefs, refusal skills, and gender-specific differences, neurological aspects and the effects of substance use.  Children from families where the mother reported using alcohol weekly or less frequently however, reported a decrease in drunkenness in the last 30 days compared to the control group. However, the study found no statistically measurable effect on drinking among children who came from families where the mother uses alcohol more than weekly.  These results were observed at a 24-month follow-up, which implies that the Unplugged programme may not be sufficient for high-risk children. Moreover, it is unclear how randomisation took place, and what the drop-out rate and baseline equivalence was. Additionally, in this study a universal programme was used as a targeted intervention.

The programme has been evaluated in three longitudinal quasi-experimental studies (and a 4.5 year follow-up study) in Germany, involving children with a mean age of about 10 years, one cluster randomised controlled trial (RCT) conducted across Germany and Italy, involving children with a mean age of about 11 years, and one quasi-experimental design in Italy and Germany.

In one of the quasi-experimental studies in Germany, there was a statistically significant positive impact on 30-day self-reported frequencies of beer, wine and mixed drinks consumption at one and two year follow-ups (but not at post-test). Two years after the booster sessions of the programme ended (four years after the programme ended), there was a statistically significant effect favouring the intervention in terms of reduced smoking and illicit drug use but no effect on alcohol consumption.

The other German study compared a teacher-led and a student-led version of the intervention to a control condition. At post-test, the frequency and prevalence of self-reported alcohol use was higher in the peer-led version, compared to the teacher-led version and the control condition (this difference was statistically significant). There was no intervention effect on resistance skills towards the offer of alcoholic drinks by peers, and there was also no effect on smoking at post-test. However, students’ expectations about whether they would use cigarettes regularly in the future were lower in students in the teacher-led condition compared to those in the peer-led and control conditions. In addition, students who participated in the teacher-led version of the programme became more resistant to offers of cigarettes (but not alcoholic drinks) compared to students in the other two conditions. Two years after the intervention, the effects were similar. These effects were statistically significant.

The third longitudinal evaluation used a quasi-experimental design (2015) among 1,657 students form Thuringia, Germany. The intervention effects were analysed over a 3-year study interval from ages 10 to 13, covering a life phase when smoking is frequently initiated in the context and under the influence of peers. This study showed that the intervention reduced the age-typical increase in smoking frequency during early adolescence. In addition, it revealed that resistance to peer pressure seems to reflect a major protective factor for smoking in early adolescence. There was a small but substantial effect of intervention. IPSY participants had a slower increase in yielding to peer pressure (ES = .20), and there was an indirect effect on tobacco use (ES = .12).

Finally, in the German follow-up study: the programme was evaluated over 4,5 years based on a longitudinal quasi-experimental design with an experimental and a control group and included 2 follow ups. 1657 German students from 40 schools with a mean age of 10.5 years were included. The programme was proven to be an effective tool for preventing tobacco use and increasing proneness to illicit drug (cannabis and ecstasy) use during adolescence. Moreover, the frequency of alcohol use was also affected during the time of the programme but diminished 2 years later. Even though the programme was effective the study suggest that it is necessary to implement booster sessions in later adolescence.

In the cross-country RCT, the programme had a positive effect on some alcohol-related outcomes in Germany at post-test; specifically, there was a statistically significant effect favouring the intervention on self-reported expected alcohol use during the next 12 months. However, there were no statistically significant differences between conditions at post-test in: self-reported positive cognition towards alcohol use; self-reported 30-day frequency of beer, wine, mixed drinks or spirits; and self-reported consumption quantity of beer, wine, mixed drinks or spirits. There was also a statistically significant effect favouring the intervention in Germany at post-test on three out of nine variables measuring relevant risk and protective factors: resistance to peer pressure; knowledge of assertive behaviours; and school involvement. There were no statistically significant effects on the other six variables: self concept of assertiveness towards groups; self concept of appreciation from others; self concept of general self-esteem; self concept of general problem solving; knowledge of speaker rules; and knowledge of listener rules. There was also a one-year follow-up (participants in the intervention condition were offered seven booster sessions during this period), at which point there was a statistically significant effect favouring the intervention on knowledge of assertive behaviours, resistance/susceptibility to peer pressure, school involvement, expected regular alcohol use during the next 12 months and 30-day frequency of wine consumption.

In the sample in Italy, there was a statistically significant effect at post-test favouring the intervention on consumption quantity of wine, but there were no effects on any of the other seven behaviour variables or the nine variables representing relevant risk and protective factors. There were no effects on any outcome at follow-up.

The second evaluation study in Germany and Italy (2016) used a quasi-experimental design, with one-year follow up. The study sample was composed of 1131 German adolescents with a mean age of 10.45 years, and 159 Italian adolescents with a mean age of 11.14 years. This study demonstrated that the adolescents that received IPSY showed a less steep increase of alcohol use one year after the implementation in both countries, compared to control groups. Also, the program positively affected students’ behaviours and skills that represent prominent protective factors against substance use. German participants increased their knowledge about assertiveness, resistance to peer pressure, and school bonding, and Italian students increased their assertiveness skills. The IPSY programme decreased the expected alcohol consumption in German adolescents, and decreased the consumption of wine in Italian adolescents. The study suggests that life skills-based programmes may be a useful tool in a broader European context.

The programme has been evaluated in one quasi-experimental study in Italy, one quasi-experimental study in Sweden comparing the intervention to a behavioural parent training programme and one randomised controlled trial (RCT) in Sweden.

The Italian study, involving children with a mean age of 12.4 years, found a statistically significant effect favouring the intervention on self-reported frequencies of beer and wine consumption but not on self-reported tobacco or alcohol use. There was no intervention effect on parent outcomes or parent-reported child behaviour problems.

The quasi-experimental study in Sweden, involving children aged 8-12 years with behavioural problems, found that there was either no statistically significant difference between conditions or that the other programme performed better than Connect in improving parent-reported child behaviour and parent outcomes (stress, competence and discipline strategies).

The Swedish RCT, involving children aged 3-12 years, found statistically significant effects favouring the intervention on some measures of parent-reported child behaviour problems (intensity of specified behaviours, and whether the behaviours are problematic) but not on others (inattention, hyperactivity or oppositional behaviour). There were also statistically significant effects favouring the intervention for some self-reported parent outcomes (stress, depression, competence and positive rewards).

The programme has been evaluated in: one randomised controlled trial (RCT) in the Netherlands involving children aged 8-13 years with disruptive behaviour disorder; one quasi-experimental study in Italy involving children aged 8-9 years with disruptive behaviour disorders and comparing Coping Power to, respectively, no treatment and a multi-component programme based on cognitive-behavioural therapy; and three cluster RCTs in Italy, evaluating the programme in a classroom-based format – one with children aged approximately 7-8 years, one with children aged approximately nine years, and one with children aged four years.

In the Dutch RCT, there was a statistically significant effect favouring the intervention at post-test on parent-reported child aggression but not on parent-reported oppositional behaviour or teacher-reported externalising behaviour. About four years after the programme ended, there was a statistically significant effect favouring the intervention on self-reported use of cigarettes in the last month, but not on the use of alcohol or marijuana. However, there was a statistically significant effect favouring the intervention on self-reported lifetime use of marijuana (although not on alcohol or cigarettes). About four years after the programme ended, there was no difference between intervention and control conditions in self-reported delinquency.

In the Italian quasi-experimental study evaluating the targeted version of the programme, assessments were conducted at post-test and one year after the intervention ended but they were analysed together as longitudinal data. There was a statistically significant effect favouring Coping Power (compared to both the control and alternative intervention conditions) on parent-reported child aggressive behaviour and parent- and self-reported child callous traits, but not on parent-reported child rule-breaking behaviour. In terms of parent outcomes, there was a statistically significant effect on positive parenting and inconsistent discipline, but not on harsh discipline or involvement (parent-reported, compared to the other intervention).

In the Italian study with 7-8 year olds (in a universal version of the programme), there were statistically significant effects at post-test and 12 months after the programme ended on three of six teacher-reported child behaviours – prosocial behaviour, hyperactivity and total difficulties – but not on conduct problems, emotional symptoms or peer problems. At the follow-up, academic grades were also analysed, showing a statistically significant effect favouring the intervention.

In the Italian study with 4 year olds (in a universal version of the programme), there were statistically significant effects favouring the intervention at post-test on teacher-reported child conduct problems, prosocial behaviour and total difficulties (but not on emotional symptoms, hyperactivity or peer problems). Considering parent-reported measures, there was a statistically significant effect favouring the intervention at post-test for conduct problems only.

In the Italian study with 9-10 year olds, there was a statistically significant effect on all child outcomes measured, namely teacher-reported emotional symptoms, conduct problems and hyperactivity at post-test.

The programme has been rated as Promising by Blueprints for Healthy Youth Development database based on a review of studies conducted world-wide.