Learning Outcomes of e-Learning in Psychotherapy Training and Comparison With Conventional Training Methods: Systematic Review

Background Mental disorders pose a major public health problem in most western countries. The demand for services for common mental health disorders has been on the rise despite the widespread accessibility of medication. Especially, the supply and demand for evidence-based psychotherapy do not align. Large-scale increase of modern psychotherapy is difficult with current methods of training which are often expensive, time consuming, and dependent on a small number of top-level professionals as trainers. E-learning has been proposed to enhance psychotherapy training accessibility, quality, and scalability. Objective This systematic review aims to provide an overview of the current evidence regarding e-learning in psychotherapy training. In particular, the review examines the usability, acceptability, and learning outcomes associated with e-learning. Learning outcomes are assessed in different modalities including trainee experiences, knowledge acquisition, skill acquisition, and application of trained content in daily practice. Furthermore, the equivalence of web-based training and conventional training methods is evaluated. Methods Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, a search from Ovid, MEDLINE, PsycINFO, and Scopus databases between 2008 and June 2022 was conducted. Inclusion criteria required studies to describe e-learning systems for psychotherapy training and assess acceptability, feasibility, or learning outcomes. The risk of bias was evaluated for both randomized and nonrandomized studies. Learning outcomes were categorized using the Kirkpatrick model. Effect sizes comparing e-learning and traditional methods were calculated. Results The search yielded 3380 publications, of which 34 fulfilled the inclusion criteria. Positive learning outcomes are generally associated with various e-learning programs in psychotherapy training including trainee satisfaction, knowledge, and skill acquisition, and in application of trained content in clinical practice. Learning outcomes generally show equivalence between e-learning and conventional training methods. The overall effect size, indicating this disparity, was 0.01, suggesting no significant difference. This literature displays a high level of heterogeneity in e-learning solutions and assessment methods. Conclusions e-Learning seems to have good potential to enhance psychotherapy training by increasing access, scalability, and cost-effectiveness while maintaining quality in terms of learning outcomes. Results are congruent with findings related to e-learning in health education in general where e-learning as a pedagogy is linked to an opportunity to carry out learner-centric practices. Recommendations for conducting psychotherapy training programs in blended settings supported by activating learning methods are presented. However, due to the heterogeneity and limitations in the existing literature, further research is necessary to replicate these findings and to establish global standards for e-learning, as well as for the assessment of training outcomes in psychotherapy education. Research is especially needed on the effects of training on patient outcomes and optimal ways to combine e-learning and conventional training methods in blended learning settings.


METHODS
Eligibility criteria 5 Specify the inclusion and exclusion criteria for the review and how studies were grouped for the syntheses.
Pages 5-6 under 'Selection criteria' and 'Framework for classifying studies' Information sources 6 Specify all databases, registers, websites, organisations, reference lists and other sources searched or consulted to identify studies.Specify the date when each source was last searched or consulted.
Pages 4-5 under 'Search method' Search strategy 7 Present the full search strategies for all databases, registers and websites, including any filters and limits used.Pages 4-5 under 'Search method' Selection process 8 Specify the methods used to decide whether a study met the inclusion criteria of the review, including how many reviewers screened each record and each report retrieved, whether they worked independently, and if applicable, details of automation tools used in the process.

Data collection process
9 Specify the methods used to collect data from reports, including how many reviewers collected data from each report, whether they worked independently, any processes for obtaining or confirming data from study investigators, and if applicable, details of automation tools used in the process.

Data items 10a
List and define all outcomes for which data were sought.Specify whether all results that were compatible with each outcome domain in each study were sought (e.g. for all measures, time points, analyses), and if not, the methods used to decide which results to collect.
Page 8 in paragraph under figure 1 10b List and define all other variables for which data were sought (e.g.participant and intervention characteristics, funding sources).Describe any assumptions made about any missing or unclear information.
Page 8 in paragraph under Figure 13a Describe the processes used to decide which studies were eligible for each synthesis (e.g.tabulating the study intervention characteristics and comparing against the planned groups for each synthesis (item #5)).
Pages  1 and Tables 1 and 2.
13d Describe any methods used to synthesize results and provide a rationale for the choice(s).If meta-analysis was performed, describe the model(s), method(s) to identify the presence and extent of statistical heterogeneity, and software package(s) used.  1 and Tables 1 and 2.

Study selection 16a
Describe the results of the search and selection process, from the number of records identified in the search to the number of studies included in the review, ideally using a flow diagram.
Page 7 under 'Search outcome' and in Figure 1 16b Cite studies that might appear to meet the inclusion criteria, but which were excluded, and explain why they were excluded.
In Figure 1 Study characteristics 17 Cite each included study and present its characteristics.
In Tables 1  and 2 Risk of bias in studies 18 Present assessments of risk of bias for each included study.
In Figure 2 and in Appendices 1

Results of individual studies
19 For all outcomes, present, for each study: (a) summary statistics for each group (where appropriate) and (b) an effect estimate and its precision (e.g.confidence/credible interval), ideally using structured tables or plots.
In Tables 1  and 2 and in Figure 3 Results of syntheses 20a For each synthesis, briefly summarise the characteristics and risk of bias among contributing studies.

OTHER INFORMATION Registration and protocol
24a Provide registration information for the review, including register name and registration number, or state that the review was not registered.Review was not registered.
24b Indicate where the review protocol can be accessed, or state that a protocol was not prepared.Protocol was not prepared.
explicit statement of the objective(s) or question(s) the review addresses.Page 4in introduction (paragraph 6) methods used to assess risk of bias in the included studies, including details of the tool(s) used, how many reviewers assessed each study and whether they worked independently, and if applicable, details of automation tools used in the process.Page 5 under 'Risk of bias'Effect measures12 Specify for each outcome the effect measure(s) (e.g.risk ratio, mean difference) used in the synthesis or presentation of results.Page 6

24c
Describe and explain any amendments to information provided at registration or in the protocol.NA Support 25 Describe sources of financial or non-financial support for the review, and the role of the funders or sponsors in the review.Page 30 under 'Financial and non-financial support' Competing 26 Declare any competing interests of review authors.Page 30

Location where item is reported 20d
Present results of all statistical syntheses conducted.If meta-analysis was done, present for each the summary estimate and its precision (e.g.confidence/credible interval) and measures of statistical heterogeneity.If comparing groups, describe the direction of the effect.Present results of all sensitivity analyses conducted to assess the robustness of the synthesized results.NAReporting biases 21 Present assessments of risk of bias due to missing results (arising from reporting biases) for each synthesis assessed. 20b