From Clinical Trial to Education: Methodologies, Assumptions, and Directions
DOI:
https://doi.org/10.21467/ijm.1.1.4668Abstract
In the field of education, emphasis on evidence-based practice, randomized controlled trials (RCTs), causal inference, and process evaluation can all find their roots back in clinical trials and medical research. This response paper surveys contemporary literature in psychometrics, process evaluation, and RCTs aiming to evaluate the feasibility and limitations of RCTs as a methodology in education and provide future directions. Based on the systematic literature review, the author argues: (1) A lack of significant positive treatment effect does not indicate that RCTs are not worth the investment. (2) A careful evaluation of the intervention itself, implementation process, and measurement instrument is recommended for RCTs. (3) There is the need to reframe some causal inference assumptions in an education setting. The paper also provides several examples of reframing assumptions and comments on the caveats. In conclusion, the author foresees a promising future for RCTs in education with the appropriate reframing of assumptions, process evaluation and replication, and recognition of the validity of parallel methodologies.
Keywords:
Randomized Control Trials, Education, Causal InferenceDownloads
References
Barnes, C., Yoong, S. L., Nathan, N., Wolfenden, L., Wedesweiler, T., Kerr, J., ... & Grady, A. (2021). Feasibility of a Web-Based Implementation Intervention to Improve Child Dietary Intake in Early Childhood Education and Care: Pilot Randomized Controlled Trial. Journal of medical Internet research, 23(12), e25902.
Biesta, G. (2007). Why “what works” won’t work: Evidence‐based practice and the democratic deficit in educational research. Educational theory, 57(1), 1-22.
Blanchard, R. D., Artino Jr, A. R., & Visintainer, P. F. (2014). Applying clinical research skills to conduct education research: important recommendations for success.
Boylan, M., & Demack, S. (2018). Innovation, evaluation design and typologies of professional learning. Educational Research, 60(3), 336-356.
Byrne, B. M. (2013). Structural equation modeling with LISREL, PRELIS, and SIMPLIS: Basic concepts, applications, and programming. Psychology Press.
Cochrane, A. L. (1972). Effectiveness and efficiency: random reflections on health services.
Cohen, E., Uleryk, E., Jasuja, M., & Parkin, P. C. (2007). An absence of pediatric randomized controlled trials in general medical journals, 1985–2004. Journal of clinical epidemiology, 60(2), 118-123.
Connolly, P., Biggart, A., Miller, S., O'Hare, L., & Thurston, A. (2017). Using randomised controlled trials in education. Sage.
Connolly, P., Keenan, C., & Urbanska, K. (2018). The trials of evidence-based practice in education: a systematic review of randomised controlled trials in education research 1980–2016. Educational Research, 60(3), 276-291.
Dawson, A., Yeomans, E., & Brown, E. R. (2018). Methodological challenges in education RCTs: reflections from England’s Education Endowment Foundation. Educational Research, 60(3), 292-310.
Delavar, F., Pashaeypoor, S., & Negarandeh, R. (2020). The effects of self-management education tailored to health literacy on medication adherence and blood pressure control among elderly people with primary hypertension: A randomized controlled trial. Patient education and counseling, 103(2), 336-342.
Espelage, D. L., Rose, C. A., & Polanin, J. R. (2016). Social-Emotional Learning Program to Promote Prosocial and Academic Skills among Middle School Students with Disabilities. Remedial and Special Education, 37(6), 323-332. http://dx.doi.org/10.1177/0741932515627475
Grässel, S., &, D. (2020). Recent advances in the treatment of osteoarthritis. F1000Research, 9.
Hedges, L. V., & Schauer, J. (2018). Randomised trials in education in the USA. Educational Research, 60(3), 265-275.
Hitchcock, J. H., Johnson, R. B., & Schoonenboom, J. (2018). Idiographic and nomothetic causal inference in special education research and practice: Mixed methods perspectives. Research in the Schools, 25(2), 56-67.
Holland, P. W. (1986). Statistics and causal inference. Journal of the American statistical Association, 81(396), 945-960.
Hong, G., & Raudenbush, S. W. (2006). Evaluating kindergarten retention policy: A case study of causal inference for multilevel observational data. Journal of the American Statistical Association, 101(475), 901-910.
Ingersoll, R. M., & Smith, T. M. (2003). The wrong solution to the teacher shortage. Educational leadership, 60(8), 30-33.
Linnan, L., & Steckler, A. (2002). Process evaluation for public health interventions and research.
Lipsey, M. W., Puzio, K., Yun, C., Hebert, M. A., Steinka-Fry, K., Cole, M. W., Roberts, M., Anthony, K. S., & Busick, M. D. (2012). Translating the Statistical Representation of the Effects of Education Interventions into More Readily Interpretable Forms. National Center for Special Education Research.
Lortie-Forgues, H., & Inglis, M. (2019). Rigorous large-scale educational RCTs are often uninformative: Should we be concerned?. Educational Researcher, 48(3), 158-166.
Mandy, W., Murin, M., Baykaner, O., Staunton, S., Cobb, R., Hellriegel, J., Anderson, S., & Skuse, D. (2016). Easing the Transition to Secondary Education for Children with Autism Spectrum Disorder: An Evaluation of the Systemic Transition in Education Programme for Autism Spectrum Disorder (STEP-ASD). Autism: The International Journal of Research and Practice, 20(5), 580-590. http://dx.doi.org/10.1177/1362361315598892
McKnight, L., & Morgan, A. (2020). A broken paradigm? What education needs to learn from evidence-based medicine. Journal of Education Policy, 35(5), 648-664.
Odom, S. L. (2021). Education of Students with Disabilities, Science, and Randomized Controlled Trials. Research and Practice for Persons with Severe Disabilities, 46(3), 132-145.
O’Toole, T., Logemann, J. A., & Baum, H. M. (1998). Conducting clinical trials in the public schools. Language, Speech, and Hearing Services in Schools, 29(4), 257-262.
Parker, A. T. (2009). Orientation and Mobility with Persons Who Are Deaf-Blind: An Initial Examination of Single-Subject Design Research. Journal of Visual Impairment & Blindness, 103(6), 372-377. https://ezaccess.libraries.psu.edu/login?url=https://www.proquest.com/scholarly-journals/orientation-mobility-with-persons-who-are-deaf/docview/61869802/se-2?accountid=13158
Pring, R., & Thomas, G. (2004). Evidence-based practice in education. McGraw-Hill Education (UK).
Raudenbush, S. W. (2008). Advancing educational policy by advancing research on instruction. American Educational Research Journal, 45(1), 206-230.
Robins, J. M., Hernan, M. A., & Brumback, B. (2000). Marginal structural models and causal inference in epidemiology.
Rubin, D. B. (1986). Comment: Which ifs have causal answers. Journal of the American statistical association, 81(396), 961-962.
Ruble, L. A., McGrew, J. H., Toland, M. D., Dalrymple, N. J., & Jung, L. A. (2013). A randomized controlled trial of COMPASS web-based and face-to-face teacher coaching in autism. Journal of Consulting and Clinical Psychology, 81(3), 566-572. http://dx.doi.org/10.1037/a0032003
Salinas Cisneros, G., & Thein, S. L. (2020). Recent advances in the treatment of sickle cell disease. Frontiers in Physiology, 11, 435.
Schneider, M. (2019, January). Opening plenary: IES director welcome (Paper presentation). IES principal investigators meeting, Washington, DC.
Siddiqui, N., Gorard, S., & See, B. H. (2018). The importance of process evaluation for randomised control trials in education. Educational Research, 60(3), 357-370.
Styles, B., & Torgerson, C. (2018). Randomised controlled trials (RCTs) in education research–methodological debates, questions, challenges.
Sutherland, K. S., Lewis-Palmer, T., Stichter, J., & Morgan, P. L. (2008). Examining the influence of teacher behavior and classroom context on the behavioral and academic outcomes for students with emotional or behavioral disorders. The journal of special education, 41(4), 223-233.
Sutherland, K. S., Conroy, M. A., Algina, J., Ladwig, C., Jessee, G., & Gyure, M. (2018). Reducing child problem behaviors and improving teacher-child interactions and relationships: A randomized controlled trial of BEST in CLASS. Early Childhood Research Quarterly, 42, 31-43.
Torgerson, D. (2008). Designing randomised trials in health, education and the social sciences: an introduction. Springer.
Wehman, P., Schall, C. M., McDonough, J., Graham, C., Brooke, V., Riehle, J. E., Brooke, A., Ham, W., Lau, S., Allen, J., & Avellone, L. (2017). Effects of an Employer-Based Intervention on Employment Outcomes for Youth with Significant Support Needs Due to Autism. Autism: The International Journal of Research and Practice, 21(3), 276-290. http://dx.doi.org/10.1177/1362361316635826
Downloads
Published
Issue
Section
How to Cite
License
Copyright (c) 2022 Ann Yinqi Zhang
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
