INTRODUCTION

In school curricula, science, technology, engineering, and mathematics (STEM) subjects have received much attention as individual disciplines. Dugger (2010) argued that “the concept of integrating subjects in U.S. schools, especially at the secondary school level, generally is not new and has not been very successful in the past” (p. 117). STEM is seen as an interdisciplinary approach to learning. According to Wahono et al. (2020), although STEM learning has not yet become a regular part of most education systems, many researchers or teachers have enacted STEM. Integrating these four disciplines into an already crowded curriculum has seen a profound change in education and is challenging for schools and teachers to execute. Nevertheless, learning STEM is essential to helping students make connections between STEM concepts. It is crucial for preparing future citizens to compete globally and be successful in the technological world of the 21^st century (Rahman et al., 2022).

Existing research gives some insight into the five models of STEM learning, as presented in Figure 1 (Dugger, 2010; Hobbs et al., 2018). Learning STEM disciplines by integrating them would be more in line with the nature of each subject. Integrated STEM explicitly assimilates concepts from all disciplines and applies equal attention (Dugger, 2010). The engineering and technology elements that fit well with the science and mathematics concepts can be incorporated through integrated STEM. However, more work must be done to examine the best model or strategy in a specific educational context or setting (Dugger, 2010). For example, in Australia, teachers individually and collaboratively develop curricula on STEM in their schools in ways that reflect the priorities and cultures of the school (Hobbs et al., 2018) because no specific teaching can fit all students’ needs and learning styles.

Figure 1. Integration of STEM models (Dugger, 2010; Hobbs et al., 2018)

As providing high-quality STEM learning faces many issues and challenges, researchers have been investigating ways to provide rigorous curricula, instruction, and assessment in integrating technology and engineering into the science and mathematics curriculum to meet the demands of helping students to develop 21^st-century knowledge, skills, and dispositions (Bell et al., 2018). Generally, kindergarten through high school students use their formal and informal STEM educational experiences to explore various interests and build the skills required to pursue their goals. While formal learning is a well-structured activity within educational institutions, informal learning provides unique opportunities to all learners because of the flexibility and ease of access to learning specific knowledge or skills (Jenson et al., 2023; Tedeschi, 2023) and takes place outside the school environment (Ayar, 2015). Informal learning shifts power away from instructor-driven pedagogy toward student-centred learning that nurtures students’ 21^st-century skills such as creative thinking, critical thinking, collaboration, and communication (4C).

In the United States (U.S.), integrating STEM through informal learning has gained momentum apart from formal schooling settings (Dugger, 2010). The National Research Council states that informal STEM learning occurs throughout one’s everyday experiences, designed settings, and activities outside the classrooms that focus on a single STEM area or utilise an integrative approach to STEM (e.g., summer camp). It contains informal activities that advocate making connections between the different STEM disciplines. For instance, cross-cutting STEM concepts from science and mathematics subjects can be widely covered in informal learning activities (Dugger, 2010). Most of the findings from previous studies proved that informal STEM learning has positive impacts. For example, the real-life applications combined with the opportunity for students to choose subjects that matched their interests may have provided relevance to learning that could have improved their STEM perceptions (Kwon et al., 2021). In addition, research findings showed that STEM summer camps have increased students’ interest in a STEM career (Roberts et al., 2018; Siregar et al., 2023) and raised the chances that a student will pursue a STEM career (Kitchen et al., 2018). Djonko-Moore et al. (2018) also suggested that informal learning positively impacted children’s learning, engagement, and interest in science. Nevertheless, Üçgül and Altıok (2021) had an opposite overall result from a study on a robotics camp that showed that a STEM-related robotics camp has a non-significant effect on students’ perceptions of STEM, which could be attributed to the short 8–10-day duration of the robotics camps and the study’s small sample size.

Therefore, we set out to conduct a review to systematically examine previous empirical research to identify the integration of STEM disciplines and the characteristics of STEM learning activities in informal settings. By examining these two elements, the present study provides valuable insights for researchers and practitioners in conducting STEM informal learning.

METHODS

This study utilised a comprehensive review that adapted the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and flow chart. The PRISMA guidelines comprise a four-step flow chart (Figure 2) and a 27-item checklist that lists the criteria for finding, screening, including, and excluding reports relevant to a review (Selçuk, 2019). We used the university library’s federated search service, which includes two well-known databases Scopus and Web of Science (WOS) that are published high-index journal articles. These are the most respected platforms for finding peer-reviewed literature. Additionally, 6,680 results—sorted by relevance—came from searching for all pertinent articles on Google Scholar. Haddaway et al. (2015) recommend that searches of article titles focus on the first 200 to 300 results from Google Scholar to find any missing literature, so we screened the abstracts of the first 300 articles. The search strings used in each database were “STEM education” AND “summer camp” AND “STEM education” AND “informal learning,” as displayed in Table 1. We used the following criteria to examine the abstracts and select the studies sequentially:

Criteria 1: Publication between 2012 and 2022 and written in English.

Criteria 2: Publication of empirical research in peer-reviewed journals.

Criteria 3: Empirical research that includes school students (K-12).

Figure 2. PRISMA flow chart

A total of 37 articles were retained after screening the title, abstract, and keywords aligned with the study’s objectives. For quality assessment, the first author checked whether the retained articles met the set criteria. Discussion was held with other authors until a consensus was reached, and only 17 articles were eligible for the next phase. Given the small number of articles, backward snowballing techniques were used to obtain more comprehensive results (Raharjana et al., 2021). Reference lists of all selected articles were inspected, and eight relevant articles that met the inclusion criteria were added to the dataset. A total of twenty-five articles were studied and examined in depth. The authors organised the extracted data regarding similarities or relevant sub-themes and categorized them around the central themes established by typology (Adams et al., 2021).

RESULTS AND DISCUSSION

The results are presented on the focus on STEM integration and the characteristics of informal STEM activities.

Integration of STEM Disciplines

Most studies, 14 out of 25 articles, focus on integrated STEM, and some incorporate at least one discipline (Table 2). Referring to the STEM integration by Dugger (2010) and Hobbs et al. (2018), integrated STEM can be viewed as five distinct models. The first model separates the STEM disciplines into different teaching and learning sessions, which is the current practice in most schools (Dugger, 2010). The next integrated STEM model focuses on all four disciplines but strongly focuses on one or two disciplines. This model type can be seen in McManimon’s (2021) study, which only focused on science and engineering. McManimon (2021) also implicated a third model, where one of the disciplines was integrated into the other disciplines. Thus, the study integrated technology into science and engineering. Many studies applied the fourth model, where technology, engineering, and mathematics disciplines were integrated into science, as studied by Khanaposhtani et al. (2018), Shahali et al. (2018), Smith-Jackson et al. (2012), and Struminger et al. (2018). Other than that, Ayar (2015), Maiorca et al. (2021), Roberts et al. (2018), and Üçgül and Altıok (2021) studied robotics modules in which science, technology, and mathematics were integrated with engineering. This integrated STEM model was known as total integration, based on Hobbs et al. (2018). Then, Anand and Dogan (2021), Kim and Keyhani (2019), as well as Kwon et al. (2021) practised the last model, in which all four disciplines are distributed into separate subjects.

On the other hand, Baucum (2022) and Kney et al. (2016) have not mentioned any specific integration STEM model that was applied to their studies. Nevertheless, most of the studies applied to almost all the STEM disciplines. Therefore, the findings from these studies would give a better point of view on STEM informal learning as a whole based on the results from the reviewed articles. In contradiction, other studies, such as Djonko-Moore et al. (2018), Habig and Gupta (2021), Hahn et al. (2019), Morris et al. (2019) as well as Struminger et al. (2021), only focused on one STEM discipline that could not represent a comprehensive informal STEM learning.

To be successful with physics concepts and problem-solving skills, learners must hold fundamental schemas or cognitive frameworks that support reasoning from a science and mathematics perspective (Smith-Jackson et al., 2012). Engineering, science, and mathematics activities may give students an authentic learning experience where each subject is interconnected through an applied context. Hence, different sub-dimensions of STEM can be used together when applying their knowledge to real-world problems and challenges (Üçgül and Altıok, 2022).

Considering the relevance of the studies that covered all four disciplines of STEM, the findings from Anand and Dogan (2021) showed that the students enjoyed most of the projects because they were involved in hands-on learning and felt honored by taking ownership of their work. The findings indicate that the robotics summer camp differed from the traditional science classroom in terms of goals, practical work, and social structure (Ayar, 2015). Students at school tend to memorise theories and be familiar with the exam questions as preparation for the examination, not for real-world problems. Ayar (2015) also stated that succeeding on the university entrance exam demanded less practical work and less interaction with real contexts.

According to Khanaposhtani et al. (2018), the combination of STEM content and teacher-specific pedagogical techniques had a positive cognitive and affective impact on participants’ experience in the informal learning environment. Most reviewed articles agree that informal STEM activities help increase interest in students learning STEM subjects (Shahali et al., 2014; Morris et al., 2019). This was also in line with the study by Kwon et al. (2021), who found that informal STEM activities may improve students’ perceptions of STEM. They believed factors that enhance the effectiveness of STEM lessons include informal learning settings, hands-on learning experiences, real-life application, collaboration, and motivation.

Maiorca et al. (2021) stated that the activities during the summer learning experience either expanded or broadened students’ interests in STEM future careers. In addition, students interacting with STEM professionals in authentic ways has also heightened and piqued (i.e., students were more curious) their interest in STEM and STEM-related careers. The increase in students’ interest in STEM careers could be due to the understanding and exposure they gain from the STEM activities they attend (Shahali et al., 2014). Meanwhile, Roberts et al. (2018) proved that the applicability of the activities completed during the STEM summer learning experience not only provided more context for these subjects at school but also helped students to create an understanding of learning them.

The STEM activities promote awareness of integrated knowledge to the public through informal learning experiences (Struminger et al., 2018). Students commented that the addition of engineering would be significant. Students appear more comfortable speaking to STEM professionals through technological platforms (Baucum, 2022). Although engineering is one of the fastest-growing fields, the number of graduates and new admissions in engineering fields is disproportionate (Ayar, 2015). Apart from engineering and technology, Morris et al., (2019) argued that informal science activities are also critical for supporting long-term learning in STEM fields and would contribute to building a healthier pipeline to science careers by promoting STEM learning.

Characteristics of Informal STEM Activities

The characteristics of informal STEM activities can be classified into inquiry-based, problem-based project-based, design-based, cooperative learning, student-centred, hands-on, and 21^st-century skills.

Inquiry-based

Through inquiry-based learning in STEM, the summer learning experience provided an outlet to develop student’s curiosity (Maiorca et al., 2021; Roberts et al., 2018). In the reviewed papers, several essential aspects of inquiry-based learning are mentioned. Firstly, questioning is an integral part of inquiry-based learning where students are asked about their capability to deal with STEM subjects (Bamberger, 2014). Secondly, students used their prior knowledge to generate new ideas. By engaging in the scientific process, students developed several specific skills and practices of science, which they used to address several fundamental questions (Habig and Gupta, 2021) or even simply to participate in a thought experiment with a question asking them to consider how they relate (Semmens et al., 2020). Moreover, inquiry-based learning with prior knowledge might be effective because students reported having no exposure to engineering careers in school, so their engineering understanding could have been improved (Shahali et al., 2018). Therefore, a naturalistic inquiry methodology was also used to explore students’ interest in STEM careers while participating in a weeklong STEM summer learning experience (Maiorca et al., 2021). When students’ conceptions improved, allowing them to learn about STEM-related jobs (Shahali et al., 2018). Finally, students should be given appropriate guidance with open-ended tasks that build up their divergent thinking. Diverse solutions may increase positive perceptions of STEM subjects and careers (Kwon et al., 2021).

Problem-based and project-based

Project-based and problem-based learning allows students to connect to real-world issues (Roberts et al., 2018). In project-based learning, students expressed their ideas through authentic problem-solving hands-on activities, they could relate what they have learned to daily situations in real-world contexts and apply it to the tasks given (Shahali et al., 2014). By contrast, problem-based learning is more open-ended compared to project-based learning. There is no end product, and students are required to identify the problem on their own. Problem-based learning aims to help students develop problem-solving skills by going through the solving process. For instance, students can get real-life exposure through presentations from STEM professionals (e.g., oil and gas engineer, doctor) about an interesting topic, instruction by STEM professionals who are addressing similar problems in the real world, and student can learn about a specific job scope and career path (Kwon et al., 2019). This kind of informal learning experience can support individuals in developing engineering problem-solving and design skills as they engage in engineering challenges (Ayar, 2015).

Design-based

Some studies focus on the element of design wherein students collaboratively designed, built, and improved structures while attending to the mathematically significant shapes they were using. The design-based activity allowed them to engage in the engineering design process (Roberts et al., 2018). In the reviewed papers, the engineering design process occurred in an outdoor science camp (Kney et al., 2016; Puttick and Tucker-Raymond, 2018). The components of the activities make students fully engage with the topic and positively influence their interest and learning motivation (Khanaposhtani et al., 2018). In addition, throughout the engineering design process, parents’ input helped students to design a logic model to ensure the research methods and activities to elicit requirements and conduct usability evaluation were culturally competent and acceptable to the target users (Smith-Jackson et al., 2014). Students should be able to explore STEM activities and engage with engineering design challenges, highlighting that the robotics summer camp fosters interest in engineering and shows that summer camps are distinct from regular science classrooms regarding goals, practical work, and social structure (Ayar, 2015).

Cooperative learning

In informal STEM learning, we found the activities in the studies focus on collaboration and cooperation with others. In collaborative learning, students structure their work without formal training. Through STEM activities, students develop leadership skills, which is perceived to be the outcome of working collaboratively (Evans et al., 2014; Khanaposhtani et al., 2018; Hahn et al., 2019; McManimon, 2021; Shahali et al., 2014). In cooperative learning, on the other hand, training is provided to students to improve participation. Students work in a community of practice in planning, creating, and refining their ideas using the engineering design process (Roberts et al., 2018). Roberts et al. also stated that students’ critical thinking skills could be improved through STEM approaches by cultivating and inspiring curiosity in their participants.

Student-centred

Student-centred is one of the important characteristics of informal STEM learning. Students were given the authority to accomplish their tasks with less supervision. In other words, they were responsible for completing their tasks and actively performing STEM activities under the guidance of facilitators to complete work and parents’ support (Ayar, 2015; Chiang et al., 2022; Habig and Gupta, 2021). Student’s perceptions of the STEM activities helped them not only understand the purpose of the content they were learning but to be aware the STEM connections can help them excel in the subjects they learn in school (Robert et al., 2018).

Hands-on

Hands-on activities play a major role in informal STEM learning. Through a hands-on activity, students enjoyed most of the STEM projects because they were involved in doing the task and felt invested by taking ownership of their work (Anand and Dogan, 2021; Roberts et al. 2018). For instance, opportunities for early STEM learning are often lacking in preschool classroom curricula. Thus, it would be a best practice in engaging young children to play and learn STEM in a natural environment, outside the traditional classroom setting (Anand and Dogan, 2021). Also, many researchers believe place-based and environment-focused informal learning can provide an exciting way to explore science literacy gains because their outreach efforts often encourage participants to experience a particular place and be hands-on with their learning (Ayar, 2015; Khanaposhtani, 2018; Maiorca et al., 2021; Shahali et al., 2014; Morris et al., 2019; Roberts et al., 2018; Struminger et al., 2021). Although there is an attempt to develop STEM culturally relevant and hands-on activities, teachers must be able to show the STEM connections to students for their understanding of the important concepts (Djonko-Moore et al., 2018).

21^st-century skills

The reviewed studies include activities that focus on developing students 21^st-century through informal learning environments, such as STEM camps. Semmens et al. (2020) found that STEM camps have increased students’ STEM knowledge and encouraged students to develop and enhance leadership, creativity, problem-solving skills, and technology literacy (Khanaposhtani et al., 2018). In addition, Semmens et al. (2020) participated students demonstrated through identifying connections discussed in the STEM activities, accurately explaining connections in their own words, and creating mobiles that depict such connections with personal relevance (Semmens et al., 2020). Roberts et al. (2018) reported that activities in the STEM camps are engaging and motivating for students to actively build, explore, investigate, inquire, and communicate while completing their programming using problem-solving skills. Meanwhile Üçgül and Altıok (2022) stated that creativity, critical thinking, communication, and collaboration are thought to be the core skills that will enable students to be successful in the future world. Thus, informal STEM learning can provide students with valuable experiences to develop 21st-century skills and generate STEM knowledge (Struminger et al., 2021).

IMPLICATIONS

In order to develop STEM talent and support students through informal STEM learning, necessary provisions must be provided by the educational communities. One possible reason for the decrease in interest in STEM could be due to the different experiences that the students gained from the activities and the learning experiences in the classroom as compared to their real-life situations (Shahali et al., 2014). We strongly believe students need a quality curriculum that aligns with the needs of informal STEM learning. One of the suggestions is school administrators must be aware of the need for increased time for informal STEM learning through STEM clubs and competitions. In addition, students must be encouraged to participate in informal STEM activities during school holidays that focus on the characteristics discussed above such as hands-on learning experiences, real-life application, and collaboration will positively impact their academic achievement and motivation (Anand and Dogan, 2021).

Over one-third of the children in the world population clearly expressed a lack of interest in science and technology. Nevertheless, research shows that increased confidence leads to better performance during informal STEM learning. Also, increase in students’ interest in STEM careers could be due to the understanding and exposure they gain from STEM activities. The STEM activity’s influence, in terms of helping them understand and know much more about careers in STEM, lasted even though they had little exposure to information about STEM careers in their subsequent schooling (Shahali et al., 2014). Through informal STEM learning, we can educate these underrepresented children to understand not only different STEM topics but also to become curious to raise questions about these topics and motivated to think critically, identify problems, actively seek solutions and feel empowered to take action (Khanaposhtani et al., 2018). Thus, the results across multiple studies from this study should be used to guide informal STEM initiatives, in informing the future development of activities to improve informal STEM learning. Although this paper provides an essential step towards improving the implementation of informal STEM learning, more research is needed. Future research should focus on examining specific STEM interventions utilized in each study and their impacts on student learning outcomes.

CONCLUSION

While this study is vital in highlighting students’ perceptions of how participating in an informal STEM learning environment prepares them for future STEM learning, further research is needed to examine the lasting impacts of participating in this type of informal STEM learning. Exploring students’ future course-taking patterns, success, and perseverance in STEM-related courses and choice of college majors or careers needs further research (Roberts et al., 2018) as to whether the tracked students have maintained their level of interest (Ayar, 2015). Moreover, well-designed educational media can support foundational STEM learning for young children with or without the interactive component. When used with other forms of education and instruction, digital technologies may contribute to children’s academic performance in science, technology, engineering, and mathematics (Aladé et al., 2015). Hence, a study that can assess the effect of exposure to various professions under the umbrella of STEM regarding students’ interest in STEM careers (Semmens et al., 2020) would be interesting.

ACKNOWLEDGEMENTS

The research project is funded by the internal university grants of GG-2023-004 and GUP-2021-028 from The National University of Malaysia.