INTRODUCTION

An integrated, problem-centered approach to secondary STEM education is believed to be the key to piquing students’ interest and at the same time fostering their robust understanding of STEM subjects (Kennedy et al., 2014). STEM teachers have a key role in establishing such integrated STEM (iSTEM) education, but are not trained nor supported to do so (Shernoff et al., 2017; Dare et al., 2018). The research literature on educational innovation (Voogt et al., 2011; Parke et al., 1997) suggests that teachers will be better prepared for implementing iSTEM education when they are at the driver’s seat not only of its implementation, but of its design as well.

With these aspirations for secondary STEM education, the 4-year research project STEM@school was launched in Flanders (Belgium) in 2014 (De Meester et al., 2020). STEM@school engaged 30 teams of STEM teachers to implement iSTEM learning materials in their classroom, 10 of which were also involved as teacher design teams (TDTs) in the design of these learning materials. At the start of STEM@school, no concept for effective iSTEM education or learning materials tailored to the Flemish education system were available. In the scope of STEM@school, the study presented in this article investigated the activities teachers undertake to integrate learning contents from separate STEM subjects into the curricula of grades 9 to 12.

THEORETICAL FRAMEWORK

iSTEM Key Principles and Requirements

From a comprehensive review of literature on integrated STEM, Thibaut et al. (2018) extracted four key principles that constitute effective iSTEM education: (1) problem-centered learning, (2) inquiry- and design-based learning, (3) integration of STEM learning contents, and (4) cooperative learning. To endorse the value of long-lasting (educational) research in each of the different STEM disciplines (NRC, 2012), we add the extra principle of research-based learning. As a result, we advocate the five key principles for establishing high-quality iSTEM education in secondary school classrooms shown at the left of Figure 1 (De Meester et al., 2020). Based on the research literature on instructional design for integrated STEM education (NAE et al., 2014; Felder et al., 2016), we translated these five key principles into 10 requirements for high-quality iSTEM education, shown at the right of Figure 1.

Figure 1. Key principles and requirements for high-quality integrated STEM education

iSTEM Curriculum Design

An effective way to prepare teachers for implementing a new instructional approach is to immerse them in the key principles underpinning this new approach (Loucks-Horsley et al., 2009). This particular strategy of active teacher involvement fosters their self-efficacy (Ingvarson et al., 2005), their beliefs (Girvan et al., 2016), and their classroom practices (Jeanpierre et al., 2005) in favor of the targeted principles. Involving teachers in curriculum design results in findings that are in line with these effects (Voogt et al., 2011; Mooney Simmie, 2007). Design can be considered a form of complex problem solving, involving interdisciplinary thinking, handling of multiple decisions and unanticipated problems, inquiry, collaboration in multidisciplinary teams, and a scientific basis to start from (de Vries, 2020; Jonassen et al., 2006; Stanovich et al., 2003). Teachers will thus experience the key principles of iSTEM education (Figure 1) firsthand when they participate in cooperative, interdisciplinary design of iSTEM learning materials, which, in turn, will prepare them to implement these principles in their classrooms.

The role of TDTs in curricular innovation has gained much interest in educational research, often in the context of professional development (Handelzalts, 2019; Coenders et al., 2015; Huizinga et al., 2014). However, there is little in-depth research on teacher endeavors to design iSTEM learning materials. Some studies report on teachers’ experiences and struggles while designing integrated or context-based STEM materials (James et al., 2000; Stolk et al., 2016). Guzey et al. (2016), who examined 20 teacher-designed iSTEM learning units, concluded from their study:

[I]ntegrated or interdisciplinary science curriculum is not a new concept; however, designing instructional materials for integrated STEM education is new for many teachers. Put simply, there are few resources available for teachers to help them develop integrated STEM curriculum materials and designing curriculum materials is a complex process. (Guzey et al., 2016: 14)

Little research has focused on the teachers’ process itself when they are designing iSTEM learning materials. McFadden et al. (2017) observed two TDTs’ design processes and visualized these processes via flowcharts. These flowcharts contain dilemmas and design decisions that reflect the particular group dynamics in the TDTs, but do not seem to be meant as general guideline for iSTEM curriculum design. Better generalizable are the design activities mentioned in the study of Satchwell et al. (2002). In their study, a team of nine competent middle-school teachers and researchers from different STEM disciplines engaged in (1) choosing a shared theme based on the national math and science standards and key STEM concepts, (2) developing a series of discipline-oriented objectives aligned with the national standards, (3) creating an introductory challenge to raise students’ interest, (4) designing learning materials and authentic assessments to have students explore, study, and apply the new key concepts. The designed learning units were reviewed on content by STEM experts, and feedback from field-tests and classroom visits was incorporated afterwards. Satchwell et al. state that “[t]he development of a standards-based, integrated mathematics, science, and technology curriculum is a challenging task” (p. 16) and indicate that the development of a framework, field testing, collection of feedback, reflection, and revision are important parts of the design process.

These studies suggest that secondary STEM education would considerably benefit from more in-depth research on the specific activities constituting TDTs’ actual iSTEM design process and on how these activities affect the designed learning materials. In this study, we translated the research literature’s calls for guidance regarding the iSTEM design process into two research questions:

RQ1: How can we model the process teacher teams go through when they are designing integrated STEM learning materials from scratch? Which activities can we identify in their design process?

RQ2: Based on the learning materials designed by the teacher teams and the requirements for high-quality integrated STEM education, which crucial, profitable, detrimental, and missing activities can we distinguish in the design process we found in RQ1?

An evidence-based model and assessment of such activities could support school organization, teacher education, and professional development programs to better prepare and support STEM teachers to establish high-quality integrated STEM education (Guzey et al., 2016; Shernoff et al., 2017). It could facilitate teachers’ design process, empower them to work in a systematic and organized way, and to communicate with a common language (Woods, 2000).

Outset of This Study

Regarding this study, the following initial conditions should be taken into account: (1) the participating teacher teams were not guided by the iSTEM key principles shown in Figure 1, since these were established during the successive research cycles of STEM@school; (2) the teacher teams were not guided by any design or problem solving framework. They were only guided by the aspiration of deepening students’ understanding and stimulating their interest in STEM subjects by meaningfully integrating STEM learning contents. Given these conditions, we looked open-mindedly at how the teacher teams approached the design of iSTEM learning materials, and the design activities they engaged in. The iSTEM requirements in Figure 1 are the criteria against which we evaluated the iSTEM learning materials designed by the TDTs, to assess the effect of each of the TDTs’ design activities on the designed learning materials.

METHODS

An inductive, data-driven approach was applied to build a model describing teachers’ process of designing iSTEM learning materials. We chose case study as research method because of the real-world and incontrollable character of the context in which this study would take place (Kitchenham et al., 1995; Yin, 2003). Each project year of STEM@school, a TDT was selected for the case study by means of purposive sampling (Mortelmans, 2007) until we obtained theoretical saturation (Eisenhardt, 1989). In total, four cases (C1, C2, C3, and C4) were studied, each involving a TDT engaged in the process of designing iSTEM learning materials from scratch. In order to construct a valid model of this process, we gathered and triangulated data from multiple sources of evidence (Yin, 2003), as shown in Figure 2 and elaborated below.

Construction of the Case-based Model of the TDTs’ iSTEM Design Process (RQ1)

Model construction via within- and cross-case process analysis (step 1.1) – Each case was first analyzed separately, in order to be able to identify unique patterns before generalizing them across cases (Eisenhardt, 1989). Therefore, the main researcher (first author) kept field notes and reports on the TDTs’ design meetings, and memos on team dynamics. These data were supplemented by secondary data, such as e-mails sent among TDT members and coaches, annual planning schemes, materials, prototypes and presentations produced by the TDTs, as shown in Figure 2. First, open coding was used to label excerpts of meeting notes as particular actions, by means of direct interpretation (Stake, 1995). The main researcher then engaged in axial coding (Charmaz, 2006; Stake, 1995): by means of a codebook, she interrelated codes that seemed to lead to one and the same outcome. Such outcomes were, e.g., a target group, an inventory of prescribed STEM curriculum standards, and a list of constraints for the central student task. Actions leading up to one outcome were classified under one category, i.e., a design activity. These categories were refined based on the secondary data. From C2 on, the main researcher performed a cross-case analysis (Eisenhardt, 1989) after the design process of a case had come to an end (see Figure 2): via constant comparison (Charmaz, 2006), she checked the data collected and codes generated in the different cases to see whether and which actions could be classified under design activities identified in the previous cases. On a weekly basis, the researcher discussed the data and her interpretations with other STEM@school researchers. They shared their perspectives until a consensus on the categorization was reached. Appendix 3 shows the identified categories with their major actions (clustered codes) and outcomes.

Figure 2. Data collection and analysis of this study

Model validation (step 2.2) – To verify the veracity (or representativity) of the constructed model (Van Driel et al., 1999), we asked a teacher who had participated in three of the cases to draft a model based on his own impression of the iSTEM design process. He was asked to send it per e-mail. We compared his digital model to our case-based model to look for discrepancies.

Evaluation of the Design Activities Constituting the Case-based Model (RQ2)

To determine the crucial, detrimental, and missing activities in the observed design processes (RQ2), we triangulated data from multiple sources:

• the learning materials designed by the TDTs of each case,

• the iSTEM requirements shown at the right of Figure 1,

• the design activities constituting the case-based model resulting from RQ1.

Per-case product evaluation (step 2.1) – Three STEM@school researchers (among whom the first and last author of this paper) collectively evaluated the products of the studied design processes (i.e., the learning materials designed by the TDTs in cases C1, C2, C3, and C4) against the iSTEM requirements (see Figure 1).

Cross-case linking of the design activities to the iSTEM requirements (step 2.2) – Based on a concise list of their constituting actions, each of the design activities of the process modeled in step 1.1 was linked to one or multiple iSTEM requirements. Therefore, the main researcher and an external researcher (not associated with STEM@school and therefore unbiased by the events) individually assessed per design activity whether or not its actions (as shown in Appendix 3) could affect the fulfillment or failing of each iSTEM requirement. Afterwards, they compared and discussed the resulting associations until they reached a consensus.

Cross-case process evaluation (step 2.3) – We interconnected the result of step 2.1 (i.e., the evaluation of the materials designed in each case against the iSTEM requirements), the result of step 2.2 (i.e., the links between the iSTEM requirements and the design activities of the case-based model), and the result of step 1.1 (i.e., the case-based model with the occurrence or absence of the design activities in each case). Through this interconnection, the quality of the product of a design process was traced back to the occurrence or absence of certain design activities in the modeled process, using determination Table 1.

Context and Selected Cases

Table 1. Determination of the crucial, profitable, detrimental, and missing design activities

In autumn 2013, the STEM@school project team launched an open call among Flemish schools to sign up for trying out a new, integrated approach to STEM in their secondary education. The schools had to indicate whether they also wanted to participate in the design of this approach and associated learning materials. Thirty schools signed up to pilot the new iSTEM approach. The STEM@school team initially selected 10 schools to delegate a teacher team for the iSTEM curriculum design, based on whether their motivation favored interest and enthusiasm over political agenda (such as using the new approach merely as a pretext for student recruitment). The selection held a balanced mix between schools providing education with a science focus (S), and schools providing education with a technical focus (T). Each of these schools delegated one team of two to five teachers for the iSTEM curriculum design. From this ‘big group’ of teacher teams, each project year, the STEM@school team composed several multidisciplinary, cross-school TDTs based on considerations prevalent at that time (see Table 2). Out of these TDTs, we selected one team per project year for our case study by means of purposive sampling. The school team of C1 had presented itself to the STEM@school staff before the project was actually launched and was therefore included as an exploratory case. The context of the selected cases is shown in Table 2 and Table 3.

For our purposive sampling, we implemented the following criteria (Mortelmans, 2007):

• informational contribution and convenience − TDTs who did not manage to get started properly in the iSTEM design were excluded from the case study. C1, C2, and C3 were selected because their TDTs were coached by the main researcher, which gave her plenty of opportunities to gather data on the iSTEM design process.

• homogeneity – To identify the influence of commonalities and to explore evolution in time on the identified pattern of design activities, we selected C3 based on its correspondence with C2 in terms of team composition, learning contents to cover, and team coach.

• heterogeneity – In order to identify commonalities in the design processes of TDTs despite variation, C4 was selected because its TDT composition differed completely from the TDT composition of C3 (different TDT members and coach). Furthermore, the stage of the TDT of each case in the project varied from zero years of design experience (C1) to three years of design experience (C4).

Apart from schools B_T and C_S (see Table 3), all participating schools agreed to the following terms proposed by the STEM@school team:

• The learning materials designed in year i would be implemented in the pilot classrooms in school year i+1.

• A new subject ‘iSTEM’ would be incorporated into the students’ weekly timetable (see Table 4), especially designated for activities concerning STEM integration, inquiry, and design.

• Truly integrated learning materials cover contents from different STEM subjects. Therefore, not only the new ‘iSTEM’ subject but all of the involved STEM classes and their teachers would have to be engaged with the implementation of the materials.

• To consistently and profoundly expose students to integrated STEM activities, two to four large learning units were to be implemented per school year, each unit covering several weeks to up to a whole semester. Each TDT would design iSTEM learning materials for one unit.

• In the timetables of the teachers participating in the iSTEM design, Tuesday afternoons would be reserved for weekly within-school design meetings and monthly cross-school TDT meetings.

• The design process would be limited to the period from September until April each project year.

• Each TDT would be coached by a researcher from the STEM@school project team. These coaches would take the lead in the TDT composition, planning of TDT meetings, writing of a script for the learning materials, and finally bundling the materials developed by the different school teams in the TDT.

RESULTS

Figure 3. Model of the phases and activities (indicated by acronyms) identified in the design processes studied in cases C1, C2, C3, and C4

Construction of the Case-based Model of the TDTs’ iSTEM Design Process (RQ1)

Figure 3 shows the model of the design processes of the four studied cases. In these processes, we distinguished five phases, each consisting of several design activities, which we briefly discuss below. Appendix 2 gives a detailed overview and illustrates the design activities we identified in each case.

In the Context analysis phase, the teachers defined the target group of students for whom they would design the iSTEM learning materials (TGI) and analyzed the context, i.e., scanned the national curricula of the different STEM subjects of this target group to look for learning contents that could be integrated (SCA).

In the Theme selection phase, based on the learning contents extracted from the STEM subject curricula, the TDTs discussed possible themes for the learning units and a sequence in which these units should be implemented in the target group’s school year (BCT). Based on the conditions shown in Table 2, the STEM@school team then composed the TDTs and divided the themes among them (TFD).

In the Content/challenge brainstorm phase, the TDT members discussed the learning contents relevant for the theme that was assigned to them (CIL), as exemplified by the following excerpt of a meeting in C4 (theme algae):

We made the exercise: which learning contents can be addressed from (…) the curriculum of [grades 11 and 12]? In biology, it’s indeed more about microscopy, mitosis: what is that? Eh, the connection with the growth charts, but that is the math. Photosynthesis came up (…) And then, the biochemistry of what plants and algae consist of. The link with lipids, proteins, and which nutrients do we need to [grow] them. And then we get to the link with growth medium. (…) Then, within chemistry, we [distinguished] determination of concentration and dilution, possibly, in preparing the media. (…) And also, if we are working with acidity, if we are measuring that in our medium, eh... we could develop some learning materials on that as well. And eh... then, of course, also the separation and extraction techniques, if we want to go into... how to isolate the lipids. (C4_Bio/Chem2)

What we would like to develop is, eh... about spectrophotometry: specifically what it is; what the relationship is between the [algae] concentration and the [light spectrum]. Eh, we also want to develop learning materials on LDRs and phototransistors: what the relationship is between the light intensity and the electric current. (C4_Phys)

Based on the listed contents, they formulated a student challenge that would raise the need to learn and link these contents (CF). In C2 and C4, the TDTs iterated these activities to better match the challenge with the learning contents. These TDTs also studied the feasibility and open-endedness of the challenge (RFS) and broke it down into smaller subproblems (CBS). The TDTs then conceived learning activities that would lead students of the target group to learn and link the targeted contents in order to solve the challenge (DLA). Figure 5 illustrates this phase for C2. The TDTs also carried out experiments and built example prototypes of their students’ future products (EPD). The result of this activity in C3 (theme ball launcher and car) is shown in Figure 4. The TDT of C3 started this phase by forming monodisciplinary subteams who further developed discipline-specific materials regarding the targeted learning contents (SFD).

Figure 4. Result of design activity EPD in C3 (theme ball launcher and car)

Figure 5. Overview of the results of some important activities in C2

The Reporting phase took place in one plenary meeting within the big group of TDTs, organized by the STEM@school team in order for the TDTs to present their preliminary designed materials and exchange feedback (REF). In C2 and C3, the TDTs also discussed the rescheduling of the learning unit implementation within the students’ school year (TPR).

In the Development phase, the TDT coaches wrote a script (or learning-unit scenario) to interconnect the student challenge and the learning activities that had been conceived by the different school teams constituting the TDT (SW). Meanwhile, the TDT members refined their syllabi (SF), elaborating the learning activities and instructions for the target group. These materials were bundled by the TDT coach (MB) and revised and fine-tuned a last time by the TDT members (MRF).

The revision of the learning materials, based on the feedback from the teachers who implemented them in the pilot classrooms, fell out of the scope of this study, as it was merely conducted by the STEM@school staff. The final learning materials can be found on the project webpage (STEM@school, 2017).

The left of Figure 6 shows the digital model made by one of the case-study participants (translated to English) upon our request in order to validate our model (step 1.2 of our study design). On the right, we depict the activities of our case-based model, of which the labels and descriptions strongly match the descriptions in the participant’s model. Comparison suggests that our case analysis produced a representative and more fine-grained abstraction of the design process, and that we did not overlook any activity.

Figure 6. Participant AS_C123_ Phys’s model of the design process activities (a) and matching activities identified in our case study (b)

Evaluation of the Design Activities Constituting the Case-based Model (RQ2)

Figure 7. Results of the per-case product evaluation (step 2.1) and the cross-case linking of the design activities to the iSTEM requirements (step 2.2 of the study design)

To identify the crucial, detrimental, and missing activities in our case-based model (Figure 3), we utilized the list of iSTEM requirements shown at the right of Figure 1. The left side of Figure 7 shows how the learning materials designed in each case were evaluated with respect to each of the iSTEM requirements (step 2.1 of our study design). The right side of Figure 7 shows the links between the design activities identified in our case study and each of the iSTEM requirements (step 2.2 of our study design).

The combination of the results shown in Figure 7 and the case-based model in Figure 3, which shows the occurrence or absence of the design activities in each case, is shown in Table 5. This table formed the basis to further analyze the effect of each design activity on the learning materials designed in each case (step 2.3 of our study design). We note that, in these analyses, we did not take into account other, unidentified activities or factors that might have played a role in the design processes.

Table 5. Fulfillment of each iSTEM requirement per case (see step 2.1), together with the occurrence or absence in each case (see step 1.1) of the design activities linked to each requirement (see step 2.2)

In order to determine the crucial and missing design activities in the modeled design processes, we performed a cross-case analysis per iSTEM requirement based on Table 5. For each requirement, we determined (a) the minimal combination of design activities linked to this requirement that sufficed to fulfill () the requirement in at least one case, and (b) the maximal combination of design activities linked to the requirement that had not sufficed to fulfill the requirement and had thus resulted in its failing () (see Table 6).

Table 6. Cross-case determination of the activities crucial for each iSTEM requirement to be fulfilled, derived from Table 5

By subtracting the insufficient combination of activities from the sufficient combination (i.e., (a) – (b) in Table 6) and taking into account the causalities between activities (see at the right of Figure 7), we can conclude for certain that the following activities are crucial to fulfill some of the requirements for high-quality iSTEM education:

• CIL (learning-content identification and linking), since CIL is also an important precondition for DLA (design of STEM linking learning activities), which is in turn an important precondition for SF (syllabus finalization),

• CF (challenge formulation),

• as important preconditions for CIL and CF:

  • BCT (brainstorm on learning contents and themes), and therefore:

    • SCA (specific-context analysis), and therefore:

      • TGI (target-group identification),

• CBS (challenge breakdown into subproblems), and

• RFS (requirements and feasibility study).

Furthermore, the learning materials of none of the cases met requirements 8 and 9, and Table 6 is not conclusive about the activities necessary to fulfill requirement 6. These findings suggest that one or a few design activities are missing in order for requirements 6, 8, and 9 to be fulfilled. More precisely, actions should have been taken in the design processes to ensure:

• student cooperation (~req6);

• goal-oriented strategies (~req8); and

• integration of know-how from discipline-specific educational research (~req9).

In order to determine the profitable and detrimental design activities in the modeled design processes, we performed a cross-case analysis per design activity based on Table 5. For each design activity, we counted the total number of iSTEM requirements linked to this activity: (a) those that were fulfilled () across all cases in which this activity had occurred, and (b) those that were failed () across all cases in which this activity had occurred. We did the same for all cases in which this design activity had not occurred. Taking into account the causalities between activities (see at the right of Figure 7), this resulted in Table 7.

Table 7. Cross-case number of fulfilled versus failed iSTEM requirements linked to each design activity, derived from Table 5 and the causalities in Figure 7

Using determination Table 1, Table 7 suggests that most activities were (rather) profitable, having a moderate to very positive effect on the fulfillment of the requirements for high-quality iSTEM education. Design activities SFD (subteam formation and content division) and SF (syllabus finalization) appear to be rather detrimental.

Interpretation of the Findings

In this section we relate the most significant results on the assessment of each design activity to our observations during the case study. The identification and cross-disciplinary linking of STEM learning contents (CIL) and the formulation of a student challenge (CF) are shown to be crucial activities for the designed materials to fulfill the iSTEM requirements. Particularly in C2 and C4, we saw how iteration of these two activities played a pivotal role in the TDTs’ design processes. This finding is remarkable, given the fact that the TDTs of these two cases had no team member or coach in common. This iteration manifested itself in lively cross-disciplinary discussions during TDT meetings, which reflected the iSTEM key principles problem-centered learning, integration and cooperative learning among the participating teachers: The teachers were trying to find a common ground, for which they had to understand each other, and therefore, they had to learn from each other across STEM disciplines. This resulted in a clever alignment of an engaging student challenge and the targeted learning contents from different STEM subjects. In C1, no challenge was formulated. As a result, the learning activities in the designed materials of C1 contained an incoherent set of cross-disciplinary but unrelated experiments and exercises (De Meester et al., 2015). In the other cases, we found the formulated challenge to be the main focus and motivator for the TDTs. In C3, no actual learning activities were designed (no activity DLA), probably because the specific learning contents to be addressed, had not been articulated or discussed (i.e., activity CIL was not found in C3).

The requirements and feasibility study (RFS) appeared to be a crucial design activity as well. This activity had manifested itself in two different ways. In C2, this activity was carried out to ensure that the challenge would be open-ended (i.e., allow for different possible solutions). In C4, the TDT undertook company visits to assess the feasibility of different algae applications, such as waste water treatment, biofuel production, and nutritional supplements. In this design activity, as well as in the investigation of learning contents, we saw teachers engaging in the iSTEM key principle of inquiry-based learning.

The challenge breakdown into subproblems (CBS) seemed to help the TDTs of C2 and C4 to identify the sequence in which to address the targeted learning contents throughout the learning unit, which, in turn, guided the design of the learning activities (DLA). The outcome of this activity formed the basis of the script writing (SW), in which the scenario of the learning activities was outlined (except from C3, in which the TDT coach had to build the script from scratch).

We attribute the absence of activities CIL, RFS, CBS, and DLA in C3 to the fact that the TDT split up into smaller, monodisciplinary subteams, who independently developed materials for the separate STEM classes (SFD). These materials contained no links across STEM disciplines and did not contribute to solving the challenge. This could explain why activity SFD came out of the analysis as a ‘rather detrimental’ activity. Also the syllabus finalization (SF) came out of our analysis as rather detrimental, but no observations in our case study could corroborate this finding.

In C2, C3, and C4, the TDTs devoted a lot of design time to a tryout of the experiments or the design of a prototype (activity EPD). In C2 and C4, this activity fed into the design of the learning activities (DLA), as the pedagogical approach of the learning activities relied on the process and results of the experimentation and prototyping. In C3, the majority of the design process was absorbed by this activity. The intermediary reports of the subteams of C3 and the discussions during the TDT meetings were dominated by the teachers’ efforts to produce working prototypes (see Figure 4) and test their accuracy. This resulted in many technical documents which lacked a pedagogical approach. The final designs were so complex that the students, in order to get working prototypes, would have to follow a cookbook plan.

We believe that the composition of multidisciplinary TDTs (activity TFD) has strongly facilitated the cross-disciplinary brainstorming that took place in activities CIL, CF, and the design of cross-disciplinary learning activities (DLA) in C2 and C4. Activity REF (the review and exchange of feedback) was much appreciated by several teachers, as evidenced by their reflections:

Actually, I find it quite important: explaining what we want to do, and that [other teams] can say: “yeah, that is maybe a bit odd,” or “that can be done better like this,” or “I would put more emphasis on that.” So that we can take this into account before we [proceed], ‘cause... yeah, if we would already have developed everything, and then they [would come up with] a lot of comments, that [would be] quite annoying. (C4_Phys)

In none of the observed cases, and nowhere in the design processes did the TDTs take the time to construct a clear and comprehensive list of learning goals they wanted their students to achieve. We believe that this missing though important design activity, explains the fact that certain requirements for high-quality iSTEM education were failed by all designed materials. Although the teachers’ main drive was to address the learning contents mentioned in the national curriculum standards, they never discussed what they considered to be important competences with regard to these contents. Furthermore, none of the TDTs consulted scientific articles on students’ difficulties with regard to these learning contents. Consequently, the iSTEM key principle of research-based learning was missing in the TDTs’ design practices.

DISCUSSION

We conducted a case study to model the design processes of four TDTs involved in the development of integrated STEM learning materials. Assessing the learning materials designed by each of these TDTs against the requirements for high-quality iSTEM education and linking the modeled design activities to these requirements, allowed us to distinguish the crucial, detrimental, and missing activities in the design processes. In sum, our study shows evidence that a successful iSTEM design process is constituted by (1) the multidisciplinary TDT formation (activity TFD), and (2) the alignment of (a) the identification and linking of STEM learning contents and (b) the formulation of a real-world student challenge (activities CIL and CF). We found that, via these activities, STEM teachers engage in constructive, cross-disciplinary conversations. In turn, these conversations lead to engaging student challenges and activities that do not only address learning contents from different STEM disciplines but also add meaning to students’ learning in the light of solving these challenges. On the other hand, our study also revealed that, in order to fulfill all requirements for high-quality iSTEM education, TDTs should explicitly be encouraged to discuss and create learning opportunities that ensure (1) students’ achievement of important learning goals associated with (higher-order) STEM competences, (2) student cooperation, and (3) the use of research-based strategies. In this discussion we first situate our findings in the broader research field on interdisciplinary curriculum design and in its national educational context. We then elaborate the limitations of our study. We conclude with suggestions for future research and opportunities entailed by our findings.

This study presents a first explicit model of the particular activities constituting the design of integrated STEM learning materials as carried out spontaneously by teams of in-service high-school STEM teachers. Table 8 shows the design activities we identified via our inductive approach and compares them to actions and activities we could detect via deeper analysis of other research on interdisciplinary curriculum design, using our case-based model as a lens. A testing stage did not fall within the scope of this study. Table 8 shows that our model portrays a comprehensive account of the activities comprised by the design of interdisciplinary materials for STEM. The endeavors of the teams in Satchwell et al.’s (2002) study show the biggest overlap with the design activities carried out by the TDTs in our study. Satchwell et al. and the theoretical model of Krajcik et al. (2008) mention the development of student learning objectives and assessments. This activity was lacking in McFadden et al.’s (2017) as well as in our study. Teachers enthusiastically dive into the design of a fun challenge (Guzey et al., 2016) and the inherent hands-on design (see activity EPD in our case study), without setting their objectives first. Furthermore, since teachers do not easily find their way to research literature (See et al., 2016), they do not tend to use evidence-based instructional strategies in their design. Much like in McFadden et al.’s (2017) study, the design discussions in our cases often “remained geared around what types of experiences students should be engaged in and not how students learn best during integrated learning experiences” (p. 13). Unlike the teams of McFadden et al., however, the TDTs in our case study did not seem reluctant to explore the multitude of learning opportunities encompassed by open-ended challenges (see activity RFS). We believe this courage stems from the fact that the TDTs in our study were composed by teachers from different STEM disciplines, from different grade levels and from different schools (see activity TFD). Each TDT thus held a lot of expertise in diverse areas for the TDT members to rely upon.

As the findings we present result from a case study, they should be situated in the specific context in which the study was carried out. Before STEM@school, the subject of ‘how to teach integrated STEM’ did not make part of STEM teacher education programs in Flanders (Belgium). The high-school teachers participating in this study have been schooled in teaching one or two STEM subjects (e.g., physics, biology, mathematics) in a monodisciplinary way. They were, therefore, used to adhering to subject-specific national curricula and handbooks, and to consulting with colleagues teaching the same STEM subject(s). This traditional way of teaching STEM subjects did, however, not require them (and, therefore, they were not used) to consider viewpoints and curricula from colleagues teaching other STEM subjects. The consequential tendency to resort to monodisciplinary teaching is reflected in activity SFD (monodisciplinary subteam formation) which occurred in C3 and was found to be harmful to the design of high-quality integrated STEM materials. Furthermore, before the start of STEM@school, the concept of ‘integrated STEM’ at school had been promoted in Flanders via short-term projects or events, such as a STEM day, workshop, or company visit. Such activities did address learning contents from different STEM subjects, but often in a ‘fun’ and not necessarily integrated or instructive (i.e., standards-based or competence-oriented) way. This idea may have played a role in the fact that the high-school teachers in this study (1) have not considered articulating meaningful learning objectives and assessments around the contents they extracted from their subject-specific curricula, and (2) sometimes prioritized the development of fun, hands-on activities addressing these contents over the design of minds-on learning activities aiming at meaningful STEM competences related to these contents. We believe that guidelines based on our evidence-based model can favor teachers, teacher educators, and schools in any educational context that shows characteristics similar to those of the Flemish context described here.

This study has some constraints. As in other case study research, certain features, such as team dynamics, the participants’ knowledge and beliefs, and the role of the team coach could not be clearly distinguished from the process under study. As a result, we cannot indicate which variables might have influenced the occurrence and quality of the design activities and how. In order to construct a valid model, we implemented several literature-based methods such as purposeful sampling and theoretical saturation, and recursive cycling through and triangulation of different data sources (Creswell et al., 2000; Eisenhardt, 1989). However, the research was mainly conducted from an insider’s view by the STEM@school project staff, and some bias can thus not be ruled out. On the other hand, complete immersion in the process enabled the researchers to experience the endeavors of iSTEM design first-hand, and to interpret the observed activities in their specific context, which they got to know from inside out (Mortelmans, 2007). Another limitation of the study is that we cannot claim that certain design activities re-emerged in the different cases by coincidence, since some learning effect might have set in among the TDTs during the course of STEM@school. However, taking into account these limitations, the combination of observation-based modeling, criterium-based product assessment, linking, and logical deduction, provides a useful new approach to evaluating curriculum design processes.

Future research could advance the case-based model presented in this work. Firstly, the classroom implementation and corresponding revision of the designed materials could be examined and included (e.g., as extra phases) in the model (Satchwell et al., 2002; Berland, 2013). Additionally, further research among pre- and in-service STEM teachers could examine the extent to which the case-based model and findings of this study support other TDTs’ endeavors to design integrated STEM education. Our observations showed indications that participants in our TDTs were being immersed in some of the key principles of high-quality iSTEM education. In order to further inform teacher training programs, future research could focus on (1) teachers’ immersion in these principles when being involved in iSTEM curriculum design that is guided by our findings, and (2) the effect on their knowledge and self-efficacy and those of their students (van Keulen et al., 2015).

With the notion of its crucial, detrimental, and missing activities, the evidence-based model constructed from our case study can inform (student) teachers and teacher educators who want to transform high-school STEM education into high-quality integrated STEM education. Since the model reflects a path towards iSTEM learning materials that was fully conceptualized by in-service teacher teams during this study, in- and pre-service teachers will naturally relate to these findings. Currently, this particular model together with the good practices and pitfalls found in the case study already form the basis of pre- and in-service teacher training for integrated STEM in several higher education and professional development programs in Flanders. The model and implications of this study provide (student) teachers with a common language and encourage them to engage in cross-disciplinary discourse, to search for ways to create competence-oriented, meaningful, and thus motivating STEM learning opportunities for their students.

LIST OF ABBREVIATIONS

iSTEM integrated Science, Technology, Engineering, and Mathematics

S secondary education with a focus on science

T secondary education with a focus on technology

Bio biology

Chem chemistry

Elec electricity-electronics

Math mathematics

Mech mechanics

Phys physics

Eng engineering

ACKNOWLEDGEMENTS

This study was carried out in the scope of the research project STEM@school, which was funded by the Agency for Innovation through Science and Technology (IWT) of the Flemish government (Belgium). We want to thank all teachers who participated in this study, and who worked tirelessly to design engaging and relevant STEM education for their students. We also want to thank Leen Goovaerts and Anthony Coyette for the important contribution they made to this study and for their invaluable support. Finally, we want to thank Marie-Paule Buyse, Stijn Ceuppens, Jan De Lange, Hanne Deprez, Yves Vanbilsen, and fellow teacher educators for promoting and advancing the results of this work.

APPENDIX 1 - BACKGROUND INFORMATION TDT MEMBERS

Abbreviations

iSTEM integrated Science, Technology, Engineering, and Mathematics

S secondary education with a focus on science

T secondary education with a focus on technology

F female

M male

Bio biology

Chem chemistry

Elec electricity-electronics

Math mathematics

Mech mechanics

Phys physics

Eng engineering

5B short-cycle tertiary education (ISCED level 5)

BEd bachelor in education (ISCED level 6)

BSc bachelor of science (ISCED level 6)

MSc master of science (ISCED level 7)

APPENDIX 2 - DETAILED DESCRIPTION AND EXAMPLES FOR EACH DESIGN ACTIVITY IN THE CASE-BASED MODEL

^aat the rate of 1/2 day per week (on average)

APPENDIX 3 - ACTIONS PER DESIGN ACTIVITY

(Process starts with a big, motivated group of STEM teachers)

TGI

• deciding for which target group of students (i.e., grade and study program) to develop

Outcome: the target group of the learning materials

SCA

• analysis of the curricula of the STEM subjects (science (or more specifically physics, chemistry, biology, geography), technology (or applied sciences, computer science), engineering, math) of the target group to determine:

  • the prior knowledge of the target group

  • the standards (curriculum goals) that must be achieved by the target group

    • extracting from these curricula the contents that must be learned by the target group

    • discussing these learning contents

Outcome: prior knowledge and STEM curriculum standards prescribed for the target group

BCT

• grouping/categorization of learning contents to identify overarching themes based on the separate STEM curriculum standards

• identifying/choosing overarching themes

• choosing a sequence in which the themes will be scheduled in the students’ school year based upon logical succession of learning contents

Outcome: themes for the learning materials, and a sequence in which to address them

TFD

• splitting up the big group of enthusiasts into “smaller”, multidisciplinary teams (TDTs) of teachers with differing discipline-specific knowledge, and differing classroom or practical experience in one or more of the STEM disciplines, with differing degrees and/or teaching in different grades, of different schools

• allocating the chosen themes to the formed TDTs

Outcome: small(er) multidisciplinary TDTs

(Big group of STEM teachers is now split up into teams of about eight teachers from different STEM disciplines)

CIL

• collaboratively listing the learning contents relevant to the theme per discipline (S, T, E, M),

• collaboratively establishing cross-disciplinary links between these contents,

• (when iterated) collaboratively adding learning contents to the list and cross-disciplinary links, necessary to solve the formulated challenge,

Outcome: list of learning contents from different STEM subjects with cross-disciplinary links

SFD

• dividing the relevant discipline-specific learning contents

• splitting up the team of teachers into small, monodisciplinary subteams

Outcome: small monodisciplinary subteams

CF

• collaboratively searching for a challenge

  • that is fun / meaningful to the target group

  • that includes purposeful design or inquiry

  • that is more or less authentic / situated in a real-world context

• (when iterated) collaboratively fine-tuning the challenge so that the targeted STEM contents will be addressed in a meaningful way and linked when solving

• (when iterated) collaboratively adjusting the challenge, taking into account the studied the requirements and feasibility (e.g., open endedness)

Outcome: a central student challenge

RFS

• consulting experts in the field of the chosen theme

• collaboratively identifying the requirements and constraints of the chosen challenge,

• collaboratively checking these constraints for allowing enough student creativity, and for open-endedness

• collaboratively checking the practical feasibility of the formulated challenge: making an inventory of the accommodation and equipment needed to solve the challenge and listing extra constraints based on what is (not) possible

Outcome: requirements and feasibility criteria for the student challenge

CBS

• collaboratively identifying the different subproblems encompassed by the challenge

Outcome: list of successive subproblems to be encountered when solving the challenge

DLA

• collaboratively determining what the target group will be doing: brainstorming cross-disciplinary learning activities per subproblem

  • that form the bridge between the challenge and the targeted learning contents;

  • that address and link learning contents from different STEM disciplines, e.g., via modeling

  • that rely on previously constructed learning contents

  • in which students are actively engaged: involving

    • theory processing and/or information gathering,

    • making exercises

    • programming

    • experimenting, performing an inquiry

Outcome: collection of learning activities

EPD

• designing a prototype, or

• trying out experiments related to the challenge

• ordering sample components / equipment

• keeping documentation:

  • codes, drawings, experiment results

  • required equipment

Outcome: experiment guidelines, prototypes, drawings, building plans, order lists, practical tips

REF

• presenting intermediary ideas and drafted learning activities to other TDTs (of the big group) and to other stakeholders

• listening/being updated about the other TDTs’ ideas and activities

• collaboratively giving feedback on the other TDTs’ ideas and activities

• collaboratively documenting feedback from the other TDTs on own ideas and activities

Outcome: feedback (suggestions for improvement/adjustment) of the materials

TPR

• collectively revising the initial scheduling of the themes (i.e. verifying if the learning contents are addressed in a logical sequence in the school-year scheduling of the themes)

• in case of doubts about the sequence: balancing pros and contras for switching the themes in the school-year schedule

• collectively deciding upon the final scheduling of the themes within the target group’s school year

Outcome: an approved theme planning

SW

• writing the scenario (script) for solving the challenge, based on the identified subproblems

• including scaffolding questions that successively introduce students to a new subproblem,
  and scaffolding questions that recapitulate what students (should) have learned when solving that subproblem and reconnect with the challenge

• identifying gaps among the outlined learning activities and communicating them to the TDT members

Outcome: script (scenario) for the learning activities, starting from the student challenge

SF

• elaborating the drafted learning activities, taking into account the feedback from the other TDTs

• gathering information based on the notes and information gathered during the experiment tryout and prototype design

Outcome: elaborated syllabi

MB

• bundling of the script and the syllabi

Outcome: a coherent collection of syllabi

SRF

• individually reviewing the syllabi

• fine-tuning the syllabi based on the feedback of the TDT members

Outcome: an approved set of iSTEM syllabi (the designed learning materials)

• Berland, L. K. (2013). Designing for STEM integration. Journal of Pre-College Engineering Education Research (J-PEER), 3(1), 3. https://doi.org/10.7771/2157-9288.1078
• Charmaz, K. (2006). Constructing grounded theory: A practical guide through qualitative analysis. Sage Publications Ltd.
• Coenders, F. and Terlouw, C. (2015). A model for in-service teacher learning in the context of an innovation. Journal of Science Teacher Education, 26(5), 451–470. https://doi.org/10.1007/s10972-015-9432-5
• Creswell, J. W. and Miller, D. L. (2000). Determining validity in qualitative inquiry. Theory into Practice, 39(3), 124–130. https://doi.org/10.1207/s15430421tip3903_2
• Dare, E., Ellis, J. and Roehrig, G. (2018). Understanding Science Teachers’ Implementations of Integrated STEM Curricular Units through a Phenomenological Multiple Case Study. International Journal of STEM Education, 5(1), 19. https://doi.org/10.1186/s40594-018-0101-z
• De Meester, J., Boeve-De Pauw, J., Buyse, M. P., Ceuppens, S., De Cock, M., De Loof, H., Goovaerts, L., Hellinckx, L., Knipprath, H., Struyf, A., Thibaut, L., Van de Velde, D., Van Petegem, P. and Dehaene, W. (2020). Bridging the Gap between Secondary and Higher STEM Education–the Case of STEM@ school. European Review, 28(S1), S135-S157. https://doi.org/10.1017/S1062798720000964
• De Meester, J., Knipprath, H., Thielemans, J., De Cock, M., Langie, G. and Dehaene, W. (2015). Integrated STEM in secondary education: A case study. Il nuovo cimento C, 38(3), 1-10.
• de Vries, M. J. (2020). Wicked Problems in a Technological World. Philosophia Reformata, 85(2), 125-137. https://doi.org/10.1163/23528230-8502A002
• Eisenhardt, K. M. (1989). Building theories from case study research. Academy of Management Review, 14(4), 532-550. https://doi.org/10.5465/amr.1989.4308385
• Felder, R. M. and Brent, R. (2016). Teaching and learning STEM: A practical guide. San Francisco, CA: John Wiley & Sons.
• Girvan, C., Conneely, C. and Tangney, B. (2016). Extending experiential learning in teacher professional development. Teaching and Teacher Education, 58, 129-139. https://doi.org/10.1016/j.tate.2016.04.009
• Guzey, S. S., Moore, T. J. and Harwell, M. (2016). Building up STEM: An analysis of teacher-developed engineering design-based STEM integration curricular materials. Journal of Pre-College Engineering Education Research (JPEER), 6(1), 2. https://doi.org/10.7771/2157-9288.1129
• Handelzalts, A. (2019). Collaborative curriculum development in teacher design teams. In Collaborative curriculum design for sustainable innovation and teacher learning (pp. 159-173). Springer, Cham. https://doi.org/10.1007/978-3-030-20062-6_9
• Hommes, B.-J. and Van Reijswoud, V. (2000). Assessing the quality of business process modelling techniques. In E. Dennis (ed.), Proceedings of the 33rd Annual Hawaii International Conference on System Sciences. Maui, Hawaii, 4 January, 1–10. https://doi.org/10.1109/HICSS.2000.926591
• Huizinga, T., Handelzalts, A., Nieveen, N. and Voogt, J. M. (2014). Teacher involvement in curriculum design: Need for support to enhance teachers’ design expertise. Journal of Curriculum Studies, 46(1), 33–57. https://doi.org/10.1080/00220272.2013.834077
• Ingvarson, L., Meiers, M. and Beavis, A. (2005). Factors affecting the impact of professional development programs on teachers’ knowledge, practice, student outcomes & efficacy. Education Policy Analysis Archives, 13. https://doi.org/10.14507/epaa.v13n10.2005
• Jacobs, H. H. (1989). The interdisciplinary concept model: a step-by-step approach for developing integrated units of study. In H.H. Jacobs (ed.), Interdisciplinary curriculum: Design and implementation. Alexandria, VA: ASCD.
• James, R. K., Lamb, C. E., Householder, D. L. and Bailey, M. A. (2000). Integrating science, mathematics, and technology in middle school technology-rich environments: A study of implementation and change. School Science and Mathematics, 100(1), 27–35. https://doi.org/10.1111/j.1949-8594.2000.tb17317.x
• Jeanpierre, B., Oberhauser, K. and Freeman, C. (2005). Characteristics of professional development that effect change in secondary science teachers’ classroom practices. Journal of Research in Science Teaching, 42(6), 668-690. https://doi.org/10.1002/tea.20069
• Jonassen, D., Strobel, J. and Lee, C. B. (2006). Everyday problem solving in engineering: Lessons for engineering educators. Journal of Engineering Education, 95(2), 139-151. https://doi.org/10.1002/j.2168-9830.2006.tb00885.x
• Kennedy, T. J. and Odell, M. R. L. (2014). Engaging students in STEM education. Science Education International, 25(3), 246-258.
• Kitchenham, B., Pickard, L. and Pfleeger, S. L. (1995). Case studies for method and tool evaluation. IEEE Software, 12(4), 52–62. https://doi.org/10.1109/52.391832
• Krajcik, J., McNeill, K. L. and Reiser, B. J. (2008). Learning‐goals‐driven design model: Developing curriculum materials that align with national standards and incorporate project‐based pedagogy. Science Education, 92(1), 1-32. https://doi.org/10.1002/sce.20240
• Loucks-Horsley, S., Stiles, K. E., Mundry, S., Love, N. and Hewson, P. W. (2009). Designing professional development for teachers of science and mathematics. Thousand Oaks, CA: Corwin Press. https://doi.org/10.4135/9781452219103
• McFadden, J. R. and Roehrig, G. H. (2017). Exploring teacher design team endeavors while creating an elementary-focused STEM-integrated curriculum. International Journal of STEM Education, 4(1), 21. https://doi.org/10.1186/s40594-017-0084-1
• Mooney Simmie, G. (2007). Teacher Design Teams (TDTs)—building capacity for innovation, learning and curriculum implementation in the continuing professional development of in-career teachers. Irish Educational Studies, 26(2), 163–176. https://doi.org/10.1080/03323310701295914
• Mortelmans, D. (2007). Handboek kwalitatieve onderzoeksmethoden [Manual qualitative research methods]. Leuven: Acco.
• NAE & NRC (2014). STEM Integration in K-12 Education: Status, Prospects, and an Agenda for Research. Washington, DC: The National Academies Press. https://doi.org/10.17226/18612
• NRC (2012). Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering. Washington, DC: The National Academies Press. https://doi.org/10.17226/13362
• Parke, H. M. and Coble, C.R. (1997). Teachers designing curriculum as professional development: A model for transformational science teaching. Journal of Research in Science Teaching, 34(8), 773–789. https://doi.org/10.1002/(sici)1098-2736(199710)34:8<773::aid-tea2>3.0.co;2-s
• Satchwell, R. E. and Loepp, F. L. (2002). Designing and implementing an integrated mathematics, science, and technology curriculum for the middle school. Journal of Industrial Teacher Education, 39(3). https://eric.ed.gov/?id=EJ782300
• See, B. H., Gorard, S. and Siddiqui, N. (2016). Teachers’ use of research evidence in practice: a pilot study of feedback to enhance learning. Educational Research, 58(1), 56-72. https://doi.org/10.1080/00131881.2015.1117798
• Shernoff, D. J., Sinha, S., Bressler, D. M. and Ginsburg, L. (2017). Assessing teacher education and professional development needs for the implementation of integrated approaches to STEM education. International Journal of STEM Education, 4(1), 13. https://doi.org/10.1186/s40594-017-0068-1
• Stake, R. E. (1995). The art of case study research. Thousand Oaks, CA: Sage Publications.
• Stanovich, P. J. and Stanovich, K. E. (2003). Using research and reason in education: How teachers can use scientifically based research to make curricular & instructional decisions. Partnership for Reading (Project), National Institute for Literacy, US Department of Education. https://doi.org/10.1037/e563842009-001
• STEM@school (2017). Our learning modules. Available at http://www.stematschool.be/en/our-learning-modules (Accessed 12 April 2021).
• Stolk, M. J., Bulte, A. M. W., Jong, O. D. and Pilot, A. (2016). A Framework for Empowering Teachers for Teaching and Designing Context-Based Chemistry Education. In R. Taconis, P. Brok, A. Pilot (eds.), Teachers Creating Context-Based Learning Environments in Science (pp. 191-211). Rotterdam: SensePublishers. https://doi.org/10.1007/978-94-6300-684-2_11
• Thibaut, L., Ceuppens, S., De Loof, H., De Meester, J., Goovaerts, … and Depaepe, F. (2018). Integrated STEM education: A systematic review of instructional practices in secondary education. European Journal of STEM Education, 3(1) 2. https://doi.org/10.20897/ejsteme/85525
• Van Driel, J. H. and Verloop, N. (1999). Teachers’ knowledge of models and modelling in science. International Journal of Science Education, 21(11), 1141–1153. https://doi.org/10.1080/095006999290110
• van Keulen, H., Voogt, J., van Wessum, L., Cornelissen, F. and Schelfhout, W. (2015). Professionele leergemeenschappen in onderwijs en lerarenopleiding. [Professional Learning Communities in education and professional development]. Tijdschrift voor lerarenopleiders, 36(4), 143-160.
• Voogt, J., Westbroek, H., Handelzalts, A., Walraven, A., McKenney, S., Pieters, J. and De Vries, B. (2011). Teacher learning in collaborative curriculum design. Teaching and Teacher Education, 27(8), 1235–1244. https://doi.org/10.1016/j.tate.2011.07.003
• Woods, D. R. (2000). An evidence‐based strategy for problem solving. Journal of Engineering Education, 89(4), 443-459. https://doi.org/10.1002/j.2168-9830.2000.tb00551.x
• Yin, R. K. (2003). Case study research: design and methods (3rd edition). Applied Social Research Method Series (Vol.5). Thousand Oaks, CA: Sage Publications.