INTRODUCTION

Worldwide there has been an historical decline in the number of college students pursuing engineering and science undergraduate degrees, especially in industrialized nations (NSB, 2018; OECD, 2014; Pretz, 2016). For example, the Organisation for Economic Co-operation and Development reported that only 15% of students are awarded engineering degrees in OECD countries whereas 32% are awarded degrees in social science programs (OECD, 2014). While these numbers have started to rise, this is extremely discouraging for countries as many reports have shown that there is and will remain a shortfall in the number of engineers available to fill highly technical jobs for years to come (e.g. Becker, 2010; Holdren et al., 2010; Sjøberg and Schreiner, 2010).

To help mitigate these statistics, science, technology, engineering and mathematics (STEM) projects in schools introduced consistently from early childhood onward could help to shift not only student interest in STEM but also enhance self-efficacy with respect to STEM careers (Parker, 2018). Ultimately, this strategy might encourage young people to consider careers in engineering. Positive shifts in interest have been observed internationally when incorporating engineering projects into secondary schools (Malone, Schuchardt and Schunn, 2018) and informal outreach programs to K12 schools (Johnson et al., 2013; Karp and Maloney, 2013; Shahali et al., 2017). The incorporation of engineering at the early elementary school level (students five to 10 years old) has included the use of engineering picture and trade books to introduce engineering challenges (Milto et al., 2016; Pantoya et al., 2015), engineering design projects (Lachapelle and Cunningham, 2014; Silver and Rushton, 2008b), and animated videos (Johnson et al., 2013). The lack of inclination to become an engineer may be caused by the negative stereotypical images held by students rather than enjoyment of science and engineering (Silver and Rushton, 2008a; Wang and Degol, 2017). Alternatively, lack of interest in engineering careers might be simply caused by the lack of awareness of what engineers do. Several studies have shown that elementary students consider engineers to be laborers, technicians, or train drivers (Capobianco et al., 2011; Chou and Chen, 2017). The question may be how to introduce engineering to young children in a way that brings forth their creativity and helps to develop their identity as budding engineers while also helping them to understand the concepts of engineering and technology.

A possible method that would develop young children’s identity as potential engineers is the use of dramatic inquiry (DI), a dialogic inquiry and dramatic play-based pedagogy (Edmiston, 1998; Edmiston, 2014) and other artistic approaches such as art and dance. Studies have shown that during play students exhibit behaviors that are consistent with those used during the engineering design process (Gold et al., 2015; Park et al., 2018) and earlier engineering play can help them to be more successful in later engineering challenges (Tõugu et al., 2017). Tõugu et al. (2017) asked families with young children in a museum setting to solve an engineering design problem together using spatial play. The children were then asked to solve a second problem on their own and the researchers found that the children experienced even greater success completing the second engineering design problem after first experiencing play and engineering design. The exploratory study described in this paper is a first attempt to fill a gap in the literature by determining the effect of integrated STEAM (i.e., science, technology, engineering, art and mathematics) units incorporating engineering design process units infused with dramatic inquiry, art, and/or dance on student knowledge of engineering and technology in early primary school, ages 4 years to 8 years old (i.e., PreK3).

ENGINEERING IN THE ELEMENTARY STEM CONTEXT

The incorporation of engineering in elementary schools has been on the rise in the last decade. Several studies have shown positive benefits for students when engineering design challenges are incorporated into elementary classrooms (e.g. Guzey et al., 2017; Toma and Greca, 2018; Wendell and Rogers, 2013). The common dominator amongst these research-based projects has been the use of engineering design challenges.

One of the most widely used interventions is the Engineering is Elementary (EiE) project developed in the USA at the Boston Museum of Science (Cunningham and Lachapelle, 2014). The EiE units use guided inquiry to assist students in designing solutions to challenging engineering problems. The units focus on having students use the engineering design process that includes brainstorming ideas, researching, and gathering information in STEM to help solve a problem, test solutions, and then improve a selected solution. This improvement of the solution allows students to confront failure (or partial success) thus supporting the conception that failure is not a reason to quit but to continue to improve on their solutions (Johnson, 2016). A further description of the units is in the methods section.

Research has shown that use of EiE units can produce an improvement in student knowledge of science (Lachapelle et al., 2017), engineering and technology (Lachapelle and Cunningham, 2007) as well as foster increased student interest (Lachapelle and Cunningham, 2017) and identity in engineering (Capobianco et al., 2015) while decreasing negative perceptions of engineers (Rynearson, 2016). However, most of these studies focused on elementary students enrolled in the third grade and up. Only a handful of studies focus on younger grade levels. Lachapelle and Cunningham (2007) did include second graders in a study looking at changes in understanding of engineering and technology of students in grades 2 through 6. Aquirre-Munoz and Pantoya (2016) and Kelly, Cunningham and Ricketts (2017) demonstrated that the use of engineering design challenges resulted in increased engineering related discourse in classrooms with young children. These studies suggest that continually referring to the students as engineers allowed for increases in student engineering identity. In conclusion, there is a definite paucity of engineering design studies at the early elementary level.

ENGINEERING IN THE ELEMENTARY INTEGRATED STEAM CONTEXT

Focusing only on STEM in elementary classes ignores eliminates the positive aspects that inclusion of the arts can bring to the engineering design process. The arts (i.e., dance, visual arts, music and drama) allow students to use other modes of expression such as kinesthetic and physical models that can facilitate learning in science and engineering (Hadzigeorgiou, 2016). In addition, the artistic design process is very similar to the engineering design process (Bequette and Bequette, 2012). Steele et al. (2006) studied the incorporation of dance into a 5^th grade engineering unit focused on energy and wind turbines. They found that the students were able to interpret the transfer of energy in terms of dance movements while the teacher reported that student engagement in science had improved. However, student knowledge of engineering and technology was not assessed.

The visual arts also have a positive impact on engagement and student mental development when incorporated with their science lessons as shown by a Malaysian case study conducted in 3^rd grade classrooms (Dhanapal et al., 2014). Jakobsen and Wickman (2015) studied two learning sequences in a 1^st grade class studying leaves during a science lesson. In the first sequence, the students studied the leaves using a magnifier while in the second they made pictures of the leaves using a rubbing technique. The authors found that the artistic sequence enhanced the students’ meaning-making of the science concepts being studied.

One dramatic inquiry pedagogical approach known as ‘Mantle of the Expert’ positions students as if they are professional people with expertise who need to engage in inquiry to complete a commission (Heathcote and Bolton, 1995). Over time, students take on the “mantle” of “experts” as they dialogue while engaging in the sort of authentic inquiry that professionals engage in. Two case studies in 4^th and 5^th grade classrooms that incorporated this dramatic inquiry approach in science classes in New Zealand and the UK found that the students were more engaged and enthusiastic about their science classes (Fraser et al., 2012; McGregor, 2017). The use of dramatic inquiry in science classrooms suggests that students develop a deeper understanding of the science concepts in an inquiry oriented classroom where the students are in greater control of meaning-making (Odegaard, 2003). This idea was supported by a case comparison using dramatic inquiry with 2^nd graders learning about the science of snails (Warner and Anderson, 2004). In this study one group was taught with scientific inquiry only methods while the second infused dramatic injury. While the two groups learned the same scientific content, the DI group was much more focused, engaged in more dialogue, and made more detailed observations during the final writing task when conceptualizing scientific ideas. During authentic scientific inquiry instruction students need to engage in discursive Dialogue with their peers (Ford and Wargo, 2011). Thus, research shows that dramatic inquiry seems to be able to create pedagogical conditions that allows for an increase in student dialogue about science which might enhance their identity as scientists but none report gains in understanding of technology and engineering. In addition, the use of dramatic inquiry to teach engineering concepts does not appear to have been reported in past literature.

The components of dance, visual arts, music and drama may be associated with positive benefits when individually incorporated into science classes. Graham and Brouillette (2016) studied the combined effects of these techniques in the 3^rd to 5^th grades (8 to 10 year olds). They developed an integrated arts/science program called San Diego Teaching Artist Project (TAP) and evaluated its effects using quasi-experimental methods. This program integrated dance, drama, music, visual arts and science within nine hours of instruction. The researchers determined that the students in the TAP program outperformed the control group in terms of knowledge of scientific concepts and teachers reported higher levels of engagement in science over that of the control teachers. Thus at upper elementary grades the incorporation of artistic components can enhance scientific knowledge as well as increase engagement in science when used separately or when incorporated together in a format we will call integrated STEAM (science, technology, engineering, arts (visual, dance, drama) and mathematics.

In summary, while current research concerning the use of integrated STEAM (i-STEAM) in the elementary classroom is present in the literature, the majority of the results are anecdotal in nature with very little integration of the arts with engineering design and dramatic inquiry. In addition, very few studies focus on early childhood years (4 – 8 years of age). This study seeks to contribute robust findings to the literature in these areas by assessing the effects of i-STEAM units that incorporate dramatic inquiry, dance, visual arts and engineering design challenges on students’ knowledge of and interest in engineering and technology in early childhood classrooms.

GOALS AND RESEARCH QUESTIONS

Research suggests that there should be positive implications on student concept knowledge by including drama and other arts (e.g., dance, 2 and 3-dimensional art, etc.) into an i-STEAM unit, especially at the younger age groups. This study attempts to fill a void by evaluating the use of curriculum grounded in the engineering design process that incorporates the use of dance, artistic design, physical education and dramatic inquiry (i.e., integrated STEAM) in PreK3 classrooms.

The goal of this study was to (1) examine the effects of the STEAM unit on students’ conceptual understanding of engineering and technology; and (2) determine teachers’ views about student engagement within the context of the i-STEAM unit. To meet the goals of this study a mixed methods approach was designed that included pre/post student assessments about engineering and technology as well as pre/post teacher interviews. The pre/post assessments evaluated students’ knowledge about what engineers do in real life (i.e., fix cars vs design bridges) as well as what types of objects are considered technology (i.e., lightening vs forks) by engineers. The following research questions guided the study:

1. How do integrated STEAM units impact PreK3 student conceptual understanding of what engineering is?

2. How do integrated STEAM units impact PreK3student conceptual understanding of technology?

3. How do integrated STEAM units impact PreK3student interest in engineering?

METHODS

Participants and Settings:

This study targeted five PreK8 schools in an urban district located in a Midwestern state in the USA. The schools targeted were low achievers in science according to standardized testing earning performance grades of D or F and considered to be struggling schools by their district. After obtaining permission from the targeted school’s principals to participate in the project, teachers were recruited to be part of the study. The recruited teachers included self-contained classroom teachers as well as subject specialists. The self-contained classroom teachers usually taught mathematics, science, social studies, literacy and reading to their students. Some of the specialist teachers taught dance, art and physical education (PE) to the students for a few hours each week depending on each individual school. Some reading and mathematics specialists were also part of the teacher team and served as consultants for the classroom teachers within the area of these two subjects. School demographic data is in Table 1. The table includes not only the ethnic background of students but also the percentage of students considered proficient in math at grade 3 and in science at grade 5.

In these five schools, a total of 14 teachers participated in the study from PreK to 3^rd grade. Convenience sampling was used since the teachers were volunteers and their students were assigned to them by a registrar. Table 2 denotes the number of students at each grade level who participated in the engineering part of the study as well as the numbers that participated in the technology portion of the study in order to answer research questions 1 and 2.

Classroom Materials and Practices

STEM Instructional Units - Engineering is Elementary

The instructional units from Engineering is Elementary (EiE) were chosen as the basis for our i-STEAM curriculum units (Cunningham and Lachapelle, 2014). The EiE units were designed by interdisplinary teams to incorporate engineering design as well as science standards. The units were aligned with standards across all states in the United States as well as the United States’ Next Generation Science Standards (NGSS). The NGSS specifically has standards for both engineering design challenges as well as science standards (NGSS Lead States, 2013). These units utilize story books designed to engage the students in an engineering problem. The books set up the context of the problem and introduce the engineering field as well as relevant vocabulary. In one example unit, the story is situated in the Netherlands and the students are presented with the need to design a windmill to aerate fish ponds. The multicultural settings of the story books support both student literacy, as well as world geography.

This introduction leads to a second lesson that exposes them to the engineering field being investigated. For example, in the case of the windmill unit the students are introduced to egg beaters, glue sticks and mechanical pencils. They are encouraged to consider what a mechanical engineer does and how they might use machines to design a windmill. Next the students are presented with the engineering design process similar to that shown in Figure 1.

Figure 1. Engineering Design Process developed by Kristy Rothbrush and Whitehall City School District teachers

The students are then introduced to the materials and science concepts needed to be successful in the engineering design project. For example, in the windmill unit the students are given kits with different samples of materials that they can use to investigate how wind would affect the materials when they are used to catch the wind. They are encouraged to build and design sails with the varied materials. The students can then test their sails by fixing them onto Styrofoam food trays which can be set in a small tub of water to see how the wind from a fan affects their sails. At this point they also learn basic physics about how the angle of the sail relative to the wind can affect how fast the sailboat moves across the water. Based on the results of these activities the students can choose the materials they think will work best on a windmill blade.

In the last lesson, students work on designing the windmill blade. To test the blade, it is attached to a dowel rod that can be turned as the blade spins. The dowel rod is attached by a string to a cup. The goal is for the windmill blades to lift as many bolts placed in the cup as possible. After the students build their first prototype, they test it. After the initial testing they are encouraged to identify strengths and weaknesses of the initial prototype and then to improve their design. Thus, the students experience failure and revision as normal parts of the engineering design process.

The books that come with the EiE units require a reading proficiency well above the ability of many of the students in our high needs PreK3 classrooms. Teachers can use many scaffolds to introduce the story to their students including reading only sections to the students, using gallery walks with pictures of the story, and using projected images as the story was read out loud.

Dramatic Inquiry in the Context of the Integrated STEAM Units

Dramatic inquiry was presented to the teachers during their professional development days as an alternative to using the books as an introduction to the units. Teachers using dramatic inquiry positioned the students as if they were already engineers with expertise who were presented with a problem to solve. Through dramatic playing, students took on the role of an engineer. In doing so there was a significant shift in the classroom atmosphere to become both playful and exploratory as well as seriously focused on professional inquiry.

One example of the i-STEAM units incorporation of dramatic inquiry occurred within the EiE windmill unit discussed in the last section. For this unit the students were introduced to The Boy Who Harnessed the Wind (Kamkwamba and Mealer, 2012). This true story tells about a young boy in Africa whose village suffered a severe drought. By positioning students as if they were designers in the story world, they took on more ownership of the problem as they engaged in the design challenge. An adult, sometimes the teacher, would take on the role of the young boy, William from Africa, and have students posing as engineers ask questions about the problem his village was experiencing. The village did not have electricity to run irrigation pumps thus their harvest each year was minimal. However, the village usually had a strong wind current. Thus, William wanted to harness the wind to produce electricity. After participating in the asking phase of the engineering design process the teacher would give the young engineering teams the challenge of helping the village by producing a windmill that would harness the wind in order to run the irrigation pumps. The student teams are now part of William’s project serving as the engineers who will help the village produce more crops. As engineering teams the students test out different materials that can be used, choose the materials, then design and test their windmill prototypes. These prototypes are then improved based on the data the students collect concerning how well they functioned.

A second example dealt with bridge building. The students were positioned as if they were a team of scientists with expertise in rebuilding after natural disasters. The problem they would encounter was introduced via YouTube: the tragedy of Typhoon Haiyan (2013) in the Philippines. Students created a model of one of the islands in the Philippines archipelago using colored paper (brown to represent land and blue to represent sea) and blocks. Blocks were used to make bridges connecting the islands, hospitals, and airports. Making the sounds and movements of weather and buildings, the students imagined what might happen to the islands when the typhoon arrived. Students then moved the blocks to show how bridges between islands might be damaged and how buildings might collapse. Embodying what might have happened to people inside the buildings the students demonstrated the destruction of the typhoon. They were then asked to brainstorm what help might be needed after the storm. Students suggested hurt people needed transportation to the hospital and food from the airport, but with the bridges broken this would be difficult. With the problem clearly defined, students were then assigned to engineering teams as if they were working for an international group, Engineers without Borders. The engineering teams conducted activities with a variety of materials to determine how much mass diverse types of bridge designs could hold. Then students used the materials they predicted would work the best to build a bridge prototype. The bridges were built and tested to failure. Students then returned to their designs to improve their bridge prototypes to hold as much weight as possible. Using a metric for ‘best bridge’, student engineers selected what they identified as the best bridges to help the people of the Philippines.

A kindergarten example situated children as experts in playdough. They were given the problem to chemically engineer the best possible playdough for a schoolwide STEM night. A group of parent volunteers arranging the night came to the class and asked the students to participate in helping them make a recipe for playdough. The students were shown the parents failed low quality playdough and were asked to help improve upon it. The students then took on the mantle of chemical engineer and worked to formulate the best playdough possible. They developed a rubric of high quality versus low quality playdough and as a class decided on the three best playdough recipes. They presented these formulations to the parent committee who as the clients chose the one they liked best. The students mass produced the playdough for sale at the STEM night. They had to calculate the cost of the materials and decide upon a price that would allow for some return on the investments. To decide the colors for the playdough they would sell they decided as a group to survey the parent committee (see Figure 2). These students took this task very seriously and were engineers in the true sense of the word.

Figure 2. Student graph of survey data to determine the most popular playdough color (Kindergarten Class)

In these examples, the students imagined they were interacting in a real-life engineering context. In all three cases, in their bodies and minds they could ‘become’ an engineer and develop innovative technologies to help people. This allowed the students to take up the professional position of an engineer who works to improve other peoples’ lives through technology.

Using Artistic and Physical Education Specialists to Produce Integrated STEAM Units

Normally, the elementary teacher in a self-contained classroom does not interact with the arts specialists at the school. In the i-STEAM unit, the teachers put into place truly integrated units where both specialist and classroom teachers would participate in an explicit manner to assist students in achieving their engineering goal.

Dance and art teachers connected with the engineering design process as it was very close to that of the design process that is used by art professionals. The dance teachers developed interpretative dance lessons to go along with the units being studied. For example, in the case of the windmill the dance teachers designed an activity that asked the pupils to interpret what would happen to the windmill when wind with different amounts of energy encounters the windmill blade. The students were divided into small groups to develop their dance that they would then perform for the class. The class would need to interpret what the dance meant in terms of the windmill unit and the dance assignment. In another unit on designing playdough in the classroom, the students determined a rubric of high quality and low-quality playdough. When the students went to their dance class, the dance teacher would ask them to develop an interpretive dance that would demonstrate the differences between high quality and low-quality playdough. Students would create dances that would show how the high and low-quality playdough moved.

In the art class the teacher would help students incorporate drawings and sculpture into the unit. For example, in the bridge unit the art teachers were the ones who helped the students’ think about the aesthetics of the bridge and ultimately design the bridge models. In another unit using casts, the art teacher taught the students about casting and then the classroom teacher connected that knowledge to the engineering design challenge they were working towards.

The PE teachers in the schools were also active in the integration of the units. The activities they designed for each unit had the students kinetically becoming involved in the activity. To support a unit on parachute design, the PE teacher introduced the students to juggling. She allowed the students to use a number of materials. However, the students had to predict based upon the material testing that occurred during the implementation of their i-STEAM unit inside their self-contained classroom which materials would be best to use when learning to juggle. Thus, the students were able to put into play in a new context their knowledge of hang time.

Teacher Professional Development

The Teacher Professional Development program included two distinct tiers. In the first tier, a team of teachers and university personnel were sent to the Science Museum of Boston to complete a training course to become certified EIE teacher leaders. During this four day experience, the leadership team met each other and began the process of planning the summer PD programs. In the second tier of the PD program, the EiE Leadership team planned and implemented both the summer PD program and year-long follow-up sessions for teachers at their schools. The other members of the team consisted of faculty from science education (one of whom was a EiE certified Teacher Educator), drama education, science, and engineering departments. This interdisciplinary team was able to confirm that the final i-STEAM units developed incorporated the appropriate science, technology and engineering.

The first week of the Summer Institute focused on training the teachers in engineering design projects using an EiE unit led by the teacher leaders from their schools. The participants learned about technology, mathematical problem solving, data analysis, and graphing as well as engineering and scientific practices. These workshops stressed hands-on learning with the participants in student mode. Student mode facilitated the urban school district participants’ learning of the content focus as well as introducing them to the hands-on active pedagogy that stresses engineering and scientific practices and increasing student discourse. The workshop incorporated teacher mode discussion as they negotiated how they would implement the units as well as training in the use of Dramatic Inquiry. The first part of the summer workshop culminated with the participants working within school groups to choose the EiE modules that align with their state science standards based on grade band as well as cultural fit for the diverse learners at each school site.

The second week of the summer workshop occurred right before the start of the school year. At this session the teachers worked in school based teams to determine how to use the units at their schools as well as to start incorporating the arts into the EiE units. The Summer Institutes were offered in two experiences in an attempt to allow for teacher reflection throughout the summer so that many of their concerns could be answered before the start of the school year. Thus, at the beginning of the second session, teachers had the opportunity to pose questions and concerns that needed to be addressed regarding implementation of the units in their classrooms. During the follow-up year the participating teachers met several times after school and as a whole group on three Saturdays.

At the Saturday follow-up meetings, the participants presented a summary of the implementation of the units in their classrooms and their successes and challenges as well as worked on new ways to incorporate DI into existing EiE units. At the end of the school year cohorts discussed how they could utilize the knowledge of student learning to reflect back over the year to determine what they could do in the future to improve the implementation at each school.

SCHOOL IMPLEMENTATION

During the school implementation the teacher leader at each school met with teachers regularly to discuss implementation issues and to maintain fidelity of implementation of the units at after school meetings. The graduate students on the project also visited the schools to observe the implementation of the units as well as assist in any issues. The graduate students did not participate in the actual teaching of any of the units. In addition, the science education faculty visited the schools throughout the year of the implementation.

Each teacher completed at least one i-STEAM unit. The units and the engineering focus was quite broad. Table 3 lists the engineering and conceptual focus of units implemented by grade levels, at times units were used for multiple grade levels. The units covered many different concepts so they were taught by teachers when it was appropriate to do so. All of the units followed the EiE format described in an earlier section of this paper but some teachers taught the unit straight through all day for several days while others only taught parts of the unit per day or even per week thus stretching the unit over several weeks. The engineering content was consistent across implementations of the same unit, but the length of time varied with each teacher. The science content changed based upon grade level. This was the first year that all the teachers, including teacher leaders, had implemented these types of i-STEAM units. However, two of the teachers in this study were teacher leaders at their school and did experience the three-day EiE training. The teacher leaders taught in two different grade levels, second and third. The third grade teacher leader had prior training in dramatic inquiry.

Research Tools

Student Conceptual Assessments – What is Engineering? and What is Technology?

The students were given two assessments to test their conceptual understanding of engineering and technology. The assessments were designed to be given prior to the unit as well as a post assessment. If students were absent for either assessment they were not asked to complete them. The teachers were asked to give the assessments a week before and a week after the unit was taught. The units were taught at various times throughout the school year to best align with the school curriculum. Teachers selected units appropriate for their grade levels and curriculum needs. Since students at different grade levels participated, a variety of EiE units were taught. As a result, no common conceptual assessment was produced to assess students on science content. However, all students completed the pre and post instruction engineering and technology assessments.

The engineering assessment utilized for the study was the “What is Engineering?” assessment produced by the Museum of Science for the EiE project since it had been tested and revised (Lapachelle and Cunningham, 2014). This instrument was initially assessed using 1126 third and fourth grade students attending public schools located in a midwestern state located in the United States. Each survey item showed a picture and a textual description of an activity kids might think an engineer would engage in. The students were asked to circle “yes” if an engineer would do that or “no” if an engineer would not. The yes and no answers were converted to a 1 or 0, respectively. These scores were totaled and used in the factor analysis as well as being analyzed for internal reliability. The Cronbach alpha score determined for this assessment was 0.881. An exploratory factor analysis was conducted by Lachapelle and colleagues’ (2012) using a principal components analysis (PCA) with oblimin rotation. The factors determined by the factor analysis included works with electronics (i.e., develops cell phones), works with construction (i.e., roofs houses), works with non-electronic (i.e., develops better bubble gum), works with the environment (i.e., designs ways to clean polluted air), repairs or installs items (i.e., installs cable), or industrial (i.e., driving a truck). These factors accounted for 49.37% of the variance they observed.

The final assessment presents students with 19 activities engineers can participate in. For example, the students see both a picture of a person driving a truck with the words “drives a truck” under the picture. The students then are asked to circle which pictures contain an activity in which engineers would engage. The following activities are examples included on the assessment:

1. Develop better bubble gum

2. Steer ships

3. Fix headlights on a car

4. Drive trains

5. Develop smaller cell phones

6. Invent warmer kinds of cloth

A number of these highlight known misconceptions that students have about engineering. For example, draw an engineer studies have shown that student’s think that engineers would drive trains or build houses (Capobianco et al., 2011).

The technology assessment utilized for the study was the “What is Technology?” assessment produced for the EiE project since it had been used to compare STEM student performance (Lachapelle and Cunningham, 2007). This instrument underwent validity and reliability testing with elementary students (Lachapelle et al., 2013). The instrument was judged to have strong construct validity. To test for reliability paired pre and post assessments were collected from 550 students in grades 3 to 5. The students took the post assessment after completing an EiE STEM unit appropriate for their grade level. The pre assessment Cronbach’s alpha was 0.729 while the post assessment alpha was 0.858.

The technology assessment used in this study has a set of 20 pictures. The pictures consist of items that may or may not be considered as examples of technology. The students are asked to circle which items are examples of technology. The following items are examples from the assessment:

1. Wind-up toy

2. Bird

3. Volcano

4. Fan

5. Windmill

Since the ability levels of students may not have allowed for easy reading of either of these two assessments, teachers could opt to read the descriptions of the activities and names of the technology aloud to the students and have them select yes or no. In lower level classrooms, a card sort was provided as another option for assessing preliterate children. The card sort had detailed pictures of either activities that engineers might participate in or pieces of technology or non-technology placed on 3x5 cards. Students were asked to place the cards that showed an activity an engineer might do on the “yes” card or the “no” card. The results were then tallied by counting the number of cards in each location that were correctly placed and well as the types of activities. The outcomes of both of these types of assessments in the younger grade levels were combined with other scores in the final scores seen in Table 3 and 4.

Teacher Interviews

A subset of the participating school’s teachers’ were interviewed in the context of focus groups prior to the professional development they attended in the summer and again after the implementation year. The interviews were semi-structured such that there were a standard set of questions used for both pre and post interviews, but the interviewees also guided where the interview went based on their comments.

The interviews were conducted in focus groups consisting of one to five teachers. The focus group format allowed teachers who were accustomed to working in an integrated fashion to provide more detail about their experiences. The focus group interviews were recorded and transcribed. All transcriptions were checked for accuracy by the researchers.

All interview transcripts were coded by members of the project team using a grounded theory approach (i.e., constant comparative method) where meaning was derived from the data (Glaser and Strauss, 1967). The open-coding consisted of segmenting the transcript into sentences, selecting each sentence and commenting with a word or phrase that captured the sentence meaning. After coding (one pre-transcript and one post-transcript), a computer macro developed by Fredborg (2013) was used to extract coded segments into a separate document. Initially, two researchers completed this task and then were able to compare their individual coding to find similarities in the transcripts. This process led to the creation of a codebook, consisting of sixteen codes with corresponding definitions (Porter, West, Kajfez, Malone and Irving, 2018).

This initial code book was then turned over to two different researchers who individually attempted to use the codebook with two additional transcripts after they had been re-segmented into idea chunks. During this process the researchers developed additional codes and sub codes that they thought should be added to the codebook using grounded theory (Strauss and Corbin, 1990). After they discussed the transcripts they agreed on deleting several main codes and added sub codes. In addition, the definitions were expanded. The final codebook had 12 main codes with 20 sub codes.

These two inter-raters then coded all seven of the transcripts. Their coding was compared for inter-rater reliability using Cohen’s Kappa. The Cohen’s Kappa for all the transcripts was 0.723, an adequate level for analysis. However, the inter-raters then compared any codes upon which they disagreed and arrived at a consensus on all codes. Thus, the final agreement was 100%. For this paper a sub section of the codes was analyzed to answer the research questions. Table 4 details the five main codes utilized in this analysis, as well as their respective sub-codes.

FINDINGS

Conceptual Assessment: What is Engineering?

Five schools and 14 teachers participated in the pre/post assessment entitled: What is Engineering? Statistical tests were performed between the average pre and post test scores to determine the effect of the i-STEAM units on student conceptual understanding of what engineers do. Paired t-tests were conducted for each grade level. The results were significant for all five grade levels. Through the use of a Games-Howell test, it was determined that the pre tests across grades were not statistically significant from each other. The effect sizes in each grade level are considered large (i.e., a shift of 0.8 of a standard deviation) with the smallest effect of 0.71 showing a high moderate effect (Cohen, 1988). The overall findings from the five elementary schools are presented in Table 5.

The assessment items were categorized based upon the groupings determined by Lachapelle and colleagues’ (2012) factor analysis study discussed in the methods section. The factors analyzed in this study include the following: works with electronics (i.e., develops cell phones), works with construction (i.e., roofs houses), works with non-electronic (i.e., develops better bubble gum), works with the environment (i.e., designs ways to clean polluted air), repairs or installs items (i.e., installs cable), or industrial (i.e., driving a truck). In Figure 3 the pre and post scores for each category can be seen by grade level. Figure 3 demonstrates that there was improvement in students’ conceptual understanding of engineering and across grade levels a greater percentage of students considered engineers to work with not only electronics but also non-electronics and the environment. However, student misconceptions about what an engineer does seem to be difficult to shift since students continue to believe that engineers repair cell phones, build roofs and drive trucks for a living. In many cases, these particular misconceptions were enhanced by the units.

Figure 3. Student Conceptions about What Is Engineering?

Conceptual Assessment: What is Technology?

Four schools and 11 classrooms participated in the pre/post assessment entitled: What is Technology? The students were enrolled in kindergarten through the 3^rd grade. Paired t-tests using pre and post assessment scores were conducted for each grade level. The results were significant for all four grade levels. A Games-Howell test was used to determine if the pre test scores were significantly different across grade levels. The pre tests across grades were not statistically different from each at the p<0.01 level. The pre to post effect sizes in each grade level except for the 2^nd are considered large effects (Cohen, 1988), with the 2^nd grade demonstrating only a moderate effect size. The number of students from each grade level can be seen in Table 6 along with their average pre and post test score.

The technological and non-technological objects featured on the assessment were grouped for analysis into three categories as used by Lachapelle et al. (2013) in a study with grades 3 -5. The three categories were electrical technology, non-electrical technology and natural non-technology items and Table 7 shows the items in each category.

Figure 4 shows that at all grade levels the student post test ideas about technology move towards more correct conceptions and at all grade levels the students are mostly correct about non-technology and electrical technology. At the time of the pre test, students at all levels have less than a 30% chance of recognizing these items as a type of technology. These findings are similar to technology research with 3^rd through 5^th graders (Lachapelle et al., 2013; Solomonidou and Tassios, 2007). The students had the most difficulty classifying the non-electrical technology items at all grade levels. However, the third graders experienced the greatest jump in this area with a 49% increase in ability to classify non-technological items. However, the third graders also had a much lower pretest in this category. Therefore, to further analyze this situation the average normalized gain was calculated for each technology sub-score by grade level (Hake, 1998). These gain scores are in Table 8. The normalized gain was chosen since it differentiates between the amount students learned vs how much they were able to learn based on their pre test scores. Thus it is independent of the pre test score at each grade level. The normalized gains over 0.4 can be seen to occur at different points in each grade. For example, kindergarteners (5 year olds) have the greatest shifts in categorizing electrical technology, first graders (6 year olds) have the greatest change in natural and electrical, while second graders experienced the change in the natural category. However, the third graders greatest gains are in the category or electrical and non-electrical. The third graders actually experienced a drop in the natural category. Unfortunately, the technology assessment items were not evenly divided between the categories with only 4 items in the natural category, 5 in the electrical category and 11 in the non-electrical category.

Figure 4. Pre and post Technology Scores by Grade level and category type

If the non-electrical technology is further categorized into clothing, household technology (i.e., broom and basket) vs mechanical (i.e., wind-up toy) technology the findings across grade level can be seen in Figure 5. When comparing these non-electrical technologies, one finds that students have a tough time distinguishing that clothing and household objects can be technology. Mechanical objects are easier for them to recognize as technology. However, by third grade the students seem to have a stronger grasp of these concepts on the pre assessment compared to the other grade levels. However, there was a large decline in third graders classifications of household objects pre to post. Although the i-STEAM units improved students’ conceptual understanding of technological vs non-technological objects overall; the findings suggest that students are still having difficulties distinguishing non-mechanic types of objects.

Figure 5. Non-electrical Technology by grade level

Teacher Focus Groups:

To determine if there were differences in teacher’s comments pre to post the statements transcribed from each focus group were coded and then the percentages of difference coding types were computed. This was done since some focus groups simply talked much more than others. Thus, the proportions of comments being assigned each code allowed for a macro level look at differences pre to post. This method allowed us to be able to focus on the differences between the two sets of focus groups.

Collaboration, Cross-curricular Activities and Fidelity of Implementation

When looking at the codes focused on collaboration and cross-curricular activities there was no change in the number of codes pre to post. However, there was a large difference in the substance of the comments. In the pre-interviews the comments about teacher collaboration were mostly general in nature and not STEM specific. For example, teacher 1 said in the focus pre-interviews:

“I think we always share ideas, like if we find something that's really creative, oh, do you want to do this kind of thing, you know.”

However, in the post interviews they were much more specific and targeted towards STEM. For example, teacher 2 said:

‘We have a group of ten people who (…) worked together to help (….) school learn about engineering design process.’

One school did a bridge building unit across all grades in the school thus fostering collaboration between teachers and students. As one specialist at that school stated in the post interview about the relationship between classroom teachers and specialist teachers:

Yes, it really helped, I think, bond their relationships. Because as an art teacher or a gym teacher, it's just you take your kids and you go. And it's your free period, it's your planning period, you don't stick around to see what's going on. But they were involved together, working together to support those students as they were working on their projects.

Several teachers made comments about how they changed their class instruction in terms of cross-curricular activities due to the project allowing for students to experience cross-curricular connections. Such as:

“I know that I added literature in ways that I not necessarily would have prior to this unit.” T1

“And I made more of a point to talk about the engineering design process when we were doing other things, including writing. And so, I'd put the thing up on the board and we'd talk about each of the steps that we worked through in order to get things done.” T5

‘I also worked with social studies as an integration part of it. Not only place and geography. But also, timeline and specifying what would happen [if] things changed, if we change one variable in time. What might've happened, so I used it different ways.’ T3

‘Our third grade art [teacher] she also had them doing bridges while we were doing bridges.’T4

While many teachers mentioned integration in the pre-interviews the comments focused mostly on integration of math and science except for one, where the teacher said:

“I'm the art teacher and I do a lot of "read alouds that integrate science into art. So, they first see a book and we talk about what we learned, and then they create something about it.” T6

And,

“We talked about the process of clay and the science of clay and the chemical change that happens with clay, and then they create sculptures out of it.” T6

Thus, prior to the i-STEAM project, students did not experience cross-curricular activities where teachers of different disciplines were explicitly working on the same project. At best prior to the i-STEAM units, individual teachers might attempt to do an activity like the clay pot suggested by the art teacher that integrated the concepts within their individual classrooms. From the interviews we can see that the implementation of the i-STEAM units produced greater collaboration across classroom and specialist teachers in the elementary schools.

Student Interest in Engineering

The differences in the proportion of statements dealing with student interest and use of EDP between the pre and post interviews was quite large (5% vs 20%, respectively). As two teachers said in the post interview

“The improved process, yes. Yeah, we did that for several days, because they just, they were so determined to get it. They all got the windmill to spin, but then it was the lifting up the weights. And so, they were so determined to get it to lift weights. And so, they were so into it and I couldn't tell them no, so we worked on it far longer than planned and pushed other things. So, I mean they loved it. Loved it.” T1

And….

‘They're just designing up a storm and they loved it.’ T7

In the pre-interviews the mentions of the engineering design were usually attached to students building something and no discussion of the EDP steps such as improvement phase. Thus, the i-STEAM units seemed to pique students’ interest in engineering as perceived by their teachers.

GENERAL DISCUSSION

Students Develop a Greater Understanding of Engineering and Technology

The use of the integrated i-STEAM unit supported significant improvement in PreK3 students’ conceptual understanding of engineering and technology. Overall, there was on average a 55% increase in student understanding of what engineers do, and a 36% increase in their understanding of technology. The effect sizes per grade level for both assessments exhibited generally large effects.

Unfortunately, the scores on both assessments cannot be compared to other interventions using the Engineering is Elementary STEM units as there are no published articles to our knowledge that show average scores per grade level and in the studies we found grade levels were usually collapsed together. Lachapelle et al. (2011) reported the technology pre and post scores for third graders in an internal report. The assessment used in the report was similar to the one used in this paper, both had 20 questions, but some objects were slightly different (for example, their assessment had a picture of an ecosystem where ours did not). In addition, the students in both studies are from urban school districts located in the Midwest of the United States. The report showed a shift in assessment scores from 8.7 to 11.0, a 26% increase overall. The third graders using our i-STEAM unit showed an increase on average from 9.93 to 16.06, a 61% overall increase. This large increase in the third grade was very much due to the large normalized gain third graders had in categorizing non-electrical technological objects. However, it is difficult to point out why this might have occurred. In the third grade multiple units were taught; however, second grade also taught multiple units. Therefore, it is not a dosage effect. The only teachers who had additional training were the teacher leaders and two of them participated in this study. However, they taught in two different grade levels. The other possibility for the large effect size in third grade is that teacher leader in this level had had previous training in dramatic inquiry in terms of literacy. Therefore, the prior training in dramatic inquiry could have caused the greater effect. All of the curriculum unit effect sizes demonstrate the powerful impact that the integrated STEAM units may be having on students above and beyond that of typical STEM units.

Students’ Misconceptions of Engineering and Technology Show an Overall Decline

All students demonstrated increases in their conceptual understanding of technology from kindergarten to 3^rd grade. Students had the greatest difficulty recognizing clothing and household items as pieces of technology especially from grades K to 2. These are new findings as early elementary students have not been assessed previously in this area to our knowledge.

Student misconceptions about engineers being fixers and installers continued to surface both pre and post the i-STEAM unit. The unit did produce a shift in student understanding that engineers are involved in the design of non-electronic items, however, several misconceptions especially in the area of clothing and household technology remained and, in some cases, increased. This dilemma may have occurred because while the students learned about real engineering work via the units, their misconceptions were not specifically addressed. For example, if a class did not use an i-STEAM unit that focused on clothing or household technology then these misconceptions would not have been addressed. In Table 3 the units taught did not focus specifically on household technology except for the playdough, and solar ovens, all of which used common household items in their construction. All of these units except for solar ovens occurred at the lower grade levels and then only in one classroom. This might have caused the third graders to demonstrate the negative change in household technology shown in Figure 5. None of the units focused on clothing as a technological advance. This should be an avenue of future unit and lesson development.

Students Develop Greater Interest in Engineering

Using interviews, we were able to show that the i-STEAM units did indeed produce greater collaboration between teachers. In addition, the teachers perceived that student interest was heightened due to the enactment of the i-STEAM units. The proportion of comments concerning student interest and work with engineering was four times greater in the post interviews.

The use of the i-STEAM unit allowed the students to not only experience the engineering design process but to also see it enacted in multiple contexts from art, dance to literature. We believe, these experiences allowed students to see the importance of engineering thus piquing their interest. In many cases the students had difficulty disconnecting from the process and not continuing to improve their designs. As one teacher said: ‘They were mad at me when I sent [them] home.’

CONCLUSIONS AND IMPLICATIONS

This study is one of the first to demonstrate how dramatic inquiry, dance, visual arts and physical education can be incorporated into STEM units that focus on the engineering design process. In addition, the data demonstrates that these types of units can be quite successful even with our earliest elementary grade levels.

By implementing these integrated i-STEAM units with our younger students, we should be able to nurture their interest and understanding of engineering and technology at the primary educational level. It may not be surprising that the students achieved so well in this context since it parallels the use of techniques appropriate for their developmental age and learning. Our work suggests that engineering units with dramatic inquiry and art infusion should be introduced across the grade levels. This study demonstrates that the pre test scores across grade levels for engineering are not significantly different from each other implying that students may not learn about these important topics outside of school hours. By increasing the number of young children interested in engineering, we may be able to sustain their interest across the grade levels and contribute to a growing number of college students succeeding in STEM majors. At the very least, the students will be able to appreciate the skills they learned and possibly transfer their perseverance in engineering into other disciplines.

LIMITATIONS

The main limitation of this study is the ability to generalize across context since the number of participants is limited and they are all from Midwestern urban schools in the United States. However, the results do demonstrate the possibility of this pedagogy to impart an understanding for engineering and technology. Researchers might consider a quasi-experimental study in the future by adding a control group. The other limitation is that the students were not assessed for changes in aspirations towards future careers in engineering or science. Therefore, the study could have been improved by conducting interviews with the student participants to further gauge their understanding of engineering and technology. In addition, if a technology assessment is used it should be developed such that the questions are balanced across types of technology (i.e., mechanical vs clothing). These suggestions might be an excellent addition to future studies.

ACKNOWLEDGEMENTS

We would like to thank Meg West and Teresa Porter for their work on the original project interview codebook. We would also like to thank the teachers who participated in this work.

This project was funded by a grant under the federally funded Improving Teacher Quality (ITQ) State Grants Program, administered by the Ohio Board of Regents. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the funding organizations.