Introducing and nurturing STEM programs in schools and districts where finances are tight can be challenging. What is the best way to accomplish this task? To test its theories on implementing and expanding STEM education, the Arizona STEM Network conducted a three-year study among selected rural schools and districts in its state. The results are in. To provide information on, and outcomes of, the study, we contacted Linda Coyle, Director of Education at the Arizona STEM Network:

Linda Coyle, Director of Education for the Arizona STEM Network, details the three-year, state-wide study about implementing STEM education.

Q: Tell us about the study.

A: The three-year Helios STEM Schools Pilot program (HSSP) was part of a strategic partnership and investment by the Helios Education Foundation and Science Foundation Arizona (SFAz). The aim was to help the Arizona STEM Network develop a coordinated and effective STEM education system that provided the infrastructure, resources and metrics needed to improve educational outcomes and achievement for the state’s students.

The purpose of the HSSP was to identify models of effective implementation of STEM education teaching and learning that could be sustained, measured and replicated throughout Arizona.

To achieve this goal, the HSSP assisted sites with the integration of STEM programming by introducing the Arizona STEM Network’s STEM Immersion Guide as an overarching structure to equip educators with the knowledge, resources and skills they needed to provide their students access to quality STEM learning experiences.

To begin the process, Helios and the Arizona STEM Network released a competitive request for proposals in September 2012. This competition was open to all public and public charter schools in Arizona wanting to either begin or deepen their level of STEM implementation.

The resulting HSSP sites were selected through a competitive process from among 57 applicants. The selected pilot sites represented districts and schools in Maricopa, Coconino, Yavapai, Pima and Yuma counties and included:

Q: What is your STEM Immersion Guide? How did the study schools use the guide?

A: Educators from throughout the country are looking for a resource to establish and implement a vision for 21st century schools and classrooms. The STEM Immersion Guide (SIG) is written as an overarching framework to help chart a course for integrating STEM education in Arizona schools and districts, and, we hope, provide a resource for schools nationwide.

The SIG contains elements that support project-based, interdisciplinary STEM instruction by providing practical tools and information to enable teachers, schools, districts and administrators that want to improve student outcomes by integrating STEM.

The SIG is broken down into four STEM school models: Exploratory, Introductory, Partial Immersion and Full Immersion. Each model is further refined into six categories: leading, teaching, learning, evaluating, budgeting and sustaining. Each has specific design considerations, or attributes.

The HSSP schools began by taking the online STEM Immersion Assessment. Based on the results of their current STEM model, they dove deeper into the SIG by identifying three key attributes out of each of the six design categories. These became objectives, or “action items,” for further refinement of their existing model or could lead into a deeper model of immersion.

It is interesting to note that out of the seven districts in the HSSP program, each moved at least one to three models (levels) above where it started.

After each district identified its initial model (level) of immersion, it began to form Design Teams to create a strategic plan for school-/district-wide implementation. Throughout the process, all sites followed a carefully staged and supported process of STEM implementation that included:

Essential support provided by SFAz, helped the sites remain faithful to the process. Collaborative work with SFAz required sites to address competing factors across the system (e.g., budget limitations, teaching needs, community involvement); prioritize activities aimed at building capacity and reducing barriers; proactively address problems in one area that might negatively impact another; and utilize success in one aspect of implementation to strengthen another area.

Although site contexts and needs varied greatly, a structured process that included targeted and consistent support enabled all sites to increase the amount and quality of STEM educational experiences within their schools/district. The cumulative outcome was a higher level of STEM immersion that is likely to be sustained.

Q: Why were rural districts the focus of the study? Were any Native American schools involved?

A: The seven proposals that were funded represented a diverse range of geographic and demographic communities from across the state, spanning multiple grade levels, and represent varied educational opportunities.

Rural districts were selected due to the lack of available resources and opportunities for quality STEM professional development. In addition, Arizona is home to the second-largest Native American population nationally. Based on the tribe’s interest and commitment to integrating quality STEM programming, the Salt River Pima-Maricopa Indian Community was chosen as part of this study.

During the three-year period, these sites received a total of $1.85 million in financial and on-sight technical support, professional development and targeted STEM immersion strategies to implement STEM programs and activities.

Q: What were the results of your study? Were some programs found to be more effective, or less effective, than others?

A: The STEM Education Evaluation Center at TERC (a nonprofit education research and development organization in Cambridge, Mass.) was contracted to conduct a three-year evaluation of the HSSP. Results of the mixed methods study were shared with stakeholders via quarterly updates and annual reports.

In the final report, TERC provided data on the overarching evaluation goals established for the project, providing evidence of:

Programmatic improvements influenced teacher and student growth. Descriptive T-STEM survey data indicated that participating teachers (who were involved for varying amounts of time) developed more positive attitudes in all STEM areas.

More compelling were the results from additional and more rigorous analyses of teachers who participated in the entire project. These showed statistically significant positive changes in attitude on the science, technology use, student engagement and career awareness subscales.

Although students’ long-held beliefs and attitudes are often slowest to change, we typically expect changes in classroom instruction to impact student learning. This turned out to be the case.

Across all participating sites this year, the combined average gains in mathematics and science scores were statistically significant on Galileo assessments. The results mirror those of the previous year and suggest that students made steady progress, expanding their math and science knowledge and skills during years 2 and 3 of the project.

Moreover, the analysis of individual site data identified statistically significant gains in math and science in at least two grade levels per site, with most sites achieving significant gains for students in all or almost all grade levels.

Last, the data suggest that much of the work of the HSSP is likely to continue after the grant ends. Many of the sites carefully and systematically transitioned from having grant-funded project activities to embedding them in normal educational practice within their school system. Evidence of funding through school budgets and new grants will enable key STEM programs to continue.

In addition, recent and planned expansion of STEM activities, structures that allow staff to maintain and upgrade STEM resources, and STEM awareness and cultural shifts within the schools help to ensure sustainability.

At the same time, individual sites must be vigilant in carrying out these plans, for there is also evidence of barriers to sustainability. If addressed, the notable progress of HSSP sites may be sustained, and these strong STEM educational communities will thrive well into the future.

Q: How will the results be implemented in your programs or recommendations going forward?

A: The results of the HSSP program provided the necessary tools and data to support the use of the SIG to establish a baseline and create a plan that would help schools reach their goals; look at systemic changes that would need to occur across strands (leading, teaching, learning evaluating, budgeting, and sustaining); and evaluate progress over the long-term implementation phase. We also noted limitations and offered suggestions for improvement.

Overall, the benefits of the SIG far outweighed its limitations, and in Year 3, Guides were still in use, informing the work that would impact sustainability once the project ended.

Based on each site’s systematic evaluation of its STEM attributes, reported use of the SIG, and from SFAz monitoring reports, we found that all sites increased their level of STEM immersion. This encouraged us to try the process again with other schools interested in taking the challenge.

Q: How can other districts and states benefit from the results your study?

A: We are working with an additional 20 schools in eight new districts across Arizona using the same processes described above. While these schools will not receive funding for equipment or materials, we believe the structural process for program implementation will again produce the desired results.

By providing a tool for STEM immersion that can be individually tailored to grow STEM programs organically within a school or community, we feel that other districts and states can use our resources and the results of our study to foster STEM integration in their own communities.

Q: Is there anything else you would like to tell us about the study, your programs or processes?

A: If you are interested in our STEM Immersion Guide or processes for STEM implementation, contact me at lcoyle@sfaz.org. We are always interested in finding the best possible tools and resources for our teachers and welcome the opportunity to collaborate.