<DOC> [110th Congress House Hearings] [From the U.S. Government Printing Office via GPO Access] [DOCID: f:35233.wais] FEDERAL STEM EDUCATION PROGRAMS ======================================================================= HEARINGS BEFORE THE SUBCOMMITTEE ON RESEARCH AND SCIENCE EDUCATION COMMITTEE ON SCIENCE AND TECHNOLOGY ONE HUNDRED TENTH CONGRESS FIRST SESSION __________ MAY 15, 2007 and JUNE 6, 2007 __________ Serial No. 110-28 and Serial No. 110-35 __________ Printed for the use of the Committee on Science and Technology Available via the World Wide Web: http://www.house.gov/science U.S. GOVERNMENT PRINTING OFFICE 35-233 WASHINGTON : 2007 _____________________________________________________________________________ For Sale by the Superintendent of Documents, U.S. Government Printing Office Internet: bookstore.gpo.gov Phone: toll free (866) 512-1800; (202) 512ÿ091800 Fax: (202) 512ÿ092104 Mail: Stop IDCC, Washington, DC 20402ÿ090001 ______ COMMITTEE ON SCIENCE AND TECHNOLOGY HON. BART GORDON, Tennessee, Chairman JERRY F. COSTELLO, Illinois RALPH M. HALL, Texas EDDIE BERNICE JOHNSON, Texas F. JAMES SENSENBRENNER JR., LYNN C. WOOLSEY, California Wisconsin MARK UDALL, Colorado LAMAR S. SMITH, Texas DAVID WU, Oregon DANA ROHRABACHER, California BRIAN BAIRD, Washington KEN CALVERT, California BRAD MILLER, North Carolina ROSCOE G. BARTLETT, Maryland DANIEL LIPINSKI, Illinois VERNON J. EHLERS, Michigan NICK LAMPSON, Texas FRANK D. LUCAS, Oklahoma GABRIELLE GIFFORDS, Arizona JUDY BIGGERT, Illinois JERRY MCNERNEY, California W. TODD AKIN, Missouri PAUL KANJORSKI, Pennsylvania JO BONNER, Alabama DARLENE HOOLEY, Oregon TOM FEENEY, Florida STEVEN R. ROTHMAN, New Jersey RANDY NEUGEBAUER, Texas MICHAEL M. HONDA, California BOB INGLIS, South Carolina JIM MATHESON, Utah DAVID G. REICHERT, Washington MIKE ROSS, Arkansas MICHAEL T. MCCAUL, Texas BEN CHANDLER, Kentucky MARIO DIAZ-BALART, Florida RUSS CARNAHAN, Missouri PHIL GINGREY, Georgia CHARLIE MELANCON, Louisiana BRIAN P. BILBRAY, California BARON P. HILL, Indiana ADRIAN SMITH, Nebraska HARRY E. MITCHELL, Arizona CHARLES A. WILSON, Ohio ------ Subcommittee on Research and Science Education HON. BRIAN BAIRD, Washington, Chairman EDDIE BERNICE JOHNSON, Texas VERNON J. EHLERS, Michigan DANIEL LIPINSKI, Illinois ROSCOE G. BARTLETT, Maryland JERRY MCNERNEY, California FRANK D. LUCAS, Oklahoma DARLENE HOOLEY, Oregon RANDY NEUGEBAUER, Texas RUSS CARNAHAN, Missouri BRIAN P. BILBRAY, California BARON P. HILL, Indiana BART GORDON, Tennessee RALPH M. HALL, Texas JIM WILSON Subcommittee Staff Director DAHLIA SOKOLOV Democratic Professional Staff Member MELE WILLIAMS Republican Professional Staff Member MEGHAN HOUSEWRIGHT Research Assistant C O N T E N T S May 15, 2007 Page Witness List..................................................... 2 Hearing Charter.................................................. 3 Opening Statements Statement by Representative Brian Baird, Chairman, Subcommittee on Research and Science Education, Committee on Science and Technology, U.S. House of Representatives...................... 6 Written Statement............................................ 7 Statement by Representative Vernon J. Ehlers, Ranking Minority Member, Subcommittee on Research and Science Education, Committee on Science and Technology, U.S. House of Representatives................................................ 8 Written Statement............................................ 9 Prepared Statement by Representative Eddie Bernice Johnson, Member, Subcommittee on Research and Science Education, Committee on Science and Technology, U.S. House of Representatives................................................ 10 Witnesses: Ms. Linda K. Froschauer, President, National Science Teachers Association Oral Statement............................................... 11 Written Statement............................................ 12 Biography.................................................... 15 Mr. Michael C. Lach, Director of Mathematics and Science, Chicago Public Schools Oral Statement............................................... 15 Written Statement............................................ 16 Biography.................................................... 21 Dr. George D. Nelson, Director of Science, Technology, and Mathematics Education, Western Washington University Oral Statement............................................... 22 Written Statement............................................ 23 Biography.................................................... 25 Mr. Van R. Reiner, President and CEO, Maryland Science Center, Maryland Academy of Sciences Oral Statement............................................... 26 Written Statement............................................ 28 Biography.................................................... 132 Dr. Iris R. Weiss, President, Horizon Research, Inc. Oral Statement............................................... 33 Written Statement............................................ 34 Biography.................................................... 39 Discussion....................................................... 40 Appendix 1: Answers to Post-Hearing Questions Ms. Linda K. Froschauer, President, National Science Teachers Association.................................................... 58 Mr. Michael C. Lach, Director of Mathematics and Science, Chicago Public Schools................................................. 59 Dr. George D. Nelson, Director of Science, Technology, and Mathematics Education, Western Washington University........... 61 Mr. Van R. Reiner, President and CEO, Maryland Science Center, Maryland Academy of Sciences................................... 62 Dr. Iris R. Weiss, President, Horizon Research, Inc.............. 63 Appendix 2: Additional Material for the Record Science on a Sphere, Front-end Evaluation, prepared for the Maryland Science Center, August 20, 2004....................... 68 Dino Quest, Front-End Evaluation Focus Groups, prepared by Minda Borun, Museum Solutions........................................ 141 Dinosaur Mysteries, Summative Evaluation, by Minda Borun, Museum Solutions...................................................... 165 C O N T E N T S June 6, 2007 Page Witness List..................................................... 184 Hearing Charter.................................................. 185 Opening Statements Statement by Representative Jerry McNerney, Vice Chairman, Subcommittee on Research and Science Education, Committee on Science and Technology, U.S. House of Representatives.......... 192 Written Statement............................................ 193 Statement by Representative Vernon J. Ehlers, Ranking Minority Member, Subcommittee on Research and Science Education, Committee on Science and Technology, U.S. House of Representatives................................................ 194 Written Statement............................................ 195 Prepared Statement by Representative Eddie Bernice Johnson, Member, Subcommittee on Research and Science Education, Committee on Science and Technology, U.S. House of Representatives................................................ 195 Prepared Statement by Representative Russ Carnahan, Member, Subcommittee on Research and Science Education, Committee on Science and Technology, U.S. House of Representatives.......... 196 Witnesses: Dr. Cora B. Marrett, Assistant Director, Education and Human Resources Directorate, National Science Foundation Oral Statement............................................... 196 Written Statement............................................ 198 Biography.................................................... 201 Dr. Joyce L. Winterton, Assistant Administrator, Office of Education, National Aeronautics and Space Administration (NASA) Oral Statement............................................... 202 Written Statement............................................ 204 Biography.................................................... 209 Mr. William J. Valdez, Director, Office of Workforce Development for Teachers and Scientists, Office of Science, Department of Energy Oral Statement............................................... 210 Written Statement............................................ 212 Biography.................................................... 215 Dr. Bruce A. Fuchs, Director, Office of Science Education, National Institutes of Health Oral Statement............................................... 216 Written Statement............................................ 217 Biography.................................................... 222 Discussion....................................................... 222 Appendix 1: Answers to Post-Hearing Questions Dr. Cora B. Marrett, Assistant Director, Education and Human Resources Directorate, National Science Foundation............. 234 Dr. Joyce L. Winterton, Assistant Administrator, Office of Education, National Aeronautics and Space Administration (NASA) 236 Mr. William J. Valdez, Director, Office of Workforce Development for Teachers and Scientists, Office of Science, Department of Energy......................................................... 237 Dr. Bruce A. Fuchs, Director, Office of Science Education, National Institutes of Health.................................. 238 Appendix 2: Additional Material for the Record Statement of the Office of Education, National Oceanic and Atmospheric Administration, U.S. Department of Commerce........ 240 FEDERAL STEM EDUCATION PROGRAMS: EDUCATORS' PERSPECTIVES ---------- TUESDAY, MAY 15, 2007 House of Representatives, Subcommittee on Research and Science Education, Committee on Science and Technology, Washington, DC. The Subcommittee met, pursuant to call, at 10:05 a.m., in Room 2318 of the Rayburn House Office Building, Hon. Brian Baird [Chairman of the Subcommittee] presiding. <GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT> hearing charter SUBCOMMITTEE ON RESEARCH AND SCIENCE EDUCATION COMMITTEE ON SCIENCE AND TECHNOLOGY U.S. HOUSE OF REPRESENTATIVES Federal STEM Education Programs: Educators' Perspectives tuesday, may 15, 2007 10:00 a.m.-12:00 p.m. 2318 rayburn house office building Purpose The purpose of this hearing is to inform the Subcommittee of educators' experiences working with science, technology, engineering, and math (STEM) education programs for K-16 students supported by the federal R&D mission agencies: National Aeronautics & Space Administration (NASA), National Oceanographic & Atmospheric Administration (NOAA), National Institute of Standards & Technology (NIST), Environmental Protection Agency (EPA), and Department of Energy (DOE). This hearing will explore whether such issues as the lack of coordination between the agencies, difficulty by educators in finding information about the programs, and the absence of robust evaluation techniques hinder the potential of the federal programs for improving STEM education in America. Most importantly, the hearing will highlight how the federal R&D mission agencies can best contribute to raising the level of scientific literacy of all students. Witnesses Ms. Linda K. Froschauer, President, National Science Teachers Association Mr. Michael C. Lach, Director of Mathematics and Science, Chicago Public Schools Dr. George D. Nelson, Director, Science, Technology, and Mathematics Education, Western Washington University Mr. Van R. Reiner, President, Maryland Science Center Dr. Iris R. Weiss, President, Horizon Research, Inc. Overarching Questions <bullet> What are the experiences of educators in finding and leveraging resources for STEM education from the federal R&D mission agencies? What challenges have they encountered? <bullet> What do educators perceive to be successful STEM education programs at the federal R&D mission agencies? How do they determine success? What should the agencies improve? <bullet> What support that the federal R&D mission agencies could provide would have the most impact on improving STEM education? Background A multitude of studies over the past twenty years have documented the downward slide of American students' proficiency and participation in science, technology, engineering and mathematics (STEM) fields. In October 2005, the National Academies released the report, Rising Above the Gathering Storm: Energizing and Employing America for a Brighter Economic Future, which warned that ``the scientific and technological building blocks critical to our economic leadership are eroding at a time when many other nations are gathering strength.'' The authoring committee deemed the highest priority action to be vast improvement of science and mathematics education in order to increase the number of students interested in and prepared for entering careers in STEM fields. The scientific and technical expertise of the R&D mission agencies has been applied to varying degrees and with varying success to programs and activities relevant to improving STEM education. Currently, the STEM education programs at these agencies for K-16 grades are relatively small and vary widely in methods, target audiences, evaluation measurements, and funding. In an inventory of STEM education programs, the Academic Competitiveness Council (ACC) identified approximately $39 million in FY06 appropriated funds at the federal R&D mission agencies for K-12 programs and $6.6 million for undergraduate programs. Until recently, the agencies have developed their programs independently and without a strategic plan for accomplishing a set of overarching goals and objectives. Unfortunately, this led to a need for each program to discover a cadre of ``best practices'' on its own rather than collaborating with other program and agency experts. Each program also developed a unique method of evaluation, making a comparison of effectiveness across the programs impossible. Lastly, the agencies have had trouble building widespread awareness of their programs among teachers nationwide. In response to these issues, the National Science and Technology Council, which serves as the principal body for coordinating federal research and development, has re-established the Education and Workforce Development subcommittee to encourage the agencies to share knowledge and develop a federal strategic plan for effectively increasing STEM proficiency nationwide. The plans for this new federal coordination and planning activity will be reviewed by the Subcommittee in a subsequent hearing. The Subcommittee recognizes that critical guidance for how to improve the federal STEM education programs must come from the people who work directly with teachers and students. The witnesses for today's hearing were chosen because of their experience working with these programs and have been asked to provide insight from the field on what the agencies are doing well and where they need to improve. Specific Questions for the Witnesses Ms. Linda K. Froschauer <bullet> In your experience, what are the federal R&D mission agencies doing well in their respective STEM education programs? What could they do better? Can you give examples of any particularly effective programs? <bullet> How do your teacher members learn about STEM education programs sponsored by the federal research and development agencies? <bullet> What resources of the agencies would be most valuable in supporting your teacher members in the classroom? Mr. Michael C. Lach <bullet> How do you find resources for improving science and mathematics education in the Chicago Public Schools? <bullet> What resources have you garnered from the federal R&D mission agencies? How has this contributed to improving your students' understanding of science? <bullet> What type of support that the federal R&D mission agencies could provide would have the most impact on STEM education for your teachers and students in Chicago Public Schools? Dr. George D. Nelson <bullet> In what ways can federal R&D mission agencies contribute most effectively to improve K-12 STEM education? Can you give examples of any particularly effective programs? <bullet> At the undergraduate level, what type of support could the federal R&D mission agencies provide that would recruit more students into pursuing careers in the physical sciences? <bullet> How does the lack of coordination and overarching strategy for STEM education programs hinder the agencies from making an impact? Mr. Van R. Reiner <bullet> Please describe the informal education programs that you have partnered with federal R&D mission agencies to provide for school-aged children. <bullet> How well do the federal R&D mission agencies develop evaluation methods to determine the effectiveness of informal STEM education programs? <bullet> What informal education activities should the federal R&D mission agencies increase to help raise the level of scientific literacy in American students? Dr. Iris R. Weiss <bullet> Do you feel that the federal R&D mission agencies develop evaluation methods for STEM education programs that demonstrate effectiveness? What recommendations would you give for improving their evaluation methods? <bullet> To what extent do the federal R&D mission agencies incorporate best practices which have proven to be effective in STEM education into their programs? <bullet> Based on your research on teacher training and professional development, what guidance would you give for developing programs for pre-service and in-service STEM teachers? Chairman Baird. This hearing will come to order. I want to thank the witnesses and our guests and thank my good friend and colleague, Dr. Ehlers, Ranking Member. Our Subcommittee on Research and Science Education Committee is interested in hearing from educators in science, technology, engineering and mathematics, STEM fields, about their experiences working with the federal R&D mission agencies. This hearing is part of the ongoing effort led by Chairman Gordon that the Committee is undertaking to determine how to improve the level of scientific understanding of students in the United States and how to attract more students to careers in science and engineering. There have been at least a half-dozen reports released over the past 20 years documenting how American students have fallen behind students in other countries. The National Academies report, Rising Above the Gathering Storm, warned us that this will threaten the standing of our country in the future. The authors of that paper wrote ``the scientific and technological building blocks critical to our economic leadership are eroding at a time when many other nations are gathering strength.'' They recommended, they being the authors, recommended that the highest priority should be a vast improvement of science and mathematics education in this country in order to increase the number of students interested in and prepared to enter careers in STEM fields. The Science and Technology Committee held a hearing in March with leading voices in private industry and higher education to discuss research and education needs in STEM fields. Every one of the witnesses, including a retired CEO of Lockheed Martin, the current CEO of McGraw-Hill, the CEO of Intel, and the President of the Council on Competitiveness, testified that companies in America need a workforce well- trained in STEM fields in order to continue the innovative solutions that keep them profitable. The Committee has taken this advice to heart. H.R. 362, also known as ``10,000 Teachers, 10 Million Minds'' Science and Math Scholarship Act, was introduced by Chairman Gordon earlier this year. The bill implements most of the K-12 education recommendations of the Gathering Storm report and was passed by the House with strong bipartisan support last month. The Research and Science Education Subcommittee will next be exploring ways that federal efforts in STEM education can be better focused and more effective. This is the first in a planned series of hearings to address these issues. Today we are reviewing the role of the federal R&D mission agencies in improving STEM education. Specifically, we are referring to NASA, NOAA, NIST, EPA, and the Department of Energy. I believe there is a great deal of untapped potential residing in the expertise of scientists and engineers at these agencies. Not only do these scientists and engineers possess impressive content knowledge of the sciences, they also have real-world experience with the ``wow'' factor that gets kids excited about learning science. Space travel, discovering new forms of ocean life, creating renewable energy sources, improving air and water quality, testing bullet-proof vests, we could list hundreds of more activities that make science and math captivating to young people. Although the agencies have made commendable efforts to share their knowledge and passion for science with students, I believe those efforts have been relatively small and have varied widely in their methods, target audiences, and methods of evaluation. The programs have been developed independently and without a strategic plan for accomplishing a common set of goals and objectives. With a unified effort, I am convinced these programs could have a much bigger impact on the approximately 52 million K-12 students in America. So we have asked our witnesses today to tell us about their experiences participating in these programs. We have asked them to respond to a series of questions. What do the agencies do well? What should they improve? Which programs do educators consider successful? And how do they define that success? The Committee is devoted to improving science education, so devoted that we added science education to the name of this subcommittee. We are very concerned that American students are not achieving their potential in science and math education. It is a concern not only as we look at competing in a knowledge- based global economy but also when we look at access to high- paying, technology-based jobs in this country. I look forward to hearing from today's witnesses and recognize my colleague and Ranking Member, Dr. Ehlers. [The prepared statement of Chairman Baird follows:] Prepared Statement of Chairman Brian Baird Good morning and thank you for attending today's Subcommittee on Research and Science Education hearing. Today, we are going to hear from educators in science, technology, engineering, and mathematics-- STEM fields--about their experiences working with the federal R&D mission agencies. This hearing is part of an ongoing effort that the Committee is undertaking to determine how to improve the level of scientific understanding of students in the U.S. and how to attract more students to careers in science and engineering. There have been at least a half dozen reports released over the past 20 years documenting how American students have fallen behind students in other countries. The National Academies report, Rising Above the Gathering Storm, warned us that this will threaten the standing of our country in the future. The authors wrote ``the scientific and technological building blocks critical to our economic leadership are eroding at a time when many other nations are gathering strength.'' They recommended that the highest priority should be a vast improvement of science and mathematics education in this country in order to increase the number of students interested in and prepared to enter careers in STEM fields. The Science and Technology Committee held a hearing in March with leading voices in private industry and higher education to discuss research and education needs in STEM fields. Every one of the witnesses, including a retired CEO of Lockheed Martin, the current CEO of McGraw-Hill, the CEO of Intel, and the President of the Council on Competitiveness, testified that companies in America need a workforce well-trained in STEM fields in order to continue the innovative solutions that keep them profitable. The Committee has taken this advice to heart. H.R. 362, ``10,000 Teachers, 10 Millions Minds'' Science and Math Scholarship Act, was introduced by Chairman Gordon early this year. The bill implements most of the K-12 education recommendations of the Gathering Storm report and was passed by the House last month. The Research and Science Education Subcommittee will next be exploring ways that federal efforts in STEM education can be better focused and more effective. This is the first in a planned series of hearings to address these issues. Today we are reviewing the role of the federal R&D mission agencies in improving STEM education. Specifically, I am referring to NASA, NOAA, NIST, EPA, and the Department of Energy. I believe there is a great deal of untapped potential residing in the expertise of scientists and engineers at these agencies. Not only do these scientists and engineers possess impressive content knowledge in the sciences, they also have real-world experience with the ``wow'' factors that gets kids excited about learning science. Space travel, discovering new forms of ocean life, creating renewable energy sources, improving air and water quality, testing bullet-proof vests--I could list hundreds more activities that make science and math captivating. Although these agencies have made commendable efforts to share their knowledge and passion for science with students, I fear that those efforts have been relatively small and have varied widely in their methods, target audiences, and methods of evaluation. The programs have been developed independently and without a strategic plan for accomplishing a common set of goals and objectives. With a unified effort, I am convinced that these programs could have a much bigger impact on the approximately 52 million K-12 students in America. We have asked our witnesses today to tell us about their experiences participating in these programs. We have asked them to respond to a series of questions: What do the agencies do well? What should they improve? Which programs do educators consider successful? And how do they define that success? The Committee is devoted to improving science education--so devoted that we added science education to the name of this subcommittee. We are very concerned that American students are not achieving their potential in science and math education. This is a concern not only as we look at competing in a knowledge-based global economy, but also when we look at access to high-paying, technology-based jobs in this country. I look forward to hearing from today's witnesses. Mr. Ehlers. Thank you very much, Chairman Baird, and I do apologize to you and to the group for my due late arrival. I was speaking at another session, and unfortunately I ended up being the last speaker; and even though it was difficult for me, I did cut my words short. I am pleased today that we have a cadre of consumers of science, technology, engineering and mathematics, better known as STEM educational programs across the federal agencies before us to hear about their experiences. I believe it is the desire of all Members of the Science and Technology Committee that we support the implementation of programs that are well-designed and effective. But often as legislators we are so distanced from the final implementation of the programs that we hear little about personal challenges and personal successes, and this is an opportunity for us to hear what some of our grand schemes have resulted in and hear it from the people who are sort of the boots on the ground in the STEM education battle. Today's hearing will delve into what was happening with the consumers of federal STEM agency programs. I might mention many of these programs don't come directly from this committee. There are a number of federal agencies that instigate their own programs without Congressional direction, and I have worried for some years how those all intertwine with each other and with what we have passed. While each of our panelists brings unique perspectives to the table today, I note there are a few common themes running through your prepared testimony. Several of you have identified the Federal Science and Technology Workforce and Facilities and under-utilized resources for K-12 classrooms. I am interested to learn more about programs that successfully leverage these resources. Secondly, many of you remarked that the best programs are those that excite and inform teachers and students. Finally, your testimony, coupled with the recent release of the Academic Competitiveness Council's report on federal STEM programs emphasizes a need to reduce the number of programs that are not evaluated or clearly do not provide a benefit to teachers and students. Alternatively, faced with a maze of resources, you need help identifying programs that have been evaluated as successful to know what may be useful to you. Our challenge in Congress is to target limited federal funds at programs which leverage relevant federal resources and also complement the local education requirement. Today we will have achieved a win-win scenario to promote STEM literacy at all levels if we manage to do that. I am particularly pleased to see that today's panel includes Michael Lach. As an Einstein Fellow in my office from 1999 to 2000, Michael provided extremely valuable insight to me on STEM education and science policy. He has moved onto much grander things, now directing the math and science curricula for the entire Chicago Public School System. He has been an outspoken pioneer for effective teaching in math and science from the time that he started teaching high school science through the Teach for America program. Granted that I will allow that I am a little biased since Michael is a physicist by training, but I have been told by others that he is an exceptional teacher, one that other teachers look up to as an example that they aspire to be. We are fortunate to have him here today. Welcome Michael, and for that I yield back. [The prepared statement of Mr. Ehlers follows:] Prepared Statement of Representative Vernon J. Ehlers I am pleased today that we have a cadre of consumers of Science, Technology, Engineering and Mathematics (STEM) educational programs across the federal agencies before us to hear about their experiences. I believe it is the desire of all the Members of the Science and Technology Committee that we support the implementation of programs that are well-designed and effective. Often as legislators we are so distanced from final implementation of programs that we hear little about personal challenges and successes. Today's hearing will delve into what is happening with the consumers of federal STEM agency programs. While each of our panelists brings unique perspectives to the table today, I note that there are a few common themes running through your prepared testimony. Several of you have identified the federal science and technology workforce and facilities as under-utilized resources for K-16 classrooms. I am interested to learn more about programs that successfully leverage these resources. Secondly, many of you remark that the best programs are those that excite and inform students and teachers. Finally, your testimony--coupled with the recent release of the Academic Competitiveness Council's (ACC) report on federal STEM programs--emphasizes a need to reduce the number of programs that are not evaluated or clearly do not provide a benefit to teachers and students. Alternatively, faced with a maze of resources, you need help identifying programs that have been evaluated as successful to know what may be useful to you. Our challenge in Congress is to target limited federal funds at programs which leverage relevant federal resources and also complement the local educational requirements. Then we will have achieved a win-win scenario to promote STEM literacy at all levels. I am particularly pleased to see that today's panel includes Michael Lach. As an Einstein Fellow in my office from 1999-2000, Michael provided extremely valuable insight to me on STEM education and science policy. He has moved on to much grander things, now directing the science and math curricula for the entire Chicago Public School system. He has been an outspoken pioneer for effective teaching in math and science from the time that he started teaching high school science through the Teach for America program. Granted, I will allow that I am a little biased since Michael is a physicist by training, but I have been told by others that he is an exceptional teacher, one that other teachers look up to as the example they aspire to be. We are fortunate to have him here with us today. Welcome, Michael. Chairman Baird. Thank you, Dr. Ehlers. If there are other Members who wish to submit additional opening statements, the statements will be added to the record. [The prepared statement of Ms. Johnson follows:] Prepared Statement of Representative Eddie Bernice Johnson Thank you, Mr. Chairman. Our nation's future competitiveness depends on whether or not tomorrow's generation is prepared for the high-tech jobs of the future. Enthusiastic, adequately prepared teachers who utilize successful teaching methods are needed in many of today's math and science classrooms. Mr. Chairman, the Federal Government must invest in math and science education, as these investments help fuel our nation's economic growth. Economists agree that no other investment generates a greater long- term return to the economy than scientific R&D, and that starts with educational systems. Research, education, the technical workforce, scientific discovery, innovation and economic growth are intertwined. The Science, Technology, Engineering and Math Caucus has reported that Texas ranked 20th in the Nation on the 2005 National Assessment of Educational Progress scores for mathematics with a score of 281. The national average was 278. Texas did not report on the percentage of Texas middle school teachers who were certified in math (the national average was 49 percent) or science (the national average was 54 percent). Sadly, only seven percent of Texas' 12th grade students took the AP Calculus exam in 2004. Students should be challenged so that they are able to master these subjects, if we want to compete globally. Today's witnesses are here to provide a critical view of federal STEM education programs. Members of Congress need to know what works and what doesn't work so that we can build policies that offer the most benefit per taxpayer dollar. Thank you, Mr. Chairman. I yield back. Chairman Baird. At this time I would like to introduce the witnesses on the panel. Ms. Linda Froschauer is the President of National Science Teachers Association. She is also the K-12 Science Department Chair for the Weston Public Schools in Weston, Connecticut. It is good to have you with us again. We always value your remarks and insights. Mr. Michael Lach is the Director of Mathematics and Science for Chicago Public Schools. Dr. George D. Nelson is Director of the Science, Mathematics, and Technology Education and is Professor of Physics and Astronomy at Western Washington University, my home state. He is a former astronaut and flew on the Space Shuttle. Mr. Van Reiner is the President of the Maryland Science Center and formerly was the President of the Sparrow Point Division of Bethlehem Steel. And Dr. Iris Weiss is President of Horizon Research, Incorporated, which specializes in mathematics and science education and research evaluation. As you can see, we have a very distinguished and well-qualified panel before us today to learn from. I would remind our witnesses--first of all, we have all had a chance to look at your written testimony. Thank you for preparing that, some of it quite lengthy, but very, very informative; but I would remind you that today for the purpose of testimony--spoken testimony is limited to five minutes each. There are little lights there that will come on; and as Dr. Ehlers used to remind people, after the red light comes on, you have about five seconds and a trapdoor appears underneath your chair and you disappear from sight, after which Members of the Subcommittee will have five minutes each to ask questions. This is a collegial, friendly atmosphere, so we very much look forward to a good exchange of ideas. And with that, we will start with Ms. Froschauer. STATEMENT OF MS. LINDA K. FROSCHAUER, PRESIDENT, NATIONAL SCIENCE TEACHERS ASSOCIATION Ms. Froschauer. Thank you for this opportunity to present testimony on behalf of the National Science Teachers Association. My name is Linda Froschauer, and I am the President of the NSTA. For 32 years I have been a science teacher, currently teaching eighth-grade science, and I am Department Chair at Weston Middle School in Connecticut. This is the second opportunity I have had in recent months to testify before this subcommittee. A few weeks ago I appeared in support of H.R. 524 which is for Partnerships for Access to Laboratory Science Grants. I want to sincerely thank the Members of the Science Committee and the House for passing this extremely important legislation as part of H.R. 362. Today I would like to talk about federal STEM education programs for K-12 teachers. As you know, the vast majority of STEM education programs are generated from the U.S. Department of Education and the National Science Foundation. We consider the NSF to be the engine of innovation for K-12 STEM education. Information about NSF initiatives and other federal STEM education programs are promoted extensively through NSTA print and online channels and on the NSTA website. Combined, these communication vehicles reach hundreds of thousands of teachers, teacher leaders, and others in the science education community. Federal agencies also share information about programs for science educators at the NSTA conferences which draw approximately 12,000 teachers each year. During the last NSTA conference, the National Institutes of Health featured the NIH Research Zone, a coordinated effort that involved 27 institutes and centers from NIH professional societies and other supporting partners. The NIH Research Zone provided one-stop shopping for teachers interested in discovering the resources available from the NIH research community. Workshops and exhibits on NASA's education programs are also prominent parts of NSTA conferences. These include the NASA Educator Astronaut Launch where teachers can join NASA's first educator astronaut, Barbara Morgan, on her upcoming Shuttle launch. The Student Observation Network, 21st Century Explorer, and the Engineering Design Challenge allow students to use NASA data to conduct their own analysis and apply engineering principles to solve scientific problems. One of the challenges with many federal education programs, however, is that they reach only a miniscule proportion of our nation's science teachers. We must continue to find new and effective ways to get quality, professional development programs up to scale so they reach a large number of teachers. To address this issue, NSTA is working with NASA, NOAA, and the FDA to develop face-to-face training and online experiences that we believe have potential to reach hundreds of thousands of K-12 teachers. Why is professional development so important? Last year the National Research Council report titled Taking Science to School, Learning and Teaching Science in Grades K-8, said that professional development is key to supporting effective science instruction in the critical early years of a child's education. All teachers need opportunities to deepen their knowledge of the science content. In fact, the NRC and Taking Science to School Report says that federal agencies that support professional development should require that the programs they fund incorporate models of instruction that combine the four strands of science proficiency, focus on core ideas in science, and enhance teacher's science content knowledge, knowledge of how students learn science, and knowledge of how to teach science. In conclusion, recent reports have made it clear that better coordination and communication is desperately needed among federal agencies, bureaus, divisions, and centers that are involved with STEM education research and programs. The federal agencies do not appear to work together to facilitate the dissemination of research or to discuss possible new ideas and avoid those duplicative programs. In addition, an inventory of STEM education programs across the federal agencies would inform future priorities and initiatives, an oversight entity at the federal level that works to coordinate STEM education programs and could work with state and local officials and with science education stakeholders is critical. Improvements in STEM education require a commitment of leadership at the local, State, and federal levels. Education programs at the federal agencies should continue to play a role in improving STEM education. We hope that any changes to existing programs, especially at the National Science Foundation, that may come about as a result of this ACC report will be carefully reviewed and considered. Thank you for allowing me the opportunity to address you today, and I look forward to your questions. [The prepared statement of Ms. Froschauer follows:] Prepared Statement of Linda K. Froschauer Mr. Chairman and Members of the Committee: Thank you for this opportunity to present testimony on behalf of the National Science Teachers Association. My name is Linda Froschauer, and I am President of the NSTA. For 32 years I have been a science teacher and I am currently an 8th grade science teacher and Department Chair at the Weston Public Schools in Connecticut. This is the second opportunity I have had in recent months to testify before this subcommittee. A few months ago I appeared in support of H.R. 524, the Partnerships for Access to Laboratory Science grants. I want to sincerely thank the members of the Science Committee and the House for passing this extremely important legislation as part of H.R. 362. Today I will talk about the Federal STEM education programs for K- 12 teachers. As you know the vast majority of STEM education programs originate from the U.S. Department of Education and the National Science Foundation. Strengthening science and math education is a core mission of the NSF. Science education is unique because it is concerned with the special character of science and its related disciplines--it is at once a body of knowledge and a dynamic questioning activity. Because of the nature of science it is important to have scientists involved in critical questions of science education. It was the recognition of this interdependence between scientists and the science education enterprise that drove the identification of science education as a key part of the NSF agenda when the agency was founded. We consider the NSF to be the engine of innovation for K-12 STEM education. The new NSF Education and Human Resources Division of Research on Learning in Formal and Informal Settings--known as DRL--is working to advance discovery and innovation at the frontiers of STEM learning and teaching. NSF supports the highly innovative models and approaches to learning in formal and informal settings. NSF works to advance equity and participation for all, to foster linkages between STEM education research and practice, and to unite education research and evaluation activities across the Foundation and with other federal agencies. The NSF has the capacity to incorporate the best from both the science and education R&D communities and can enlist scientists, academicians and researchers in a peer review process that generates and tests innovations in science-related disciplines for education. Unlike the Department of Education, the NSF has the ability to tap into basic cognitive research, fold in new content and new ways of teaching this content from the disciplines, and explore new technologies for the delivery of professional development and for assessing teachers and their students. One of the most effective education programs at NSF is the Math and Science Partnerships. An analysis of 123 schools participating in the NSF MSP program shows continued increases in student proficiency in math and science since the program was first established in 2002. Students showed the most significant improvements in mathematics proficiency, with a 13.7 percent increase for elementary, 6.2 percent increase for middle school, and 17.1 percent increase for high-school students. Science proficiency at each level showed marked gains as well, with a 5.3 percent increase for elementary, 4.5 percent increase for middle school, and 1.4 percent increase for high-school students. African-American, Hispanic, and white students showed significant improvements in elementary level mathematics, as did students designated as special education or as limited English-proficiency students. In addition to working with NSF on a MSP grant, NSTA has worked directly with federal agencies such as NASA, NOAA, and FDA to develop a combination of face-to-face training and online experiences that we believe has the potential to reach hundreds of thousands of K-12 science teachers. NASA, NOAA, DOT and the FDA have partnered with NSTA to develop SciPacks on topics supporting their mission. SciPacks are designed for educators who want or need to learn core science content. SciPacks contain three to five Science Objects, which are stand-alone, content- based units aligned with National Science Education Standards and Benchmarks for Science Literacy. These discrete online learning experiences are especially beneficial to teachers who are forced to teach out-of-field, elementary and middle level teachers who lack degrees in science, or those who need to increase their science knowledge of a particular content area. Each SciPak also contains a pedagogical implications section highlighting age-appropriate concepts and common student misconceptions. Teachers utilizing SciPaks get individualized e-mail support from a content expert and can complete a graded assessment demonstrating content mastery. SciPacks recently unveiled in the NSTA Learning Center focus on these content areas: Gravity and Orbits; the Universe and the Solar System; Earth, Sun and Moon; Coral Reef Ecosystems; Ocean's Effect on Weather and Climate; Plate Tectonics, The Rock Cycle, Force and Motion and Energy. In addition, thousands of teachers have taken advantage of weekly NSTA Web Seminars on these topics. In addition NSTA Symposiums provide face-to-face training with experts on these content areas from federal agencies, who interact one-on-one with K-12 teachers. Other STEM education programs from federal agencies are promoted extensively through NSTA print and online channels, and on the NSTA website. These communication vehicles reach hundreds of thousands of teachers, teacher leaders, and others in the science education community. Federal agencies also share information about programs for science educators at the NSTA conferences, which draw approximately 25,000 teachers each year. During the last NSTA annual conference the National Institutes of Health featured the NIH Research Zone, a coordinated effort that involved 27 institutes and centers from NIH professional societies and other supporting partners. The NIH Research Zone provided one stop shopping for teachers interested in discovering the resources available from the NIH research community. The groups represented included the National Biomedical Imaging and Bioengineering; the National Center for Research Resources; the National Human Genome Research Institute; the National Institute of Allergy and Infectious Diseases; the National Institute of General Medical Sciences; the National Institute of Neurological Disorders and Stroke; the National Institutes of Health Office of Science Education; the National Library of Medicine; and the Society for Neuroscience. In addition, the NIH Office of Science Education provides medical and life science curriculum supplements for grades K-12, as well as posters and videos promoting health science careers. Workshops and exhibits on NASA's education programs are also prominent at NSTA conferences. These include the NASA Educator Astronaut Launch, where teachers can join NASA's first educator astronaut, Barbara Morgan, on her flight to the International Space Station later this summer. NASA is offering a website, classroom activities and challenges to teachers and students. Other NASA programs highlighted at the conference include the Student Observation Network, 21st Century Explorer, and the Engineering Design Challenge. These programs allow students to use NASA data to conduct their own analyses and apply engineering principles to solve scientific problems. The NASA Smart Skies features a web-based simulator with real world air traffic control motion problems between two or more planes. Students apply proportional reasoning and distance rate time relationships to resolve conflicts by changing plane routes and speeds. While I cannot speak to the efficacy or the outcomes of these federal programs, we have found that many of these programs do provide key research and content to classroom teachers and help to excite teachers and students about science. One of the challenges with federal education programs, however, is that they reach only a minuscule proportion of our nation's science teachers. We must find new ways to get proven, effective professional development programs up to scale so they reach a large number of teachers. Why is professional development so important? Last year the National Research Council report titled Taking Science to School: Learning and Teaching Science in Grades K-8 said that professional development was key to supporting effective science instruction in the critical, early years of a child's education. The NRC called for a dramatic departure from current professional development practice, both in scope and kind. All teachers need opportunities to deepen their knowledge of the science content. The NRC also believes that teachers need opportunities to learn how students learn science and how to teach it. They need to know how children's understanding of core ideas in science builds across K-8, not just at a given grade or grade band. Teachers need to learn about the conceptual ideas that students have in the earliest grades and their ideas about science itself. They need to learn how to assess children's developing ideas over time and how to interpret and respond (instructionally) to the results of assessment. In short, teachers need opportunities to learn how to teach science as an integrated body of knowledge and practice--to teach for scientific proficiency. They need to learn how to teach science to diverse student populations, and to provide adequate opportunities for all students to learn science. We believe federal agencies have a key role in providing programs that will enhance teacher content knowledge, help them to deliver effective instruction, and provide insight into how students learn. It is interesting to note that the NRC report also asserts that ``Federal agencies that support professional development should require that the programs they fund incorporate models of instruction that combine the four strands of science proficiency; focus on core ideas in science; and enhance teachers' science content knowledge, knowledge of how students learn science, and knowledge of how to teach science.'' Looking to the future we anticipate that the soon-to-be released Academic Competitiveness Council report on the myriad of federal STEM education programs will bring about needed changes. From our observations, there is an overlap in many of the programs offered at the federal level. There is no oversight entity at the federal level that works to coordinate these STEM programs. The federal agencies do not appear to work together to facilitate the dissemination of research, or to discuss possible new ideas and avoid duplicative programs. We believe that better coordination and communication is desperately needed among federal agencies, bureaus, divisions, and centers that are involved with STEM education research and programs. Finally an inventory of STEM education programs across the federal agencies would inform future priorities and initiatives. Federal agencies should also work to coordinate their STEM education initiatives with states, local districts, the higher education community, and other key stakeholders. Improvements in STEM education require a commitment of leadership at the local, state, and federal levels. Education programs at the federal agencies will always have a critical role to play in improving STEM education. We hope that any changes to existing programs, especially at the National Science Foundation, that may be come about as a result of the Academic Competitiveness Council report will be carefully reviewed and considered. Thank you for allowing me the opportunity to address you today and I look forward to any questions you may have. Biography for Linda K. Froschauer National Science Teachers Association President, 2006-2007 Linda K. Froschauer, K-8 Science Department Chair at the Weston Public Schools, in Weston Connecticut, is president of the National Science Teachers Association (NSTA). She began her one-year term on June 1, 2006. Froschauer has been a devoted teacher and dedicated leader in science education. She began her teaching career as an elementary school teacher in Matteson, Illinois; moved on to middle level teaching at the Greenwich Public Schools, in Greenwich, Connecticut; and has been with the Weston Public Schools since 1985. She combines her work in the classroom with a leadership role in her school, serving as grades K-8 Science Department Chair/mentor teacher. Outside the classroom she has worked as an instructor for Chicago's Museum of Science and Industry; as a writer/consultant for many publications; and as a field editor, reviewer, and consultant for numerous organizations. For more than 30 years, Froschauer has been a leader and active member of NSTA. In 1976, she was named the first Preschool/Elementary Division Director to serve on the NSTA Board of Directors. She later worked on many NSTA committees, including the International Convention Planning Committee, the Preschool/Elementary Committee, and the Informal Education Committee, and she has chaired both the Awards and Recognition Committee and the Committee on Nominations. She also has served as Middle Level Division Director, worked on the Committee and Board Operations Task Force, and led the development of NSTA's first Family Science Day, which was held in conjunction with the NSTA National Convention in Boston. Froschauer's devotion to science education is evidenced by her involvement in numerous other professional organizations. She has served as president of the Connecticut Science Supervisors Association (CSSA), the National Middle Level Science Teachers Association (NMLSTA), and the Council for Elementary Science International (CESI). She is also a member of the Connecticut Academy for Education in Mathematics, Science, and Technology; the Association of Presidential Awardees in Science Teaching; and the Society of Elementary Presidential Awardees. She has been actively involved in Project 2061, a national effort to improve science education sponsored by the American Association for the Advancement of Science. Froschauer was chosen as a Connecticut Science Educators Fellow and named Weston Teacher of the Year in 1999. Her other awards and accomplishments include receiving the NSTA Distinguished Teaching Award, Middle Level, in 2001; National Board for Professional Teaching Standards certification, also in 2001; the CSSA Charles Simone Award for Outstanding Leadership in Science Education in 1998; a Presidential Award for Excellence in Mathematics and Science Teaching in 1993; and the Educational Press Association of America's Distinguished Achievement Award in 1991. Froschauer earned a BS degree in education from Northern Illinois University, an MA in science teaching from Governors State University, and a sixth-year degree in curriculum and supervision from Southern Connecticut State University. Chairman Baird. Thank you, Ms. Froschauer. Mr. Lach. STATEMENT OF MR. MICHAEL C. LACH, DIRECTOR OF MATHEMATICS AND SCIENCE, CHICAGO PUBLIC SCHOOLS Mr. Lach. Thank you, Mr. Chairman, Members of the Subcommittee. Thank you for inviting me here today to speak to you about this issue. It is an honor to sit before you along side colleagues who I have worked with and learned much from. We have made great progress with mathematics and science instruction in Chicago. Student performance has risen considerably over the past five years, and our rate of improvement is greater than that of the rest of the state. To do this, we developed a comprehensive plan to coordinate all aspects of mathematics and science improvement which we call the Chicago Math and Science Initiative. As part of this work, we created a vision for high-quality instruction; built the support infrastructure to provide high-quality, content-rich, professional development to thousands of teachers over the course of an academic year; forged partnerships with local businesses, museums, laboratories, and universities to increase the content knowledge of our teachers; and enhanced their after-school offerings to include mathematics and science enrichment. We have done this in the traditional urban context. Most of our students are poor, our facilities are crumbling, and we are limited on resources. I would argue that there are two major assets of the federal R&D mission agencies that will help K-12 STEM education. The first asset is human capital. The scientists and engineers of NASA, NOAA, NIST, EPA, and the Department of Energy are the best and brightest in the world. They are the ones making new discoveries, creating new technologies, and literally exploring new worlds. The more we can connect students, parents, and teachers with their insights, energy, and perspectives, the better. The second major asset is the facilities. The laboratories and tools that are part of a federal R&D infrastructure are top notch, the particle accelerators, the spacecraft, the computers, the data sets. Most of our students have a very incomplete picture of the real work of scientists and engineers. Many teachers have never been part of a real scientific project. The facilities that are part of the federal R&D mission agencies should be utilized not only to ground science learning in a well-defined context but to enable students and teachers to grasp a vision of what they are trying to do. Communication between districts and the federal R&D mission agencies generally differs by the amount of collaboration that is intended in the partnership. The projects that are designed by the federal R&D mission agencies, individual teachers and schools find them by the usual methods, NSTA mailings and publications, websites, email lists. We regularly email our teachers of any opportunities that we hear about, and generally because of our lack of resources, it is unconscionable for me not to encourage our teachers to participate in anything. The more strategic partnerships, programs are often developed jointly and are the result of an ongoing dialogue so the strengths of the partnering institutions are all leveraged. These partnerships require intense collaboration and flexibility from all sides as well as resources to support and create and maintain them. In my written comments, I have mentioned several partnerships we have used with Fermilab and Argonne National Laboratory. The federal R&D mission agencies have an important role to play in improving K-12 STEM education. By leveraging the human capital and facilities these possess and connecting these to the existing plans and strategies of districts we will collectively be able to advance the mathematic and science achievement of our students. Thank you. [The prepared statement of Mr. Lach follows:] Prepared Statement of Representative Michael C. Lach Mr. Chairman and Members of the Subcommittee, thank you for inviting me here today to speak to you about this issue. It is an honor to sit before you alongside colleagues whom I've worked with and learned much from. I am the Director of Mathematics and Science for the Chicago Public Schools. The Chicago Public School system consists of over 600 schools, nearly 25,000 teachers, and more than 400,000 students. We have made great progress with mathematics and science instruction in Chicago. Student performance has risen considerably over the past five years, and the rate of improvement is faster than that of the state. (See Figure 1 and Figure 2.) To do this, we developed a comprehensive plan to coordinate all aspects of mathematics and science improvement, which we call the Chicago Math & Science Initiative. As part of this work, we created a vision for high quality instruction; built the support infrastructure to provide high quality, content-rich professional development to thousands of teachers over the course of an academic year; forged partnerships with local businesses, museums, laboratories, and universities to increase the content knowledge of our teachers; and enhanced our after-school offerings to include mathematics and science enrichment. We've done this in a challenging context. Eighty-five percent of our students come from low-income families. Our resources are low; Illinois ranks 47th in the Nation in the level of state support for education. Our capacity is limited--less than five percent of our K-8 teachers possess a State endorsement in mathematics. The Chicago Public Schools is an extremely decentralized school district. By State law, decisions about local school budgets, principal contracts, and curriculum are made by an elected body called the ``Local School Council,'' not the Chief Executive Officer. While we feel proud of our accomplishments, we know that we still have much work to do. An achievement gap remains in many of our schools. The number of students meeting and exceeding standards remains far too low. Our high schools, in particular, still have graduation rates that are not acceptable. In as much as possible, we connect with external resources to help us improve mathematics and science in the Chicago Public Schools. Much of the intellectual design of our work comes from insights my colleagues on this panel have provided, from Dr. Nelson's leadership, to Dr. Weiss's insightful evaluations of large-scale change efforts, and to the National Science Teacher Association's consistent support for teachers. Most of the funding for our efforts comes from the district; we work in every manner possible to leverage additional funding from corporate and university partners in the Chicagoland area. In particular, we're happy to have several major universities that we work with in close partnership, and our relationship with Argonne National Laboratory has resulted in several programs that we have enacted together. The gaps we face, and the resource and capacity limitations that we operate under make it unconscionable for us to turn down assistance. So my most important point today is that we really depend on the assistance and partnership of others--including the federal R&D mission agencies. They have an important role to place in science, technology, engineering, and mathematics education in the United States. I'd argue that there are two major assets of the federal R&D mission agencies that will help K-12 STEM education. As the Committee considers the most appropriate way to connect these agencies with K-12 teachers and schools, programs should be designed so that these assets are highlighted. The first asset is human capital. The scientists and engineers of NASA, NOAA, NIST, EPA and DOE are the best and brightest in the world. They are the ones making new discoveries, creating new technologies, and literally exploring new worlds. The more we can connect students, parents, and teachers with their insights, energy, and perspectives, the better. The people of federal R&D mission agencies can both educate and inspire our students and teachers. A key priority should be to leverage this human capital so that they can assist schools and school districts in their work. The second major asset is the facilities. The laboratories and tools that are part of the federal R&D infrastructure are top notch-- the particle accelerators, the space craft, the computers, the data sets. Most of our students have a very incomplete picture of the real work of scientists and engineers. Many teachers have never been part of a real scientific project. The facilities that are part of the federal R&D mission agencies should be utilized not only to ground science learning in a well defined context, but to enable students and teachers to understand a vision of what they're trying to do. A second key priority of the federal R&D thus is to make the places where science and engineering are practices accessible in meaningful ways to students and teachers. I'd like to highlight a few examples of these that come from my experience with Argonne National Laboratory and Fermilab. <bullet> The Academies Creating Teacher Scientists program provides summer internships for teachers to conduct scientific research with ANL scientists.\1\ In this program, both the human capital and the facilities of ANL are made available to select teachers in a sustained, supportive manner. --------------------------------------------------------------------------- \1\ http://www.dep.anl.gov/p<INF>-</INF>k-12/acts/ <bullet> Fermilab's Saturday Morning Physics sessions--in which I participated as a student--brings students to Fermilab to learn about modern physics topics and see real scientists in action.\2\ Similarly, Argonne's distance learning project uses modern technology to provide the opportunity for CPS students to meet and interact with professionals in technical fields.\3\ Both of these programs enable students to access the human capital and facilities of these laboratories. --------------------------------------------------------------------------- \2\ http://ed.fnal.gov/talks/fermilab1994/web/ ed<INF>-</INF>prog<INF>-</INF>sec<INF>-</INF>student.html#saturday \3\ http://www.dep.anl.gov/p<INF>-</INF>k-12/distancelearning/ <bullet> The online ask-a-scientist provides a mechanism for student and teachers to get accurate answers to scientific questions from practicing scientists.\4\ --------------------------------------------------------------------------- \4\ http://www.newton.dep.anl.gov/ <bullet> And, in an expression of our work as partners, the director of education at Argonne participates in our annual --------------------------------------------------------------------------- Principal For A Day project. Given these comments, a picture emerges about the sort of work that isn't very helpful. Curriculum development is one. We know from decades of instructional material development that writing curriculum is a complicated, difficult process. More acutely, we know that robust curriculum is necessary but not sufficient for classroom improvement. In addition to strong materials, teachers need equipment, professional development workshops, coaching, and good assessments. Within a school, leaders need to understand how to support curriculum implementation, and manage improvement throughout grades and courses. Collections of lessons plans, by themselves, are only a small piece of the puzzle. The proliferation of state and national standards and content also makes implementation difficult. Special topics can be motivating and interesting to both teachers and students, but given the now-famous finding from the TIMSS study that our curriculum is ``a mile wide and an inch deep,'' adding more topics to cover only makes things difficult for teachers. If programs or projects are parochial, they're harder to connect to our work. We also know that transforming classroom practice involves intensive capacity development sustained over time. It doesn't happen over night--or in a one-day field trip or workshop. A brief visit to a laboratory or launch can be inspiring--and I don't mean to downplay the importance of inspiring teachers and students about the world of science--but real change takes sustained work over time. Within an overall strategy, there's certainly a need for both. I want to say a few words about the type of human capital development that we provide for teachers at the Chicago Public Schools. Our work falls into three major categories. The first is support for core instructional materials implementation, focusing on the direct application of content and pedagogy to the classroom. This is almost always led by the district, and is difficult to conceive any outside institutions other than curriculum developers with the capacity to provide this work. The second is to enhance the content knowledge of teachers via university course work. The highly qualified teacher demands of the No Child Left Behind legislation as enacted in Illinois use course work as the main driver for this work. Seminars and sessions that don't provide credit for teachers don't enable me to very easily meet my goals. The third is activities that inspire the study of science and mathematics; generally, we use outside institutions such as museums and laboratories to do most of this work. In the Chicago Math & Science Initiative, the Chicago Public Schools was able to develop a coherent and comprehensive strategy for mathematics and science improvement thanks to NSF systemic initiatives. It took us some time both to arrange the human capital and organization in order to structure such a strategy, but the results to date are quite positive. The more the federal R&D mission agencies can align their work to similar district strategies, the better the chance of success. Without a clear connection to district's vision, there will be no traction. A plan enables forward movement. And it takes resources to develop and drive such plans. When the proposals come to me as existing plans with little opportunity for localization, their chance of effectiveness is reduced considerably. Small programs that are aligned only peripherally to our strategies often just add complexity. We've had success because of our commitment to coherence, and the more the federal R&D mission agencies can align with that, the better. I can't think of any proposals that have come to me with an evaluation report documenting their effectiveness. Communication between districts and the federal R&D mission agencies generally differs by the amount of collaboration that is intended in the partnership. For projects that are designed by the federal R&D mission agencies, individual teachers and schools find them by the usual methods--NSTA mailings, websites, e-mail lists. We regularly e-mail our teachers any opportunities that we hear about. For more strategic partnerships, programs are often developed jointly and are the result of an ongoing dialogue so that the strengths of the partnering institutions are leveraged. These partnerships require intense collaboration and flexibility from all sides. The federal R&D mission agencies have an important role to play in improving K-12 STEM education. By leveraging the human capital land facilities that these institutions possess, and connecting these to the existing plans and strategies of the district, we'll collectively be able to advance the mathematics and science achievement of our students. Answers 1. How do you find resources for improving science and mathematics education in the Chicago Public Schools? Individual teachers find text, lesson plans, and other classroom resources via the usual methods--NSTA mailings, websites, e-mails. When the central office learns of opportunities such as this, we distribute them via e-mail to our schools. More strategic partnerships--such as the ones described above--come about via ongoing dialogue with our partners in museums, laboratories, and universities. These are generally designed together. 2. What resources have you garnered from the federal R&D mission agencies? How has this contributed to improving your students' understanding of science? As mentioned, our partnerships in particular with Argonne National Laboratory and the Fermi National Accelerator Laboratory have resulted in several successful programs that connect teachers and students with the scientists, engineers, and facilities of these institutions. 3. What type of support that the federal R&D mission agencies could provide would have the most impact on STEM education for your teachers and students in Chicago Public Schools? The most productive supports are those that (1) inspire students and teachers to study science and mathematics and (2) provide students and teachers with a deep understanding of the real-world work of scientists and engineers. Supports that are not particularly effective include (1) lesson plans and curriculum development, (2) workshops that don't connect directly to specific instructional materials or university credit. To enable deeper collaboration, resources need to be allocated with the expressed purpose of connecting K-12 schools and districts with the federal R&D mission agencies. <GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT> Biography for Michael C. Lach Michael C. Lach is Director of Mathematics and Science for the Chicago Public Schools, overseeing mathematics and science teaching and learning in the 500 elementary schools that comprise the Nation's third largest school district. Mr. Lach began teaching high school biology and general science at Alcee Fortier Senior High School in New Orleans in 1990 as a charter member of Teach For America, the national teacher corps. After three years in Louisiana, he joined the national office of Teach For America as Director of Program Design, developing a portfolio based alternative-certification system that was adopted by several states. Returning to the science classroom in 1994 in New York City Public Schools, and then back to Chicago in 1995 to Lake View High School, he was named one of Radio Shack's Top 100 Technology Teachers, earned National Board Certification, and was named Illinois Physics Teacher of the Year. He has served as an Albert Einstein Distinguished Educator Fellow, advising Congressman Vernon Ehlers (R-MI) on science, technology and education issues. He was lead curriculum developer for the Investigations in Environmental Science curriculum developed at the Center for Learning Technologies in Urban Schools at Northwestern University and published by It's About Time, Inc. He has written extensively about science teaching and learning for publications such as The Science Teacher, The American Biology Teacher, and Scientific American. He earned a Bachelor's degree in physics from Carleton College, and Master's degrees from Columbia University and Northeastern Illinois University. Chairman Baird. Thank you, Mr. Lach. Dr. Nelson. STATEMENT OF DR. GEORGE D. NELSON, DIRECTOR OF SCIENCE, TECHNOLOGY, AND MATHEMATICS EDUCATION, WESTERN WASHINGTON UNIVERSITY Dr. Nelson. Good morning, Mr. Chairman, and Members of the Committee. My name is Pinky Nelson, and today I am wearing my science educator hat. What resources can the mission agencies focus on? The two goals of literacy and workforce development, have skilled and knowledgeable workforce of scientists, engineers and technicians engaged in cutting-edge science and technology development focused on missions critical to the country, research and technology partnerships with industry and universities, world-class and unique laboratories and facilities, and the capacity for long-term funding. What resources do the mission agencies generally lack? Knowledge of the K-12 education system and how it is structured and regulated, internal expertise in education research, curriculum development, effective instruction, or teacher preparation. The agencies should combine their programs so those that can take advantage of their strengths and be sure to include appropriate partners when working in areas where they lack the expertise. They have the capacity to sustain and grow programs that are working and axe those that are not. Possible areas where I think the mission agencies can contribute effectively include career pathways for high-school students and mission-related undergraduate and graduate research. More challenging is participation in K-12 curriculum development and evaluation, and teacher preparation. I will briefly discuss the areas of challenge before moving on to discuss career pathways and support for research. Working towards achieving universal literacy by improving K-12 schooling requires deep collaboration with professionals across the education system, often in a non-leadership role, creating the capacity and improving the system comes first, the agencies' short and intermediate-term goals come second. There is a huge inventory of poorly-designed and under- evaluated mission-related curricula, posters, and lesson plans and associated professional development rarely used in classrooms and with no natural home in a coherent standards- based curriculum. The constant barrage of new resources adds to the noise in the system and contributes to the mile-wide, inch- deep problem. However, I do have one positive example. I recently received a copy of an astronomy curriculum for grades three to five that was developed collaboratively by NASA and the Professional Science Educators and Developers at the Lawrence Hall of Science at UC-Berkley. It is high-quality and fills a real need for instructional materials at this level. A collaborative curriculum development model such as this is rare. Adding the evaluation component could make it exemplary. My current work includes exploring the preparation of effective new STEM teachers and helping current teachers improve their practice. This is not a part-time job or one for the faint of heart. Agencies should encourage and provide incentives for their STEM retirees to become teachers. In addition, they should collaborate with excellent teacher preparation programs and support their rigorous evaluation. In high schools and community colleges, agencies can collaborate with appropriate education organizations and industry to develop and support career pathways for students, for example, in high school in high-need areas like photonics or nanotechnology. The agency can promote its mission through carefully designed, implemented, and evaluated technology programs targeting the future workforce. These programs can take full advantage of the agency talent pool. The NSF Advanced Technology Education Program has created some effective models at the community college level. Agencies could expand this work, bring it into high school, career, and technical education programs and provide sustaining funding that is not available from NSF R&D programs. Research scientists, engineers, and technicians can help museums or other informal education entities display and communicate, both in real and cyberspace, the new science and technology that is coming out of the agencies to excite and inform students, parents, and voters. Additionally, the personal stories of STEM workers at all levels, including clear maps of the paths through school that qualify them for those jobs can help motivate students to enter career pathways. Agencies can support undergraduate, graduate, and postdoctoral students to engage in mission-related research and then hire the best of them into meaningful jobs. They can provide undergraduate and graduate students authentic research experiences in their centers and laboratories, again with the prospect of meaningful jobs. As a graduate student, I spent two invaluable summers at the Air Force Cambridge Research Laboratory solar observatory in Sunspot, New Mexico. The NASA Space Grant program in Washington State is a positive example. NASA funds leveraged with a one-to-one match support around 150 graduate students every year to engage in STEM research mentored by faculty at institutions throughout the state, internships at companies, or NASA centers, or participation on student design teams. Last year 100 percent of the Space Grant scholar graduates went on to STEM graduate work or employment. While the program keeps good statistics, it could benefit from a more sophisticated effort. Thank you. I look forward to your questions. [The prepared statement of Dr. Nelson follows:] Prepared Statement of George D. Nelson Chairman Baird and Members of the Committee, it is a privilege to accept your invitation to participate in the hearing and provide my perspective on the STEM education programs of the federal mission agencies. My primary perspective comes from my recent roles in STEM education reform as Directory of Science, Mathematics, and Technology Education at Western Washington University, and my previous position as Director of Project 2061 at the American Association for the Advancement of Science. I am also Principal Investigator of a targeted Mathematics and Science Partnership grant from NSF that brings together 28 regional school districts, Washington State LASER, three state community colleges, the Northwest Indian College, and Western Washington University in an effort to reform science education with a particular focus on improving K-16 science teacher preparation. Personal experiences from previous positions have profoundly influenced my perspective towards STEM education and general education reform. I have worked as a research astrophysicist, flown three missions on the U.S. Space Shuttle as a NASA astronaut, served as Associate Vice Provost for Research at the University of Washington, and taught at all levels in higher education. I have spent considerable time thinking about and engaging in discussions with NASA and the Department of Energy about their K-12 education programs, and served on numerous advisory committees, commissions, and on boards of directors including the Pacific Science Center, the Art Institute of Seattle, and the Center for Occupational Research and Development (CORD). I am also the proud father of a dedicated and outspoken middle school mathematics and science teacher from Katy, Texas. This testimony will focus on the role of the federal mission agencies, but it is always good to keep the big picture in mind. The American education system is enormous, with over 50 million students and 3.1 million teachers. Counting the critical role of STEM learning in the elementary grades, more than half of these teachers are responsible for teaching mathematics and science. The system is also decentralized, locally funded and governed, and subject to myriad regulations. Mr. Lach has provided a compelling picture of the Chicago system. There are 15,000 other districts in America, each with its own unique strengths and challenges. Since the federal mission agencies depend so heavily on both a literate citizenry for continued support and STEM professionals at all levels to carry out their missions, it is in the interest of the agencies to contribute appropriately to achieving two STEM education goals: 1) universal math and science literacy and 2) significantly increasing the number and diversity of American students entering and successfully exiting the STEM pipeline. I shall now address the Committee's specific questions. To approach a model for how the federal mission agencies can contribute, it is reasonable to ask, what resources can the mission agencies focus on the two goals of literacy and workforce development? Here is my short list. <bullet> A skilled and knowledgeable workforce of scientists, engineers, and technicians engaged in cutting edge science and technology development focused on missions critical to the country <bullet> Research and technology partnerships with industry and universities <bullet> World-class and unique laboratories and facilities <bullet> Long-term funding. It is also important to ask, what resources do the mission agencies generally lack? <bullet> Knowledge of the K-12 education system, how it is structured and regulated <bullet> Internal expertise in education research, curriculum development, effective instruction, or teacher preparation. 1. In what ways can federal R&D mission agencies contribute most effectively to improve K-12 STEM education? Can you give examples of particularly effective programs? Taking advantage of their strengths, agency professionals can collaborate with appropriate education organizations and industry to develop and support Career Pathways for students in high schools and community colleges, for example in high need areas like photonics or nanotechnology. The agency can promote its mission through carefully designed, implemented, and evaluated technology education programs targeting the future workforce. These programs can take full advantage of the agency talent pool. The NSF Advanced Technology Education program has created some effective models at the community college level. Agencies could expand this work, help bring it into high school Career and Technical Education programs, and provide sustaining funding that is not available from NSF R&D programs. Research scientists, engineers, and technicians can help museums or other informal education entities display and communicate--both in real- and cyberspace--the new science and technology that is coming out of the agencies to excite and inform students, parents, and voters. Additionally, the personal stories of STEM workers at all levels, including clear maps of the paths through school that qualify them for those jobs can help motivate students to enter the Career Pathways. My current work includes exploring the preparation of effective new STEM teachers and helping current teachers improve their practice. This is not a part-time job, or one for the feint of heart. Agencies should encourage and provide incentives for their STEM retirees to become teachers, again making use of their talented workforce. They should also collaborate with excellent teacher preparation programs and support their rigorous evaluation. Poor preparation for entering the classroom results in ineffective instruction and low retention. 2. At the undergraduate level, what type of support could the federal R&D mission agencies provide that would recruit more students into pursuing careers in the physical sciences? Agencies can support undergraduate, graduate, and postdoctoral students to engage in mission-related research, and then hire the best of them into meaningful jobs. They can support students on campuses to work with faculty engaged in mission-relevant research. They can also provide undergraduate and graduate students authentic research experiences in their centers and laboratories--again with the prospect of meaningful jobs. As a graduate student, I spent two invaluable stints at the Air Force Cambridge Research Laboratory solar observatory in Sunspot, New Mexico engaged in cutting edge research with world- class instruments. The NASA Space Grant program in Washington State is a positive example. NASA funds support around 150 undergraduate students every year to engage in STEM research, mentored by faculty at institutions throughout the state, internships at companies or NASA centers, or participation on student design teams. Last year 100 percent of the Space Grant scholar graduates went on to STEM graduate work or employment. While the program keeps good statistics, it could benefit from a more sophisticated evaluation effort. 3. How does the lack of coordination and overarching strategy for STEM education programs hinder the agencies from making an impact? There is a huge inventory of poorly designed and under-evaluated mission-related curricula (posters and lesson plans and associated professional development) rarely used in classrooms and with no natural home in a coherent standards-based curriculum. The constant barrage of new ``resources'' adds to the noise in the system and contributes to the ``mile wide, inch deep'' problem. Effective curriculum development requires a deep collaboration with a team of professional curriculum developers, education researchers, and classroom teachers. In that light, I do have one positive example. I recently received a copy an astronomy curriculum for grades 3-5 that was developed collaboratively by NASA and the professional science educators and developers at the Lawrence Hall of Science and UC-Berkeley. It is high quality and it fills a real need for instructional materials at this level. A collaborative curriculum development model such as this is rare. Adding a rigorous evaluation component to explore ho well the curriculum helps teachers teach and students learn could make it exemplary. Summary A focus on 1) partnering with high schools and community colleges along with appropriate education professionals and industry partners on mission-related technology education programs for the future technical workforce, and 2) supporting mission-related research for undergraduate and graduate students both in agency facilities and on university campuses could pay major dividends. This would require an achievable overarching strategy, but not necessarily significant coordination among the agencies. The critical collaboration would be with STEM education professionals (not just K-12 teachers), university faculty, and industry partners. Biography for George D. Nelson Dr. George D. Nelson is the Director of Science Mathematics, and Technology Education and Professor of Physics and Astronomy at Western Washington University in Bellingham, Washington. The program is responsible for the preparation of future K-12 science, mathematics, and technology teachers. It is also a research and development center with a focus on teacher preparation and science, mathematics, and technology education reform. He is currently the principal investigator on a $12 million NSF project, the North Cascades and Olympic Science Partnership. Prior to joining Western Washington University in 2002, Dr. Nelson was Director of Project 2061 and a member of the senior staff of the American Association for the Advancement of Science. Project 2061 is engaged in the reform of science, mathematics, and technology education at all levels with a focus on helping to create a system where all high school graduates are literate in science, mathematics, and technology. Under Dr. Nelson's leadership Project 2061 produced a number of ground- breaking publications including Blueprints for Reform, Designs for Science Literacy, and the Atlas of Science Literacy. The project also developed unique and rigorous procedures for evaluating curriculum materials and assessments, and greatly expanded its professional development activities. From 1989 to 1996 Dr. Nelson was Associate Vice Provost for Research and Associate Professor of Astronomy and Education at the University of Washington. His administrative responsibilities included research policy, government-university-industry interactions, university-K-12 education interactions, and federal relations. He taught graduate courses in stellar atmospheres and solar physics and undergraduate courses in general astronomy. In the college of education he taught an innovative seminar on science education for scientists, graduate students, and teachers using Project 2061 as the underlying foundation. During the 1992-93 academic year, Dr. Nelson was a fellow of the American Council on Education. From 1978 to 1989 he served as a NASA astronaut and flew as a mission specialist aboard three Space Shuttle flights. These missions included the first on-orbit satellite repair in 1984. Dr. Nelson was the pilot of the first operational flight of the manned maneuvering unit and the primary extra-vehicular crewman. He also served on the crew of the flight of Discovery in September 1989 immediately following the loss of the Challenger and was extensively involved in the rework of all crew procedures and the re-engineering of Space Shuttle components and software. He has advised NASA through service on a number of committees, most recently as Chair the Hubble Space Telescope Servicing Missions 3A and 3B External Independent Readiness Review Team. Dr. Nelson has served on several boards of directors including the Art Institute of Seattle, Analytic Services Inc., and the Pacific Science Center. He received his B.S. in physics from Harvey Mudd College and M.S. and Ph.D. in astronomy from the University of Washington. His research interests include science education, education reform, and radiative transfer and hydrodynamics applied to interesting problems in astrophysics. He lives in Bellingham with his wife, Susie. They have two grown daughters; Aimee Nelson-Engle and Marti Nelson-Frazier and three perfect grandsons, Pierce, Langston, and Andrew. Chairman Baird. Thank you, Dr. Nelson. Mr. Van Reiner. STATEMENT OF MR. VAN R. REINER, PRESIDENT AND CEO, MARYLAND SCIENCE CENTER, MARYLAND ACADEMY OF SCIENCES Mr. Reiner. Good morning, Mr. Chairman and Members of the Subcommittee. I am Van Reiner, President of the Maryland Science Center located in Baltimore's Inner Harbor. We have three levels of interactive, hands-on exhibits, a planetarium, an IMAX theater, classroom space, and a program space for live science demonstrations. Each year we admit 100,000 students attending with school groups to augment their science and mathematics curriculum. We are a member of the Association of Science Technology Centers, Incorporated. We, like hundreds of institutions across the country, employ what is known as informal education as a way to connect people with science and technology. Learning by doing is the basis for our approach. Showing how, rather than stating why, gives visitors the information they need to make informed decisions about how to relate the topic at hand. When we are successful, we go from global to local to individual by giving the facts, a better understanding of the topic or how it relates to them, and hopefully a quest for more knowledge. Motivating students to take interest in science, technology, engineering, and math, whether or not they choose to pursue a career in those fields, puts science centers in a unique position to spark an initial interest. Collaborations are essential to the success of science centers, and we have a history of collaborating with the federal R&D mission agencies. The longest running collaboration is with NASA. We have co-hosted events with NASA such as having students participate in televideo conferences with the astronauts aboard the International Space Station and the Shuttle, as well as watching a European solar eclipse while talking to NASA Goddard scientists on board a ship in the Black Sea. We have helped to develop after-school astronomy programming and are embarking on a citizen science project to measure the amount of UV radiation that reaches the earth at Baltimore's Inner Harbor. Scientists from federal agencies participate in our ``scientist of the month'' program to interact directly with our visitors to discuss current research findings. Other instances with NASA and other agencies are listed in my written testimony. Evaluation of these programs and exhibits have been performed in our institution by us. The accepted practice for informal education institutions such as ASTC members has been to do front-end, formative, remedial, and summative evaluations of the program or exhibit by a third party to be sure that stated goals of the project are met. These evaluations are required for NSF or NIH grants, and we use them for other federally sponsored exhibits and programs. Several of these evaluations are included in the attachments to my written testimony, and I apologize for the length of them. [Evaluations included in Appendix 2: Additional Material for the Record.] These evaluations are thorough and complete and help us to know if we have met the requirements of the project and if the audience understands the subject presented. We feel that without this evaluation we would quickly lose our relevancy. Currently we are collaborating with NASA and NOAA on evaluating a project called Science on a Sphere, the globe identical to the one that Queen Elizabeth II visited at Goddard Space Center last week. We have been asked by NOAA to lead the users group to work with the agencies to develop evaluation methods specifically for the exhibit as well as the programming that the group develops around the exhibit. This collaboration between NOAA and NASA is unique and should be encouraged. Scientists from the two agencies are working together to ensure that the data presented is clear and meaningful. Increased collaboration between federal R&D mission agencies and science centers can better accomplish the goals of STEM education programs. The dialogue between federal mission agencies should be expanded so that the general public and students can be presented with knowledge in larger and more meaningful ways. We believe that greater understanding leads to greater acceptance that science is resident in everything we do. It just doesn't happen in a laboratory. Science centers are a resource in every sense of the word and should be viewed as such. We know how visitors react and how to best present scientific discovery and scientific progress to the public. We believe greater utilization of science centers as resources for federal R&D mission agencies is the best way to help raise the level of scientific literacy with the general public, including students. We can, and do augment, the formal classroom instruction using resources that would be either too expensive or too impractical for the classroom. I believe that federal R&D mission agencies should be required to allocate a portion of their resources to educate the public as is now required by the National Science Foundation. I thank you for this opportunity, and I look forward to your questions. [The prepared statement of Mr. Reiner follows:] Prepared Statement of Van R. Reiner Chairman Baird, Ranking Member Ehlers, and Members of the Subcommittee: The Maryland Science Center, located at Baltimore's Inner Harbor, is a private non-profit that had its genesis in 1797 as the Maryland Academy of Sciences, which is still our parent organization. The current building was put in service in 1976 and expanded in 2004. The facility houses three levels of highly interactive, hands-on exhibits, a planetarium, an IMAX Theater, classroom space, and program space for live demonstrations on a variety of scientific phenomena. Each year, Maryland Science Center welcomes over 400,000 visitors to our facility with about 100,000 students visiting with school groups to augment their science and mathematics curriculum. Major areas of concentration--our core programming areas--are Earth system science, space and aerospace science, health sciences and the human body, and early childhood education. All of the permanent exhibits at the Maryland Science Center were designed to be in concert with the Maryland Department of Education Science Curriculum, and where applicable, under the influence of The Benchmarks for Science Literacy published by the American Academy for the Advancement of Science as part of Project 2061, the National Science Education Standards prepared under the auspices of the National Research Council, and the Principles and Standards for School Mathematics from the National Council of Teachers of Mathematics. The Maryland Science Center, like hundreds of institutions across the country and around the world, employ what is known in the field as ``informal education'' as its means of connecting people with science and technology. Learning by doing is the foundation of informal education, along with demonstrating practical examples. Building on that foundation, the role of the science center as distiller and interpreter of the latest in scientific discovery and connecting the public, especially school children, to the everyday application of these advancements, is a key strategy we employ. Showing how, rather than stating why, gives visitors the information they need to make informed decisions about how they can relate to the topic at hand. When we're successful, we go from global to local to individual along a continuum giving the individual the facts, a better understanding or how it relates to them, and hopefully, a quest for more knowledge. Science centers in particular have an ability to affect change by engaging school children through their use of informal education methods. Motivating students to take interest in science, technology, engineering and math, whether or not they choose to pursue a career in those fields, puts science centers in a unique position to spark an initial interest. Collaboration has always been a means to an end for the institution. Seeking partnerships to ensure that our exhibits and programs are the best they can be has yielded quality educational and enjoyable product for our visitors. The Maryland Science Center has long cultivated a history of working with the National Aeronautics and Space Administration (NASA), National Oceanographic and Atmospheric Administration (NOAA), National Institute of Standards and Technology (NIST), as well as the National Science Foundation (NSF) and the National Institutes of Health (NIH). Working with these agencies, and others, we develop permanent exhibits, traveling exhibits, and programs designed to inform and educate the general public--from those school children to their parents and teachers--about not only the basics of science, math and technology, but also the latest events and discoveries in our core programming areas. In our core exhibits we use high tech, multimedia update centers known as Links. Each Link is designed to offer real time investigation of science topics in the news that are appealing to mass audiences. When something happens in the world of science and technology, our Link areas and Link staff key in on the news releases, scientific data, and information resources to process and present the findings in a relevant, thought-provoking manner. Our Links are designed to give visitors who seek cursory, as well as deeper understanding of science as it happens, a chance to speak with our content experts, and to explore areas that can be a resource for the quest for deeper knowledge. Maryland Science Center currently features three Links. TerraLink focuses on Earth system science, SpaceLink concentrates on space and aerospace science and BodyLink examines health sciences. The fabrication and design of these centers was accomplished with input from the aforementioned agencies, and we continue to interface with these agencies to stay on top of current research. In the SpaceLink update center, NASA has been a primary partner. The partnership has taken many forms. With the Goddard Space Flight Center, we have helped develop an after school astronomy club format. The results can be seen at www.afterschoolastronomy.org. This site is a resource that provides information for students with an interest in astronomy and put the information to use in practical applications, a hallmark of informal education. We have also co-hosted individual events with Goddard such as Sun-Earth Day, where educators from Goddard are at the Maryland Science Center to explain that we live in the atmosphere of a star, which has many effects on our planet. We are part of a larger system and understanding that system is vital to other scientific endeavors we attempt to explain. This is an annual event where visitors engage in activities and receive print materials to take with them that allow them to further explore the topics at home. Again, relevancy and real world examples--informal education in process. Maryland Science Center and Goddard also co-hosted an Earth Explorer Institute where we convened 25 informal educators representing science centers and museums from across America to discuss and recommend NASA Earth science programming for informal audiences. As an outgrowth of that, we are participating in a UV Citizen Science project. We will enlist citizens to measure the amount of UV radiation that hits the Earth at Baltimore's Inner Harbor, enter the data into a nationwide database, and then participate in ongoing work to measure changes in UV radiation across the country. People are exposed to scientific equipment, scientific methods, and will be able to participate in a nationwide study. They also gain a greater understanding of the implications of changing UV levels as it relates to their everyday lives. Maryland Science Center makes its resources available to provide opportunities for students to witness events such as a solar eclipse. In August of 1999, a group of students observed a solar eclipse in Europe via a link with Goddard scientists on board a research vessel in the Black Sea. Students came to the Maryland Science Center to learn about the eclipse and participated in two televideo conferences before and during the event to observe and ask questions. In partnership with the Johnson Space Center, Maryland Science Center has hosted Baltimore City School students for four live downlinks from the International Space Station and the Space Shuttle since 2001. Students have conversations as well as question and answer sessions with the astronauts aboard the Station and the Shuttle. Prior to the downlinks, students visited the Maryland Science Center to learn about the specific mission and prepare questions for the astronauts. The missions included the delivery of the Destiny component to the International Space Station, and the Hubble Space Telescope servicing mission. After these two missions, the entire Shuttle astronaut crew involved in the downlink visited the Maryland Science Center to meet with the students who participated in the conference and the general public. We were the first informal education institution to have this opportunity, and mentored other museums on how to replicate the experience. With the Space Telescope Science Institute, we were advisors on the making of a short IMAX film entitled ``Hubble: Galaxies Across Space and Time.'' This three-minute film has been shown in over a dozen IMAX theaters nationwide. At the Maryland Science Center, it has had 1,564 screenings for over 110,000 visitors. We have also developed a planetarium show to explain what the Hubble has allowed us to see and how those images have helped to shape the way we view the universe as well as increasing our knowledge of our place in the universe. Through collaboration, the Maryland Science Center participates in other smaller programs funded by NASA as part of a larger grant to another institution. We will create a series of programs and events highlighting the New Horizons mission to Pluto and the Kuiper Belt. We will host a small exhibit, offer a teacher workshop and host a Pluto Family Science Night. Keying on the recent news and popular culture references to Pluto's status as a planet or not, Maryland Science Center will present the latest Pluto information as part of a popular planetarium show ``Planet Trek.'' In conjunction with the Howard Owens Science Center in the Prince George's County, Maryland school system, we are to develop a planetarium program on Pluto and the New Horizons mission for distribution to school planetariums nationwide--currently numbering in excess of 600. In TerraLink, the Earth systems science update center, Maryland Science Center has partnered with NASA and NOAA to provide ongoing support for programming including visual material and scientific expertise. NASA and NOAA scientists periodically visit to work with students as part of our Scientist of the Month program and on special programming days such as Earth Day. Students and visitors have a chance to see science pursuits as both a vocational option and simply as a means to broaden awareness that science, technology, engineering and math is not a narrow cast field of inquiry and exploration. The focus of this program is to provide science and technology careerists as role models for students as well as being able to offer another thread for visitors and students to seek out information about how things like atmospheric phenomena occurs. TerraLink staff and the Science Person of the Month collaborate to present topics and information to the public in understandable terms. Using NASA and NOAA data and visual imagery, as well as utilizing experts from the agencies in the Science Person of the Month program currently defines the extent of Maryland Science Center collaboration with the agencies named in this inquiry. It should be noted however that prior to the institution's recent expansion and broadening of its core competencies, the programmatic synergies between Maryland Science Center and these agencies was limited by definition of scope and mission. Currently BodyLink, the Maryland Science Center's health sciences update center, collaborates and partners with other federal agencies-- primarily NIH, through its Science Education Partnership Award program--but does not at this time enjoy a relationship with the agencies named in this inquiry. However, topics like the studies of the effects on the human body of extended durations of time spent living in space are of interest to BodyLink staffers and Maryland Science Center and the opportunities to collaborate and deliver programming similar to that which is already in place in SpaceLink and TerraLink are currently tracking with our institutional collaborative goals. Beyond the Link areas of our core exhibits and programming, and in partnership with NOAA, Maryland Science Center has embarked on an exciting project entitled Science On a Sphere (SOS). This is an Earth visualization system developed by NOAA that projects a wide variety of data sets onto a large sphere to create dynamic global views of the entire Earth. Visitors observe hurricane development and prediction, tectonic plate movement and earthquakes, sea surface temperatures and their effect on global weather conditions, as well as observe global warming models and the potential effect on the Earth. Science On a Sphere is now a permanent exhibit at the Maryland Science Center. Having this technology also allows us to compare Earth to other planets and NASA data sets have been converted to show the Moon, Mars, Saturn and the Sun on the same sphere. We have, in collaboration with NOAA and NASA, developed Maryland Science Center staff-delivered, visitor- centered, programs as well as produced prerecorded programs that explain the images being observed. We have also developed traveling exhibits. NIST was instrumental in providing technical information for our Titanic Science exhibit. NIST performed analyses of rivets from the Titanic hull that were found at the wreck site to determine the strength level of the rivets, so our information would be factual. One of the questions surrounding the Titanic disaster was whether or not the steel used to make the rivets was of poor quality--and if that might explain how the ``watertight'' features failed. By presenting the data and the surrounding conditions, visitors were left to their own conclusions as to how the rivets might have contributed to the Titanic's end. Although we were not asked to speak directly to our collaboration with other federal agencies, two examples of Maryland Science Center collaboration with the National Institutes of Health are funding and content expertise for a traveling exhibit titled: The Changing Faces of Women's Health and funding and content expertise for our permanent health sciences update center BodyLink. With the exception of Science On a Sphere, the evaluation of permanent exhibits has been done by the Maryland Science Center. The accepted practice for informal education institutions, as exemplified by the Association of Science-Technology Centers, (ASTC), has been to include a front-end evaluation as part of any project. This involves determining what the public knows about the subject through focus groups and questionnaires. The project is then judged as to what is feasible to build and install, and through prototyping of exhibit pieces, determining if the public will understand the idea or concept presented by the various exhibit pieces. When the project is complete, a summative evaluation is performed to see if the stated goals of the project have been met. This is done through direct observation and public feedback solicitation, usually by an independent third party. If there are changes to be made with the project, a final, remedial evaluation is made to ensure that the intended knowledge transfer has been made. (Attachment B and Attachment C accompanying this testimony illustrate an example of this evaluation process which we completed as part of our development of our permanent Earth science and dinosaur exhibit. The exhibit was produced with funding and content support from NASA).[Located in Appendix 2: Additional Material for the Record.] With educational programs, the process is similar. Educators are solicited for areas where an informal experience can add to the students' understanding of the subject matter. When the programs are developed and delivered, feedback is given directly by the educators who bring their students to the center for the educational enrichment. These accepted evaluation procedures are required for National Science Foundation or National Institutes of Health grants and have been used by science museums for other federally-sponsored exhibits and programs. Using the accepted evaluation practices mentioned above, NOAA, NASA and the Science On a Sphere users group (made up of all centers with a sphere installed as well as those centers where spheres are being installed), have embarked on specific evaluation methods for the exhibit as well as the programs centered around the exhibit. Each funded NOAA project contains a detailed evaluation plan. NOAA asked the Maryland Science Center to lead a discussion of all SOS users on the different SOS evaluation methods used to date and what method of prototyping and evaluation will best help science museums develop understandable exhibits and programs for the target school group audiences and the general public. Using front end evaluation (a copy of the full evaluation can be found in Attachment A; see Appendix 2: Additional Material for the Record) of the Science On a Sphere exhibit--again conducted by third party evaluators--Maryland Science Center was able to implement and utilize the SOS exhibit in response to the feedback collected during the evaluation process. In our case, we developed specific staff-led programs to augment the SOS experience for our visitors. Overwhelmingly, the display of the information and data, the quality of the presentation, overall appeal of the technology, and understanding of the purpose of the exhibit was extremely positive. Once operational at Maryland Science Center however, our exhibits team noticed that when the SOS exhibit ran in auto-play mode using ``canned'' presentations, and no staff members were available to augment the presentation and answer questions about the data being presented, the level of engagement was short in duration. In general, when the SOS exhibit was facilitated by Maryland Science Center staffers, questions were answered, programming could be paused for explanation, and dwell time (time people spend at a specific exhibit) by visitors was very long in duration. When the SOS exhibit ran in auto-play mode--meaning the canned programs, with their taped narrative--the dwell time for visitors was far shorter. Visitors could not fully comprehend the auto- play presentations and moved on to other exhibit areas more quickly. ``Canned'' programming for SOS, absent a subject expert who could interpret the presentation for the casual enthusiast, was at too high a level. Programmers may have assumed too high a level of understanding on the part of the museum-goer and the exhibit was losing audience as the visitor became confused or could not fully understand the presentation. In response to this, Maryland Science Center exhibit team members installed interactive computer kiosks around the SOS exhibit that offered a more basic interpretation of the imagery and programming being presented when the exhibit is in auto-play mode. Visitors can glean basic understanding of the programming's more technical aspects by viewing a more basic interpretation on the interactive kiosk screen. Given this more basic knowledge, the visitor is given the tools necessary to gain a deeper understanding of the original intent of the more specialized canned programming. Program staff have also inserted more facilitated programs into the presentation schedule to engage more visitors more often using the SOS exhibit. To evaluate and measure the success of the remedial actions the exhibit team completed a dwell time study of visitor interaction with the SOS exhibit prior to the installation of the interactive kiosks to create benchmark dwell time statistics. Now that the kiosks are in place, the dwell time study will be repeated and the data will be compared to the benchmarks created prior to the kiosk's arrival. All the information gathered, the remedial actions employed, and the measures of success are being shared and reported to the NASA/NOAA led SOS users group so that the exhibit is as successful as possible at all locations around the country. As the installation of SOS exhibits began their roll out, greater collaboration between program creators and informal educators earlier on could have led to programs that did not need as much remedial modifications and augmentation. Partnering in the development stage may have gained SOS more audience and enthusiasm more quickly and in greater numbers. The case with SOS illustrates an example of how to improve scientific literacy. To improve the effectiveness of using informal education to help raise the level of scientific literacy in the United States, emphasis should be placed upon how non-classified information could be made available to the general public. Informal educators such as ASTC members have the ability to dispense highly technical knowledge in a manner that the non-scientific public can understand. In the case of the Maryland Science Center, we employ a cadre of on-the-floor explainers, many of whom have received training from the various federal R&D mission agencies to augment their own formal education. For every hour that we are open, we have staff members ready to engage our visitors to answer questions or offer ideas that stimulate meaningful discussions about the subject areas. Our goal is to make gaining this knowledge engaging and fun, while showing how science and technology affect our daily lives. In the process, our hope is that we will excite and encourage some of our student visitors to consider careers in science and technology. In the example provided about the installation, evaluation, and ongoing collaboration with the Science On a Sphere exhibit there is a working example of how federal agencies and science centers can better accomplish the goals of STEM education programs. We would encourage federal agencies to continue to expand ways that researchers and engineers collaborate with informal science education professionals to better engage the public. With SOS, scientists were made available to us, evaluation was encouraged, and NASA and NOAA sought our help in getting the message out. There was recognition that science centers, through their use of informal education, know how to engage visitors and spark their interest in the sciences. We know how visitors react and how best to present scientific discovery and scientific progress. And we know how to present it in ways that matter to them as individuals. The dialogue with science centers should be expanded--we want access to the knowledge and the discovery so we can distill, interpret and present it to the general public and school children in larger and more meaningful ways. We want to reach greater numbers of people more often so that scientific discovery becomes as much a part of a person's everyday life as it can be. We believe that greater understanding leads to greater acceptance that science is resident in everything we do--it doesn't just happen in a laboratory. Science centers like the Maryland Science Center are a resource in every sense of the word and deserve to be viewed as such--from resources (financial and otherwise) to expertise and knowledge. We believe greater utilization of science centers as resources for federal R&D mission agencies is the best way help raise the level of scientific literacy with the general public, including school children. Biography for Van R. Reiner Mr. Reiner, a native of Lakewood, Ohio, holds a Bachelor's degree in chemistry from Wittenberg University and a Master's degree in chemistry from Lehigh University. He also has completed executive education programs at Duke University and the Wharton School of Business at the University of Pennsylvania. He joined Bethlehem Steel in 1974 and spent the first 10 years of his career as an engineer in the coke oven department at Bethlehem's Lackawanna, NY, plant. In 1984, he was transferred to the Burns Harbor, IN, Division serving as Assistant Superintendent, coke oven division. In 1987 he moved to Assistant Superintendent of the slab mill/plate mills department and in 1990 was promoted to Superintendent of that department. In 1995, Mr. Reiner was appointed Superintendent of the Galvanized Products Division, an operating unit of Burns Harbor located at Lackawanna. He became Senior Manager, Operations, for the Burns Harbor Division in 1997, and President, Bethlehem Lukens Plate, when that division was formed in May 1998 following Bethlehem's acquisition of Lukens Inc. He was then named President, Bethlehem Sparrows Point Division in August 2000. Professionally, Mr. Reiner is a past member of the board of directors of the American Institute of Steel Construction. He also held memberships in the American Iron and Steel Institute, Association of Iron and Steel Engineers, American Chemical Society, Western States Blast Furnace and Coke Oven Operators Association. Since the closing of Bethlehem Steel, Mr. Reiner has served as interim Director to the Maryland Science Center in Baltimore, MD, and in March of 2005, was named its permanent President and Chief Executive Officer. Mr. Reiner also serves on the Board of the Maryland World Class Manufacturing Consortia and is a founding member and Treasurer of the Partnership for Baltimore's Waterfront. In the past, he has served on the following Boards: Chester County, PA Chamber of Commerce, United Way of Chester County, Lackawanna, NY Chamber of Commerce, President of the Board of Lackawanna Community Development Zone. He and his wife, Shirley, are the parents of three children. They reside in Bel Air, MD. Chairman Baird. Thank you, Mr. Reiner. I would mention that there is no need to apologize to this committee for providing additional material and particularly evaluative. We appreciate the effort you folks have done to evaluate your program effectiveness and value the information very much. Dr. Weiss. STATEMENT OF DR. IRIS R. WEISS, PRESIDENT, HORIZON RESEARCH, INC. Dr. Weiss. Mr. Chairman, Members of the Subcommittee, thank you for the opportunity to participate in this hearing. My name is Iris Weiss, and I have spent the last three decades in research and evaluation in STEM education. I would like to share my thoughts on two issues, the first is how program evaluation can help the federal R&D mission agencies be more efficient and effective wherever they choose to focus their efforts to increase scientific literacy, and the second is where I believe these agencies should focus their efforts. To date the federal R&D mission agencies have not had a great deal of success in evaluating their STEM education programs. The same can be said for other federal agencies and for the broader field as well. How could evaluations be improved? First, the designs of proposed programs should be critiqued to determine if the interventions are likely to lead to the desired outcomes and how broad the impact would likely be, so programs could be improved before major costs are incurred. To take one example, the Department of Energy offered science teachers summer employment in their research labs. Program goals include deepening participating teachers' knowledge of science and improving classroom practice at their schools. But a design critique would suggest that the program would be unlikely to achieve its classroom impact goal. Few teachers would have the time and expertise needed to develop student activities that were accurate, developmentally appropriate, and feasible to implement with the resources likely to be available, nor would the participating teachers be likely to have the time to help other teachers improve their classroom practice. A design critique might well have predicted what in fact happened. Teachers appreciated being involved in the program, reported that it deepened their understanding of scientific content and scientific research, but it did not have much of an impact on classroom practice. Similarly, formative evaluation of pilot programs would help the agencies be more efficient and effective in their STEM education efforts. At the pilot stage, the focus is not on impact but rather on whether the program can be implemented as intended, how it might be improved, or if it needs to be discontinued. There is no question that impact evaluations need to be improved as well, as the just-released report of the ACC makes clear. At the same time I believe the challenges associated with rigorous evaluations of education programs have not been adequately acknowledged in that report. In addition, effective evaluations require not only strong research design but also appropriate outcome measures. Although developing instruments to assess teacher content knowledge and similar goals is not the responsibility of the federal R&D mission agencies, I believe that the lack of appropriate measures will continue to hamper the mission agencies in efforts to increase their program effectiveness. Where should the federal R&D mission agencies focus their efforts to improve scientific literacy? Based on my understanding of the complexities of the K-12 education system and the expertise of these agencies, I believe they should play a relatively small role in efforts to improve the formal K-12 education system and a larger role in the informal science arena. For example, current evidence suggests that teacher professional development is most effective in improving classroom practice when it is closely tied to instruction. We know that teacher content knowledge is necessary, but it is becoming increasingly clear that it is not sufficient. Teachers also need to learn how to use their instructional materials well, how to figure out what their students understand and where they are struggling, and how to make appropriate instructional decisions based on that information. And teachers need opportunities to apply what they are learning in their own classrooms and to get constructive feedback. The federal R&D mission agencies certainly have the content expertise to provide professional development, but they have only limited understanding of K-12 education and they are not well- positioned to provide professional development that is practice-oriented and sustained over time. For greater and broader impact, rather than developing their own programs, I believe the federal R&D mission agencies should consider making scientists available to serve as content resources for local professional development, helping shore up a major weakness of many of those programs. In contrast, I believe the federal R&D mission agencies are very well-positioned to make major contributions in the informal education arena along the lines we have just heard. Lack of coherence is not a problem here, as it is when we talked about curriculum development. In fact, having multiple pathways increases the likelihood that a large number of people will benefit from the available resources. Thank you. [The prepared statement of Dr. Weiss follows:] Prepared Statement of Iris R. Weiss I appreciate the opportunity to testify before the Research and Science Education Subcommittee as it explores how the federal R&D mission agencies can contribute to improved scientific literacy for all students. There is no question that there is a wealth of scientific expertise in the various agencies, and considerable interest in helping to improve K-12 STEM education. Moreover, much of the work of these mission agencies focuses on areas that are of intrinsic interest to students, and can help motivate students both to engage in learning science and to consider STEM careers. With appropriate programs, carefully designed and well-implemented, the federal R&D mission agencies can both enhance levels of scientific literacy in the population as a whole and help ensure an adequate supply of well- qualified STEM professionals for the future. It is important to recognize, however, that there are many more ``good ideas'' (i.e., possible programs in areas of relevance to the agency's mission that have the potential to increase teacher knowledge, improve classroom practice, and enhance student knowledge and aspirations) than can possibly be implemented. There are substantial costs involved in designing, implementing, and evaluating new programs, and very limited resources available for these activities. Clearly there need to be criteria for deciding which of the many potentially good ideas should be implemented by a particular agency, and processes for deciding how to refine promising programs, which ones to scale up, and which ones to drop. The hearing charter makes clear that the goal is to increase the level of scientific literacy for all students. It is important, therefore, to consider the nature and scope of the K-12 education system that the mission agencies are trying to influence--50 states, more than 15,000 school districts, more than 100,000 schools, and millions of teachers responsible for STEM education, textbook publishers, test developers, etc. all making decisions that affect student opportunities to become scientifically literate. In addition, while there have been efforts to identify the core understandings that constitute scientific literacy, the volume of content included in national and State standards documents is still much more than can possibly be addressed in depth in the time available. Teachers and curriculum developers are faced with the unenviable choice of trying to cover it all, and doing so superficially; or taking seriously the recommendation for in-depth, inquiry-based learning, and leaving out some of the required content. In this context, current and potential programs have to be examined not only to see if they are effective in terms of adding value to the participating teachers/students, but also whether there are likely to be sufficient indirect benefits to a large enough number of students to make a meaningful difference in overall scientific literacy. At present, the problem I see with many federal programs, including those of the R&D mission agencies, is that they have very limited potential for leverage and in some cases simply add to the confusion. How can program evaluation help federal programs be more efficient and effective in improving STEM education? The federal R&D mission agencies have not had a great deal of success in evaluating their STEM education programs; the same can be said for other federal agencies, and for the broader field as well. The natural desire to address the pressing problems in science education has taken precedence over the need to ensure that the investments will in fact have the intended impact. I believe that existing program evaluation tools and approaches can help increase the likelihood that STEM education programs supported by the federal R&D mission agencies (and others as well) will have a broad, positive impact. Evaluation is useful at various stages of a program. It can and should be used in (1) critiquing proposed programs to help make decisions about which ones to offer and to improve their designs; (2) monitoring program quality both to allow appropriate mid-course corrections and determine if the program is ready for rigorous evaluation; and (3) assessing program impact. At present, it appears that some of the tools and approaches that evaluation has to offer are used some of the time in some of the STEM education programs supported by the federal R&D mission agencies; their more consistent application would help improve the quality, impact, and cost-effectiveness of the agencies' efforts to enhance overall scientific literacy. Evaluation as design critique In terms of program design, the first step any agency needs to take is to identify needs relevant to their mission and expertise. The federal R&D mission agencies have been very successful in this regard; virtually every program they offer can be readily mapped both to the mission of the agency and to the needs of the designated target audience(s). But targeting an appropriate need does not necessarily mean that the programs are addressing priority needs; one can assume that at least some students and teachers lack knowledge in any given area of science, and that many more students and teachers are likely to lack knowledge in areas at the cutting edge of science. Since any program aimed at increasing teacher or student knowledge could be justified by making the case that it addresses an existing need, simply being able to demonstrate need is not an adequate criterion for making decisions among potential programs. Given scarce resources, agencies need to be able to decide which of the many needs that are consistent with their mission are the most important to address, and which of those they have the capacity to address well. Only then does it make sense to move ahead with program development. Not having been part of the program planning discussions, I can't tell the extent to which the mission agencies' STEM education program rationales were made explicit and the various priorities debated. But my impression from the multitude of topics, grade ranges, and approaches the various agencies are using is that decisions have been made based on whether a particular idea was of interest to someone in a decision-making position, rather than whether the program was part of an overall, coordinated strategy for maximum leverage on K-12 education. Even more important than whether a program targets a priority need is whether the proposed intervention is likely to have the desired impact; no matter how important the need, ineffective programs are a wasted investment. Conducting a ``design critique'' of a proposed program can help improve the design, or in some cases lead to a decision not to go forward with programs where the odds are stacked against them. And the very good news is that design critiques are not an expensive undertaking; they require only modest amounts of time from people who understand both the system that is being targeted for improvement and what has been learned in prior efforts. We need to pay more attention to the fact that STEM education programs that either have little likelihood of impact, or will impact only a small number of teachers/students, are not going to make much of a difference in overall scientific literacy. Again, the criterion of likely impact based on prior research and the ``wisdom of practice'' seems not to have been uniformly applied in the STEM education programs offered by the federal R&D mission agencies. To take one example, the Department of Energy has at various times offered science teachers summer employment in their research labs, an expensive undertaking given the costs of salary, transportation, and lodging. The goals of the program have been to deepen participating teachers' knowledge of science, and to improve instruction not only in the participating teachers' classrooms, but in those of their colleagues at the school as well. Developing a ``logic model,'' a standard tool in program evaluation, would have enabled the designers of that program to see that there were major holes in the program's theory of action, places where the links between activity and impact were weak at best. One could readily make the case that teachers would learn more science, and learn more about scientific research, by being placed in a research laboratory. However, the science content teachers were learning was likely to be well beyond what their students would be expected to learn, and they would not have the sophisticated equipment needed to carry out the investigations. Few teachers would have the time and expertise needed to develop instructional activities to make the activities developmentally appropriate for their students and feasible to implement with the available resources; nor would participating teachers have the time to help other teachers apply what they had learned. Thus, while teachers who participate in these kinds of programs often report that they gained a great deal from these experiences, it should not be surprising that the improved classroom practice that was a major goal of the programs rarely materialized. In this case and many others supported by federal, state, and local agencies, considerable resources have been devoted to programs where lack of classroom-level impact could have been anticipated. Formative evaluation to enable mid-course corrections and determine if PROGRAMS ARE READY FOR RIGOROUS EVALUATION Given the start-up costs associated with the development of any new program, it make sense to fund only those that have great potential to begin with, and then based on the lessons learned during implementation to refine the programs to get the kinks out. Evaluations of the mission agencies' STEM education programs would also be improved by more systematic attention to monitoring the quality of program implementation and use of the resulting feedback. From an external perspective, the fact that some initiatives have been modified over time suggests that at least some of the federal R&D mission agencies employ formative evaluation strategies for at least some of their STEM education programs. It is less clear whether the STEM education programs supported by the federal R&D mission agencies use evaluation for quality control purposes when programs are expanded. Often the people who design a program, e.g., for teacher professional development, are able to implement it well, but when the program is expanded the quality tends to suffer. It is important both to monitor initial program implementation and fine-tune the design as needed, and to monitor the quality of implementation during scale up. Ideally, evaluations of the quality of implementation would include observations of program activities by people who have expertise in both content and the target populations; interviews with key stakeholders, including in many cases students, teachers, administrators, and parents. Often it is appropriate to collect some interim data on impact to see if the design needs to be fine-tuned, or additional support provided to program implementers. Sometimes a preliminary evaluation provides evidence that a program is unlikely to achieve its goals, so a more rigorous evaluation is not necessary. For example, my organization was once asked to evaluate a statewide program that had the goal of ``transforming elementary science education.'' One of the primary interventions was having STEM faculty visit classrooms--typically once a semester--and model for teachers how to conduct science demonstrations. The client wanted evidence to see if this strategy was paying off in terms of improved classroom practice. Recognizing the limitations of survey self-report data, they asked that we do classroom observations, which would have required site visits to a fairly large number of treatment and comparison classes, clearly an expensive undertaking. From our perspective, finding out that something that could not possibly work in fact did not work seemed to us to be a poor use of both our time and taxpayer money; we convinced the client to let us interview a small number of teachers before committing to a more extensive evaluation. Teachers told us that (1) they were happy to have scientists visit their classrooms because the kids enjoyed it and got a better sense that scientists were like most people, not nerdy beings in laboratory coats; (2) they thought it would be a good idea if they did demonstrations like the scientists had done, but acknowledged that they rarely did so--they didn't know whether the demonstrations would ``work;'' they didn't have the necessary materials; and they were concerned that they wouldn't be able to answer questions students raised. In this case we were able to convince the client to forego a rigorous evaluation, but not, unfortunately, to revamp the clearly ineffective program. As another example, if materials have been developed for classroom use, but initial evaluation data show that teachers aren't using the materials because of they do not appear to be well-aligned with state standards, time and effort spent doing a careful evaluation of impact on student learning would not be warranted. Given the substantial costs involved, only programs that have a reasonable likelihood of substantial impact and can be implemented well should be subjected to rigorous evaluation. It is particularly important to provide incentives for agency personnel to use evaluation feedback for program improvement, rather than allowing people to continue to implement poorly designed, inadequately implemented, or ineffective programs. Unfortunately, there appear to be pressures at every level of the system for people to overstate the success of their programs, highlighting positive aspects and glossing over problems, which may help explain the observation that almost everything appears to work, but nothing much changes. Summative evaluation to assess program impact What most people mean by program evaluation has nothing to do with design critique or studying the quality of implementation; rather evaluation is typically equated with an assessment of the impact of a particular activity or set of activities. It is important to recognize that rigorous evaluation is very difficult, and it is therefore not surprising that the federal R&D mission agencies have encountered many challenges in assessing the impact of their STEM education programs. First, as a profession, we lack instruments to measure many of the outcomes we care about. For example, many STEM education programs over the last several decades have aimed to deepen teacher content knowledge, but until recently there were no instruments of demonstrated validity and reliability that were feasible for use on a large scale; even now such instruments exist for only a few topics. As a result, program evaluations have had to depend on notoriously suspect measures, such as asking teachers if they thought their content knowledge had improved! Programs targeting student knowledge have faced similar problems, as it has proven difficult and costly to develop measures of conceptual understanding; existing instruments are more likely to assess student knowledge of vocabulary or the apocryphal n steps in the ``scientific method,'' rather than the in-depth understanding sought by STEM education programs. Even if appropriate measures were available, program evaluation has to navigate many other difficult challenges as well. Much attention has been paid of late to randomized field trials as the ``gold standard'' for evaluating program effectiveness. There is no question about the value of this approach, but there are many questions about its cost and feasibility. (It is particularly ironic that at a time when school districts are very interested in ``research-based'' programs, they are reluctant to participate in research because of the many pressures they are dealing with.) And as the recent report by the American Competitive Council notes, decisions about education policy and practice shouldn't be based on single studies, however well-designed. To be most helpful, an evaluation of program effectiveness should include multiple studies to answer question not only about whether the program achieves its desired outcomes, but also with whom and under what conditions. Finally, summative evaluations need to determine if programs have had unintended negative consequences. Where should the federal R&D mission agencies focus their STEM education efforts? Based on my understanding of both the expertise of the federal R&D mission agencies, and the complexities of the K-12 education system, I believe these agencies should play a relatively small, supporting role in efforts to improve the K-12 education system, and a more direct and major role in the informal science arena. I suspect that was not the advice I was expected to provide, as I was asked to use what we have learned from research to make recommendations for the development of programs for pre-service and in- service STEM teachers. (Before I explain my reasoning, I would like to point out that the research I and others have conducted on effective professional development has not progressed as far as one would hope, for a myriad of reasons. I already mentioned the lack of valid and reliable measures of teacher learning that are feasible for large-scale administration. In my view, it is both appropriate and essential that the Federal Government support such development efforts, as the private sector has few if any incentives to undertake this difficult and expensive work. But that is probably an appropriate task for the National Science Foundation rather than for the federal R&D mission agencies that are under consideration in this hearing. A second major problem has been the lack of a system to help ensure the steady accumulation of knowledge in key areas such as professional development for STEM teachers, again a challenge more for NSF than for the federal R&D mission agencies. Much of what we ``know'' about effective professional development is based on the insights of expert practitioners, rather than on clear empirical evidence. Richard Elmore has characterized the emerging consensus not as a substitute for research, but as a set of sensible propositions that can be used to guide practice and as hypotheses to be tested.) Although the research is far from definitive, the emerging consensus in the field is that professional development is most effective in changing classroom practice when it is closely tied to classroom instruction. Although there is no question that teacher content knowledge is necessary, it is becoming increasingly clear that teacher knowledge of content is not sufficient. Teachers also need to learn how their instructional materials can be used to help students learn science concepts; how to figure out what their students understand and where they are struggling; and how to appropriate instructional decisions based on that information. Teachers also need opportunities to apply what they are learning in their own classrooms; to share their struggles and triumphs with other teachers; and to get feedback they can use in improving their instruction. To be effective, it appears, professional development programs need to be intensive, extensive, and sustained over time. The federal R&D mission agencies clearly have the necessary content expertise, but they have only limited expertise in improving classroom practice. Thus they do not appear to be well-positioned to make a substantial contribution to teacher professional development of the nature and scale needed to increase overall science literacy. A number of the federal R&D agencies have offered professional programs for many years, but those programs typically reach only small numbers of teachers, in many cases ``volunteers'' who tend to be already relatively strong in content knowledge. To be effective in providing professional development, the mission agencies would need to create mechanisms to be able to stay current about what is being learned about effective professional development, and apply that knowledge to their professional development programs. And they would have to develop and maintain on-going relationships with a sufficient number of districts to make much of a difference. In my view, rather than having the federal R&D mission agencies develop and implement their own professional development programs, it would make sense to have agency scientists available to serve as content resources for local professional development. It would also be helpful if agency scientists were available to assist organizations engaged in the development of professional development materials for more widespread use. Similarly, I would not recommend that the mission agencies continue to develop instructional modules for classroom use. That is not to say that the materials the federal R&D agencies have developed are of poor quality, but rather that the K-12 education system lacks incentives for teachers to find those materials, or once found, to use them in their classrooms. Many science teachers are already hard-pressed to address the content included in state standards in anywhere close to the depth needed to develop student understanding, so adding in supplemental activities may be a difficult sell. In fact, having the mission agencies provide activities for classroom use can actually have a negative effect, adding to the incoherence in the system as different teachers make different decisions about what to leave out in order to make room for these activities. The teacher of the next course may well have some students who have engaged with the topic as addressed in the ``regular'' materials, some with the supplemental activity, others with both, and still others with neither. In that situation, teachers can't win no matter what they decide to do. Just as serving as content resources for others engaged in professional development would be helpful, in my opinion the federal R&D mission agencies can contribute to the improvement of the K-12 education system by making relevant data accessible to people who develop curricula, assisting them in understanding their potential not only for engaging students but also for helping them learn important content as outlined in national and State standards. In contrast to the cautious approach I recommend for involvement in the formal K-12 education system, I believe the federal R&D mission agencies are well positioned to make major contributions in the informal science arena, e.g., through the development of interactive exhibits for science centers on phenomena of interest to students, parents, and the general public; speakers' bureaus; activities for after-school programs; newspaper inserts; television programs, etc. Informal science education vehicles can also be used by the mission agencies to help ensure an adequate science pipeline, for example disseminating information about science career opportunities requiring different levels of education. The federal R&D mission agencies can sponsor programs for interested students to interact with scientists, with special efforts to encourage participation of students from under- represented groups. Other efforts could target parents, to help ensure that their children keep their options open by enrolling in elective mathematics and science courses. While coordination of efforts among agencies to avoid unnecessary duplication of either infrastructure or resources is appropriate, lack of coherence is not an issue as it is in the formal K-12 education system. Different people will access different resources in different ways and at different times; having multiple pathways increases the likelihood that people will benefit from the available resources. REFERENCES: American Association for the Advancement of Science/Project 2061. (1993). Benchmarks for science literacy. New York: Oxford University Press. Banilower, E.R., Heck, D.J., & Weiss, I.R. (2007). Can professional development make the vision of the standards a reality? The impact of NSF's local systemic change through teacher enhancement initiative. Journal of Research in Science Teaching, 44(3), 375-395. Elmore, R.F. (2002). Bridging the gap between standards and achievement: The imperative for professional development in education. Washington, DC: Albert Shanker Institute. Garet, M.S., Porter, A.C., Desimone, L., Birman, B.F., & Yoon, K.S. (2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38(4), 915-945. National Research Council. (1996). National science education standards. Washington, DC: National Academy Press. Biography for Iris R. Weiss Iris R. Weiss is President of Horizon Research, Inc. (HRI), a contract research firm in Chapel Hill, NC specializing in mathematics and science education research and evaluation. She has had extensive experience in survey design and analysis and in mathematics and science education development, evaluation, and policy research. Dr. Weiss received a Bachelor's Degree in Biology from Cornell University, a Master's Degree in Science Education from Harvard University, and a Ph.D. in Curriculum and Instruction from the University of North Carolina at Chapel Hill. Before establishing HRI in 1987, Dr. Weiss was Senior Educational Research Scientist at the Research Triangle Institute, where she directed numerous education research, development, and evaluation projects. Prior to that, she taught science at the high school level in Ithaca, NY, and Chapel Hill, NC. Dr. Weiss has directed many of HRI's research, development, and evaluation projects and is responsible for quality control of all operations. She participated in the evaluation of NSF's model middle school mathematics and science teacher preparation and Triad curriculum programs, served on the assessment working group for the National Standards of Science Education, and chaired the National Research Council's Committee on Understanding the Influence of National Standards. Dr. Weiss has served on numerous advisory boards, and has provided consultation to the National Science Foundation, the US Department of Education, the American Association for the Advancement of Science, the National Science Teachers Association, the National Council of Teachers of Mathematics, the Congressional Office of Technology Assessment, the Council of Chief State School Officers, and the National Assessment of Educational Progress, and several private foundations. In addition to directing a series of national surveys of mathematics and science education, Dr. Weiss coordinated the Inside the Classroom national observation study. She served as Principal Investigator for several studies of systemic reform, including the cross-site evaluation of the Local Systemic Change professional development program, and co-authored the Handbook for Strategic Leadership to help mathematics and science educators apply the lessons learned from those initiatives to their practice. Dr. Weiss is currently Principal Investigator of a Knowledge Management and Dissemination project for NSF's Math Science Partnership program and co-PI of the Center for the Study of Mathematics Curriculum. Discussion Chairman Baird. Thank you, Dr. Weiss. Fascinating series of perspectives from all of you, and I am grateful for your insights. I will begin the questioning and then in five minutes or so we will yield to Dr. Ehlers. One of the terms that came up repeatedly in various testimony was outcomes and the importance of looking at outcomes. What would each of you--and I will let each of you take a shot at this--what would be the most important outcome that you think could derive from participation by the mission agencies in the education endeavor? If you had to measure it-- and I recognize frankly some of it is rather ephemeral and may be difficult to quantify and I respect that. Set aside the issue--don't define the goal as something measurable, define something desirable first and we will worry about measure in a second. What would you think are the most important, left to right, Ms. Froschauer? Ms. Froschauer. One of the things I believe that would help classroom teachers the most, and that is my perspective, is research that would provide us with information concerning how best to teach concepts to students so that they truly can conceptually develop the ideas. The research base in many areas is lacking, and expanding that research base would be very valuable. And most of that can be done quite well with some longitudinal studies and look at how students actually learn over a long period of time and add to their conceptual understanding. Chairman Baird. Mr. Lach. Mr. Lach. I think what would help most is having the federal R&D mission agencies measure the way that they connect students and parents and teachers to the practice of science through their laboratories and their facilities and the way that they inspire that same population to get excited about the world of science and its practice, focus on the informal side. Chairman Baird. Dr. Nelson. Dr. Nelson. I would like to see an outcome that would allow the mission agencies to be full partners with the schools and the community colleges and the programs that help prepare not necessarily the very high-end, top 10 percent students who are going to be scientists and engineers, but help the forgotten majority of the students below that who are very necessary. We need three or four or five good technicians for every engineer we have got in the field. Those students need to be both excited to participate in a career like that, prepared through a good K-10 kind of preparation in the schools, but where the agencies can help is to take the next step through their last couple of years of high school into community college to prepare them to work on the cutting edge in the mission agencies as high-paid, well-prepared technicians and then support personnel. Chairman Baird. Mr. Reiner. Mr. Reiner. In my, or our arena, if you will I think the mission agencies have a role to play in terms of exciting not only the students but their parents and the general public because I think that we need to improve science literacy, we need to have people understand that science is all around them. I have a couple anecdotes that I think apply to that. Dr. Tom Jones, a former Shuttle astronaut, has told me that he first became interested in astronomy when he looked through the telescope at the Maryland Science Center as a young child. Also, the current doctor in charge of the Baltimore Public Health System remembers going to a science summer camp at the Maryland Science Center where he dissected a cow's eye, and that gave him his first example of medicine. So we have that opportunity, I think the federal R&D mission agencies can help us in terms of getting people, the public, students, to relate to the field. Chairman Baird. I notice in those comments the absolute centrality of hands-on experience which I know your facility is--just really the hallmark of your facility and in both those anecdotes, it was a hands-on experience that excited someone. Mr. Reiner. There is nothing like the face of a student who, in interacting with an exhibit, suddenly gets it. Chairman Baird. Dr. Weiss. Dr. Weiss. One outcome would be student interest in science, the wow factor that you referenced. A second would be general science literacy. We have measured that over time, and it has been pretty disheartening what the results have been. And if the agencies choose to work in the formal system, then I would say the outcome would be improvement at scale, for example, having science textbooks used by millions of kids have relevant applications as opposed to just a small number of people benefiting. Chairman Baird. The scale issue also is something many of you mentioned, and I think that is a central question: How do we scale this up? It is terrific if 100 teachers or 50 teachers or even only 20 can go to a summer workshop, but how do we scale it to reach the forgotten majority that Dr. Nelson alluded to? Maybe we will be able to get to that question in a moment. I would just conclude that we had a workshop out in my district with NSF and a bunch of teachers, and someone asked at one point what do you think the goal should be; and for me, the essence is, as a science teacher myself actually before this job, is wonder and discipline. You said the wow factor. I want somehow, and I think what the agencies can do is the wonder part. There is nothing like talking to a Space Shuttle astronaut or somebody under the ocean or dissecting a cow's eye. That wonder part is critical but the discipline part that helps them understand it takes some rigor to answer these questions. And with that, let me yield appropriately to a gentleman who knows first hand about that, the Ranking Member. I would also note the presence of Eddie Bernie Johnson, former Ranking Member of this subcommittee and a valuable asset to it. Thank you. Dr. Ehlers? Mr. Ehlers. Thank you very much. I really appreciate the testimony. I am a hands-on person as an experimental physicist, and I am a great believer in hands-on education as well. I am just curious, of the--you know, we are talking here about agency programs, agency ideas. How much of them are directed to the high-school student and how much to the elementary school student? Let me just get an idea from each of you what you think it is, let me say the ratio of high-school to elementary school? Ms. Froschauer? Ms. Froschauer. I don't know if I can give you an exact figure on that, however---- Mr. Ehlers. I didn't expect exact figures. Ms. Froschauer.--there is a great deal more emphasis on high school than there is on elementary school which we know needs to--we need to have a shift in that. There needs to be more emphasis on elementary school because as so many people have expressed, when you are beginning in their education, we want to excite students so they actually consider taking more science as they go through the K-12 system. Mr. Ehlers. Michael? Mr. Lach. [no response]. Mr. Ehlers. Okay. Dr. Nelson. Dr. Nelson. I think there has been--it moves around. It is a moving target. I think recently people have been focused on middle school, you know, which has been kind of a great wasteland of education in terms of what the focus and the coherence of the programs there, so it has been getting a lot of attention. My impression is it is kind of spread evenly across the board. Mr. Ehlers. Mr. Reiner. Mr. Reiner. I think the mission agencies would like to focus on high school. In our case we force them to tailor their content or we tailor the content to upper elementary and middle school. Mr. Ehlers. Okay. Dr. Weiss. Dr. Weiss. I haven't a clue. Mr. Ehlers. Okay. That is a perfectly valid answer. I don't either, otherwise I wouldn't have asked the question. Years ago I proposed, and unfortunately that part of the bill got removed as it went through the process. This is when we still had Eisenhower funding and we had a clearinghouse in Columbus. I proposed that that clearinghouse be charged to have a listing, an Amazon.com type of listing of all of the different units available from all the different public agencies, from the corporations, the chemical society, et cetera, et cetera. And by Amazon.com style I mean teachers who used a particular unit would send back their evaluations, you know, one star up to five stars and tell other teachers how they have used it so a high school chemistry teacher wanting to teach something about the gas laws would just go to that website, punch in gas laws. There might be 20 units that would fit, should read the evaluations, download the best one for her class the next day. Unfortunately, as I say, that got lost; but I still think it would be very useful. The other thing it seems to me would be useful in terms of the government agencies is some sort of STEM czar, and I don't mean that literally but something that coordinates all the different programs because we have an incredible hodgepodge out there; and how is a teacher to sort them out? How do they relate to each other? How can you effectually use them in the curriculum? It seems to me that having the soliciting I talked about plus some coherence to the Federal Government's efforts might be very beneficial for all teachers. Finally, let me just make a pitch. In picking up on what Mr. Reiner said about getting kids excited. I really think you have to start in the elementary school very strongly. If we are going to get the type of technicians we need, and I always say the jobs of the future require an understanding of the basic principles of math and science. I mean, I think that is pretty self-evident. So how do we convey that to the kids? I think you have to start in preschool already, emphasizing these ideas. And I am pleased I just managed to get attached to the Head Start bill last week. The Head Start programs also have to deal with what we call math readiness and science readiness, just learning simple skills of classification and enumeration, things of that sort. If we don't get them started early, they are not going to do it in middle school. If they don't do it there, they don't do it in high school, they get to the university and it is too late. They have to spend six years if they want to become an engineer. So I think it is crucial that these programs that we, collective we, develop for use in school be able to span the spectrum and really develop the interest. With that I will yield back. Chairman Baird. I will yield to Dr. McNerney in one moment but I would like to give the panelists the chance to respond to the particular issue Dr. Ehlers raised which seems intriguing to me. Is there such a clearinghouse as he has described and if not, would it be useful to you and in what way would it be most useful? Ms. Froschauer. There isn't a single clearinghouse for all materials that I have ever heard of. There have been some attempts to have clearinghouses for materials, and currently there is nothing; and it is beneficial to have a clearinghouse. Something else that is interesting is that currently when programs come out of an agency such as NSF, there can't be any kind of a rating system coming from NSF. Actually, they can't put their good NSF stamp of approval on it and say this is what everyone should be using. And so it even makes it more of a dilemma as to what really quality programs are out there for teachers to pull from. Chairman Baird. Mr. Lach. Mr. Lach. There is not a clearinghouse that I know of other than a web search which I know many of our teachers use. I would sort of add to the discussion, one of the sensibilities we have learned in Chicago is that just using curriculum by itself is necessary but not sufficient to get where we need to go. I spend an awful lot of time and energy connecting well- designed curriculum to workshops, to in-school coaching, to assessments, to leadership development work, to grade-to-grade, school-to-school sequencing; and I think that part is really, really important and I would want to make sure any such clearinghouse really highlighted the connections between all the things that a teacher has to worry about. If we make it just a place to download a PDF of a gas law experiment, it is not going to be--it is necessary but it is not sufficient to get the kind of change we need. Chairman Baird. Other comments on that? Mr. Reiner. On the informal science education, the members of the Association of Science Technology Centers do share successful programs; and the website of ASTC is a place that you can go. Additionally, when we receive an NSF or NIH grant, we are required to share any findings we get with other member institutions, but I know of no other clearinghouse. Dr. Weiss. I would like to comment on that as well. Some teachers can pull together excellent materials and organize them into a coherent curriculum, but most teachers have neither the time nor the capacity to do that; and in our research, when teachers have been faced with more than they can cover in the time available, they make choices. But the choices tend to be based on what they believe is engaging to their students, and a lot of the prerequisites get lost, the coherent whole of children getting an opportunity to learn important science goes away. So the kind of clearinghouse approach I would recommend would be to make these ``wow'' factor types of things available to curriculum developers so they can get into the system at scale as opposed to through the work of individual teachers. Chairman Baird. Dr. McNerney. Mr. McNerney. Thank you, Mr. Chairman. One of the big challenges we face is that STEM education is hard work. It is not easy to get a degree in math or physics or engineering. It takes a lot of hard work. And so part of our job is finding out what it is going to take to inspire this coming generation to do that hard work, to get involved and instead of going to the frat party that they want to go to, to actually do the work and this starts when they are young. This sort of drive to achieve something in science or engineering starts when they are young. Dr. Van Reiner, you had mentioned that you have scientists come in to your science center. I am wondering, what is the most effective thing in your observation to get kids wowed, to get kids excited and inspired about science? Is it direct interaction with scientists or is it hands-on? What works the best? Mr. Reiner. At the early ages, it is definitely the hands on. As they get into upper elementary or middle school, it is a combination of the hands on and the direct interaction. I can remember we had a USGS geologist, a young woman who was explaining what she did for her job; and this young seventh- grader said to her, you really like what you are doing, don't you? And she said yes. And they pay you for that? So I think that is--I have got a thousand anecdotes. But I do believe that it is important for the scientist to have a face-to-face time with the upcoming generation if you will in order to be able to practice explaining things in everyday language. Dr. Nelson. I would like to comment on that, too. I think one of the issues of the pipeline that we don't talk about very often that is really important is we need to focus on getting kids in the front end of the pipeline. That is certainly true. But one of the things I have found in working with lots of students and trying to convince them that they might be interested in being scientists or engineers or going to work and even being science teachers is this notion that the pull on the other end of the pipeline isn't that strong. A lot of times people say, well, why should I be an engineer? It is not such a great job anymore. How attractive is it to get my Ph.D. in biology when I could get my business degree and become a post doc for the next eight years, and when I am 40 I might get an assistant professorship job? So I think we need to work on both ends of the pipeline to make the jobs for students very appealing, and the agencies can certainly work on that at the front end so that students see this as a possibility. Think back in the '60s when you were going through school and the community, the government, and everybody else was paying people to go to graduate school. The universities were booming, everybody who graduated could get a job right away. And now we have this huge pool of post docs and others out there. So the system is different today, and I think that does have an impact. Mr. McNerney. I will yield back. Chairman Baird. Ms. Johnson. Ms. Johnson. No questions. Chairman Baird. Okay. We will go to a second round. This is very interesting, and I appreciate it greatly. Let us move to the issue of scale a little bit because that is something that many of you mentioned and a common thread and also, Dr. Nelson, you talked about the challenge of--I want to sort of put out two issues, one is the issue of scale and the second one is the issue that Dr. Nelson talked about, about NASA proliferating educational material, posters, pamphlets, et cetera, but lacking some of the direction, the kind of things that Dr. Weiss maybe mentioned. So I am going to throw out those two topics and open it up to any of you about either of those topics that you want to address, either how we scale things up or how we watch out for this proliferation of materials that may be well-intentioned but not well-targeted. Mr. Lach, you have got direct experience with some of this, a big-scale system. Mr. Lach. I know a thing or two about scale. And I think that is really the key point. A lot of what we have done in Chicago has been based on Iris' work among others, and I think what we found out is we may not have all the answers but we know an awful lot more about how to leverage pretty dramatic change in a large system. It involves coherence, it involves an intense focus on capacity building, it means connecting instructional materials to assessments, to coaching, to support, it means focusing on leadership, and it means pulling everybody in the community together, all the museums, the universities, the labs, you know, to work on this together. It takes a long time and it takes an awful lot of work. Chairman Baird. How did you find time for the people and participating to do this? How did you get their buy-in in that system? Mr. Lach. Our work in Chicago began through a series of NSF systemic initiative grants. It took us several years to sort of figure out how to use them and how to use them well. And we began with a--we knew that need. That need was very clear. We began with a vision that said this is how we are going to move things ahead. We had 87 different math curriculum in Chicago at the K-8 level when we started. Now we are down to two at the K- 5 level. In a local-controlled district that takes an awful lot of convincing and cajoling to do. But other groups would sort of come along once they would see that sort of coherence. And it means, you know, tending to your stakeholders. I spent a lot of time and energy making sure our friends at universities, our friends at labs and museums understand the role that they are to play; and we do a lot of back and forth to make sure that that makes sense. Chairman Baird. What is your portfolio? Did someone crown you as the science education czar and give you some authority or is it just your persuasive personality that gets you through the day? Mr. Lach. I was a classroom teacher and had enough of a loud mouth that was sent to Washington to be a fellow, and then I learned an awful lot from Congressman Ehlers who sort of gave me a perspective about scale and about policy that I just didn't have at the classroom level. And then when I came back to Chicago, it took a little bit of time but they had me in charge of science and then science and mathematics. Chairman Baird. And you are given that respect. People acknowledge that is your role and they look to you for help and you get to give some guidance and governance I guess? Mr. Lach. Yes. I pause because I think--and this is a little bit of an aside. I think one of the things we are learning is that instruction and leadership in mathematics and science is really, really important; and it doesn't exist very much in the educational system. There is lots of pretty compelling research now that shows principals and school leaders, when they lead, the practice of leadership differs around leading and around mathematics and around science; and that is not something most of the people--most local and state education agencies understand and I don't think the system quite addresses yet. And it is a really important factor if we are going to leverage the kind of changes we need. Chairman Baird. Well, we appreciate the work you do, and I am also very grateful to hear that an NSF grant was used so well. You may be interested to know in our NSF reauthorization bill which Dr. Ehlers and I wrote together we have actually lengthened the time period for some of these demonstration projects that had been three years and you are out, basically. So that right when you get things where you have tweaked it enough to think it is finally working, you get the first class of students through, then it runs out. We are actually going to make that longer. It is also nice to know that somebody who has worked for Dr. Ehlers has gone on to do some great things. Not a surprise at all. You learned from the feet of the master. Dr. Ehlers is recognized. Mr. Ehlers. I am not sure how I could follow that up. Let me just say that when Michael spent a year in my office, he did a better job of learning how Congress operates than any fellow I ever had. He had an instinctive approach. And I think what really happened when he got back to Chicago is he realized that if anyone can understand how Congress works they can figure out how the educational system works. I think it wasn't a factor also that you arrived there shortly after the Chicago Public Schools were so-called privatized and that resulted in much greater centralization of authority and power, and that is something you could leverage. Is that correct? Mr. Lach. Yes. Mr. Ehlers. So it was a combination. But you have done a beautiful job there. Let me also answer one other question that I heard raised at some point and I don't recall where or how but that was about how one propagates this. And the best example of that I have seen is the American Meteorological Society offers a summer program for teachers at I think various grade levels. But one requirement of signing up for that course, and it is a very good course, it is about a month or more and the teachers are paid for their expenses, et cetera--I think there is a stipend--but one requirement is that every teacher, when they go back to their school system, have to set up workshops to teach 10 other teachers the same material. And then those teachers have to make a commitment to propagate it to their own school buildings. And so in a short time, the AMS curriculum went from being just taught to a workshop and ended up with 100,000 teachers using it. And you know, if somebody looked at that, perhaps it is easier than most to develop a concise unit. Kids can study clouds and weather and so forth so you know, there is no expense involved with equipment. But nevertheless, I thought it was a brilliant idea and something that we might pursue as well. I don't have any other questions on my mind at the moment. I just very much appreciate the breadth of experience represented here and the comments that you have made. It has given me a lot of insight of what we should be trying to do legislatively as well and to take into account the concerns you have raised. But I don't see any way we are going to break it without, as Michael has indicated--I believe our emphasis has to be professional development because before I came here I worked with a lot of schools and a lot of teachers; and I found the teachers wanting to teach science well, wanting to teach math well, but many were scared because they didn't know the subject material. Many others didn't really--they weren't scared but they didn't know how to tackle it and do it right. And professional development is the only way you are going to deal with that, and that is why the Eisenhower program was a good thing, even though it didn't always do it well. But that is something we lost in No Child Left Behind only because the mechanisms are there but the money was never provided. And so we actually lost something going from Eisenhower to No Child Left Behind. With that I will yield back. Chairman Baird. Did any others want to comment on the issue of scale or the other couple of questions? Dr. Nelson. I would like to make a quick comment on that, and it relates to professional development, too. One of the reasons I went back to the university that I went to, Western Washington University, is that it is a former normal school. It prepares roughly 500 teachers a year, and with the bulge of baby boomers moving through the system now, we have a real opportunity to provide the system with new teachers, with young teachers, with new ideas. And one of my stated goals at the university is that I have been at this for five years now. I am giving myself another five years to say in that time I hope to be graduating teachers from this institution who don't need remedial professional development. We can't always continue at the same model of assuming that the teachers in the schools are not at the level we want them to be. And so we are working very hard to try and graduate new teachers who know how to choose and use the best curriculum that is out there, who know what good instruction looks like and can have a beginning at least level of practice for that but also really importantly know how to collaborate with their peers and to partner with us at the university and others to improve their instruction focused around the performance of their students. So now I am starting to focus on not just teachers, but the administrators are such a huge role. We are finding in our NSF grant that probably the biggest factor on whether a school is making real progress in improving their science education programs is the principal who can either allow it or not allow those programs to happen and principal preparation programs now. Again, we are having a big turnover of principals. We have an opportunity to prepare administrators who understand what good instruction is, who can support professional learning communities of teachers. So hopefully in the future we can--our professional development will look differently. We will be able to ratchet it up to a different level. Chairman Baird. As the son of a principal who talked a great deal about the challenge of aspiring to be the academic leader of the institution but being often burdened with the budgeter, disciplinarian, police, liaison, et cetera, I think you are right. The academic leadership provided by the principal is absolutely critical, and I admire the notion that we are going to graduate people who actually know what they are doing when they graduate. It really is well put. Dr. Nelson. Yes, it is not so much the teachers in the field now don't know what they are doing. It is a different world. We have learned a lot in the last 20 years, and we are hopefully going to get that into the system. Chairman Baird. Dr. Weiss, you had some comments? Dr. Weiss. Yes, I want to build on what Pinky just said. It was a while back, a number of years ago, said that we are putting teachers out in immediate need of a 50,000-mile tune-up and that unless we improve pre-service education, we will always be at the point of remediation rather than continuing education. And we need teachers--like all professionals, we need continuing education. One of the root causes of the mess we are in I believe is that teachers are asked to try and address far too much content. As a result, our preparation can't be as focused as it needs to be, our professional development can't be as focused as it needs to be. It is scattered resources. A second comment I wanted to make, for reasons I have never understood, we are doing better in developing systems to support teachers and principals in mathematics education than we are in science education. The notion of professional development materials, models and materials that have been carefully crafted, evaluated, and improved, scaffolds the efforts of lots of people and lowers the capacity that is needed to do these well in the field. Building on what Michael said, we need efforts, direct efforts, and I don't believe this is a federal R&D mission agency responsibility, but direct efforts to build the capacity so that we people our school systems with people who are ready to take advantage of the knowledge and tools that are out there. I could go on longer but I will stop. Chairman Baird. Very well put. I am going to yield some time for Dr. Ehlers. He has a follow-up question. Mr. Ehlers. Thank you for yielding. I totally agree with you on that issue, and in terms of why science takes second seat to math I think is pretty evident. Everyone thinks that reading and math is something that everyone should understand. I find a lot of people who still think that science is only for someone who is going to be a scientist or an engineer. And even if they are teachers they don't regard it as highly important. That is changing. But the way I got into science education was just when I was a young professor and I was very concerned about scientific illiteracy and I asked myself, what can I do as one person? And I decided to set up a special course to teach future teachers both science and how to teach science. And I thought that was my role in life until it inadvertently brought me here. It is absolutely crucial. And I continue to speak constantly to university presidents, deans, about the importance of this and above all something I learned the hard way, that you have to get the schools of education to work with the academic departments. Right now almost every campus I visit, it is not that they dislike each other, they disdain each other. And the academic folks think the people in the Department of Education are all aflutter about education, psychology, and theories of education and they don't know how to teach themselves. And people in the education department think about scientists as up in the lofty skies. They don't know--have the slightest ideas about what it is like to teach in elementary school. They just don't talk to each other. I found I had to teach myself the lingo of the educators, studied educational psychology on my own so that I could communicate with them; and once that happened, we had a very good relationship and actually worked together. But it is tough. The easiest thing to do, easier than professional development, is to train the teachers right in the first place. It is going to take a lot of work on a national scale to make that happen. It is happening, you are doing it, Dr. Nelson, Arizona State University has done it quite well and Kansas started some good programs, Western Michigan University has. So it is coming. It is spreading across the country, but it is still not highly regarded in the academic world and it should be. Thank you. Chairman Baird. Thank you for those insights, Dr. Ehlers. Mr. Carnahan, five minutes. Mr. Carnahan. Thank you, Mr. Chairman, and welcome to the panel. I apologize for getting here late but I did want to jump into these questions here. In particular I want to talk about any successes that you have seen or ideas you can share with us about how we can do a better job with partnerships between the front-line teachers and the private sector and other science resources. I mean, my hometown is St. Louis as we have a wealth of higher institution entities there, higher education facilities there, private-sector entitles there like Monsanto, Boeing, non-profits like Missouri Botanical Gardens and the Dane Forest Plant Science Center. So we have got this wealth of science and engineering there in the community, yet we still seem to have the difficulty getting some of that translated into the classroom. And so I guess, kind of a two-part question, how can we do a better job partnering with those kind of resources to supplement what we do in the classroom, and the second part of that is with regard to streamlining our teacher certification process for some of those retirees that maybe have had full careers in one of those institutions, how they could become qualified to teach and be a part of really beefing up what we do on the front line of our schools. And I would start with Ms. Froschauer. Ms. Froschauer. Thank you, Congressman Carnahan. It is good to see you again. Mr. Carnahan. Good to see you. Ms. Froschauer. As you know, we recently were in your town---- Mr. Carnahan. Yes, indeed. Ms. Froschauer.--for our major and national conference. And we had the ability to enjoy all of those wonderful science resources you just mentioned. We had about 10,000 science teachers there a couple months ago, so it is good to see you. I do want to say something about the cooperation and working with agencies. I know that Dr. Nelson had mentioned things like posters and things of that sort that have come out of NASA and other agencies are not highly utilized by teachers. But there are many partnerships and many collaboratives that have brought about materials and opportunities that have been very beneficial to science teachers. But they must be done carefully, they must be well-thought-out. They must be things that are actually going to be utilized by the teachers or beneficial to the teachers and translate into student learning. NSTA partnered with NASA, and in that partnership we considered what science teachers need. Now, many people have already alluded to the fact that science teachers are sometimes are uncomfortable with the content, especially when you get into elementary grades, middle schools many times, and even a high- school teacher who may be teaching outside of their discipline. We know that that content is necessary. We also know that NASA has the capacity of providing scientists and the people who can actually contribute the content knowledge. And so we utilized that content knowledge, that expertise with NASA, pulled it together into something that would provide teachers with contact information. We also know that teachers are busy, and so they want to be able to gain that content information kind of in an on-demand sort of basis and so we have provided that actually on the web so that they can self-instruct, they can also self-evaluate and actually can develop that content knowledge, and we do that through something called science objects. And they are available free. Mr. Carnahan. Excuse me, is that a specific website, Science Objects? Ms. Froschauer. You can go into the National Science Teachers Association website. You will find science objects, and that is actually at the front of the wall so that everyone can have access to them. And we also partnered with NOAA and FDA on pulling together those sorts of things. And then during the St. Louis conference for instance we had many sessions that were delivered by NASA scientists, by NOAA scientists, and also they had space on our exhibit floor where they could disseminate materials and share information with teachers. So there are some very strong partnerships when you consider what the needs of the teachers are and what the resources are of the agency and how you could pull those together to benefit teachers. Mr. Carnahan. And so you think we can do the same type of things with local resources as well? Ms. Froschauer. You certainly can. And I think you probably already realize that there's a great deal of effort right now at the state level to have coordinating bodies established similar to what is being recommended through the NSB Council and also through the current ACC recommendations, that there would be a coordinating body that would look at all of the efforts that are taking place at a state level and that came out of the governor's report. Mr. Carnahan. I guess the second part of my question with regard to potentially being able to certify some of those retired experts, engineers, scientists that are in communities like that. Has there been any work going on that has helped to facilitate that that you're aware of? Ms. Froschauer. Well, every state seems to have their way of providing for that kind of certification. There are many opportunities for people to change a career path and go into science education, and it doesn't always demand that they go back to college for four years to get an education degree. It is actually an alternate path to science teaching. And every state seems to have a different way of going about that. That is something else that could really use some coordination because obviously in some situations it is more successful than in other situations. Mr. Carnahan. I guess I will ask this of anyone on the panel. Are you aware of any--certainly the state certification process varies widely, but are there any sort of best practices out there that, you know, a certain state is doing that we might try to copy or other states could look to as a leader in that area? Mr. Lach. One of the problems we have in Chicago is the way we certify teachers is complicated, it is bureaucratic, and it is not the same as our neighboring states. So I think there is possibility for ways to streamline that process and perhaps find some commonalities about what a high-school biology teacher or middle-grade science teacher ought to know and be able to move things ahead. Let me speak a moment about your previous question which I think is a fascinating one, sort of how would you think this through in Missouri or in St. Louis. I think it begins by having a clear plan, probably at the state or the LEA level. That takes a fair amount of work to put that together and a clear theory of action so you would understand why the different aspects of the plan might result in a boost in student achievement. If you have a plan you can then position the various partners to take on different roles of that. In Chicago, for instance, a big part of our plan is a core curriculum adoption and implementation. UFC does one of our math programs, UIC does another, Loyola does our science programs. They have become implementation centers within the district to move that along. The second part of our plan is to increase the content knowledge of our teachers by taking university courses. We have 10 local universities that we provide tuition stipends for so teachers can go back to school and earn the state endorsement in mathematics or science. In our plan we also have a place for this inspirational, this wow factor. We use the tremendous museums that are in Chicago, the Museum of Science and Industry, the Field Museum, Adler Planetarium, others. We the Argonne and Fermilab. We use a lot of the community resources to do that inspiration sort of work. It has only been in the past year or two when we have had this plan and--we have kept with this plan for four years but it has only been in the past year or two where I have had enough political clout to be able to tell someone, you know, those posters that you are providing and those one-shot lesson plans, I don't think we really need that. We have got a plan that is working, we are sticking to that. But it is very, very difficult to do. We don't have a lot of resources in Chicago. It is pretty unconscionable to turn them away. Having a plan, having lots of partners invested in that plan enables us to bring that coherence and that support. And the last part of that that I also think that you might be able to help with is we need tools to sustain those sorts of plans and that kind of work. That means high level of capacity in districts and in states for leadership in math and science education and the appropriate amount of political cover so those people can make the kind of decisions that are going to help kids in the long run. It is very, very difficult work. It is really difficult to scale, and we need lots of resources and help from you all if we are going to give our kids what they really deserve. Chairman Baird. Thank you. I am going to ask Dr. Weiss a somewhat different question but appreciate the tremendous insights, and then Dr. Ehlers, and then we will open it up for final remarks if anybody has any burning issues. So you will get each--if you have got something you just haven't had a chance to say, we will get to that, too. Just one fairly brief question. Dr. Weiss, questions can be brief, the answers often are much longer but the ACC report talked about evaluation and the need for a control group design, outcome studies. I wonder if you have any--personally I have some mixed feelings about that with the internal and external validity issues but I would welcome your insights as an expert in this field. Dr. Weiss. There is no question that we need to be doing a more rigorous job of evaluating programs. I think that the report ignores or underplays some really key issues, one I mentioned earlier in terms of having measures of outcomes of interest. Another would be, when people talk about randomized control trials, they tend, although the report did talk about this, they tend to not realize that we need multiple such studies. A program that works in a rich, suburban district, you can't just now say it is going to work everywhere else, and that has been a problem. But the other is the realities of school districts. If we were going to try and evaluate let us say a good set of instructional materials, you would want to have some teachers using it and some other teachers not and you would want to do that for a long enough period of time so you could look at the differences. Teachers need opportunities to learn how to use the materials, practice, get feedback. School districts are not willing to do that. They can't have two simultaneous programs going on and create whole systems around it, and in these days with the pressures, all of the pressures on school districts, we are in this not-in-my-backyard. Everybody wants research- based programs but nobody wants to participate in the research that will generate that knowledge. So I think there is a whole host of issues, on theoretical grounds you cannot argue. On practical grounds you can. Chairman Baird. Dr. Ehlers had a question and then we will conclude in a second. Mr. Ehlers. Just something I wanted to get on the record. If there is a lack--first of all, I wanted to ask, if there is a lack of coordination among federal agencies regarding STEM education programs, how would you recommend solving the problem? How should we coordinate it and who should be tasked with the coordination. Any ideas on that? Chairman Baird. I nominate Mr. Lach. Mr. Lach. I was afraid of that. The need is clearly there. I think what we found in Illinois is that different districts have different needs so I suspect there would have to be some sort of statewide localization to address some of those sorts of things. I think there also needs to be--I would encourage that the work of the coordinating--this coordinating body, I am not sure who it should be. I think an important role might be to provide really formative work to districts and states to help them get better. I get a report card every year when I see my test scores, but changing education systems, particular at scale, is a very complicated, murky business; and if I had a well-regarded report that told me based on the inputs I am putting in as well as the outputs, what I need to do a little more of, what I need to do a little less of, that would help me organize resources, redeploy my people in a way that would help sustain that sort of program. In Chicago, we would love that to be really transparent. We don't have anything to hide. We think we are doing pretty well, but we know we have a long way to go. So some sort of public report cards about both the inputs and our outputs which I think are already public I think would help us improve and would help make sure that mathematics and science were really on the agenda of everybody as we move things forward. Mr. Ehlers. Does anyone else want to respond? Dr. Weiss. Dr. Weiss. I think we need research that is focused on a smaller number of problems of practice. Right now the research enterprise, it is pretty wide open and so it doesn't tend--we don't have mechanisms right now for accumulating knowledge on key problems. Mr. Carnahan talked about best practice. We don't really have mechanisms now for knowing. I thought about the question of where are the good, lateral entry programs, and I suspect I know the parameters of what an effective approach would be but I don't have any data and I don't know that anybody has pulled that together. So we need mechanisms for accumulating knowledge. But we also need incentives for using that knowledge. I was struck by the comment in the ACC report that they saw no examples of a federal agency building on the knowledge or models of another federal agency. So I think it is capacity issues but it is also incentives issues. Mr. Ehlers. Who do you think would be the best agency or person to coordinate all that? Dr. Weiss. Every candidate that comes to mind has baggage, so nothing is quite coming to mind right now. Chairman Baird. The distinguished Ranking Member would be an outstanding choice. Dr. Nelson. Can I make one quick comment on that? I remember back it must have been in the late '80s there was a group called the Fix-it Committee, the federal Coordinating Committee on Science and Technology; and it started out with great promise I think because it was supported at a very high level from many of the different agencies in the--federal agencies, and then kind of petered out as the level of participation filtered down deeper into the organizations. And it seems like a committee like that that could stay at a very high level and provide a focus and some level of coordination supported somehow by Congress or someway. That is what you guys are good at, right? And would be able to maintain that focus rather than expecting to--you know, if you could keep a committee like that going for long enough, you might be able to bring some kind of a coherence and focus to the program. Mr. Ehlers. Any other comments? Thank you very much. I yield back. Chairman Baird. We have time in an outstanding group like this. I would open it up if anybody among you has any critical insights or issues we haven't had a chance to address throughout questioning that you feel would be a shortcoming if we didn't raise, let me give you an opportunity now. You don't have to, but if you feel like there is something that really we should put down. Dr. Nelson. I have a brief comment about research that I would like to make. There is no doubt that if you have a controlled laboratory condition or you can do controlled experiments like double-blind drug testing, things that that is a great way to learn things, but as a research astronomer, I can tell you somehow astronomers learn things and we can't control anything. And so it is possible by posing questions well and by doing a carefully designed observation and analysis, it is important to learn things about complex systems. To make a point that we can learn important things about the education system by carefully--by designing and carrying out careful observations and analysis of existing systems, then we can apply so not all research has to be--has to fit into this narrow---- Chairman Baird. I think that is a great example and maybe a way of an appropriate analogy but would we have seen some star programs ourselves today. We didn't necessarily need a double- blind study to observe the effectiveness of those stars. Dr. Nelson. Certainly they are important. Chairman Baird. Let me suggest--we were kicking around as this hearing moved forward, I think there is so much information that has been useful today that what we are going to do, we will have a--we have already scheduled a hearing with the mission agencies themselves for June 6th. We will provide the heads of those agencies with the testimony provided by you folks today because I think it is so outstanding I would hope they would incorporate the insights in their own work on the educational front. We are also exploring the idea of possibly posting this hearing, and Ms. Froschauer, we might like to talk to you about posting this hearing in some way where it would be accessible to your members. They could share some of the models they have heard about how, for example, the science museums are working, how the graduate education programs are working, how Chicago is making their changes, how they might design their research interventions, and then offer comments analogous to the approach Dr. Ehlers was talking about so that we could actually get further impact of your profound insights but also maybe some additional people pitching in and create a little bit of a dialogue, and that way we have this tool. So we will, if we may, talk to you about if there is a forum to do that, a bulletin board kind of model or something like that. Without any final comments on your part, Dr. Ehlers? Mr. Ehlers. Well, put it on YouTube. Chairman Baird. He said we could put it on You Tube and the students--well, we do have this extraordinary tool right now. I was saying to Dr. Ehlers when I was teaching psychology I came up with a pretty neat way to teach about standard deviations, little lab experiments you could do, published it in the Journal of Teaching of Psych if I remember and gosh, that is a rather inefficient way. It was a nifty little thing and it worked I think and hopefully some people adopted it and used it. But with the Internet you can get those things out there so well if there is a coherent way of using it. So we are going to try to model that through this hearing itself. With that, I want to thank our witnesses and our guests today and the Ranking Member and the staff for all their work and look forward to continue dialogue; and I am confident this will actually have some positive impact and appreciate very much your time and expertise, and the hearing stands adjourned. Thank you very much. [Whereupon, at 11:40 a.m., the Subcommittee was adjourned.] Appendix 1: ---------- Answers to Post-Hearing Questions Responses by Linda K. Froschauer, President, National Science Teachers Association Question submitted by Representative Eddie Bernice Johnson Q1. The witnesses provided a range of views on how R&D mission agencies can best contribute to STEM education. To what extent is there a consensus among panelists about what the agencies can do well and what they are less well suited to do? A1. Federal agencies should focus primarily on improving student achievement, teacher quality, and student engagement. Specifically, the agencies are well suited to: Make scientific laboratory experiences and equipment available to teachers and students (Dept. of Energy) Encourage scientists to work with teachers to further their content knowledge (Dept. of Energy) Work with universities, the science community and others on model, research-based programs to increase teacher effectiveness that can be replicated (Math and Science Partnerships) Develop and implement proven, effective, research based instructional materials and methods (NSF Instructional Materials) Recruit and retain teachers with majors in STEM fields (NSF, Noyce) Increase the content knowledge of in service teachers with long- term, quality professional develop (NSF, Math and Science Partnerships) Support activities that encourage under-represented groups to enter and remain in the STEM fields (NSF) Increase the awareness and interest in STEM through informal science activities (NSF) Conduct and disseminate key research in all STEM education fields (NSF) Provide undergraduate grants and loan forgiveness for STEM teachers (Dept. of Education) The agencies are less well suited to develop curriculum or provide professional development. If this is done, it should be accomplished through strong partnerships. Although teachers enjoy receiving posters and activities for use in their classrooms, these materials do not support the development of conceptual understanding through a strong curriculum. In addition, many of the federal agency programs sponsor smaller, issue based contests and training for educators. These programs simply do not reach the number of teachers necessary to have an impact on student achievement. Answers to Post-Hearing Questions Responses by Michael C. Lach, Director of Mathematics and Science, Chicago Public Schools Questions submitted by Representative Vernon J. Ehlers Q1. You mentioned that the Federal Government could provide some critical ``political cover'' to decision-makers at local levels to help advance STEM education. Do you have any ideas on the best mechanisms to provide that cover/leadership? A1. To best answer this question, let me take a step backwards and articulate some of the factors that in my perspective constrain the system for improving K-12 STEM education. <bullet> The general public does not understand science or its practice. Sadly, most members of the education establishment (school boards, district offices, administrators, state boards of education, principals, etc.) share this lack of understanding about STEM issues. <bullet> The political dynamics in most school systems work against long term solutions. Given that most school boards are elected to short terms, most superintendents in urban areas last only a few years, it's difficult to institute broad, 5-10 year plans. <bullet> We know much about large scale change, but have far from a precise understanding about how to move large systems of schools forward. These three factors are at the core of our inability to move solutions forward. I think the potential exists for the Federal Government to help alleviate some of these issues by establishing: Now, imagine if every state superintendent and the board president of the 50 largest school districts received a detailed ``audit'' of their mathematics and science program once a year. They already receive a performance report--in the form of the annual test that the state mandates. Yet test data merely tells states and districts what to improve, and doesn't help them decide how to improve it. The difference is critical. Most districts know the general direction they must travel, but lack a roadmap that shows them how to get there. The audit I'm imagining would be conducted by a well-respected, external organization. It would review the output data, certainly, but also the inputs--the systems and structures the state or district had in place that resulted in the outputs. For instance, there's pretty compelling data that shows a ``managed curriculum'' approach--a standards set of instructional materials for mathematics and science, coupled with workshop professional development and in school coaching support--is necessary for significant improvement. Another example is local school leadership capacity, as principal understanding of mathematics and science improvement strategies and processes also is a major contributor to student achievement. This set of standards or conditions for state/district improvement would be challenging to develop, but I'd posit that there exists enough consensus within the education community that they could be established. In the course of the audit, a team would visit the state/district, spend some time digging into both the achievement data and the programmatic data, and then prepare a public report to be shared. Perhaps the audit or report uses letter grades or a ranking to indicate the status of key aspects of a STEM program, so it is easily understood by the public and news media. Why would this help? Such a report would highlight both the summative and formative aspects of systemic improvement to all interested parties. As the reports were issued each year, trends and tendencies would emerge, enabling state/district staff to make adjustments and new decisions in light of their review. If they are issued with regular frequency from an external organization, much of the internal politics within states and districts would be muted. If a board decided to fire a superintendent and change strategies, they'd do so knowing there will be a report forthcoming that will hold them accountable to established national understandings. If a superintendent decided to put a large portion of next year's budget into a new reading program, the shift in resources would likely reflect negatively on their STEM education report. A board president could use lukewarm ratings on such a report to galvanize support for reform efforts throughout the state or city. Who could do this work? That's a difficult question, and probably the biggest problem with this entire idea. I think a research-focused organization like the National Academies could lead the development of the standards or conditions for large-scale improvement, probably commissioning some additional research in the process. I worry that the National Science Foundation doesn't have enough reach to other federal agencies--NASA, NIH, DOE, etc.--involved in STEM education, and that the Department of Education just doesn't have the respect of the scientific community to pull this off. A new not-for-profit entity, funded by foundations and business, probably wouldn't have the credibility either without really strong connections to the Federal Government, and I can't foresee a business model where this work would pay for itself in the out years. I'm left thinking that the best (though imperfect) answer would be a new agency under the governance of the National Academies--there's enough clout there to handle the academic needs, and enough impartiality to preserve the inevitable political fights that will ensue. But that's a considerably more activist role for them that would entail considerable rethinking of their organization and mission. Questions submitted by Representative Eddie Bernice Johnson Q1. The witnesses provided a range of views on how R&D mission agencies can best contribute to STEM education. To what extent is there a consensus among the panelists about what the agencies can do well and what they are less well suited to do? A1. I believe that the witnesses exhibited a large degree of agreement, particularly about the important role that the federal R&D mission agencies have to play in K-12 STEM education, the fact that much of the current work merely adds confusion and incoherence to an already complicated system, and that partnerships with states and district education agencies are essential if the work is going to significantly advance student learning. Answers to Post-Hearing Questions Responses by George D. Nelson, Director of Science, Technology, and Mathematics Education, Western Washington University Question submitted by Representative Eddie Bernice Johnson Q1. The witnesses provided a range of views on how R&D mission agencies can best Contribute to STEM education. To what extent is there a consensus among the panelists about what the agencies can do well and what they are less well suited to do? A1. I think there is considerable consensus among the panelists. I would also like to raise one point where there is not disagreement among the panelists, but not consensus either. We agreed on the strengths that the R&D mission agencies could bring to STEM education: <bullet> A high quality science and engineering, workforce engaged in cutting edge research and technology development. <bullet> World-class laboratories and facilities. <bullet> The capacity to engage in long-term projects. <bullet> Existing partnerships with university and industry researchers focused on mission-related research. <bullet> Charters to disseminate the results of mission research broadly within the government, to industry and to the general public. We also agreed on general weaknesses in the agencies with respect to STEM education: <bullet> Lack of knowledge of the K-12 education system, now it is structured and regulated. <bullet> Lack of internal expertise in curriculum development, effective instruction, and teacher preparation. <bullet> Lack of expertise in education research and program evaluation. If we accept these ideas, then it makes most sense for the agencies to target their efforts on informing the public and inspiring the next generation of STEM workers through the informal education community-- museums and media. Interactions with the formal K-12 education system should involve close partnerships with STEM education professionals and education researches/evaluators. There is some danger that a group deeply engaged its K-12 STEM reform, like our panel, sees the world predominantly through a single lens. It brings to mind the old saying, when all you have is a hammer, everything looks like a nail. As the only member of the panel with a background as a research scientist as well as K-12 STEM education reform, I feel I should comment on the role of the R&D mission agencies beyond K-12. I believe that the agencies can contribute more significantly to increasing the quantity and quality of the STEM workforce by focusing their efforts primarily on supporting faculty and students to engage in mission-related research. Given limited funding and the wide range of STEM education activities that can contribute to the goals stated in the recent reports Rising Above the Gathering Storm, and the Report of the Academic Competitiveness Council, each agency should focus their resources where they can have the largest impact. In the case of the R&D mission agencies, this is mission-related research as discussed in my formal testimony. Answers to Post-Hearing Questions Responses by Van R. Reiner, President and CEO, Maryland Science Center, Maryland Academy of Sciences Question submitted by Representative Eddie Bernice Johnson Q1. The witnesses provided a range of views on how R&D mission agencies can best contribute to STEM education. To what extent is there a consensus among the panelists about what the agencies can do well and what they are less well suited to do? A1. The R&D mission agencies should stay away from writing curriculum and concentrate on being a content resource. As such, the agencies should partner with informal education institutions to help communicate the science behind the research and mission. Educating the general public will help the formal education institutions by having a knowledgeable public to reinforce the students' experience in the classroom. Answers to Post-Hearing Questions Responses by Iris R. Weiss, President, Horizon Research, Inc. Questions submitted by Chairman Brian Baird Q1. The Academic Competitiveness Council recommended in its May 2007 report that ``Funding for federal STEM education programs designed to improve STEM education outcomes should not increase unless a plan for rigorous, independent evaluation is in place, appropriate to the types of activities funded.'' The report later describes ``rigorous evaluation'' methods in a hierarchy with experimental methods, such as Randomized Controlled Trials (RCTs), as being the best methodology for showing the effectiveness of a program. In cases where an RCT was not feasible, the next best methodology would be to use a Well-Matched Comparison Group Study. Q1a. As an expert in evaluation techniques for STEM education programs, do you feel that these evaluation techniques are the most appropriate for determining the effectiveness of STEM education programs at the federal R&D mission agencies? A1a. As I noted in my testimony to the Subcommittee, there is no question that STEM education programs, including those of the federal R&D mission agencies, should have more rigorous evaluations. Randomized control trials (RCTs) are an excellent evaluation strategy when feasible. However, there are a number of constraints to the feasibility of this approach that were, in my view, not adequately addressed in the May 2007 report of the American Competitiveness Council (ACC). First, random assignment to experimental and control groups, or quasi- experimental designs with well-matched comparison groups for that matter, are of little use unless there are appropriate instruments available to measure outcomes of interest. For example, in Goal 2, a suggested metric for teacher quality is percentage of teachers demonstrating increased competency in a given area. However, there are only a handful of measures of teacher competency in specific science content areas, and even fewer in areas closely aligned with the missions of the federal R&D agencies, so in many cases even the most rigorous experimental designs would not provide useful information about the impact of a program on teacher knowledge. (I strongly recommend that the federal R&D mission agencies identify a set of key goals, and that NSF be asked to coordinate the development of appropriate measures of those goals so the agencies will be in a better position to judge the effectiveness of their programs in the future.) Second, in my view the ACC report underestimates the difficulties and costs of mounting randomized control trials, especially if the intervention is provided over an extended duration or if the impact needs to be studied over a substantial period of time. To avoid spill- over from the experimental to the control group, teachers would need to refrain from collaborating with one another, which is directly counter to current recommendations for establishing learning communities within schools and districts. Alternatively, the school or district rather than individual teachers could be the unit of random assignment, but that typically results in a larger and much more costly experiment. Third, discussions of RCTs as the gold standard for evaluation tend to focus primarily on judging impact, downplaying the need to understand not just if a program is effective, but also how it works and under what conditions. A truly rigorous program evaluation would include exploring the nature and quality of implementation, as well as both proximal and distal outcomes, in a variety of contexts; one or two RCTs or quasi-experimental studies focusing only on the ultimate impact of a program, e.g., student achievement, leave many important questions unanswered. Q1b. What advice would you give the federal R&D agencies for developing evaluation methods to help them judge which programs are effective and whether proposed programs will be successful? A1b. There are many, many needs in science education, and many possible approaches to improving the situation. But given the magnitude of the problems and the scarcity of resources for addressing them, it is important that funding be limited to programs that will make a substantial difference. In my view, the first step is for each proposed program--and continuations/modifications of existing programs--to specify its goals, in effect describing what would count as success. Agencies would be held accountable for achieving their specified goals, so there would be an incentive to identify goals that are both realistic and measurable. Once the goals of a proposed program are specified, the agency should describe the theory of action underlying the program design--how the planned activities are expected to lead to the desired outcomes-- citing available evidence that supports particular elements of the program design. For example, teacher professional development programs do not directly affect student attitudes, aspirations, or achievement. If a professional development program's goals include impact on students, then a case needs to be made how the program activities will result in those impacts. A sample theory of action might be: Professional development activities will focus on concepts A, B, and C (a small number of important concepts addressed in national standards for the target grade range that are relevant to the mission of the agency), providing opportunities for teachers to explore these science concepts, to learn about applications of these concepts in the work of the agency, and to consider how their instructional materials can best be used to develop student understanding of these concepts. Teachers will gain a better understanding of the content, applications of the content, and use of their instructional materials, and be able to use their enhanced knowledge in their instruction. Improved instruction will enable students to see the relevance of the targeted content, motivate them to learn these concepts, and lead to improved student knowledge of those science ideas. Is the program well designed to achieve its goals? Prior to expending resources on implementation, it would be important to have the program design, theory of action, and evidence presented in support of the design critiqued by a small number of external experts, similar to the process used by the National Research Council for review of committee reports. Program designers' responses to these critiques, indicating how reviewers' concerns will be addressed, would be reviewed by program managers with input from another external expert acting as ``monitor''; only those programs that appear to have responded adequately to reviewers' concerns should be allowed to go forward. The next step that should take place before program implementation is the design of an evaluation to assess both the quality of implementation and its impact, using the program's theory of action as a guide. Suggested connections, for example between program activities and teacher content knowledge, or between teacher content knowledge and classroom practice, can be considered as hypotheses to be tested, and the evaluation plan should describe how that testing will take place. (It is important to note that an evaluation plan needs to specify both the measures to be used and the research design that will enable the evaluation to make the case that any gains that are found are due to the program and not extraneous influences.) Only programs that can be adequately evaluated should be considered for funding. It makes little sense to devote resources to an endeavor if you won't be able to determine if it is successful. If a program has a promising design, and can be evaluated, it should be piloted on a small scale to see if it can be implemented with quality and to check to see if the proposed measures are sensitive to the interventions. If not, there may be problems with the program and/ or there may be problems with the instruments, the program should not be scaled up until these problems are identified and resolved. Only when a program has cleared these hurdles--a program design that seems promising based on prior evidence, can be implemented with quality, and can be adequately evaluated--does it merit broader implementation, and then it needs to be carefully evaluated to see if the program is in fact achieving its goals, and under what conditions. Questions submitted by Representative Eddie Bernice Johnson Q1. The witnesses provided a range of views on how R&D mission agencies can best contribute to STEM education. To what extent is there a consensus among the panelists about what the agencies can do well and what they are less well suited to do? A1. I believe there was considerable consensus on a number of issues: <bullet> The federal R&D mission agencies have an important role to play in the informal education arena--including museums and science centers--in increasing students' interest in science, encouraging them to consider science careers, and enhancing science literacy of the general public. <bullet> It is typically not helpful for the agencies to develop posters and activities for classroom use. Even if these materials are very well aligned with national and state standards, and many are not, they add noise to an already noisy system, making it less likely that students will encounter a coherent science curriculum. <bullet> If the agencies are going to be involved in teacher professional development, they need to partner with groups that have expertise that goes beyond content knowledge to include developing teacher pedagogical content knowledge and skill in applying what they learn to the classroom using the instructional materials selected by their districts. Q2. How would these programs need to be designed so that they can achieve classroom effectiveness? A2. The teacher professional development programs carried out by the federal R&D mission agencies have the potential to enhance teacher content knowledge. (It is difficult to know the extent to which they have actually done so, as in the absence of appropriate measures, evaluations of these and many other programs have had to rely on teacher self-report to assess impact.) Getting transfer to classroom practice has proven to be more difficult. The evidence we have available suggests that to foster improved classroom instruction, professional development needs to be sustained over time, include a focus on pedagogical content knowledge, such as common student misconceptions in a specific content area, how to assess student understanding of the target concepts, and how to use that information to improve teaching and learning. In my view, the agencies are not well-positioned to provide sustained, professional development focused on helping teachers not only learn content, but also apply that content to their instruction, but if they are going to do so, it would be helpful if the agencies partnered with groups that have the necessary expertise. In contrast, I believe the agencies could make an important contribution by providing opportunities for prospective secondary science teachers to have an authentic research experience as part of their pre-service, or lateral entry, preparation, a much less expensive and potentially more scalable strategy than current programs targeted at that goal. Each of the agencies could develop research modules relevant to their mission and aligned with national standards that could be incorporated into existing college/university courses, with college/university faculty and/or agency scientists serving as mentors depending on the context. Appendix 2: ---------- Additional Material for the Record <GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT> FEDERAL STEM EDUCATION PROGRAMS ---------- WEDNESDAY, JUNE 6, 2007 House of Representatives, Subcommittee on Research and Science Education, Committee on Science and Technology, Washington, DC. The Subcommittee met, pursuant to call, at 2:40 p.m., in Room 2318 of the Rayburn House Office Building, Hon. Jerry McNerney [Acting Chairman of the Subcommittee] presiding. <GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT> hearing charter SUBCOMMITTEE ON RESEARCH AND SCIENCE EDUCATION COMMITTEE ON SCIENCE AND TECHNOLOGY U.S. HOUSE OF REPRESENTATIVES Federal STEM Education Programs wednesday, june 6, 2007 2:00 p.m.-4:00 p.m. 2318 rayburn house office building Purpose The purpose of the hearing is to review the K-16 science, technology, engineering, and mathematics (STEM) education activities of federal agencies and to explore current efforts for the improvement of interagency coordination and evaluation of programs. In addition, agencies will be asked to respond to the witness testimonies given on May 15, 2007, about the educators' views on the STEM education programs at federal R&D mission agencies. The witnesses provided Subcommittee Members with their suggestions for how those agencies could best contribute to STEM education nationwide and strongly recommended closely collaborating with educators in the field when developing programs. Witnesses Dr. Cora Marrett, Assistant Director, Directorate for Education and Human Resources, National Science Foundation; Co-Chair, Education and Workforce Development Subcommittee, National Science and Technology Committee Dr. Joyce Winterton, Assistant Administrator, Office of Education, National Aeronautics and Space Administration Mr. William Valdez, Director, Office of Workforce Development for Teachers and Scientists, Office of Science, Department of Energy Dr. Bruce Fuchs, Director, Office of Science Education, National Institutes of Health Overarching Questions <bullet> What steps have agencies taken to improve coordination with other federal agencies' STEM education activities and, in particular, what is the status of the new coordinating committee under the National Science and Technology Committee (NSTC)? To what extent do agencies collaborate with educators in the states and school districts in developing STEM education programs? <bullet> The recent report of the Academic Competitiveness Council reinforces the need for better evaluation and performance metrics for federal STEM education programs. What plans do agencies have to improve evaluation of STEM programs? <bullet> The Subcommittee received testimony at a hearing on 15 May on how the R&D mission agencies could improve the effectiveness of their STEM education programs. The witnesses were skeptical of the ability of the R&D mission agencies to develop curricular materials for formal classroom instruction and questioned the effectiveness of their teacher professional development programs to improve teacher classroom performance, while suggesting that the agencies' most important role is in informal STEM education. The witnesses also strongly recommended closer collaboration by the agencies with educators in the field when developing STEM programs. What are agencies' responses to the recommendations from these witnesses? <bullet> How do the agencies determine priorities for their K- 16 STEM education portfolios? Have the agencies' balance of programs at graduate/post doctoral, undergraduate, K-12, and informal education changed much over the past few years? Is there a likelihood of a change in that balance in the future? <bullet> How do agencies disseminate information about STEM education programs? What organizations, both government and private, have agencies partnered with to reach educators in the field? Background STEM Education Funding In an effort to identify the contributions of federal agencies to improving STEM education, the Academic Competitiveness Council (ACC) was created in the Deficit Reduction Act of 2005 (P.L. 109-171) and charged with creating an inventory of STEM education programs across federal agencies, identifying the effectiveness of those programs, determining areas of overlap or duplication among programs, identifying target populations served by the programs, and recommending processes to integrate and coordinate those programs. After a year long study, the ACC released a report containing an inventory of $3.12 billion in funding for Fiscal Year (FY) 2006 for 105 STEM education programs. This inventory showed that nearly 50 percent of funding was directed toward Graduate/Post Doctoral programs ($1.4 billion) and another 30 percent was directed toward Undergraduate Programs ($943 million). K-12 programs received approximately $574 million in funding and informal education programs received $137 million in funding. <GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT> The agencies under the jurisdiction of the Committee on Science and Technology showed a similar balance of funding for STEM education programs with about two-thirds of funding going to post-secondary programs. The National Institutes of Health, whose representative has been included as a witness in this hearing due to agency's large investment in STEM education, reported a total of $856 million in FY06 funding with 89 percent dedicated to the Kirschstein National Research Service Award for graduate/post doctoral fellowships ($761 million). NIH provided approximately $52 million for K-12 programs (six percent), $37 million for undergraduate programs (4.4 percent), and $5 million for informal education programs (0.6 percent). <GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT> The ACC set parameters of its inventory, limiting the programs for inclusion to those ``primarily intended to provide support for, or to strengthen, science, technology, engineering, or mathematics education.'' The Subcommittee on Research and Science Education, realizing that many educational activities carried out by the federal R&D mission agencies are contained within larger programs, worked with those agencies to provide a more in-depth view of those efforts. Excluding graduate education programs which already dominate mission agencies' STEM funding, an additional $256.65 million in FY06 appropriated funds for K-12, undergraduate, and informal education activities was identified for a total of $1.01 billion in K-16 funding at NSF and the federal R&D mission agencies. (EPA reported no FY06 funding for K-16 STEM education activities.) <GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT> As can be seen in the table in Figure 3 K-16 STEM Ed FY06 Appropriated Funds, roughly an equal amount of funding is dedicated to undergraduate activities as K-12 and informal education activities combined. <GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT> Evaluation of Programs Another consistent criticism of federal STEM education programs is a lack of evaluation methods which would show the level of effectiveness of a program. The ACC studied evaluation processes used by the identified STEM education programs and concluded in its report that ``there is a general dearth of evidence of effective practices and activities in STEM education.'' The ACC recommended that funding for any program should not be increased until it can show effectiveness as determined by rigorous evaluation methods. The report points to randomized control trials or, when that is not feasible, well-matched comparison group studies as the optimal methods for determining if a program is effective. This sentiment was echoed by the witnesses at the Subcommittee's hearing on May 15, 2007, Federal STEM Education Programs: Educators' Perspectives, who explained to Members that the absence of consistent performance measurements makes choosing among the vast array of programs difficult and time-consuming. Although all of the witnesses agreed that evaluations should be improved, two stated that they did not think that research methodologies such as randomized controlled trials would be practical or necessary for the majority of programs. Instead, they recommended that programs focus on developing design critiques of proposed programs and formative evaluations of current programs to guide decisions for building highly effective programs. Coordination and Collaboration The federal agencies have also been criticized for their lack of coordination and collaboration between agencies and with state and district education agencies when developing programs. All of the witnesses in the May 15th hearing agreed strongly with the need for more effort by the federal agencies to work with educators in the field. The Mathematics and Science Director of Chicago Public Schools explained that materials and programs developed by federal agencies that do not fit into the district's curriculum and the state standards only add confusion and distract from the successful program they have built. The Director of Science, Technology, and Mathematics at Western Washington University commented, ``There is a huge inventory of poorly- designed and under-evaluated mission-related curricula, posters, and lesson plans and associated professional development rarely used in classrooms and with no natural home in a coherent standards-based curriculum. The constant barrage of new resources adds to the noise in the system and contributes to the mile-wide, inch-deep problem.'' In response to this issue, the ACC recommended that the National Science and Technology Committee (NSTC), which serves as the principal body for coordinating federal research and development, re-establish the Education and Workforce Development subcommittee to encourage the agencies to share knowledge and develop a federal strategic plan for effectively increasing STEM proficiency nationwide. The NSTC recently announced the subcommittee will be co-chaired by NSF, the Department of Education, and the National Institutes of Health. Dr. Cora Marrett, as the Co-Chair from NSF, has been asked to provide an update on the status of the NSTC subcommittee in this hearing. Specific Questions for the Witnesses Dr. Marrett <bullet> As Co-Chair of the NSTC Subcommittee on Education and Workforce Development, please describe the make up of the group, current activities, and planned activities. <bullet> What steps has your agency taken to improve its coordination with other federal agencies' STEM education activities? How has your agency improved its collaboration with states and districts in developing STEM education programs? Please describe your agency's commitment to establishing formal mechanisms to improve in these areas. <bullet> The ACC report reinforces the need for better evaluation and performance metrics for federal STEM education programs. How has your agency made improvements in its evaluation of programs? How has this affected your agency's funding for STEM education programs? <bullet> How does your agency determine priorities for its K- 16 STEM education portfolio? Has your agency's balance of programs at graduate/post doctoral, undergraduate, K-12, and informal education changed? Do you foresee a change in that balance in the future? <bullet> How does your agency disseminate information about its STEM education programs? What organizations, both government and private, have you partnered with to reach educators in the field? Dr. Joyce Winterton <bullet> What steps has your agency taken to improve its coordination with other federal agencies' STEM education activities? To what extent does your agency collaborate with educators in the states and school districts in developing STEM education programs? <bullet> The recent report of the Academic Competitiveness Council reinforces the need for better evaluation and performance metrics for federal STEM education programs. What plans does your agency have for improvements in its evaluation of its STEM programs? <bullet> The Subcommittee received testimony at a hearing on 15 May on how the R&D mission agencies could improve the effectiveness of their STEM education programs. (Witness statements and video of the hearing can be downloaded at http:/ /www.science.house.gov/publications/hearings<INF>-</INF> markups<INF>-</INF>details.aspx?NewsID=1814). The witnesses were skeptical of the ability of the agencies to develop curricular materials for formal classroom instruction and questioned the effectiveness of their teacher professional development programs to improve teacher classroom performance, while suggesting that the agencies' most important role is in informal STEM education. The witnesses also strongly recommended closer collaboration by the agencies with educators in the field when developing STEM programs. What is your response to the recommendations from these witnesses? <bullet> How does your agency determine priorities for its K- 16 STEM education portfolio? Has your agency's balance of programs at graduate/post doctoral, undergraduate, K-12, and informal education changed much over the past few years? Do you foresee a change in that balance in the future? <bullet> How does your agency disseminate information about its STEM education programs? What organizations, both government and private, have you partnered with to reach educators in the field? Mr. William Valdez <bullet> What steps has your agency taken to improve its coordination with other federal agencies' STEM education activities? How has your agency improved its collaboration with states and districts in developing STEM education programs? Please describe your agency's commitment to improving in these areas. <bullet> The ACC report reinforces the need for better evaluation and performance metrics for federal STEM education programs. How has your agency made improvements in its evaluation of programs? How has this affected your agency's funding for STEM education programs? <bullet> In response to the testimonies given on May 15th by STEM educators, what do you recommend as the most effective role your agency can play in improving STEM literacy? <bullet> How does your agency determine priorities for its K- 16 STEM education portfolio? Has your agency's balance of programs at graduate/post doctoral, undergraduate, K-12, and informal education changed? Do you foresee a change in that balance in the future? <bullet> How does your agency disseminate information about its STEM education programs? What organizations, both government and private, have you partnered with to reach educators in the field? <bullet> Please describe the process you utilized to gather information for creating a strategic plan for the OWDTS education programs. Include a synopsis of the information gathered. Dr. Bruce Fuchs <bullet> What steps has your agency taken to improve its coordination with other federal agencies' STEM education activities? To what extent does your agency collaborate with educators in the states and school districts in developing STEM education programs? <bullet> The recent report of the Academic Competitiveness Council reinforces the need for better evaluation and performance metrics for federal STEM education programs. What plans does your agency have for improvements in its evaluation of its STEM programs? <bullet> The Subcommittee received testimony at a hearing on 15 May on how the R&D mission agencies could improve the effectiveness of their STEM education programs. (Witness statements and video of the hearing can be downloaded at http:/ /www.science.house.gov/publications/hearings<INF>-</INF> markups<INF>-</INF>details.aspx?NewsID=1814). The witnesses were skeptical of the ability of the agencies to develop curricular materials for formal classroom instruction and questioned the effectiveness of their teacher professional development programs to improve teacher classroom performance, while suggesting that the agencies' most important role is in informal STEM education. The witnesses also strongly recommended closer collaboration by the agencies with educators in the field when developing STEM programs. What is your response to the recommendations from these witnesses? <bullet> How does your agency determine priorities for its K- 16 STEM education portfolio? Has your agency's balance of programs at graduate/post doctoral, undergraduate, K-12, and informal education changed much over the past few years? Do you foresee a change in that balance in the future? <bullet> How does your agency disseminate information about its STEM education programs? What organizations, both government and private, have you partnered with to reach educators in the field? Mr. McNerney. Welcome to today's hearing, entitled ``Federal STEM Education Programs.'' I personally want to thank everyone for coming here today. It is a nice audience for the proceedings of the Subcommittee on Research and Science Education. This hearing is part of an ongoing assessment that the Committee is undertaking to determine the role of the Federal Government in science, technology, engineering, and math education. We will be hearing today from four federal agencies about their progress in improving STEM programs. As a mathematician and someone who believes strongly in the need to expand educational opportunities for the next generation, today's hearing holds particular importance for me. In the past few months, this committee has done a commendable job in drawing attention to the difficulties that our country will face if we are unable to increase the number of future scientists and researchers. From the difficult but not insurmountable challenges laid out in the National Academies' ``Rising Above the Gathering Storm'' report, to witness testimony from businesses and universities, we have heard time and again that we need more STEM education graduates and teachers if we are to compete successfully in the global economy. This committee, under the leadership of Chairman Gordon, has succeeded in passing legislation that will pay significant future dividends, and will be a great legacy for this Congress. The issues we are discussing today are no less important. By hearing from educators on the front lines of the educational system, as well as the federal agencies that must implement STEM programs, we are laying out the groundwork for how to maximize the benefits of existing programs, which will invariably lead to future initiatives. Last month, the Subcommittee held a hearing with local STEM educators to learn their perspectives on federal programs. During witness questioning, I made a point to highlight what I believe to be one of the most important aspects of any discussion on STEM education, and that is how we can reach more students, and make sure that the United States is not only keeping up with the rest of the world, but is outpacing other countries. It is hard work to earn a degree in math or physics, as our Ranking Member might testify, and we need to make sure that our federal policies first and foremost benefit our students. Our previous witnesses shared with us their recommendations for STEM education improvements. That included working with states and districts to align federal programs to local standards. The witnesses also expressed frustration that there are currently no consistent guidelines for evaluating STEM programs across government agencies, and we have asked today's witnesses to respond to the comments and recommendations from the previous panel. In addition, the week before last month's hearing, the Academic Competitiveness Council released a comprehensive report on federal STEM education programs. Not surprisingly, the ACC assessment was the same as that of our witnesses. Collaboration between agencies and with State and local governments needs improvement. Programs should be designed around best practices, and a wide variety of educational measurements should be simplified into common ones. Each of the agencies invited to testify this afternoon were deeply involved in the ACC's process, and we have asked each of them to communicate today what changes they have made over the past year in order to achieve these goals. I am hopeful that today's witnesses will shed some light on how we have been spending scarce federal dollars on STEM education, and how we can improve our current practices so that we are benefiting students from kindergarten all the way up. I look forward to the discussion with our witnesses and how their agencies are planning, coordinating, and evaluating their efforts to improve STEM education. The Chair now recognizes Dr. Ehlers for an opening statement. [The prepared statement of Vice Chairman McNerney follows:] Prepared Statement of Vice Chairman Jerry McNerney Good afternoon, and thanks to everyone for attending today's proceedings of the Subcommittee on Research and Science Education. This hearing is part of an ongoing assessment that the Committee is undertaking to determine the role of the Federal Government in science, technology, engineering, and math education. We'll be hearing today from four federal agencies about their progress improving STEM programs. As a Mathematician, and someone who believes strongly in the need to expand educational opportunities for the next generation, today's hearing holds particular importance for me. In the past few months, this committee has done a commendable job in drawing attention to the difficulties our country will face if we are unable to increase the number of future researchers and scientists. From the difficult--but not insurmountable--challenges laid out in the National Academies' Rising Above the Gathering Storm report to witness testimony from businesses and universities, we've heard time and again that we need more STEM educated graduates and teachers if we can compete in a global economy. This committee--under the leadership of Chairman Gordon--has succeeded in passing legislation that will pay significant future dividends, and will be a great legacy for Congress. The issues we're discussing today are no less important. By hearing from educators on the front lines of the education system as well as the federal agencies who must implement STEM programs, we're laying the groundwork for how to maximize the benefit of existing programs, which will invariably improve future initiatives. Last month, the Subcommittee held a hearing with local STEM educators to learn their perspectives on federal programs. During witness testimony and in questioning, I made it a point to highlight what I believe to be one of the most important aspects of any discussion on STEM education and that is how we can reach more students and make sure that the United States is not only keeping up with the rest of the world, but out-pacing other countries. It is hard work to earn a degree in math or physics, and we need to make sure that our federal policies--first and foremost--benefit students. Our previous witnesses shared with us their recommendations for STEM education improvements that included working with states and districts to align federal programs to local standards. The witnesses also expressed frustration that there are currently no consistent guidelines for evaluating STEM programs across government agencies, and we have asked today's witnesses to respond to the comments and recommendations from the previous panel. The week before last month's hearing, the Academic Competitiveness Council released a comprehensive report on federal STEM education programs. Not surprisingly, the ACC assessment was the same as that of our witnesses--collaboration between agencies and with State and local governments needs improvement, programs should be designed around best practices, and a wide array of evaluation measurements should be simplified into common ones. Each of the agencies invited to testify this afternoon were deeply involved in the ACC's process and we have asked each of them to communicate today what changes they have made over the past year in order to achieve these goals. I'm hopeful that today's witnesses will shed some light on how we've been spending scarce federal dollars on STEM education, and how we can improve on current practices so that we're benefiting students from kindergarten and up. I look forward to the discussion with our witnesses on how their agencies are planning, coordinating, and evaluating their efforts to improve STEM education. Mr. Ehlers. Thank you, Mr. Chairman. It is a pleasure to be here to participate in yet another event centered on STEM education. You and I, I think, have both spent a good part of our lives dedicated to education, particularly science, technology, engineering, and math education. I do have to clarify that what the S, T, M, and E stand for, even though everyone in here should be familiar with it, but I have to do that for anyone who isn't familiar, because tomorrow, we are going to have debate, discussion, and votes on two stem cell bills, and so, all the papers are going to talk about for the next few days is votes on stem cell bills. We are not talking about a stem cell bill here today. In fact, I think this is, for the future of our nation, probably considerably more interesting. STEM education is a priority for this nation. It has to be if our nation is to survive, to continue to improve in the welfare of its own citizens, and frankly, also in the welfare of the world. Thanks to constantly increasing understanding of the importance of STEM education to our national competitiveness, I no longer have to go into a lot of details on this, but even with this improving awareness of STEM education, there is still more that the Federal Government can do to improve K-12 STEM education in the United States, and we clearly need improvement, because we are still ranked near the bottom of the developed countries in STEM education. The Academic Competitiveness Council was created by Congress to catalog and coordinate the STEM education projects and programs currently supported by the Federal Government. I commend the agencies that participated in this endeavor. For some time, I have been very concerned about the fact that the Federal Government and its agencies and also some private entrepreneurs in industry are developing these fantastically good programs, but there is no overall correlation of how they should be used in the classroom, how they fit in with the curriculum, and I think we should be working on that. I commend the agencies that have participated in this endeavor. The charge to the Academic Competitiveness Council was a challenging one, and the report reflects the breadth and depth of programs that exist at our federal agencies. From the start, I harbored a general concern that the ACC might overzealously seek out seemingly duplicative programs, and inadvertently encourage their demise, which happened a few years ago with the Math Science Partnerships. Someone in the basement of the White House noticed that we had two Math Science Partnerships, one in the National Science Foundation, one in the Department of Education, and said, ``A ha, duplication, we must get rid of one.'' Fortunately, this has not happened in your case on these issues. I think the ACC report sheds light on the diversity and uniqueness of the programs that are developed, and sends a clear message that Congress must authorize adequate evaluation capacity for federal STEM education programs. It is crucial that we evaluate these programs with the most appropriate and rigorous techniques available. Overall, the ACC report provides a useful foundation for future coordination and collaboration, so that federal agencies can work together to leverage STEM resources and communicate successes as well as failures. I am pleased that the recently reestablished National Science and Technology Council Subcommittee on Education and Workforce Development will follow through on actions recommended by the ACC. I look forward to hearing from our witnesses today about how they are moving towards increasing collaboration, as well as becoming more educated about the STEM education programs at their respective agencies. With that, I yield back the balance of my time. [The prepared statement of Mr. Ehlers follows:] Prepared Statement of Representative Vernon J. Ehlers STEM education is a priority for this nation. Thanks to a constantly increasing understanding of the importance of STEM to our national competitiveness, I no longer must define what the ``S,'' ``T,'' ``E'' and ``M'' stand for; today, my colleagues are familiar with the acronym. Even with this improving awareness of STEM, there is still more that the Federal Government can do to improve K-16 STEM education in the U.S. The Academic Competitiveness Council (ACC) was created by Congress to catalog and coordinate the STEM education projects and programs currently supported by the federal government. I commend the agencies that participated in this endeavor. The charge to the Council was a challenging one, and the report reflects the breadth and depth of programs that exist at our federal agencies. From the start, I harbored a general concern that the ACC might overzealously seek out seemingly duplicative programs and inadvertently encourage their demise. Instead, I think the ACC report sheds light on the diversity and uniqueness of programs, and sends a clear message that Congress must authorize adequate evaluation capacity for federal STEM education programs. It is crucial that we evaluate these programs with the most appropriate and rigorous techniques available. Overall, the ACC report provides a useful foundation for future coordination and collaboration, so that federal agencies can work together to leverage STEM resources and communicate successes as well as failures. I am pleased that the recently re-established National Science and Technology Council (NSTC) will follow through on actions recommended by the ACC. I look forward to hearing from our witnesses today about how they are moving toward increased collaboration, as well as becoming more educated about the STEM education programs at their respective agencies. Mr. McNerney. Thank you. The Chair thanks the gentleman from Michigan. If there are any Members, I don't see any here, but if there are any Members who wish to submit additional opening statements, your statements will be added to the record at this point. [The prepared statement of Ms. Johnson follows:] Prepared Statement of Representative Eddie Bernice Johnson Thank you, Mr. Chairman. Today's hearing will be valuable to determine how to help the federal science agencies become more efficient in working together in their efforts toward STEM education. As you know, the Academic Competitiveness Council recently studied evaluation processes used by the various federal STEM education programs and concluded in its report that ``there is a general dearth of evidence of effective practices and activities in STEM education.'' The ACC recommended that funding for any program should not be increased until it can show effectiveness as determined by rigorous evaluation methods. Witnesses at the May 15, 2007, Subcommittee hearing on ``Federal STEM Education Programs: Educators' Perspectives'' concurred with this sentiment. The absence of consistent performance measurements makes choosing among the vast array of programs difficult and time-consuming. Mr. Chairman, it can safely be said that all Members of this subcommittee care deeply about our scientific enterprise and are committed to supporting and streamlining it to most responsibly use taxpayer dollars. We hope to determine, from witness feedback, how to work toward that goal. Again, welcome to today's witnesses. Thank you, Mr. Chairman. I yield back. [The prepared statement of Mr. Carnahan follows:] Prepared Statement of Representative Russ Carnahan Mr. Chairman, thank you for hosting this hearing to examine the participation of federal agencies in STEM education and investigate approaches to improving coordination and evaluation of their programs. As we have all mentioned time and again, the Rising Above the Gathering Storm report provided us with both the knowledge that our nation's standing as the global leader in the STEM field is at risk as well as solid tools for policy-makers to counteract this worrisome trend. Chairman Gordon, you have been a tremendous advocate for improving STEM education in this nation. I am proud to be a Member of the Committee on Science and Technology--under your leadership we have successfully moved four major innovation initiatives through the House just in these past couple months. I am pleased that today's hearing again focuses on the important task of ensuring that our STEM programs are working to the best of their abilities. I am eager to hear our witnesses' assessments of these agencies' contributions to STEM education programs so that we can reflect on the successes and inefficiencies of the programs and seek to make modifications for improvement. Your first-hand experiences are vital to maximizing the resources we are offering our nation's aspiring students. To all the witnesses--thank you for taking time out of your busy schedules to appear before us today. I look forward to hearing your testimony. Mr. McNerney. At this time, I would like to introduce our distinguished witnesses. First, we have Dr. Cora Marrett, is that being pronounced correctly? She is the Assistant Director for the National Science Foundation Directorate for Education and Human Resources. Dr. Marrett is also the Co-Chair of the Education and Workforce Development Subcommittee under the NSTC. Welcome aboard, Dr. Marrett, and she is going to be chairing the new subcommittee, so it is a big burden on your shoulders, and I am looking forward to your words. Our second panelist is Dr. Joyce Winterton. She is the Assistant Administrator of NASA's Office of Education. Welcome. Third, we have Mr. Bill Valdez, Director of Office for Workforce Development for Teachers and Scientists at the Department of Energy. Welcome, this afternoon. And finally, we have Dr. Bruce Fuchs, and he is the Director of the Office of Science Education at the National Institutes of Health. Welcome all, and as our witnesses know, spoken testimony is limited to five minutes each, after which each Member of the Committee will have five minutes to ask questions. We will try and limit you to five minutes, but I understand if you have a few extra minutes to run over, but we do want to keep things in line. So, we will start with Dr. Marrett at this point. Would you begin your testimony? STATEMENT OF DR. CORA B. MARRETT, ASSISTANT DIRECTOR, EDUCATION AND HUMAN RESOURCES DIRECTORATE, NATIONAL SCIENCE FOUNDATION Dr. Marrett. Thank you very much, Chairman McNerney, and Ranking Member Ehlers. I do appreciate the opportunity today, and want to express my gratitude to the entire Subcommittee, for your longstanding support for excellence in science, technology, engineering, mathematics education, or STEM. The National Science Foundation appreciates the interest expressed by this subcommittee and others in coordination and evaluation of STEM-centered activities. In fact, the National Foundation, or NSF, takes pride in the actions we have undertaken over the years to enhance excellence across all levels of education and all fields of science and engineering. We owe our successes to the interactions we have had with communities of researchers, educators, diverse organizations, and indeed, other agencies. Those interactions have shaped significantly the content of our efforts, our evaluation of them, and our dissemination strategies. We are aware, however, that we must revisit continually the approaches and connections we cultivate. This said, these are some of the ways in which we are looking at what is the current situation, and how we respond to the challenges that now exist. The reconstitution of the Subcommittee on Education and Workforce Development of the National Science and Technology Council, to which you have just referred, that reconstitution should help us in NSF strengthen our ties to other federal agencies. I should note we already have a number of ties. One of our most recent is a partnership through a memorandum of understanding with NASA, and I am delighted that Dr. Winterton and I had an opportunity quite recently to host a conference in which we had participation from our other colleagues here at the table and other agencies. The subcommittee, the reconstituted subcommittee of the NSTC, the membership for it will come from the agencies represented on NSTC's larger Committee on Science. The representatives are to possess substantive knowledge of their STEM education portfolio within the agency, and these representatives are to have experience with evaluation research and, possibly, with the development and application of performance measures. So, in looking at what we need for that committee, we realize it is knowledge both of what agencies are doing, but familiarity to work on these very important issues of evaluation research performance measures. The subcommittee that I am referring to will address a broad range of issues related to STEM education. To use a phrase that is often heard these days, this will be attending from K to gray, because the education portfolio does indeed cover all phases of education and workforce activities. The subcommittee will provide a forum for exchanges of information and expertise regarding research and evaluation. On the one side, then, we have the development, the reconstitution of the Subcommittee on Education and Workforce Development. Another important development, of course, is the report you have referred to from the Academic Competitiveness Council, or the ACC. As we look at the report, it certainly serves to enhance our attention to evaluation. Now, within the National Science Foundation, there has been a requirement that there is evaluation associated with every program within the Directorate of Education and Human Resources. In fact, Congress had a lot to do with the directive that set this as expected. We have responded and indeed now have a requirement that all such programs within this part of the organization do have to be subject to evaluation. But we have other work to do, and we see ourselves as attending increasingly to such matters as clarifying the objectives of particular programs. Evaluation must depend on how clear is the intent of any activity. We see ourselves as attending far more perhaps to the definition of concepts, and ensuring that those definitions are shared, especially with the other agencies that must be involved. These become fundamental issues to ensure that we will be able to conduct the rigorous evaluations that we agree must be essential for determining how resources are being used. In collaboration with others, we will strive to enhance the capacity for and knowledge base of such evaluation. There is a need for expanding the community, the experts who can, in fact, bring to the matter of evaluation and research the strong conceptual theoretical work that is essential. There are, in addition to these matters of evaluation, extensive possibilities for expanded opportunities to improve STEM education, opportunities for engaging with others on research, for example. The hearing that you had with STEM educators, the inquiries we have received from foundations, lots of private foundations, industrial groups, the responsiveness we have received from school districts and higher education associations, all of these developments prompt our heightened attention and commitment to collaboration, collaboration in the cause of excellence. In closing, then, we at the National Science Foundation will not rest on our past achievements. Rather, we will continue to foster and tap the creativity this Nation needs for the success of our citizenry in the years ahead. I am willing, of course, to respond to questions. This could be nothing more than a quick snapshot of what we have in mind, what we have done, and where we intend to go at the National Science Foundation. Thank you. [The prepared statement of Dr. Marrett follows:] Prepared Statement of Cora B. Marrett Chairman Baird, Ranking Member Ehlers, and Members of the Subcommittee. Thank you very much for inviting me to testify before you today on science, technology, engineering, and mathematics (STEM) education. This subcommittee's commitment to excellence in STEM education at the National Science Foundation (NSF) is well known, and we are extremely appreciative of your long-standing support. As you are well aware, the NSF provides leadership at the federal level to advance learning and discovery in all disciplines of science and engineering and to foster connections among the disciplines. The Director of NSF, Dr. Arden Bement, has presented the case eloquently: ``Our job is to keep science and engineering visionaries focused on the furthest frontier, to recognize and nurture emerging fields, to prepare the next generation of scientific talent, and to ensure that all Americans gain an understanding of what science and technology have to offer.'' The questions for today's hearing center on the coordination of STEM-related programs, the evaluation of those programs, and the dissemination of information about effective strategies. These long have been central concerns for NSF, as is evident in activities we have undertaken over the years. But we are cognizant of changes looming on the horizon that will require heightened attention to coordination, research and evaluation as well as dissemination. On Coordination and Collaboration. The National Science Foundation works in partnership with the research and education community to promote excellence. Hence, for us effectiveness is indicated in no small part by the connections we establish and maintain with researchers and educators as well as with agencies and organizations that share our commitment to excellence in STEM education. We seek opportunities to foster exchanges on matters critical to such excellence. An example: the conference held recently on state standards for mathematics. What gave rise to the conference were the development by states of different standards, the efforts of several national organizations to align those standards, and the interest of state supervisors of mathematics in exchanging ideas and experiences. The conference, held in February 2007, featured presentations on recommendations regarding standards and engaged ``users'' of standards--State and district curriculum specialists, textbook and assessment publishers, K-12 district and teacher leaders, and representatives from higher education and business. The National Science Foundation served as a co-sponsor of the conference, along with Achieve, Inc., the American Statistical Association, the College Board, the Mathematical Association of America, and the National Council of Teachers of Mathematics. The idea for the conference emerged from an NSF-sponsored entity: the Center for the Study of the Mathematics Curriculum at Michigan State University. That Center organized the conference, in concert with the State Supervisors of Mathematics. The case illustrates that NSF takes a broad approach to the challenges associated with coordination and collaboration. Our approach to coordination and collaboration extends beyond the formal education sector to include important activities in promoting understanding of science in the wider public. Towards that end, NSF organized in March 2007 a workshop on informal science activities conducted through science centers, museums, community projects and the media. The workshop brought designers of informal science initiatives together with program evaluators, to generate guides for the evaluation of such initiatives. The workshop included representatives from other federal agencies. Again, the action reinforces the theme that NSF supports coordination through outreach--to various communities and agencies--on matters relevant to STEM educational policies and practices. The informal science workshop demonstrates, too, that NSF both endorses and seeks to provide leadership on program evaluation. The evaluation efforts are tailored to the goals and state of development for any given program. Moreover, NSF invests in research and evaluation, not just to assess outcomes, but also to build knowledge about and a community prepared to advance STEM research and evaluation. A distinctive feature of the NSF STEM education portfolio is its breadth. Not only does it incorporate program development as well as research, and the informal as well as formal sectors; it addresses the pre-college realm, undergraduate and graduate education, post-doctoral experiences, and the STEM workforce of the Nation. This breadth has profound implications for the collaborations NSF undertakes, the evaluations it supports, and the dissemination strategies it pursues. Subcommittee Questions Having provided a general context for the questions central to this hearing, let me now turn more specifically to those questions. 1. As Co-Chair of the NSTC Subcommittee on Education and Workforce Development, please describe the make-up of the group, current activities, and planned activities. In response to the Academic Competitiveness Council (ACC) report, the Subcommittee is being re-constituted through representation from the agencies that comprise the Committee on Science of the National Science and Technology Council (NSTC). The representatives are to possess (1) substantive knowledge of STEM education programs within the given agency's portfolio, and (2) experience with evaluation research and/or the development and application of performance measures. These requirements will enable the Subcommittee to meets its initial goal to coordinate and facilitate implementation of the ACC recommendations. The Subcommittee is also expected to address a range of issues related to STEM education at all levels. 2. What steps has your agency taken to improve its coordination with other federal agencies' STEM education activities? How has your agency improved its collaboration with states and districts in developing STEM education programs? Please describe your agency's commitment to establishing formal mechanisms to improve in these areas. Past coordination activities include formal memoranda of understanding with the Department of Education (ED) and the National Institutes of Health (NIH) in 1992 and with the Department of Energy in 1995. Through the Interagency Educational Research Initiative, launched in 1999, NSF, NIH, and the U.S. Department of Education sponsored a program of research designed to develop and/or investigate the effectiveness of educational interventions in classrooms across the United States. Earlier this year, NSF signed a memorandum of understanding on STEM education cooperation with National Aeronautics and Space Administration (NASA). The goal of this partnership is to support the development of a creative and diverse engineering workforce that comprehends the technical and social impacts of technology applications and needs in a rapidly changing environment. Interactions with NASA precede the memorandum, however, and include our joint participation on a task force ``to examine the feasibility and benefits of using a portion of the International Space Station payload resources and accommodations for education.'' Among the ways in which NSF cooperates with the Department of Education, these especially warrant notice. A memorandum of understanding enabled the NSF and ED to fund jointly two of the large projects in our Math and Science Partnership (MSP) programs. Moreover, almost two-thirds of the sites in the NSF portfolio have some involvement as well with the state MSP projects that ED supports. In 2005 the U.S. Department of Education and the Education and Human Resources (EHR) Directorate of the National Science Foundation began collaborating on a Mathematics Education Toolkit. The Toolkit provides resources for state and district leaders on how to improve mathematics teaching and learning for Title I students. The Toolkit represents a response to concerns that states and districts have expressed. The workshop on standards, cited earlier, provides another example of the NSF connections beyond the federal level. The coordination challenges in the years ahead will extend beyond those found among federal agencies. Increasingly, foundations and corporations are investing in STEM education and the workforce. The National Science Foundation has a leadership role within the ACC and is committed to establishing whatever connections and mechanisms offer heightened possibilities for innovation in STEM education within the United States. 3. The ACC report reinforces the need for better evaluation and performance metrics for federal STEM education programs. How has your agency made improvements in its evaluation of programs? How has this affected your agency's funding for STEM education programs? The emphasis in NSF on program evaluation precedes the ACC report. A Congressional mandate in 1992 set in motion a systematic plan for assessment of programs within the EHR portfolio. The approach has evolved quite significantly over time, from one focused largely on the monitoring of developments to evaluations of impacts. The evolution has reinforced the importance of enhanced capacity for evaluation of STEM programs and accounts, then, for investments NSF has made in increasing expertise on evaluation. EHR education programs require project and program evaluations, and there is now greater emphasis on collecting evaluation information at the start of a program. The evaluation of a program's value, worth, and impacts is based on a multiplicity of assessment and review studies. NSF evaluation efforts range from periodic measures of project activities to in-depth analyses of a program's success. Quantitative and qualitative data are obtained to measure a program's success in achieving its goals. Our current approach encompasses a multiple method evaluation framework that combines theory and research to better understand and assess the R&D educational investment. This methodological pluralism enables programmatic decision-making to be based on the preponderance of the evidence from external studies. Through the NSTC, we will work to improve evaluation for STEM education initiatives across the Federal Government, including at NSF, to ensure that the most rigorous methods appropriate are used to assess federal programs. 4. How does your agency determine priorities for its K-16 STEM education portfolio? Has your agency's balance of programs at graduate/ post doctoral, undergraduate, K-12, and informal education changed? Do you foresee a change in that balance in the future? Issues for the K-16 STEM education portfolio emerge from various sources. The staff within NSF consists of specialists on STEM education within given disciplines, researchers with on-going connections to resources and knowledge, and experts on trends in STEM education in the United States and elsewhere. The panels that review proposals, the Committees of Visitors for our programs, and the Advisory Committees for each directorate keep us abreast of developments and interests. In recent years, reports on STEM education have yielded many recommendations, as have the priorities established in both the Executive and Legislative branches. In determining priorities for NSF funding, consideration is given to the capacity of external communities to pursue given lines of inquiry, the activities underway through other agencies and organizations, and the appropriateness of the topics for the NSF portfolio. The Foundation strives to address a broad portfolio for STEM education, but does not have a formula for investments at each educational level. The substance of those investments does not remain static, however, for it must reflect changes over time in knowledge, national needs, and capacities within our communities and NSF. 5. How does your agency disseminate information about its STEM education programs? What organizations, both government and private, have your partnered with to reach educators in the field? The National Science Foundation disseminates information about its programs and the results of its investments through various channels. There are websites for particular programs. These include IGERT.ORG, a website produced by the Integrative Graduate Education Research and Traineeship (IGERT) program that seeks to attract to STEM research groups now under-represented in science and engineering. Communication and collaboration among MSP partners is promoted through MSPNet. Similarly, the Center for Learning in Out of School Environments (UPCLOSE) at the University of Pittsburgh serves to link researchers and educators who want to enhance teaching and learning in informal environments. Publications from the National Academy of Sciences serve to share widely the results from NSF-investments. Among these: the path-breaking volumes, Adding It Up: Helping Children Learn Mathematics, and Taking Science to School. Our outreach efforts are extensive. What we intend to undertake in the near future is an assessment of the effectiveness of our strategies in reaching under-served communities--of educators, researchers, and institutions. Such an assessment, to be pursued in connection with our panels, advisory communities, and public and private partners, may result in modifications to our outreach efforts. We in NSF will not rest on past achievements or accolades. Rather, we will continue to strive to foster and tap the creativity this nation needs for the success of our citizenry in the years ahead. Biography for Cora B. Marrett Dr. Cora B. Marrett is the Assistant Director of the Directorate for Education and Human Resources (EHR) at the National Science Foundation (NSF). She leads the NSF's mission to achieve excellence in U.S. science, technology, engineering and mathematics (STEM) education with oversight of a budget of approximately $800 million and a staff of 150. EHR is the principal source of federal support for strengthening STEM education through education research and development (R&D). Prior to her appointment at the NSF, Dr. Marrett served as the Senior Vice President for Academic Affairs in the University of Wisconsin System. Her NSF position is in conjunction with the UW- Madison Department of Sociology, where she remains a tenured faculty member. Earlier, she held the post of Senior Vice Chancellor for Academic Affairs and Provost at the University of Massachusetts-Amherst. Her current position represents a return to NSF. From 1992-1996, she served at NSF as the first Assistant Director of the Directorate for Social, Behavioral and Economic Sciences. She received the NSF's Distinguished Service Award for her leadership in developing new research programs and articulating the scientific projects of the directorate. In addition to her faculty appointment at the University of Wisconsin-Madison, she has been a faculty member at the University of North Carolina and Western Michigan University. Dr. Marrett holds a B.A. degree from Virginia Union University, and M.A. (1965) and Ph.D. (1968) degrees from UW-Madison. She has an honorary doctorate from Wake Forest University (1996). She is a Fellow of the American Association for the Advancement of Science, the American Academy of Arts and Sciences, and Sigma Xi, the Science Research Society. In 2005, Dr. Marrett received the Erich Bloch Distinguished Service Award from the Quality Education for Minorities (QEM) Network, given annually to an individual who has made singular contributions to the advancement of science and to the participation of groups under- represented in science, technology, engineering and mathematics. She is widely published in the field of sociology, and has held a number of public and professional service positions. Mr. McNerney. Thank you, Dr. Marrett. Now, we will recognize Dr. Winterton. STATEMENT OF DR. JOYCE L. WINTERTON, ASSISTANT ADMINISTRATOR, OFFICE OF EDUCATION, NATIONAL AERONAUTICS AND SPACE ADMINISTRATION Dr. Winterton. Thank you, Chairman McNerney and Congressman Ehlers. I appreciate the opportunity to testify today, and have a discussion that we know is so important around STEM education. NASA certainly recognizes the role that education has in providing that next generation of scientists, engineers, and people that are experts in technology, that will really advance the Nation's economic well-being. The United States does have a tremendous need to sustain our competitive international collaborations, and we want to keep United States' preeminence in that area. NASA certainly serves as a contributor towards that goal. As was mentioned by Dr. Marrett, we have a growing number of agency collaborations: our memorandum of understanding with the National Science Foundation; also, we recently signed a memorandum of understanding with the Federal Aviation Administration. So, it was very rewarding, as we signed that, to see a group of middle school students experiencing our Smart Skies Initiative, that is a simulated online resource, where students get a feeling of what it is like to actually land a sequence of planes safely, using mathematics as part of that learning, and it was very interesting to see their motivation of seeing the real world context, and how you apply your math in a type of job that is in demand. I do serve on the interagency taskforce to revitalize the aerospace workforce that has been mandated by Congress, and through that, we are collaborating with the U.S. Department of Labor, the NSF, NIH, Department of Energy, Department of Defense, and looking forward to a substantial look at where we are currently since the Gathering Storm report, and what needs to be done to reenergize, and make sure we are on target with those. We certainly look at how we can work with other agencies. For instance, recently, at Johnson Space Center, we conducted a teacher-to-teacher training that is part of the U.S. Department of Education's initiative to provide professional development, and I think we had over 350 teachers who signed up and participated in that effort in Texas. As are other agencies, we have a renewed effort to re-look at what are vigorous metrics to evaluate the effectiveness of our program, the efficiency, but also, the long-term impact. How do we know that we are really investing our dollars in the right place? So, we actually have a specific schedule of how we will be looking at each of our major programs, and looking at the impact of those, as well. So, we are looking forward to our National Academies study on our pre-college programs that will be available in November, and again, that will be another opportunity for us to see how we are doing with those programs, how we can improve, and how we are meeting our customers' needs. The role of our agency certainly includes professional development. Being a former high school teacher, that is close to my heart, and a teacher educator. And it is very important for NASA to work closely with educators as we develop resources in a formal K-12 setting, or as we work with our informal partners in museums and community-based groups like Boy Scouts, Girl Scouts, and 4-H. Some of the things that we are doing, for instance, with our educator astronaut, who is a mission specialist, and will be part of our STS-118 launch in August, we have worked with the International Technology Education Association and the National Science Teachers Education, to develop resources so students can actually develop a growth chamber. They can do the engineering. Obviously, that is a skill we may need some day if we plan to go back to the Moon and beyond. So, students can use their science background, their engineering, at an elementary, middle school, and high school level to develop growth chambers, and then, we actually have basil seeds that will be flying on the Shuttle, that they can test, and see if their growth chamber will actually work. So, that is an example of the collaboration, making sure that we are connecting to the existing curriculum, bringing our resources that fit the standards, and what teachers can afford to do, both cost- effective and time-wise, because we know that is very important in the curriculum today. Our NASA Explorer Schools is an example of how, over three years, we work with a team within a school. That is a competitive process. Those educators help us determine what are the needs in that particular school, how we can meet their needs, but also, how we sustain that afterwards. In fact, at the National Science Teachers Association Conference, I had a teacher from Kentucky come up to me and say, ``I was part of an Explorer School. I am teaching at a different school now, but I can tell you I changed the way I teach because of that experience. I am teaching more real world, using NASA content, with my students today, so they see where they are going to apply their science and math.'' She said it has made such a difference in the responsiveness and interest of her students in STEM. Now, we certainly use our Digital Learning Network, which is an opportunity for us to connect schools to our engineers and scientists at our centers. That is really what NASA has to add, our content that is new and relevant, and information that may or probably isn't in a textbook, our facilities and our experts and our people. So our Digital Learning Networks, for instance, I saw a school from the State of Washington in a dialogue with the scientists at Johnson Space Center. So through that vehicle, we can reach every school in every state, even if they are not within a radius where they can travel to one of our NASA centers, although we certainly encourage that. We balance our portfolio, looking at higher education, undergraduate, graduate, and the K-12 as a really important part of our continuum, to have a pipeline to our workforce, not only for NASA, but also our contractors. And informal education is certainly a way to engage the public at large. We look forward to more opportunities to work with our counterparts in other agencies, educators, to really make sure we are on target meeting the needs of educators and students today. Thank you. [The prepared statement of Dr. Winterton follows:] Prepared Statement of Joyce L. Winterton Chairman Baird and Members of the Subcommittee, thank you for the opportunity to appear before the Subcommittee today to discuss NASA activities that support K-16 science, technology, engineering, and mathematics (STEM) educational programs. NASA recognizes the important role education plays in developing the diverse scientific and technological workforce required to advance this Nation's economic leadership. The United States has a tremendous need to build, sustain, and deploy the skilled talent that will be required to continue America's preeminence in space and aeronautics research and development in the coming decades. NASA serves as a contributor for achieving such goals. To ensure our future explorers will be ready to continue the journey, NASA is working with one of its most vital partners-- educators. This summer, NASA will ignite the flame of knowledge with the first space flight of one of NASA's most famous educator. Mission Specialist and Educator Barbara Morgan will engage students and educators worldwide from 240 miles above Earth aboard the International Space Station. NASA Administrator Michael Griffin recently stated, ``The greatest contribution that NASA makes in educating the next generation of Americans is by providing worthy endeavors for which students will be inspired to study difficult subjects like math, science and engineering because they too share the dream of exploring the cosmos.'' To this end, NASA educational investments are designed to: 1. Strengthen NASA and the Nation's future workforce--NASA will identify and develop the critical skills and capabilities needed to ensure achievement of exploration, science, and aeronautics. 2. Attract and retain students in STEM disciplines through a progression of educational opportunities for students, teachers, and faculty--To compete effectively for the minds, imaginations, and career ambitions of America's young people, NASA will focus on engaging and retaining students in STEM education programs to encourage their pursuit of educational disciplines critical to NASA's future engineering, scientific, and technical missions. 3. Engage Americans in NASA's mission--NASA will build strategic partnerships and linkages between STEM formal and informal education providers. Through hands-on, interactive, educational activities, NASA will engage students, educators, families, the general public, and all agency stakeholders to increase America's science and technology literacy. Experience has shown that exciting and compelling NASA missions truly can inspire the next generation of explorers, innovators, and leaders. NASA's unique program content, people, and facilities can be leveraged to spark interest, capture imaginations, and guide students toward careers in STEM fields while increasing their scientific and technologic literacy to the benefit of the Nation. To prepare future generations to manage and lead the cutting-edge research of tomorrow, strategic planning is essential. NASA has identified strategic goals and objectives that align its portfolio of education programs with the Human Capital Initiatives under the President's Management Agenda to build the workforce needed to meet core competences. All of NASA's education efforts are part of an integrated Agency-wide approach to human capital management. NASA Education Programs support multiple goals and sub-goals in the 2006 NASA Strategic Plan. Specifically, the education programs of the Agency contribute to the following outcomes: <bullet> Outcome 1: Contribute to the development of the STEM workforce in disciplines needed to achieve NASA's strategic goals through a portfolio of programs. <bullet> Outcome 2: Attract students and retain them in STEM disciplines through a progression of educational opportunities for students, teachers, and faculty. <bullet> Outcome 3: Build strategic partnerships and linkages between STEM formal and informal education providers that promote STEM literacy and awareness of NASA's mission. NASA Education Programs The manner in which the Agency will achieve these outcomes is detailed in the NASA Education Strategic Coordination Framework. The Framework was approved by the NASA Strategic Management Council in 2006 and guides the planning, implementation, assessment and validation of the following portfolio of programs: The Higher Education Program focuses on supporting institutions of higher education in strengthening their research capabilities and providing opportunities that attract and prepare increasing numbers of students for NASA-related careers. The research conducted by the institutions contributes to the research needs of NASA's Mission Directorates. The Minority University Research and Education Program (MUREP) engages under-represented populations through a wide variety of initiatives. Multi-year grants are awarded to engage minority institutions, faculty and students in research pertinent to NASA missions. The program focuses on retaining under-represented and under-served students in STEM disciplines through completion of undergraduate or graduate degrees and entry into the scientific and technical workforce. The Elementary and Secondary Education Program provides K-12 educators with tools, experiences, and opportunities to further their education and participate in unique NASA learning experiences to enhance their knowledge of STEM and inspire pursuit of STEM careers. The program supports the role of educational institutions, which provide the framework to unite students, families, and educators for educational improvement. Education Technology and Products (e-Education) sustains the research and development of technology applications, products, services and implementation of technology-enriched infrastructure in facilitating the appropriate and effective technology based applications to enhance the educational process for formal and informal education. In addition, e-Education identifies projects that will meet the objective of the President's Management Agenda to provide citizen-centric services related to NASA Education efforts. The Informal Education Program is focused on increasing learning, educating students, educators and the general public on specific STEM content areas, and expanding the Nation's future STEM workforce. Projects within the program produce supplemental educational materials that are standards based and designed to support facilitators who are trained or qualified in STEM education fields, and are actively working with participants to further enhance their understanding. Informal Education Programs also develop content based on educational standards and learning objectives to supplement and enrich an experience, visual, or activity. The breadth of our portfolio, and how these programs have been implemented nationally, can be illustrated through the following examples: <bullet> Attracting students to the teaching profession, the NASA Educator Astronaut project uses the visibility and educational opportunities created by the activities of the Educator Astronauts to inspire greater K-12 STEM achievement, promote STEM careers, and elevate public esteem for the teaching profession. In selecting Educator Astronauts, NASA identified and trained hundreds of our country's top educators who are members of the Network of Educator Astronaut Teachers (NEAT). Approximately 180 NEAT members are now in communities across America, each conducting workshops that reach about 90 educators per session. These efforts result in strengthening the STEM skills of approximately 10,000 teachers annually. <bullet> NASA Explorers Schools (NES) provide intensive training and on-site professional development to teachers in classrooms across the country. The NES project assists middle schools with improving teaching and learning in STEM education through professional development, stipends, grants, and curricular support based on NASA resources. In 2006, 5,339 teachers received intensive training as part of the NES project. Additionally, our Aerospace Education Services personnel conducted sessions across the Nation, reaching 13,938 educators in other schools. <bullet> In addition to in-service workshops based on our missions, NASA is committed to the pre-service training of our future educators. Through the National Pre-Service Teacher Conference, Pre-Service Teacher Institutes and Online Professional Development, NASA recruits STEM teachers to develop the confidence and skills to effectively teach mathematics and science using cutting-edge technology and educational materials. Such efforts have led to 200 STEM- enhanced teachers instructing an average of 25 students per classroom for three years, impacting a projected total of 15,000 students. <bullet> NASA's four Mission Directorates provide opportunities for students to engage in NASA mission related experiences. For example, within NASA's Science Mission Directorate, a broad spectrum of education activities are sponsored, ranging from kindergarten to post-graduate levels. All of NASA's science missions and programs are required to have an education and public outreach component. Through a competitive, peer-review selection process, NASA provides funding dedicated to education and public outreach to researchers. NASA also sponsors graduate and post-doctoral fellowship opportunities. In addition, the Agency is looking for new ways to provide increased opportunities for students to gain greater experience developing and launching their own science instruments, either in conjunction with science missions or through its suborbital rocket and balloon programs. <bullet> Launched in January 2006 as part of the New Horizons Mission, the Student Dust Counter is the first student-built instrument selected by NASA to fly on a planetary mission. Built by students at the University of Colorado at Boulder, the counter will monitor the density of dust grains in space. This data is of particular interest to researchers. Given the nine- year travel time, discoveries from this mission will engage today's elementary school student until college when this spacecraft encounters Pluto. <bullet> Aeronomy of Ice in the Mesosphere (AIM) began its two-year mission on April 25, 2007, after a flawless ride to Earth orbit aboard an Orbital Sciences Pegasus XL rocket. AIM is the first mission dedicated to exploring mysterious ice clouds that dot the edge of space in Earth's polar regions. With AIM, Hampton University in Virginia has become the first Historically Black College and University to lead a NASA satellite mission. Undergraduate and graduate students from various STEM disciplines will have an opportunity to join faculty researchers in the analysis of collected data. <bullet> In February 17, 2007, NASA launched five Time History of Events and Macroscale Interactions during Substorms (THEMIS) microsatellites to study the Earth's magnetosphere. THEMIS will help scientists understand how and why space storms create havoc on satellites, power grids, and communication systems. Students will work with scientists to unravel a variety of scientific mysteries. <bullet> NASA's support of higher education students is embodied by the National Space Grant College and Fellowship Program, which continues to provide fellowships and scholarships to students across the country. Recent statistics show that, of the pool of students who completed their degrees, 31 percent were employed in STEM careers and 48 percent continued their education to the Master's, Ph.D., or post- doctoral levels. Many consortia have implemented hands-on, university student-led projects in aeronautics, rocketry, scientific ballooning, rocketry, and nano- and micro-satellite development. These types of projects provide the professional training that enable students to be fully prepared to enter the STEM workforce. Portfolio Management Process Such a diverse portfolio requires effective management of the Agency's education portfolio both internally and externally, with clear roles and responsibilities. As the Assistant Administrator for Education, I am responsible for ensuring that the Education Outcomes as reflected in the 2006 NASA Strategic Plan are achieved. I serve as both the head of the Office for Education, managing all responsibilities assigned to the Office and also as the Chair of the Education Coordinating Committee (ECC), ensuring the overall planning, coordination, and integration of the Agency's entire education portfolio. NASA's ECC is a collaborative structure that maximizes NASA's ability to maintain an integrated education portfolio and strategically manage the implementation of numerous programs, projects and activities in a distributed system. The committee consists of representatives of the Agency's Office of Education, the four Mission Directorates that provide mission related content, and the ten NASA Center Education Offices, among others. The committee develops education strategy and supports me in coordinating education efforts throughout the Agency. The ECC also provides checks and balances for effective internal control and ensures the successful achievement of education goals and portfolio effectiveness. Collaboration and Coordination with Other Federal Agencies NASA's Office of Education is continually engaged in collaboration with other federal agencies, including: the Department of Education, National Science Foundation, Federal Aviation Administration, Department of Commerce, Smithsonian Institution, Department of the Interior, and Department of Energy. Additionally, NASA collaborates with state STEM education coalitions, through the National Alliance of State Science and Mathematics Coalitions, the District of Columbia, Puerto Rico, and the U.S. territories. Each of our Centers works closely with State and local departments of education to ensure that our resources are tailored to support the needs of the education community. We have worked hard to ensure that we understand and can respond to the needs of State or local districts. Collaboration and coordination also occur in a number of forums in the Federal Government to ensure that NASA's activities in K-16 STEM education are complementary and not redundant with the programs of other federal agencies. Additionally, NASA has actively participated in the Congressionally-mandated Academic Competitiveness Council (ACC), which found there is a dearth of evidence of effective practices and activities in STEM education and made recommendations to integrate and coordinate federal STEM programs. In February 2007, NASA and the National Science Foundation (NSF) signed an historic agreement to work together and coordinate efforts to expand opportunities for promoting STEM education and to broaden the participation of the under-represented in those areas. The Memorandum of Understanding (MOU) between NASA and NSF promotes a comprehensive knowledge base to be shared between the agencies to address national challenges and manage the agencies' resources more effectively. It reflects the goals of the Administration's American Competitiveness Initiative, whose cornerstone is a commitment to increase investments in basic research in the physical sciences and engineering, strengthen K-12 math and science education, and build a well-educated, skilled workforce. One of the first results of the collaboration was a three day joint NASA-NSF Research Education Opportunity Conference for Principal Investigators, Faculty, and Partners. Over three hundred members of the academic community gathered to be trained on ways to strengthen their ability to compete for research grants and to leverage their partnerships with the agencies. Earlier this month, NASA and the Federal Aviation Administration (FAA) signed an MOU to foster the development of students' skills in STEM. The agreement supports the FAA's mission to provide the safest, most efficient airspace system in the world and NASA's mission to pioneer the future in space exploration, scientific discovery and aeronautics research. The partnership includes a broad range of cooperative outreach activities. The agencies' initial focus is on a NASA resource called, ``Smart Skies.'' Smart Skies is an online air traffic control simulator for students in fifth through ninth grades. It offers a fun and exciting way to learn math and skills central to air traffic control while providing multiple modes of problem solving for students who learn in different ways. The agreement unites the strengths of both agencies to provide the best of aviation-related educational products and experiences to the widest possible population of students and educators. Sharing Best Practices Through our work with the ACC, we are strengthening our evaluation methodologies and sharing some of our best practices, for example: The Harriett G. Jenkins Pre-doctoral Fellowship Program (JPFP) is a model of a STEM education pipeline program that can be replicated by other agencies as a best practice. While the success of the JPFP can easily be quantified by counting the number of students participants (121), the number of awards provided to conduct research at a NASA center (90) or the number of successful mentoring relationships that were established through this program (121), the greatest accomplishment of the program is an exceptionally diverse group of under-represented STEM scholars who are excited about pursuing NASA- related advanced degrees that will equip them to participate in the space exploration workforce. To date, the Jenkins project has produced 34 M.S. degrees and 32 Ph.D. degrees in NASA-related disciplines. Another stellar NASA project identified as a best practice is the Science, Engineering, Mathematics, and Aerospace Academy (SEMAA). The Ash Institute for Democratic Governance and Innovation at Harvard University's John F. Kennedy School of Government announced that SEMAA was among the top eighteen programs in the 2007 Innovations in the American Government Awards competition. Selected from a pool of nearly 1,000 applicants, these initiatives are being recognized as the government's best efforts for their novelty and creativity, effectiveness at addressing significant issues, and potential to be replicated by other jurisdictions. Finalists presented before the National Selection Committee at the Kennedy School on May 15, 2007, and winners will be announced in September. Evaluation of NASA Education Programs The Agency's many Education initiatives have not been evaluated in a comprehensive, rigorous manner to indicate how well all of our programs are performing in support of our outcome goals. We are committed, however, to enhancing and improving our evaluation procedures. The Agency has taken several major steps to improve the evaluation function by: (a) incorporating a detailed evaluation plan into its Education Strategy Framework; (b) defining an enhanced set of outcome-based performance measures; articulating specific roles and responsibilities to ensure accountability; and, (c) allocating the resources necessary to support rigorous evaluations and the overall evaluation function. A range of processes will be used to capture the total picture of education across NASA and to assess the education portfolio for its effectiveness in: achieving the stated outcomes; establishing linkages within the framework; and determining the level of quality, impact and comprehensiveness of the portfolio. The ECC will employ an appropriate mix of methodologies, ranging from basic quantitative data to qualitative information, to assess the overall condition of the education portfolio. Coincident with the adoption of a new education framework and outcomes in FY06, NASA developed a corresponding set of objectives and outcome measures. Baselines for these measures are being established with FY07 data. The outcome measures include, but are not limited to the following: <bullet> Percentage of student participants employed by NASA, aerospace contractors, universities, & other educational institutions. <bullet> Percentage of undergraduate students who move on to advanced education in NASA-related disciplines. <bullet> Level of student interest in science and technology careers resulting from elementary and secondary NASA education programs. The most significant improvement NASA is making to its evaluation efforts is to make use of independent, credible evaluators to measure the effectiveness of education investments. Project-level evaluations will be conducted on three to five of our major projects each year, with the objective of evaluating each project at least once every five years. In collaboration with the National Science Foundation and the Office of Management and Budget, we are working to determine the best ways to apply a Randomized Controlled Trial (RCT) model of evaluation to demonstrate the impact of our portfolio of programs. Projects that cannot be reliably evaluated using RCT methods will be evaluated in an objective and credible manner, conforming to the standards of professional practices. Public Awareness and Access to NASA Education Programs NASA Education is a cross-cutting process that engages the public in shaping and sharing the experience of exploration and discovery. The President's FY 2008 budget request for NASA's Education program is $153.7 million. Through the Office of Strategic Communications, the Agency is building and maintaining public awareness for the activities and goals focusing on science, education, aeronautical research and exploration. As part of the Agency's long-term strategy in promoting public awareness, National Education Campaigns designed to build a comprehensive education initiative that engage diverse audiences with tailored modes of interaction have become common practice for assisting the Agency with public engagement and the formation of national and international visibility and recognition. STS-118, the first Space Flight of an Educator Astronaut, is a good example of a National Education Campaign designed not only to engage students and educators but also increase America's science and technology literacy. NASA disseminates its education content including STEM-related materials through resources designed to reach all education audiences-- formal, informal and the public at-large--as well internal dissemination networks such as Aerospace Education Services Program (AESP) and Space Grant. The NASA Portal opens the door to all the resources that NASA has available. From there, educators can either download materials for use, or obtain copies from the Central Operations of Resources for Educators (CORE). CORE is a worldwide distribution center for NASA's educational material. The Role of Partnerships Strategic alliances with non-governmental organizations provide an immediate springboard as unfunded collaborators to produce, market, and distribute educational information about NASA's projects and programs. NASA's partnership with the International Technology Education Association is one of many venues the Agency uses to reach students and educators across the country. Other organizations include the National Science Foundation, National Institute of Aerospace, National Science Teachers Association, AOL's Kids On-Line, the Girl Scouts of the USA, Imaginary Lines, and Reader's Digest. Imagine, with the right partners, what NASA can do to strengthen and support STEM education. Powerful technologies can enable new learning environments using simulations, visualizations, immersive environments, game playing, intelligent tutors and avatars, learner networking, and usable building blocks of content. These capabilities can create rich and compelling learning opportunities that meet the needs of learners while empowering educators to unlock the potential in each student's heart and mind. NASA can unite with the technology and education communities in dialogue, understanding and action. Students and educators can have access to a new renaissance of learning for the benefit of the Nation and the world. Conclusion I would like to commend the Subcommittee for its efforts to improve K-16 STEM education. The educational achievement of America's next generation is an issue that reaches our nation at all levels. NASA will continue to partner with federal, industry, State and local organizations and invest our resources toward a shared vision to secure those jobs critical to the 21st century workforce. This means not only inspiring the next generation of leaders and explorers but also providing educators with unique resources to support educational excellence in STEM while improving scientific literacy. The President, Administrator Griffin, and all of NASA share the belief that a highly educated and well-prepared workforce has been and continues to be essential to this country and the Agency. NASA's investment in education is indeed an investment in America's future. Thank you for the opportunity to participate in this important hearing. I am prepared to respond to any questions you may have. Biography for Joyce L. Winterton Dr. Joyce Leavitt Winterton, NASA's Assistant Administrator for Education, directs the development and implementation of the Agency's education programs that strengthen student involvement and public awareness of its scientific goals and missions. In this role, she leads the agency in inspiring interest in science, technology, engineering, and mathematics, as few other organizations can through its unique mission, workforce, facilities, research and innovations. As Assistant Administrator for Education, Winterton chairs the Education Coordinating Committee, an agency-wide collaborative structure that maximizes NASA's ability to manage and implement its education portfolio. The ECC works to ensure that the Agency's education investments are focused on supporting the Nation's education efforts to develop the skilled workforce necessary to achieve the Agency's goals and objectives. Before coming to NASA, Winterton served as the Director of Education Programs for USA TODAY, and developed educational strategies, resources and partnerships for its K-12 and collegiate programs. During her nine years at USA TODAY, she created innovative cross-curricular educational approaches, including case studies, content development and on-line collaborations. She was the founder and President of Winterton Associates, a consulting firm that specializes in working on joint projects with business and industry, education, and government. The firm has served as the evaluator for National Science Foundation projects and U.S. Department of Education-funded programs, including six national skill standards projects since 1991. Winterton's previous experience includes serving as the team leader for partner development for the National Future Farmers of America student organization, where she planned and developed partnerships and strategies to communicate the benefits of agricultural education and a student organization with over 450,000 members. She has also been an education training consultant for FranklinCovey Inc. where she facilitated time management and personal effectiveness workshops for national student leadership organizations. In 1986, Winterton became the executive director of the National Council on Vocational Technical Education, a Presidential Advisory Council providing recommendations to the President, Congress and the Secretary of Education. Additionally, Winterton served as the deputy assistant secretary for vocational and adult education in the United States Department of Education and was the first Director of the Presidential Academic Fitness Awards program. She also was a professional staff member for the U.S. Senate Committee on Labor and Human Resources. She has served on a number of national education boards and advisory panels. Winterton has been a high school teacher, a teacher educator and a home economist in business. She received the Lawrence Prakken Professional Cooperation award from the International Technology Education Association and was recognized as an outstanding alumna from Colorado State University and also the Family, Career and Community Leaders of America. She earned her Bachelor's and Master's degrees in home economics education from Utah State University in Logan. In 1978, she completed her doctorate in teacher education and administration at Colorado State University in Fort Collins. Mr. McNerney. Thank you, Dr. Winterton. Mr. Valdez. STATEMENT OF MR. WILLIAM J. VALDEZ, DIRECTOR, OFFICE OF WORKFORCE DEVELOPMENT FOR TEACHERS AND SCIENTISTS, OFFICE OF SCIENCE, DEPARTMENT OF ENERGY Mr. Valdez. Thank you. Mr. Chairman and Mr. Ehlers, thank you for inviting me to testify at this important hearing. I have submitted written testimony that makes three points. First, federal S&T mission agencies, such as the Department of Energy, have enormous resources that could be devoted to STEM education and workforce development. Those resources naturally complement what is offered by the Department of Education and the National Science Foundation. We are developing partnerships with other federal agencies, NSF, the Department of Education, and other organizations that have a strong interest in STEM education, as a way to leverage our resources within the Department of Energy. Second, the Office I manage, Workforce Development for Teachers and Scientists, has a great deal of work to do in the area of program evaluation. We are in the process of developing a comprehensive and rigorous evaluation program that builds upon the recommendations of the Academic Competitiveness Council, and what experts told this committee at the May 15 hearing. Third, my Office is engaged in an extensive planning process that is leading to a reprioritization of our programs to meet today's challenges in STEM education and workforce development. We are now developing business plans for all of our programs that describe their goals, resource requirements, and connection to the DOE mission. These three points are discussed in more detail in my written testimony, and I welcome an opportunity to answer any questions you might have about them. But I would like to take a moment to discuss one of the questions that you asked in your May 29 letter of invitation to this hearing. What do you recommend as the most effective role your agency can play in improving STEM literacy? I have had literally dozens of conversations with experts in the field on this question, including most of my colleagues at this table, and just in the past two weeks with Frank Owens of the National Science Teachers Association, Sally Shuler of the National Science Resource Center, and Iris Weiss of Horizon Research, who testified at this hearing on May 15. The conversations I had with these experts, who I consider to be among the thought leaders in STEM education in the United States, have led me to the following observations. All three agreed that DOE programs, which emphasize experiential learning, hands-on opportunities for students and educators, and rigorously designed programs at our National Laboratories, fill a critical void in STEM education and workforce development. Iris Weiss, for example, said that DOE needs to provide an authentic research experience well beyond the brief cookbook experience that students and educators tend to get even at the undergraduate level. Students and educators learn by doing. We provide them with an opportunity at our National Laboratories to learn with some of the best mentor scientists in the world, but, and this is a very big caution, our programs must be properly designed to maximize their effectiveness. This is why we are developing business plans for all of our programs, and will have those plans reviewed by outside experts. Our program design must be open, transparent, and conform with the best standards known to the STEM education community. Iris Weiss asked me if we really would take their suggestions and criticisms to heart, and my answer was, ``Of course.'' All three experts were firm in their belief that we must find a way to sustain our programs, and link them to what matters to educators and students. Frank Owens suggested that DOE partner with NSTA to help develop voluntary national science certification standards for educators. This would take two forms: first, making a structured laboratory research experience part of the voluntary national certification standards that NSTA is developing; and second, utilizing DOE's world-class scientific talent to partner with NSTA to develop their online science content modules. NSTA will propose a partnership structure to accomplish this with the Department of Energy. Iris Weiss suggested that we also work with the Council of State Science Supervisors and other groups, which we will do. Finally, every expert we have spoken with has said that we must carefully evaluate our programs. Iris Weiss and Sally Shuler, who really know this business inside and out, agree that we currently are only able to measure improvements to content knowledge and interest in pursuing a science career after participation in the kinds of experiential learning programs that we manage, but we currently cannot measure whether those increases lead to improved test scores and an ability to perform science. I am very interested in talking with NSF and my colleagues at the table and other experts about ways to fill these gaps in knowledge and improve our program evaluation. Overall, all three experts agree that federal S&T agencies need to do a better job of talking with one another, sharing best practices, and leveraging resources. They expressed enormous frustration that we don't have a ``one-stop shopping'' resource for programs, evaluation techniques, and outreach. Even the simple things, such as a common application for K-12 educators who want to enter into a research experience, would be a big help, they said. The series of hearings that this committee is holding should help in that regard. I would like to thank you, Mr. Chairman, and other Members of the Committee, for investing your time and energy into studying this challenge. I look forward to working with you and your staff, and answering any questions you might have. [The prepared statement of Mr. Valdez follows:] Prepared Statement of William J. Valdez Thank you, Mr. Chairman and Members of the Subcommittee, for this opportunity to discuss the role that the Department of Energy's (DOE) Office of Science plays in scientific and technical workforce development and education. We appreciate your strong commitment to improving science and math education and training in the United States. The Office of Science is the Federal Government's largest supporter of civilian basic research in the physical sciences. This basic research supports the Department's missions in energy, the environment, and national security. The Office of Science manages 10 national laboratories and more than 30 major scientific user facilities that provide the scientific community with state-of-the-art research tools that help accomplish the Department's goals and maintain U.S. competitiveness in science and technology. The Department's most significant contribution to the development of a scientific and technical workforce has been through the support of graduate students pursuing advanced degrees, post-doctoral students who work on research projects, and, to a much smaller degree, hands-on research opportunities for undergraduate students and K-12 educators and informal experiential learning opportunities for K-12 students. These individuals utilize DOE research facilities and work side-by-side with the scientific and technical staff at the national laboratories. Those national laboratories are unique settings for research, mentoring, and collaboration. Through structured and unstructured workforce development and science education programs at DOE's 17 national laboratories, the power authorities and other DOE facilities, the Department engages with more than 250,000 students of all ages and 19,000 K-12 educators on an annual basis. The Office of Workforce Development for Teachers and Scientists (WDTS), which I manage, is the only program office in DOE that has a specific mission in science, technology, engineering and mathematics (STEM) workforce development and education. Our programs reach 600 undergraduate students, 16,000 K-12 students, and 150 K-12 educators annually. We do this with an $8 million annual budget and under specific statutory authority (Public Laws 93-438 and 101-510, and most recently the Energy Policy Act of 2005). The 17 DOE national laboratories, the power authorities and other DOE facilities use WDTS funding as ``seed money'' to develop complementary programs that are designed to meet their local needs. Our programmatic philosophy is ``nationally designed programs, locally delivered.'' This model has relied on partnerships within the Department of Energy and with external organizations. While WDTS directly funds 600 undergraduate students for summer internships, the total number of undergraduate research interns at all of the DOE laboratories is 4,100. Similarly, WDTS directly funds 150 K- 12 educators, but a total of 19,300 K-12 educators are involved in programs at DOE laboratories and facilities. Partnerships enable WDTS to coordinate with and leverage the resources and capacity of the Office of Science (SC). SC works with more than 300 of the top universities in the Nation, manages 10 of the biggest national laboratories in the Federal Government, and deals directly with hundreds of high technology companies. In recognition of widespread concern about STEM workforce development, the Secretary of Energy, in 2006, commissioned a review by the Secretary of Energy Advisory Board of the Department's activities in STEM education. That review concluded that DOE has a clear role in STEM education and that partnerships are the primary vehicle we should use to achieve our goals. The Board stated: ``[A] review of the Department's educational programs as well as a review of the educational efforts in other federal agencies, leads us to our conclusion that DOE has a significant opportunity to enhance STEM education in the Nation. Moreover, it is clear from our review (as well as from the GAO reports) that the educational activities of DOE and other federal agencies could benefit from increased cooperative activities with one another, with industry, with colleges and universities, and with science teachers' professional organizations. In both nationwide influence and in cooperative partnerships, DOE is already positioned to take a leadership role. DOE's national laboratories are geographically distributed over the country, allowing access to teachers across the Nation. Moreover, the network of national laboratories is also tightly linked with industrial and academic resources, giving DOE the ability to forge educational partnerships that can extend its reach, and therefore also its capacity to enhance STEM education nationwide.'' As a result of its stakeholder meetings and other outreach efforts, WDTS has had discussions with a wide range of organizations proposing partnerships. Let me give you four examples of partnerships that could make our programs more effective: <bullet> WDTS has an existing partnership with the National Science Foundation (NSF) that illustrates how federal resources can be effectively leveraged. DOE has the 17 national laboratories, but NSF has greater access to undergraduate and educator populations. Our agreement with NSF enables us to share programs, with a result that in FY 2006 NSF supported 195 educators and students at seven of our national laboratories. (Table 1) <GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT> This is a beginning, but we could do more. DOE mentor scientists who participate in Office of Science and other DOE programs have a long history of working with students, and many have indicated they are eager to expand their efforts. One resource that could help is the federal laboratory system. The Federal Government owns more than 250 national laboratories across the Nation, and many of these have STEM workforce needs similar to those of DOE. Thus, WDTS could partner with USDA labs, for example, to prepare the future workforce to support the expanding bio-fuels industry, or with Department of Defense laboratories to develop our national security workforce. <bullet> WDTS is engaged in extensive discussions with the Department of Education on better support for the Administration's Adjunct Teacher Corps initiative. The mentor scientists at DOE's national laboratories could constitute a promising potential core of the Adjunct Teacher Corps. Sandia National Laboratories, which is part of the National Nuclear Security Administration within DOE, is taking a leadership role with WDTS to structure a program that would enable us to work with the Department of Education to achieve the Administration's goal of placing 30,000 adjunct teachers in the Nation's classrooms by 2015. If Sandia's pilot program with the Department of Education is successful, the concept could be expanded to other federal agencies with national laboratories and pools of mentor scientists. Secretary of Education Margaret Spellings on May 9, 2007 commented on this emerging partnership: ``When I was in Senator [Jeff] Bingaman's state of New Mexico I visited a local high school where scientists from Sandia Labs were teaching chemistry. We need to make this the norm around the country.'' <bullet> The core element of WDTS's programs and other programs carried out at the DOE laboratories is providing educators and students with hands-on research experience. These research experiences supplement what students learn in the classroom and help educators better understand the process of science. Thus, we want to partner with organizations like universities and corporate laboratories that have similar infrastructure to that of the DOE national laboratories. As a start, we have entered into discussions with a university and a major non-profit science educational group in Boston about pilot programs that would share resources and capabilities. <bullet> WDTS is developing what we are calling a ``trusted partners'' approach to reach under-represented populations. Students and educators tend to learn about our programs primarily through recommendations from individuals and organizations whose opinions the students and educators themselves trust. This is particularly true of students and educators from under-represented populations who have not built a trust relationship with the Department of Energy. As a result, we are exploring partnerships with several national organizations to help identify their most promising students and educators for our programs. We have had discussions, for example, with a major Hispanic communications network about developing innovative approaches that reach the best and brightest Hispanic students and teachers for our programs. Evaluation I would now like to turn to the need for evaluation and intelligent program design. In this regard, I would like to commend the Department of Education and the Office of Management and Budget (OMB) for the work they have done through the Academic Competitiveness Council (ACC). The catalogue of existing STEM education programs in the Federal Government and the emphasis the ACC Report places on the need for rigorous evaluation catalyzed a discussion in Washington, D.C. policy circles about the need for rigorous evaluation of STEM education and workforce programs. The ACC Report's recommendations were influential in the development of WDTS's future direction. This was a discussion that was much needed because, frankly, WDTS has done a poor job over the past 10 years of rigorously evaluating our programs. As a result, and under the specific direction of Under Secretary for Science Raymond Orbach, we are committed to improving our ability to evaluate the impact and effectiveness of our programs. We have data that indicate our experiential learning programs are yielding good results (i.e., promoting interest in STEM fields). We are in the process of developing a plan for more rigorous study of the program that will enable us to demonstrate the program's impact. Based on the results, we will be able to refine the program and pursue the most effective strategies going forward. One lesson that was reinforced by the ACC process is that evaluation and assessment are crucial to the effective design of STEM workforce programs. For example, during the 1980s and 1990s, the Department funded rigorous longitudinal workforce studies that enabled program managers to identify specific future workforce needs. WDTS is in the process of re-invigorating that effort and within the next 12 months will have completed a pilot workforce study that identifies the workforce needs, by scientific discipline, for the Office of Science federal and national laboratory staff. This workforce study is being done in collaboration with the National Nuclear Security Administration (NNSA). By including the 10 national laboratories managed by the Office of Science and the three defense national laboratories managed by NNSA (Lawrence Livermore National Laboratory, Sandia National Laboratories, and Los Alamos National Laboratory) in the study, we will account for the majority of the R&D performed by the Department. In future years, we hope to include other DOE laboratories and R&D programs in this effort. Identifying gaps Rigorous evaluation of programs and the use of workforce data from the analysis that we will do over the next 12 months will enable WDTS to identify opportunities to improve our STEM workforce development and education efforts. WDTS currently manages nine programs for students and educators. Those programs emphasize experiential learning opportunities for students and educators, such as the Academies Creating Teacher Scientists (ACTS) program; and world class celebrations of scientific achievement for students, such as the National Science Bowl. We have two decades of experience managing these types of programs and believe that they are effective and are contributing to our nation's efforts to improve STEM education and proficiency, although more rigorous evaluations are in order. But conversations with our stakeholders and our own internal analysis have revealed that there are opportunities for our programs to better achieve their objectives. Let me give you two examples: <bullet> One potential gap is in the development of talent in our federal workforce. While one of our goals is to encourage students to join federal service, we do not have programs in place that provide a clear link for them to seek employment with the Department, such as helping them navigate the difficult federal hiring process. In addition, we do not have the workforce assessment tools in place that would inform us about whether we need more physicists, chemists, or engineers. The workforce assessment we are doing will help in that regard. We are also working with DOE's Chief Human Capital Officer, Dr. Jeff Pon, to develop programs specifically targeted at the federal STEM workforce. <bullet> Another area for improvement is our collective need to better align agency STEM efforts with larger federal mission needs. Representatives from various federal agencies have emphasized the need to work collaboratively to solve our mutual STEM education and workforce challenges. One result is that my colleague, Dr. Joyce Winterton of NASA, has taken the initiative to form a brown bag lunch group of federal science and technology agencies as a forum for discussion and collaboration. I am also talking with the Federal Laboratory Consortium, the Triangle Coalition and a host of other groups about partnerships designed to bring federal agencies together with the educational community and industry. We need to work together and, in fact, a grassroots process supported by federal S&T agencies has already begun. Conclusion: I would like to conclude by highlighting several statistics: <bullet> $135 billion--the annual federal investment in R&D that is managed by 34 agencies <bullet> 257--federal laboratories that belong to the Federal Laboratory Consortium and are active in communities nationwide <bullet> 206,000--federal scientists and engineers (not including contractors) When all of these numbers are put together, it is evident that we have excellent resources for a coordinated federal response to the Nation's STEM education challenge. The 34 federal R&D mission agencies--such as NASA, NOAA, DOD, NIH, USDA and EPA--have a long-term and enduring interest in their workforces and STEM education. This has been a strong federal resource that can continue to support our efforts to address the national challenge of educating the future U.S. workforce and helping to prepare our citizens for the emerging era of scientific discovery and innovation. Thank you for offering me this opportunity to provide a perspective on this important issue. I look forward to answering the Committee's questions. Biography for William J. Valdez Bill Valdez is the Director of the Office of Workforce Development for Teachers and Scientists within the Department of Energy's Office of Science. His responsibilities include developing workforce strategies for the Department's scientific and technical workforce, and creating opportunities for students and educators to participate in the Nation's research enterprise as a means to improving the competitiveness of U.S. industry and overall scientific literacy. In addition, Mr. Valdez has been leading an interagency effort, coordinated by the White House Office of Science and Technology Policy, that is designed to establish credible outcome measures for basic research, create new evaluation methods that focus on systems level analysis, and promote business models that will enable federal R&D managers to improve investment decisions. Previously, Mr. Valdez was the Director of Planning and Analysis at the Department of Energy's Office of Science. His responsibilities included corporate strategic planning, R&D evaluation, and federal S&T policy development. Mr. Valdez was elected as a Fellow of the American Association for the Advancement of Science in 2006 and is Vice Chair of the Senior Executive Association's Board of Directors. He was elected to the Board of Directors of the Senior Executive Association in 2005. Mr. Valdez has held various positions at the Department of Energy since 1994, including serving as Executive Director of the DOE R&D Council and developing evaluation techniques for technology transfer programs. Mr. Valdez also served at the White House Office of Science and Technology Policy from 1998-99. His responsibilities included co- authoring a report on strategies designed to improve the future scientific workforce as the Nation's demographics change, developing interagency technology initiatives, and advising on international energy initiatives. Prior to working at DOE, Mr. Valdez worked as a Senior Project Manager in private industry where he provided strategic planning services to Asian and European multi-national corporations. Mr. Valdez received a Bachelor of Arts from the University of Texas and his Master of Arts in International Economics and Energy Policy from the Johns Hopkins School of Advanced International Studies. Mr. McNerney. Thank you, Mr. Valdez. Dr. Fuchs. STATEMENT OF DR. BRUCE A. FUCHS, DIRECTOR, OFFICE OF SCIENCE EDUCATION, NATIONAL INSTITUTES OF HEALTH Dr. Fuchs. Thank you, Chairman McNerney and Congressman Ehlers. I want to thank you for this invitation to appear before you to discuss some of NIH's STEM programs. It is an honor for me to appear before this subcommittee that has worked so hard to improve STEM education in this nation. I will apologize for the PowerPoint. I am an old professor, and old habits are hard to break. I would like to briefly discuss some of the things that I believe federal agencies can do to help with STEM education in this country. First, we can partner with outside agencies and experts to design exemplary model programs for a variety of things, instructional materials, teacher professional development, and then rigorously evaluate those programs. It is not that every agency should get involved in each activity, but I think every agency can select something to do well. We need to know what works, what doesn't, and most importantly, why. This kind of in-depth design and evaluation research is unlikely to come either from states or private industry working alone. Second, I believe that the responsible federal agencies, and this is primarily NSF, Department of Education, and NIH through its National Institute on Child Health and Development, need to support high-quality, scientifically based education research. Unfortunately, we have scientific evidence to support only a small number of items related to math and science teaching that we didn't know 25 years ago. We must not be in the same situation 25 years from now. Lastly, I would like to suggest that federal scientists, because of their many interactions with students and scientists from around the world, have an important insight into what it takes to compete in today's world. This insight should be used to help define the world-class standards to which our schools must aspire. Sadly, many of our states' science and math standards cannot presently be considered to be world-class. I would like to tell you briefly about two programs that we have at the NIH, and illustrate how they are working with educators in the states. The first is a grants program known as the Science Education Partnership Award. This is a peer- reviewed program located within NIH's National Center for Research Resources. These grants are used to establish partnerships within a community to enhance the teaching of science within that community. For example, a partnership might be developed between a university and a science museum, or a university and a school district. There are currently 70 active SEPA grants in 39 states, a number of which focus on underserved populations within those communities. The second program I would like to highlight is the NIH Curriculum Supplement Series. I must say that I agree with the testimony that you received last month relating to how difficult it is to develop high-quality instructional materials. This is a time-consuming and an expensive process. I believe that we have avoided the pitfalls described by forming appropriate partnerships. In short, we know what we know, and we know what we don't know. We have been able to combine the scientific insights of some of the world's leading scientists, people like Dr. Anthony Fauci and Dr. Francis Collins, with the professional expertise of some of the most highly respected curriculum development organizations in this country. We are currently engaged in aligning our supplements to the science, math, health, and language arts standards for each state. We have included, in an online appendix, a sample of alignments for a number of sample states, and I have included for Committee Members some examples of curriculum supplements for you to review. We believe that this project is one way that we can bring some of the excitement, hope, and promise of NIH research to schools around the Nation. But they won't help if they don't get out there, so I would like to close with a brief discussion of dissemination. We have worked very hard to let educators know about the availability of our now 16 different curriculum supplements aimed at elementary, middle, and high school. At the end of May, we had had requests from more than 70,000 teachers from across the country, for almost 285,000 supplement titles. In the online appendix, I have included maps showing the number and location of these requests for some sample states. I think if you, each of you, would look at your home state, as I do, you can look simply from the requests, and identify various towns. I don't have time to show you all these state maps, but I can give you, show you a national map that gives you a sense of this. This map places one blue dot in each zip code from which we have received one or more requests for an NIH curriculum supplement. Each blue dot could represent one curriculum supplement or hundreds, but the map does give you a good sense of where the orders have come from. The next map is actually a NOAA satellite image of the U.S. at night, with lights showing approximately the population distribution across the U.S. Now, if I can successfully toggle this back and forth, I will give you some sense of how well we have done at finding people where they live and connecting them with our curriculum supplement project. Thank you very much for the opportunity to discuss a few of our STEM programs. I will be happy to answer any of the questions that the Committee Members might have. [The prepared statement of Dr. Fuchs follows:] Prepared Statement of Bruce A. Fuchs Chairman Baird and Members of the Committee, it is a privilege to accept your invitation to participate in this hearing and provide you with information about STEM education efforts at the National Institutes of Health (NIH), an agency of the Department of Health and Human Services. The mission of the NIH is to uncover new knowledge that will lead to better health for everyone. NIH has long been involved in directing programs for the collection, dissemination, and exchange of information in medicine and health, including the development and support of medical libraries and the training of medical librarians and other health information specialists. In 1991, the NIH formed an Office of Science Education Policy (now the Office of Science Education under the Office of Science Policy) in the Office of the Director because of concerns surrounding the state of science education in the Nation. The NIH Office of Science Education (OSE) coordinates a program to strengthen and enhance efforts of the NIH to attract young people to biomedical and behavioral science careers and to improve science literacy in both adults and children. The function of the OSE is to: 1) develop, support, and direct program activities at all levels, with special emphasis on targeting students in grades kindergarten to 16, their educators and parents, and the general public; 2) advise NIH leadership on science education issues; 3) examine and evaluate research and emerging trends in science education and literacy for policy-making; 4) work closely with the NIH extramural, intramural, women's health, laboratory animal research, and minority program offices on science education special issues and programs to ensure coordination of NIH efforts; 5) work with NIH Institutes and Centers to enhance communication of science education activities; and 6) work cooperatively with other public- and private-sector organizations to develop and coordinate activities. NIH contributes to K-16 STEM education in three main ways: 1) by partnering with educators on high-quality model programs to create instructional materials, conduct teacher professional development, and support informal science education in museums and science centers; 2) by conducting rigorous research into science and mathematics learning and teaching through NIH's National Institute of Child Health and Human Development (NICHD); and 3) because of its interactions with scientists and students from around the world, by helping to understand the ``world class standards'' our students will need to compete in today's world. 1a. What steps has your agency taken to improve its coordination with other federal agencies' STEM education activities? NIH was actively engaged in the Academic Competitiveness Council (ACC) deliberations. NIH Director Elias Zerhouni joined the ACC at the invitation of Secretary Spellings and made clear his support for the process. NIH participated in all three ACC working groups: K-12, Graduate/Postgraduate, and Outreach and Informal Education. Dr. Zerhouni has also committed NIH to a leadership role on the new National Science and Technology Council (NSTC) Subcommittee on STEM Education that will follow through on the ACC recommendations. Dr. Zerhouni appointed Dr. Duane Alexander, NICHD Director, to serve as one of the co-chairs of the subcommittee, along with Dr. Cora Marrett, National Science Foundation, and Dr. Russ Whitehurst, Department of Education. Additionally, the NIH is in discussions with the Department of Defense and the National Aeronautics and Space Administration (NASA) about ways to extend the ACC database (ACC Recommendation #1). By expanding the program database to include project-level information, federal program managers with shared interests (e.g., teacher professional development) would be able to find one another in order to share information. 1b. To what extent does your agency collaborate with educators in the states and school districts in developing STEM education programs? Some of the resources that teachers request from the NIH were not originally targeted for classroom use. Most of the large number of publications created by NIH are directed to specific health conditions or are directed at specific audiences, such as patients, family members, and health care professionals. However, once these publications were discovered by science teachers, they began to be requested for use in classrooms as well. (Two popular examples are of this type of publication are Inside the Cell, available at http:// publications.nigms.nih.gov/insidethecell/, and Understanding the Immune System, at http://www.niaid.nih.gov/publications/immune/ the<INF>-</INF>immune<INF>-</INF>system.pdf.) However, today the majority of NIH programs and resources requested by educators were created expressly for, and with, teachers. Below are two examples of NIH resources currently available to science educators. First, the NIH National Center for Research Resources Science Education Partnership Award (SEPA) Grant Program is the largest single K-12 (and informal) science education program at NIH. SEPA's goals are: 1) to stimulate career opportunities in basic science and clinical research by providing inquiry-based curricula to K-12 students, teachers, and parents; and 2) through SEPA projects at science centers and museums, to increase the public's understanding of NIH-funded medical research and to provide information about healthy life style choices. Because these awards are made to a community organization, the projects can be specifically designed to meet the needs of that community. For examples of SEPA-funded projects in selected States, see Appendix A at http://science.education.nih.gov/HSTC, and for additional information about SEPA, see http://www.ncrr.nih.gov/ science<INF>-</INF>education<INF>-</INF>partnership<INF>-</INF>awards/. Second, OSE has collaborated with a number of NIH Institutes and Centers to create a series of free curriculum supplements (currently 16 titles) for science educators (available at http:// science.education.nih.gov/supplements). Teachers have input into the development, writing, and editing of each supplement. The supplements are field-tested by teachers across the Nation and modified to address their concerns before being released to the public. Our state-level collaborations have included working with state departments of education and state-wide education advocacy groups (in New York, North Carolina, Ohio, Missouri, and Tennessee) to determine whether a supplement meets a need in the state-wide science curriculum and to help with state health education standards development. The North Carolina Department of Public Instruction has recommended one of the NIH supplements as a primary resource for their eighth-grade science teachers since 2005. (http://www.dpi.state.nc.us/docs/curriculum/ science/middlegrades/8thsciencesupport.pdf.) 2. The recent report of the Academic Competitiveness Council reinforces the need for better evaluation and performance metrics for federal STEM education programs. What plans does your agency have for improvements in its evaluation of its STEM programs? NIH supports the ACC goal of conducting increasingly rigorous evaluations of its STEM education activities using multiple evaluation strategies. These strategies will include working toward conducting randomized controlled trials where appropriate. NIH has agreed to align its goals and metrics to those defined through the ACC process. The first NIH-wide meeting of K-12 project directors was held in April 2007 to discuss this alignment as well as ways to begin collaborating on increasingly rigorous evaluations. The science education grants programs at NIH (for example, SEPA grants) are currently considering changes in their funding opportunity announcements to require increasingly rigorous project evaluations. While this process will take some time, OSE is committed to helping the community (extramural grantees and NIH intramural project managers) solve the problems it may encounter on the road to rigorous evaluations. 3. The Subcommittee received testimony at a hearing on 15 May on how the R&D mission agencies could improve the effectiveness of their STEM education programs. The witnesses were skeptical of the ability of the agencies to develop curricular materials for formal classroom instruction and questioned the effectiveness of their teacher professional development programs to improve teacher classroom performance, while suggesting that the agencies' most important role is in informal STEM education. The witnesses also strongly recommend closer collaboration by the agencies with educators in the field when developing STEM programs. What is your response to the recommendations from these witnesses? Several witnesses expressed concerns at the May 15 hearing regarding the potential pitfalls related to developing curricular materials for formal classroom instruction. Mr. Michael Lach made comments about the problems of ``adding more topics to cover'' and of parochial projects' being ``harder to connect to our work'' in terms of curriculum materials. Dr. Nelson noted, ``There is a huge inventory of poorly designed and under-evaluated mission-related curricula (posters and lesson plans and associated professional development) rarely used in classrooms and with no natural home in a coherent standards-based curriculum. Effective curriculum development requires a deep collaboration with a team of professional curriculum developers, education researchers, and classroom teachers.'' We could not agree more. Development of high-quality instructional materials is a difficult, time-consuming, and expensive undertaking. It is true that well-meaning scientists have unwittingly added to Dr. Nelson's ``huge inventory of poorly designed and under-evaluated'' curricula. We believe that OSE has avoided these pitfalls by proceeding slowly, doing our homework, understanding where we have expertise, and, most importantly, understanding where we do not have expertise. Before starting the NIH Curriculum Supplements Series, we conducted nearly two years of research, discussions, and interviews with leading curriculum developers across the U.S. We also conducted focus groups with educators at a number of conferences around the Nation to determine whether there was interest in having NIH create supplemental materials for the classroom. (There was.) We discussed with educators the topic areas where they felt they needed help and how these might be fit into biology courses. Interestingly, the teachers also strongly warned us ``not to let our scientists write the curricula,'' advice that we took to heart. When considering what NIH can bring to the creation of supplemental instructional materials, it is important to note that our employees include some of the world's leading scientific minds. Dr. Anthony Fauci, Director of the National Institute of Allergy and Infectious Diseases, and Dr. Francis Collins, Director of the National Human Genome Research Institute, are only two such individuals who have contributed their scientific understanding and foresight to the NIH Curriculum Supplements Series. However, while NIH has this kind of scientific expertise in abundance, we do not have in-house expertise in instructional materials development. Instead, we have sought out professional curriculum development organizations that are as well known and respected in their field of expertise as NIH is in its own. We have contracted with BSCS (Biological Sciences Curriculum Study) and EDC (Educational Development Corporation), two of the most highly respected science instructional materials developers in the Nation. Both of these organizations rely on research into how children learn science, use professional curriculum developers, and depend on classroom teachers as advisors, writers, and field-testers. Both of these organizations trace their genesis back to the early post-Sputnik days and have established long track records of creating innovative and effective curricula. NIH curriculum supplements were designed from the start to align with the National Science Education Standards (NSES). Most States have used the NSES to create their own standards documents. Since implementation of the No Child Left Behind Act, alignment to the NSES is no longer sufficient. As a result, we are undertaking the task of aligning each of our 16 curriculum supplement titles to each state's science, mathematics, health, and language arts standards (34 States and the District of Columbia are done so far; see Appendix B at http:// science.education.nih.gov/HSTC for samples of alignments of one supplement to selected States). When this project is complete, we will be able to demonstrate for each state how a specific NIH curriculum supplement directly addresses the science and cross-curricular content standards that educators are expected to cover. Many educators have reported being especially excited to receive materials that can transmit some of the thrill and sense of discovery arising from the latest NIH research as a way to inspire and motivate their students. Each supplement provides activities for students to investigate science content knowledge they can apply directly to some aspect of their daily lives. The fact that the materials cover the biological concepts that teachers are required to cover but do it through references to human health and disease is seen as a strong positive. For instance, in general, children do not get very excited by studying onion root tips. It is far more engaging to study the mechanisms that control cell growth by relating it to a human disease like cancer. In creating these instructional materials, we were also motivated by the fact that research into the poor performance of our students in international comparisons has concluded that curricula in the U.S. are ``a mile wide and an inch deep'' and that content is often years out of date. The American Association for the Advancement of Science Project 2061 has evaluated many middle and high school science textbooks and found all of them wanting. None of the 10 evaluated high school biology textbooks received even a ``good'' rating. We were convinced that teachers would benefit from free, accurate, interesting, standards- based instructional materials that incorporate the latest research into how people learn, so we developed curriculum supplements that allow students to think like present-day researchers and engage in practical applications. The extent to which we have created a curriculum series that is of interest to educators is indicated by the fact that as of late May 2007, more than 70,000 educators have requested almost 285,000 supplements across the Nation. We would like to emphasize that each of these supplements has been shipped out in response to a specific request for that title coming in from an educator. In other words, each of these requests is a record of a positive action taken by an educator to come to our website, fill out a post card, send us an e-mail, etc. (See Appendix C at http://science.education.nih.gov/HSTC for distribution maps showing how many, and from where, requests have come for selected States.) We are also proud to report that NIH curriculum supplements are frequently used as exemplary instructional resources by university- based professors engaged in teaching future science teachers in ``methods'' courses. The middle school supplement Doing Science: The Process of Scientific Inquiry has been especially well received by this audience. To our knowledge, very few other entities have created educational materials that are deemed so useful that they are requested both by STEM teachers and by the university-based professors who train them. We created the NIH curriculum supplements as models for how challenging content can be combined with engaging, realistic situations to give students the opportunity to think like scientists. For a report on how the instructional model underpinning the NIH curriculum supplements aligns with current research into how people learn, see Appendix D at http://science.education.nih.gov/HSTC. Last, although we share an enthusiasm for informal science education, we are concerned by the inequities that would result if it were our only approach. Many educators, particularly those in small, rural, or impoverished urban school districts, cannot afford a field trip to a science center or museum, nor is every school district within driving distance of a museum, major university, or federal laboratory installation. We must not forget those teachers and students who cannot, for financial or other reasons, travel to a wonderful science museum, or have a scientist visit the classroom. These teachers and students also deserve to have access to high-quality science experiences. I would like to illustrate this last point with a personal anecdote. I was one of those rare individuals who knew from early childhood that he wanted to be a scientist. Undoubtedly, this was due, at least in part, to the post-Sputnik efforts that allowed my parents to order me pictures of astronauts, rockets, and stars from NASA. However, living where I did in central Illinois, I was a senior in high school before I got to meet my first working scientist--after a four- hour bus ride to Argonne National Laboratory. We have designed the NIH curriculum supplements to bring some of the excitement, promise, and hope of NIH research to any school--urban or rural, rich or poor, with the best laboratory facilities or none at all. 4. How does your agency determine priorities for its K-16 STEM education portfolio? Has your agency's balance of programs at graduate/ post doctoral, undergraduate, K-12, and informal education changed much over the past few years? Do you foresee a change in that balance in the future? Approximately 95 percent of the education activities (in dollar terms) that NIH submitted to the ACC inventory fell into the ``Graduate/Postgraduate'' category. NIH has no plans to change that balance. In the future, this priority setting will be more formal and coordinated. As previously mentioned, NIH has agreed to align its goals and metrics to those defined through the ACC process. The first NIH- wide meeting of K-12 project directors was held in April 2007 to discuss this alignment as well as ways to begin collaborating on increasingly rigorous evaluations. 5. How does your agency disseminate information about its STEM education programs? What organizations, both government and private, have you partnered with to reach educators in the field? OSE has created a web site specifically designed to help educators find NIH resources that meet their needs (http:// science.education.nih.gov). We also responded to input from teachers regarding the ways that they search for materials (for example, by topic, by grade level, by resource format). As OSE identifies new NIH resources, it codes them using this scheme to facilitate easy retrieval by teachers. Shortly before our last web site redesign, we began using the evaluation services of the American Consumer Satisfaction Index (ACSI), which publishes an e-government Satisfaction Index. ACSI is a cross- industry measure of consumers' satisfaction. It measures the performance of over 200 private-sector companies as well as many government agencies, using scores calculated on data gathered from voluntary online surveys of randomly selected site visitors. For the past few years, the OSE web site has been one of the top 10 sites in the entire government in terms of customer satisfaction. Since March 2000, site traffic has increased from 17,000 visitor sessions per month to well over 250,000 visitor sessions per month. Web pages viewed each month have increased over the same time period from 36,000 to almost 2.5 million. For the past two years, OSE coordinated an ``NIH Research Zone'' at National Science Teachers Association (NSTA) national conferences. This year, eight NIH Institutes and Centers joined OSE, along with the Society for Neuroscience and other organizations. This effort has been greatly appreciated by the NSTA members. NSTA President Linda Froschauer cited it as a good example of how NSTA benefits from interactions with federal agencies in her May 15, 2007, testimony before this committee. OSE also attends the National Association of Biology Teachers meetings, the National Middle School Association meetings, and, on occasion, state meetings of science teachers. Thank you for this opportunity to discuss NIH's STEM education efforts with you. I will be happy to answer any questions you may have. Biography for Bruce A. Fuchs Dr. Bruce A. Fuchs is currently the Director of the National Institutes of Health's (NIH) Office of Science Education (OSE). Dr. Fuchs is responsible for monitoring a range of science education policy issues and providing advice to NIH leadership. He also directs the creation of a series of K-12 science education curriculum supplements that highlight the medical research findings of the NIH. The NIH Curriculum Supplement Series is designed to meet teacher's educational goals as outlined in the National Science Education Standards and is available free to teachers across the Nation. The office also actively creates innovative science and career education web resources, such as the LifeWorks career exploration site, accessible to teachers and students across the Nation. These resources are available at http:// science.education.nih.gov. Dr. Fuchs is serving on the Education and Workforce Development Working Group of the National Science and Technology Council and on working groups of the Department of Education's Academic Competitiveness Council. He was a member of the K-12 education focus group for the National Academy of Science's report Rising Above the Gathering Storm, which was utilized in the Administration's development of the American Competitiveness Initiative, which President Bush introduced in his 2006 State of the Union address. In 2005, the Department of Education asked Dr. Fuchs to serve as the U.S. representative to the Asian Pacific Economic Cooperation meeting on Best Practices in Math and Science Education. For a number of years, Dr. Fuchs was the NIH representative to the Department of Education's National Education Research Policy and Priorities Board. That experience led to his continuing interest in the debate over how to make educational research more effective. Before coming to NIH, Dr. Fuchs--an immunologist who did research on the interaction between the brain and the immune system--was a researcher and teacher on the faculty of the Medical College of Virginia. He had grant support from both NIH's National Institute of Mental Health and the National Institute on Drug Abuse. He has a B.S. in Biology from the University of Illinois and a Ph.D. in Immunology from Indiana State University. He was born and raised in Springfield, Illinois. Discussion Mr. McNerney. Thank you, Dr. Fuchs. And I want to thank all the witnesses for making the effort. I know it is a challenge to come over here and prepare yourselves and all, and sit in front of this committee, so it is very good testimony. At this point, we will open our first round of questions, and the Chair recognizes himself for approximately five minutes, or maybe a little bit longer. I think the purpose of this hearing is really to give this committee comfort in what is about to happen in STEM education, and I see it as this committee's--that is about to be formed, or is already formed--responsibility to make sure that this is executed in a way that takes advantage of the resources that are available. I think it is a large responsibility, so I applaud, again, all of you for wanting to participate. I see Dr. Marrett as the leader of this effort, rightly or wrongly, and so I am going to be picking on you a little bit more than the others perhaps, but there is no malicious intent here. Will the subcommittee be developing a strategic plan for federal STEM education programs to help guide the priorities of the agencies? Is that something you see a part of your charter? Dr. Marrett. Yes, I do. This is an important question, because we are, at this point, developing the charter for the committee. And as I see it, yes, there has got to be, we must undertake some activities that will strengthen, enhance what is taking place with agencies, that will recognize some of the distinctiveness, but will also work towards the level of coordination that is going to be expected. When you ask, then, should there be some impact, some sense of what the impact should be, we will certainly be looking for how to think about the entire enterprise, and what is the role of the individual agencies within that? I am saying this, expressing to you our desire to have any suggestions, any questions and ideas, as we develop both the charter and the set of activities. Mr. McNerney. Well, will you be developing an annual plan, then, or some sort of an organization chart that we can look at? I know that prior to this meeting, the ACC report had catalogued the way that money is being spent in K-12 education. That is very important, I think that is, in some sense, the relatively easy part of this task. We can find out where we are through a certain amount of research, and we can determine what our plans are. The real hard part is evaluating what we are going to get once we start doing it, and I was appreciating Dr. Winterton's discussion of how some programs had worked, and they were exciting, and the kids got involved. It would be nice to have that sort of thing, not codified, but in some sort of description that people can refer to in a way that will be helpful. But then again, the question of evaluation, I think Dr. Valdez mentioned that we can evaluate programs after they are done, but the problem is that we need sort of a feedback mechanism, so that when a program is in progress, we can tell if it is working or not. So, the evaluation is really the most difficult aspect, in my opinion, here. So, can we have a comment, Dr. Marrett? Dr. Marrett. Several comments. I will try to keep them brief. One of the things that you also heard during the testimony is the imperative for enhanced research, STEM education research, understanding a lot more of what works, under what conditions, for whom, and through what processes. Since we know we have got to enhance that a lot more, and I am very encouraged when I hear my colleagues indicate what that will mean. That will have to play into the plans for evaluation. So, on the one hand, we will be proceeding, building on the knowledge that does exist about evaluation, having those conversations of what are the appropriate kinds of measures, of indicators. How do we develop indicators that would be common enough across the agencies? We will do that, but we won't, I won't promise that we will have everything on an annual basis at the beginning, just because there are these very thorny but important questions about how do we build the knowledge base, the knowledge base that will be so important for the way the evaluation activities will have to take place. Mr. McNerney. Well, I like the sound of using the scientific method in evaluating this scientific education, but honestly, that is going to be our challenge. How do we know when we are succeeding? How do we make sure that the program is organized well? I think it is going to be important to see that plan to move forward. Mr. Valdez, a question, if I have time. I understand that you have been developing a strategic plan for the Department of Energy education programs, to ensure that they align with STEM workforce needs for the future. Would you tell the subcommittee the process you underwent to determine those educational needs, and how you used that information to create your strategic plan? Mr. Valdez. Yes, the strategic plan, as most strategic plans, is based on what our stakeholders think we should be doing, and aligning that to the mission of the Department of Energy. So, for example, we had a series of nine focus groups, involving 110 stakeholders, everything from the educational community to corporate America, to groups representing underrepresented populations, in a very structured way, talking about how the Department of Energy, which is a mission agency, could participate in STEM education and workforce development. And I think this gets to the heart of what this hearing is all about. Why would an agency like the Department of Energy be involved in this? And the reason is because we care about our future workforce. We support specific disciplines and specific areas of industrial competitiveness that are important to the Nation, and as a result, we have to work with the educational community at all levels, not just K-12, but undergraduate, postdoc, graduate school, and then continuing lifelong learning. So, we developed a process where we brought in members of all those communities, to help us rationalize the resources that we have within the Department of Energy, to more effectively use them. And frankly, our most effective use is through the National Laboratories. You, Mr. Chairman, were at Sandia National Labs, and you know that these National Laboratories are embedded in the communities. They work with local school districts. They are the experts; and so, we deliver our products and services through them, and we try to align the strategic plan in a way that makes full and maximum use of our National Laboratories. Mr. McNerney. Thank you, Mr. Valdez. The Chair now recognizes Dr. Ehlers for five minutes. Mr. Ehlers. Thank you, Mr. Chairman. First of all, Dr. Fuchs, you mentioned as an old professor, you felt compelled to use PowerPoint. Let me point out, as a younger professor, that I think you did the right thing. I have read a number of these articles about how terrible it is to use PowerPoint, and we shouldn't use them so much, et cetera. I have decided all of those are written by people who don't know how to prepare good slides. Dr. Marrett, first question for you. You described, you are going to reconstitute the NSTC, the National Science and Technology Council's Subcommittee on Education and Workforce Development, for which I applaud you. It should never have been disbanded. You described the necessary qualifications as knowledge and experience. So, would it also be helpful for you, in that process, to ensure that a minimum level of experience, or maybe the better word is stature within their agency, their seniority or whatever, be required, so that you can ensure that the appropriate individuals will be at the NSTC's disposal through this means? And I think you know what I am getting at. I have nothing against junior personnel. They usually have the best ideas, but you need some upper level people there to really make sure that this gets carried out, and that when someone on your subcommittee goes back home, and says well, we really should do this, that you don't have some other person there who frowns and says, well, we really don't have the money in the budget. But what is your response? Dr. Marrett. My response is that the way this is being set up is through the Committee on Science. The Committee on Science consists of the principals from the agencies. Hence, when we have asked for their nominees, the assumption is that whatever the level of the person, that person will have direct access to the top of the agency. You are quite right that it will be very important that there will be access to the centers of communication, to funding, but this is what we anticipate the direct, the process should help enhance that. Mr. Ehlers. Okay. I hope you are right. You may want to build something into the structure to ensure that. Next question, I know what the ranking of the budgets of the agencies are. Obviously, Dr. Fuchs, you have the richest agency, and then, we go down to NASA, then Energy, and then, finally, NSF. And we could have lots of arguments of how it should be. But I am curious how much is spent on education in each of your agencies. And we will go right to left, and ask Dr. Fuchs, what does NIH spend on these educational issues? What is your budget? Dr. Fuchs. I believe we submitted to the ACC around $800 million worth of activity, and that would have been in the year when our total budget was about $28 billion. Mr. Ehlers. Yeah. Okay. Dr. Fuchs. The majority, 95 percent of that, was actually graduate, postgraduate activity. Mr. Ehlers. Okay. And how much, any estimate how much is K- 12? Dr. Fuchs. We submitted $36 million worth of K-12 activities, and about $6 million worth of informal education. Mr. Ehlers. Okay. Mr. Valdez. Mr. Valdez. My Office has a budget of $8 million per year. Through appropriated funds, we think the Department spends about $20 to $25 million per year on educational activities; and then, through the National Laboratories, both through overhead, other kinds of activities, partnerships with industry and other groups, there is probably about another $30 million that is spent by the National Laboratories. So that is a total of about $50 to $60 million. Mr. Ehlers. Okay. And Dr. Winterton, I know NASA spends a huge amount of money on educational activities. What is yours? Dr. Winterton. Well, we do spend on education, in various places within NASA, so within the Office of Education, we plan on spending about spending about $153 million. Then, we also have each of our Mission Directorates, who also invest in education, because it ties, again, directly back to their missions. Mr. Ehlers. Right. Dr. Winterton. So, for the Science Mission, it is about $72 million, for Explorations, $4.8, Aeronautics, $2.8, and our Space Operation Mission Directorate spends and supports activities directly of the centers, so if you want those specifics, I can provide that as well, so---- Mr. Ehlers. I think this gives me a general idea. And finally, Dr. Marrett, what is---- Dr. Marrett. Yes. For NSF, across the board, it is nearly $1 billion, but this includes what is spent inside the Directorate for Education and Human Resources, as well as outside. So the Directorate itself has a budget of just over $700 million, but this is where I would again indicate that there are efforts, every part of the Foundation has education outreach public service activities that relate to education, and so, it comes to about nearly $1 billion. Mr. Ehlers. Okay. That is very helpful. It gives me some perspective, and I may have to speak to Dr. Bodman or Mr. Rohrabacher about pumping up your budget, Mr. Valdez. I really appreciate what you have done. I have encountered, in my experience in the schools, the K-12 schools, products of what you have done, and I think it is outstanding. The one part that bothers me, which I have tried to correct legislatively, but so far have not had enough support, is the inability of teachers to locate what they may want or need at a particular time, and so what I have advocated is a clearinghouse, where all of the programs that you have available for schools would be listed, and this would be on a website, and any teacher who wanted to could access that, if they had a particular unit they were working on, and wanted to just look at what was available, that teacher could go on the website, type that in, the units would be displayed. And I really advocate an Amazon.com type of approach, where teachers who have used something would then evaluate, and just write a short evaluative statement, give a ranking from one to four, and this would be very useful. I hope that we can get that passed, because I think that would be immensely useful to the teachers, and would give much greater exposure to what you have done and what you have solved. With that, I yield back. Mr. McNerney. Thank you, Mr. Ehlers. The Chair now recognizes my good friend and mentor from Illinois, Mr. Lipinski. Mr. Lipinski. Thank you, Mr. Chairman. I would like to thank all of the witnesses for their testimony today, and the work that they are doing. I don't know if there is anything really more important in ensuring America's future than to improve our STEM education. I think it is very critical for our country. I have a degree in mechanical engineering, and I always point to, and many people have heard this now, my physics teacher when I was a junior in high school, who really is the one who inspired me to go ahead and go to college and major in mechanical engineering, really got me interested and excited in studying engineering. A couple of things that I wanted to ask about. First, Dr. Marrett, the new NSTC Subcommittee on STEM Education is a subcommittee of the NSTC Committee on Science. Do you think that this is a sufficiently high level organization to get the agencies' attention, or do you think there should be something at a higher level than that? Dr. Marrett. I am quite willing to try it at the level where it is. There was, in fact, a previous committee that did exist. The charter expired in last December, and so, it is not as if this is a brand new activity, and my colleagues, who were involved before, will know that we have some items that should come onto that agenda. I know there has been this question of what should be the level. I think that we can be effective when there is the level of conversation that is needed, the planning, wherever, as long as, as Chairman McNerney indicated, there is the strong connection to the top of agencies and to the conversations that must take place across and within agencies. But I am not bothered by the fact that this is being structured as a subcommittee of the Committee on Science. Mr. Lipinski. Now, back in November, I spoke to the National Science Board. They were in Chicago meeting, talking about STEM education, and right now, we are awaiting their report and recommendation on what to do on STEM education. I understand it has been delayed now, until August. Dr. Marrett, or if any other witnesses have any, do you have any ideas about what is going to be coming out of this, or where do you see this going? Dr. Marrett. Well, let me first thank you very much for the presentation that you made, and in fact, I heard that it was an impressive set of comments that you gave. The National Science Board Committee is in the process of redrafting the report. That is why I am not sure my colleagues have seen what is still, for now, an internal document for the National Science Board, thus it would be difficult for me to describe where everything is likely to lead, except that I would say it, again, is likely to reinforce the imperative for our coordinated efforts, for our heightened attention for what must take place. So, to that extent, I see it as responding to and helping to clarify many of the issues this committee has dealt with, as well as the several reports that people have referred to. Mr. Lipinski. Well, I am very much looking forward to seeing that, as I know all of us are. In my last bit of time here, I wanted to ask Mr. Valdez about some of the programs that are going on with the DOE labs in helping with STEM education. I have talked to the Museum of Science and Industry in Chicago, who is working and has more plans to work with Argonne National Lab, to try to bring students down to the Museum to really get them interested, excited about these various areas. Now, a couple things that you talked about in your testimony. First, you describe activities that support K-12 teachers who are involved in research activities at the National Labs. Has there been an evaluation that DOE has done about the effectiveness of doing this? Mr. Valdez. We have not done a specific evaluation of that program. It is a relatively new program. It is a three year program. The first cadre of teachers completed their participation in the program just this past summer, and we have been developing the evaluation techniques that will enable us to track what happens to them after they finish the program, and what they have learned through the program. So, it is too soon to say whether or not we have evaluated it right. Mr. Lipinski. Well, I definitely want to support that. I think that is something we should be doing more of, and also, in addition, something you talked about was that Sandia National Lab, if the Chair will indulge me to ask one last question here, at Sandia National Lab, you mentioned how there is a pilot program for the Administration's Adjunct Teacher Corps Initiative. Can you explain how this works? Mr. Valdez. Well, the Adjunct Teacher Corps has a goal of putting 30,000 adjunct teachers into classrooms by the year 2012 or 2013, I forget which one, and the Department of Education came to the Department of Energy and said, ``We don't have the ability to recruit adjunct teachers the way that you do,'' because adjunct teachers would come from industry, National Laboratories, working scientists, retired scientists. We were very excited about working with them, and Sandia stepped up to the plate, and decided they would pilot the concept. And so, over the next fiscal year, we are going to take a look at what Sandia does, and then see if we can model this to the other 17 National Laboratories in the DOE system. I think that illustrates what we do at the Department of Energy. We work, we design programs nationally, but we implement them locally through the National Laboratories, and through our other partners. And if you do that, even though our budget is small, Dr. Ehlers--and I do appreciate your voice of support for this--the fact of the matter is we reach a lot of people. You know, through our National Laboratories, we touch 250,000 K-12 students every year, and 19,000 K-12 educators. The National Laboratories use the money that we have at Department of Energy headquarters as seed money to then go out and work with industry, local school districts, and everybody else, to implement the programs, and it is a very effective model, and one that I think could be useful for other federal agencies. Mr. Lipinski. Thank you. Thank you, Mr. Chairman. Mr. McNerney. Thank you, Mr. Lipinski. I believe the Committee was interested in another round of questions, so I will recognize myself for another five minutes. Dr. Marrett, will the subcommittee develop and maintain a catalogue of the federal STEM education programs, as the ACC did? I think that is important, at least having a quantifiable, a really quantifiable way to understand the connection and how money is being spent, whether there is overlap, waste, and so on. Dr. Marrett. Because the committee is still being put together, I have to hesitate to answer some questions about what the committee will do, but this one, I can be confident about, because we have already had the conversations that said yes, the ACC report had, of course, had recommended this kind of continued compendium of programs, and that will be done. So, that part is going to certainly be continued, as related to the ACC process. I think it is likely, too, that we will come back to an issue that had been on the table earlier, and that is trying to understand what the very concepts mean in the programs. As my colleague Dr. Fuchs has said, that sometimes the term that is used to describe a program need not mean the same thing from one place to another. So, as we try to develop a useful compendium of what takes place, that will mean looking very much at how the programs are defined to develop something that will be across the agencies, too, on that front. But yes, we will be a source of trying to provide that information about what takes place. Mr. McNerney. What about priorities? How are we going to go ahead, or how is the committee going to go ahead and determine what the priorities are, in terms of resource usage? Dr. Marrett. I am a bit hesitant to answer that one, on behalf of all of my colleagues from the committee yet to be established. I think I would say with some level of confidence we certainly will be looking at matters of priorities, paying attention to a lot of the information that is already around about what the Nation's needs are for now and into the future, this cannot be just looking at what might have been appropriate some 20 years ago, but exactly how those will evolve is to be determined, through a process that we intend to be as engaging and engaged as possible. For that, I would say we would welcome any observations you might have. Mr. McNerney. Well, it was interesting to hear Dr. Winterton's testimony, because she was talking about inspiring kids, and that is critically important, because no matter how many resources we offer, if the generation doesn't accept the offer and get engaged in the process, then we are not going to get as far as we need to get. And I see some of our national challenges as being a tool for getting kids engaged, the threat of global warming, for example, a great national and international threat that cannot only show challenge, in terms of risk, and the fear that that might bring, but also, the opportunity for intellectual achievement, for financial success, and so on. So some of these great national challenges ought to be involved in some way, if we can get that as a part of the program. I don't know exactly how, but perhaps Dr. Valdez, you have, or Dr. Winterton, have a comment on that? Dr. Winterton. I think, especially, as we have the opportunity to engage faculty and students directly with our scientists, through our science missions, or through our space exploration, so students see the real world application, and we provide mentorship, so even in high school, students are working at Goddard Spaceflight Center, and actually doing research with our scientists. So, I think it is that really, understanding, you really have to be very good, not only when you work in our labs, but when you are working on the International Space Station, with international colleagues, that they really apply themselves in their math and their science classes, that they start seeing a career for them within the aerospace industry, and understand that it is a great opportunity, that they see the kind of skills they need, and the competencies to be there. Mr. Valdez. I am a big fan of prioritizing things, and I think you need to go back to what are the core capabilities and the core missions of the federal agencies, and in the case of the federal mission agencies, I think it really comes down to two things. First, we need to be supportive of the workforce and the missions that we have been entrusted to us by the Congress and by the U.S. taxpayer. And so, for the Department of Energy, it is support for energy, environment, national security, and basic discovery science. And that helps inform what kinds of programs we should have for STEM education and STEM workforce. And then, we have a larger responsibility through the American Competitiveness Initiative, to support STEM education and literacy in general. And I think this is where your notion, Mr. Chairman, of having students become excited about science plays in, and agencies are uniquely placed to have students become excited, whether it is by going to a DOE National Laboratory, and seeing the Advanced Photon Source, or riding the National Oceanographic and Atmospheric Administration's ship, you know, an Antarctic ship. I mean, there are ways that we can do this, but we need to set those priorities, and rationalize the resources in the appropriate way. Mr. McNerney. Well, I agree with the idea, but that only reaches a fairly small segment of the population, I mean, in terms of children in inner city schools, I mean, they are not going to get out to the labs. We need to find a way to reach out and show them the national need, the national priority, and the need for them to get engaged for our nation's future, and that is something that the Committee might consider in its future debates. Dr. Marrett. Well, I would say that the Committee will need to look at a number of options, because what is also been represented a bit here, and even more so, with some of our agencies not at the table, is that we work to enhance the level of excitement, the motivation, through a number of informal processes, through the activities with Boy Scouts, for example, with community groups. One is not constrained to the very formal kinds of settings, and that is where we are also very interested in what is appropriate for the other kinds of settings, because we do not intend to leave behind segments of the population we know that must be important for where the Nation moves. Mr. McNerney. Well, my time has expired. I am going to recognize Mr. Ehlers. Mr. Ehlers. Thank you, Mr. Chairman. First of all, let me just make a comment, Mr. Chairman. I have been through many hearings over the years, and I have been working on improving math, science education in the K-12 system for close to 50 years now, but especially in the last 14, in the Congress. This is the biggest and best audience we have ever had for this, so I think, Mr. Chairman, we are finally getting the word out, and people are interested in the topic. I find that very heartening. I know that the National Science Foundation declares education and public outreach as part of research proposals that are submitted. Is that common in your other agencies as well, when investigators submit a request for proposal? Do they have to also include a statement about the educational aspects that will come out of that? Dr. Fuchs. It is not a feature in the NIH. It was considered for a while back in the 1990s, under the previous Director, and the decision was not to make that similar requirement, so we don't have that. Mr. Ehlers. Okay. Mr. Valdez. Mr. Valdez. No, it is not part of the Department of Energy's grant programs. Mr. Ehlers. Okay. And Dr. Winterton. Dr. Winterton. It is part of our Science Mission Directorate, so each of their principal investigators with their science missions are required to do an education and public outreach plan and implement that. Mr. Ehlers. And the two agencies that do this, is there the proposal of how they are going to implement that education and outreach? Is that considered in deciding whether or not to give the grant, or is that just sort of an automatic add-on? Dr. Marrett. No, for NSF, it is critical for what takes place, because we have got the criteria of intellectual merit and broader impacts, and in that context of broader impacts, this is where, generally, the outreach efforts, the education activities, are to be undertaken. Those, then, become critical in the decisions made about the awarding of grants. We wrote this process, trying to think of some ways to make sure that we can evaluate effectively the outcomes from those efforts. Mr. Ehlers. Okay. So, if someone submits a proposal for an experiment to measure Einstein's effect in general relativity, that, the educational part, is considered as well as the scientific part. Is that true in NASA also, Dr. Winterton? Dr. Winterton. It is a really critical part of their plan, and we work closely, and offer our assistance in the Office of Education, to assist in educational outreach and dissemination of those opportunities as well. Mr. Ehlers. Okay. And I would just encourage Energy and NIH to consider doing the same thing. It is a very useful thing. I also just, Dr. Winterton, you mentioned something about a mathematical exercise relating to flying. Do you use flight simulators at all your programs for the schools? Dr. Winterton. We do have---- Mr. Ehlers. Commercially available flight simulators. Dr. Winterton. We provide those opportunities through our centers. Now, the type of simulation the students were doing, and I know you are a budding pilot, and you might want to try this simulator, to make sure we are getting the right mission control experts, is---- Mr. Ehlers. I have already flown a Shuttle. Dr. Winterton.--is a computer-based simulation, so that part is available at any school. Mr. Ehlers. Yeah. Dr. Winterton. So, we like that combination of affordable, readily available, but you can also go to a center to do the simulation as well. Mr. Ehlers. Okay. Dr. Winterton. And I think through some of our university and our space grants, that certainly is another opportunity to provide that in a broader sense. Mr. Ehlers. Okay. And I can't give commercials, but I am fascinated with the potentials of the X-Plane program, because it actually allows students to construct their own airplane, and you have to be fairly sophisticated, at least junior high, perhaps higher, but it is an incredibly good educational experience, and the program is $50 or something like that, so any school could afford to have that. Well, thank you, Mr. Chairman. This has been an excellent hearing. The only missing component, and I don't want to criticize, even though I should, since I am part of the minority, and that is our job, but there is one other agency that spends a huge amount of money on K-12 education, and that is the Defense Department. It is a smaller fraction of their budget than any of these agencies, but we might want to hear from them some time, too, about what they are doing. And they, incidentally, one plus factor of the Defense Department is their Army Schools, which are located around the world--I shouldn't say Army--the schools are for their employees' children--consistently rank higher than the average American school, particularly in the science and technology, and also, often equal the countries, the record of the countries that they are in. So, obviously, the Defense Department is doing something right in their schools, and we might be interested in talking to them about that some time. With that, I will conclude, and thank you again, Mr. Chairman, for having this hearing. Mr. McNerney. Well, the Chair thanks the Ranking Member for his thoughtful comments, and I think that is something that we would want to move forward with. And before closing, I would like to thank the witnesses for, again, spending their afternoon, and also, the members of the audience. It is a nice day out there, so it is appreciated that you would come in here to hear our testimony. And the record will remain open for additional questions that Subcommittee Members may ask of the witnesses, so be prepared for that. And the witnesses are excused, and the hearing is now adjourned. [Whereupon, at 4:00 p.m., the Subcommittee was adjourned.] Appendix 1: ---------- Answers to Post-Hearing Questions Responses by Cora B. Marrett, Assistant Director, Education and Human Resources Directorate, National Science Foundation Questions submitted by Representative Eddie Bernice Johnson Q1. Dr. Marrett, the new National Science and Technology Council (NSTC) subcommittee on STEM education is a subcommittee of the NSTC Committee on Science. Is this of a sufficiently high level organizationally to get the agencies' attention? Because of the widely recognized importance of STEM education, shouldn't this be constituted as a new, independent NSTC committee? A1. The membership of the STEM Education Subcommittee has now been established. The timely response from the agencies to the request for names and the responsibilities of the persons designated suggests that the subcommittee structure can be effective. The persons selected to represent their agencies have sufficient oversight for personnel and budgets to act on behalf of their organizations. There are substantive reasons for an arrangement that connects STEM education to the larger Committee on Science (COS). First, the agenda on STEM education must reflect current developments in the wider realm of STEM research and development. The connection to the COS, the body charged with the broader world of research and development, helps ensure that the priorities for STEM education are aligned with the directions at the federal level of science, technology, engineering, and mathematics. Second, within the agencies STEM education is integrated into and not isolated from the objectives agencies have identified for science and engineering. The subcommittee structure, then replicates the pattern that agencies employ and have found to be beneficial. The structure does not signal that the subcommittee is of secondary importance in the NSTC complex. The principals for the Committee on Science regard STEM education as of profound significance for the Nation and consequently anticipate regular exchanges about the work of the subcommittee. They, in turn, are committed to facilitating and participating in exchanges between the subcommittee and any other relevant NSTC bodies. Likewise, the co-chairs of the subcommittee have ready access to the principals and aim to work cooperatively with them. In sum, then, the subcommittee has obtained from the agencies and the Committee on Science the support it will need to be an effective force in the pursuit of excellence for education in science, technology, engineering, and mathematics. Q2. The Academic Competitiveness Council (ACC) has recommended that no funding should be increased for a STEM education program ``unless a plan for rigorous, independent evaluation is in place,'' which is defined in the report as being Randomized Controlled Trials or, when that is not feasible, Well-Matched Comparison Group Studies. One of our witnesses at the May 15th hearing, Dr. Weiss, stated that this approach would not be practical for the majority of programs in the federal R&D mission agencies. I would like to ask our witnesses to respond to Dr. Weiss' opinion. Is this a practical way of evaluating programs at NASA, Energy, and NIH? What other types of evaluation methods might be better suited? A2. The Academic Competitiveness Council (ACC) report does indeed recommend no additional funding to STEM education programs, in the absence of plans for rigorous, independent evaluation. The report points out, however, that ``no single design or evaluation methodology is appropriate for all education studies,'' and ``the appropriate methodology should be selected based on the maturity of the activity'' (p. 13). Furthermore, it acknowledges that programs can be arrayed along a continuum that starts ``generally with small-scale studies to test new ideas and generate hypotheses, leading to increasingly larger and more rigorous studies to test the effects of a given intervention or activity on a variety of students and in a variety of settings'' (p. 13). Thus, the report does not advocate for the premature use of experimental or quasi-experimental methods to determine causality. It is also important to remember as well that the ACC discussion is centered on measuring the impact of an educational activity on student outcomes (p. 15), not educational differences at the institutional or system level. The National Science Foundation recognizes the importance of using randomized controlled trials (RCTs) to establish cause-and-effect relationships between education programs and student outcomes. As Dr. Weiss stated, and the ACC report supports, RCTs and well-matched comparison group studies are not always feasible and applicable, nonetheless. For short-term outreach efforts or efforts to enhance the institutional structure for STEM education, some activities cannot be standardized and controlled. For projects federal agencies and others undertake, the randomized assignment of participants is not always feasible, practical, or ethical. Thus, the use of RCTs is appropriate depending on the program design and the research question. The ACC report recognizes this point; Dr. Weiss sought especially to reinforce it. Policy-making bodies frequently draw on evidence from multi-year studies that use mixed methods. What seems significant for the making of policy is the rigor of the approach that is taken, coupled with attention to the question or problem that is central to analysis. Answers to Post-Hearing Questions Responses by Joyce L. Winterton, Assistant Administrator, Office of Education, National Aeronautics and Space Administration (NASA) Questions submitted by Representative Eddie Bernice Johnson Q1. The Academic Competitiveness Council (ACC) has recommended that no funding should be increased for a STEM education program ``unless a plan for rigorous, independent evaluation is in place'' which is defined in the report as being Randomized Controlled Trials or, when that is not feasible, Well-Matched Comparison Group Studies. One of our witnesses at the May 15, 2007, hearing, Dr. Weiss, stated that this approach would not be practical for the majority of programs in the federal R&D mission agencies. I would like to ask our witnesses to respond to Dr. Weiss' opinion. Is this a practical way of evaluating programs at NASA, Energy, and NIH? What other types of evaluation methods might be better suited? A1. Establishment of standards for the evaluation of Federal STEM education programs was extensively discussed within the ACC working groups. A hierarchy of study designs, weighted in favor of a research- oriented model focused on Randomized Control Trials (RCTs), was adopted. However, several concerns were raised by NASA and other working group members. The main concern is that RCTs are primarily applicable to interventions, defined broadly by the education evaluation community, as projects with highly specific features that target a precisely defined audience in order to achieve a specific outcome under controlled, standardized conditions. There are a variety of other reasons why RCT models are not practical for some of NASA's portfolio, including: (a) schools are not typically prepared to match control and treatment groups; (b) RCTs are complex and costly to properly implement; and, (c) some of NASA's education projects are not specific interventions according to the above definition but are instead designed to enhance the capabilities of the education and outreach community (e.g., the NASA Explorer Institutes project seeks to enhance the ability of science centers and museums to use NASA's unique resources). NASA is in full agreement that sound, rigorous evaluations should be implemented to provide the best possible evidence of effectiveness but the methodologies must be appropriate to the program. NASA supports and is implementing a definition of rigorous evaluation that includes RCT-based methodologies, as feasible. Where RCTs are not feasible, NASA plans to implement alternate evaluation methodologies, commonly called ``Mixed Methods'' approaches, to capture a complete picture of education investments to determine effectiveness in achieving outcomes, impact and comprehensiveness. These Mixed Methods are based on the accepted professional standards for educational evaluations articulated in The Program Evaluation Standards: How to Assess Evaluations of Educational Programs, 2nd edition, (Joint Committee on Educational Evaluation, 1994). Like RCTs, these rigorous methods will lead to credible, objective, reliable, and valid evaluations of program performance and effectiveness. Mixed-methods evaluations use quantitative data, such as experimental, quasi-experimental, and correlational studies, and also use qualitative methodologies, such as case studies, surveys, and focus groups. NASA will use RCTs when practical and relevant, however the Agency will typically use mixed- methods evaluations. NASA has developed and submitted to the Office of Management and Budget a plan for evaluating each of the Agency's major projects. Answers to Post-Hearing Questions Responses by William J. Valdez, Director, Office of Workforce Development for Teachers and Scientists, Office of Science, Department of Energy Questions submitted by Representative Eddie Bernice Johnson Federal STEM Education Programs Q1. The Academic Competitiveness Council (ACC) has recommended that no funding should be increased for a STEM education program ``unless a plan for rigorous, independent evaluation is in place'' which is defined in the report as being Randomized Controlled Trials or, when that is not feasible, Well-Matched Comparison Group Studies. One of our witnesses at the May 15th hearing, Dr. Weiss, stated that this approach would not be practical for the majority of programs in the federal R&D mission agencies. I would like to ask our witnesses to respond to Dr. Weiss' opinion. Is this a practical way of evaluating programs at NASA, Energy, and NIH? What other types of evaluation methods might be better suited? A1. The Office of Science's Workforce Development for Teachers and Scientists (WDTS) program fully supports the Academic Competitiveness Council (ACC) recommendations and has developed a rigorous evaluation program currently under review by the Office of Management and Budget. That review program, which is being designed to ensure that the cost is commensurate with the overall size of the WDTS program, emphasizes three evaluation protocols that are consistent with the comments made by Dr. Weiss. Those three protocols, which we believe are appropriate to the needs of mission agencies such as NASA, the Department of Energy and NIH, are: <bullet> Quasi-experimental approaches that utilize well- matched groups to understand improvements to the learning of scientific content, STEM career choices, and improvements to STEM teaching approaches; <bullet> Long-range longitudinal studies of student participants designed to verify whether WDTS programs are achieving their goal of contributing to the DOE STEM workforce; and, <bullet> Management effectiveness studies, such as external reviews by Committees of Visitors, to validate that WDTS programs are managed efficiently. This evaluation program is designed for the mission needs of the Department of Energy, which requires development of a highly qualified pool of scientific and technical workers in the mission areas it supports (defense, environment, energy, and scientific discovery). Answers to Post-Hearing Questions Responses by Bruce A. Fuchs, Director, Office of Science Education, National Institutes of Health Questions submitted by Representative Eddie Bernice Johnson Federal STEM Education Programs Q1. The Academic Competitiveness Council (ACC) has recommended that no funding should be increased for a STEM education program ``unless a plan for rigorous, independent evaluation is in place'' which is defined in the report as being Randomized Controlled Trials or, when that is not feasible, Well-Matched Comparison Group Studies. One of our witnesses at the May 15th hearing, Dr. Weiss, stated that this approach would not be practical for the majority of programs in the federal R&D mission agencies. I would like to ask our witnesses to respond to Dr. Weiss' opinion. Is this a practical way of evaluation programs at NASA, Energy, and NIH? What other types of evaluation methods might be better suited? A1. It is important to articulate the final goal of federal STEM education efforts. We should seek to fund programs that really make a difference in the lives of our children. We want those students with talent and inclination to consider STEM careers in order to help the government, and the Nation, meet its need for technical professionals. But additionally, in an increasingly competitive world, we want all students to obtain the knowledge and skills (for example, problem solving and critical thinking) that they will need to find good jobs and lead fulfilling lives. The kinds of skills that students can learn in STEM courses will help prepare them for the 21st century--even if they never put on a white coat or work in a laboratory. The ACC report does not really define rigorous research as ``Randomized Controlled Trials, or when that is not feasible, Well- Matched Comparison Group Studies.'' The report does emphasize the importance of these evaluation methods for ``those study designs whose purpose is to estimate a project's impact on education outcomes, such as student math and science achievement.'' But the report also acknowledges that much of the federal STEM effort does not fall into this category. The report recognizes that ``no single study design or evaluation methodology is appropriate for all education studies, and that the appropriate methodology should be selected based on the maturity of the activity, the intended use of the data, and the inferences to be drawn from study results. . ..'' Early in any STEM education research endeavor, most studies will not be candidates for a randomized controlled trial design. These studies will more likely focus on classroom observations, developing early-stage instructional materials or approaches, and generating testable hypotheses with regard to student performance. The ACC recognizes that these types of studies ``are a key part of the research agenda needed to improve U.S. STEM education, can be `rigorous' in their own context, and can serve as valuable precursors and-or complements to impact studies.'' However, the goal should ultimately be to determine what works for students. Promising programs at different stages of maturity should be identified and subjected to evaluation methods that have increasing power to discern whether the student outcomes observed are really related to the program interventions. This goal will push study designs up the ``hierarchy pyramid'' toward randomized controlled trials. However, even in these cases, a variety of evaluation methods will be used to ``complement'' the impact study. This is because we will seldom be satisfied to know simply whether a particular educational intervention works or not--we want to know why it succeeds, or fails, to achieve the intended objective. We do know more about how to effectively teach STEM subjects to students than we did a generation ago. However, the number of such insights that have been rigorously tested and that are known to be valid for large numbers of students under a variety of conditions is much smaller than we would like it to be. Carefully applying the recommendations of the ACC report will help ensure that we are not in the same situation a generation from now. Appendix 2: ---------- Additional Material for the Record Statement of the Office of Education National Oceanic and Atmospheric Administration U.S. Department of Commerce Question 1: What steps have agencies taken to improve coordination with other federal agencies' STEM education activities? To what extent do agencies collaborate with educators in the states and school districts in developing STEM education programs? The National Oceanic and Atmospheric Administration (NOAA) is leveraging several relationships established through the Academic Competitiveness Council (ACC) to foster improved coordination among Federal Science, Technology, Engineering and Mathematics (STEM) education activities. For example, such relationships include: (1) the synthesis of independent evaluations of federal STEM activities (as previously demonstrated by the National Aeronautics and Space Administration); (2) providing advice to other agencies (e.g., the National Science Foundation) on how to develop and implement evaluations programs consistent with the ACC recommendations and metrics; and (3) seeking external advice and guidance to inform the design of NOAA STEM education activities (e.g., from the Department of Education on NOAA's Teacher at Sea Program). NOAA looks forward to exploring additional opportunities through the National Science and Technology Council (NSTC) to build on the collaborative activities initiated under the ACC. NOAA is working with educators to develop and improve individual STEM education programs in many states and school districts. These programs are designed to support the long-term development of quality educational programs for all educators and students while, simultaneously, meeting the goals of NOAA and the Nation. For example NOAA is working with California on the state's Environmental Education Initiative, and NOAA is working with Hawaii to develop a Marine Science Curriculum. Question 2: The recent report of the Academic Competitiveness Council reinforces the need for better evaluation and performance metrics for federal STEM education programs. What plans do agencies have for improvements in evaluation of STEM programs? NOAA is taking direct action to improve the consistency, rigor, and frequency of evaluation activities for each of its education programs. Specifically, NOAA will measure the effectiveness of all its current and future education activities using methodologies appropriate to the types of activities funded. NOAA has recently adopted a program logic model to inform the design and evaluation of all education activities in the agency, address the performance metrics of the Academic Competitiveness Council, and promote consistency, coordination, and information sharing with other federal entities. NOAA's Educational Partnership Program (EPP) has adjusted its operation and implementation activities based on annual client evaluation; monitored the output and products of its programs against established performance measures; and assessed program statistics of students who have trained and/or graduated in STEM fields. In addition, NOAA's Cooperative Science Centers, which are associated with EPP, were rigorously evaluated after three years of operation using metrics that were established previously to evaluate NOAA's Cooperative Institutes. Evaluation metrics were tailored to address a variety of components such as education, outreach, research, administration, recruitment, budgeting, and capacity building being conducted at each of the Cooperative Science Centers. The accomplishments of each program were compared to the objectives and performance measures established during the development phase. A plan to conduct rigorous, independent evaluation is currently being established for NOAA's student scholarship programs, which include undergraduate and graduate fellowships, as well as the Ernest F. Hollings and Dr. Nancy Foster scholarship programs. Although these programs have not yet been formally evaluated, it should be noted that there are established performance measures in place for each program. Question 3: The Subcommittee received testimony at a hearing on 15 May on how the R&D mission agencies could improve the effectiveness of their STEM education programs. The witnesses were skeptical of the ability of the agencies to develop curricular materials for formal classroom instruction and questioned the effectiveness of their teacher professional development programs to improve teacher classroom performance, while suggesting that the agencies' most important role is in informal STEM education. The witnesses also strongly recommended closer collaboration by the agencies with educators in the field when developing STEM programs. What are agencies' responses to the recommendations from these witnesses? NOAA's efforts do not replace or supplant the critical role of the State and local governments in education. Because there are limited resources to invest and large needs to support in science education, NOAA believes investments in formal and informal education are most effective and efficient at producing outcomes when they supplement or complement efforts supported by others. NOAA's efforts are aimed at providing supplemental materials in subject areas where NOAA has unique expertise and where sufficient public knowledge is required to ensure understanding and response to warnings, forecasts, and stewardship efforts. NOAA education maximizes use of place-based learning opportunities afforded by field offices, including National Marine Sanctuaries, Sea Grant Colleges, National Estuarine Research Reserves and Weather Forecast Offices. We often work through external partners, such as education associations or aquaria and science museums, to enhance our connections to the public and the education community. Students and educators are one of many groups of users of NOAA data and information. The NOAA Outreach Unit in Silver Spring, MD receives over 4,000 unique requests each year for NOAA's education materials from teachers, students and librarians. NOAA recognizes that education is primarily the responsibility of state and local governments. However, state and local education programs may not specifically focus on topics relevant to NOAA's mission. For example, the ocean sciences are under-represented in the national science education standards for grades K-12. In addition, teacher content knowledge is not always sufficient in the ocean and atmospheric sciences. As a result, students graduating from U.S. high schools may not possess sufficient understanding of the earth processes and phenomena that are the focus of NOAA research, monitoring, and prediction efforts. NOAA's informal education activities provide educational experiences that typically involve taking students to unique settings outside of the classroom. Informal education combines well-established educational methods with the excitement of hands-on activities and field experiences and develops life long interest in the ocean and atmosphere. NOAA's informal education activities include hosting school children, community groups, and the general public at NOAA sites, supporting hands-on experiences in NOAA-related sciences and increasing the inclusion of NOAA-related topics at science centers, museums, and aquaria. Question 4: How do agencies determine priorities for K-16 STEM education portfolio? Has agencies' balance of programs at graduate/post doctoral, undergraduate, K-12, and informal education changed much over the past few years? Is there a likelihood of a change in that balance in the future? Most of the investment priorities and the general direction of NOAA education activities over the past few years has been determined by Congress--either through legislation (e.g., the National Marine Sanctuaries Act, Coastal Zone Management Act, Sea Grant Act) or through appropriations language (Environmental Literacy Grants). Other education activities are supported by across the board percentages which Congress has instigated (e.g., NOAA-wide in the case of the Hollings Scholarship Program and Sanctuaries-wide in the case of the Nancy Foster Scholarship Program). Within these external constraints, NOAA's process for determining priorities is based on national peer reviewed competitions focused on advancing earth system science education. Although there is no formal review process to determine priorities for funding, the following criteria are used informally: Proposed projects should: <bullet> deliver NOAA-wide benefit; <bullet> have hard schedule drivers that require action within a one to two year time frame; <bullet> reach a large, high priority audience; <bullet> result in a significant impact on the audience reached; <bullet> increase understanding of NOAA science and service; <bullet> leverage partnerships; <bullet> build on existing NOAA investments; and <bullet> target NOAA priority areas. Question 5: How do agencies disseminate information about STEM education programs? What organizations, both government and private, have agencies partnered with to reach educators in the field? NOAA relies on partnerships with a variety of government, non- government, non-profit, and private organizations to disseminate information about our STEM education programs. Partnerships mentioned above with State departments of education allow NOAA to directly assist state efforts to improve STEM-related education in areas specific to NOAA science (e.g., watershed and environmental education in the states that are party to the Chesapeake Bay agreement and its education commitments; environmental education in California; marine science education in Hawaii). NOAA also depends heavily on the dissemination networks of State and university partners for the education programs funded by the National Sea Grant College Program, the National Estuarine Research Reserve System, and cooperative research programs including the Educational Partnership Program, Joint Research Institutes, and Cooperative Institutes. Other federal partners, such as the National Aeronautics and Space Administration and the National Science Foundation, leverage NOAA investments for broader impact including improving the rigor and credibility of high school Earth systems science course work and build centers for Coastal Ocean Science Educational Excellence. In the past few years, multiple partners joined NOAA to develop Ocean Literacy Principles to assist teachers in using ocean concepts and examples to teach state science standards, and similar efforts are underway for climate literacy. NOAA also partners with the American Association for the Advancement of Science and the National Academy of Sciences to maintain the currency of the science education standards and benchmarks disseminated by these organizations. To reach classroom teachers directly, NOAA partners with organizations with large teacher membership and distribution networks and teacher training efforts (e.g., National Science Teacher Association, National Marine Educators Association, American Meteorological Society, the Jason Project). NOAA also supports online, searchable education resource libraries that provide access to standards-referenced, peer-reviewed education materials and lesson plans for teachers (e.g., Digital Library for Earth Systems Science, www.dlese.org; the Bridge, http://www2.vims.edu/bridge/noaa/). Partnerships are essential to NOAA's informal education efforts to promote STEM-related education related to NOAA science in the general public as well as supplementing the activities of the formal education system. NOAA's partnerships with individual as well as networks of science centers, museums, aquaria, and zoos produce innovative exhibits and displays and complementary educational programming. These partnerships include world renowned institutions such as the American Museum of Natural History and the Smithsonian Institution National Museum of Natural History, and organizations such as American Zoo and Aquarium Association, Association of Science-Technical Centers, and Coastal Ecosystem Learning Centers). Partnerships with non-profit, private, and government organizations also enable NOAA's efforts to improve public understanding related to STEM through radio and television programming and public media campaigns.