ENGINEERINGCHANGE
A Study of the
Impact of EC2000
Executive Summary
ENGINEERING CHANGE
A Study of the
Impact of EC2000
Executive Summary
Copyright © 2006 by ABET, Inc.
Printed in the United States of America.
All rights reserved. No part of this report may be reproduced in any form or by any means
without written permission from the publisher.
Published by: ABET, Inc.
111 Market Place
Suite 1050
Baltimore, MD 21202
www.abet.org
Authors: Lisa R. Lattuca, Patrick T. Terenzini, and J. Fredricks Volkwein
Center for the Study of Higher Education, The Pennsylvania State University
400 Rackley Building
University Park, PA 16802
www.ed.psu.edu/cshe
Acknowledgements
A study of this complexity and magnitude requires the assistance of a large number of organi-
zations and individuals. We wish to acknowledge the many and important contributions of the
following to the success of this study:
� The 28 professional and technical societies that constitute ABET, Inc., and thus sup-
ported this study.
� Those societies representing the seven disciplines on which this study focused, for
their endorsements and assistance, and their members who responded to the survey
of engineering employers: The American Institute of Aeronautics and Astronautics,
the American Institute of Chemical Engineers, the Institute of Electrical and
Electronics Engineers, Inc., the American Society of Civil Engineers, the American
Society of Mechanical Engineers, and the Institute of Industrial Engineers.
� The American Society for Engineering Education for access to its engineering pro-
gram database, which enabled us to identify and specify the study population and
develop our sampling design.
� The National Science Foundation for its financial support of portions of our study
(NSF Grant No. EEC-9812888).
� The members of our National Advisory Board (see Appendix C) for their sage advice
and steady support.
� The deans, department chairs, faculty members, 1994 graduates, and 2004 graduates
of the 40 institutions that participated in our study; without their cooperation, this
project could not have happened.
� The Penn State College of Engineering faculty members and students, the ABET
Industry Advisory Council, and others who assisted us with instrument refinement,
pilot testing, and other forms of advice.
� The ABET executive staff members who provided ongoing support, guidance, and
autonomy.
We are grateful to all these people and organizations. Without their assistance, the study
would have been greatly diminished. We also wish to say it has been a pleasure and honor to
participate in the national conversation on quality in engineering education.
Lisa R. Lattuca
Patrick T. Terenzini
J. Fredricks Volkwein
University Park, PA
March 2006
-i-
Table of Contents
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1
Findings from the EC2000 Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3
Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12
Appendix A: Study Design and Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14
Appendix B: Engineering Change Study Team . . . . . . . . . . . . . . . . . . . . . . . . . .16
Appendix C: Engineering Change National Advisory Board . . . . . . . . . . . . .17
Appendix D: Engineering Criteria 2000 (EC2000) . . . . . . . . . . . . . . . . . . . . . . .18
Engineering Change:
A Study of the Impact of EC2000
Lisa R. Lattuca, Patrick T. Terenzini, J. Fredricks Volkwein
Center for the Study of Higher Education,
The Pennsylvania State University
Are post-EC2000 engineering graduates any better prepared to enter the profession than
were their pre-EC2000 counterparts of a decade ago? That question is at the heart of this
three-year study, titled Engineering Change: A Study of the Impact of EC2000.
In 1996, the ABET Board of Directors adopted the
new set of standards, called Engineering Criteria 2000
(EC2000). EC2000 shifted the basis for accreditation
from inputs, such as what is taught, to outputs —
what is learned. The new criteria specify 11 learning
outcomes and require programs to assess and demon-
strate their students’ achievement in each of those
areas. EC2000 retains earlier accreditation standards’
emphases on the development of students’ mathemati-
cal, scientific, and technical knowledge, as well as
standards for program faculty and facilities, but it also
emphasizes developing other professional skills, such
as solving unstructured problems, communicating
effectively, and working in teams. In addition, EC2000
stresses awareness of ethical and contextual considera-
tions in engineering.
In 2002, ABET, Inc., commissioned the Center for
the Study of Higher Education at Pennsylvania State
University to undertake a three-and-a-half-year study
to assess whether the implementation of the new
EC2000 evaluation criteria is having the intended
effects. Engineering Change: A Study of the Impact of
EC2000 was designed to answer two primary questions:
� What impact, if any, has EC2000 had on stu-
dent learning outcomes in ABET-accredited
programs and institutions?
� What impact, if any, has EC2000 had on orga-
nizational and educational policies and prac-
tices that may have led to improved student
learning outcomes?
Engineering Change -1-
Origins of EC2000
Since 1932, ABET, Inc., has
been the recognized U.S.
accreditor of postsecondary
degree-granting programs in
engineering. ABET currently
accredits nearly 2,000 engi-
neering programs at more than
350 institutions.
ABET accreditation requires,
among other things, that engi-
neering programs adhere to a
set of quality standards called
accreditation criteria. For most
of the second half of the 20th
century, ABET’s accreditation
criteria dictated all major ele-
ments of an accredited engi-
neering program, including
program curricula, faculty, and
facilities. In the mid-1990s,
however, the engineering com-
munity collectively began to
question the validity of such
rigid accreditation require-
ments. Following a year of
intense dialogue within the
engineering community, ABET
crafted new and unique
accreditation criteria for engi-
neering programs. These crite-
ria became known as
Engineering Criteria 2000 or
EC2000.
To answer these questions, the Penn State research team examined educational practices
in engineering programs and assessed student performance pre- and post-implementation of
EC2000. The conceptual model guiding the study (see Figure 1) summarizes the logic of the
study’s design.
Figure 1 assumes that, if implementation of the EC2000 evaluation criteria is having the
desired effect, several changes in engineering programs would be evident:
� Engineering programs would make changes to align their curricula and instructional
practices with the 11 learning outcomes specified by EC2000 (Criterion 3.a-k, see
Appendix D).
� Alterations in the faculty culture would be evident as faculty members engaged at a
higher rate than before EC2000 in activities such as outcomes assessment and curricu-
lum revision.
� Faculty and program administrators would adjust program practices and policies
regarding faculty hiring, salary merit increases, tenure, and promotion criteria to give
greater recognition to the kinds of teaching and learning required by EC2000.
� All of those program changes would reshape students’ educational experiences inside
and outside the classroom, which would in turn enhance student learning (defined as
improved student performance on measures of the 11 EC2000 learning outcomes).
-2- Engineering Change
PROGRAM
CHANGES
EC2000
OUTCOMES
Continuous Improvement
STUDENT
EXPERIENCES
Curriculum &
Instruction
Faculty
Culture
Policies &
Practices
In-Class
Out-of-Class
Student
Learning
(3.a-k)
Employer
Ratings
Figure 1. Conceptual Framework for the Engineering Change Study
� Employers would report improvements in the knowledge and competencies of the
engineering graduates they have hired since implementation of EC2000.
The EC2000 study evaluated these connections to assess whether any changes in engi-
neering programs and improvements in student learning are a consequence of EC2000 rather
than other factors.
Research Design and Sample Selection
The EC2000 study utilized a cross-sectional, pre- and post-EC2000 design that drew on
multiple sources of evidence to provide a 360-degree view of the impact of the EC2000
accreditation criteria on the preparation of undergraduates for careers in engineering. Table 1
provides the sample size and response rate for each population studied. By social science
standards, the response rates are quite respectable for studies of these populations, and the
numbers of respondents provide more than adequate statistical power. [A summary of the
study’s research design and methods appears in Appendix A.]
Findings from the EC2000 Study
Major findings from the EC2000 study are reported according to the logic of the conceptu-
al model given in Figure 1.
Changes in Engineering Programs
According to program chairs and faculty members, engineering program curricula changed
considerably following implementation of the EC2000 criteria. Although few programs
reduced their emphasis on the foundational topics in
mathematics, basic science, and engineering science,
both program chairs and faculty members report
increased emphasis on nearly all of the professional
skills and knowledge sets associated with EC2000
Criterion 3.a-k. Three-quarters or more of the chairs
report moderate or significant increases in their pro-
gram’s emphasis on communication, teamwork, use of
modern engineering tools, technical writing, lifelong
learning, and engineering design. Similarly, more than
half of the faculty respondents report a moderate to
Engineering Change -3-
Table 1. Response Rates for Target Populations
Data Sources Target Population Number of Responses Response Rate
Programs 203 147 72%
Faculty 2,971 1,243 42%
Deans 40 39 98%
1994 Graduates (Pre-) 13,054 5,494 34%
2004 Graduates (Post-) 12,921 4,330 36%
Employers unknown 1,622 N/A
Key Findings: Changes in
Engineering Programs
� Greater emphasis on
professional skills and active
learning after EC2000.
� High levels of faculty support
for continuous improvement.
� Mixed emphasis on teaching
in faculty reward structure.
significant increase in their emphasis on the use of modern engineering tools, teamwork, and
engineering design in a course they taught regularly.
EC2000’s focus on professional skills might also be expected to lead to changes in teach-
ing methods as faculty members seek to provide students with opportunities to learn and
practice their teamwork, design, and communication skills. Consistent with that expectation,
half to two-thirds of the faculty report that they have increased their use of active learning
methods, such as group work, design projects, case studies, and application exercises, in a
course they teach regularly (See Figure 2).
Are these curricular and instructional changes attributable to EC2000 or to other influ-
ences shaping engineering education? Program chairs are much more likely than faculty to
credit ABET (70% vs. 28%, respectively) and industry (78% vs. 28%) with having a moderate
to strong influence on curricular changes in their programs. Faculty are more likely to take
personal credit (82%) for changes they have made in their courses or to cite student feedback
(54%) as having a moderate or great deal of influence. When other possible influences on cur-
ricular change at the course and program levels are controlled, however, faculty members
believe ABET has had a statistically significant and independent influence on all measures of
curricular or instructional change, and program chairs see a significant and independent
ABET influence in two of three curricular areas. Industry feedback, however, is viewed as
having a significant influence on only two of three course-level changes. Program chairs did
not consider industry feedback as having a significant, independent influence on any changes
at the program level.
EC2000 also requires that engineering programs assess student performance on the a-k learn-
ing outcomes and use the findings for program improvement. Program chairs report high levels
-4- Engineering Change
0204060
Some to significant
decrease
Lectures
Case Studies
No
change
Some to significant
increase
0 20 40 60
5 -point scale, where 1 = “Significant Decrease and 5 = “Significant Increase
25%
54%42%
54%
52%
40%
43%
20% 20% 60%
17%
Open-Ended Problems
Use of Groups in Class
Textbook Problems 61%22%
38% 60%
67%31%Computer Simulations
Design Projects
Application Exercises
6%
5%
4
2
2
2 65%33%
5 -
Figure 2. Faculty’s Reports of Changes in Teaching Methods
Since First Teaching the Course
of faculty support for these practices (see Figure 3). More than 75 percent of the chairs estimate
that either more than half or almost all of their faculty supported continuous improvement
efforts, and more than 60 percent report moderate to strong support for the assessment of stu-
dent learning. Faculty corroborated this finding: Nearly 90 percent of the faculty respondents
report some personal effort in assessment, and more than half report moderate to significant lev-
els of personal effort in this area. For the most part, moreover, faculty members do not perceive
their assessment efforts to be overly burdensome: Nearly 70 percent think their level of effort
was “about right.”
Learning how to do assessment or incorporate active learning methods into courses may also
influence faculty members’ engagement in professional development opportunities focused on
teaching and learning. This study finds that more than two-thirds of the faculty members report
reading more about teaching in the past year, and about half engage in formal professional
development activities, such as attending seminars or workshops on teaching, learning, and
assessment, or participating in a project to improve engineering education. Depending on the
activity, one-fifth to one-quarter of the faculty members say that in the past five years they have
increased their teaching-and-learning-related professional development efforts.
One of the most important influences on faculty work in colleges and universities is the
institutional reward system, which can encourage or discourage attention to teaching. The
EC2000 accreditation criteria require that engineering programs be responsible for the quality
of teaching, learning, and assessment, but do faculty members believe that their institutions
value their contributions in these areas when making decisions about promotion, tenure, and
merit-based salary increases? About half of the program chairs and faculty surveyed see no
change in their institution’s reward system over the past decade. About one third of the pro-
Engineering Change -5-
37
35
29
35
35
20
0
10
20
30
40
50
60
70
80
Systematic Efforts
to Improve
Assessment of
Student Learning
Data-based
Decision-Making
More than half the faculty Almost all faculty
5 -point scale, where 1 = "Almost None" and 5 = "Almost All"
72%
64%
55%
Figure 3. Program Chairs’ Reports of Faculty Support for EC2000 Initiatives
gram chairs, however, report an increase over the past decade on the emphasis given to teach-
ing in faculty hiring, promotion, tenure, and salary and merit decisions. In contrast, roughly
one-quarter of the faculty respondents believed the emphasis on teaching in their reward sys-
tems had decreased in the same time period. Senior faculty members, however, tend to report
increased emphasis on teaching in promotion and tenure decisions whereas untenured faculty
are more likely to report decreased emphasis.
Differences in Student Experiences
Have the program changes reported by chairs and faculty had a measurable impact on the
educational experiences of engineering undergraduates? The evidence suggests they have.
Indeed, the experiences of the 2004 graduates differ in a number of ways from those of their
counterparts of a decade earlier. The direction of seven of the 10 differences, moreover, is
consistent with what one would expect if EC2000 were putting down roots. Compared to
their 1994 counterparts, and after taking differences in graduates’ and institutional character-
istics into account, 2004 graduates reported:
� More active engagement in their own learning;
� More interaction with instructors;
� More instructor feedback on their work;
� More time spent studying abroad;
� More international travel;
� More involvement in engineering design competitions; and
� More emphasis in their programs on openness to diverse ideas and people.
Although they tend to be small, seven of 10 statistically significant differences between
pre- and post-EC2000 graduates persist even after adjusting for an array of graduate and insti-
tutional characteristics.
The exceptions are the absence of differences in instructor teaching skills and the hours
spent in cooperative or internship experiences, as well as the 2004 graduates’ reports of a
somewhat chillier diversity climate than that cited by their predecessors. The latter finding
may be related to several factors: differences in the gender and racial/ethnic mix in 1994 and
2004, graduates’ awareness of diversity issues, and/or their willingness to discuss and challenge
prejudice or discrimination. The evidence provides no guidance in the way of an explanation.
Differences in Learning Outcomes
Assessments of graduates’ skill levels on each of nine scales1 reflecting EC2000 Criterion
3.a-k learning outcomes are based on graduates’ self-reports of their ability levels at the time
of graduation (using a five-point scale, where 1=“no ability” and 5=“high ability”). A growing
body of research over the past 30 years has examined the adequacy of self-reported measures
of learning and skill development as proxies for objective measures of the same traits or skills.
When self-reports are aggregated to compare the performance of groups, they are generally
considered to be valid measures of the skills under study. Although results vary depending on
-6- Engineering Change
1 In statistical analyses, two of the 11 scales developed a priori to operationalize the a-k criteria collapsed into other
scales, leaving a total of nine measurement scales to reflect student learning.
the traits and instruments examined, these studies
report correlations of .50 to .70, on average, between
self-reports and such objective criterion measures as
the ACT Comprehensive Test, the College Basic
Academic Subjects Examination, and the Graduate
Record Examination. The original research design
called for comparison of graduates’ scores on the 1996
and 2004 Fundamentals of Engineering examination as
a measure of graduates’ content mastery. However, the
research team was unable to obtain permission to use those scores.
Figures 4 - 6 show the differences between 1994 and 2004 graduates’ reports of their
achievements on each of the nine scales reflecting the Criterion 3.a-k learning outcomes. In
all cases, the differences are consistent with what one would expect under the assumption
that EC2000 is having an impact on student learning. All differences, moreover, are statisti-
cally significant (p < .001), with effect sizes ranging from +.07 to +.80 of a standard deviation
(mean = +.36).2 Five of the nine effect sizes exceeded .3 of a standard deviation, an effect
size that might be characterized as “moderate.”
Engineering Change -7-
4.02
3.73
3.56
4.07
3.91 3.95
1.00
2.00
3.00
4.00
5.00
Applying Math
and Science
(Criterion 3.a)
Experimental
Skills
(Criterion 3.b)
Ap
本文档为【ABET-EC2000(EngineeringChange-executive-summary)】,请使用软件OFFICE或WPS软件打开。作品中的文字与图均可以修改和编辑,
图片更改请在作品中右键图片并更换,文字修改请直接点击文字进行修改,也可以新增和删除文档中的内容。
该文档来自用户分享,如有侵权行为请发邮件ishare@vip.sina.com联系网站客服,我们会及时删除。
[版权声明] 本站所有资料为用户分享产生,若发现您的权利被侵害,请联系客服邮件isharekefu@iask.cn,我们尽快处理。
本作品所展示的图片、画像、字体、音乐的版权可能需版权方额外授权,请谨慎使用。
网站提供的党政主题相关内容(国旗、国徽、党徽..)目的在于配合国家政策宣传,仅限个人学习分享使用,禁止用于任何广告和商用目的。