Plan for Assessment of Student Achievement at IUPUI

Submitted to:

North Central Association of Colleges and Schools

December 22, 1994

By the

Program Review and Assessment Committee  
and the  
Office of the Vice Chancellor for Planning and Institutional Improvement 
355 N. Lansing St., AO 140 
Indianapolis, IN 46202-2896 
317-274-4111

APPENDIXES A through I 
(Please contact Planning and Institutional Improvement at 274-4111 for appendixes)

PLAN FOR ASSESSMENT OF STUDENT ACHIEVEMENT AT IUPUI
Executive Summary

IUPUI is an urban university enrolling some 27,000 students that has no degrees of its own, but awards Indiana University or Purdue University degrees or certificates in 174 academic programs. Over its brief history--25 years--IUPUI has experienced remarkable growth; enrollment in the 1990s is 2.5 times that of the 1970s and the range of degree programs is broader than at any other institution in the state. IUPUI's growth has occurred principally as a result of the development of 17 relatively autonomous academic units, some with long histories as independent institutions in their own right within the city of Indianapolis and others with strong collegial and administrative links with their counterparts at Indiana University Bloomington and Purdue University West Lafayette.

Its almost unparalleled complexity, with allegiances to two other institutions and to strong disciplinary traditions, has made it unusually difficult for IUPUI to establish its own identity as a university. This difficulty was at the root of three principal concerns expressed by the NCA reviewers who visited in 1992. The team could not find evidence of a clearly understood institution-wide mission or set of purposes; it noted the absence of "a sound and coherent experience in general education that is monitored at the campus level;" and despite the presence of a campus-wide Assessment Committee dating from 1988, the team pronounced assessment at IUPUI "widely dispersed and uneven." The team directed IUPUI to develop a comprehensive plan for institution-wide assessment of student achievement by January 1, 1995.

Since 1992 progress has occurred in each of the areas that caused concern among the NCA reviewers. In that year a vice chancellorship for planning and institutional improvement, including responsibilities for assessment, was created and the first incumbent of that post was recruited. With leadership from the Chancellor and the new vice chancellor and participation by representative groups of faculty, students, and staff, a new mission and goals statement for IUPUI has been drafted. The contents of this document are beginning to influence school missions, curricula, assessment, and budgeting decisions.

Consensus on principles for campus-wide general education has not yet been achieved, but there are many signs that this broad agreement will occur in the next few months. The Faculty Council is currently considering a proposal that it empower IUPUI's first campus-wide undergraduate curriculum committee which can, among other matters, support the development of general education. The Program Review and Assessment Committee formed in 1993, with membership from all schools, has been successful in producing the first drafts of school plans for assessment of student achievement in general education and the primary field of study. Moreover, this committee has provided guidance for the implementation of IUPUI's first campus-wide peer review process for undergraduate and graduate programs.

Since mission development, agreement on general education principles, and assessment planning are all occurring simultaneously, it is difficult to predict future outcomes. Nevertheless, it is clear that IUPUI faculty and administrators have made a commitment to address all issues of concern to the 1992 NCA reviewers and that demonstrable progress has been made on these issues in the two years since their visit. Assessment planning is occurring in every school because this activity has a strong basis in the work of a representative faculty committee and strong administrative support from the Chancellor, the Dean of Faculties, and a vice chancellor for whom assessment is a primary area of responsibility. Assessment at IUPUI is proceeding according to a realistic timeline, and abundant evidence exists to support the expectation that assessment results will be used to suggest improvements in academic programs and student services.

PLAN FOR ASSESSMENT OF STUDENT 
ACHIEVEMENT AT IUPUI

Part I. CONTEXT FOR ASSESSMENT PLANNING AT IUPUI 

A. Brief History of IUPUI

IUPUI has just celebrated its 25th anniversary. Today IUPUI is a large public urban university with almost 27,000 students, 17 schools, and 174 academic programs. The Indiana Higher Education Commission has designated this campus as the state's principal site for graduate professional education.

Twenty-five years ago both Indiana University and Purdue University had small regional campuses in Indianapolis, the state's capital and largest population center. The two campuses were four miles apart. Purdue emphasized science and engineering and technology, while Indiana supported schools of medicine, dentistry, and nursing and several liberal arts programs. Many faculty were part-time, most administrative functions and decisions were handled by the home campuses, and most of the physical facilities were neglected hotels and office buildings. Classrooms were converted hotel rooms or offices.

Over the last 25 years a new campus has been built on a site near the center of the city. In 1993 the School of Science finally moved all its programs to the main campus. Now only the Herron School of Art remains in a facility remote from the central campus. A new library with multi-million dollar technological features was dedicated in Spring 1994. Sports facilities have been developed, with the venues for swimming, diving, tennis, and track among the best in the world. Academically, individual schools have developed very strong programs. External program reviewers say that in recent national searches IUPUI has begun to attract and hire some of the best faculty in the country.

Although IUPUI's first quarter-century has been one of remarkable growth--enrollment is now 2.5 times that in 1970--and extraordinary accomplishments, 25 years has not been a sufficient period in which to meld 17 schools representing two parent institutions into a unified university. The administrative structure is so complex that it almost defies explanation; it confuses new faculty and students as well as external audiences. The establishment of a clear identity for IUPUI, both on campus and in the external environment, is one of the paramount goals of the 1990s for this institution.

IUPUI's history has been characterized by the development of strong, relatively autonomous academic units, each with a long-standing tradition of independence. Many schools also have strong collegial and administrative links with their counterparts at Indiana University Bloomington and Purdue University West Lafayette. While the presence of academic programs from two different universities has given IUPUI an academic diversity and richness not possible at most institutions, it has also made the process of developing a clear identity more time- and work-intensive. Further exacerbating this situation, most of the schools have the additional pressures of trying to meet the academic demands of accrediting agencies. And for system schools like Nursing and Medicine, which have state-wide responsibilities, the academic agendas of seven other campuses must be accommodated by the faculty and administrators in Indianapolis.

Growth during these 25 years has been rapid but stepwise--occurring in phases. Every four or five years, the faculty's conception of the institution changes, often coinciding with the completion of one building project and the start of another. Just as faculty seem to grasp the possibilities of their current status, the external and internal forces of change are creating yet another definition of IUPUI.

All of these dynamic forces come together and are evident in the most important process of the university, developing a mission statement. At a university with the complexity and youth of IUPUI, crafting a mission takes many months and even years if it is to come, as it should, from the roots of the university: the faculty, the students, the staff and the community. In contemplating the pattern of its development over the next decade--the beginning of its second quarter-century--IUPUI is taking a fresh look at its mission and goals. Simultaneously, discussions aimed at creating some campus-wide conceptualization of general education, as opposed to a collection of individual school plans, are taking place. Complementing these two major undertakings is the development of a campus-wide plan for assessing student academic achievement.

The developing story of the interweaving of mission, general education, and assessment in a new institutional fabric at IUPUI will be told in Part II of this report, the Assessment Plan. But four additional pieces of context are essential to a full understanding of the way assessment has evolved and is emerging at IUPUI. The first, Section I.B, is a brief history of campus assessment initiatives prior to 1993. The second, Section I.C, is the series of observations and recommendations reported to the campus by the North Central Association (NCA) accrediting team following its visit in November 1992. Finally, Sections I.D and I.E describe developments currently taking place at IUPUI with respect to the evolution of its mission and the emergence of a campus-wide consensus about general education.

B. Brief History of Assessment at IUPUI

In 1988 the Dean of Faculties at IUPUI established an Assessment Committee under the auspices of the Office of Faculty Development. The Associate Dean for Faculty Development first convened the Committee, comprised of faculty and/or administrative representatives from each of the schools, and then served as staff to the group once a faculty chair was selected.

The Assessment Committee reviewed current assessment literature and sent representatives to national and regional conferences on the topic. The members of the committee also began the process of educating other IUPUI faculty about assessment. A newsletter was issued periodically and mailed to all faculty. Faculty development workshops on various topics were held. The IUPUI committee was strengthened by affiliation with a system-wide assessment initiative staffed by a special assistant to the President of Indiana University.

IUPUI Assessment Committee members decided that the best way to encourage faculty colleagues to undertake assessment for their own reasons was to offer a series of small grants awarded competitively. The Dean of Faculties set aside $15,000 annually for assessment projects and publicized the availability of these funds through announcements issued by the Office of Faculty Development. A listing of these projects and an example of one project report that was published as a journal article appear in Appendix A.

C. Observations and Recommendations of the 1992 NCA Visiting Team

The report developed by the NCA visiting team following its study of IUPUI in November 1992 identified four areas of concern that have subsequently received considerable attention on campus: mission, general education, assessment of student achievement, and assessment of graduate programs.

The team's first concern was related to IUPUI's mission. In their report the reviewers observed that "administrators, faculty, staff and community supporters with whom team members spoke seem to share a broad vision of IUPUI's potential as an institution of higher education . . . but the vision itself has not yet crystallized into a single statement of purposes that can be broadly promulgated and used to define the institution to its various constituencies" (p. 11). In addition, the team noted that "IUPUI will continue to be disadvantaged in important ways until it develops more fully and articulates more clearly an institution-wide mission and a set of purposes consistent with that mission" (p. 12).

The visiting team devoted several long sections of its report to its second area of concern, general education at IUPUI. Specifically, the team suggested:

The process of building a consensus or reaching accommodations about what general education should mean at IUPUI will help the university define and make known the special objectives of the education it provides for undergraduate students. Once those objectives are clearly defined, it will be possible for the institution to undertake a meaningful assessment of the extent to which students are attaining them. Until IUPUI's objectives in general education are clarified, systematic assessment of student attainment in general education will be impossible, even on a school-by-school basis. (pp. 8-9)

Furthermore, the report stated that "if IUPUI is to produce and implement a program of general education--campus-wide or school-by-school--that is subject to meaningful outcomes assessment, discussion should be focused and concepts tightened. Administrators and curriculum committees of IUPUI's undergraduate schools should be brought into the process so they can influence the nature of the discussions and the conclusions that flow therefrom" (p. 9). Finally, the team recommended that "recipients of bachelor's degrees from all IUPUI schools (should) have a sound and coherent experience in general education that is monitored at the campus level . . ." (p. 30).

Assessment of student academic achievement was the third area of concern identified by the NCA reviewers. Team members acknowledged the progress in advancing assessment achieved by the IUPUI Assessment Committee appointed in 1988 as follows: "A great deal of important groundwork has been accomplished by informing faculty about the importance of assessment initiatives and exposing them to a variety of assessment methods" (p. 21). However, the team also noted that at IUPUI "the assessment effort is widely dispersed and uneven" (p. 21). The team therefore recommended that IUPUI "file a report by January 1, 1995 on the continuing development of its program to assess student academic achievement" (p. 33).

Finally, the NCA team found "no program for regularly assessing the effectiveness of (IUPUI's) academic programs" (p. 22). Specifically, the report pointed to the assessment of graduate programs as its fourth concern.

We are particularly concerned about the absence of a regular assessment plan for graduate programs that are not subject to review by professional accrediting agencies and operate with a large measure of independence in Indianapolis. It is not that we have doubts about the quality of these programs, but that we feel it is important for their quality to be assessed and validated periodically by objective parties. (p. 22)

The team recommended that IUPUI "develop a procedure for periodic review by graduate faculty on the Indianapolis campus, with the assistance of disciplinary experts from other universities, of graduate programs offered by IUPUI and covered by IUPUI's NCA accreditation" (p. 29).

Sections I.D and I.E and Part II below summarize the actions and progress that have taken place since 1992 in each of the four areas of concern identified by the NCA visiting team. The current status of developments related to the mission statement and campus-wide goals for general education is characterized in Sections D and E of Part I. The Assessment Plan is described in Part II, and Subsection C.2 provides highlights of the new centrally-coordinated process of peer review of both undergraduate and graduate programs that is now in its second year of implementation. 

D. Current Status of Mission Development at IUPUI

The first task in providing a foundation for an assessment initiative is to ensure that the mission statement reflects the current state of the institution. Given its history, IUPUI has experienced several redirections of mission. The most recent was presented in the document IUPUI Development Plan 1988-2000 , which was referred to in the report by the NCA visiting team. In 1992 IUPUI created a new office to coordinate institutional planning, assessment, and program improvement and hired a vice chancellor to direct it. The new vice chancellor, Trudy Banta, initiated upon her arrival a planning process involving collaborative work with administrators, faculty, student representatives, and a group of community advisers to articulate yet another reformulation of IUPUI's mission statement. This will become the "institution-wide comprehensive statement of purposes . . . (that) sets forth clearly and unequivocally . . . things the university as a whole intends to do ( NCA Report of the Visiting Team , 1992, p. 11).

The planning process began in Fall 1992 with a series of six three-hour sessions on quality improvement concepts provided to the Chancellor's staff by an external consultant. In the course of these sessions, a draft of a new campus vision, mission, values and goals emerged. Preliminary drafts were shared with members of the Faculty Council, Council of Deans, Administrative Council, Staff Council, and student government. Reviews by these groups convinced the Chancellor and vice chancellors that a broader campus dialogue was needed to achieve consensus on mission and goals.

In Fall 1993 the school deans and campus vice chancellors were brought together under the leadership of the Chancellor to engage in campus-wide planning. The current working draft of the mission statement (see Appendix B) has emanated from the deliberations of this group. Once again the draft has been the topic of focused discussions by appropriate groups of faculty, students, staff and community members. In October 1994 the text of this draft was delivered via electronic mail to every faculty member with a request for comment. The suggestions collected from these groups are being reviewed, and the mission draft will be revised accordingly in January 1995. 

E. Current Status of the Development of General Education Goals for IUPUI

Concurrent with the effort to develop consensus for a new, more focused mission statement clearly understandable by faculty, students, staff, and external publics, various initiatives have been undertaken to define general education for the campus. In 1991 the Council on Undergraduate Learning and the Faculty Council appointed a Commission on General Education to develop this component of the mission of IUPUI. Previous attempts to create a campus-wide general education program had failed to produce consensus among the faculty. The Commission examined these previous efforts, which essentially proposed core or distribution approaches to general education. In a bold move, the Commission adopted as an alternative a "process" approach to general education. The essential feature of this approach is initiating and nurturing a conversation about general education among all of IUPUI's stakeholders: faculty, students, and the general public in central Indiana.

The Commission held three all-day retreats for faculty, one in each of three academic semesters, and involved faculty in smaller working groups for two additional semesters. Based on the input from the 300+ faculty involved in these activities, the Commission forged a plan for general education at IUPUI. The central feature of this plan was a set of eight Principles of General Education. These principles were conceived as starting points for discussion among faculty in each of the schools (see Appendix C).

This process approach to general education has been successful in generating discussion about the foundations for undergraduate education at IUPUI. As a result of the activities just described, as well as an unmoderated electronic mail discussion, nearly 1,000 faculty have been engaged in some conversation about the eight principles. The idea of having principles of general education has been endorsed by most faculty groups. But Commission members had hoped that extensive faculty involvement in the evolution of the eight principles would assure acceptance of them. Instead, discussion of general education across campus has brought to light pockets of substantial disagreement. It is clear that broad acceptance of the principles must include not only school-based discussions and decision-making but also consensus on the part of a campus-wide body.

Following a suggestion of one of the NCA visiting team members, the Dean of Faculties initiated in Fall 1994 a plan for advancing the general education discussion that required cooperation between two pivotal schools, Liberal Arts and Science. An unprecedented level of dialogue between these two schools on the subject of undergraduate education has ensued. Although the discussion is not complete, these two schools have preliminarily adopted a set of five principles which are similar to five of the eight principles of the Commission (see Appendix D).

At present there is no undergraduate curriculum committee at IUPUI; therefore, no group can approve any set of general education principles as a cornerstone of the undergraduate program. Under debate is a proposal that a newly-constituted Council on Undergraduate Learning will become the group that makes campus-wide curricular decisions and monitors general education activities. The Faculty Council is considering a resolution to empower this group with such responsibilities. The proposed Council on Undergraduate Learning would include administrative and faculty representatives from each school. The situation with respect to graduate programs is somewhat different. A committee currently exists, the Graduate Council, which oversees curricular matters at the graduate level.

Prior to this most recent effort to develop a campus-wide general education program, general education requirements have been set by the 13 individual schools at IUPUI that offer undergraduate majors. As might be expected, each school has a different view of what general education should be. The courses specified in each set of distribution and core requirements have been compiled by the Undergraduate Education Center in a matrix that is distributed to all incoming students in an undergraduate education manual (see Appendix E). The matrix is an expedient to assist students in choosing introductory courses, but does not serve to inform students of the broad purposes of a curricular foundation in general education. Establishment of a campus-wide set of principles and identification of school-specific and campus-wide implementation strategies will contribute to a clearer definition of general education for students as well as other IUPUI constituents.

--------------------------------------------------------------------------------

Part II. THE ASSESSMENT PLAN

A. Mission, Scope and Stages of Assessment

1. Mission and Scope of Assessment

The new IUPUI mission and goals statements (see Appendix B) are currently in draft form and revised working drafts are scheduled for completion in January 1995 (see Part I, Section D). To paraphrase the new mission statement, in fulfilling its role as an urban university, IUPUI intends to "raise educational achievement and intellectual aspiration" in Indianapolis and the state, to offer "effective academic programs" that feature "partnerships with Indiana University and Purdue University and the community" and to draw upon "the distinctive strengths . . . of the capital city and state." Among IUPUI's goals are those of "Providing student academic and support programs that serve the needs of a wide array of learners and promote their ability to engage in life-long learning" and "Having all units committed to continuous improvement of academic programs, student services, and administrative activities."

These excerpts from the IUPUI mission and goals statements support an assessment program that begins in the first year of the student's experience with the measurement of academic readiness via placement testing, continues with the tracking of student progress in general education and the major, and includes monitoring of community involvement to promote learning on the part of students and graduates. In addition, the goal expressing commitment to continuous improvement should help to ensure that assessment data are used to suggest directions for program modification and improvement.

Part II, The Assessment Plan, begins with a characterization of the status of assessment planning with respect to measuring student achievement in general education (Section B) and in undergraduate and graduate majors (Section C). Subsequent sections describe on-going campus assessment efforts which illustrate the uses being made of assessment data to improve activities in the three areas of programming just identified: first-year experiences (Section D), general education and the major (Section E), and community involvement designed to support learning (Section F). Section G contains an explanation of the campus structures that support assessment at IUPUI. The Plan concludes with Section H, Evaluation of the Assessment Plan and a summary statement (Section I).

2. Stages of Assessment

Concomitant with IUPUI's focus on mission and goals has been an equally important focus on developing an assessment plan. An assessment plan, in general, may be viewed as progressing through three stages. In Stage 1, the concern is with getting started: How can a plan be developed? In Stage 2, the details of the plan are worked out, including specifications for the kinds of data to be collected. The third stage is one centered on evaluation and use: How successfully was the plan implemented? How good is the program, based on the data? What decisions or changes are made on the basis of findings? The assessment planning process is iterative since the evaluation which takes place in Stage 3 may lead to reconceptualization of the plan developed in Stage 1.

IUPUI programs and departments are at various stages of the assessment planning process. The status of each of the assessment components described below will be noted.

B. Assessment of Student Achievement in General Education

In the area of general education, IUPUI is currently at Stage 1, putting an assessment plan together. To complete Stage 1, three steps are required:

1. Identify the principles of general education.
2. Determine how these principles will be implemented. That is, what are the processes of general education?
3. Establish measures and procedures that can be used to evaluate the implementation strategies, processes, and achievement of principles.

1. Principles of General Education

The first task is to achieve agreement on the principles of general education across all schools. During the Fall 1994 Semester each school has been asked to consider the eight principles from the Commission (see Appendix C), the five principles from the Schools of Liberal Arts and Science (see Appendix D), and any work that has been done by other schools. From this information, the faculty of each school should endorse one of the existing statements of principles and/or develop its own. At the same time, each school and department is developing a statement of learning outcomes for each of its undergraduate and graduate degree programs and demonstrating how these goals at the undergraduate level encompass principles of general education.

During the 1995 Spring and Fall Semesters, the principles from each school will be compiled. Table 1 provides a timeline for activities associated with the further development of general education programming and assessment. The Council on Undergraduate Learning, which includes faculty representatives from each school, will develop a statement of principles for IUPUI.

2. Processes of General Education

Aspects of the IUPUI mission statement are reflected not only in the general education outcomes that are identified, but also in processes or strategies for implementing the outcomes. Thus, the processes of general education reflect the mission of the university and shape the course of general education. The Commission has proposed two such processes: collaborative learning and service learning. Campus efforts have identified two others: first-year and senior capstone experiences. In addition, each school may specify its own processes.

During Spring and Fall 1995, the processes from each school will be compiled by the proposed Council on Undergraduate Learning, which will, through an interdisciplinary collaborative process of decision-making, develop a set of processes for IUPUI. Following review and approval of this set of processes by individual school faculties, the Council on Undergraduate Learning will recommend a final version.

Table 1. Timeline for Assessment at IUPUI

Spring and Fall 1995 Steps to Complete Units Responsible 
General Education Develop campus-wide consensus on general education principles 

School Faculty 
Council on Undergraduate Learning (CUL) 
Faculty Council 

Develop implementing processes for general principles School Faculty 
CUL 
Faculty Council 

Undergraduate Majors and Graduate Programs Provide campus workshops, consulting visits to schools to clarify meaning of assessment School Faculty 
Program Review and Assessment Committee (PRAC) 
Vice Chancellor Banta 

Develop concrete plans for assessing student achievement in every academic unit; ensure that assessment data are used in improvement in schools already conducting outcomes assessment School Faculty 
PRAC 
Vice Chancellor Banta 

Spring 1996
Steps to Complete Units Responsible 
General Education Develop campus-wide implementation strategies and measures of assessment for general education principles School Faculty 
PRAC 
CUL 

Undergraduate Majors and Graduate Programs Begin to apply assessment measures in every school/department School Faculty 
PRAC 
Vice Chancellor Banta 

Fall 1996 and continuing
Steps to Complete Units Responsible 
General Education Administer campus-wide assessment measures in general education and begin to make improvements in curricula and methods of instruction as warranted School Faculty 
PRAC 
CUL 
Vice Chancellor Banta 

Undergraduate Majors and Graduate Programs Monitor use of assessment data in improvement School Faculty 
PRAC 
CUL 
Vice Chancellor Banta 


3. Measurement of Student Learning Outcomes Based on General Education Principles

During Fall 1995 and Spring 1996, each school will identify any methods that are currently used to evaluate its general education principles and will generate additional measures as appropriate. The methods of the several schools that have already started this process can serve as examples for others. As each school generates its assessment measures, the results will be shared with other schools. During Spring 1996 and following, representatives from each school will meet to identify assessment measures common to all schools and make recommendations for campus-wide measures in general education. These recommendations will then be taken to the Council on Undergraduate Learning for campus-wide endorsement. In addition, each school and department will be asked to review the set of measures for its major(s) and show how these measures fit with the campus assessment procedures for general education.

C. Assessment of Student Achievement in the Undergraduate Major and in Graduate Programs

1. School Assessment Plans for Undergraduate and Graduate Programs

a. Developing assessment processes at school and campus levels. Assessment of student learning, a campus-wide priority since 1988, continues to be an important activity for all academic and professional schools at IUPUI. Over the past six months faculty in all schools have begun to draft plans for assessing student achievement in their 2-year, baccalaureate, and graduate degree programs. Excerpts from these plans are included in Appendix F. Table 1. provides a timeline for further development of the school plans. In these plans faculty have identified student outcomes that are consistent with school mission and philosophy statements; program goals; professional accrediting standards and employer expectations when appropriate (e.g., Allied Health, Nursing, Pharmacy, Social Work); and the proposed principles of general education (see Appendixes C and D). Assessment is in a state of evolution in every academic unit. Some schools and departments are struggling at Stage 1 while others have reached Stage 3 in their assessment planning.

All schools have selected persons to be responsible for assessment activities. Some schools have designated or added administrative leaders. For example, in 1992 the School of Nursing established the Office of Program Evaluator and in 1994 the School of Social Work added a new associate dean for quality improvement. Many schools have convened assessment committees: examples include the Evaluation Committee of the School of Education, the Teaching Standards Committee of the School of Journalism, and the Program Review and Assessment Task Force of the School of Allied Health.

During the last two years much time and energy have been given to the development, refinement, and implementation of a formal, standardized assessment plan for the campus. Faculty identified as leaders, facilitators and coordinators in developing and implementing their own school's assessment plan also represent their school on the campus-wide Program Review and Assessment Committee (see Section II.G2). This committee is charged to integrate the schools' assessment efforts into an assessment mission for the campus which shapes campus-wide assessment activities that, in turn, support needs of the schools.

b. Building campus-wide consensus and communication. An examination of the school assessment plans and the content of discussion about general education suggests that faculty in all schools are in philosophical agreement about the need to assure that students graduating from IUPUI will have developed skills in the areas of critical thinking, communication, discipline-specific knowledge and competence, cultural sensitivity, and informed citizenship. Each school faculty has incorporated these outcomes in plans (as in Appendix F) that are under development or refinement. Although the faculty of each school are charged with the responsibility for defining the parameters of the above outcomes, the high level of consistency in student performance expectations bodes well for the future of campus-wide general education principles. A specific example about critical thinking is demonstrative. The School of Liberal Arts faculty assesses critical thinking from the perspective of "moving from an expressive to a critical mode in textual analysis," "recognizing ambiguity," and "developing an ability to value various critical approaches." The School of Dentistry plan discusses critical thinking as students' ability to "gather and analyze relevant data" and "formulate treatment plans [consistent with findings]" and to "apply basic biological principles to the practice of clinical dentistry." The School of Business faculty requires graduates to be able to "apply analytical tools to problem solving" and "apply concepts from an area of curricular concentration" to real life problems. Clearly these schools have similar expected learning outcomes for students in the area of critical thinking.

Faculty also have identified some similar categories of competence within the major. For example, the School of Education looks at professionalism, knowledge, and field-based skills; the School of Dentistry, at professionalism, patient care, and practice administration; and the School of Physical Education, at professionalism, content mastery, and application. As faculties continue to identify instructional strategies and assessment measures for these competencies, the campus may well be able to find common ground across majors. Campus-wide assessment through the kind of capstone experience already proposed by one campus-wide committee may be an outcome.

The sharing of individual school outcomes in the Program Review and Assessment Committee (PRAC) ensures cross-fertilization of ideas. During Spring 1994 campus workshops and consulting visits to schools by PRAC members, including Director of Assessment John Kremer, Vice Chancellor Trudy Banta, and other faculty members from the 1988-93 Assessment Committee, will further understanding of assessment methods and their uses in program improvement. As this process continues to be refined, it should yield a rich set of student learning outcomes both within and across schools. It is anticipated that this sharing will result in internalization of expectations that are valued both by faculty and students regardless of school allegiance.

c. Gathering assessment data. Plans for identifying what data on student achievement will be collected and how it will be collected are as individual as the faculty, the schools they represent, and their students. However, some common themes in the steps for gathering information are described in the school plans. In general, data collection is occurring in a progressive fashion within and at the end of courses and periodically at the completion of a certain set or core of courses. Many schools (e.g., Nursing, Dentistry, Public and Environmental Affairs, and Education) have end-of-program measures including capstone evaluation experiences, final projects or papers, licensure or certification examinations, and thesis or dissertation projects. Course assessment methods include tests, written projects and oral presentations. The list is much more exhaustive than presented here, although these activities seem to be among the more popular, regardless of degree program and school.

Materials in Appendix F demonstrate the scope and complexity of assessment practices at IUPUI. End-of-program measures are employed primarily in programs that are externally accredited and/or require graduates to sit for licensure or certification exams. Faculty in more traditional academic disciplines are in the process of exploring which end-of-program measures are essential in examining comprehensive learning outcomes and which methods of data collection provide the most effective means of review.

d. Using assessment data. How the faculty in different degree programs are using assessment information in continuous improvement of their programs is the least developed step in any school's assessment plan. However, two schools have used assessment information to revise curricula after carefully considering the practice needs of their graduates for the 21st century. The School of Nursing faculty is in the process of revising its ASN, BSN and MSN programs and proposing the replacement of its DNS program with a Ph.D. program. The School of Dentistry faculty, having rethought the mission of the dentistry program, is "completing the final revisions in the list of competencies students will be required to have at graduation." This activity was initiated following a site visit in Fall 1992 by the Commission on Dental Accreditation of the American Dental Association.

Despite the diversity in mission, philosophies, goals, faculty, and students within and across schools, assessment planning and implementation are becoming an integral part of life on the IUPUI campus. The efforts to merge assessment and evaluation activities have not only allowed faculty and students to re-examine their program missions, but have enhanced a broader understanding of and support for the diversity that constitutes the very essence of IUPUI.

2. Centrally-Coordinated Peer Review of Undergraduate and Graduate Programs

Worldwide, the evaluation strategy that has gained widest acceptance among academics is peer judgment. Time and again it has been demonstrated that faculty and administrators pay attention to suggestions for improvement made by respected peers. Utilizing peer judgment in program review is thus the overall strategy at IUPUI for outcomes assessment in the major field. A working draft of the guidelines for the review process, which was inaugurated in Fall 1993, is included in Appendix G.

Over a five-year period, all IUPUI schools and departments are scheduled for an academic review (see Appendix G for review schedule). Each department develops a list of potential internal and external reviewers and prepares a self-study. The self-study is sent to the reviewers prior to their 2½-day site visit. This visit concludes with an oral overview of the department being reviewed and is followed by a full written report to the department, the dean of the school in which the department is located, and the Chancellor.

The guidelines for the self-study ask for a delineation of the educational goals, processes, and assessment methods for all graduate and undergraduate programs offered by the department. The guidelines emphasize community involvement, building collegiality, and attention to student outcomes. As the IUPUI mission and accompanying goals and implementation strategies are developed and disseminated, program review will also address the congruence between the university mission and instructional programs, research, and service activities in the schools and departments.

Thus far, program reviews have been conducted in three academic departments and one service unit. Two more departments are scheduled for Spring 1995. A conscientious follow-up process ensures that the principal recommendations made by reviewers on the basis of their comprehensive study of the academic units will be implemented.

D. Uses of Assessment Data in Improving First-Year Experiences

Coordinated campus-wide assessment of student achievement in general education must await the identification of campus goals or principles. With respect to the development of assessment in the discipline, many campus departments and schools are still at Stages 1 or 2 of their planning--still working on measurable comprehensive learning outcome statements or identifying appropriate assessment strategies. But other campus initiatives already provide clear evidence that IUPUI is committed to applying assessment data to improve its programs for students; these initiatives are at Stage 3.

1. Undergraduate Education Center

The Undergraduate Education Center (UEC) is the academic unit that enrolls IUPUI students who do not have an academic major or who have not met the requirements to enter the major of their choice. The UEC serves approximately 7,000 of the 18,500 undergraduate students on campus. Its purpose of helping students select a major and meet the requirements for entry is accomplished through advising and a variety of support programs for students.

a. Advising. On their own, students may advise themselves through the computer-based Student Advising System. In addition, students must see an advisor for counseling. To help in advising, counselors have available on-line or batch-mode audits of students whom they are counseling. Counselors typically see from 50-150 student advisees per month in scheduled or walk-in sessions. After counseling, students may enroll at their convenience using touchtone registration. Prior to entering classes students participate in orientation sessions.

Process evaluation is part of the daily operations of the UEC. Several series of reports evaluate critical advising functions: effectiveness of the computer advising system, errors in counselors' advising, errors in computer audits, problems with touchtone registration, and new students' experiences with orientation and registration. The data on counselors' advising sessions includes: average waiting period of walk-ins, number of appointments per counselor, and number of students at orientation sessions.

b. The Student Mentor Program. This program has two components: the Peer Support Center (PSC) and the Learning Assistance Program. The PSC assists students in academic, social, and personal areas. The PSC may provide a tutor, a reference for a personal problem, or a study buddy. Process evaluation data include participation rates. In the Learning Assistance Program student mentors facilitate group learning in small, collaborative, weekly sessions. The focus is on study skills, learning styles, subject content, and building a strong community at IUPUI. The mentors are used for the seven most popular (percent of UEC students enrolled) and most difficult (percent D/W/F) courses on campus. This program is modeled after a similar program at the University of Missouri at Kansas City. Process evaluation data include participation rates. Outcome evaluation involves conducting a study of the effectiveness of this program. Students who have participated in sections taught by mentors have performed better (though not significantly so) than students taught by full-time faculty. Retention data for two subsequent semesters are currently being examined.

c. Learning Communities. Small groups (N=20) of new students are formed for the purpose of enabling the students to take three introductory courses together: writing, reading, and introductory psychology. Faculty teaching these courses occasionally meet together and some attempt is made to link the content of these courses. Outcome evaluation includes examining the GPAs of these students and retention rates.

d. Cornerstone Experience. This program offers IUPUI students who have 18-plus credit hours and a GPA of less than 2.0 the opportunity to take introductory psychology or a basic level math course. The program offers supportive courses (mentoring or study skills) along with the math or psychology course. Outcome evaluation data include scores on exams compared to other students taking the course. The data indicate that the program is effective.

e. Square One. This program assists 40 students who do not fully satisfy present IUPUI admissions standards, but are admitted conditionally. Students complete an academic skills preparatory program which helps them make a successful transition into a regular academic program. Outcome evaluation data include a comparison of pretest and posttest data on the IUPUI placement exams and the number of students who eventually enroll in the next level of courses, especially Math 001. These students are also tracked throughout their educational process.

f. Overall evaluation. The director and staff regularly examine data that relate to the routine operation of UEC programs. Process data include the following: percent of students with an undeclared major that are in the UEC, percentage of students enrolling in subsequent semesters, the overall failure rate of UEC students, percent of students with GPAs below 2.0, number of dismissals, percent of students making the honor lists, percent of exploratory students, and graduation rate 10-18 semesters following graduation. One indicator of effectiveness is that all but one of the highest undergraduate awards at IUPUI have been given to individuals who were at one time UEC students. A total of 30 students has achieved this award over 5 years.

2. First-Year Courses in Mathematics

The work on first-year courses in mathematics, writing, and psychology is at Stage 3 of assessment planning. Several outcome measures have been developed and resulting data have been used in making significant course changes.

The Department of Mathematical Sciences at IUPUI has instituted a number of assessment efforts directed at Developmental/Introductory mathematics courses. Three major techniques have been used to examine the Developmental/Introductory curriculum: (1) course-by-course analysis of success rates of students; (2) tracking of student progress through the Developmental curriculum; and (3) various strategies to measure and foster instructional quality. The brief synopsis which follows highlights a few examples of the ways in which these assessment activities have been used to improve student learning in mathematics.

All entering IUPUI students (except those with transfer credits in mathematics from another postsecondary institution) are required to take the mathematics placement examination. Based on the results, students are placed in various mathematics courses.

Over the years, faculty in the department have charted the success and failure rates of students in the various mathematics courses. Based on information from these success and failure rates, several changes have been made in the curriculum, including the addition of Developmental/Introductory courses. For example, faculty created MATH M010 (Pre-Algebra) in 1988 in response to the high student failure rate of 76% in MATH 001 (Introduction to Algebra). During the first year following the introduction of MATH M010, the failure rate in the follow-up course, MATH 001, dropped to 58%. By 1993, the failure rate had dropped to 43%. In fact, longitudinal tracking of students who completed MATH M010 suggests that even those students who were most in need of remediation were successful in the next two courses in the Developmental Mathematics curriculum.

An examination of data for MATH 111 (Algebra), the last of the three courses in the Developmental curriculum, has also demonstrated improvements in student completion rates and final examination scores. The withdrawal rate has decreased from 42% in 1988 to 28% in 1993. At the same time, student mean scores on the MATH 111 departmental final have steadily increased, while the standard deviation across different sections has decreased.

Similar student tracking has been conducted across the many Introductory mathematics courses. In general, tracking data suggest several courses in which improvements have been made (e.g., MATH M118, Finite Mathematics) and courses in which the current student failure rates are too high (MATH 151/153-154, Algebra and Trigonometry I & II; MATH 163, Integrated Calculus and Analytic Geometry I). The department is continuing its efforts to find solutions to the problem of high failure rates.

Another strategy used by the department to examine student learning is longitudinal tracking of student outcomes. Longitudinal tracking was implemented in order to ensure a seamless curriculum for students. Two separate groups of students have been tracked. One group consisted of 157 students who entered the University Access Center--now part of the Undergraduate Education Center (UEC)--in 1988; the second group included 294 students who entered the UEC in 1991. Students in both groups were tracked for a period of three years using a wide variety of data, including: SAT scores, placement test results, counseling decisions, mathematics course grades, chosen major, and demographic information.

In general, findings from longitudinal tracking suggest that: (1) students starting with and passing the first of the Developmental courses (MATH M010) have a higher probability of passing the next Developmental course as compared to students who are placed directly in the last Developmental course; (2) success rates for students climb as they progress through the Developmental mathematics curriculum; (3) a change from pass/fail to A/B/C/F grading in all Developmental courses has resulted in increased student motivation to perform at higher levels (as demonstrated by tracking the 1991 cohort); and (4) students in the 1991 cohort averaged slightly higher success rates (though not statistically significant) when compared to the 1988 cohort, yet fewer students in the 1991 cohort attempted the full sequence of Developmental courses within the three-year period.

As a result of data collected from longitudinal tracking, faculty have concluded that the Developmental mathematics courses are well articulated. Additionally, the findings suggest that the change from a pass/fail to graded system has had the intended effect of raising student motivation for higher levels of performance.

Finally, a number of strategies have been introduced to ensure instructional quality. One technique is based on student performance on departmental finals for courses. Student mean scores and withdrawal rates are compared across classes and instructors, most of whom are part-time. The results are shared throughout the department. Those instructors with the lowest average student performance scores and highest withdrawal rates are asked to improve their teaching. Those instructors whose performance does not improve, as illustrated through student performance, are not rehired.

Another set of techniques for ensuring instructional quality consists of peer, student, and instructor evaluation of classroom instruction. Peer evaluation of teaching has already been established (along with a mentor program) for all junior faculty (see Appendix H). A new student evaluation of instruction instrument has been designed and will soon be voted on by the faculty. And finally, faculty will be encouraged to provide their own course assessments at the end of each semester (see Appendix H).

3. First-Year Courses in Writing 
a. Placement. When placement test readings indicated a wide range of student abilities at the low end of the scale, the Department of English developed E010, a developmental writing course for students with the lowest ratings. The success rate of E010 students when they retake the placement exam is high, as is the success rate in the first course after E010. In addition, because the placement test was not discriminating among students at the top end of the scale for exemption and W140 (Honors) placement, the Department instituted a portfolio submission for top end placement. This change resulted in more accurate designation of Honors students and, therefore, more successful W140 classes.

b. Course evaluation. In 1988 the Department of English instituted portfolio assessment of E010, W001, and W131, three elementary writing courses. After a pilot year in which writing faculty conducted practice readings and ratings of individual papers versus portfolios, rubrics were developed for excellent, acceptable, and unacceptable performance. Rubrics were based on student learning outcomes devised by faculty for common syllabi in these courses. After individual evaluation by faculty, portfolios are rated at end-of-semester teaching team meetings. Each portfolio brought to the meetings is read by at least two faculty members.

Information gleaned from portfolio readings influences continual revision of course content and pedagogy. In 1994, for example, portfolio reviewers noted a need for more attention to specific features of academic writing; the W131 curriculum was subsequently revised. Also students consistently performed less well on one departmentally-mandated paper assignment, which was rewritten and emphasized in faculty development workshops. Each year curricular changes are made based on student learning demonstrated in course portfolios. Curriculum is rewritten by a team of faculty from the Writing Coordinating Committee and associate faculty who teach the courses being revised.

4. First-Year Courses in Psychology

In 1992, an analysis of data on Introductory Psychology indicated that the traditional lecture format of teaching this course to underprepared students (average SATs less than 800) was not working. The data included 40% D/W/F rate, 30% attendance during the weeks of class, low student satisfaction with the course, and highly variable instructor grading patterns.

To change this situation, the course was redesigned. This design included development of systematic ways to measure student outcomes, support for teaching innovations, and faculty development.

The course now includes five common exams across all sections. Students take these exams on computer at their convenience anytime during a week. Student attitude measures are given at the beginning and end of the course. These measures include: self-esteem, degree of involvement, acceptance of technology, and perceived academic support. Each of these four areas is measured at three levels: the course, the university, and the community. Data over the past two years indicate that the measures have adequate reliability, and some validity has been established. Pretest and posttest measures have provided information on how the course is working and how it can be changed. Because of continuous changes in the tests and course, comparison of the traditional and new methods of instruction have not been possible. Nevertheless, the data have been useful in identifying effective teachers and strategies. In addition, the most successful approaches have been communicated to all faculty and some have been implemented course-wide.

To facilitate learning, a common set of course objectives has been established along with department-wide testing. Learning is facilitated through computerized testing, videotaped lectures, and the use of study guides and practice tests. Information about student use of study guides and practice tests is given to faculty. Process evaluation data are routinely collected on the number of lectures watched and number of chapters read. These data provide instructors with information on student motivation and success of course interventions to increase study behavior. Regular focus groups are held with counselors in the UEC, mentors, and teachers in the learning community sections (see Section II D.1c) to discuss improvements in the courses.

5. Placement Testing

Assessment of the entry level reading, writing and math skills of IUPUI undergraduates is well established at IUPUI. With respect to assessment planning, this program is clearly at Stage 3.

The rationale for placement testing at IUPUI is threefold. First, students who enroll in appropriate levels of courses should have a more positive experience than those who enroll in courses that are either too difficult or too easy. They should be more satisfied with their college experience, and therefore more likely to persist in their studies. Second, because students are more likely to stay in classes that are appropriate to their ability level, departmental administrators can more carefully plan how to allocate faculty resources to class sections. Finally, the placement exams may in the future serve as one of the bases for measuring the contributions of the university to the development of students' general education skills.

The math test is a departmentally-developed objective instrument that assesses skills ranging from basic operations to introductory calculus. IUPUI's reading test consists of the Nelson-Denny, Form E, an objective reading assessment from the Riverside Publishing Company, and consists of three parts--reading speed, comprehension, and vocabulary. The English written essay is a holistically-scored writing sample based on a rotating set of prompts developed by faculty from the English Writing Program. These prompts are designed to focus students' discourse on a topic in the realm of their normal experience. Students are given an hour to brainstorm, organize, and write an essay that is usually about 500 words in length.

Based on careful monitoring of the process over several years, a number of improvements to the current placement testing procedures were proposed by the Testing Center in June 1994. For math, it was recommended that the 40-item linear test be transformed into a computerized adaptive test drawn from a 168-item bank. The adaptive test should produce more variable results within each class placement and should be more precise than the non-adaptive test. Moreover, a recommendation was made to use an outcome that was more variable than course grades while still measuring the same domains. The use of a common final exam for those courses that are linked to placement testing has been implemented on an experimental basis.

For Reading, a recommendation was made to use a two-stage conversion from the current reading test to a computerized adaptive version of a new test. During stage one (which is now completed), a new linear reading test that could be administered by computer would be created. After the appropriate cut-off scores were developed and sufficient item information had been collected, the transition would be made to a computerized adaptive test. Since the Reading Program assigns P/F grades, the recommendation was made that alternate outcome information be collected upon which to base placement validity coefficients.

For English, the recommendation was made to convert the assignment of course placement to an interval or quasi-interval scale so that some variance on the predictor could be calculated. Because only one score is currently available for each class assignment, it is impossible to calculate a predictive validity coefficient. The English Department has been working to develop the new scoring scale, which should be ready by the beginning of 1995.

Placement tests are being considered for an additional use, that of establishing benchmark measures for assessing student gains in general education skills. At least three issues should be addressed before embarking on any transformation of the placement tests for this purpose: (1) Is there a good match between the general education goals and the skills measured on the placement tests? (2) Will the precision levels of the ability estimates required for benchmarking undo the efficiencies of computerized adaptive testing as it is implemented for placement tests? and (3) What will the impact on test security be once the testing operation is promoted from "low-" to "high-stakes" testing?

Some issues could be resolved by calling together individuals involved in both general education and placement testing. Our objective is to create systematic assessment which serves instructional purposes across the curriculum at IUPUI.

6. Student Enrollment Support Services Action Teams

After considering results of surveys and interviews with school deans, the Student Enrollment Support Services group, consisting of front line student services offices, formed several action teams to address specific issues identified by the deans as important for recruiting and retaining committed learners. The action teams began work in September 1994 and will present findings and recommendations to the Chancellor's staff in April 1995. Among the action teams are: Academic Advising, Internal Recruitment, and The First Year Experience.

a. The Academic Advising Team. This team exists to identify and develop the tools needed to support advisors in their work with students. The purpose or focus is to enable advisors to assist students in entering their desired major and completing a degree program in a timely manner.

b. The Internal Recruitment Team. This team plans to identify the means of assisting students who are already enrolled at IUPUI but are not yet admitted to a school of their choice. This team is looking at the processes, structures, and resources needed to facilitate internal recruitment of students by units other than the one they had planned to enter. The focus will be on new activities that are non-advising processes.

c. The First Year Experience Team. This team is to recommend improvements in the orientation process that follows registration on the basis of evaluations of what is currently taking place. The team also will consider the value of coordinating and consolidating campus-wide orientation efforts.

E. Uses of Assessment Data to Improve Learning in General Education and Major Fields 

1. Student Tracking

Like most other colleges and universities, IUPUI must respond to an increasing number of external mandates to monitor student progress toward degree completion. Indiana University maintains extensive data systems to support student tracking. These systems have been used to develop responses to external mandates such as the recently-implemented evaluation requirements associated with Carl Perkins Act for vocational and technical programs.

Because of IUPUI's large, non-traditional student population and the decentralized management structure of the academic schools, traditional student progress measures have fallen short of providing information that is truly useful for formulating and evaluating student retention efforts. Over the past year, IUPUI's Office of Information Management and Institutional Research has been developing new methods and measures for assessing student progress at IUPUI. Most notable among these is a "New-to-Program/Student Tracking" analysis that provides a "view" of student progress at the department and school level (see excerpt in Appendix I).

The "New-to-Program/Student Tracking" methodology establishes cohorts according to program entry points. It characterizes new students according to mode of entry (e.g., new to university, inter-campus transfer, inter-school transfer, intra-school transfer) and tracks them through programs according to several characteristics (mode of entry, sex, ethnicity, etc.). This methodology has been deployed so far for programs participating in academic program reviews. The first campus-wide dissemination of these data will occur in Spring 1995. It is expected that the "New-to-Program/Student Tracking" analysis will generate a range of indicators that will prove more useful than currently available ones for evaluating programmatic efforts to increase student retention and graduation rates.

2. Interrelated Surveys for Current Students and Graduates

Since its formation in 1992 the IUPUI Office of Information Management and Institutional Research has established a survey program that includes a questionnaire for recent alumni: IUPUI's 1992-93 Undergraduate Degree Recipients Survey (see excerpt in Appendix J); one for currently enrolled students: The IUPUI Continuing Student Satisfaction and Priorities Survey (see excerpt in Appendix K); and a planned IUPUI Freshman Survey.

These surveys were designed in part to ensure that a mechanism is in place for assessing the principles of general education once these principles have been approved by the faculty. Drawing upon research which suggests that good practice in undergraduate education encourages student-faculty contact, encourages active learning, and gives prompt feedback (The Wingspread Report on the Seven Principles for Good Practice in Undergraduate Education), both the alumni and student satisfaction surveys probe these issues. Thus these surveys provide a beginning point for monitoring such activities as student involvement in faculty research; participation in discussions about learning with friends, family or co-workers; and interaction with faculty concerning the evaluation of classwork.

Examples of using survey data in program improvement include the recent decision to add resources to the Offices of Undergraduate Admission, Student Financial Aid, Bursar, and Registrar; and the decision to designate the recently-vacated library building as a Student Activity Center. Likewise, faculty, staff and student groups are beginning to draw upon the data to plan and to inform discussions, ranging from the Chancellor's State of the Campus Address to the recent deliberations about individual learning plans. In addition to informing the decision-making process the data are also proving useful in helping faculty and staff understand the IUPUI student population and the educational needs of the Indianapolis community.

3. Course Evaluations as Assessment and Improvement Tools

Most IUPUI schools use student evaluation forms to supply information about satisfaction with the quality of instruction. Within the past year the IUPUI Office of Faculty Development has begun to provide individual consulting sessions with faculty interested in strengthening their teaching. The number of requests for such consulting has been even greater than anticipated. Data from end-of-course evaluations provide the primary impetus for faculty to contact the Office of Faculty Development for advice.

a. Student satisfaction. Eleven years ago, the School of Science developed an 11-item scale with good reliability for use in evaluating students' satisfaction with teaching. Over this period 80% of sections have been evaluated. Each semester this means that 400 sections and nearly 10,000 students participate. Since 1983, over 200,000 students have evaluated 10,000 sections.

Several studies indicate that this scale also has some validity with respect to assessing student learning; one study produced a correlation of .6 with scores on a cumulative final. This study analyzed the data from 2 semesters, 40 sections and 2,500 students. In the absence of direct data on student learning, student satisfaction apparently can be a good proxy indicator. Student satisfaction scores also correlated .53 with peer ratings of teaching competence.

The student satisfaction ratings are provided as a service to faculty. The purpose of this work is to provide evidence for faculty to use in important personnel decisions. In 50% of the departments in the School, student satisfaction ratings are an important part of salary decisions. For this subset of departments, student satisfaction ratings predict 25% of the variance in salary increases. These ratings are also used routinely by faculty to document effectiveness in teaching for promotion and tenure decisions. The ratings are also used to monitor the effectiveness of part-time instructors. In addition, they are frequently used by full-time and part-time faculty who are seeking an award in teaching.

Over the course of the last 10 years, the average rating on the evaluations for full-time faculty in the School of Science has risen .2, an increase that is highly statistically significant. Considering that the distribution of satisfaction ratings is always skewed toward the positive side of the scale, the .2 increase is also a meaningful increase. Student satisfaction with instruction is used in various forms by many schools, including Science, Liberal Arts, and Nursing.

b. Student learning. Another assessment method has been developed to make comparisons among faculty in terms of student learning. Any course with multiple sections and common exams can use actual student learning as an evaluative tool by converting scores on common exam(s) to z-scores. These standardized scores permit comparisons across faculty, across courses, and even across departments.

This method of assessing student learning has been used in the Psychology Department which has over 40 sections of its introductory course. For the past four years, faculty have been given feedback on the amount of student learning taking place in their classes. High scores translate directly into salary increases, just as does meritorious performance in research.

c. Student mentoring. Graduate students in the Schools of Science, Liberal Arts, and Nursing identify mentors by responding affirmatively to the following statement: From this list, identify faculty who have had a remarkable and positive influence on you. Students also identify outstanding mentors by responding affirmatively to the following statement: This person influenced the whole course of my life and this person's effect on me was invaluable. Ninety-five percent of the students identify at least one faculty member who was a mentor and 25% of students identify at least one faculty member who was an outstanding mentor. As is the case with student satisfaction data, faculty report this information for salary, promotion, and tenure considerations.

d. Evaluation of course content. Faculty in several schools are using external review in low stakes and high stakes evaluation. In the School of Liberal Arts, for example, several faculty have earned tenure and promotion based primarily on external review of teaching portfolios. In the School of Science faculty submit documentation on a given course which includes: syllabus, significant aids for student learning, class presentation materials, and a rationale for the course addressing an established list of items. This material is sent to two external reviewers. Using the letters of evaluation and the material, the members of the Teaching Evaluation Committee rate the quality of the course content on an established scale. Six faculty have used this option over the last five years to document teaching effectiveness.

e. Evaluation of products. Science faculty prepare a four-page document that identifies significant work done or products they have developed. In the document, faculty identify the need addressed, and present data on effectiveness and impact. The members of the Teaching Evaluation Committee rate the quality of the course content on an established scale. Four faculty have used this option over the last five years to document teaching effectiveness.

f. Tracking evaluations. One of the drawbacks to utilizing course instructor surveys for program improvement is that no campus-wide mechanism exists to track these data over time. The School of Science is the only IUPUI academic unit that has course evaluation information systematically stored in a way to make longitudinal comparisons. The ability to track the impact of faculty improvement efforts is a key component in evaluating the efficacy of individual or group interventions.

The Testing Center has embarked on an ambitious programming development project that permits it to monitor course evaluation indicators longitudinally, and permits deans, department chairs, or authorized others to create individual or aggregate reports easily. Utilizing distributed networking and relational database technology, participating units can create their own evaluation tools (such as Indiana's Multi-Op and Purdue's Cafeteria instruments) and generate standard or customized reports for monitoring faculty performance as perceived by students. The Center performed a pilot test of the system with IUPUI's School of Public and Environmental Affairs (SPEA) faculty and subsequently has implemented the system across all of SPEA's five campuses.

F. Improving Community Involvement Experiences Through Service Learning

A major aspect of IUPUI's mission is its intentional linkage to the larger community in which it is located. Although the university values its centrality in Indiana--and especially its location in the heart of Indianapolis--geographic locale does not in itself bind the institution to the community. Instead, IUPUI is linked to the community through involvement of its faculty, staff, and students "both to provide educational programs and patient care and to apply learning to community needs through service" (see IUPUI Statement of Mission in Appendix B).

In order to accomplish this aspect of the university's mission, the Office of Service Learning was established at IUPUI in November 1993. An assistant director for the program was hired at that time. In February 1994, the university appointed the first faculty director of the program, an associate professor of psychology with teaching and research interests in service learning.

The office exists to support faculty in integrating service components into the curriculum. With support from Campus Compact, the director and assistant director have been involved in educating the campus community--through workshops, faculty seminar series, faculty meetings, and one-on-one contact--about the benefits of integrating service with academic study. The office also assists nonprofit community agencies in developing partnerships between the community and IUPUI.

Involvement in the community has been a major aspect of an IUPUI education since the university's founding, especially as related to the "practice based" education which characterizes student learning in medicine, dentistry, nursing, social work, education, and other health-related professions. The office builds upon these historic ties and attempts to formalize and foster additional growth in community service efforts.

Recent projects of the Office of Service Learning have included: (1) the development of a summer and fall seminar series designed to assist faculty in developing service learning courses; (2) the awarding of course development stipends to ten faculty; (3) the initiation of a student scholarship program to recognize students for exemplary public service (two students were awarded scholarships in 1994-95); (4) the establishment of a mini-grant program to encourage faculty service learning course implementation; (5) the preparation of a proposal to establish the IUPUI Center for Public Service, an office devoted to coordinating the university's efforts in integrating service and academic study and encouraging voluntary service; (6) the representation of the university in regional, state, and national forums on service learning; and (7) the participation of IUPUI in the Vanderbilt (FIPSE funded) study which will explore outcome measures for service learning.

Finally, a framework has been laid to assess and document the activities and impact of the center on faculty, students, and the community. In August 1994, detailed goals and objectives of the center were created, including a timetable for completion and indicators of progress. Input for the creation of the office and its goals has come from a number of sources, including the Service Learning Advisory Committee, a group comprised of students, faculty, staff, and community members which provides direction for all office activities. Additionally, strategies for assessing student learning have been compiled from a variety of sources and are maintained by the office.

Baseline data on service learning have been gathered in a number of areas. In August 1993, a survey was conducted to obtain information on the number and nature of faculty course offerings involving a service learning component. In January 1994, a survey of student organizations was completed on student community service activities. Additionally, questions regarding student satisfaction with service learning opportunities were included in the 1994 Continuing Student Satisfaction and Priorities Survey (Sample size=1,643) coordinated through the Office of Information Management and Institutional Research. Results from this survey suggest that: (1) more than half of the respondents were satisfied with the opportunities to participate in community service afforded at IUPUI (361 very satisfied or satisfied, 208 dissatisfied or very dissatisfied); (2) more than half were satisfied with the information available on service opportunities (403 very satisfied or satisfied, 269 dissatisfied or very dissatisfied); (3) slightly more than half of the students were dissatisfied with the availability of courses providing credit for service learning (179 very satisfied or satisfied, 200 dissatisfied or very dissatisfied); and (4) although few students indicated that they had participated in voluntary community service (154 very often or often, 1,461 occasionally or never), an overwhelming majority of students felt that they had a civic responsibility to become involved in the community (851 strongly agreed or agreed, 172 disagreed or strongly disagreed). Clearly, a need exists for the programs and services offered through the Office of Service Learning. Finally, information has also been collected on community service agency interests in, and needs for, developing service learning partnerships. The data collected from all of these surveys will be used to provide benchmarks against which future activities will be measured.

G. Structure to Support Assessment

1. Decision-Making

The concept of faculty governance in matters involving the curriculum is carefully nurtured at IUPUI. Thus the Faculty Council is the ultimate source of authority on the structure of general education and the shape of the Assessment Plan. The Faculty Council is considering a proposal that the responsibility for studying and making decisions about specific issues related to the undergraduate curriculum be delegated to the Council on Undergraduate Learning (CUL). The membership of the CUL would include faculty and administrative representatives from all IUPUI schools. The CUL, if endorsed by the Faculty Council, would oversee the full development and implementation of the goals of general education and the Assessment Plan. The important decisions of this group would have a strong foundation in widespread faculty discussion. Similarly, the Graduate Council would be the body with primary responsibility for the quality of graduate programs.

2. Implementation

The Program Review and Assessment Committee (PRAC) was established in Fall 1993 as a successor to the Assessment Committee that had been in operation since 1988. Its purpose is to provide broad faculty involvement in implementing the campus Assessment Plan. The committee reports to the CUL. The PRAC has two faculty representatives from each school who work with school and departmental academic policies and assessment committees.

The PRAC membership spent most of 1993-94 increasing its collective knowledge about assessment. Some members attended national and regional assessment conferences, and all read recommended materials on the topic. In addition, the group provided the Vice Chancellor for Planning and Institutional Improvement with valuable guidance in developing the guidelines for academic program review and subsequently in applying the guidelines in the initial reviews.

During Fall 1994, PRAC representatives worked with their colleagues in each school and department to develop student learning goals, related instructional strategies, and assessment methods (see Appendix F). In addition, PRAC members will provide leadership on similar issues related to general education.

A faculty member chairs the PRAC and is supported by staff in the Office of Planning and Institutional Improvement, including Vice Chancellor Trudy Banta and Director of Assessment John Kremer. Kremer is also chair of the Department of Psychology and thus has only a part-time role in assessment. Associate Dean of the Faculties Barbara Cambridge, whose primary responsibilities include general education matters, also serves the PRAC in a staff capacity.

Membership on the PRAC is recognized as being a major service commitment for faculty. The Office of Planning and Institutional Improvement is working to develop a policy to recognize this effort with the appropriate academic support and reward.

3. Support Offices

IUPUI has two offices whose principal responsibility is to support assessment: the Office of Information Management and Institutional Research and the Testing Center, both of which report to the Vice Chancellor for Planning and Institutional Improvement. These offices provide survey, tracking, and achievement data for all the departments and programs at IUPUI and have developed some creative assessment initiatives of their own which have been described elsewhere in this plan. 


4. Training and Development Support

Several IUPUI offices, most notably the Office of Faculty Development and the Office of Planning and Institutional Improvement, have offered programs that support assessment at IUPUI. Several nationally recognized assessment scholars have been invited to campus. In Spring 1994, Alexander Astin, Peter Ewell and K. Patricia Cross came to make addresses and to meet for discussion with the PRAC. In Fall 1994 Pat Hutchings from AAHE, Charles Karelis from FIPSE, and Lee Shulman from Stanford University were campus visitors. In the course of his visit, Shulman met with an ad hoc group of faculty from the Schools of Medicine, Dentistry, Allied Health and Nursing to discuss their interests in performance assessment. Workshops by experienced local practitioners have been offered to faculty on a variety of assessment topics, including how to set measurable objectives and how to match assessment strategies to such objectives. For the past two years PRAC members as well as other faculty have had the opportunity to attend the national Assessment Conference in Indianapolis sponsored annually by Banta and her staff.

During 1995 PRAC members will be encouraged to ask for individual consultation on their assessment plans from Kremer, Cambridge, Banta and others. Additionally, half-day workshops directed by IUPUI faculty have been planned to assist schools and departments at Stages 1 and 2 to move to Stage 3 in their assessment planning.

Finally, the PRAC continues the tradition established by the Assessment Committee of awarding departmental and school faculty small grants to begin assessment projects. This practice has encouraged faculty to collect better data for program review as well as to make assessment the focus of their own disciplinary scholarship.

H. Evaluation of the Assessment Plan

In 1999 a peer review will be conducted for the assessment program. The evaluation team will include assessment specialists from institutions outside Indiana as well as IUPUI faculty and a community representative.

I. Summary

Since the 1992 NCA accreditation visit, a great deal of progress has been made with respect to addressing the four areas of concern in the team's final report. Significant attention has been focused on discussing, drafting, and establishing the mechanisms for agreement on IUPUI's mission as well as its general education goals. Furthermore, improvements in assessing student achievement in undergraduate and graduate education are underway at both the school level and institution-wide through program review. Clearly, IUPUI has made a commitment to, and progress on, addressing the areas noted in the NCA report.

This progress has not been easy, nor has it been evenly spread throughout the institution. This is in part due to the historic independence of IUPUI's schools; the somewhat confusing administrative structures which have resulted from IUPUI's ties to two separate research universities; and the absence of a clear and distinctive mission, institutional identity, and widely-shared educational goals. It is also, in part, a reflection of the profound paradigm shift for many in attempting to focus greater attention on what students are learning as contrasted with the more traditional focus on what faculty are doing. Nonetheless, progress has been made.

As an institution, IUPUI still has much work to do in the area of assessment. Yet, the efforts currently underway to develop a focused mission and identity, and to stress the assessment of student learning will, no doubt, position IUPUI to play an even greater role in the lives of the students, faculty, staff, administrators, and community members who interact with this institution.