Program Review at UNC-Chapel Hill
Instructions for Implementing the Self-Study
Program Overview
Curriculum
Faculty
Students
Leadership, Administrative Support, Facilities and Equipment, Institutional Relationships
The Future
A NOTE ON COMBINED GRADUATE AND UNDERGRADUATE PROGRAM REVIEWS
A combined review of both graduate and undergraduate programs of a school, department, or curriculum offers the opportunity to evaluate each degree level within the context of its own needs, goals, and objectives, as well as within the context of the unit’s overall mission and strengths. When preparing the self-study in such a case, it is important to provide assessment and commentary on each degree level separately, wherever possible. This is done in order to conduct a meaningful evaluation of the teaching, research and training activities of each degree level on its own merits. Afterward, an assessment of the programs together should focus on the dynamic relationship between the undergraduate and graduate degree levels. The combined review should also address the program’s effectiveness in representing the discipline on campus.
PROGRAM OVERVIEW
The program overview presents a comprehensive assessment of the program’s degree offerings, within the parameters of its stated mission, goals, and objectives, and its position within the history of the discipline (past, present, projected), as well as within the family of its peer institutions and research centers. This is an opportunity for the program to reassess itself, restating or modifying its mission and the consequent goals and objectives. This statement serves as the framework for the evaluation process. Specifically, the reviewers will be charged to assess whether the stated mission is realistic and feasible, and whether it meets the needs of the profession for both scholarship and research.
The following suggestions may help in developing this section:
MISSION, GOALS, AND OBJECTIVES
State the unit’s overall mission and goals – as well as the distinct mission and goals of the graduate and undergraduate programs, if appropriate – and the objectives to achieve those goals. There should be reference to the program’s position within the university’s mission (e.g., the Academic Plan), including the role of the undergraduate major and minor in relation to the program’s other functions (graduate program, research mission, service courses and activities). A description of the program’s organization, and policies which guide its operations, is essential.
Units with both graduate and undergraduate programs should organize subsequent materials into separate sections for each level, and then proceed to develop mission/goal/objective statements for each level, as well as for different degree programs within each.
NEED/DEMAND
For each degree or area of concentration within a degree program, including the undergraduate major (if any), please provide evidence concerning the need and/or demand for the program.
INTERDISCIPLINARY ACTIVITIES
Comment on the degree of rapport and exchange that exists with related programs, and on any means for encouraging students to take courses in other programs. Note joint faculty appointments, joint or dual degree programs, and participation by program faculty in curricula outside the program. Comment on the program’s involvement in research centers and institutes.
INTERINSTITUTIONAL PERSPECTIVE
Summarize evidence that compares the quality of your program with those on other campuses within the state, the region, and the nation.
PREVIOUS EVALUATIONS
Include a synopsis of any previous evaluations or accreditation reviews of the program.
CURRICULUM
The section on curriculum is to describe the design and requirements of all degree programs, and to correlate these with the educational goals and learning outcomes they are intended to achieve for the students – content knowledge, thinking and expression skills, and professional skills. If programs sponsor certificate programs, either for matriculated or off-campus/professional students, they should be discussed as a separate program.
Specifically, the reviewers will be charged to assess the curriculum in terms of the stated mission of the program and their understanding of the needs of the profession. They will look at the breadth and depth of content, integration, opportunities for practice and research, and preparation for professional life in the 21st century. Among other things, they will look for balance between coursework and research for each concentration area, as well as balance between required and elective courses; the progression of courses, seminars, research opportunities, and integrative projects; the length and size of the program relative to its stated goals; the balance between graduate and undergraduate programs; and the timeliness of course offerings. Figure 1, “Curricular Matrix,” may help in developing this section.
For programs offering courses for General College students: Explain the program’s role in the university’s undergraduate general education program for non-majors. How do undergraduate courses that enroll general education students differ from those aimed predominately at majors or minors? How are faculty made aware of these differences? How does the program ensure that is offers general education courses in sufficient numbers to satisfy demand and to attract majors?
FACULTY
This section is to frame the discussion by the faculty and the program’s administration of the faculty’s strengths and areas of concern in research, teaching, mentoring/advising, and participating in professional, program/department, and university committees and activities. This should be accomplished by:
- Stating the goals for research, teaching, mentoring/advising, and participation in professional activities,
- Determining the means to assess how well these goals are being met, and
- Discussing how to improve in each of these areas.
The Program Review Team will be asked to look at faculty size, quality and distribution by field of expertise, in relation to your program’s stated mission, and to identify priority areas of scholarship and research. We will ask them to look at how the faculty portrait relates to the university’s commitment to maintain a diverse population (e.g., women, underrepresented populations). They will consider both students and faculty evaluations of teaching, as well as whether there are sufficient opportunities for faculty to improve their teaching skills. They will look at research strength as compared to that of faculties at peer institutions, and will assess whether faculty research is effectively integrated into their teaching. We will ask them whether faculty compensation and rewards are appropriate in comparison to peer institutions. Finally, they will take into account the morale of the faculty and collegiality within the program.
It may be useful to organize the information into sections addressing overall assessment, research activities, and teaching responsibilities:
Overall Assessment of the Faculty
The overall assessment should include descriptive statements, supported by both aggregate data and individual data:
- Begin with a brief assessment of the state of the faculty, including a description of its organization and committee activities, interactions among faculty of the various curriculum tracks/concentrations (if any), and a report of the results of any faculty surveys conducted as part of the self-study. Include an assessment of faculty strengths and areas of concern.
- Provide data on faculty distribution over ranks, including relevant comments and assessment.
- Describe how the salary range at each rank in the program compares with that in the discipline at comparable institutions.
- Indicate the age distribution of the faculty, along with projected faculty retirements. What is the expected impact of these retirements, and what are the program’s plans to address this impact?
- Comment on the racial, ethnic, and gender diversity in the program’s faculty. Note any significant trends in the last five years. Be certain to present the programs’ recruitment, retention, and support procedures for racial and ethnic minority faculty and other underrepresented populations within the program.
- An assessment by and of faculty with administrative responsibilities (e.g., chairs, deans, directors) – their role and their effectiveness — is encouraged.
- Include (an appendix is preferable) an abbreviated curriculum vitae for each faculty member which summarizes publications; honors and awards; participation in national and international societies and meetings; editorial responsibilities; university, regional, national and/or international committees; research grants and/or contracts during the past five years.
Faculty Research
State the research goals of the faculty in each program area, including standards of quality and quantity. What are the rewards and results of meeting or exceeding these goals? How do these goals compare to those of similar programs at peer institutions?
- Begin by describing arrangements for research or study leaves, or alternative means by which faculty are encouraged to broaden their perspectives and to renew their qualifications for teaching and research. Comment on the success of faculty in obtaining outside or competitive funding for leaves – Guggenheim, Fullbright, etc.
- Comment on faculty grants held in the program over the last five years.
- Provide a précis of honors and distinctions of the faculty for the last five years.
Teaching
Describe and discuss the program’s policies on teaching, (supplementing with data, as shown in Figure 2, “Teaching Activities,” and Figure 3, “Teaching Activities/Faculty Member”):
- Describe the policy for the distribution of teaching loads during the most recently completed academic year. Distinguish between the teaching of graduate and undergraduate courses when possible, but include both when applicable.
- Discuss the rationale for this policy, and how it responds to the program’s curricular goals.
- Describe who does the undergraduate teaching (if any) in the program. If graduate students teach undergraduate courses, describe the process by which they are selected, trained, supervised, and evaluated. Describe, and comment upon the adequacy of, the compensation they receive.
- Describe the program’s process for evaluating teaching, and explain the evaluations that are used. Present evidence regarding instructional effectiveness as indicated by student evaluations.
- Describe the program’s efforts to assist faculty to improve their teaching.
- State the program’s goals and policies for faculty advising/mentoring of graduate students, undergraduate majors and minors, and postdoctoral fellows, and describe how this is accomplished.
For each faculty member individually, indicate:
- # completed master’s theses/papers chaired/advised last five years
- # completed dissertations chaired/advised last five years
- noncommittee involvement in supervision of doctoral students or postdoctoral fellows
- active involvement in improvement of undergraduate instruction/research
- whether s/he has undertaken course development work (developing new courses, revising courses, preparing new course materials), either independently or in conjunction with programs in the Center for Teaching and Learning, or received various course development awards. Please describe any significant achievements in detail.
STUDENTS
This section is to summarize data about and by the students in the program, for the most part separating the presentation for graduate and undergraduate students. The Program Review Team will be asked to assess the standards of the program’s student scholarship and research, as well as the placements of graduates, as compared to peer institutions. They will consider the adequacy of student funding, the quality of advising and mentoring of students, student morale, student learning outcomes assessment results, average time to degree trends, and the distribution of students in relation to the university’s commitment to maintain diversity (e.g., women, underrepresented populations). They will talk with students and they will assess whether students feel that they can participate effectively to improve or revise the program.
To assist in preparing this section, there are several types of data the program can obtain, described in detail later in this manual. The program being reviewed should comment on the trends revealed by those data. Information should be obtained for the past five years and includes various configurations of data on enrollment; graduate applications and acceptances; credit hours generated; and degrees conferred.
You may want to present quantitative data in the format of Figure 4, “Demographic Profile of Graduate Students” and Figure 5, “Demographic Profile of Undergraduate Majors”. Please note significant trends. In addition, the following descriptive information should be included:
- How does the program seek to ensure a hospitable environment for all of its students?
- Comment on the racial, ethnic, and gender diversity in the program’s graduate and undergraduate student populations. Note any significant trends in the last five years.
- Present the program’s recruitment, retention, support and placement procedures for underrepresented populations within the discipline.
- Describe the program’s set of goals and expected outcomes for undergraduate education, measures that will enable them to tell how well these goals and outcomes are met, a description of the student learning outcomes assessment process, and examples of how assessment results have been used to improve the program in recent years.
- Describe the criteria by which graduate applicants are chosen for offers of admission.
- From what sources (e.g., federal funds, training grants, university awards and assistantships, program assistantships), and in what amounts, are graduate students supported financially? What proportions are currently supported, and what is the average rate of support/student? If 5-year data are available, please provide; otherwise note trends.
- Describe procedures for evaluating the progress of graduate students during the course of their training.
- Incorporate time-to-degree statistics and trends, including any efforts to streamline time-to-degree in recent years.
- Provide any available statistics and an analysis of those data for matriculated students who left before completion of their program. What steps have been put in place to increase overall retention or identify “attrites” earlier?
- Identify significant professional or intellectual contributions by students while in the program or after graduation/completion of training (e.g., faculty ratings, awards and honors, notable publications, leadership roles in professional organizations, etc.)
- Provide a detailed record of the employment placement or further advanced studies of graduates upon completion of the program for the most recent five years, for each degree or area of concentration.
- Provide a précis of initial post baccalaureate pursuits of undergraduate majors and minors. Are the program’s baccalaureate graduates well prepared for graduate and/or professional studies and employment in their specialty? Are they aided in obtaining employment? If so, describe how.
In preparing this section, please obtain student input and evaluations of all phases of the program; this is typically accomplished through the use of program-generated assessment tools and surveys. Comment on the results of any such assessments. Include a discussion of the impact of teaching by graduate student teaching assistants (GTAs) on the effectiveness of the undergraduate curriculum. Summaries and representative samples of student comments are encouraged. Comment on the legitimacy of student criticisms.
LEADERSHIP, ADMINISTRATIVE SUPPORT, FACILITIES AND EQUIPMENT, INSTITUTIONAL RELATIONSHIPS
The goal of this section is to assess the effectiveness of the program’s governance and administration, and the adequacy, currency, and distribution of space, equipment, and support services, especially as related to achieving the program’s stated mission.
LEADERSHIP
Describe the program’s internal organization for governance and administration. Include an organization chart, if available.
ADMINISTRATIVE SUPPORT
Comment on the adequacy of staff support, or support of others that may be appropriate, indicating the financial base of this support and clearly delineating state versus nonstate support levels. This should include technical, clerical, secretarial, and administrative support.
FACILITIES AND EQUIPMENT
Comment on the status and adequacy of physical facilities, including:
- physical space for teaching, research, and administration;
- instructional, research, and administrative equipment;
- library holdings both within the program and university wide;
- computer capacity available to the program from the campus and/or from other agencies.
INSTITUTIONAL RELATIONSHIPS
Delineate relationships with other academic and research units.
THE FUTURE
The program is asked to engage in an intellectual five-to ten-year planning process, taking into account the self-study data generated. This section should also share with the review team the vision that the program faculty and administration hold for the discipline. Where is the program going? Stimulate faculty dialogue and report both majority and minority views.
Discuss how the program integrates an ongoing assessment of its progress toward its stated goals into planning for the future. How does the program recognize its strengths and concerns, and how is this recognition translated into planning? Where applicable, delineate between undergraduate and graduate aspects of program planning.
The Program Review Team will be asked whether the program’s plans for the future, and its new initiatives, are realistic, and reflect the needs of the discipline. They will be asked to provide insights as to significant new developments in education and/or research that they see as likely to occur in the next five years, and to assess whether the program is positioned to capitalize on these developments. They will be asked for recommendations for program growth, retraction, and/or other changes.
In preparing this section, the program may wish to address the following questions (among others):
- What are the major disciplinary subspecialties represented in the program, and what are their strengths and the program’s concerns regarding them?
- Which of these subspecialties are likely to remain vigorous, and which are likely to be de-emphasized in the future, especially as current faculty retire? Are there subspecialties that should be eliminated or merged with others?
- Develop and comment on student enrollment projections for each subspecialty.
- Are there subspecialties not currently represented in the program for which development should be initiated? If so, what resources would be needed to mount such initiatives?
- Given that the total faculty size at Chapel Hill is likely to remain fixed in the future, and that graduate student enrollment has historically been most influenced by the ability of a program to support its students, what do you project for the size of your program (faculty, staff, graduate students, undergraduate majors) in the next five years? . . . ten years? Explain. How does undergraduate enrollment affect the program?
- Are there specific resource needs, other than additional faculty slots and graduate student stipends, which might inhibit nourishment of current subspecialties or the initiation of new ones? Explain.
- Are there important curricular changes that are to be made, or which should be made, during the next five years? If so, what are the plans for their implementation?
- How can the quality of graduate and undergraduate education be improved? What steps would be of assistance in enhancing the quality of the program in comparison to those of other programs in the state? The region, and the nation-both short-term and long-term?
- What are the plans to attract graduate students with higher academic qualifications than those presently enrolled in the program?
- What plans exist to attract highly performing undergraduate majors?
- What plans have been made to ensure and enhance racial, ethnic, and gender diversity in the graduate and undergraduate programs?
- Does the program have or intend to have a postdoctoral program that will prepare new doctorates from diverse backgrounds to enter and remain in the professorate?
- What plans have been made to recruit, support, and retain minority and underrepresented faculty?
Next: Suggestions for the Self-Study Outline
(A printable version of Program Review at UNC-Chapel Hill is also available.)