Elements of the Program Review

The Self Study

  • What do you do?
  • Why do you do it?
  • How well do you do it?
  • What difference does it make whether you do it or not?
  • How well does what you do relate to why you say you do it?

A complete description of the suggested substance of a program’s self-study comprises Part II of this document, “Instructions for Implementing a Self-Study.” The self-study process usually takes approximately a year to complete, and involves a significant amount of planning, meeting, data collecting, discussing, writing, reviewing, and editing. It is strongly suggested that a schedule for various components of the review be developed early in the process in order to facilitate accountability and avoid placing an inordinate amount of the work for the self-study on any one person or administrative office. Optimal implementation of the process is genuinely collegial, embracing all actors and perspectives.

The FINAL FORMAT of the self-study should ideally be electronic only via a website (e.g., Canvas, Sharepoint, or MS Teams with security rights given to all participants), or a combined master PDF that can be emailed to all participants, assuming these electronic versions are easily accessible, organized, and readable. The Graduate School should receive the self-study materials approximately one month prior to the scheduled start of the on-campus visit of the Program Review Team. The Graduate School will handle distribution to everyone involved in the review and archival activities.

If flash drives or printing must occur, eight copies of the self-study are to be delivered to The Graduate School. They should be bound or in ring binders, with a table of contents, and tabbed or otherwise easily identifiable sections; it is appropriate for appendices to be provided only on a flash drive.

The On-Campus Visit

Selecting the Team

The Program Review Team comprises two highly qualified individuals, external to the campus. An additional member from the UNC-Chapel Hill faculty is appointed to participate and is drawn from a discipline complementary to the program undergoing review. Criteria for selecting team members include a history of involvement and success in scholarship or research, and experience in both graduate and undergraduate teaching. Ideally team members will have some direct administrative experience (e.g., chair, director of under/graduate studies, center experiences) to enable a full review.

The program submits a prioritized list of proposed reviewers to The Graduate School, taking care to cover the different areas of expertise necessary to provide a complete and knowledgeable assessment of the program. Usually, these are faculty members from other academic institutions, who are well respected in the field, but without direct connection to the program. Program alumni, former faculty (or those we have previously recruited), direct research collaborators, and former team members from past reviews are not appropriate choices and should not be submitted. Occasionally, in fields in which certain kinds of technical expertise are critical, an industry or practitioner representative may be included.

The Graduate School reviews the proposed names, together with the program dean's office, and an agreement is reached on a priority list for extending invitations. Attention is given to selecting individuals from public institutions, where possible, and for maintaining diversity of reviewers and perspectives. The Graduate School takes responsibility for inviting individuals to participate and for appointing them to the team.

A team member is selected to serve as chair of the team and to act as a liaison between the team and the University. The team chair provides input into the development of the on-campus visit schedule (when necessary), coordinates the preparation of the team’s report, and is responsible for submitting the final report to The Graduate School.

The Graduate School pays travel expenses (i.e., budget roundtrip ticket to/from home location with reservations made approximately two months in advance and hotel in Chapel Hill) and a modest honorarium for the two external team members. Unless flight times do not allow a return late-afternoon or evening on the final day of the review, The Graduate School cannot pay for additional hotel stays. If the program wishes to invite an additional reviewer, the invitation process remains as above, but the program assumes all of the costs of the extra team member. Exceptions to this policy can be requested for valid academic reasons (e.g., an attempt to cover all specializations in a large program) but are dependent on funds available in a given academic year.

Elements of the Review and Preparation of the Schedule

After reviewing the self-study, the external team of reviewers visits the Chapel Hill campus to assess the program. This assessment is based in part on the self-study, with particular attention given to the strength of the instructional and research programs, and the resources available to the program. In programs where there is an undergraduate program, it is expected that the undergraduate, and graduate program where applicable, will each be evaluated on its own merits before an assessment of the program as a whole is made.

The on-campus visit opens with a working dinner where the team meets with the program chair and other program faculty members. The next day, there is a meeting with the review team, along with representatives of the Provost’s Office, the Dean’s office of the respective school, and The Graduate School. Representatives from the program being reviewed are not present at the opening meeting. The visit closes with an exit interview with the team and this same group. The program chair, in consultation with the chair of the review team and The Graduate School, plans the remainder of the visit.

During their visit, the team typically interviews program faculty and students, and often alumni, as well as meeting with key external constituents or internal working groups. Unless there are specific requests by the team, it is at the program's discretion to determine how to select and group individuals for these meetings while keeping guidance from The Graduate School in mind. These participants may include directors of program degree programs (e.g., undergraduate, master's, doctoral), research groups, key committees, and/or representatives of other units or groups that play a critical role in the program's work. Depending on the size of the Program, it is advisable to build in opportunities for certain groups to meet with the Review Team alone (e.g., assistant professors, associate professors, graduate students, and the unit chair or director). A schedule of all individual, one-on-one meetings is not recommended, though the schedule may have one 20-30 minute "open session" for any individual drop-ins or provide a way for someone to contact the team individually if desired. Open blocks of writing time are not necessary to schedule, instead spending time while on site in active meetings gathering information. Program leadership should not be present at every session during the site visit. (See attached sample schedules.)

In some cases, it is appropriate for the team to visit off-campus facilities, e.g., the Institute of Marine Sciences in Morehead City (Department of Earth, Marine, and Environmental Sciences), Art Lab Facility and Hanes Art Center (Department of Art and Art History). The team may ask to examine sample student files, or dissertations and theses, or to review other additional material/data.

Social events should not be scheduled, as the team typically uses evenings to work, in order to produce the beginnings of a draft of its report by the end of the on-campus visit.

Other Logistics of the On-Campus Visit

Reviewers are asked to arrive on campus in time for a dinner the night before the start of the review. The on-campus visits usually last one full day, and most of a second day (depending on the size and complexity of the program being reviewed). Reviewers are usually able to depart from campus in early evening on the final day.

The unit being reviewed is expected to arrange to have the external team members met at the airport and taken to the airport for their return, and to handle all other transporting or escorting of team members to meetings, on-campus and off.

The Graduate School will pay for the welcoming dinner for the review team and up to three additional program representatives. The Graduate School will also pay for the breakfast meeting on the first day, dinner on the first full day for the review team only, and the breakfast on the second day for the review team only. The program will assume the costs of other events (e.g., lunches, self-study preparation). Note: The Graduate School is unable to pay for alcoholic beverages; bills will be split at the restaurant and/or with journal entries afterwards.

A memo detailing the logistics and finances will be provided to each program as their review date nears.

Virtual Visits

It is possible to conduct site visits virtually, by choice or due to external factors such as public health or weather. If your program believes a virtual visit is appropriate please communicate with The Graduate School early in your planning. The guidance above for selection of team members and schedule preparations should be followed, with the exception that the schedule may be elongated over more days to accommodate virtual capabilities, more frequent breaks, and more sessions for the Review Team alone. Please see a compilation of Best Practices for Virtual Review Experiences.

The Program Review Team’s Report

On the second day of the on-campus visit, the Program Review Team participates in an exit interview, making an oral report to representatives of the Provost’s Office, the Dean’s office of the respective school, and The Graduate School. The team is requested to have a written report to The Graduate School, detailing their findings and recommendations, within a month of their on-campus visit. The report should reflect an assessment of mission, curriculum, faculty, students, leadership, support and resources, and strategy for the future, and typically is ten to fifteen pages long. Most teams try to have a plan for drafting of the report before they leave campus.

Once received by The Graduate School, copies of the report are forwarded to the program chair and to representatives of the Provost's Office and the Dean's office of the respective school. It is expected that the program chair will share the report with program faculty.

The Program Response

The program chair facilitates the preparation of a written response to the Program Review Team’s report, reflecting the deliberations of the faculty in response to the report, which is subsequently sent to The Graduate School. This step normally occurs 6-9 months after the site visit to allow time for program deliberation and to put recommendations in place. Once reviewed by The Graduate School, copies of the response are forwarded to the Provost’s Office and the Dean's office of the respective school.

The Closure Meeting And Memo

After receiving the program's response to the Program Review team’s report, The Graduate School schedules a closure meeting. Attending are the program chair (and others whom s/he may wish to involve), and representatives of the Provost’s Office, the Dean’s office of the respective school, and The Graduate School. The purpose of the meeting is for the program to discuss the report of the Program Review Team and the program’s response, to provide an update on changes since the report and the response were written, to respond to questions, and to develop a shared understanding of the steps needed to address any concerns raised by the review, including improvement of the educational program. This meeting is the time when an agenda and action plan for the future are established.

The discussion at the closure meeting is captured by The Graduate School and serves as a reference for subsequent program reviews, and for any interim deliberations regarding program expansion or new initiatives.

The Midpoint Review

For many programs, approximately four years after the closure meeting is held, The Graduate School will conduct a midpoint review with the program by requesting an update on the action steps identified. This step ensures continuous assessment and improvement of our academic units. The Graduate School will contact the program chair with additional information when it is time for the midpoint review. Occasionally, depending on the nature of the recommendations and action items, additional check-ins will be scheduled on alternate schedules.

Next: Instructions for Implementing the Self-Study