Implementing the Self-Study

Program Overview
Curriculum
Faculty
Students
Leadership, Administrative Support, Facilities and Equipment, Institutional Relationships
The Future

Please note that these instructions are intended to be a helpful guide in preparing the program self-study. There is scope for interpretation and program-level flexibility, particularly if certain data elements or thematic areas do not apply to a given program. Please see the self-study outline for required key elements and use the following sections and questions as a guide for developing your program narrative. Additionally, those data and themes that are routinely monitored and assessed within the program should be relied on whenever possible to ease new data collection and reflect continuous assessment best practices.

Combined graduate and undergraduate program reviews

Program review at UNC-Chapel Hill includes all levels of program — undergraduate only, graduate only, and graduate-undergraduate combined, where appropriate. A combined review of both graduate and undergraduate programs of a school, department, or curriculum offers the opportunity to evaluate each degree level within the context of its own needs, goals, and objectives, as well as within the context of the unit's overall mission and strengths. When preparing the self-study in such a case, it is important to provide assessment and commentary on each degree level separately, wherever possible. This is done in order to conduct a meaningful evaluation of the teaching, research and training activities of each degree level on its own merits. Afterward, an assessment of the programs together should focus on the dynamic relationship between the undergraduate and graduate degree levels. The combined review should also address the program’s effectiveness in representing the discipline on campus.

Program Overview

The program overview presents a comprehensive assessment of the program's degree offerings, within the parameters of its stated mission, goals, and objectives, and its position within the history of the discipline (past, present, projected), as well as within the family of its peer institutions and research centers. This is an opportunity for the program to reassess itself, restating or modifying its mission and the consequent goals and objectives. This statement serves as the framework for the evaluation process. Specifically, the reviewers will be charged to assess whether the stated mission is realistic and feasible, and whether it meets the needs of the profession for both scholarship and research.

The following suggestions may help in developing this section:

Mission, Goals, and Objectives

• State the unit’s overall mission and goals as well as the distinct mission and goals of the graduate and undergraduate programs, if appropriate and the objectives to achieve those goals. There should be reference to the program’s position within the university’s mission, including the role of the undergraduate major and minor in relation to the program’s other functions (graduate program, research mission, service courses and activities). A description of the program’s organization, and policies which guide its operations, is important to include and provide context.

  Units with both graduate and undergraduate programs should organize subsequent materials into separate sections for each level, and then proceed to develop mission/goal/objective statements for each level, as well as for different degree programs within each.

Need/Demand

• For each degree or area of concentration within a degree program, including the undergraduate major (if any), please provide evidence concerning the need and/or demand for the program. Metrics should include at minimum enrollments and degrees awarded in each degree program.

Interdisciplinary Activities

• Comment on the degree of rapport and exchange that exists with related programs, and on any means for encouraging students to take courses in other programs.
• Note joint faculty appointments, joint or dual degree programs, and participation by program faculty in curricula outside the program.
• Comment on the program’s involvement in research centers and institutes.
• Include descriptions and evaluation of any agreements (informal as well as official MOUs) with foreign partners and international collaborations.

Interinstitutional Perspective

• Summarize evidence that compares the quality of your program with those on other campuses within the state, the region, and the nation.

Previous Evaluations

• Include a synopsis of any previous evaluations or accreditation reviews of the program. It is appropriate to include the previous program review report, response, closure memo, and midpoint as a self-study appendices. Copies of these are on file in The Graduate School if needed.

Curriculum

The section on curriculum is to describe the design and requirements of all degree programs, and to correlate these with the educational goals and learning outcomes they are intended to achieve for the students content knowledge, thinking and expression skills, and professional skills. If programs sponsor certificate programs, either for matriculated or off-campus/professional students, they should be discussed as a separate program in the report.

Learning outcomes and assessment reports for both undergraduate and graduate curricula are important to include and assess. Programs must submit their last three years of learning outcomes assessment reports as part of the self-study; please contact the Office of Institutional Research and Assessment for prior year reports if necessary.

Specifically, the reviewers will be charged to assess the curriculum in terms of the stated mission of the program and their understanding of the needs of the profession. They will look at the breadth and depth of content, integration, opportunities for practice and research, and preparation for professional life in the 21st century. Among other things, they will look for balance between coursework and research for each concentration area, as well as balance between required and elective courses; the progression of courses, seminars, research opportunities, and integrative projects; the length and size of the program relative to its stated goals; the balance between graduate and undergraduate programs; and the timeliness of course offerings. Figure 1, “Curricular Matrix,” may help in developing this section.

For programs offering courses for General College students: Explain the program's role in the university's undergraduate general education program for non-majors. How do undergraduate courses that enroll general education students differ from those aimed predominately at majors or minors? How are faculty made aware of these differences? How does the program ensure that is offers general education courses in sufficient numbers to satisfy demand and to attract majors?

Faculty

This section is to frame the discussion by the faculty and the program's administration of the faculty's strengths and areas of concern in research, teaching, mentoring/advising, service, and participating in professional, program/department, and university committees and activities. This would be accomplished by:

• Stating the goals for research, teaching, mentoring/advising, service, and participation in professional activities,
• Determining the means to assess how well these goals are being met, and
• Discussing how to improve in each of these areas.

The Program Review Team will be asked to look at faculty size, quality and distribution by field of expertise, in relation to your program's stated mission, and to identify priority areas of scholarship and research. We will ask them to look at how the faculty portrait relates to the university's commitment to maintain a diverse population (e.g., women, underrepresented populations). They will consider both student and faculty evaluations of teaching, as well as whether there are sufficient opportunities for faculty to improve their teaching skills. They will look at research strength as compared to that of faculties at peer institutions, and will assess whether faculty research is effectively integrated into their teaching.

We will ask them to review and evaluate mentoring policies and practices within the unit, including special emphasis on mentoring junior faculty as they prepare for tenure and promotion as well as graduate students (if applicable). They review the service and engagement of the faculty research and activities with the campus, state, nation, and international communities. We will ask them whether faculty compensation and rewards are appropriate in comparison to peer institutions. Finally, they will take into account the morale of the faculty and collegiality within the program.

It may be useful to organize the information into sections addressing overall assessment, research activities, and teaching responsibilities:

Overall Assessment of the Faculty

The overall assessment would include descriptive statements, supported by both aggregate data and individual data:

• Begin with a brief assessment of the state of the faculty, including a description of its organization and committee activities, interactions among faculty of the various curriculum tracks/concentrations (if any), and a report of the results of faculty surveys conducted as part of the self-study. Include an assessment of faculty strengths and areas of concern.
• Provide data on faculty distribution over ranks, including relevant comments and assessment.
• Describe how the salary range at each rank in the program compares with that in the discipline at comparable institutions.
• Indicate the age distribution of the faculty, along with projected faculty retirements. What is the expected impact of these retirements, and what are the program's plans to address this impact?
• Comment on the racial, ethnic, and gender diversity in the program's faculty. Note any significant trends in the last five years. Be certain to present the program’s recruitment, retention, and support procedures for racial and ethnic minority faculty and other underrepresented populations within the program.
• Describe the program’s promotion and tenure policies, including their clarity and transparency among faculty. As part of program review, the unit's promotion and tenure process should be reviewed and revised (if not already done so recently), described, and included in the self-study.
• An assessment by and of faculty with administrative responsibilities (e.g., chairs, deans, directors) their role and their effectiveness is encouraged.
• Provide a description and assessment of the mentoring policies and practices in the program, including faculty-to-faculty mentoring, faculty-to-student mentoring, and the quality of the chair's or dean's feedback to faculty via annual and third-year reviews.
• Include an abbreviated curriculum vitae for each faculty member which summarizes publications; honors and awards; participation in national and international societies and meetings; editorial responsibilities; university, regional, national and/or international committees; faculty and graduate student mentoring activities; research grants and/or contracts during the past five years. Include CVs as an appendix at minimum.

Faculty Research

State the research goals of the faculty in each program area, including standards of quality and quantity. What are the rewards and results of meeting or exceeding these goals? How do these goals compare to those of similar programs at peer institutions?

• Begin by describing arrangements for research or study leaves, or alternative means by which faculty are encouraged to broaden their perspectives and to renew their qualifications for teaching and research. Comment on the success of faculty in obtaining outside or competitive funding for leaves Guggenheim, Fulbright, etc.
• Comment on faculty grants held in the program over the last five years.
• Describe the program’s mentoring and other efforts to assist faculty to improve their research skills and ability to secure external funding.
• Provide a précis of honors and distinctions of the faculty for the last five years.

Teaching

Describe and discuss the program’s policies on teaching, (supplementing with data, as shown in Figure 2, “Teaching Activities,” and Figure 3, “Teaching Activities/Faculty Member”):

• Describe the policy for the distribution of teaching loads during the most recently completed academic year. Distinguish between the teaching of graduate and undergraduate courses when possible, but include both when applicable.
• Discuss the rationale for this policy, and how it responds to the program’s curricular goals.
• Describe who does the undergraduate teaching (if any) in the program. If graduate students teach undergraduate courses, describe the process by which they are selected, trained, supervised, and evaluated. Describe, and comment upon the adequacy of, the compensation they receive.
• Describe the program’s process for evaluating teaching, and explain the evaluations that are used. Present evidence regarding instructional effectiveness as indicated by student evaluations.
• Describe the program's mentoring and other efforts to assist instructors (faculty and graduate students) to improve their teaching.
• State the program's goals and policies for faculty advising/mentoring of graduate students, undergraduate majors and minors, and postdoctoral fellows, and describe how this is accomplished.

For each faculty member individually, indicate:

• # completed master's theses/papers chaired/advised last five years
• # completed dissertations chaired/advised last five years
• non-committee involvement in supervision of doctoral students or postdoctoral fellows
• active involvement in improvement of undergraduate instruction/research
• whether s/he has undertaken course development work (developing new courses, revising courses, preparing new course materials), either independently or in conjunction with programs in the Center for Faculty Excellence, or received various course development awards. Please describe any significant achievements in detail.

Students

This section is to summarize data about and by the students in the program, for the most part separating the presentation for graduate and undergraduate students. The Program Review Team will be asked to assess the standards of the program's student scholarship and research, as well as the placements of graduates, as compared to peer institutions. They will consider the adequacy of student funding, the quality of advising and mentoring of students, student morale, student learning outcomes assessment results, average time to degree trends, career preparation activities and the distribution of students in relation to the university’s commitment to maintain diversity (e.g., women, underrepresented populations). They will talk with students and they will assess whether students feel that they can participate effectively to improve or revise the program.

To assist in preparing this section, there are several types of data the program can obtain, described in detail later in this manual. The program being reviewed should comment on the trends revealed by those data. Information can be obtained for the past five years and includes various configurations of data on enrollment; graduate applications and acceptances; credit hours generated; and degrees conferred.

You may want to present quantitative data in the format of Figure 4, “Demographic Profile of Graduate Students” and Figure 5, “Demographic Profile of Undergraduate Majors”. Please note significant trends. In addition, the following descriptive information would be included:

• How does the program seek to ensure a hospitable environment for all of its students?
• Comment on the racial, ethnic, and gender diversity in the program's graduate and undergraduate student populations. Note any significant trends in the last five years. Be sure to use multiple selection options for race and ethnicity to adhere to recent Department of Education changes in collecting these data.
• Present the program's recruitment, retention, support and placement procedures for underrepresented populations within the discipline.
• Describe the program's set of goals and expected outcomes for undergraduate education, measures that will enable them to tell how well these goals and outcomes are met, a description of the student learning outcomes assessment process, and examples of how assessment results have been used to improve the program in recent years.
• Describe the criteria by which graduate applicants are chosen for offers of admission.
• From what sources (e.g., federal funds, training grants, university awards and assistantships, program assistantships), and in what amounts, are graduate students supported financially? What proportions are currently supported, and what is the average rate of support/student? If 5-year data are available, please provide; otherwise note trends.
• Describe procedures for evaluating the progress of graduate students during the course of their training, including mentoring practices to provide feedback to students.
• Incorporate time-to-degree statistics and trends, including any efforts to streamline time-to-degree in recent years.
• Provide any available statistics and an analysis of those data for matriculated students who left before completion of their program. What steps have been put in place to increase overall retention or identify “attrites” earlier?
• Identify significant professional or intellectual contributions by students while in the program or after graduation/completion of training (e.g., faculty ratings, awards and honors, notable publications, and leadership roles in professional organizations.)
• Provide a précis of initial post baccalaureate pursuits of undergraduate majors and minors. Are the program's baccalaureate graduates well-prepared for graduate and/or professional studies and employment in their specialty? Are they aided in obtaining employment? If so, describe how.
• Describe career exploration and job search support services and activities within the program. Are graduate and undergraduate students encouraged to prepare for a variety of career outcomes?
• Provide a detailed record of the employment placement or further advanced studies of graduates upon completion of the program for the most recent five years, for each degree or area of concentration.
• Provide narrative and data on current and projected workforce demand, focusing on job growth in the field and how our programs are contributing to workforce needs in North Carolina and beyond.

In preparing this section, please obtain student input and evaluations of all phases of the program; this is accomplished through the use of program-generated assessment tools and student surveys and focus groups. Comment on the results of any such assessments.

Include a discussion of the impact of teaching by graduate student teaching assistants (GTAs) on the effectiveness of the undergraduate curriculum. Summaries and representative samples of student comments are encouraged. Comment on the legitimacy of student criticisms.

Leadership, Administrative Support, Facilities and Equipment, Institutional Relationships

The goal of this section is to assess the effectiveness of the program's governance and administration, and the adequacy, currency, and distribution of space, equipment, and support services, especially as related to achieving the program’s stated mission.

Leadership

Describe the program’s internal organization for governance and administration. Include an organization chart, if available.

Administrative Support

Comment on the adequacy of staff support, or support of others that may be appropriate, indicating the financial base of this support and clearly delineating state versus nonstate support levels. This should include technical, clerical, secretarial, and administrative support.

Facilities and Equipment

Comment on the status and adequacy of physical facilities, including:

• physical space for teaching, research, and administration;
• instructional, research, and administrative equipment;
• library holdings both within the program and university wide;
• computer capacity available to the program from the campus and/or from other agencies.

Institutional Relationships

Delineate relationships with other academic and research units, both on our campus and with other universities and outside units.

The Future

The program is asked to engage in an intellectual five-to ten-year planning process, taking into account the self-study data generated. This section should also share with the review team the vision that the program faculty and administration hold for the discipline. Where is the program going? Stimulate faculty dialogue and report both majority and minority views.

Discuss how the program integrates an ongoing assessment of its progress toward its stated goals into planning for the future. How does the program recognize its strengths and concerns, and how is this recognition translated into planning? Where applicable, delineate between undergraduate and graduate aspects of program planning.

The Program Review Team will be asked whether the program's plans for the future, and its new initiatives, are realistic, and reflect the needs of the discipline. They will be asked to provide insights as to significant new developments in education and/or research that they see as likely to occur in the next five to ten years, and to assess whether the program is positioned to capitalize on these developments. They will be asked for recommendations for program growth, retraction, and other changes.

In preparing this section, the program may wish to address the following questions (among others):

• What are the major disciplinary subspecialties represented in the program, and what are their strengths and the program’s concerns regarding them?
  
• Which of these subspecialties are likely to remain vigorous, and which are likely to be de-emphasized in the future, especially as current faculty retire? Are there subspecialties that should be eliminated or merged with others?
• Develop and comment on student enrollment projections for each subspecialty.
• Are there subspecialties not currently represented in the program for which development should be initiated? If so, what resources would be needed to mount such initiatives?
• Given that the total faculty size at Chapel Hill is likely to remain fixed in the future, and that graduate student enrollment has historically been most influenced by the ability of a program to support its students, what do you project for the size of your program (faculty, staff, graduate students, undergraduate majors) in the next five years?ten years? Explain. How does undergraduate enrollment affect the program?
• Are there specific resource needs, other than additional faculty slots and graduate student stipends, which might inhibit nourishment of current subspecialties or the initiation of new ones? Explain.
• Are there important curricular changes that are to be made, or which should be made, during the next five years? If so, what are the plans for their implementation?
• How can the quality of graduate and undergraduate education be improved? What steps would be of assistance in enhancing the quality of the program in comparison to those of other programs in the state? The region, and the nation both short-term and long-term?
• What are the plans to attract graduate students with higher academic qualifications than those presently enrolled in the program?
• What plans exist to attract highly performing undergraduate majors?
• What plans have been made to ensure and enhance racial, ethnic, and gender diversity in the graduate and undergraduate programs?
• How can the quality of mentoring in the program be improved? What policies could be implemented to assist junior faculty in their promotion and tenure activities? What best practices would aid in strengthening both faculty-to-faculty and faculty-to-student mentoring?
• Does the program have or intend to have a postdoctoral program that will prepare new doctorates from diverse backgrounds to enter and remain in the professorate?
• What plans have been made to recruit, support, and retain underrepresented faculty?

Next: Suggestions for the Self-Study Outline