Contact Information

 

AO Assessment Office homepage

Project Description

BACKGROUND
Assessment of student learning is not new to UHM. Existing assessments at UHM focus on the students’ degree program, grades, and retention & graduation rates.

This project builds upon existing assessment practices by

  • emphasizing the importance of assessing core areas (general education): written and oral communication; ethical reasoning; symbolic reasoning; global understanding; and Hawaiian, Asian, and Pacific issues;
  • defining and evaluating proficiency in the core areas (general education);
  • going beyond student grades and test scores by including student perspectives and voices to investigate the when, how, and why behind the student performance; and
  • following a student cohort throughout the undergraduate experience so that student development over time can be studied.

The results of this integrated, longitudinal approach to student learning assessment provides insights into what and how the millennial generation student at UHM learns. Because the project combines performance assessment with student explanations, the results can better guide updates to the curriculum and pedagogy; the results reveal how well students are achieving in the core areas and, as importantly, when, how and why.

OVERVIEW
The project started with 250 first-time fall 2010 freshmen in August 2010. Students are part of the project for their entire undergraduate experience (note: participation is voluntary and students may withdraw at any time without penalty). The data collection will end in May 2016. The project includes the following activities:

  • Students submit assignments and exams from general education classes; faculty assessment teams examine the student materials to determine students’ levels of knowledge, skills, and values in the core areas. No personal student or faculty information is reported; results are reported in the aggregate.
  • Students describe the factors that enhance or hinder their learning and their perception of the "Mānoa Experience" through in-depth surveys, written reflections on their learning, and discussion groups.
  • The Assessment Office disseminates and uses the project’s results to increase student learning, improve students’ educational experiences, and ensure UHM graduates are prepared and equipped to contribute to the global workforce.

Student commitment to participation is vital to the success of this project. The 250-student cohort is divided into two groups of 125 students. Each semester, students in Group A complete 3 in-depth on-line surveys and submit written reflections on their learning and educational experiences. Students in Group B also complete 3 on-line surveys and submit written reflections on their learning and educational experiences each semester. In addition, Group B students submit coursework (e.g., exams, papers, projects, presentations) each semester to be evaluated by faculty assessment teams, and they participate in annual discussion groups.

GOALS & OBJECTIVES
The overarching goal of this project is to improve students’ educational experience and ensure that the general education program is effective in preparing students in core academic areas..

Goal 1. Determine minimum levels of proficiency for the core academic areas.

Objective A. Evaluate student work to establish the minimum proficiency of students in the core general education areas: written and oral communication; ethical reasoning; symbolic reasoning; global understanding; and knowledge of Hawaiian, Asian, and Pacific issues. When faculty teams evaluate the student work, they will set minimum standards for student proficiency. Currently, there are only individual course standards; this project seeks to establish core curriculum standards.

Goal 2. Improve the general education curriculum and pedagogy.

Objective B. Identify the key factors that contribute to student learning and the factors that hinder learning. Having evidence on what helps learning and what hinders learning helps the faculty make decisions about how to improve or modify the curriculum and pedagogy. Evidence has been limited to individual anecdotes; this project tackles systemic issues related to general education teaching and learning.
Objective C. Create faculty development materials based on the key factors and disseminate via workshops and the internet. Because faculty will need assistance in improving the general education curriculum and pedagogy, the Assessment Office and other faculty groups will offer engaging and interactive faculty development workshops.

Goal 3. Ensure the general education curriculum and employer needs are aligned.

Objective D. Hold an annual conversation with local employers in the private sector to discuss the general education curriculum and employer needs. There is little to no interaction between UH Mānoa and employers regarding general education core areas and thus these conversations are useful for both groups. UH Mānoa can use the information gained to guide general education modifications.

POPULATION SERVED
The immediate beneficiaries are the students who participate in the project: they receive UHM Bookstore Gift Cards that they can use to offset the cost of books and supplies; they may benefit by regularly reflecting on their learning and experiences in the general education program. The faculty who oversee the general education program (35-40 faculty members) benefit by receiving information about the effectiveness of the general education program. They use the results to identify strengths and weaknesses and take action to improve student learning.

Ultimately, this project will impact all of the students at Mānoa (Spring 2010 UHM enrollment 12,941 students), faculty who teach general education courses (approximately 800 faculty members), and in turn the businesses and communities in which our graduates will work and serve. Investment in this project has an immediate impact on the student participants and a long-term impact on future UHM students. The long-term impact includes curriculum improvements, establishment of best teaching practices, and students better equipped to contribute to Hawai‘i’s workforce. As UHM is the flagship campus for the only public university system in the state of Hawai‘i and annually produces more graduates than any other university in Hawai‘i, the improvement of its curriculum and environment therefore directly impacts the quality of the workforce and the state at large.

EVALUATION PLAN
The faculty members in the UHM Assessment Office lead the project and evaluate its success. They monitor the process and implementation of the project and ensure timely progress. In addition, they take a continuous improvement approach by using evaluation results to modify procedures and create subsequent survey and focus group questions

Process Measures

Timeline

Faculty teams evaluate student work collected from student participants (Group B).

Student work will be evaluated every January, 2012 to 2017.

Student participants (Groups A & B) complete six online surveys each year.

Ongoing, September 2010 to May 2016.

Student participants (Group B) submit coursework/exams.

Each semester, December 2010 to May 2016.

Student participants (Group B) take part in an annual focus group.

Annually, March 2011 to April 2016.

UH Mānoa engages in an annual conversation with local private employers.

Annually, July 2011 to July 2016.

Assessment Office and other faculty groups create and disseminate faculty development materials to improve or modify the general education program.

Draft materials and seek faculty feedback, 2012-2013. Faculty development workshops start in February 2013 and are regularly offered.

Outcome Measures

Timeline

Faculty teams set minimum proficiency standards for each general education core area.

Student work discussed every January and draft standards established by 2013. Standards established by 2015.

Assessment Office identifies at least three key factors that help learning and three that hinder learning.

Ongoing, November 2010 to May 2016.

UH Mānoa determines extent of alignment between general education core areas and Hawai‘i private employer needs.

Ongoing, July 2011 to July 2016.

Faculty who oversee the general education program modify criteria for the curriculum and/or pedagogy.

Starting August 2013 and continuing through 2016.

Faculty who teach in the general education program use the faculty development materials to improve/modify courses.

Starting August 2013 and continuing through 2016.

PRINCIPAL INVESTIGATOR
Monica Stitt-Bergh, PhD, University of Hawai‘i at Mānoa Assessment Office (AO). The AO’s main functions are (a) to support the use of program and institutional assessment to improve the quality of student learning and (b) to collaborate with faculty, staff, students, and administrators to establish meaningful, manageable, and sustainable assessment. It has successfully conducted workshops and consultations for over 200 faculty members during the last year.

AOAssessment Office
University of Hawaiʻi at Mānoa
2500 Campus Road, Hawaiʻi Hall 107
Honolulu, HI 96822
Office email: airo@hawaii.edu

updated 06/01/2012 (author: Monica Stitt-Bergh)