Overview of University Learning Objectives (ULO) Assessment Project
Begun in Fall 2008, the ULO-based assessment commonly known as the ULO Project was coordinated by the Director of General Education (GE) under the auspices of Academic Programs. The project marked a concerted effort to define measurable outcomes for the ULOs and to directly assess student attainment of these outcomes. Although the individual assessments are at various stages of completion, the project as a whole aims to measure “value added,” i.e., progress from the freshman year to the senior year, and, where possible, to close the loop by recommending improvements to pedagogy and curriculum.
The project began with the appointment of five faculty members as ULO Consultants, each representing a different ULO-based skill: writing, oral communication, diversity learning, lifelong learning, and ethics. Each consultant formed a broadly representative committee composed of faculty members representing GE and various majors across the university, as well as staff members from Student Affairs. After reviewing nationwide best practices, two committees (Writing and Oral Communication) reviewed class assignments, three (Diversity Learning, Lifelong Learning, and Ethics) developed survey/test instruments to collect essay/multiple-choice responses, and one (Diversity Learning) used focus groups to explore student attitudes. All developed rubrics to identify traits and articulate levels of development.
The committees intended to use student work from lower and upper-division GE as well as major courses to determine freshman/sophomore and junior/senior levels of attainment and thereby measure the value added during a Cal Poly education. Only three were able to accomplish this goal (Writing, Diversity Learning, and Lifelong Learning). While these assessments are best considered as pilots, the committees have made some modest recommendations for educational improvement based on the evidence collected. The university has already implemented some improvements, most notably workshops sponsored by the Center for Teaching and Learning (CTL) on ULO-based assessment of writing and critical thinking in the senior project. In connection with the ULO Project, Academic Programs revised the program review process to include the mapping of major courses and co-curricular activities into the ULOs. Each program identifies where the ULOs are introduced, developed, and mastered in the major curriculum. A map of the GE curriculum is provided, although programs are not explicitly required to consider the GE and major maps together. The intention is to encourage the faculty to locate and address any significant gaps in the students’ education.
As an experiment in the assessment of transferable skills across the GE/major divide, faculty members from GE and the Orfalea College of Business ran a pilot of Integrated Program Review in Spring 2009. They applied the University Expository Writing Rubric to the work of Business students and used the assessment results to discuss how to improve student attainment of the ULO on effective communication. Though the group identified a number of opportunities for strengthening student writing, the integrated model has not been repeated nor revisited.
ULO Summary Reports
The most recent ULO summary reports are contained in the 2012 WASC EER Report (PDF).
- Writing Proficiency (pages 9 -11)
- Oral Communication (pages 11-13)
- Diversity (pages 12-14)
- Ethics (pages 14-16)
- Lifelong Learning (pages 15-16)
Description of Consultant/Committee Activities
Provided a central source of information on the teaching, learning, and assessment for the specific ULO they are responsible for.
For example: To take an example from the Writing ULO Committee :"How well are Cal Poly students writing? How is this measured? Have there been measurable improvements?"
- Analyzed student learning data reported by the faculty to pinpoint areas in need of attention from an institutional perspective, and made recommendations to programs when asked to. This process would be addressed most intensively during academic program review.
- Devised scoring rubrics with campus-wide input so that the assessment of student learning would be valid and reliable across the different divisions and throughout a student's academic career. In this way, we tracked students' development as they achieved growth in the attainment of the specific learning objective.
- Analyzed the assessment data and then CLOSED THE LOOP, explaining where specific pedagogical and curricular changes could be made in order to improve student learning. Once changes were made, the plan was for another round of assessment to begin (three-year cycle) to determine how much student learning actually improved, as measured against the baseline established in the previous assessment.
- Worked with the Center for Teaching and Learning (CTL) and offered faculty development workshops, which served as a central venue for conversation across divisions and which provided teaching strategies for faculty on how to implement changes. This approach provided for robust faculty engagement with an assessment process that led to potentially improving the learning environment.
- Developed information and content for the ULO website.
- Informed the Academic Assessment Council and the GE Governance Committee quarterly on progress.
- Provided an annual report/presentation to be shared with the Academic Assessment Council, the College Assessment Councils, the GE Governance Committee, and the Academic Senate.