Assessing the University Learning Objectives

Starting in 2012, Cal Poly moved from assessing the ULOs to the five core competencies as outlined by WSCUC's Re-Accreditation Handbook (2013). To learn more about current Cal Poly assessment efforts, please visit our Academic Assessment home page. For a list of current university-wide learning objectives, please visit the webpages that follow. (Please note: You will be redirected to the Academic Programs and Planning website).

University Learning Objectives
Diversity Learning Objectives
Sustainability Learning Objectives

To learn more about the ULO Project from 2008-11, continue reading the sections that follow.

introduction

In Fall 2008, Cal Poly began a concerted effort to assess its University Learning Objectives (ULOs). As part of that project, faculty, staff, and administrator worked together to define measurable outcomes for the ULOs and to evaluate student attainment of those outcomes directly. This multiyear undertaking was commonly known as the ULO Project.

Coordinated by the Director of General Education (GE) under the auspices of Academic Programs and Planning, the ULO Project aimed to measure the value added between freshmen and senior years and, where possible, to close the loop by recommending improvements to pedagogy and curriculum. 
 

Forming the Committees

• writing,
• oral communication,
• diversity learning,
• lifelong learning, and
• ethics.

Each consultant formed a broadly representative committee composed of faculty members representing GE and various majors across the university, as well as staff members from Student Affairs.

Developing and designing the assessments

• three committees (Diversity Learning, Lifelong Learning, and Ethics) developed survey/test instruments to collect essay/multiple-choice responses;
• two committees (Writing and Oral Communication) reviewed class assignments;
• and one committee (Diversity Learning) used focus groups to explore student attitudes.

Each committee developed rubrics to identify traits and articulate levels of development.

The committees intended to use student work from lower and upper-division GE as well as major courses to determine the value added during a Cal Poly education. Only three were able to accomplish this goal: Writing, Diversity Learning, and Lifelong Learning. While these assessments are best considered as pilots, the committees have made some modest recommendations for educational improvement based on the evidence collected.

The results (through 2011)

The university has already implemented some improvements, most notably with workshops sponsored by the Center for Teaching and Learning (now known as the Center for Teaching, Learning, and Technology) on ULO-based assessment of writing and critical thinking in the senior project.

In connection with the ULO Project, Academic Programs and Planning revised the program review process to include the mapping of major courses and co-curricular activities into the ULOs. Each program identifies where the ULOs are introduced, developed, and mastered in the major curriculum. A map of the GE curriculum is provided, although programs are not explicitly required to consider the GE and major maps together. The intention is to encourage the faculty to locate and address any significant gaps in the students’ education.

As an experiment in the assessment of transferable skills across the GE/major divide, faculty members from GE and the Orfalea College of Business ran a pilot of Integrated Program Review in Spring 2009. They applied the University Expository Writing Rubric to the work of Business students and used the assessment results to discuss how to improve student attainment of the ULO on effective communication. Though the group identified a number of opportunities for strengthening student writing, the integrated model has not been repeated nor revisited.
 

ULO Summary Reports

• Writing Proficiency (pages 9 -11)
• Oral Communication (pages 11-13)
• Diversity (pages 12-14)
• Ethics (pages 14-16)
• Lifelong Learning (pages 15-16)
  
   

Description of Consultant/Committee Activities

• Provided a central source of information on the teaching, learning, and assessment for the specific ULO they are responsible for. 
  For example: To take an example from the Writing ULO Committee :"How well are Cal Poly students writing? How is this measured? Have there been measurable improvements?"
• Analyzed student learning data reported by the faculty to pinpoint areas in need of attention from an institutional perspective, and made recommendations to programs when asked to. This process would be addressed most intensively during academic program review.
• Devised scoring rubrics with campus-wide input so that the assessment of student learning would be valid and reliable across the different divisions and throughout a student's academic career. In this way, we tracked students' development as they achieved growth in the attainment of the specific learning objective.
• Analyzed the assessment data and then CLOSED THE LOOP, explaining where specific pedagogical and curricular changes could be made in order to improve student learning. Once changes were made, the plan was for another round of assessment to begin (three-year cycle) to determine how much student learning actually improved, as measured against the baseline established in the previous assessment.
• Worked with the Center for Teaching and Learning (CTL) and offered faculty development workshops, which served as a central venue for conversation across divisions and which provided teaching strategies for faculty on how to implement changes. This approach provided for robust faculty engagement with an assessment process that led to potentially improving the learning environment.
• Developed information and content for the ULO website.
• Informed the Academic Assessment Council and the GE Governance Committee quarterly on progress.
• Provided an annual report/presentation to be shared with the Academic Assessment Council, the College Assessment Councils, the GE Governance Committee, and the Academic Senate.