Lifelong Learning Assessment
Summary/Reports
The ULO Project on Lifelong Learning began in Spring 2010, when Kennedy Library conducted a survey of student information skills in consultation with the ULO Lifelong Learning Committee. Information skills are a foundational component of lifelong learning, and they contribute to other ULOs including written and oral communication.
Information Skills Assessment Reports/Rubrics
Method
The survey was designed to identify student competencies by measuring performance on the Information Literacy Learning Objectives, which the library established in 2009. The survey presented students with a research scenario and asked them to respond to a series of 20 questions. Two versions were administered during a one-month period: one for lower-division and one for upper-division students. The versions differed by the order in which questions were asked and the wording of some questions.
Invitations to participate were emailed to 1,332 lower-division and 2,905 upper-division students. In addition, an open invitation was posted on the library website, and instructors who had previously brought students for library instruction were encouraged to announce the survey to current students. Approximately 98% of the responses came from the email invitations. Without adjusting for the remaining 2%, the lower-division response rate was 28% (367 respondents) and the upper-division response rate was 20% (578 respondents). The high response rate likely resulted from the promise of cash prizes; however, not all respondents answered all questions.
Results
Figure 1.10 (PDF) presents the mean scores in terms of percent correct for five questions for which there was a single response. A statistical analysis was conducted to determine whether the correct response to each item was related to Class Level and Instruction; the latter factor distinguished between students who had and had not received library instruction in research methods. In all cases, upper-division students did better than lower-division students. For three of the five items—thesis statement/promising research question, correct identification of citation example, and correct selection of the search term that would yield the fewest results—Class Level had a significant effect, demonstrating value added. There was a marginal effect of Class Level on the correct selection of the search term that would yield the most results. Significant effects of Instruction were found for the thesis statement and correct identification of the citation example. The question on the ethical use of ideas showed no significant effects of either Class Level or Instruction. Across all analyses, no significant interactions between variables were present
Appendix 1.1 has the full statistical analysis of WASC Student Learning Report (PDF)
The results demonstrate value added across several items on the survey, indicating higher levels of information literacy at the upper-division level. In addition, promising results for the educational effectiveness of library-related instruction were also found, with some indication that lower-division students attending such instruction consistently scored almost as well as upper-division students who had not attended such sessions. It should be noted that the outcomes measured in this scenario-based questionnaire necessarily focused on the means of finding and identifying information rather than on the more complex evaluative and synthetic skills associated with the critical-thinking aspects of information literacy.
Future Plans
The library plans to re-administer the information literacy survey in Spring 2012 to provide more and better data about student learning as a function of Library Instruction and Class Level. When revising the survey, more attention will be paid to the planned analysis, making sure that the upper- and lower-division questions are directly comparable.
Committee Membership
Members of the committee were composed of faculty and staff and were led by Navjit Brar (University Librarian and Assessment Coordinator). Other members included Thomas Bernard (Theatre and Dance), Brett Boedemer ( College of Liberal Arts Librarian), Fred DePiero (Associate Dean, College of Engineering), and Patricia Stoneman (Continuing Education).
Detailed Plans/Abstract of Plan for Assessment
Upon the development of the lifelong learning document (2008/09), it was decided to focus on the second attribute to help students learn the information and critical thinking skills.
- Beginning: Integrate the basic information and critical thinking skills (Attribute 2) into the General Education (GE) Area A: Communication A1–A3 lower-division course series. This was implemented in 2009, and is currently a practice in GE A1, A2, and A3 courses.
- Developing: Integrate the intermediate-level information and critical thinking skills into the GE Area C: Arts and Humanities (C1-C3) lower-division course series, the GE Area D: Society and the Individual (D1-D4) lower-division course series, and selected major courses. This process has been informally implemented in a few classes, and plans are being developed to formally integrate in all of the GE Area C and D classes, as well as specific major courses.
- Mastery-Level: Integrate the advanced level information and critical thinking skills into GE upper-division Arts and Humanities C4 synthesis courses, the GE upper-division Society and the Individual D5 synthesis courses, and Senior Project courses. This integration is presently occurring in the Senior Project courses. This process has been informally implemented in a few classes, and plans are being developed to formally integrate in all of the GE Area C and D upper-division courses, as well as specific major courses.
Assessment Methods
- Administer online surveys to determine the effectiveness of the information and critical thinking skills taught through online and classroom instruction
- Seek feedback from the faculty to assess students' understanding and use of information resources and critical thinking in their class research project(s)
- Administer both lower and upper-division scenario-based assessments to determine the level of student information and critical thinking skills
Results
- The scenario-based information skills assessment surveys provided consistent evidence of a positive correlation between higher percentages of appropriate responses and student attendance in library sessions at both the lower and upper-division levels.
- The in-class survey responses strongly suggested that library instruction was a positive learning experience for all students during their first two years at Cal Poly.
- The teaching faculty provided positive feedback and satisfaction with student research assignments.
Committee Activities and Timeline
| date | activity |
|---|---|
| 2008-2009 |
Assembled a committee to define lifelong learning, and develop the learning outcomes/lifelong learning rubric. Definition of Lifelong LearningLifelong Learning (LLL) is comprised of the foundational attributes that enable graduates to navigate a constantly changing world in which success is highly contingent upon the continuous intake, evaluation, and deployment of information. Lifelong learning is comprised of formal [1], non-formal [2], and informal [3] learning, whether intentional or unanticipated, that occurs at any point after a student has graduated. Many attributes characterize the lifelong learner, and these are best developed during the undergraduate and graduate years. Terms: Learning OutcomesA Lifelong Learner is one who: Effectively finds, critically evaluates, and formulates new ways of understanding information Lifelong Learning RubricDefining Beginning, Developing, and Mastery Skills in Lifelong Learning (PDF) |
| 2009-2010 |
|
| 2010-2011 |
|
| 2011 |
Information Skills Assessment Reports |
Resources for Students