Standing on the shoulders of a forty-year-old portfolio tradition, we are exploring all of the ways in which this rich history of assessing the quality and character of our students’ learning through their paper portfolio submissions can lead to a promising digital future of ePortfolio-based Outcomes Assessment.
Our efforts to integrate the use of ePortfolio for supporting Outcomes Assessment on our campus fall somewhere between emerging and developing. These nascent efforts, however, are significantly enhanced by the fact that the Manhattanville College strategic plan explicitly includes the integration of ePortfolio into college-wide teaching, learning and assessment practices. Members of our ePortfolio team serve on key committees charged with the development, oversight and assessment of our general education and discipline-based curricula. These conditions provide us with invaluable opportunities for influencing and facilitating curriculum development and continuous improvement planning that is outcomes driven. Our new core curriculum is being developed with ePortfolio-supported outcomes assessment in mind, and an ever-increasing number of faculty, departments and programs are beginning to use ePortfolio as the platform in/through which they conduct authentic and holistic Outcomes Assessment designed to improve student learning.
Authors: Alison Carson, Jim Frank, Gillian Greenhill Hannum and Sherie McClam
Part I: Setting the Stage: Outcomes Assessment on your campus:
While we have had a long history of the Manhattanville Portfolio System, established in 1973 with the publication of the Manhattanville Plan, these student portfolios examined individual growth across a student’s academic career, and have not been fully leveraged as an assessment of learning outcomes for a Manhattanville graduate, most likely because learning outcomes had not been established institutionally. In 2008, Manhattanville College was put on warning by our accreditation body, Middle States. Within the warning was reference to our lack of assessment practices, within courses, departments, and across the institution. Since the warning, we have hit the ground running, establishing a General Education subcommittee that developed a general education system that included learning outcomes for 8 different competencies. This group also developed some rubrics for the evaluation of the competencies. Both the learning outcomes and rubrics were based in part on the VALUE rubrics. Following this work, our faculty by-laws were revised, and a Core Curriculum Committee (CCC) and a Committee on the Assessment of Student Learning (CASL) were established, institutionalizing the oversight and assessment of student learning in both general education and programs. As a result of this and other work, we were taken off warning and required to submit a monitoring report in September 2012. CASL has been responsible for monitoring and supporting the development of and engagement in programmatic assessment. This group has been generally effective, however they initially undertook this endeavor without any support or expertise in the area of assessment. In Fall 2012, an Assistant Provost for Institutional Effectiveness was hired to support all assessment practices on campus.
In the spring of 2012, it became clear that assessment design was different from actual assessment, so once again, we had to spring into action to engage in the practice of assessment. The first working group established baseline measures using work submitted to student Portfolios. We assessed Written Communication and Global Awareness, both using a revised VALUE rubric. In the fall of 2012, a second working group was established to assess Critical Analysis and Reasoning. Again, a revised VALUE rubric was used to accomplish this work. The CCC is currently working on a revision of the 2009 general education curriculum, and our Portfolio (which moved to Digication in the spring of 2013) will play a critical role in the assessment of this new curriculum. Until now, the Manhattanville Portfolio System has been generally thought of as a separate component of general education, but CCC along with the Board on Academic Standards is working to bring the Portfolio back to the center of students’ education. In terms of student learning assessment, the Board on Academic Standards, the faculty group that oversees and evaluates all Portfolios, has established learning outcomes for the Portfolio and uses revised VALUE rubrics to evaluate student essays submitted as a Portfolio requirement. The Digication Committee-based Assessment System has been used to evaluate all student Portfolio submissions beginning in Spring 2013.
Part II: Outcomes Assessment Development Story
When thinking about the role of ePortfolio in Outcomes Assessment on our campus, it is important to note that we have administrative support at the highest levels and that ePortfolio development has been written into our strategic plan. This, of course, gives us tremendous leverage for integrating ePortfolio in campus-wide curriculum, instruction and assessment development efforts. With this in mind, we envision ePortfolio supporting our outcomes assessment work at the institutional, departmental, programmatic, course and student levels. At the institutional level, we envision ePortfolio being the primary platform in/through which we will capture and assess artifacts that have been tied to general education learning outcomes. We are exploring two different approaches to accomplishing these goals—the latter being the most likely. In the first, we imagine following a Clemson University-like approach in which students are responsible for identifying and uploading artifacts that demonstrate that they have met the required general education learning outcomes. Knowing in advance what those learning outcomes are and the kinds of artifacts that can be used to demonstrate their accomplishments, students select appropriate artifacts for making the case that they have met the specified learning outcomes.
In the second possible approach, specific courses will be identified in which general education (core curriculum) learning outcomes can be met. Those courses will have been accepted as core curriculum courses by demonstrating that students will produce work (artifacts) through which they will be able to demonstrate that they have met the core curriculum learning outcomes associated with that course. In this case, core curriculum course instructors will ask students to upload these learning outcome-linked assignments/assessments/artifacts into a separate “course” ePortfolio that is used to assess the degree to which our core curriculum learning outcomes are being met. In this model, it is likely that the core curriculum course instructors will also evaluate the work submitted in their classes using a rubric developed by CCC or CASL evaluating the core curriculum learning outcomes.
Following this second model, First Year Program instructors has already engaged in the assessment of a course signature assignment called the Freshman Essay. Students uploaded the final Freshman Essay to a specific location in their Manhattanville (e)Portfolio and then submitted to a specific course built in Digication. Using the Digication Committee-based Assessment System, these artifacts were then sampled and assigned to specific FYP faculty for evaluation using the FYP rubric for Critical Reasoning and Analysis. While we did not have 100% participation from the FYP instructors, the percentage of evaluated Freshman Essays increased dramatically from previous paper assessments.
In ways that are similar to the process by which ePortfolio will be used to assess core curriculum learning outcomes, we expect programs, departments and individual faculty members to use ePortfolios as platforms in/through which artifacts representing students’ success can be collected and assessed. In each case, instructors, departments, programs and the college will engage in the process of clearly defining outcomes and the criteria for determining the degree to which the evidence collected represents success in meeting those outcomes. Whether understanding the degree to which a student has been successful in meeting the learning outcomes associated with a particular course or the degree to which our core curriculum outcomes have been met, we foresee the ePortfolio playing a pivotal role.
Part III: Conceptual Framework
Our once externally-motivated push to identify learning outcomes that would satisfy our accrediting commission has become an internally-driven desire to improve our capacity to meet the needs of our students. This begins with exploring (inquiry) what those needs are in terms of learning outcomes, and it requires viewing our students holistically. Taking this view demands that we work in a collaborative and integrative fashion to understand the ways in which our courses, departments, programs and our institution can support our students in meeting those outcomes. With ePortfolio as an organizing platform for collecting and assessing evidence of student learning, we expect our faculty and staff to engage in an ongoing and iterative process of reflecting on and learning from our assessments of the degree to which students are meeting our desired outcomes. This process of asking ourselves where we want to go with our students, how we’ll know when we get there and what experiences will give them the greatest opportunity to be successful is truly beginning to permeate all aspects of our teaching and learning discourse. When it is done mindfully, this process inherently involves inquiry, reflection and integration.
As members of the Manhattanville faculty, our ePortfolio team members serve on key committees involved in the development of an institutional learning process driven by outcomes assessment. Through our service on the Board on Academic Standards (BOAS), we have the opportunity to demonstrate the ways in which ePortfolio can be used to facilitate the process of collecting, sampling and assessing evidence of student learning for determining not only individual student success, but also the degree to which students have met the institutional learning outcomes associated with the Manhattanville Portfolio System. Through our service on the Core Curriculum Committee (CCC), we have the opportunity to promote and facilitate the development of general education learning outcomes and the use of ePortfolios for assessing evidence of meeting those outcomes. Through our service on the Academic Policy Committee (APC), we have the opportunity to weigh in on policies defining criteria for the approval of new courses throughout our curriculum. These policies now require course developers to clearly define learning outcomes, work that will be accepted as evidence of meeting those outcomes and the experiences through which students will develop the capacity to produce that evidence. As faculty submit courses for approval by APC, they are beginning and will continue to be referred to ePortfolio as a platform for collecting, organizing and assessing that evidence for the degree to which specified learning outcomes have been met.
We envision one of our next steps to be the development of clear models for how ePortfolio can help support outcomes assessment across our campus. Each of these key committees is looking to the ePortfolio team to help them conceptualize a process by which ePortfolio can be used to support all aspects of the outcomes assessment process. Developing models that will enable us to reach/hook a broad audience will involve working with individual course instructors, departments and program coordinators, some of whom are already using ePortfolios for outcomes assessment. These faculty members and program coordinators also constitute pockets of interest on which we can capitalize. For example, the music department has developed an ePortfolio template to be used by students for their sophomore review. Using this template, students reflect on their experiences as music students and map out their plan of study.
Similarly, the studio art department is using ePortfolio as a platform for assessing work for students’ junior review, the art history department is using ePortfolio for assessing senior capstone projects, the Castle Scholars program is also using ePortfolio for assessing capstone projects, the first year program is using ePortfolio for collecting and assessing freshman essays, and any number of faculty members are using ePortfolios as places into which students accumulate evidence of their learning or as an end product that constitutes evidence of meeting course-based learning outcomes. Refining and building on these examples will be a critical next step in the process of demonstrating how ePortfolio can support outcomes assessment across our campus. With these examples/models in our kit bag, we will be able to more effectively promote ePortfolio as an approach to outcomes assessment with other faculty members, departments and programs.