newsletter header

 

NOTES FROM THE EXECUTIVE DIRECTOR

“There’s gonna be a showdown . . .”

by David C. Paris


“...so you better be better, when you get out on the floor, you better have your steps together, there's gonna be a showdown . . .”  Archie Bell and The Drells

Accreditation has been an ongoing subject of discussion and debate in higher education for years.  Within the next year, at least two groups, the American Council on Education (ACE) and the National Advisory Committee on Institutional Quality and Integrity (NACIQI), will separately be making recommendations about how accreditation ought to be adjusted or changed (or not).  A third, the Council on Higher Education Accreditation (CHEA), has an ongoing four-year review of the accreditation process.

One hopes that out of these multiple efforts there will be some clear path to improving accreditation and thereby improving the work of colleges and universities.  However, one fears that “there’s gonna be a showdown” between higher education and the government that will end up being more about issues of structure, power, and regulation than about improving student learning. 

The issues are too numerous to be discussed in depth here, but the stakes are high because of one key fact--the accreditation agencies are the gatekeepers for colleges and universities’ access to federal funds, particularly for financial aid.   Thus although accreditation is nominally a voluntary peer review process that monitors basic quality and encourages improvement, it is a de facto mandatory, quasi-regulatory process whereby colleges and universities must comply with standards in order to maintain eligibility or federal funds.  Colleges and universities in practically all cases cannot afford not to be accredited.  Therefore how accreditation is organized and to what ends has huge consequences for higher education

The fact that accreditation organizations play this role creates some peculiar dynamics and tensions.  In addition to their role in peer review, accreditation agencies are also implicitly or explicitly responsible for preventing fraud and abuse.  For example, the Higher Learning Commission (HLC) was taken to task by the Department of Education and brought before Senator Harkin’s committee to explain why certain for-profit institutions received accreditation (and thus access to federal funds) and why certain seemingly questionable practices in granting credit and degrees were not noticed and stopped.  However, it is unclear that accreditation agencies have either the tools or disposition to take on many of these issues.

At the same time, the government, given its increased expenditures in higher education, has an interest in quality assurance--that students complete degrees and that these degrees represent something of substance.  The Spellings Commission suggested that the United States was falling short in both the numbers and quality of degrees, and the Department of Education, until restrained by Congress, seemed poised to demand that accreditation agencies set standards for quality.  Many in higher education feared (rightly or wrongly) that this would bring “No Child Left Behind” requirements to colleges and universities.  However, just as the accreditation agencies may be ill equipped for law enforcement, the Department of Education may not be in the best position to define and judge quality or regulate it through accreditation.

Some have suggested the desirability of a clearer division of labor, with the government taking on consumer protection (and protection of the federal purse) while accreditation agencies focus on quality assurance.  Some have even gone so far as to suggest an end to the connection of accreditation to funding, thereby making clear the two spheres of concern and assigning authority over them accordingly.  Tempting though this may be, it immediately raises questions:  Would colleges and universities subject themselves to the accreditation process if it were no longer linked to funding?  Similarly, would the federal government pull back from its interest in quality issues, leaving it up to a peer review process it has been openly skeptical about?

And even if these questions could be answered, there are other difficult issues that need to be discussed.  For example, most people see the separate regional accreditation agencies as an historical artifact - why accredit regionally, especially as online education makes physical location and boundaries irrelevant?  Even if it were agreed that there might be one national set of accrediting standards, how that might be created and structured poses a significant challenge.  Similarly, as accreditation agencies have pushed institutions to do more assessment, some colleges and universities, typically elite and major research institutions, have pushed back, suggesting that perhaps accreditation should have different standards and processes for different categories of institutions.  Is there a “one size fits all problem” in accreditation or is it that some institutions want to create quality/assessment standards that they are more comfortable with?  Responses to these questions are also likely to be hotly contested.

No doubt the participants in each of the three endeavors want the best higher education system possible.  Moreover, every party has acknowledged that the accreditation agencies have made significant efforts, with some success, at pushing colleges and universities to focus more on the quality of student learning and assessing it.  The issue of the quality assurance is now widely acknowledged to be part of the discussion of higher education reform.

At the same time, the very existence of these separate efforts to address accreditation suggests that there are different ideas in the government and in higher education about what will make the system better.  These differences are likely to make answering the difficult questions above even more difficult, and a showdown seems inevitable.  In such a conflict it may be tempting to focus on some structural or regulatory issue - abolishing the gate keeping function, creating a national or sector accreditation process - rather than directly on getting colleges and universities to focus on measuring, reporting and improving learning.

It is doubtful that some structural change in the accreditation process is likely to truly change the gathering, reporting, and using evidence to improve student learning.  The path to change is more direct.  Accreditation agencies and colleges and universities themselves have to assume much greater responsibility for assessment and accountability.  A showdown over accreditation may be inevitable, but one can only hope that it does not distract from or otherwise impede more direct efforts.

"We better be better, have our steps together…"

 
NEWSLETTER
October 2011
PERSPECTIVES AND PRACTICE
Reflections on Issues, Efforts, and Experiences from Northern Arizona University
READING LIST
Current Industry Articles and Reports
PRESIDENTS’ ALLIANCE SPOTLIGHT
University of Central Florida
NEW MEMBERS OF THE PRESIDENTS’ ALLIANCE
Drake University, University of Mary Washington, University of Missouri-Kansas City
JOIN US ON THE ROAD
Will you be at either of the following events?

PERSPECTIVES AND PRACTICE
Reflections on Issues, Efforts, and Experiences from Northern Arizona University 

Assessing An Academic Partnership Between A Librarian And Faculty Member:
Influence On Student Use of Research Resources

by Amy Hughes and Melissa Birkett

In 2010, the Association of College and Research Librarians (ACRL) approved a set of information literacy standards for undergraduate psychology students, which aligned with the American Psychological Association’s (APA) Guidelines for the Undergraduate Psychology Major.  The ACRL Psychology Information Literacy Standards were derived from ACRL’s general Information Literacy Competency Standards for Higher Education.  The overlapping nature of these guidelines set the stage for a unique partnership between an academic librarian and a psychology faculty member at Northern Arizona University.

The primary goals of our collaborative effort were to promote students’ use of content specific information resources for a semester long portfolio project about a brain disease or disorder and to address the relevant guidelines set forth by the APA and ACRL.  This project addressed Goal 6 in the APA Guidelines for the Undergraduate Psychology Major and ARCL’s Psychology Standards.  There are four distinct learning outcomes associated with APA Goal 6; however, we focused on the learning outcomes stated in 6.1, along with the ACRL Psychology Information Literacy Standards.

Goal 6:  APA Guidelines for the Undergraduate Psychology Major
6.1  Demonstrate information competence at each stage in the following process:

  1. Formulate a researchable topic that can be supported by database search strategies.
  2. Locate and choose relevant sources from appropriate media, which may include data and perspectives outside traditional psychology and Western boundaries.
  3. Use selected sources after evaluating their suitability based on:
  1. Appropriateness, accuracy, quality, and value of the source.
  2. Potential bias of the source.
  3. The relative value of primary versus secondary sources, empirical versus nonempirical sources, and peer-reviewed versus non-peer-reviewed sources.
  4. Read and accurately summarize the general scientific literature of psychology.
     

ACRL Psychology Information Literacy Standards

  1. Help psychology liaison librarians and psychology faculty design the content of information literacy instruction for students in psychology.
  2. Make possible an evaluation of the information literacy skills of psychology students by delineating competencies that should be addressed.
  3. Encourage psychology liaison librarian and psychology faculty collaboration in the teaching of information literacy as a component of research methods in psychology. 

Working together, we drew upon expertise from our backgrounds in research and teaching to conduct this pilot project to provide a means to (1) assess student use of research resources and (2) create a learning experience aligned with the APA Guidelines for the Undergraduate Psychology Major and the ACRL Psychology Information Literacy Standards.  The undergraduate Psychology class we chose to work with was a sophomore level course with an enrollment cap of 70 students.  As part of a semester-long project in this class, students were required to create a portfolio containing a one-page letter to the reader, a table of contents, three student-selected research assignments completed at different times during the semester, a one-page reflection on each of the research assignments, a one-page conclusion and an annotated bibliography of all sources used to create the final portfolio. 

During a single class meeting, the librarian introduced students to online research resources including psychology databases, popular within the discipline, and health databases such as Consumer Health Complete and PubMed.  Medical dictionaries are included in many of these health databases and were demonstrated as a tool to assist with an etymology assignment.  In general, for each database, the benefits and disadvantages were discussed and example searches were performed.  In addition to the in-class lecture and demonstration, the resources discussed in class were embedded into the online course management site for the course.  Students could easily access specific resources without having to leave the online course site.   

The role of the faculty member in this partnership was to introduce research assignments, discuss research progress during the semester, answer student questions about resources and help direct students to appropriate resources.  To facilitate the collaborative instruction, the librarian and faculty member met two times during the semester and communicated goals for instruction and design of the research project. 

Annotated bibliographies from the semester portfolios were used as one factor to assess students’ use of research resources.  Although students completed research portfolio projects in previous semesters, annotated bibliographies were included for the first time as part of this collaborative effort.  The annotated bibliographies served as our primary assessment tool and encouraged students to think critically about the sources that they would include.  To evaluate student use of research resources, each bibliography was examined for the total number of citations, format of resource (book, journal, web site), and specific type of resource, which was divided into six categories.  The resource categories included (1) course textbook, (2) other books, (3) scholarly journal, (4) government health databases, (5) Google Health, (6) other websites.  The graded bibliographies were reviewed by the librarian and the faculty member, and reflected the learning objectives described in Goal 6 in the APA Guidelines for the Undergraduate Psychology Major and the ACRL Psychology Standards.  

The annotated bibliographies provided a rich source of data on student use of research resources.  In total sixty-seven annotated bibliographies, totaling 510 individual citations were analyzed. The number of resources cited in each of the bibliographies ranged from 0 to 19, with the majority of bibliographies citing between 9 and 11 resources.  In general, students sought answers from the course textbook, 43 out of the 67 annotated bibliographies cited the course textbook.  Databases from the National Institute of Health were also popular, including PubMed and MedlinePlus, and an EBSCOhost database, Consumer Health Complete.  Two hundred references cited these online databases.  Web sites were cited about as often as databases, accounting for 207 of all references.  The three most commonly referenced websites were MayoClinic.com, WebMD.com, and Google Health.  Online medical dictionaries, instead of the medical dictionaries included within the health databases, were cited in nearly 25 percent of the bibliographies.  Several web sites specific to a brain disease or disorder were cited.  General web sites such as about.com were cited less often.  Wikipedia was cited in only three bibliographies. 

Our secondary assessment tool was an anonymous and voluntary online survey to collect information about resource preferences and time devoted to research.  The survey was available to students throughout the semester, linked to each assignment in the course through the course management website.  Students were encouraged to complete the survey after submitting each assignment.  More specifically, questions from the survey asked students which resource was most helpful for each assignment and how long (in minutes) it took to complete the assignments.  The first question was intended to determine whether students utilized the same resources consistently for every assignment in the class.  The second question was intended to help determine the length of time students spent on research. 

In total 43 surveys were collected.  Overall results from the surveys were not considered representative due to the low response rate. Furthermore, each assignment survey differed in the number of responses.  The second assignment had the highest number of responses, in which 16 percent of the class responded.  The remaining assignment surveys had a 10 percent or lower participation rate. 

While the results from the online surveys were limited, several trends were noted across the surveys.  The most commonly used sources to complete the homework assignments were the course textbook followed by an online database, Consumer Health Complete.  From the surveys that were collected, which represented approximately 8 percent of the class, students on average completed the assignments in 20 – 25 minutes.

The assessment suggests that overlapping learning outcomes from different academic disciplines can be successfully integrated through course assignments.  Furthermore, we were encouraged by the number and diversity of resources that students used overall.  The partnership between the faculty member and the librarian reinforced the importance of using high-quality resources specific to a research topic.  Repetition of specific resources, through discussions and online links, appeared to be helpful in guiding students to high-quality resources.  We believe that students will be more prepared to begin upper division undergraduate psychology research because they have been exposed to key resources in the field. 

In the future we plan to implement a pre- and post-survey, during a class period that will ask students about their research behavior.  Allowing time in class to complete the survey may be one way to increase the response rate and collect more accurate data regarding resource preference and time spent on research.  Furthermore, we believe an in-class pre- and post-survey design could be used to provide additional information about student information literacy skills.   Undoubtedly, these skills will help students as they progress through the psychology curriculum.   

To conclude, we recommend collaborative projects between faculty members and librarians for courses that require this type of research component.  Fully utilizing resources, such as the expertise of faculty members and librarians, can also help achieve overlapping learning objectives and align coursework with nationally recognized standards such as those of the APA and ACRL.   

Amy Hughes is Academic Programs Librarian,
and Melissa Birkett is Assistant Professor of Psychology at Northern Arizona University.


REFERENCES

American Library Association (ALA).  (2000). Information Literacy Competency Standards for Higher Education.  Association of College and Research Libraries.  Chicago, IL:  American Library Association. 

American Psychological Association. (2007). APA guidelines for the undergraduate psychology major. Washington, DC: Author. Retrieved from http://www.apa.org/ed/resources.html.

READING LIST
Current Industry Articles and Reports 

The Association of American Colleges and Universities (AAC&U) announced the launch of Quality Collaboratives, a $2.2 million initiative funded by the Lumina Foundation to provide "direct funding to state systems and individual institutions for faculty and state system leaders and assessment experts to test ways to assure that students can demonstrate achievement of essential competencies across all areas and levels of learning, regardless of where they begin or end their educational journeys."

The National Institute for Learning Outcomes Assessment's (NILOA) twelfth Occasional Paper titled Assessing Learning in Online Education: The Role of Technology in Improving Student Outcomes, by Matthew Prineas and Marie Cini is now available.

In The Role of Feedback on Skill Development, faculty blogger Maryellen Weimer, Ph.D. describes from her experiences the importance of moving beyond content mastery toward the processes involved in learning.  

PRESIDENTS’ ALLIANCE SPOTLIGHT
University of Central Florida 

The University of Central Florida (UCF) first opened its doors in 1968 to nearly 2,000 students in the Orlando, Florida metro area. Since then, the university has greatly increased in size and has a current enrollment of over 56,000 undergraduate, graduate, and professional students. Classified as an institution with very high research activity, UCF offers 216 degree programs. A unique aspect of UCF is its comprehensive approach to assessment and institutional effectiveness, which engages numerous individuals, programs, and departments throughout the campus.  In fact, the university set a goal in 1994 for every academic and administrative unit to create mission statements, objectives, and learning outcomes for the improvement of programs and services. This goal was achieved, hence UCF’s well-developed Institutional Effectiveness assessment process.

The UCF community is at the forefront of assessment in higher education. For example, the university is a founding member of the Voluntary System of Accountability (VSA), a joint venture between the American Association of State Colleges and Universities (AASCU) and Association of Public Land-grant Universities (A-P-L-U), which provides accessible and comparable information about public colleges and universities to students and families. Additionally, UCF is also a founding member of the Alliance’s Presidents’ Alliance for Excellence in Student Learning and Accountability and is a pilot institution for the Excellent Practice in Student Learning Assessment institutional certification program. UCF has a strong tradition of sharing its assessment work with colleagues from other colleges and universities. Initiatives like these help UCF accomplish this objective. These national forums for sharing best practices with other institutions are helpful to improving the institution's work and communicating its value.

In addition to public statements of the importance of assessment, UCF is known for its meaningful and organized assessment work on campus. In particular, the work of the Operational Excellence and Assessment Support (OEAS) office and University Assessment Committee (UAC) are instrumental in ensuring the improvement of student and operational learning assessment. The OEAS office contributes by “providing support to all administrative units and academic programs through integrated processes that include continuous quality improvement, survey development, data collection, analysis, and guidance in assessment,” in order to maintain sustainable assessment efforts across the institution. Meanwhile the UAC, consisting of 19 Divisional Review Committee (DRC) chairs from academic and administrative units and two Ex-Officio members consisting of Vice President, Strategy, Marketing, Communications and Admissions Director and Operational Excellence and Assessment Support, maintains a hands-on role in assisting units conduct assessment for student learning, student development, and university services and operations.

At UCF, assessment is about making it useful. With full support from its president, provost, deans and VPs, faculty and staff engage in assessment within a protected supportive culture where the main objective is to improve student learning and operational practices.  To learn more about the notable assessment work at UCF, visit its Presidents’ Alliance institutional profile and campus assessment website.

NEW MEMBERS OF THE PRESIDENTS’ ALLIANCE
Drake University, University of Mary Washington, University of Missouri-Kansas City 

The Presidents' Alliance for Excellence in Student Learning and Accountability continues to grow as we welcome Drake University, University of Mary Washington, and University of Missouri-Kansas City. Learn more about how each institution is committed to improving student learning by viewing their Action Plans on the the Alliance's website.

JOIN US ON THE ROAD
Will you be at either of the following events?  

The Alliance will be presenting at the following events so be sure to join us!   

SEND US YOUR FEEDBACK
 As always, we welcome and appreciate feedback from our supporters. 
If you would like to share your comments and/or suggestions, please e-mail us at office@newleadershipalliance.org.

Find us on Facebook    View our profile on LinkedIn