In this Issue:

President's Annual Report 1999-2000

Board Members 2000-2001

ACRL/NEC 2000 Spring Conference Report
   Breakout Sessions

Women's Studies Spring 2000
Program Report

New Co-Chairs for the Preservation SIG

ITIG Launches Technology Column

Massachusetts Special Collections Directory

Continuing Education Committee
Welcoming New Members

ACRL/NEC Listserv - Special Thanks

Updating Member Information

Announcements

NEBIC & Simmons College Present
"Information Literacy into the Curriculum"

Business Librarians' Interest Group Announces
Spring 2000 Program


Return to Masthead

ACRL New England Chapter News Online
Spring 2000, No. 90

Collections
Jonas Barciauskas, Boston College,
Gianna Gifford, Simmons College, Reporter

People shared ideas about ways to get at whether what we are collecting is actually useful to our faculty members and students. Many interesting questions were raised: How do consortial relationships effect assessment? Is it still appropriate to compare our collections against standard bibliographies? How do we compare statistics that we get from database vendors when they are so different and/or incomplete? Can we assume that our approval plans are providing a solid, base collection? Is there a way to work with faculty to see if student papers and bibliographies include resources from our collections, and would this be a method to measure the quality of our collections? We all agreed that assessment seems to be linked to use and to the missions of our colleges/universities-this relationship will be increasingly important to collection development. We concurred that collaboration with faculty is necessary to insure that our collections are relevant and connected to the curriculum, and will be useful to stud
ents.

Back to Top

User Needs Assessment
Brinley Franklin, University of Connecticut
Harvey Varnet, Simmons College, Reporter

Brinley opened the session by describing some of the ARL (Assn. of Research Libraries) ongoing projects, including the ARL New Measure Initiative. Areas of interest to ARL libraries include, campus assessment activities and the importance of library participation; SERVQUAL, a methodology to assess service quality that will be tested at several institutions; an identification of high impact library functions that call for economic study (cost drivers); developing self-assisted guides for measuring activities (i.e., interlibrary loan/document delivery); and usage measures for electronic resources.

The group of 40+ librarians then spent their time developing responses to "what are the right questions" and "what are the right measures". Some of the results were: knowing the demographics of your institution, asking users to rate themselves on their library abilities, figuring out the best way to serve remote (virtual) users, asking users how they learn best and what environment and resources promote that learning; asking users to assess web sites, their utility and appropriateness; asking students which assignments are
difficult to complete and how the library could have been more helpful; measuring user confidence with using library resources; asking faculty to assess collections; and asking alumni to reflect on how past library use prepared them for life-long learning, and what they'd recommend as changes in library programming.

The participants all agreed that "assessing" services and resources is an important and serious undertaking and should be done on a regular basis.

Back to Top

Human Resources
Artemis Kirk, University of Rhode Island
Raynna Bowlby, Brown University, Reporter

Participants acknowledged that sometimes it is hard for us as library
staff to see our direct connection to student learning given our unique organizational hierarchies. Artemis challenged us to question HR models and techniques that have been designed for the corporate sector. The breakout participants did, however, embrace certain HR philosophies of leading management theorists such as Rosabeth Moss Kantor and Tom Peters, including the concepts of having value-based missions and goals, doing meaningful and challenging work, creating an environment that inspires collegiality and collaboration, and having the opportunity to grow. Among the more thought-provoking questions we might ask for assessment of library human resources and the educated student are: Why do we do [each] task? What is its contribution to our overall mission of creating educated students? Do we give smile-free service? And, if not, is there something we can do about it? Are our libraries collaborative enough?

Back to Top

Online Catalog
Sherrie Bergman, Bowdoin College
Geri Sheehan, Stonehill College, Reporter

The group, mainly from undergraduate institutions, agreed that our topic, the online catalog, would benefit from both public and technical service points of view. Most in attendance at this session were from the technical services arena. Sherrie began by outlining the evolution of the library catalog. It began as a record of physical products actually owned by the library and has expanded in recent years to include links to other collections. She discussed standards for use of the catalog, such as: timely and orderly access to materials, and provision of an accurate representation of available resources. She stressed the importance of keeping remote users in mind. She also proposed aspects of assessment including, input (raw materials), output (quantifying the work done) and outcomes (how the user is changed as a result of using the product).

The questions we formulated included, is the online catalog still the central starting point for students' research? Does the catalog design enhance the use of other resources? Are catalog records confusing? Do our catalog and web page use consistent terminology? Are all items, regardless of format, represented? What are the limitations of our Integrated Library System in terms of indexing capabilities and reporting features? Are there adequate online instructions for the effective and efficient use of the catalog? Does our standard system of subject headings really match the users search habits? How can we provide for more natural language searching opportunities? Are we collaborating with faculty in the teaching process to include the use of the catalog?

Measures discussed were, quantitative (number of times the catalog has been searched, book circulation statistics), qualitative (search terminology history, quality review of student bibliographies), number and type of ILL requests generated for materials that we already own, checking course syllabi which direct students to use the catalog (do faculty suggest subject headings or send them to specific titles?), user surveys, or focus groups of students to discuss their use of the catalog, and partnerships with vendors and faculty.

Back to Top

Instruction
Ann Schaffner, Brandeis University
Sarah Wenzel, Boston University, Reporter

We began by reviewing a few things from the morning session. Assessment should be considered a continuous/organic process. The trick is being able to translate the information gained from assessment into improvements. There is a difference between teaching and learning, as well as between opinions about a class and achievement. Information literacy is the goal.

The participants voted not to take the time to make a list of competencies, but rather to examine existing lists. Schaffner challenged us to look beyond the library. She drew a picture of the "educated student" of the 1950s: taking a core of survey courses and distribution requirements in his or her major. This curriculum evolved as survey courses were replaced with more specialized courses and distribution requirements either broadened among disciplines or disappeared. Concerns were raised about fragmentation of knowledge, skills and competencies, a depersonalization of instruction, and the existence of a canon (or not, and what?). Several reform movements have emphasized new requirements such as first-year experiences and "linked learning."

Schaffner gave us two examples of competency lists: ACRL Information Literacy and Interdisciplinary. A participant asked if she saw any trend towards bringing back comprehensive exams. Schaffner was not aware of any such movement.

We then looked at the ACRL list, which included:
life-long learning/transferability of skills
stimulating curiosity and creativity
persistence

We noted that the latter two are not skill/content-based criteria. They encompass an attitudinal aspect imbedded in the standards, although not explicitly expressed.

Next we turned to the interdisciplinary list, which are even more attitudinal. Stress is placed on the "model learner" concept, where the professor learns with the students and can show them attitudinal qualities. This technique emphasizes being able to communicate ideas and synthesize information. The Interdisciplinary list is of competencies really outside of the library, so the library must involve other departments and efforts.

We asked ourselves "How do you measure effective? How do you teach excitement?"

We noted that there are vast differences in cognitive ability between an 18-year old and a 21-year old ("right answer" vs. "ambiguity"). Recognition of need was added to the ACRL list.

Turning to measurement, Schaffner drew the following chart:
Input Output Outcome
Staffing Teaching Learning (Instruction)
Statistics Tests
Portfolio
Survey

Overriding questions include how to transfer the results of measurement into changes, and improvement versus accountability.

We looked at means of measuring:
statistics (usually output, not effect)
surveys (usually satisfaction ­what people think happened)
tests
portfolios
focus groups (similar to surveys)
in-class assessment (fast, but requires a continuing relationship)
performance (have them take what they learned and apply it)

Schaffner described some of the results of a library survey held at Brandeis last year. She noted that most users do not know what the library already has.

She also described some research being done by one of her colleagues. They have two across-the-board instructional programs (they give fifty instruction sessions per year, thirty-five of which are during the first term). This year the students have been divided into groups. One group is given a pre-test, instruction session, and a post-test. The second group receives a pre-test, then after the semester is nearly over a post-test, and then instruction. A third group is not given a pre-test, but is tested after the semester is nearly over and prior to instruction.

Suggestions for questions to ask students are:
How often do you ask for help?
List differences between two different citations.
What steps would you go through to find scholarly articles on toxic waste?
Evaluate two given web sites.
Questions about Boolean searching.
What do you find confusing about the library? (Or, other attitudinal questions)

Other suggestions for evaluating student behavior include:
Asking the instructor to send you bibliographies from their papers.
Faculty inviting librarians to witness students' final presentations.
Faculty comparing the results from those students whom came to the library and those who did not.

We then broke into two groups, and discussed two different scenarios. We were asked to determine how we would assess the students and then translate our results into a difference.

Scenario A: Upper-level psychology courses; sessions on teaching Web-searching and Psychlit; targeted assessment; this is its third-year of existence.

This group decided to set goals, in collaboration with the faculty and the students. We thought that a process-sheet narrative would be a good method of assessment. Alternatively, an assignment could be given to the students before they came to the library: "find something and bring examples." We could then use our class time to analyze what they found, instruct them, and re-do the search.

Testing could be done using the search tools. Controlled vocabulary and Boolean searching would be highlighted.

Scenario B: First-year instruction tours and workbook (either on-line or paper); asked by the Dean to evaluate the program.
It was decided to evaluate the program briefly. We would look at both quantitative data (raw counts, cost) and qualitative factors. We would examine the program to ensure that it matched the mission/goals of the institution. Pre- and post-testing would also be used (with a control group). We could involve the faculty, and ask them if they believed the tours were helpful.

We talked about following up with the students in four years (perhaps through portfolios). We would need to collaborate with faculty. Another method would be to ask the students to keep a journal as they researched. Depending on our results, we might choose to focus the sessions on research strategies versus which buttons to click for a specific database.

The session discussed the role of tours in library instruction. We also mentioned that time is a real factor. It is very hard to accomplish instruction and assessment in the limited time frame in which we usually work.

Back to Top

Library Services
Willis Bridegam, Amherst College
Dawn Thistle, Assumption College, Reporter

Dr. Bridegam began the breakout session on Library Services by recommending that participants take a look at the report done by the ACRL Task Force on Academic Library Outcomes Assessment at http://www.ala.org/acrl/outcome.html. As a member of the task force, he was able to comment on the difficulties of determining appropriate outcomes for academic libraries. For example, the task force agreed that the implementation of outcome assessment does not necessarily mean that the practice of measuring inputs and outputs is invalid. Taken together, a library can more fully assess its performance. Various ideas for outcome assessment were mentioned, including user logs, pre-, mid- and post-testing of library users, and student focus groups. Assessing Service Quality by Peter Hernon was recommended as a good text for assessment purposes.

In regard to the new Standards for College Libraries, in this month's issue of College & Research Libraries News, we discussed the necessity of identifying appropriate peer institutions with which to compare ourselves. Dr. Bridegam suggested negotiating the list with college administrators. Once the peer group is identified, a library can create ratios in different comparison areas that can then be analyzed. The question was raised of whether or not permission was needed before using data from other libraries, but to the extent that data is available from IPEDS this was not seen as necessary.

The group then explored questions that might be important to ask a library in regard to assessment. These questions included:
"Does the institution have a mission statement and does the library's mission relate to it?"
"Do the institution and its library have a strategic planning process?"
"What is the state of library staff morale?"
"Do library resources (collections, staff, space) reflect the mission of the institution?"
"How and to what extent is the library's web site used?"
"How much money is spent per student for print/electronic materials?"
"Do librarians participate in campus governance or on campus committees?"
"What is the library's organizational structure (i.e. does it make use of teams)?"

Another question, "Who makes judgments about library quality?" was asked. Several possibilities were mentioned, including librarians/faculty/administrators/students, campus library committee, and visiting committee.

Back to Top

Web site
David Carlson, Bridgewater State College
Sarah Nesbeitt, Bridgewater State College
Regina Raboin, Tufts University, Reporter

David and Sarah began the breakout session by asking what questions should we be asking about assessing library web sites and what measures could we be looking at for library web sites.

Typical concerns expressed about library web sites are:
ease of navigation/usability - minimal clicks
technically accessible
visually appealing/ graphically balanced
priorities are distinguished
intent ­ what does page convey
disability accessibility
instruction supports

The group then discussed the following questions: who are the users ­ students, faculty, staff, off-campus, etc.; searching effectiveness; timeliness; what is it that they are not finding that they should; and why are librarian's doing this? There was a focused discussion on what the users were looking for in and on a web site. What are we professionally bringing to our web sites that maybe should not be there? An important issue of marketing/promoting the library's web pages was also discussed. Who does the marketing, when and how?

Typically, the number of hits is used to determine use measurement statistics. Below are some of the additional ways to measure:
surveys/focus groups
amount of time spent searching the library web site
what are the most common navigation paths
counting sessions, not hits
bookmarks at public computer stations ­ what is being used and by whom
return paths
usability studies
marketing on the web page (have vendors help with evaluations; having questions of effectiveness)
electronic suggestion board ( perhaps popping up every 100th session/hits)

David and Sarah ended the session by discussing some creative web-based programs in use by academic libraries. Bridgewater State and University of Connecticut use a Microsoft Access database to list and search their full text electronic journals. North Carolina State provides My Library, a pull technology that allows students to "create" their own web-based library. VMI and Cornell University are making use of web agent software for subject searching. Simmons College, Graduate School of Management uses Web CT for web-based electronic reserves.

 Back to Top