Matthews, Joseph R. The Evaluation and Measurement of Library Services. Westport, CT: Libraries Unlimited, 2007. 372 pp.  50.00 USD.  ISBN-10: 1-59158-532-5.  ISBN-13: 978-1-59158-532-9. ∞

 In this very substantial treatment of the question of how to measure and evaluate library services, Joseph Matthews has chosen to place the emphasis very much on evaluation.  He laments the failure of many library administrators and directors to engage in meaningful evaluation of library services, their tendency to regard the gathering of statistical information as equivalent to evaluation, and their tendency to rely on the implicit "goodness" of libraries as justifications for the services they offer. 

The opening chapters of this book deal with evaluation issues and models as well as with the issues that arise from qualitative and quantitative forms of measurement and evaluation.   In the subsequent chapters Matthews applies a variety of evaluation techniques to topics such as library users, the library collection, electronic resources, reference services, technical services, interlibrary loan, online systems, library instruction and information literacy, and customer service.   The concluding chapters draw the reader from the specific to the more general: the economic and social impacts of libraries, communicating the value of library services to a wider audience, and methods to determine whether libraries provide life-long benefits to library users.

Most chapters begin with a "service definition," followed by a detailed discussion of the topic, a summary of the discussion, and very substantial footnotes and bibliographical information.   One of the values of this book is the very wide range, geographically and historically, of evaluation and measurement studies that Matthews has consulted and his brief reflections on these studies.  He notes areas where little research or evaluation has taken place and where more needs to be undertaken.

Interestingly, Matthews does not include a chapter on library space as a service, and he misses the opportunity for an in-depth discussion of issues such as the use of library space for cultural events (poetry readings, book launchings, displays of artwork or handicrafts, etc.) or for human conveniences such as refreshment services, places for group study, or access to wireless Internet connections.   There is a detailed discussion of electronic journals and e-books but curiously no discussion of other library electronic services such as Web site guides, pathfinders, or even the library's own Web site.

The book is impressive in its breadth of coverage of this topic, in particular its discussion of the variety of types of evaluation and measurement.  Whereas administrators of academic or public libraries will find much to benefit from in this book, administrators of special libraries may be disappointed at the limited discussion of the evaluation of the services they provide.   At times, in the summaries of the studies that have been undertaken on a topic, the reader is confronted with a seemingly contradictory series of statements about studies that prove a hypothesis, studies that disprove the same hypothesis, and studies that yielded no definite results at all.  If the majority of studies prove a hypothesis, it would be helpful to learn this, with the indication that some studies do not.   If the studies about a library service, when viewed as a group, do not yield consistent results, then it would be more helpful if Matthews were to state this, and then move on to other methods of evaluation that do yield convincing results.

Although Matthews is right in emphasizing the need for long-term evaluations (p. 243), he occasionally takes this too far: "When an individual uses a library the immediate benefits will occur within the lifetime of that person" (p. 287).  Few library administrators would wait for the conclusion of a person's lifetime to determine whether these immediate benefits were realized, and even fewer would wait for the person's afterlife or re-incarnation to see if the long-term benefits were realized. 

Some of Matthews' assumptions could be challenged.   Matthews restricts his discussion of the social impacts of libraries to public libraries, stating (p. 327) that "school and academic libraries have the potential for impacting the larger community within which they serve, but here also most of the use and therefore the benefits are going to accrue to those who use the academic library".   "Those who use the academic library" are, if one observes an academic library, a very wide cross section of society, not just the academic community and there is ample room for a discussion of the social impacts of these libraries, too.   Matthews' discussion of reference services focuses too much on studies that measure the accuracy of the information provided by reference staff.   Most of the reference questions, at least in academic libraries, do not deal with finding specific facts but with how to use the library's resources, how to select an appropriate index or database, or how to approach a research assignment.  There are also occasional jumps in logic: "… if students are tested immediately after receiving instruction, short-term gains are not likely to be sustained," (p. 233).  The testing of a skill does not cause the skill to be sustained for a short term or a long term; it only measures whether something has been learned or not.

These quibbles notwithstanding, Matthews provides a well-organized, well-documented discussion of the design and execution of studies to measure and evaluate library services as well as numerous examples of pitfalls and mistakes that should be avoided as one seeks to conduct these evaluations.

Linwood DeLong, Collections Coordinator, l.delong@uwinnipeg.ca, University of Winnipeg, Winnipeg, Manitoba