Electronic Serials Usage Patterns as Observed at a Medium-Size University: Searches and Full-Text Downloads

Alain Lamothe
Electronic Resources Librarian
J.N. Desmarais Library
Laurentian University

Abstract:

As the number of electronic serials available to libraries continues to increase while library budgets remain either stagnant or on the decrease, it becomes necessary to evaluate the use of a library's electronic collection. In 2006, usage statistics were evaluated at Laurentian University, Canada, to provide direction to collection development and identify high-cost low-use electronic serials. Searches and full-text downloads were studied. A sharp increase in use was observed in and around 2004 which can be explained by the introduction, in Ontario, of the 'double cohort', by the rapid increase in the number of electronic resources subscribed to at Laurentian, and by the adoption of OpenURL technology. Heavily used electronic serials are identified. Turnaways, connections by IP address and Bradford's 20:80 rule are also examined. The application of a cost-per-download ratio provided a practical method for identifying underused products.

Introduction

The overall purpose of this analysis was to study the usage of the electronic collection of the J.N. Desmarais Library at Laurentian University. It represents an examination of both current usage data and data from previous years, where available, in an attempt to establish general utilization patterns of the electronic collection, currently and in the past. Additionally, the study provided much needed data in the form of a cost per use analysis to be applied to collection development. Such an analysis had never before been undertaken at the library.

Laurentian University is a multi-campus university founded in 1960, with its main campus located in Sudbury, Ontario, Canada. Considered to be medium-size by the Carnegie classification of institution scale (Carnegie Foundation for the Advancement of Teaching), the total full-time student population numbered 8,400 in 2006, with over 500 enrolled in various graduate programs. Also in 2006, 370 full-time faculty members taught and performed research. Programs cover multiple fields in the sciences, social sciences and humanities with 94 undergraduate, 18 Master's and 6 doctoral degrees offered.

As with most libraries, public or academic, the J.N. Desmarais Library, Laurentian University's main library, has not been spared the effects of the explosion in the number of electronic resources available for purchase or subscription (Bevis and Graham 115-119; Creech 30-34; Kocevar-Weidinger, Kinman and McClasin 29-34; Ridi 273-344; Wakimoto 21-33). As more and more research tools became available in electronic format; it was necessary to rely to a greater extent on electronic products rather than their more cumbersome print equivalents, not only because patrons demanded it (Albanese 1-12; Brennan et al. 515-526), especially those working off-campus (Moyo 185-209), but also because physical space in the library itself was at a premium.

Between 2002 and 2007, the number of electronic journals available to the Laurentian University community jumped from 8,595 in 2002 to 28,468, a 230% increase. Current expenditures at the J.N. Desmarais Library for its electronic serials amount to $1.2 million (CDN$) accounting for about 81% of the library's serials expenditures. In contrast the average academic library tends to spend 70% of its serials budget on electronic products (Luther 119-147).

Thus, the J.N. Desmarais Library found itself with electronic journals subscribed to by individual title or in bundles or packages. They could be accessed directly from a publisher's site or from an aggregator such as BioOne, Ebsco, JSTOR, Proquest, Gale or from both, which typically leads to duplication of titles. Aggregators, such as BioOne, offer e-journals as stand-alone subject-oriented packages; others, such as Ebsco's Academic Search Premier, as searchable databases. Date and volumes available varied from one publisher to the next with some offering only the past year or so and others offering their entire collection from the first volume onward. Furthermore, some publishers impose embargoes making the most current issues or volumes unavailable to subscribers. Embargoes have ranged from 1 day to several years.

How can an academic library ensure that it is getting the best value for its money amidst the vast number of types of electronic resources now available for their patrons to use?

Without central control, managing the electronic collection effectively is almost impossible.  One of the first steps many libraries have taken was to hire an electronic resource librarian dedicated to the management and organization of the library's electronic collection (Albitz 589-600; Felt 75-112; Ginanni, Davis and Arthur 173-177; Wiles-Young, Landesman, and Terrill 253-258). Additionally, electronic resource librarians can coordinate the gathering and analysis of usage statistics. Before deciding on the addition or retention of a particular electronic product, it is important to determine if usage warrants the budget allocation. In the case of a new product, usage information can be made available from the vendor during a trial period. These are just the steps which were taken at the J.N. Desmarais Library.

In 2004, the J.N. Desmarais Library added an electronic resource librarian to its team, and in 2006, this led to the first comprehensive statistical analysis of usage of electronic resources available at Laurentian University. Its primary goal was to investigate usage patterns in order to provide clear directions for collection development, and more specifically, to identify high-cost, low-use electronic serial services as possible candidates for cancellation since, as with many institutional libraries, the question of limited funding is an issue (Balas 36-38; Blumenstein and Rogers 18; Curzon 34-35; Duhon 27-233).

To be sure, low usage may simply be a result of patrons not knowing about the availability of a particular e-resource (Hahn and Faulker 215-227; Townley and Murray 32-39; Wisniewki and Fichter 54-57). More likely however, a scarcely-used e-journal or database is not a necessary or valuable addition to the library's collection. Services that retain low usage after a period of 3 years, especially following an aggressive marketing campaign, may not be of interest to the University community (Luther 119-147; Townley and Murray 32-39). But, without numbers, it is very difficult to justify ongoing expenditures.

Gathering the Data

Since this was the first extensive effort in the gathering and interpretation of usage statistics, a great deal of time was needed to extract the data, and more importantly, to establish contact with the various vendors in question in order to request access to the appropriate statistics. For many electronic products available at the J.N. Desmarais Library, such data had previously been inaccessible, as administrative access to these accounts had not been automatically provided with subscriptions. It required a total of 350 hours to compile comprehensive statistics. When the exercise was completed, an estimated 1,500 pages of data had been collected. Recently, gathering statistical information from vendors has become easier, but overall, is still a time consuming project (Blake and Schleper 460-464). Those libraries which have yet to embark on this quest for usage information need to be aware of the amount of time compiling such information often requires.

For the purpose of this study, an electronic serial was defined as a subject database (e.g. BIOSIS Previews, Sociological Abstracts), a database with full-text content (e.g. ABI/Inform, Academic Search Premier), a full-text e-journal suite (e.g. BioOne, ScienceDirect), or an individually subscribed e-journal (i.e. Nature Methods). In total, 88 electronic serials were examined. Numbers of searches performed in subject databases and the number of full-text articles downloaded were tracked. The number of searches, rather than sessions, is presented as it has been suggested as a more accurate method of measuring usage in that it represents a deliberate action on the user's part (Blecic, Fiscella and Wiberley 26-44). A session may be a connection accidentally initiated by the user clicking on the wrong link.

Data included figures for 2006 along with, in some cases, figures from previous years. Statistical information regarding searches performed in databases was available to this author as far back as 2003. On the other hand, statistics regarding the number of articles downloaded were available as far back as 2000. Not all service providers made historical statistics available to subscribers, although 74 out of 88 electronic serials packages examined (84%) offered such statistics. It also appeared that vendors offering access to e-journal suites (e.g. Highwire Press), as opposed to those vendors with databases containing full-text (e.g. Academic Search Premier), have made available and have retained on their servers data from previous years.

It was disappointing to see that statistics continue to vary greatly between vendors. In some cases, only basic numbers, such as sessions and searches, were provided. In other cases, much more comprehensive and complete statistics were available for analysis. Out of 88 products, 59 were COUNTER-compliant1. This represents a total of 67%. Granted, this situation is far better than that of even five years ago but it still represents approximately only two thirds of all electronic services subscribed to at Laurentian University (Cole 97-102; Conyers 37-44; Luther 2007; Luther 119-147). In addition, COUNTER-compliant reports have been demonstrated to have variation in content and format, which can only mean that more work remains to be done to improve and standardize the usage statistics situation (Blecic, Fiscella and Wiberley 26-44; Bordeaux and Kraemer 295-299).

A cost-per-use analysis was also employed during this study. This cost-per-use concept has been applied for some time now when analyzing both print and electronic collections (Blake and Schleper 460-464; Franklin 241-248; Hahn and Faulker 215-227; Holstrom; Milne and Tiffany 7-20; Scigliano 43-52; Townley and Murray 32-39; Ward 4-16). In the case of this particular investigation, it was strictly applied to the download of full-text articles. Attributing a cost of a particular search is far more difficult to quantify.

The cost-per-download ratio was calculated to provide a dollar amount attributed to the value of a particular article, by taking the annual cost of a product divided by the total number of articles downloaded (cost ÷ downloads) from that same product. In this manner, it became easier to judge the value of a particular full-text product offered by the library. The library could then assess, for example, whether it would be less expensive to use document delivery (e.g. interlibrary loan) rather than subscribing to a particular e-journal suite.

General Usage Patterns

On the whole, it is clear that electronic serials at Laurentian University have seen exponential growth in use over the past few years (Figure 1). The sharp upward trend beginning between 2003 and 2004 has yet to show any sign of leveling off. This sudden growth has been observed in both the number of searches performed and the number of full-text articles downloaded as Figure 1 clearly illustrates.

image1
Figure 1: General Usage Trends in the Number of Searches
Performed and the Number of Full-Text Articles Downloaded.

Table 1 provides figures associated with searches. Since 2003, there have been over 2.9 million searches. Of these, 1.58 million were performed in 2006 alone, or 55% of the total. When comparing values for 2003 and 2004, the total number of searches from one year to the next increased by an amazing 3,000%. Subsequent to this upsurge in use, the total number of searches increased from year to year by an average of about 200%. If we compare the values for 2003 to those of 2006, a 16,000% increase is observed.

Table 1: Total Searches Performed on Laurentian University Databases.

Year

Searches

% Change From Previous Year

% Change from Initial Year

2003

9838

N/A

N/A

2004

305181

3002%

3002%

2005

1006302

330%

10128%

2006

1582087

58%

15981%

A similar trend is noted with respect to the retrieval of full-text articles. Table 2 presents figures for article downloads. In this case, statistics were available as far back as the turn of the century. Since 2000, over 1.4 million articles have been downloaded by members of the Laurentian University community, with nearly 640,000 occurring in 2006, or 46% of the total. In contrast to search results, year to year growth in the number of downloaded articles was not as pronounced. The highest increase occurred between 2001 and 2002, at 842%. Between 2003 and 2004, the increase in the number of downloads was substantial, jumping from 57,000 in 2003 to 250,000 in 2004. However, the years 2002 through 2004 show the highest growth, with annual increases ranging between 336% and 842%.

Table 2: Total Full-Text Articles Downloaded from Laurentian University's
Electronic Subscriptions.

Year

Downloads

% Change From Previous Year

% Change from Initial Year

2000

806

N/A

N/A

2001

1015

26%

26%

2002

9565

842%

1087%

2003

57072

500%

6981%

2004

248828

336%

30772%

2005

494618

100%

61267%

2006

636182

30%

78831%

It is evident from the values presented in Tables 1 and 2 that the usage of electronic collections at Laurentian University during 2006 was intensive. Nearly half of the total reported usage occurred in that one-year period. It is also clear that the University observed a sudden spike in usage between 2003 and 2004. There are three possible explanations for such rapid growth.

A possible reason can be attributed to the fact that, in 2003, universities in the province of Ontario, Canada, saw the introduction of the "double cohort", which led to an abrupt increase in postsecondary enrollment (Association of Colleges of Applied Arts and Technology of Ontario; Council of Ontario Universities). Prior to 2003, the Ontario secondary school curriculum consisted of 5 years (Grades 9 through 13). Secondary school reforms enacted by the Government of Ontario resulted in the elimination of Grade 13 from the curriculum, reducing the number of years high-school students spent in the curriculum from 5 to 4 (Grades 9 through 12). This change took affect in 2003. As a result, students from both the new Grade 12 curriculum and those of the old Grade 13 graduated from high school in the same year essentially doubling the overall number of graduating students from high school and, consequently, the number of first year students entering their post-secondary studies.

At Laurentian University, this meant an additional 1,300 students, or 20.8%, who were not only beginning their studies in 2003, but who would also continue to be present for the duration of their program of choice, typically 4 years (bachelor's honor). Many of these would also undoubtedly continue on to graduate programs. The first of these "double cohort" students graduated in the spring of 2007, with more scheduled to receive degrees after fall convocation. If this "double cohort" phenomenon proves to be the critical factor in the sudden increase in usage of Laurentian's electronic collection, it is to be expected that during the coming years, there would be a leveling off of the curve observed in Figure 1. To confirm this, it will be necessary to wait and examine the 2007 and 2008 data.

A second important factor was the increase, over the same period, in the number of electronic resources available to students and faculty. As stated previously, 8,592 electronic journals were available between 2002 and 2003. In 2004 that number jumped to 16,811, a 95.6% increase. There was also an increase in the number of online subject databases. A total of 19 new databases were added to the collection in 2004, and in 2005, another 33. Most of these databases were purchased to replace existing print subscriptions. This abrupt increase in the availability of e-resources would certainly have had an impact on use, especially after their print equivalent was cancelled.

A third possible factor was the introduction of OpenURL in 2004. At Laurentian University, as well as other Ontario academic libraries, Ex Libris' SFX is employed. With OpenURL technology, patrons no longer need to take notes while searching in a database, and use the periodical collection to locate articles of interest. They simply click on the SFX button (labeled Get it @ Laurentian; Figure 2), and they are immediately connected to the full-text of the article in question, provided that the library subscribes to the contents. This simplification of the process of getting access to full-text content obviously had a positive impact on e-collection use (Chang 2; MacDonald 39-50; Yi and Herliky 317-331), a conclusion also reached by Eason, MacIntyre and Apps in their 2005 study which reported a rapid increase in usage after the implementation of OpenURL technology. Reason: The more steps to follow in order to gain access to an article, the greater the deterrence (Apps and MacIntyre). Mooers's Law states that "an information retrieval system will tend not to be used whenever it is more painful and troublesome for a customer to have information than for him to not have it" (Mooers 22-23). This also corresponds with Zipf's principle of least effort (Kim 3-7) and Ranganathan's fourth law of library science: "save the time of the reader" (Gopinath; Rimland 24-26).

image2
Figure 2: Get It @ Laurentian - the SFX button allowing patrons to connect
to the full-text of e-journals.

The term "heavy weights" has been associated here with those electronic serials which have seen the heaviest usage. Table 3 presents the top three searchable subject databases while Table 4 displays the top three products from which full-text articles are available. In each case, the rank, product name, vendor, and counts are indicated by year.

Table 3: Heavy weights: Total Number of Searches Performed by Year.

Rank

Product Name

Vendor

 

Year

 

 

 

 

2004

2005

2006

1

ABI/Inform

Proquest

108330

204484

238666

2

Canadian Business and Current Affairs

Proquest

51855

147604

205792

3

Academic Search Premier

Ebsco

N/A

178698

191135


Table 4: Heavy weights: Total Number of Full-Text Articles Downloaded by
Year.

Rank

Product Name

Vendor

 

Year

 

 

 

 

2004

2005

2006

1

Academic Search Premier

Ebsco

N/A

160296

141571

2

ScienceDirect

Elsevier

34895

57802

66921

3

Proquest Nursing and Allied Health Source

Proquest

53806

50158

61065

In 2006, the greatest number of searches was performed in Proquest's ABI/Inform, with 239,000 searches. The second and third most popular databases were Proquest's Canadian Business and Current Affairs (206,000 searches) and Ebsco's Academic Search Premier (191,000 searches), respectively. These top databases have been described as rather general in nature as they index information from a multitude of subject areas (Ebsco Publishing; Proquest-CSA).

The "heavy weights" for full-text downloads are as follows: Academic Search Premier (142,000 downloads), ScienceDirect (67,000 downloads) and Proquest Nursing and Allied Health Source (61,000 downloads). Again, the top two products contain journals covering a wide variety of topics. Only Proquest Nursing and Allied Health Source offers information on a single subject.

In fairness, it should be also noted that Academic Search Premier is not a full-text suite in the strictest sense. While it contains much full-text material, patrons can also search it for research on a particular topic and retrieve relevant articles through the use of SFX instead of going directly to a particular suite and searching the contents of only those journals provided within the suite. Ralston, in a 2007 study, found similar results where electronic journals were preferentially accessed through an aggregator (Ralston 51-64).

Table 5 contains a list of values comparing usage seen in full-text journal suites at Laurentian University to Bradford's 20:80 rule. Bradford's Distribution - also, at times, referred to as the Pareto Principle (Crawford 15-19) - implies that typically the top 20% of library resources available for use will account for 80% of total usage (Black 20-24; Trueswell 458-461). As applied to this study, this rule would imply that 80% of all downloaded articles originated from only 20% of the journals contained in a particular suite.

Table 5: Cost per Download with Associated Value Regarding the 20:80
Rule.

Product Name

Cost / download

20:80 Rule

ACS Online Journals

$2.56

20:72

ABI/Inform

$0.35

20:97

Academic Search Premier

$0.19

20:89

AIP Online Journals

$9.00

20:73

APA Online Journals

$0.70

20:62

APS Journals

$3.87

20:54

ASBMB Journals

$1.29

20:95

Biblio Branché

$1.33

N/A

BioOne

$1.64

20:78

Blackwell

$1.65

20:79

Cambridge University Press

$7.07

N/A

Canadian Newsstand

$0.52

20:97

Canadian Periodical Index

$0.82

20:83

CBCA

$0.80

20:97

Conmunication and Mass Media

$2.25

20:89

Dissertation and Theses: Full-Text

$3.08

N/A

Duke University Press

$14.58

20:57

GeoScienceWorld

$10.75

20:72

Institute of Physics

$5.22

20:83

JSTOR

$0.60

20:64

Kluwer Press

$1.92

N/A

Literature Online

$1.32

20:92

Mental Measurements Yearbook

$12.00

N/A

Nature

$8.27

N/A

Nature Materials

$114.00

N/A

Nature Methods

$280.00

N/A

Ovid Nursing Collection - Ejournals

$0.54

20:82

Oxford University Press Journals

$1.64

20:67

Project MUSE

$1.00

20:66

Proquest Nursing and Allied Health Source

$0.10

20:97

Royal Society of Chemistry Journals

$5.94

20:67

Sage Publications

$1.05

20:63

ScienceDirect

$1.88

20:82

SIAM Journals

$554.54

20:63

Springer-Verlag Journals

$3.43

20:91

Wiley Interscience

$1.98

20:93

As can be observed from the table, this rule does not seem to be an absolute. Usage ranged from 20:54 at the lowest to 20:97 at the highest. In other words, some journal suites saw 54% of the total downloads originating from the 20% most-used titles while other collections experienced almost all downloads from the top 20%. Only 6 out of 29 suites were even close to the 20:80 ratio (± 3 points). Variations in this ratio have previously been reported (Chrzastowski and Oleski 101-111).

The mean was calculated as being 20:79, which is very close to that stated by the rule. The median, or middle number, was 20:80 - which would seem to fit the rule – but the mode, or most often occurring number, was 20:97. This implies a negatively skewed distribution. Davis, in a 2002 publication, has reported this high usage rate (96%) among the top 20 journals available at medical institutions.

Cost per Download

The cost-per-download ratio is a very useful measure. Not only does it provide a concrete dollar value for a particular electronic article downloaded, but it also allows for a comparison in value between different delivery services. In this case, a comparison was made between the cost of a particular online full-text serial, and the cost of having these articles delivered via inter-library loan.

At the J.N. Desmarais Library, it has been observed that the cost of ordering an article by ILL can vary between $0.00 and $50.00 CDN, depending on the lending library. Having an item delivered from a nearby library will obviously be less expensive than from a library overseas. On average, the cost of ordering and having an article delivered by inter-library loan is about $15.00 CDN.

Table 5 presents a list of 36 full-text individual titles and journal suites included in this study. The cost-per-download is provided. It is clear that the vast majority of these have a ratio far below the $15.00 mark.

One e-journal suite and two individual e-journals (Nature Materials, Nature Methods, and SIAM Journals) have a cost-per-download far above $15.00 ($114.00, $280.00, $554.54, respectively). These are exceptionally high. In the entire year, only 18 articles were downloaded from Nature Materials, 16 from Nature Methods and 11 from SIAM's mathematical journals. Given that Laurentian has had subscriptions to these online journals for several years and that usage has been consistently low, it would be far more cost effective to cancel these subscriptions in favor of the more economical alternative of document delivery via inter-library loan. Three other electronic serials (Duke University Press, GeoScienceWorld and Mental Measurements Yearbook) lie very near this $15.00 mark ($14.58, $10.75, $12.00, respectively). Continued monitoring of their use is required, and a decision whether to retain or cancel can be made at the appropriate time.

However, services like Academic Search Premier and Proquest Nursing and Allied Health Source have a cost-per-download measured in cents, rather than dollars. These are all services which have been heavily used and warrant the retention of the online subscriptions.

In using a cost-per-use approach, there are some assumptions involved. First, it is assumed that the library would not have in its collection the print equivalent of a particular journal or journal suite. In other words, to access a particular article, the patron would have no other alternative than a connection to the full-text journal suite. Furthermore, it is also a given that all of the articles downloaded from a journal or a suite would have been ordered through inter-library loan, if that suite had not been part of the library's collection. It is unlikely that every single download from this suite would have been ordered, as many patrons will request a download simply because it is available and not necessarily pertinent to their work. However, the cost-per-download ratio still remains an easily calculated tool that can be quickly applied to act as an aid in the decision-making process.

Some Secondary Observations

During the course of this study a couple of other trends emerged, one dealing with IP addresses and the other with Turnaways.

Access by IP address was studied in the hope of determining which segment of the University had been utilizing the electronic collection the most. While it is true that an IP address cannot identify a particular user it can, however, quickly indicate the location of that user's computer. Unfortunately, usage statistics by IP were available from only 9 out of 88 services (10.2%). This is a bit low, as McDowell and Gorman reported in 2004 that 33% of vendors supply usage by IP address.

Figure 3 illustrates the comparison between connections initiated on-campus and off-campus. Connections via the proxy IP were very high, at 45% of the total. It would seem then that a large portion of the library's patrons are working from home or perhaps while traveling. Similar findings have been reported from Cornell University (Davis and Solla 1062-1068). The decision to access library resources from off-campus can be influenced by a couple of factors. In addition to convenience for those who would normally be on-campus (Brennan et al. 515-526; Landolt 554-555; Moyo 185-209), others may be forced to work from off-campus if they are taking distance education courses, as has been observed at many institutions of higher learning (Moyo 185-209).

image3

Figure 3: Pie Chart Representing the Percentage of Sessions Accessed
from On-campus and Off-campus.

The issue of turnaways is not a big one at Laurentian since so many of its databases allow for unlimited access. In fact the issue only involved Global Books in Print, a collection development tool mainly used by librarians and library technicians. Initially, Laurentian University had a single-user license and in 2005 turnaways numbered 480. It was therefore decided that the subscription would be upgraded to a two-user license despite the extra cost. This proved to have dramatic results the following year when the number of turnaways dropped to 47. This decrease more than justified the extra spending and illustrates the importance of keeping track of turnaway figures.

Conclusions

The purpose of this usage analysis was achieved. Until now the extent to which electronic serials were being used at Laurentian University was not clear. As demonstrated, the electronic collection has seen remarkable usage. This, in itself, comes as no surprise given the current predilection for online information.

One of the main points observed during this study was the sudden increase in usage seen in the Desmarais Library's electronic collection in and around 2004, with a 3000% increase in searches from the previous year. The highest rate of usage occurred in 2006, when nearly 50% of the total accumulated usage from 2000 to 2006 took place. Three potentially influential factors have been identified: the "double cohort" which led to an abrupt increase in postsecondary enrollment in Ontario; a 70% increase in the number of electronic serials at Laurentian over the past five years; and the adoption of OpenURL technology, in 2004, facilitating patron access to electronic resources.

Further investigation into these factors is warranted. One question remains: of these three possible factors which has contributed the most to the increase, or  have all three participated synergistically? The continued analysis of data over the next few years will be necessary as it is expected that the student population at Laurentian University will return to pre-double cohort levels. It would also be interesting to see if this same pattern occurred at other affected Ontario universities, or at any other university that has experienced a sudden increase in its student population.

A cost-per-download ratio was demonstrated to be a practical method for identifying underused electronic products; in our case Nature Materials, Nature Methods and SIAM. It would be far more economical to depend on the library's inter-library loan service rather than continue such subscriptions. For this reason, it was decided that these subscriptions should not be renewed, unless, of course, a dramatic increase in usage occurs before the renewal date is reached.

Access to library databases and full-text journal suites was nearly equal between on-campus and off-campus users. Additionally, turnaways should be monitored as they were useful in justifying the addition of an extra seat to Laurentian's subscription to Global Books in Print.

Works Cited

Albanese, A. R. "The Reference Evolution." Library Journal 129.19 (2004): 10-12.

Albitz, R. S. "Electronic Resource Librarians in Academic Libraries: A Position Announcement Analysis, 1996-2001". portal: Libraries and the Academy 2.4 (2002): 589-600.

Association of Colleges of Applied Arts and Technology of Ontario. "Investing in Ontario's Economic Development. Opportunities and Issues for Increasing Capacity in Colleges." North York, ON. 1999.

Apps, A. and R. MacIntyre. "Why OpenURL?" D-Lib Magazine 125 (2006). 26 July 2007 <http://www.dlib.org/dlib/may06/apps/05apps.html>.

Balas, J.L. "Facing Budget Cuts: Must We Rob Peter to Pay Paul?" Computers in Libraries, 26.6 (2006): 36-38.

Bevis, M.D. and J.B. Graham. "The Evolution of an Integrated Electronic Journals Collection." Journal of Academic Librarianship 29.2 (2003): 115-119.

Black, S. "Bradford's Distribution, the 80/20 Rule, and Pattern of Full-Text Database Use." Against the Grain 15.6 (2003/2004): 20-24.

Blake, J.C. and S.P. Schleper. "From Data To Decisions: Using Surveys and Statistics To Make Collection Management Decisions." Library Collections, Acquisitions, and Technical Services 28 (2004): 460-464.

Blecic, D.D., J.B. Fiscella, and S.E. Wiberley, Jr. "Measurement of Use of Electronic Resources: Advances in Use Statistics and Innovations in Resource Functionality." College and Research Libraries 68.1 (2007): 26-44.

Blumenstein, L. and M. Rogers. "Budget Roundup: Cuts hit Home." Library Journal 128.10 (2003): 18.

Bordeaux, A. and A.B. Kraemer. "Making the Most of Your Usage Statistics."Serials Librarian 48.3/4 (2005): 295-299.

Brennan, M.J., J.M. Hurd, D.D. Blecic and A.C. Weller. "A Snapshot of Early
Adopters of E-Journals: Challenges to the Library." College and Research Libraries 63.6 (2002): 515-526.

Carnegie Foundation for the Advancement of Teaching. "The Carnegie Classification of Institutions of Higher Learning." (2007). 3 July 2007 <http://www.carnegiefoundation.org/classfications/index.asp>.

Chang, S.H. "Full-Text Article Linking: Where Are We Now?" Chinese Librarianship 23 (2007): 2.

Chrzastowski, T.E.  and B.M. Oleski "Chemistry Journal Use and Cost: Results of A Longitudinal Study." Library Resource and Technical Services 41.2 (1997): 101-111.

Cole, L. "Usage Data – The Academic Library Perspective." Serials 13.2 (2000): 97-102.

Conyers, A. "Building On Sand? Using Statistical Measures to Assess the Impact of Electronic Services." Performance Measurement and Metrics 7.1 (2006): 37-44.

Council of Ontario Universities. "Access to Excellence: the Double Cohort Countdown – A Progress Report from Ontario Universities." Toronto, ON. 2002.

Crawford, W. "Exceptional Institutions: Libraries and the Pareto Principle." American Libraries 32. 6 (2001): 72-74.

Creech, A.L. "Managing Digital Resources, or, How Do You Hold Electrons in Your Hand," (2006). Against the Grain 18.2 (2006): 30-34.

Curzon, S.C. "Budget Shortfalls." Library Journal 128.9 (2003): 34-35.

Davis, P.M. "Patterns in Electronic Journal Usage: Challenging the Composition of Geographic Consortia." College and Research Libraries 63.6  (2002): 484-497.

Davis, P.M. and L.R. Solla. "An IP-Level Analysis of Usage Statistics for Electronic Journals in Chemistry: Making Inferences About User Behavior." Journal of the American Society for Information Science and Technology 54.11 (2003): 1062-1068.

Duhon, L., J. Langedorfer and S. Srivastava. "Binding Journals in Tight Times: Mind the Budget." Serials Librarian 50.3/4 (2006): 227-233.

Eason, K., R. MacIntyre and A. Apps. "A 'Joined-Up' Electronic Journal Service: User Attitudes and Behaviour." (2005). 26 July 2007 <http://epub.mimas.ac.uk/papers/ lww6/easonetal-lww6.pdf>.

Ebsco Publishing. "Academic Search Premier." (2007). 26 July 2007 <http://www.ebscohost.com/thisTopic.php?marketID=1&topicID=1>.

Felt, E. C. "Holland Library's Electronic Resource Librarians: A Profile of These Positions." Reference Librarian 64 (1999): 75-112.

Franklin, B. "Managing the Electronic Collection with Cost-per-use  Data." IFLA Journal 31.3 (2005): 241-248.

Ginanni, K., S. Davis, and M. A. Arthur. "Talk About: E-Resources Librarian to the Rescue? Creating the Uber Librarian: Turning Model Job Descriptions into Practical Positions." Serials Librarian 50.1/2 (2006): 173-177.

Gopinath, M.A. "Ranganathan, Shiyali Ramamrita," in Encyclopedia of Library and Information Science. Ed. M.A. Drake. New York: Marcel Dekker, Inc. 2003.

Hahn, K.L.  and L.A. Faulkner. "Evaluative Usage-based Metrics for the Selection of E-journals." College and Research Libraries 63.3 (2002): 215-227.

Holmstrom, J. "The Return on Investment of Electronic Journals - It Is a Matter of Time." D-Lib Magazine 10.4 (2004). 19 July 2007 <http://www.dlib.org/dlib/april04/ holmstrom/04holmstrom.html>.

Kim, C. "Retrieval Language of Social Sciences and Natural Sciences: A Statistical Investigation." Journal of the American Society for Information Science 33.1 (1982): 3-7.

Kocevar-Weidinger, E., V. Kinman, and S. McClasin. "Case Study: The Inch and the Mile," Library Administration and Management, 21.1 (2007): 29-34.

Landolt, R.G. "Examining the Effects of Introducing Online Access to ACS Journals at Primarly Undergraduate Institutions." Journal of Chemical Education 84.3 (2007): 554-555.

Luther, J. "White Paper on Electronic Journal Usage Statistics." Serials Librarian 41.2 (2001): 119-147.

Luther, J. "White Paper on Online Journal Usage Statistics." (2000). 17 July 2007 <http://www.clir.org/pubs/reports/pub94/contents.html>.

MacDonald, J.D. "Understanding Journal Usage: A Statistical Analysis of Citation
and Use." Journal of the American Society for Information Science and Technology 58.1 (2007): 39-50.

McDowell,N. and G.E. Gorman. "The Relevance of Vendors' Usage Statistics in Academic Library E-Resource Management: A New Zealand Study." Australian Academic and Research Libraries. 35.4 (2004): 322-343.

Milne, D. and B Tiffany. "A Cost-Per-Use For Evaluating the Cost-Effectiveness of Serials: A detailed Discussion of Methodology." Serials Review 17.2 (1991): 7-20.

Mooers, C.N. "Mooers's Law: Or, Why Some Retrieval Systems Are Used and Others Are Not." Bulletin of the American Society for Information Science 23.1 (1996): 22-23.

Moyo, L.M. "The Virtual Patron." Science and Technology Libraries 25.1/2 (2004): 185-209.

Proquest-CSA. "ABI/INFORM Global." (2007). 26 July 2007 <http://www.bellhowell.infolearning.com/products_pq/descriptions/abi_inform_global.shtml>.

Ralston, R. "Assessing Online Use: Are Statistics from Web-based Online Journal Lists Representative?" Journal of Electronic Resources in Medical Libraries 4.1/2 (2007): 51-64.

Reed, W.J. "The Pareto, Zipf and Other Power Laws." Economic Letters 74 (2001): 15-19.

Ridi, R. "Digital Library: Definitions, Ingredients and Problems." Bolletino AIB 44.3 (2004): 273-344.

Rimland, E. "Ranganathan's Relevant Rules." Reference and User Services Quarterly 46.4 (2007): 24-26.

Scigliano, M. "Serials Use In a Small Academic Library: Determining Cost-Effectiveness." Serials Review 26.1 (2000): 43-52.

Townley, C.T. and L. Murray, "Use-Based Criteria for Selecting and Retaining Electronic Information: A Case Study." Information Technology and Libraries 18.1, (1999): 32-39.

Trueswell, R.L. "Some Behavioral Patterns of Library Users: The 80/20 Rule." Wilson Library Bulletin 43 (1969): 458-461.

Wakimoto, J.C. "Electronic Resources: Approaches in Providing Access." Journal of Internet Cataloging 6.2 (2003): 21-33.

Ward, R.K., J.O Christensen and E. Spackman. "A Systematic Approach for Evaluation and Upgrading Academic Science Journal Collections." Serials Review 32.1 (2006): 4-16.

Wiles-Young, S., B. Landesman, and L. J. Terrill. "E-Resource = E-Opportunity: Connecting Systems, Technical Services and Patrons." Serials Librarian 52.3/4 (2007): 253-258.

Wisniewski, J. and D. Fichter. "Electronic Resources Won't Sell Themselves: Marketing Tips." Online 31.1, (2007): 54-57.

Yi, H. and C.S. Herliky. "Assessment of the Impact of An Open-URL Link Resolver." New Library World 108.7/8 (2007): 317-331.


[1] Taken from the Project COUNTER web site: "Launched in March 2002, COUNTER (Counting Online Usage of Networked Electronic Resources) is an international initiative serving librarians, publishers and intermediaries by setting standards that facilitate the recording and reporting of online usage statistics in a consistent, credible and compatible way." See: http://www.projectcounter.org/about.html.