Provocation to Learn - A Study in the Use of Personal Response Systems in Information Literacy Instruction

Maura Alicia Matesic
Division of Social Science Librarian
Scott Library, York University

Jean M Adams
Assistant Professor
Schulich School of Business, York University

Abstract

The appearance of Personal Response Systems (PRS) or "clickers" in university classrooms has opened an avenue for new forms of communication between instructors and students in large-enrolment classes. Because it allows instructors to pose questions and receive tabulated responses from students in real-time, proponents of this technology herald it as an innovative means for encouraging higher levels of participation, fostering student engagement, and streamlining the assessment process. Having already been experimentally deployed across disciplines ranging from business to the arts and sciences, it is also beginning to be used in the context of information literacy instruction. In this project we employed the technology not to transfer actual skills, but to advertise the existence of online library guides, promote the use of the library within the context of the course itself, and "provoke" students to adopt a more active approach to research as a recursive process. Our findings suggest that students adapt easily to the use of this technology and feel democratically empowered to respond to their instructors in a variety of ways that include anonymous clicker responses as well as more traditional means such as the raising of hands and posing questions verbally. The particular value of this study was to show that these broader findings seem equally applicable to pedagogical settings in which learning objectives are built around and integrated with the principles of information literacy.

Introduction

The appearance of Personal Response Systems (PRS) or "clickers" in university classrooms has opened an avenue for new forms of communication between instructors and students in large-enrolment classes. Since it allows instructors to pose questions and receive tabulated responses from students in real-time, proponents of this technology herald it as an innovative means of encouraging higher levels of participation, fostering student engagement, and streamlining the assessment process. Having already been experimentally deployed across disciplines ranging from business to the arts and sciences, it is also beginning to be used in the context of information literacy instruction. Like other instructors, librarians have already begun to use the technology to achieve their own teaching objectives and to efficiently assess student learning.

For this case study, our goal was to move away from traditional information literacy instruction; instead, we employed the technology not to transfer actual skills, but to advertise the existence of online library guides, promote the use of the library within the context of the course itself, and "provoke" students to adopt a more active approach to research as a recursive process. Using the PRS technology we were able to poll students about their perceptions of libraries and librarians as well as their varying research styles and actual research practices, in order to more directly "market" library resources to them as an idiosyncratic group of learners. Our sessions were quick, focused, and aimed at piquing the students' interest in library research. Throughout, we attempted to promote active learning by encouraging students to think critically about the various ways in which library research could help them to achieve broader course objectives.  Early results indicate that students respond positively to the use of this technology in the classroom, and that the combination of technology and teaching style (provocation) encouraged greater student participation in the classroom and an ensuing increase in student activity in the library and career centre.

This conference paper will present the results of our project and suggest avenues for future study.

Personal Response Systems and Information Literacy Goals - An overview

Personal Response Systems present a variety of pedagogical opportunities and teaching benefits for Information Literacy Instruction. Many librarians have found themselves daunted by the prospect of having to capture the attention of large numbers of students gathered in lecture halls about a topic as arcane as library research.1 An unpleasant exercise in endurance often ensues when librarians adopt the same approaches used for teaching tutorial-sized library classes, such as inviting students to raise their hands to answer questions.  PRS may offer some help to these beleaguered professionals. By affording a previously unimagined degree of interactivity in the large-enrollment classroom through immediate, two-way communication, PRS technology promises to allow librarians and instructors to transcend many of the pedagogical limitations inherent in large-enrollment classes.

Increased student participation is a benefit of PRS technology often reported in the current literature on this topic.  In their article "Waking the Dead:  Using interactive technology to engage passive listeners in the classroom," Guthrie and Carlin argue that with the use of PRS, student participation approaches 100% in class sessions, "due in part to anonymity, ease of use, and the ability to see how many others answered in the same way" (1). Though lecture halls have long been lamented as a necessary evil in the modern academy, it would be difficult to argue that such a dramatic increase in classroom participation would not result in at least modest improvements in the level of student engagement and help to foster a more active approach to learning. In the context of Information Literacy Instruction, the PRS technology allows librarians to measure student engagement in the research process, present new ideas and learning objectives at point-of-need, and permit students to process the information by selecting a research path which best fits their own understanding and learning outcomes.

We designed our project to facilitate increased student engagement through the development of student driven learning and self-direction in course work and library research.  To this end, we asked yes/no and multiple choice questions which encouraged the students to reflect upon their own learning practices and goals.  Placing our work in a larger pedagogical framework we wanted to move away from "traditional" teaching modes, promote student engagement and prompt student initiated learning outcomes. In her article "Towards a User Centered Information Service," Ruth Morris outlines the differences between traditional and constructivist paradigms of learning.  Morris describes the more traditional approach to pedagogy, based on the work of Shannon and Weaver,2 which is predicated on the notion of information as object in which learning occurs through the transfer of reified knowledge directly from instructor to learner (21). Rejecting this "traditional model" of learning in favour of a constructivist approach, where each individual learner "constructs" meaning and "creates" information, we employed the technology not to transfer skills, but to advertise the existence of online research guides and promote the use of library materials within the context of existing coursework.3 We also used the PRS technology to quiz students on their actual library use and research styles, and then "marketed" the library resources interactively. Working within Berthoff's framework for "making meaning" (passim.) we encouraged students to interact with (rather than simply use) library resources by synthesizing various source materials into a coherence based on individual experiences of what might be meaningful. Since the overall course pedagogy was based, moreover, on a principle to  "provoke not tell," the librarian deliberately presented the library components in a manner that echoed the instructor's style of offering material in a way that was quick, focused, and aimed primarily at piquing student interest.

Pedagogical style, however, was not the only locus of harmonization. In keeping with the findings outlined by Barnhart-Park and Carpenter in their article "Information Literacy and Literacy Questions," the content of the library sessions was also integrated as seamlessly as possible into the broader course content (11). Barnhart-Park and Carpenter argue that this harmonization, which they call a "relational perspective," (10) allows students to absorb discrete information as constituent parts of larger interrelated bodies of knowledge. In the article "Teaching Writing and Research as Inseparable:  A Faculty-Librarian Teaching Team," Dennis Isbell and Dorothy Broaddus echo Barnhart-Park and Carpenter by arguing for a clearer understanding of the manner in which research and writing interact in academia, "weaving together and feeding off each other" (53). Isbell and Broaddus go so far as to propose that the distinctions between research and writing are artificial and that it is necessary to integrate both into an "organic whole" (52). So while we were not specifically teaching the students how to write, we did want them to understand the inseparability of the research process from the writing of a business plan and integrate these separate tasks intellectually into an "organic whole." 

Finally, to determine how successful our attempt at reorienting students was, we quizzed students about their actual research practices and library use at the mid-way point of the term and at the end of the term.

Course objectives and the PRS technology

The Course

Our pilot study was conducted within the context of an introductory undergraduate business course (MGMT 1000 - Managing Contemporary Enterprise) for 400 incoming students in the fall of 2006 with an average student age of about 18. The course was delivered using a stakeholder framework aimed at helping students understand basic principles through various lenses (e.g. leaders, marketers, employees, environmentalists, financial investors, social entrepreneurs, and so on). One of the major course projects involved the creation of a business plan for a student-selected product or service (i.e. learning about business by creating a business proposal). Randomly assigned student groups of five worked together over a period of ten weeks to select a focus for their enterprise, and to present their ideas both verbally and in writing for evaluation (making up 20% of the final course grade). This project encouraged collective self-directed learning as teams researched and explored various options with the goal of producing the top project to garner the coveted trophy for the best business plan at the end of the term. 

The decision to explore the use of "clickers" in the fall 2006 course was made to:

For this course, each student was required to purchase a hand-held component of the PRS technology at the beginning of the term and to bring it with them to each class. Although they were aware that their responses were recorded for attendance and the occasional spot quiz, answers to questions in class were always displayed anonymously. This meant that although a summary of all responses could be viewed by the whole class, individual students did not know how the person next to them had responded unless that person chose to self-disclose. Most questions posed were to survey students' perceptions and thoughts on key issues and topics (i.e. Which of the following do you think is most important?) and not to test knowledge (i.e. Which of the following is correct?). This was consistent with the course's broader pedagogical framework designed to encourage knowledge generation and sharing.

PRS technology

The basic PRS technology is relatively simple.  Each student has his or her own "clicker," a device which resembles a TV remote with numbers and/or letters on its face.  Questions can be asked either verbally by the instructor or presented on a screen at the front of the class.  Students respond by selecting a number or letter depending on the structure of the question (multiple choice or yes/no).  The handheld device sends a signal to a receptor located in a stand alone terminal.  Results are immediately tabulated and presented on the screen in a variety of formats (graphs, numerical, etc.) and survey results can also be saved for later use.  Since each receiver can be linked to a specific student, the system can also be used to take attendance and/or administer quizzes.

There are many PRS systems available on the market. It is not the purpose of this article to recommend one supplier over another based on an objective comparative assessment of these technologies. In our setting, the brand was determined by the institution since we chose to employ the product that was supported by the central computing support group at York University. We highly recommend using the supported technology at your own institution since that often eliminates many of the technical difficulties that are often attendant in the adoption of new technologies of any kind.

The Study

Introduction and Design

This case study was comprised of a series of questions built around a range of potential responses. The questions were designed to foreground actual research practices and provoke thought among students about additional potential approaches to research in an effort to encourage learners to perceive the integral nature of library research to learning in general. We used multiple choice and binary (yes/no, true/false) questions. We also attempted to employ humour to engage the audience by adopting a trope - the "way too smart dog" - as a comedic refrain.

Our goal in the design of the questions was to provoke learning, encourage library research efforts, and engage student attention in a fun and enjoyable manner using PRS in the classroom.  Earlier studies indicate that students typically respond positively to the use of PRS technology in the classroom (Kam and Sommer 6).  Author Jane Caldwell reports that "88% of students either 'frequently' or 'always' enjoyed using the clickers in class" (14).  Margie Martyn, in "Clickers in the Classroom:  An Active Learning Approach," documents a similar response from students with the majority of students choosing to "agree" or "strongly agree" with the statement "I enjoyed participation with clickers …" (74).  In our experience, it was clear that the students are familiar and comfortable with the quiz-game format and perceive the interactive nature of the technology as "fun."

Project Methodology and Sample Questions

This pilot project involved 400 first year students in the BBA program at a large business school. The students were divided into two classes of 200 each, which were scheduled on the same day in immediate succession. Library sessions were conducted in both classes on the same day, by the same librarian, and using the same teaching materials and survey questions.  For this study the students were asked a series of questions regarding research practices and library use at beginning of the term, the middle of the term (week 8), and at the end of the term during the fall of 2006. (A complete listing of the survey questions is included in Appendix A.)  Student responses were received and tabulated, both in class when the results were posted on a large screen as part of the learning experience, and later for the authors of this paper as part of the data collection segment of the study.

First Session and Initial Results

The initial library session was conducted on the first day of class at the beginning of the term. This was also the day that the students were introduced to the PRS technology for the first time. The new technology was explained and the students each logged in with their new handheld transmitter device. The library session commenced immediately after the course instructor had offered a few introductory remarks and presented a small number of test questions to identify and correct any technical problems.

The goals of this first library session were multifaceted: to quiz the students on their current research practices and library use, to provoke students into thinking about and planning for opportunities to incorporate research into their term work, and to promote library resources and services as a practical first step in successfully completing course work. Initial responses illustrated both unsurprising and unexpected approaches to research on the part of students.

Responses to an initial general question indicated that the majority of students do their research in the library (31.5%), or by consulting with friends (24%). A follow-up question illustrated student confidence in their own information literacy and research abilities: while over half had never used the school's business library, 42.5% believed they could do so effectively. Subsequent questions established that students begin their research by reviewing course materials (approximately 38%) or by consulting with friends (31.5%). At this point the librarian provided a brief overview of the library website and services. Students were then asked about their planned research strategies for the course and after this introductory session 66% planned to use the library for their research and 57.5% wanted more information about how to do business research.

In sum, the feedback gathered using PRS indicated that this group of first year BBA students were comfortable with the prospect of undertaking library research and expected to be able to meet course research expectations. Students responded positively to the library promotion and immediately began to think reflectively about how they might best access library resources to meet the course requirements. 

Mid-term visit and survey results

The librarian returned to the class during week 8, roughly mid-way through the term, to present a follow-up survey and address any research questions that may have arisen during the intervening weeks. The final assignment involving the composition of a completed business plan, for which research was required, was due in three weeks time. In spite of the initial session and the impending deadline, close to half of the class had not started their research by this point. And although there was much student interest in the library at the first meeting of the class, at the mid-term point approximately 74% of students were not using library resources. In this context it was not surprising to discover that more than half of all students were finding the research process either "very difficult" (30%) or "hard but not impossible" (30%).

In response to the need these findings reflected, a lecture on library research methods was provided by the librarian. In spite of the large class size students were encouraged to pose research questions which would be answered on the spot. All this was undertaken in the hope that it would "provoke" individual questions (e.g. How can I do this for myself?), and foster self-direction among students. That this strategy had a measurable impact on the class was indicated by the large number of students who voluntarily sought out the help of research librarians in the business school's library in subsequent weeks. It is interesting to note that although this same course has been offered (by different instructors) for a number of years, this was the first time that the library had experienced such a high number of students enrolled in MGMT 1000 seeking guidance with their research.

Final visit and survey results

Our final set of questions, presented in the last class at the end of the term, supported earlier findings. It was evident that the majority of the students did their research by asking questions of friends or asking the help of a librarian. Surprisingly, however, while the overwhelming majority of students did use the library, only half of the students actually asked a librarian for assistance to find information. Perhaps more worrying was the fact that, of that group, only half to three quarters of the students found the librarian's guidance to be worthwhile. In addition, respondents indicated that after this learning experience a large majority planned to use the library for their research in the future.

General Participation Results

From a course perspective, the use of PRS technology proved effective - one might say it was actually more effective than could have been anticipated.

Not only did we experience an increase in participation in the classroom through the use of general responses to survey questions, students were eager to share their ideas verbally. Some actually held the clickers like a microphone when they responded verbally to elaborate on personal views. Did the silent solicitation of their opinion by PRS provoke the desire to verbalize their thinking after seeing the class response? Perhaps so … More research on this phenomenon is certainly required. Participation also included many students seeking out the professor to give suggestions for the course in the following year. Students have been in touch to start mentoring teams, peer coaching for the course, as well as direct involvement in the business plan competition.

Similar to the results of earlier studies, increased student engagement was a direct result of the use PRS technology in this class. And in facilitating one form of student participation (clicker responses), we experienced increased participation in a multiplicity of ways. As well as enhanced in-class engagement, there was also a noticeable increase in the number of students utilizing the library and the career centre for course work - so much so, that librarians and career counselors commented on the influx of students. Thus, instructors venturing into teaching with this technology should be prepared for the various forms and manifestations of increased student engagement and participation both in-class and in the larger academic environment.

Library and Information Literacy Implications

Our use of PRS technology in this pedagogical setting suggests that it might be employed with desirable results by librarians working in a variety of disciplines which attract a large number of students. In this particular instance, the course instructor and librarian were able to measure, with reasonable accuracy, student perceptions regarding the importance of library research in the course setting and student engagement in the research process as the course progressed; and in response to the data, adjust their teaching approaches accordingly. The information literacy focus of this project and the nature of the survey questions in turn provoked students to reflect upon their individual research practices and to seek out the expert help of research librarians to use resources in the business library.  This allowed students to experience for themselves, and thus truly understand, the real impact that research proficiency would have on their ability to draft a credible business plan.

Findings similar to these are being reported with increasing regularity by academics who argue that PRS typically increases student enjoyment, leads to better group interaction, and helps "students gauge their own understanding" (Roschelle, Penuel, and Abrahamson 3). These benefits all flow from the primary pedagogical benefit: namely that students in large-enrollment classes appear to shift more readily from passive to active learning in environments equipped with PRS technology (see Martyn 72 and Caldwell 11). Our own findings suggest that students adapt easily to the use of this technology and feel democratically empowered to respond to their instructors in a variety of ways that include anonymous clicker responses as well as more traditional modes including raising of hands and posing questions verbally. The particular value of this case study was to show that these broader findings seem equally applicable to pedagogical settings in which learning objectives are built around and integrated with the principles of information literacy.

Moving Forward

One of the most surprising, albeit incidental findings of this case study was the determined way in which students continued to consult with friends in an effort to formulate a practicable research strategy throughout the term. Makani and WooShue in their 2006 study "Information Seeking Behaviours of Business Students and the Development of Academic Digital Libraries," write that "business students frequently interact with others when seeking information. […] Students mentioned that they often consult with their peers, professors, teaching assistants (TAs), and experts in relevant fields during the information seeking process" (Makani and WooShue 35).  And more significantly, "among these contacts, peers (classmates) are considered particularly important" (Makani and WooShue 35). Other studies, moreover, indicate that peer instruction pedagogy can be supported through the use of PRS technology (Burnstein and Lederman 8-9). The extent to which peer instruction or peer coaching seems to occur in business invites further study of the degree to which students in other disciplines view research as a collaborative endeavour. These attitudes, if different, may also influence the degree to which non-business students are willing to employ technological tools such as clickers in lecture-hall settings for information literacy and other purposes.

Appendix A

Initial Session

  1. How do you do research?

  2. How will you get started for the research for your Business Plan Project?

  3. Have you ever used the Library at the Schulich School of Business?

  4. Do you know how to use the library effectively?

  5. Was this demo helpful - did it give you ideas for your project?

  6. After this session do you plan to use the library?

  7. Would you like more information on business research?

Mid-term Session

  1. How many of you have started your research?

  2. Are you using the library resources?

  3. How do you find the research process?

Final Session

  1. How did you do your research for the Business Plan Project?

  2. Did you use the Library at the Schulich School of Business?

  3. Did you ask a librarian for help?

  4. Was the information you received at the library useful - did it help you with your project?

  5. After this experience would you use the library in the future?

Works Cited

Barnhart-Park, Anne C., and William J. Carpenter.  "Information Literacy and Literary Questions." Academic Exchange Quarterly 6.4, (2002): 10-16

Berthoff, Ann. E.  The Making of Meaning:  Metaphors, Models and Maxims for Writing Teachers.  Montclair, N.J.: Boynton/Cook Publishers, c1981.

Burnstein, Ray A., and Leon M. Lederman.  "Using Wireless Keypads in Lecture Classes." The Physics Teacher.  39, (2001): 8-11.

Caldwell, Jane E.  "Clickers in the Large Classroom:  Current Research and Best-Practice Tips."  CBE - Life Sciences Education. 6 (2007): 9-20.

Fies, Carmen, and Jill Marshall.  "Classroom Response Systems:  A Review of the Literature."  Journal of Science Education and Technology. 15.1 (2006): 101-109.

Guthrie, Rand, and Anna Carlin.  "Waking the Dead:  Using Interactive Technology to Engage Passive Listeners in the Classroom."  Proceedings of the Tenth Americas Conference on Information Systems, New York, New York.  August 2004. 1-8.

Isbell, Dennis, and Dorothy Broaddus. "Teaching Writing and Research as Inseparable: A Faculty-Librarian Teaching Team."  RSR: Reference Services Review.  23.4 (1995): 51-62.

Kam, Cindy, and Barbara Sommer.  "Real-time Polling Technology in a Public Opinion Course." PSC.  39.1 (2006): 1-5.

Makani, Joyline, and Kelli WooShue.  "Information Seeking Behaviours of Business Students and the Development of Academic Digital Libraries."  Evidence Based Library and Information Practice. 1.4 (2006): 30- 45.

Martyn, Margie. "Clickers in the Classroom:  An Active Learning Approach." Educause Quarterly. 2 (2007): 71-74.

Morris, Ruth C. T.  "Towards a user-centered information service." Journal of the American Society for Information Science. 45.1 (1994): 20-30.

Roschelle, Jeremy, William Penuel, and Louis Abrahamson.  "Classroom Response and Communication Systems:  Research Review and Theory."  Paper presented at the Annual Meeting of the American Educational Research Association, San Diego, CA, Apr. 2004.  1-8.


[1] For this paper, lecture hall classes are defined has having more than 75 students in the class.

[2] Shannon, Claude Elwood and Warren Weaver. The Mathematical Theory of Communication. Urbana, IL: University of Illinois Press, 1949.

[3] A major theme in Bruner's constructivist theoretical framework is that learning is an active process in which learners construct new ideas or concepts based upon their current and past knowledge. The learner selects and transforms information, constructs hypotheses, and makes decisions, relying on an existing cognitive structure (schema) to do so. This cognitive structure provides meaning and organization for the learning experiences and allows the individual to build upon previous understanding. - Adapted from Mark Ferrer www.west.net/~ger/Orientation/constructivist.html