Sherry Hsi and
Christopher
M. Hoadley
hsi@garnet.Berkeley.EDU * tophe@cs.berkeley.edu
(510) 642-9717 * (510) 642-9717
Graduate Group in Science and Mathematics Education (SESAME)
University of California at Berkeley, Berkeley, California 94720
USA
A software tool designed to support collaborative discourse between multiple participants is described. The Multimedia Forum Kiosk supports students' knowledge building as they read issues, reflect on comments, and develop a point of view. The system provides two representations for fostering better comprehension of discourse and reflection through classification of comments into an argument map. Multimedia elements (digitized video, audio, images) are posted to stimulate discussion. Its uses as an instructional tool and assessment tool, as well as a methodology for analyses of comments, are described. Guidelines for successful implementation in instructional settings based on classroom experiences are provided.
This paper reports on the design and testing of an electronic multimedia bulletin board called the Multimedia Forum Kiosk (MFK). Through reading about issues, reflecting on comments, and posting opinions, members of a community can use MFK to exchange thoughts, negotiate meanings, acquire knowledge, and provide data for assessment. Unlike previous technology-based tools for collaborative discourse [Scardamalia et al, 1989; Scardamalia & Bereiter, 1991; Kunz & Rittel, 1972] or earlier bulletin board models [Hsi, 1992; Kerns, 1991], MFK provides explicit rhetorical representations that support user comprehension and user/group interactions. Comments are accessed through face icons of conversation participants. Argument trees diagram conversation structure as it develops. Multimedia clips (digitized video and audio) are used to set the context for topics and serve as discussion starters. The MFK not only serves as a catalyst for discussion and learning, but also provides a vehicle appropriate for educational assessment.
We have examined how MFK supports group interaction through asynchronous discourse as well as how it documents community-wide views. Communication models have been described as one-to-one, one-to-many, and many-to-many. The Kiosk is a tool that provides a many-to-many communication, but allows a single user to participate at his or her own pace. In the following sections, we outline our theoretical perspective, explain the design of MFK system, describe our methodology of examining Kiosk comments, and then report on our experiences in using Kiosk for collaboration and assessment.
We postulate several ways in which social context and collaborative discourse can improve individual learning. People construct their understanding through everyday conversation both in and out of the classroom. In a typical classroom discussion, each individual is responsible for making sense of a discussion and making a contribution. Instructors, in turn, are responsible for facilitating the discussion, and monitoring the comprehension and participation of the group. However, discussion is difficult for those who don't interrupt or fail to get the attention of the instructor due to social norms and expectations. Also, discussion participants who have different levels of background knowledge on a subject are less likely to participate in the conversation. Learners at first may require some scaffolding before integrating community views and knowledge into their own thinking [Brown, Collins and Duguid, 1989, Newman, 1989]. Eventually, students should be encouraged to formulate their own goals, ask their own questions, direct their own inquiry, and do their own monitoring of comprehension, yet be able to contribute at their own pace. An activity such as this in which students learn by contributing to the knowledge of a group is termed knowledge building. [Scardamalia & Bereiter, 1991]. Our objective was to design a system for which this type of learning is facilitated, so that people can ask their "knowledge-building" questions in a non-threatening environment.
On the individual level, reflection is an important activity to foster this knowledge building. Students should actively think about issues, inspect individual arguments, evaluate different points of view, and organize their own view before responding constructively. Our system is designed to facilitate these reflective activities as well.
The Kiosk organizes discussion around several interesting topics that are suggested by the instructor or by students in the form of a question (see Figure 1). Although not necessary to promote constructive discourse, multimedia can be used to illustrate various aspects of the topic and to stimulate discussion. For instance, digitized video clips can show a curious lab experiment, a student expressing concern about a difficult to understand concept, or an image of a piece of instruction from class. For example, "What parts of the Bike Lab were most valuable to you?" is a question that might be posted by the instructor and stimulus materials could include a digitized video clip of students taking a bicycle apart during a laboratory session. To encourage more engaging comments, the author of the topic provides an audio or text overview that better frames the question. Participants in a Kiosk conversation access an author's overview by clicking on his or her face.
To participate in the community discussion, people need a sense of the other individuals in the community and their views. The Opinion Area screen was specifically designed to capture community-wide views on a specific topic (see figure 2). Users can hear an author's overview, watch digitized video clips, read the comments of others, or post comments of their own. Each comment is an overview of an individual's perspective on the issue. Comments are represented by a digitized icon of the face of the comment's contributor. This makes the contributor's identity salient to other users, and also encourages the contributor to self-monitor since the comment is unmistakably theirs. However, topics that address students' concerns or topics sensitive in nature may allow anonymous participation.
Discussion also entails negotiating, arguing, and taking sides. To accommodate a deeper discussion of issues, the Discussion Area screen was developed to complement the Opinion Area. The Discussion Area captures the relational structure of arguments and evolution of the conversation. Various threads of the discussion are represented so that users can trace an argument, line of reasoning, or origin of a thought (see figure 3). Information in the Discussion Area is organized like a discourse, so that people may understand it in ways analogous to understanding a conversation.
Before adding a comment in the Discussion Area, users are asked to classify their comment as an elaboration, alternative, critique, paraphrase, or question using semantic labels "and," "or," "but", "i.e.", and "?" (see figure 4). These labels serve to structure interactions, and scaffold users in following and participating in a discourse. The activity of categorizing and placing of comments in an existing map of arguments seems to foster reflection and self-monitoring, as indicated by a pilot study [Hoadley, Hsi & Schwarz, 1993]. Users were found to explore, explain, and reflect before participating in a discussion. Users cyclically comprehended others' viewpoints, reflected on those opinions , and then expressed their own. [Hoadley & Hsi, 1993].
Thus, the Kiosk is designed to help individuals learn by interpreting the words of others and reflecting on their own experiences. The Kiosk encourages metacognitive activities such as reflection, self-critique, and monitoring one's own understanding. These activities can aid in knowledge integration and comprehension [Inagaki, 1981]. Also, by interacting with a group, users encounter different levels of expertise. This contributes to scaffolding of knowledge integration [Linn & Burbules, 1992]. The types of learning conversations Pea suggests for promoting conceptual change may also be supported by the MFK [Pea, 1992].
The system is currently implemented in Hypercard with Quicktime. The Hypercard authoring tool supported on the Apple platform allowed for rapid prototyping, use of visual representations of discourse participants, and easy linking of multimedia elements. (We postulate that this will also facilitate future distribution and use of our tool.)
The Multimedia Forum Kiosk is designed to be flexible for both casual and formal learning environments. Because of its ability to foster communication and leave behind a comprehensible, structured record of the discussion, the system is ideal for use in many situations in which people communicate. Classrooms, museums, offices, laboratories: any of these settings could potentially benefit from using the Kiosk for brainstorming, consensus reaching, or soliciting opinions. In education, students, teachers, and researchers can make use of feedback collected by the Kiosk. For research purposes, all the comments and discussions are logged and time stamped electronically. This method of gathering data and instructional feedback provides an important bridge between the coarse-grained analysis of surveys and the fine-grained analysis of videotape. Surveys can collect general community views but not interactions between individuals, while video requires lengthy analysis and can overwhelm the researcher with details.
Other situations where one might find the Multimedia Forum Kiosk or systems like it are primary or secondary school classrooms. Here, the system could operate quietly in the background, collecting comments on issues related to school or home-life. The time-independent nature of the system would allow all students access to the discussion. This provides a friendly opening for students too timid to insert their comments in boisterous verbal class discussions. It also permits students with linguistic or developmental disadvantages to have extra time.
We now describe the contexts in which the MFK has been and is currently being used. The first use of the Kiosk was within our own community, for sharing views on current educational issues, discussing research videos, and talking about issues specific to our department. It was installed in the department lounge, where people often came to read, get tea or use the microwave. The topics included two discussions of general interest (about privatization of public schools and uses of multimedia in education), video clips from two research groups, two discussions directly related to the department (on allocation of computers within the department and on whether the department was a supportive community), and lastly a literally experimental topicÑthe video shown was of a short science experiment involving an egg being pushed into a flask. This video of the egg experiment was a model of how Kiosk could be used to promote scientific reasoning. For example, this Kiosk topic prompted these reflections about heat, thermal expansion, and air pressure.
"I think it's because the flame expands the flask from the heat of the flame."Our initial success in supporting a variety of discussion topics led us to try the MFK for a new purpose. The rich and interesting record produced by the Kiosk suggested using the Kiosk to document both the comments of individuals who are in the process of understanding and engaging in knowledge building, and to document community views or "classroom Gestalt". We have begun using the system as an assessment tool to collect course feedback and assess effects of curricular changes that might not show up on standardized tests or grades."... it doesn't look like it fell through the neck due to the expansion of the flask due to heat. Slow-motion reveals the egg's elongation, which would not result for the neck's expansion. Also, the heat is probably not sufficient for that..."
"Well, the egg gets into the flask because the air pressure in the flask is far less than that outside the flask. The pressure inside is decrease by the flame, which uses up all the available oxygen, creating a mini-vacuum."
"The burning paper fixes oxygen (i.e., it reacts with carbon, etc. in the paper), reducing the number of gaseous molecules in the bottle, hence creating a relative vacuum that sucks in the egg."
As part of a national effort to improve engineering education through innovative applications of technology [Ingraffea et al, 1990], we have tailored and used MFK as a method of assessing the impact of changes to new curricula by capturing views and reactions of students and faculty. MFK provides a way to document and evaluate activities for courses where feedback is often difficult to collect: in design studios, computer learning centers, and unscheduled laboratories. In our research experiment, MFK is being used (a) to monitor students' understanding of concepts and course materials (b) to contrast views of individuals with different expectations, norms, and experiences, (c) to foster dialogue and discussion about courseware innovations, (d) to document changes in student and instructor views, and (e) to examine changes over time. We are especially interested in retention issues, in reactions to technology-based curricular innovations, and in improvement of attitudes towards women and minorities in engineering. By using a Kiosk, we can both document knowledge building by capturing on-going discourse, and perform assessment by collecting student and faculty opinions.
Students who want to voice opinions about new curricular materials, the learning atmosphere, or instructional delivery can interact with the Kiosk during the semester either voluntarily or as an assigned independent activity. Kiosk comments received during the course enable instructors to modify the course and better tailor it to students' needs. An example of this is provided by a Kiosk discussion in an engineering design course in which students advocated more hands-on experiences. The question presented was "What are the best ways to teach design?".
"I tend to think that any method that delves fairly deeply into the design process would be beneficial. Any type of hands-on experience would, therefore, be helpful to really learn all of the problems associated with design..."Thus, the class had a consensus that the course should involve as much hands-on experience as possible, which enabled the instructor to respond to these needs. As a result of this Kiosk feedback, the instructor changed her curriculum and included a hands-on design project."In my experience, design has to be taught through a variety of techniques. I for one would rather learn through activities and interactions with other members of the class rather than the traditional lecture format of academics."
"In my undergraduate Aerospace Engineering studies we spent 10 weeks studying stuff like:
Kutta-Jakowski lifting theory, aerodynamics, etc. At the end of my career I realized that I did not even know what an airfoil looked like or how it was fabricated. I strongly advocate hands-on education."
"I've done both case studies and actual design in industry. I personally learn better by experience than by classroom learning and reading. However, I still think that classroom learning and reading can always help, never hinder, learning design. In fact, it can give you ideas on how to improve your designing skills."
"I think it is extremely useful to provide hands-on projects. A review of the history of design would also prove very effective. Analyzing the state of current technology, even if in a vary basic manner, I have always found to be helpful."
Comments collected by the Kiosk can be used in mid-semester assessment for instructors to monitor feedback continuously, or as end-of-course evaluation for education researchers to analyze the pooled data. For the purposes of evaluating curricular reform objectives of the Synthesis Coalition, several methods of data analysis were employed that explored usage, individual comments, and issues raised by participants.
A coding scheme based on the themes was used to categorize comments (Table 1). For each coding category, we provide a sample comment that is representative of the category. Many comments fit into multiple categories.
Work to date from using Kiosk in six different engineering courses demonstrates that the success of the Kiosk approach in classrooms depends on specific aspects of the situation. In particular, our experiences suggest the following guidelines for successful Kiosk installations.
(1) Introduce MFK as part of the course - To familiarize students with MFK, the instructor should allocate class time to demonstrate and explain the MFK. Faculty members must be responsive to student comments. Students are more interested if they feel their feedback will have consequences for the course and effect changes.
(2) Enforce mandatory participation - Based on our experiences with Kiosk, we believe that assessment should be integrated with the curricula and be part of the planned instruction. We found better quality and quantity of comments if the MFK was included in the course syllabus. Ideally, course assignments are given that require mandatory participation in the Opinion Area, and students are rewarded for their participation, reflection, and suggestion of topics for future discussion.
(3) Develop continuous monitoring and feedback on multiple topics - To support on-going discussion, Kiosk topics that change on a regular basis (for example, at 3-4 week intervals) keep students interested in the evolving conversation. To reward students for reflection and participation in collaborative discourse, instructors should give summaries of comments back to students within a week's time.
(4) Faculty participation and commitment - To have a truly successful Kiosk experience, instructors must be committed to evaluation and be interested in having their instruction assessed. To promote faculty participation, faculty should initiate the topics to be discussed and authored topics. We found Kiosk discourse successful when faculty showed enthusiasm for using Kiosk, adding to the conversation, and responding to students' comments.
(5) Ensure access - The system must be placed in a public location that is frequented by students. Students must also have time to read and make comments. Open course laboratories proved most successful, while restricted-hours laboratories or lecture halls were unacceptable. Current plans include developing networked versions of the system that could be used from a dorm room or computer center.
In general, our perspective on what the MFK represents has evolved. Initially, the software itself and the interface were the focus. Now we look upon the MFK as a collection of stimulus materials, as a general approach to data collection, and as an aspect of the curriculum itself. Fostering and maintaining a quality interaction is a far more complex task than building software to support such an interaction. It requires the participation and enthusiasm of the participants and instructors as much as the developers.
The strength of collaborative discourse lies in supporting multiple users, allowing remote access to the on-going conversation, and empowering instructors to create their own Kiosk discussions. Future plans include the development of an instructor's tool kit that will enable instructors to design topics, access a collection of stimulus materials, and select the participants in group discussions. To support sharing of Kiosk materials and knowledge building, plans are underway to design a networked version of Kiosk that can be linked to the World Wide Web. Before the Kiosk design can be improved, education researchers need to better understand the nature of social cognition and collaborative discourse. Research that includes discourse analyses of Kiosk comments is also planned.
The Multimedia Forum Kiosk has shown potential as a medium for collaborative discourse and reflection. Our experiences using the system in classrooms has shown that useful discussion can take place among many users, including those with limited English skills or low assertiveness in the instructional setting. By thinking about the questions and stimulus materials, the participants reflected on their own learning processes and those of others. A record of this is then available for researchers, instructors, and other students alike. In this way, the Kiosk acts as a powerful platform for productive discussion and interactions.
Kiosk interactions depend on the social commitment of the participants involved. Not surprisingly, Kiosk success for assessment depends on the involvement and enthusiasm of instructors. The single most important factor in using electronic communication media is the strength of the community in which it is used. Accordingly, individuals need to want to participate in collaborative discourse. Documenting community views requires active and thoughtful participation by the whole community. However, when used in conjunction with vibrant community, structured electronic discourse can enhance the possibilities for such useful interactive communication and knowledge building.
We would like to acknowledge the NSF Synthesis National Engineering Education Coalition for their support of this work, and Marcia Linn for guidance and support. Thanks also to Dawn Rickey for comments and suggestions on earlier drafts.
Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32-41.
Collins, A., Brown, J. S., & Holum, A. (1991). Cognitive Apprenticeship: Making Thinking Visible. American Educator. (Winter), 6-11, 38-46.
Hoadley, C. M. & Hsi, S. (1993). "A multimedia interface for knowledge building and collaborative learning." Paper presented at InterCHI '93, (International Computer-Human Interaction Conference), Amsterdam, The Netherlands (adjunct proceedings): Association for Computing Machinery.
Hoadley, C. M., Hsi, S. & Schwarz. (1992) Scaffolding Constructive Communication in the Multimedia Forum Kiosk. Unpublished manuscript. University of California at Berkeley, Education in Math, Science, and Technology.
Hsi, S. (1992). The video poster: a multimedia bulletin board. [Technical Report], (August, 1992). Apple Multimedia Lab, San Francisco.
Inagaki, K. (1981) "Facilitation of Knowledge Integration through Classroom Discussion" The Quarterly Newsletter of the Laboratory of Comparative Human Cognition, April ,Vol. 3, No. 2, pp. 26-28.
Ingraffea, A. R., Henderson, T. , Agogino, A. M., Eide, A., and Aceto, J. (1991) "Synthesis: A National Engineering Education Coalition," New Approaches to Undergraduate Engineering Education III, Engineering Foundation Conference, Banff, Canada, July 28-August 2
Kerns, C. Beyond the desktop: an exploratory study of multimedia computing in the classroom. [Technical Report], Apple Multimedia Laboratory, San Francisco.
Kunz, W. and Rittel, Horst W. J. (1972). "Issues as elements of information systems." The DMG 5th Anniversary report: DMG occasional paper No. 1. Berkeley, CA: The Design Methods Group, Department of Architecture, University of California, pp. 13-15.
Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation . Cambridge, MA: Cambridge University Press.
Linn, M. C., & Burbules, N. C. (in press). Construction of knowledge and group learning. In K. Tobin (Ed.), Constructivism and applications in mathematics and science Washington, DC: American Association for the Advancement of Science (AAAS).
Newman, D., Griffin, P., & Cole, M. (1989). The construction zone: Working for cognitive change in school. London: Cambridge University Press.
Papert, S. (1980). Mindstorms . New York: Basic Books.
Pea, R. D. "Learning scientific concepts through material and social activities: conversational analysis meets conceptual change" AERA paper presentation, San Francisco April, 1992.
Scardamalia, M. & Bereiter, C. (in press). "A knowledge building architecture for computer supported learning." In E. De Corte, M. C. Linn, H. Mandl & L. Verschaffel (Ed.), Computer-based learning environments and problem solving. Berlin: Springer-Verlag.
Scardamalia, M. & Bereiter, C. (1991). "Higher levels of agency for children in knowledge building: a challenge for the design of new knowledge media." Journal of the Learning Sciences, 1, 37-68.
Scardamalia, M., Bereiter, C., McLean, R., Swallow, J., & Woodrull, E. (1989). Computer supported intentional learning environments. Journal of Educational Computing Research, 5(1), 51Ð68.
