Graduate School of Education
University of California, Berkeley
4533 Tolman Hall # 1670
Berkeley, CA 94720
(Paper presented at AERA, New York, NY, April 1996.)
The Knowledge Integration Environment (KIE) is an instructional computer environment that engages middle and high school students in scaffolded inquiry with scientific evidence from the Internet. KIE consists of a set of complementary software components which provide browsing, note-taking, discussion, argument-building, and guidance capabilities. This paper presents classroom research relating to the design of KIE curriculum projects, including: measuring the overall effectiveness of projects, the cognitive effects of multimedia representations of phenomena, methods for encouraging student perspective taking, and the instructional implications of metacognitive prompts. Implications for the development of a theory of instruction are discussed. Keywords - science education, networking, pre-college computer learning environments, scientific evidence.
KIE draws on technical, cognitive, and social resources to create a productive electronic learning community for students. The learning environment helps K-12 students use the as-yet untamed (and growing) Internet to acquire skill in interpreting scientific material, gain an understanding of complex scientific ideas, and develop a propensity toward integrating knowledge in general. As more and more schools connect to the information highway, the need for successful learning environments increases. These environments must take advantage of the corpus of "classroom materials" that are added to the network daily and allow students across the nation to work together to investigate scientific problems.
Bruner (1966), in his seminal work Toward a Theory of Instruction, identifies four areas that a theory of instruction must address. These include the experiences of students that help them develop a predisposition toward learning, the structure of the body of knowledge, the sequence in which the material is presented, and the feedback provided to students. In this paper, we focus our discussion primarily on the first of these aspects, investigating students' experiences with software and curriculum materials and the learning these experiences foster. We present first the theoretical framework on which we draw. We then briefly describe the curriculum and software components of KIE, and turn next to a discussion of our investigations into developing software and curriculum that are most beneficial to students. Finally, we discuss implications of our work for instructional settings which are enhanced by technology.
Theoretical Framework
In this section, we present a discussion of KIE's instructional framework. We then present a description of other influential research that has helped us in our evaluation of KIE's success.
Scaffolded Knowledge Integration
To help students gain a robust and predictive understanding of science, our instructional goal is to foster knowledge integration by encouraging students to make connections between scientific concepts and relate these concepts to personally relevant situations and problems. The curriculum discussed in this study was developed under a pedagogical framework that has been refined over the past decade called scaffolded knowledge integration (Linn, 1995; Linn, Songer, & Eylon, in press). The framework has four main components: (a) identifying new goals for learning, (b) making thinking visible, (c) encouraging lifelong learning, and (d) providing social supports.
The first component acknowledges that students bring a repertoire of scientific ideas about the world to science class, ranging in quality and degree of refinement. Instruction should help students build on these intuitions and encourage testing, revision, and reformulation of their scientific ideas (Linn, diSessa, Pea, & Songer, 1994). In such an approach, an in-depth coverage of the material is necessary to allow students to construct their own understanding. It is also important for students to see the relevance and productivity of their scientific ideas by embedding the science instruction in real world situations with which they are familiar. Through this grounded approach to instruction, students should be building their own scientific understanding. To this end, intermediate and qualitative models of scientific concepts and processes can be used to successfully bridge to students' prior knowledge and understanding (Linn & Songer, 1991).
Constructing knowledge in science is unlike everyday thinking in important ways. For example, science involves the ability to approach conceptual topics and evidence from multiple perspectives and demands a more stringent assessment of validity (Reif & Larkin, 1991). The second component of the framework -- making thinking visible -- emphasizes modeling scientific thought processes and making alternative models accessible to students. Modeling the application of a theoretical model for phenomena is one approach (Collins, Brown, & Hollum, 1991). Students also benefit when the actual processes of comparing scientific explanations, models, or theories are made visible and approachable.
Making thinking visible is not sufficient, however, since students also need to take responsibility for reaching their own learning in order to become lifelong learners. Thus, the framework calls for helping students reflect on their own ideas and monitor their own performance. KIE's pedagogy is based on a core scientific process -- the critical evaluation of evidence in the pursuit of deeper conceptual understanding of natural phenomena. KIE projects engage students in the critique of evidence and arguments, the debate of alternative hypotheses, and the application of scientific understanding to design scenarios. Students will benefit from exposure to these authentic scientific inquiry processes; coupled with the metacognitive aspects of reflecting on and monitoring their own understanding, the students will be more inclined to be autonomous, lifelong science learners.
The final component of the framework involves enabling and orchestrating productive social interactions in the classroom while guarding against situations which would reinforce gender stereotypes or status effects (Linn & Burbules, 1993). Students can productively exchange alternative perspectives and effectively collaborate only when they respect each other. Within an equitable social arena, students share, reflect upon, and refine their scientific understanding as a group. They can also participate in on-line discussions as part of KIE, where many of the same social norms and benefits are present.
Cognition in the Wild Classroom
In his book Cognition in the Wild, Hutchins (1995) presents analyses of the real-world cognition occurring among naval personnel navigating a large vessel. The approach is one of looking at cognition distributed across individuals, where representations of information are referenced by and passed among the participants in this setting.
The classroom is a correspondingly complex cognitive setting. Students, like the sailors, all have different knowledge, different skills and abilities, and different experiences. Although a teacher is nominally in charge of the students' progress, a more constructivist approach encourages the students themselves to take control of their own learning. A group of students working together in KIE must collaborate closely, combining and integrating their joint understanding of the project goals, the content material, and their relevant prior knowledge. The students also interact with the computer software, gaining further insight into the intended goals and evaluating scientific evidence to be weighed against their current understanding of the phenomenon under consideration.
Through all of these interactions, students end up with a personal understanding of the problem at hand. Individual understandings are often compared, contrasted, and reconciled with their peers' understanding, allowing further learning to occur. The end result is a conceptual understanding of phenomena that is better integrated. A student acts as a cognitive system working within the context of the larger cognitive system consisting of their peers, the teacher, and the technology. This theoretical perspective places an emphasis on evaluating the individual and distributed cognition that is occurring by analyzing the representations of information being used during the process whether they be oral, written, diagrammatic, or digital. In order to better understand the instructional setting, our emphasis is then to study how individual students are interacting with the software and curriculum of the learning environment and communicating with others in the classroom as they engage in their inquiry.
The KIE Approach
The Knowledge Integration Environment provides a complex mix of curricula and software. This section describes the development of both aspects of our work.
KIE Curriculum Development
We turn first to the curriculum. KIE is a project-based approach to instruction, so we will first describe the types of projects we are investigating. We then present a discussion of the types of activities which make up those projects. Finally, we will discuss the approach we have been taking to describe and represent scientific evidence, which the curriculum is focused around.
KIE Projects and Activities
The KIE curriculum consists of largely independent units called "projects." Projects typically last three to ten class periods. Longer projects are conceptual and integrative in nature, while shorter projects tend to be for specific logistical purposes such as becoming familiar with particular pieces of software to be used in larger projects. We have identified three types of large projects most useful for helping students develop an integrated understanding in science. These include debates, critiques, and design projects.
In this paper, we will focus primarily on one particular debate project, "How Far Does Light Go?" We discuss research done within the context of other KIE projects when they provide additional insight to the issues being discussed. The "How Far Does Light Go?" project (hereafter abbreviated "How Far...?") asks students to contrast two theoretical positions about the propagation of light using text and multimedia evidence derived from both scientific and everyday sources. The first theoretical position in the debate is the scientifically normative view that "light goes forever until it is absorbed" (hereafter abbreviated "LGF"), while the second is the more phenomenological perception that "light dies out as you move further from a light source" (abbreviated "LDO").
Students begin the project by stating their personal position on how far light goes. They review a set of evidence and determine where each piece fits into the debate. Then after creating some evidence of their own, the students synthesize the evidence by selecting the most prominent pieces in their opinion which factor into the debate and composing written explanations to that effect, resulting in a scientific argument supporting one of the two theoretical positions. Student teams present their arguments in a classroom discussion and respond to questions from the other students and the teacher. As "How Far...?" concludes, students are asked to reflect upon issues that came up during the activity and once again state their position in the debate.
We are researching productive sense-making and integration activities involved in using scientific resources found on the Net. Projects such as "How Far...?" are comprised of such activities. Examples of such activities include surveying evidence, searching for evidence, critiquing evidence, developing a scientific argument, developing a design, and participating in an on-line discussion.
"Component activities" are productive in more than a single instructional context. Component activities become the standard practices engaged in by students using KIE. Although the specific aspects of the component activity might differ, the form remains the same. "Surveying evidence" provides an example of one such component activity. As students survey evidence, they start from a list of evidence provided in the project. In "How Far...?", this list is provided as a "bookmark" file, with pieces of evidence organized by general topic, such as "Stars" or "The Human Eye." They review the evidence from this list, and then use custom-designed software to rate the usefulness, credibility, and other aspects of the evidence and to take notes on it. Students also use KIE's cognitive guidance system ("Mildred") to get help in thinking about the evidence.
The Nature of KIE Evidence
KIE projects revolve around evidence accessed over the World-Wide-Web (see Appendix A for an example). Each piece of evidence contains: (a) an evidence body containing the principal content of the evidence, (b) descriptions of how and why the evidence was createdÑthe methods and inspiration, respectively, and (c) other descriptive information about the evidence like associated keywords and the author. Each of these components is shown in Figure 1.

Figure 1 - The Structure of a KIE Piece of Evidence
The evidence body is the most complex portion of the evidence. It can contain actual evidence content (e.g., a student lab report), or instead point to existing resources on the Web which contain the content (e.g., a link to a particular science resource). In either event, the evidence body is characterized by the following instructional dimensions. First, the content of the evidence item consists of instances of natural phenomena and potentially interpreted perspectives or summary statements. Second, the representation used relates to how the content gets presented as text, sounds, pictures, or digital movies. Third, the structure of the body involves how the content is provided with or without advanced organization and whether it is described in narrative or empirical form. The fourth dimension consists of the conceptual guidance provided for a piece of evidence about salient aspects.
Scientific evidence available on the World-Wide-Web can be quite complex and disorganized, especially for middle and high school students. The Networked Evidence Database (the NED, discussed below) allows an intermediate representation to be created which can scaffold students in working with the evidence by including advanced organization, summary information, interpreted perspectives, and/or conceptual guidance.
KIE Software Development
The other primary aspect of KIE is the software learning environment. It is composed of a complementary set of both custom and commercial software, designed to augment the desired instructional approach. KIE features the following custom software components for students to work:

Figure 2 - The Netbook and KIE Tool Palette
The Netbook
The Netbook allows students to collaboratively manage their projects and documents, as well as become authors on the Web (see Figure 2). A Netbook is opened for the team as they log onto the KIE system, providing access to their current and past work. This software uses a notebook metaphor to provide students with access to all of their projects (accessed by means of the tabs in the notebook), the sections within those projects (as folders within each project), and all of the documents that make up each of those sections. Students can create, open, or delete any of the Netbook documents. It is important to provide students with access to their prior KIE projects so they are encouraged to build on their prior ideas.
The Netbook has been explicitly designed to simplify authoring and working with World-Wide-Web documents. It hides the details of working with Web documents, so students can focus on the science involved in the project. It functions as a jumping off point to a Web browser, word processor (and HTML editor), and other multimedia authoring tools.
The Networked Evidence Database (NED)
The World-Wide-Web allows individuals to organize existing information resources from around the globe into new forms. For example, an inspired student can take disperse information relating to home insulation and turn it into an integrated information resource on the Web structured by geographic area. As part of the KIE curriculum framework, we have designed and implemented a Web-based database structure composed of scientific evidence to be used by students in KIE projects, called the Networked Evidence Database (NED).
Obviously, the NED contains evidence from traditional scientific sources as well as descriptions of everyday experiences. Figure 3 shows part of a piece of everyday evidence which relates to the science topics of light reflection and absorption, light intensity over distance, and vision. It shows frames from a digitized movie of two bicyclists, one wearing black and one wearing white, riding up a street at night. The evidence students work with contains all of the additional dimensions discussed in the section on the nature of KIE evidence.
Evidence in the NED can come from KIE project developers who wish to make specific evidence available, from existing resources on the Net, or from the students themselves as a product of their classroom inquiry. As part of the KIE projects, students collaboratively engage in the creation of evidence to be published in the NED, drawing on the scientific methods and experiences they may be engaged in as part of classroom laboratory or research activities. For example, a student team may author a piece of evidence summarizing a lab they performed in class or they may make a digital movie of a particular phenomena relating to a science concept in class. Alternatively, evidence may come from newsgroup discussions taking place on the Net or from other information resources on the Web.

Figure 3 - Frames from the "Bicyclists at Night" Evidence
On-Line Guidance -- "Mildred"
At any point while working on a KIE project, students can receive guidance to aid in their inquiry by pressing the Guide button on the tool palette. This activates the On-Line Guidance (referred to as "Mildred") which provides four different varieties of guidance (see Figure 4). First, it can provide a description of the project and the current activity within the project the students are working on-called "The Big Picture." The second type of guidance it can provide includes procedural guidance about what students should be doing as part of the current activity -- "What to do." This is closely related to the "How to do it" guidance which consists of logistical support for accomplishing the current activity within the KIE environment specifically. The last form of guidance is put under the banner "To think about" and consists of cognitive guidance for the current piece of evidence being investigated (if there is one), the current activity, and the project.
Evidence-specific hints guide the student to critically evaluate evidence in the NED. If students are having difficulty working with a piece of evidence, these hints can be helpful in pointing out salient aspects of the evidence with regard to the instructional goal of the project. Activity-specific hints help students as they work on a particular aspect of the project. For example, the student might be reminded of the goal of writing a critique of a piece of evidence, and what is appropriate to include in such a critique. Project-specific scaffolding guides students to think about what is the main idea they should keep in mind as they work on the various activities and look at the evidence for a particular project. All of these levels of scaffolding model appropriate modes of inquiry. They also provide stepping-off points for students to engage in meaningful discourse with their peers about particular activities or evidence. These hints are intended to help students develop an integrated understanding of the subject matter by encouraging them to produce personal explanations (e.g., Chi, Bassok, Lewis, Reimann, & Glaser, 1989).

Figure 4 - On-Line Guidance ("Mildred") Displayed While Surveying Evidence
SenseMaker
SenseMaker allows students to organize the entire set of evidence associated a project. Within the software students work with dots representing individual pieces of evidence and frames corresponding to conceptual categories for evidence to be sorted into. Frames can be hierarchically nested, and evidence dots can be duplicated so they may be categorized into more than a single frame. SenseMaker also includes a way for students to rate evidence based on categorical descriptive dimensions (e.g. low to high usefulness) and take notes on the evidence.
SenseMaker provides students with a means of organizing or grouping evidence into conceptual frames. Coupled with the ability for students to specify their scientific ideas for a piece of evidence as part of the note-taking functionality, SenseMaker provides a medium for student groups to produce very structured arguments or critiques (shown in Figure 5). The resulting SenseMaker representation act as a powerful artifact, representing the students' inquiry and scientific understanding. This artifact can be used to encourage individual reflection, or alternatively as a means of multiple student groups to share and compare the approaches they have taken with the evidence and project.
Instructional scaffolding can be included in SenseMaker through the specification of initial frames students receive as they begin working with the evidence. The descriptive dimensions used in a project also provides a means of structuring student inquiry. For example, in the "How Far...?" project students rated evidence credibility, methods, and overall usefulness.

Figure 5 - One Group's Understanding of the "How Far...?"
Evidence using the SenseMaker Software
SpeakEasy
The SpeakEasy facilitates discussion and collaboration across the Net about topics occurring as part of projects as well as evidence in the NED. The SpeakEasy interface is based on an earlier tool for collaborative learning in multimedia, the Multimedia Forum Kiosk (Hsi & Hoadley, 1994) . Using the SpeakEasy, students record their opinions and participate in a discussion summarized in an argument map. The multimedia interface with images, texts, sound, and video stimulates productive discussion and reflection (Hoadley & Hsi, 1993) . The SpeakEasy allows scientific discussions over the Net within a school or across sites. The instructional aspects of the SpeakEasy are discussed in detail elsewhere (Hoadley & Hsi, 1996).
Research on Fostering Integration through Instruction
As we investigate ways to improve our software and curricula, we have in mind a particular specific goal: to improve students' integrated understanding through their use of KIE. We turn now to a discussion of ways in which we assess that integrated understanding as well as ways in which we strive to improve it through instruction.
Learning the Tools
Middle school students are often understandably overwhelmed by the prospect of learning new interfaces and new science. We have designed small projects to help students learn to use the technology in KIE, so they will be able to concentrate on the content when they work on the large content-oriented projects like "How Far...?" When we started designing these smaller projects, they fell outside of the realm of science the students were learning. For example, one of our initial KIE instructional projects was called "Introduce Your Scientist." Each pair of students was assigned a scientist to research, and they developed a "home page" for that scientist. The content-free projects were successful at teaching students about the software, but took up time that could otherwise be used on actual content. In reaction we developed a contextualized KIE instructional project. "Looking at Light" served as a preliminary project to "How Far...?" and introduced students to using the SenseMaker software. It also provided a review of some of the light concepts they had been learning in class.
The software itself also helps students learn about its uses. Mildred's How To Do It also provides logistical guidance if students get "stuck" on technology issues. Both Mildred and the checklist help to lessen a student's cognitive load by providing external memory supports. And, all of the interfaces for the software evolve through trial and refinement as we learn what features are not transparent to the students. For example, we combined the various forms of guidance into a single location in response to students' needs. This guidance is all contextualized based on the specific activity and evidence students are working on.
Helping students learn to use the tools at hand improves their ability to concentrate on the conceptual issues. We turn now to a discussion of how we assess their understanding of those concepts.
Pre- and Post-Test Comparisons
Students are given written assessments at the beginning and end of the semester of instruction; the "How Far...?" project occurs toward the middle of the semester. Table 1 presents the pre- and post-comparisons for the questions designed to assess the central issue of how far light goes across three separate semesters of instruction; each semester represents a different cohort of eighth-grade students. In Semester 1, the "How Far...?" debate consisted of an off-line worksheet activity, taking approximately five days of instruction. (Pre-test data was not available in Semester 1.) Semester 2 represents the first semester of having implemented "How Far...?" on-line in KIE as well as the first semester of KIE, in general.

Table 1 - Pre- and Post- Comparisons on the "How Far...?" Written Assessment
The written assessment is an indicator of the content learning taking place within the "How Far...?" project. As evidenced by the data, there are significant gains in understanding on this topic over the course of the semester (X2=74.7, df=2, p<.001 for Semester 1; X2=64.4, df=2, p<.001 for Semester 2). By the end of the semester, many more students are responding to the test questions with an conceptual understanding involving a Light Goes Forever response supported with a corresponding Light Goes Forever explanation. When the semester starts, many students exhibit a naive realist perspective that if you cannot see something then it is not there. The type of gains shown in Table 1 symbolize a sophisticated shift in their thinking where light can still be present even if you cannot see it with your own eyes.
The student pre- and post-responses are very similar across semesters. Semester 2, in which KIE was used, shows a trend of more students moving toward a deeper conceptual understanding (X2=5.28, df=2, p<.075). This is significant considering the conversion of the project from an off-line worksheet into an on-line KIE project which was relatively complex.
The written assessment used serves as one indicator of students' conceptual understanding. Although it does not illuminate the process in which students were engaged, it does reveal students moving toward a more productive stance toward the nature of light -- one they can use to explain a broader range of phenomena in the world around them. From an instructional perceptive, Semester 2 was dramatically different than what was done previously; it was the pilot use of the KIE software and curriculum approach. We are encouraged by the students' performance on the written assessment.
Research on Perspective Taking
Much of the difficulty in having students consider alternative explanations for phenomena is tied to their tendency to see those phenomena only from the perspective of their current conceptual understanding (Gunstone, 1991; Kuhn, 1989). It is hard for them to engage in conceptual perspective taking when making sense of evidence, which is thought to be a beneficial part of recognizing alternatives in the conceptual change process. In an attempt to scaffold students' propensity to engage in perspective taking with the evidence, a subset from two different semesters were asked to defend the theoretical position they had not selected as most resembling their own personal belief at the start of the activity. In other words, some students were asked to defend a position other than their own. We hypothesized that this could be a way to provide students with the scaffolding necessary to consider the alternative theoretical position in the debate.
Student Trajectories through "How Far...?"
The perspective taking analysis was accomplished by tracking the aggregate trajectories of students through the activity using the following milestones: (a) pre-project self-reported belief, (b) position defended, (c) post-project self-reported belief, and (d) post-test response to questions on light propagation.
Figure 6 shows student trajectories through the activity for semesters 1 and 2 (the worksheet and KIE semesters, respectively). In the figure, the sequential nature of the four milestones is reflected by their placement from far left (early) to far right (late). Each milestone has several categorical possibilities and line thickness was used to represent the number of students passing the milestone categories for each eventuality. For example: in Semester 1, 70 students self-reported that they personally identified with the LDO position. Of those 70, 45 defended the LDO position while 22 were asked to defend the LGF position as part of the perspective taking intervention. Students asked to engage in perspective taking are indicated in Figure 6 by the cross symbol in the defend column. The included post-test frequencies on the right side of the figure provide a comparative measure for the self-reported positions taken by students at the end of the activity.


Figure 6 - Student Trajectories through the "How Far...?" Project
Perspective Taking Results
An analysis of the student trajectories involves assessing how students progressed through the project who were asked to defend the theory other than their own and comparing them to the students who presented the theory in which they believed. Data were compiled across theory.

(a) Based on self-report immediately after project
(b) Based on written assessment 8 weeks later
Figure 7 - The Effect of Perspective Taking in Semester 2 (KIE)
Immediately after the "How Far...?" project, students asked to engage in perspective taking were more likely to have aligned themselves with the new position (Figure 7a; X2=8.55, df=1, p<.005). This assessment was taken from a written post-opinion worksheet students completed. Students were instructed by the teacher that they were to respond to the questions in whatever way they were currently thinking about the topic, and not based on the position they had defended in the debate. The effects of perspective taking were also examined using the post-test as an assessment of the theory students were believing. As shown in Figure 7b, students showed no significant difference on the post-test based on whether they were asked to engage in perspective taking or not. Because we know that students' ideas about the topic were progressing overall (shown in Table 1), this indicates that students were perhaps self-reporting erroneously in the first place perhaps based on some social expectation. The perspective taking intervention had no perceived effect on students longer-term understanding of the topic. An alternative interpretation which warrants further examination involves the students' self-reported conceptual change representing a temporary shift in their thinking. This interpretation would fit with other research describing the complexities of conceptual change (White & Gunstone, 1989).
The Interpretation of Evidence
Students' conceptual change in the "How Far...?" project is certainly influenced by their interpretation of evidence. As discussed previously, pieces of KIE evidence are structured along certain dimensions (refer to Figure 1). From an instructional perspective, we have begun to research the effects of varying those dimensions in KIE projects. Current results focus on the media representation of the evidence body, that is, whether it makes use of text, pictures, movies, or sounds. Studying individuals working with isomorphic problem representations has been used to explore the differing cognitive processes involved with the representations (Zhang & Norman, 1995). A study was performed to contrast how students worked with text evidence within the "How Far...?" debate compared to how they worked with multimedia evidence which was isomorphic in content. Different students independently received either one version or the other.
The Media Effect
Students were asked to categorize the evidence into supporting either of the two theoretical positions or being irrelevant. Table 2 shows how students categorized the evidence within the conceptual framework of the debate. For six of the eight pieces of evidence, students categorized the multimedia evidence differently than the text evidence. As this evidence was designed to be isomorphic, this suggests that the representation used to describe phenomena plays a strong role in how it will be interpreted.

Table 2 - Student Categorization of Text and Multimedia Evidence
So, even though efforts were made to have the multimedia versions of the evidence be conceptually equivalent the text versions from a scientific perspective, students linked the evidence pairs differently to the debate in three-quarters of the cases. This provides evidence for a general effect of representation in influencing how students cognitively process scientific evidence, as found in research on diagrammatic representations (Larkin & Simon, 1987).
However, there was not a uniform effect of the multimedia evidence compared to that of the text. These differences indicate that the media effect does not uniformly polarize nor equalize student interpretation of the two versions of the evidence. In some cases, the multimedia version of the evidence was equally linked to both sides of the debate while the text version had been linked strongly to a single position, and in others there is a switch from one theoretical pole to the other. A likely hypothesis is that the differences are related to student categorization based on specific surface features of the evidence.
The representation also affected whether or not students determined a piece of evidence was relevant to the debate. In the Flashlight evidence, students linked a table of dataÑthe multimedia versionÑinto the debate more so than a text description of what the data represented. In this case, either students found the data evidence to be more accessible, or they were reacting to the data as being more relevant because it looked more "scientific." The latter interpretation, however, is not supported by how the students categorized the Reflection evidence which had a similar appearance. Students categorized the multimedia representation of the Reflection evidence as being more irrelevant to the debate than they categorized the text version as irrelevant. This indicates that students were most likely reflecting upon individual pieces evidence and not just reacting to the representation type as a form of scientific deference.
Light Conception Analysis
As part of the "How Far...?" project, students produced written explanations for evidence they included in their argument for one of the two debate positions. These explanations were coded for the light conceptions which they elicited (e.g., "telescopes look at light which is closer to the light source"). Bell (1995) presents a more complete description of the relevant light conceptions. After these explanations were coded for the light conceptions which they contained, the number of different conception categories used with a piece of evidence (i.e. the overall variety of conceptions) were compiled for the text and multimedia version of each piece of evidence. These results are presented in Figure 8.
A comparison of the number of conception categories between the text and multimedia versions of evidence did not indicate a difference that was significant. However, for most of the evidence pairs a trend exists where the multimedia versions had a broader range of conceptions associated with them than with the text.
A rating of the scientific productivity of the conceptions used with a piece of evidence (i.e. the percentage of associated high conception categories) was also obtained and examined across media types. It should be noted that the multimedia evidence did not elicit light conceptions which were scientifically more productive than the text evidence.

Figure 8 - Number of Conception Categories used
with Text and Multimedia Evidence Isomorphs
The Effect Of Instructional Context
The instructional context also plays a role in how students interpret evidence. In the "Aliens" and "All The News" projects, for example, a piece of evidence called the "Anti-Heat Shirt" was used. This evidence, a scanned advertisement from a clothing catalog, discusses the "ice cloth" that the shirt is supposedly made of, and the "anti-heat sauce" that the cloth is dipped in to improve its anti-heat properties. In the fantastic "Aliens" design project, a large percentage of the students deemed the anti-heat shirt as appropriate for the aliens who require a cold climate. The students saw no problems with using this evidence that had been presented to them; they did not think critically about it. However, in the "All The News" critique project, where the stated purpose of the project was to think critically about each piece of evidence, students almost without exception pointed out numerous faults with the evidence: no data is provided, no experiments are run, there can be no such thing as "anti-heat sauce," the chemical is not named and thus might be made up, and so forth. Clearly, students' use of evidence is largely dependent on the goals they identify for the project.
Metacognitive Scaffolding
The scaffolding provided to students also influences the development of their understanding. Metacognitive prompts represent one aspect of the scaffolding students work with in KIE. Projects include two types of metacognitive prompts: activity-focused prompts and self-monitoring prompts. Activity prompts are designed to help students to identify appropriate, detailed considerations as they work on individual activities within the context of large projects. An activity prompt might, for example, provide students with the opportunity to justify their decisions or write scientific explanations of those decisions. Self-monitoring prompts, on the other hand, are planning and reflection prompts designed to help students map out their strategies for an activity and reflect back on that activity and identify their workÕs strengths and weaknesses. Davis (1996) provides a more complete discussion of the metacognitive prompts. Both kinds of prompts have been used in "How Far...?", but the bulk of the research has focused on other projects and so those projects will provide the context for the discussion here.
Activity Prompts
Activity prompts are largely project-dependent; they are designed for each activity within a particular project. Activity prompts for a project may be inspired by those used in another project, but they are not isomorphic across projects.
The first time activity prompts were used was in the context of a project called "Aliens On Tour," in which students design houses and clothing for three types of cold-blooded aliens with different climate requirements. In "Aliens," the prompts were focused on justification for decisions: students were asked to explain why their designs would be successful.
Out of our experience with "Aliens," where activity prompts were seen to help students provide scientific explanations for their decisions, the prompts were refined and tested again in the context of a critique project called "All The News," where students critique evidence and scientific claims in the context of heat flow and energy conversion. More prompts were used and they were expanded to include more than just justification ideas. The new activity prompts helped students think about all the processes involved in working through the project. In "All The News," the prompts were seen to help students do all the pieces of the project (Davis, 1996). Significantly more students receiving the activity prompts completed the whole project than did students who received other types of prompts, using a Fisher Exact test to compare each group (p<.05).
Self-Monitoring Prompts
Self-monitoring prompts, on the other hand, are generally isomorphic across projects. Typical self-monitoring prompts might ask students to plan ahead (e.g., "Specific things we need to think about as we work on our argument include...") or to reflect back on their work (e.g., "Pieces of evidence we don't really understand include..."). Prompts are refined based on students' responses to isomorphic prompts on different projects as well as the same project. For example, the self-monitoring prompts deemed unsuccessful in the analysis of "All The News" were eliminated from the next refinement of "How Far...?"
In the "Aliens" project, a chi-square test indicates that self-monitoring prompts helped students to use principles significantly more often in making their explanations (p<.05; df=2), and in "All The News," a chi-square test indicates that they significantly increased the incidence of principled knowledge integration (p<.05; df=3). It appears that self-monitoring prompts help students to look at the big picture rather than just the steps of a project. As a result of this overview students develop, they are more able to see the applicability of multiple sources of knowledge, and to integrate that knowledge (Davis, 1996).
Activity and Self-Monitoring Prompts In Concert
Both types of prompts are evolving to filter out those that are difficult for students to understand or that do not lead to helpful responses by many students. Further research is ongoing to research the effects of these improved prompts. It appears that activity prompts alone help students to complete all the pieces of the project and self-monitoring prompts alone help students to develop or demonstrate an integrated understanding within the context of the project. Research in the "Aliens" project indicates, moreover, that the two types in concert have a synergistic effect. In "Aliens," students who received self-monitoring prompts in addition to the activity prompts were more likely to include scientific justification in their response to the activity prompts than were students who did not receive the self-monitoring prompts (Davis, 1996).
Work is underway to investigate the effects of the two kinds of prompts in "How Far...?" It is hypothesized that the results will be similar, and that the self-monitoring prompts will help students prepare coherent, integrated arguments. Although projects like "All The News" and "How FarÉ?" have different cognitive goals, our research indicates that students can be supported in attaining these goals in similar ways.
Conceptual Scaffolding
As students are interpreting evidence in KIE, there are two specific mechanisms for conceptually scaffolding the process: (1) students can receive hints for the evidence, and (2) students can be provided with an initial set of conceptual frames in the SenseMaker software for sorting the evidence.
Evidence Hints
Prompts and hints are provided through Mildred and SenseMaker, as well. The hints available from the "To Think About" button in Mildred focus on helping students understand the goals of the project and the activity, and more importantly, they provide hints for thinking about individual pieces of evidence. For example, for the "Searchlight Photo" evidence, students would receive a hint saying, "What makes the searchlights look like this? Why are they so bright at the bottom (by the source)?" If they continued to ask for more hints, they would get another hint asking, "What might it look like from a plane flying over the searchlights?" Questions like these help students identify the aspects of the evidence that are particularly important to think about. The hints in Mildred attempt to model the kinds of things scientists would think about if they were investigating evidence of this ilk.
The hints provided by Mildred primarily provide fodder for pair discussions or individual thought; students do not write responses to these hints. In the SenseMaker software, though, they are asked questions about particular pieces of evidence. Here, they might rate the usefulness of the evidence, and then receive prompts asking them to think specifically about the methods used to create the evidence, the credibility of the evidence, and the science behind it.
Evidence Organization
As discussed previously, the SenseMaker software provides students with conceptual scaffolding in the form of the frames which can initially be presented to students as well as the descriptive dimension on which to rate and explain evidence. The analyses of the first students to work with the SenseMaker software is currently underway. Groups were initially presented with four initial conceptual frames to organize their inquiry: To Be Sorted (used to hold the initial list of evidence), Supports Light Goes Forever, Supports Light Dies Out, and Irrelevant. After engaging in three days of using SenseMaker as part of the surveying evidence activity, students had created a variety of organizing sub-frames and produced explanations to describe the evidence. Figure 5 shows the resulting SenseMaker configuration for one student group, and Figure 9 shows the text of one of their evidence notes.

Figure 9 - One Group's Notes for a Piece of
Evidence from the "How Far...?" Project
Preliminary results indicate that students are capable of creating new frames to represent their understanding of the evidence, and that this creation is facilitated by modeling the process in software introduction projects such as "Looking at Light," discussed above.
Implications
Classroom science instruction can benefit from technology when that technology is used as a learning partner, supporting students as they actively engage in building an integrated understanding. The research described in this paper has described how we have approached this endeavor. When designing instruction which makes use of technology with the goal of attaining a more theoretical understanding of the process, several important aspects must be simultaneously coordinated.
First, it is important to ensure that the approach is informed by research. A theory of instruction for using technology can only be developed when important aspects of the instructional process are considered and researched. As described in this paper, we have developed through research a better understanding of the cognitive benefits of different kinds of prompts, how perspective taking can be scaffolded, and the effects of evidence representation on student interpretation. Second, it is important that this work occurs within authentic learning contexts. Classrooms are complex arenas where students can develop integrated understanding of complicated concepts; that integrated understanding should be a primary goal of the technology and the curriculum being developed. How students actually work with the new materials can only truly be assessed within the context of a classroom itself.
Third, we believe that it is important to engage in a trial-and-refinement process with the instructional materials. Iterative design, implementation, analysis, and refinement components are necessary to tease apart the effects of changing particular instructional dimensions of the curriculum or software. For instance, only by engaging students with the different pieces of software does it become apparent which aspects are instructionally effective and which components require further formulation in the next version. As a specific example, by examining how students were using (and not using) the guidance built into KIE in the pilot semester, we decided to centralize the guidance in one area in subsequent versions.
By following this iterative, research-focused program when developing instruction for real classrooms, we believe it will be possible to develop a theoretical and practical understanding of instruction that makes use of technology as a learning partner. It will require balancing the cognitive aspects of individuals and groups, the understanding of how individuals learn within a particular domain, and how instruction can be best formulated given the affordances of the technology.
Acknowledgments
The research presented in this paper has benefited from the collaborative efforts of the rest of the KIE Research Group: Steve Adams, Flavio Azevedo, Doug Clark, Alex Cuthbert, Christopher Hoadley, Sherry Hsi, Doug Kirkpatrick, Marcia Linn, Jim Slotta, and Judy Stern. We also appreciate the efforts of the eighth-grade students who participated in the studies. Additional information on KIE is available at http://www.kie.berkeley.edu/KIE.html.
This research is supported by the National Science Foundation under grant No. RED-9453861. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the authors and do not necessarily reflect the view of the National Science Foundation.
References
Bell, P. (1995). How far does light go?: Individual and collaborative sense-making of science-related evidence. Poster presented at the Annual Meeting of the American Educational Research Association, San Francisco, CA: AERA.
Bruner, J. S. (1966). Toward a theory of instruction. New York: Norton. Collins, A., Brown, J. S., & Holum, A. (1991). Cognitive apprenticeship: Making thinking visible. American Educator, 15(3), 6-11, 38-39.
Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13, 145-182.
Davis, E. A. (1996). Metacognitive scaffolding to foster scientific explanations. Paper presented at the Annual Meeting of the American Educational Research Association, New York, NY: AERA.
Gunstone, R. F. (1991). Reconstructing theory from practical experience. In Woolnough, B. E. (Ed.) Practical Science: The role of reality of practical work in school science. Philadelphia: Milton Keynes, Open University Press.
Hoadley, C. M. & Hsi, S. (1993). A multimedia interface for knowledge building and collaborative learning. Paper presented at the adjunct proceedings of InterCHI Ô93, (International Computer-Human Interaction Conference), Amsterdam, The Netherlands: Association for Computing Machinery.
Hoadley, C. M. & Hsi, S. (1996). A theory of collaborative networking in the science classroom. Paper presented at the Annual Meeting of the American Educational Research Association, New York, NY: AERA.
Hsi, S. & Hoadley, C. M. (1994). An interactive multimedia kiosk as a tool for collaborative discourse, reflection, and assessment. University of California, Hypermedia Case Studies in Computer Science.
Hutchins, E. (1995). Cognition in the wild. Cambridge, MA: MIT Press.
Kuhn, D. (1989). Children and adults as intuitive scientists. Psychological Review, 96(4), 674-689.
Larkin, J. H., & Simon, H. A. (1987). Why a diagram is (sometimes) worth ten thousand words. Cognitive Science, 11, 65-99.
Linn, M. C. (1995). Designing computer learning environments for engineering and computer science: The scaffolded knowledge integration framework. Journal of Science Education and Technology, 4(2), 103-126.
Linn, M. C. & Burbules, N. C. (1993). Construction of knowledge and group learning. In K. Tobin (Ed.), The practice of constructivism in science education (pp. 91-119). Washington, DC: American Association for the Advancement of Science (AAAS).
Linn, M. C., Songer, N. B., & Eylon, B. S. (in press). Shifts and convergences in science learning and instruction. In D. Berliner & R. Calfee (Ed.), Handbook of Educational Psychology. Riverside, NJ: Macmillan.
Linn, M. C., diSessa, A., Pea, R. D., & Songer, N. B. (1994). Can research on science learning and instruction inform standards for science education? Journal of Science Education and Technology, 3(1), 7-15.
Linn, M. C., & Songer, N. B. (1991). Teaching thermodynamics to middle school students: What are appropriate cognitive demands? Journal of Research in Science Teaching, 28(10), 885-918.
Reif, F., & Larkin, J. H. (1991). Cognition in scientific and everyday domains: Comparison and learning implications. Journal of Research in Science Teaching (Special Issue: StudentsÕ models and epistemologies), 28(9), 733-760.
White, R. T., & Gunstone, R. F. (1989). Metalearning and conceptual change. International Journal of Science Education, 11, 577-586.
Zhang, J., & Norman, D. A. (1994). Representations in distributed cognitive tasks. Cognitive Science, 18, 87-122.