The Acrobat.com Blog

Guest post by Phil Ice, Ed.Dﾠ.

(Dr. Ice is currently the Director of Course Design, Research & Development at American Public University System. He has been using Buzzword in his teaching for the past year, and has conducted extensive research on the impact of using Buzzword versus alternatives.)

In the traditional online classroom students complete papers and projects in Word and submit them to the instructor as an email attachment. When the project is collaborative in nature, the typical workflow model involves using the track changes and commenting feature. At the point that a project is deemed complete the changes and comments are removed and passed to the instructor, via an email attachment, who adds yet another layer of comments and markups.ﾠ

In the spirit of web 2.0, an alternative to the above process has emerged in the form of free, online document editors. Buzzword, Google Docs and Zoho are some of the best known products in this sphere. A core function among these is the ability to create a document online and invite others to view and / or collaborate. Other features vary by product and include the ability to include tables, images and other fundamental features found in Word. However, the means of editing vary significantly from product to product.ﾠ

In four online, graduate-level, education courses students (n = 78) were asked to submit group projects using Word and Buzzword in alternating fashion. Buzzword was selected from the available online document collaboration tools as it offers several unique features, including pagination, intuitive media editing mechanisms and a visually oriented sharing mechanism.

Evidence

Several types of evidence were reviewed to assess the efficacy of this technique. Document analysis was given the most weight. The average document submitted was 9.75 pages in length. The mean occurrence of non-text based items (e.g. hyperlinks, graphics, tables, etc.) was 5.1 for Word submissions and 14.3 for Buzzword based submissions.ﾠ

Open ended surveys were conducted with volunteers revealed that students were more likely to use graphics, charts, links, etc in Buzzword because of the ease of inclusion. Eleven students specifically stated that they did not include non-text sources in Word documents because they viewed the processes and subsequent modification as not being a fluid, user-friendly process.ﾠ

This proves to be very important in determining the effectiveness of a writing environment because, as the survey also revealed, students are more likely to explain more complex concepts using a combination of text and non-text based materials. The majority of participants expressed the view that it was easier to express themselves at a higher cognitive level when they could present material using multiple media sources. This concept is substantiated by Mayer’s work with multi-media learning.
ﾠ
Documents produced by students were analyzed to determine if differences existed with respect to cognitive engagement as a function of the medium in which the artifacts were created. Analysis revealed that instances of higher order thought (those processes that move beyond recall of information to require critical analysis and evaluation) were prevalent more than two times more frequently in artifacts created using Buzzword than Word. Though more research is needed, the interview data described above, indicates that the manifestation of higher order thought is a function of multimedia presentation capabilities.ﾠ

In other research involving Buzzword, students (n = 238) were asked to use a variety of document creation tools (Word, Google Docs and Buzzword) as a document collaboration and submission tools, over the course of a semester. Key findings from this study were:ﾠ

1. The vast majority of students (n = 202) preferred Buzzword. Qualitative analysis revealed that the sharing architecture (n = 143), ease of use (n = 127), ability to easily rearrange resources (n = 107) and ability to easily integrate resources (n = 98) were the themes associated with this preference. Of the remainder, 11 preferred Google Docs and 15 preferred Microsoft Word. The theme associated with the preference for these later groups was familiarity with Word or Google Docs.
2. The majority of students (n = 159) viewed Buzzword not as a true document editor, but as a multimedia creation tool. As an example, document analysis found frequent manipulation (n = 92) of documents to fit within a 1024X768 viewable area without scrolling.
3. The concept of a Buzzword being a personal reflective space was common among 72% of students over 34 years old. As an example these students frequently developed a document on their own and then shared it with classmates. In contrast, 82% of students 34 and younger recognized it as a collaborative tool and began workflow projects by sharing a common document. Qualitative analysis revealed that these younger students frequently (68%) likened Buzzword to a Wiki or similar collaborative tool. In contrast, only 27% of older students made this association.

ﾠﾠ

Impact on Other Measures of Effectivenessﾠ

In addition to the learning effectiveness and student satisfaction indicators discussed above, there are indications that instructor satisfaction may be impacted. Six instructors, who used Buzzword as a submission mechanism, were interviewed and 5 indicated a preference for the tool for the following reasons: 1. they perceived student artifacts of being of higher quality, and 2. the sharing architecture was perceived as being more efficient than accessing artifacts via email attachments. The sixth instructor indicated no preference. While document analysis tends to validate the perception of higher quality artifacts being generated, more research is needed to validate the perception of efficiency.

ﾠ