Development and Assessment of an Interactive Computer Program for Aerosol Education

An interactive web-based program consisting of three integrated modules has been developed to enhance the understanding of the multidisciplinary nature and the complex principles of aerosol science and technology at the introductory level. Formative evaluation resulted in many suggestions from potential users on the improvement of the format and message. Feedback from students showed that the majority thought this mode of learning was effective, and they liked the dynamic animation and interactive features. Many also indicated that they prefer the modules be used to enhance rather than to replace in-class instruction. Summative evaluation based upon preand post-tests demonstrated that the website was effective for teaching the content. The results of the t-tests showed a statistically significant difference in the preand post-test scores. In summary, the evaluation data clearly indicate both the usefulness of the modules as teaching tools and a high degree of acceptance by students and faculty.


INTRODUCTION
Aerosol science and technology (AST) is a discipline with applications in many fields ranging from science, engineering, public health, medicine, and others.For example, knowledge of the particle deposition rates in the respiratory tract will assist in assessing the health effect of bioaerosols and combustion aerosols.A similar understanding is being used by the pharmaceutical industry in the design of aerosol systems for effective drug delivery to target different regions of the respiratory tract.The growing list of aerosol applications has called for a need to increase the number of trained personnel with knowledge of this subject (Friedlander and Hopke, 1999).However, the instruction of this subject has been primarily limited to the graduate level and typically restricted to a single department with principles and applications focused to the specific discipline.
In recent years, the importance of cross-disciplinary education has received more significant attention nationwide.A recent publication by the National Science and Technology Council (NSTC, 2000) reported that "the government should encourage cross-disciplinary education," and it listed aerosol science as one of the "inherently interdisciplinary" subjects.AST also plays a prominent role in several initiatives, such as the National Nanotechnology Initiative.To truly explore and develop the potential of aerosol science and technology, it is essential to expand the students' vision and training from a strictly traditional disciplinary approach to a broad-based multidisciplinary approach.This will be possible by teaching of the subject not only at the graduate level, but also the undergraduate level.
The widespread use of the Internet has made access to information readily available.
Meanwhile the new generation D(igital), born to live with information technology (e.g.computers, cell phones, digital games), is fundamentally different from previous learners for whom educational systems were developed.They learn at a fast pace, more in parallel channels, and are active, rather than passive, participants.They want interactivity and have little patience for traditional one-way lecture and step-by-step instruction (Tapscott, 1998;Prensky, 2001).
Concurrent with the emergence of a new type of student, research has shown that the retention of students in Science, Technology, Engineering and Mathematics (STEM) is low (Seymour, 1995a).
One of the major reasons for this has been attributed to the presentation of mathematical equations in a non-interesting manner that does not connect the subjects to real-life experience, thus reducing the motivation for learning them (Seymour, 1995b).
The situation is particularly critical in STEM classrooms where there is a need to demonstrate concepts mathematically by abstract symbolic manipulations.For example, the determination of aerosols undergoing various dynamic processes, such as diffusion or coagulation, requires students to solve the corresponding differential equations; visualization of these mechanisms is critical to understanding, although it is rather difficult to do through conventional lecturing.To stimulate more students studying STEM, educational materials must be developed in a motivating manner that fits the learning style of the new generation.
Educational materials prepared in an interactive format using advanced technology are a more effective delivery mechanism than conventional lecturing (Kadiyala and Crynes, 2000;Hargis and Donnelly, 2001;Rutz et al., 2003) and can meet the challenges delineated above.The idea of using a web-based program to enhance student learning is supported by a number of studies that have shown the effectiveness of this mode of learning among university students (Hadjileontiadou et al., 2003;Rutz et al., 2003;Dearholt et al., 2004).Such a program would enhance the ability of instructors in STEM to introduce and explain difficult-to-understand concepts in a variety of classes.Additionally, the program can provide students with real-world applications that transcend departmental boundaries and that meet the need at the introductory level.
The goal of this study was to develop an interactive and dynamic computer program for enhancing the understanding of the multidisciplinary nature and the complex principles of AST at the introductory level.Formative evaluation was conducted to provide the designers with feedback on the ease of use and effectiveness of the materials.Summative evaluation was subsequently carried out to assess the effectiveness of delivering the scientific content.1. Particle Control Devices: Particles in exhaust gas streams from an industrial system pass through a control device before being emitted to the atmosphere.The design of these devices is an important element at reducing the potential exposure to the emissions.

PROGRAM DEVELOPMENT
2. Respiratory Deposition: The health impact of the inhaled aerosol is determined by the deposition region of the respiratory system that depends primarily on the particle size.parts.The first part is a narrative that illustrates background principles using dynamic animation for better visualization.This section includes real-world contextual materials that the students can relate to.The second part is an interactive web calculator (for designing a cyclone or for estimating the deposition fraction) or a web simulator (for hands-on virtual operation of the OPC).

Aerosol Measurement
Modules with dynamic visualizations and hands-on practice with a web calculator/virtual simulator were expected to provide an attractive and effective tool for better learning experiences.
Each module could be used as a stand-alone unit, or the entire program can be used as an integrated unit to study multiple aspects.For example, in the integrated format the user can set up the OPC to measure size distributions of the aerosol at different locations in the cyclone.Thus, students work with real-world examples to gain experience in optimizing performance with minimized health effects and allowed cost.
In order to maximize the ease of use and interactivity of the learning interfaces, various web and scripting technologies were used to implement the user interfaces.To allow for the widest possible exposure, a standard web-browser platform (Microsoft Internet Information Services, IIS) based on the widely used browsers (Internet Explorer, Netscape and Mozzila) was used as the client-side interface.Macromedia Flash was used to create highly engaging and active content because of its ease for developing interactive multimedia animation and audio presentations for the web.PERL was the scripting language used to produce interactive web pages, gather user-supplied data, and link to supporting software applications.

Methods
An extensive formative (during module development) and summative (after the modules were completed) evaluation program was undertaken to ensure that the three modules were effective at teaching the desired content as well as to determine the degree of satisfaction of the students using the modules as learning tools.Additionally, faculty were surveyed to allow them to provide input into what they would find useful in the classroom.

Formative evaluation: student
Formative evaluation was conducted to measure the ease and understandability of the format.
Students were recruited to test parts of the modules as they were developed to obtain feedback for the developers.A questionnaire (Table 1), composed of 13 questions, was designed to ask about the ease of navigating through the module and the participants' perceptions of the value of the exercise when compared to the traditional mode of classroom instruction.Nine of the questions asked students to provide written comments, while the other four asked them to indicate on a scale of 1 (weak agreement) to 5 (strong agreement), their level of agreement with the different statements regarding the website and this mode of learning.
In January 2003, 19 students in engineering were asked to review the module on the website and provide answers to the questionnaire.They had not previously been exposed to content on AST.Each student reviewed the module using the computer at a station in the computer lab at Particle Engineering Research Center (PERC) at University of Florida.The students were provided with the same set of instructions to follow as they reviewed the module and were allowed to take as much time as they desired.There was a wide range in time spent to complete the modules, between 15 minutes to more than an hour.
The qualitative data from this questionnaire were analyzed using guidelines established by Miles and Huberman (1994), the results of which provided valuable information for revision of the format, navigability, and other design aspects of the modules.Modifications to the modules were made according to suggestions from the module evaluation.13 Are there any other comments or concerns you would like to add about the module?

Formative evaluation: faculty
An important stakeholder group in the success of the project was the faculty that could benefit by having these materials available for use in their classes.Hence, the goal of the faculty survey was to ensure that the product would meet their needs and to gauge their willingness to use the product when it was ready.The working versions of the modules were made available for evaluation at two conferences to solicit the opinions of members in the aerosol community: 6 th International Aerosol Conference (Taipei, Taiwan, 2002) and 21 st Annual Meeting of the American Association for Aerosol Research (Charlotte, NC, 2002).Potential users (i.e.faculty from appropriate departments around the world) attending these conferences were asked to view the modules and to respond to a survey instrument.Among them, 48 individual participants tested the modules and responded to the survey.This group provided developers with many suggestions to improve the products.Table 2 displays the questions.

Summative evaluation: student
Following modifications based on evaluation information provided to the developers, summative evaluation was conducted to determine if they were effective at teaching the desired content.A pre/post test, quasi-experimental design protocol was employed.The use of a pre-test allowed evaluators to establish the level of knowledge of the content that the student possessed prior to using the module.While it was expected that it was unlikely that these students would have prior instruction in this area, it was important to test this assumption.The post-test following use of the module then measured the degree to which the students mastered the content as a result of using the material.The students did not have access to the computer when working on the post test.Testing was completed on one module at a time.
Data from the pre-and post-tests were analyzed using Statistical Package for Social Sciences (SPSS v. 10).Results from the qualitative data were analyzed for patterns in participants' responses.

RESULTS AND DISCUSSION
The three modules have been developed and are available for free public access at http://aerosol.ees.ufl.edu and www.aerosols.wustl.edu/aaqrl/courses/cycopcresp/.As of October 2006, the program has had more than 8000 visits since its formal inception in October 2003.

Formative evaluation: student
In response to questions 1-4, the majority responded positively about ease of use of the program.An interesting response was observed for question 5.While most expressed satisfaction with the website, one specified that he "learned by writing, not clicking," indicating the importance of considering different learning styles.Regarding question 6, the interactive and dynamic demonstrations were the feature that participants liked most.They also provided many suggestions for improvement and modification.Examples include "page seemed to be overwhelming; better if made a separate page for each equation" and "you make it easier on the reader if you would include a bulleted list rather than lengthy sentences." The responses to questions 8-11 are summarized in Table 3. Clearly the participants liked this form of learning (question 8).Responses with respect to a comparison to classroom lectures resulted in 33% indicating that they did not believe they learned more from the website than a lecture, 29% thought that they learned more from the website, and 40% had a neutral response.
The ability to complete the learning on their own time and pace were highly rated (questions 10 and 11).Eight participants pointed out that in a lecture they could ask questions of the teacher and that they believed the modules should be used in conjunction with lecture because of the value of interactions with faculty.Similarly, Montelpare and McPherson (2001) found that in the teaching of a Statistics course to undergraduates, the integration of on-line information with traditional lecture was more successful than the approach where most of the course was taught via the web.

Formative evaluation: faculty
Survey results overwhelmingly showed consensus on the great need and interest for developing such a program (questions 6 and 7).In response to questions 2-4, the overwhelming majority showed positive responses: • 92% indicated that the Cyclone module would be a useful tool for transport-related concepts or topics related to inertial behavior of particles; • 91% rated the OPC module a good virtual measurement tool; • 97% indicated that for aerosol health effect topics, the RD module would be useful.
It is clear that the modules were well received by this important group.
Open-ended questions allowed them to provide module developers with feedback of what they liked most and least about the program.The most common response was that they liked the interactive approach, visuals, simulations, animations, and design ability.They also provided many constructive comments that were later implemented in the revised program.Examples include: "It seems to include a lot of the 'what' and 'how' but almost not the "why", better if you would show large particles mixed with small and medium, and how large are collected and small are not."Many also encouraged the team to expand the existing three modules to cover more topics in aerosol science and engineering.As more applications are emerging in various disciplines, survey respondents also suggested that industrial practitioners who need to take on-the-job training would also benefit from it.
With regard to the ease and understandability of the format of the web-based modules, the majority of the participants thought this mode of learning was effective.The participants liked the graphics and animations on the website.The majority of participants liked this mode of learning, especially the fact that they were able to complete the modules at their own pace and on their own time.In the open-ended comments section, eight of the students commented that while they appreciated learning form the modules, they believed that they should be used as part of a classroom setting as they value lectures and the ability to ask questions and interacting with the professor.The modules were effective for teaching the content of the modules as shown on the results of the pre-and post-tests.
It can be concluded from these results that the modules will be valuable tools for enhancing students' understanding of AST in the introductory level, especially if used as part of a program that also includes lecture where students will have access to the professor.In other words, it seems that the modules may be better used to enhance, rather than to replace classroom instruction.

Fig. 1 .
Fig. 1.Algorithm chart of the web-based aerosol program.
Devices: Another important aspect is the monitoring of the aerosol size distribution.Depending on the nature of the aerosol, different instruments based on different principles have to be used.Accordingly, the program consisted of the development of three modules: (1) Cyclone for particulate control, (2) Optical Particle Counter (OPC) for particle size distribution measurement, and (3) Respiratory Deposition (RD) for health-effect assessment.Each module contains two

Table 2 .
Faculty survey questions used for conference participants.

Table 3 .
Summary of participants' level of agreement with statements in items 8 to 11 of formative evaluation.