Assessing Electronic Brainstorming Effectiveness in an Industrial Setting

...

The authors were interested in comparing the effectiveness of individual versus group electronic brainstorming via an experiment. Employees and contractors at a national laboratory participated in an electronic brainstorm to pose solutions to a real-world problem; some of the participants were working in a group while others had to perform individually. The methods were clearly presented in the article. A detailed description of the procedure was made by the authors. An entire page was dedicated to explain what the participants had to do to start working on the brainstorming and what was the procedure they had to go through during the experiment. There were two important factors that the authors considered in evaluating the results: the quality of the ideas generated and the quantity. The goal of the study was straightforward, they wanted to compare the efficiency of two types of brainstorming methods so the experiment proposed was appropriate in order to answer the study questions.

The experimenters decided to use ANOVA, which is made for analyzing the differences between group means and their associated procedures. Therefore, the method used to analyze the data was clearly appropriate. However, some numbers were not clearly explained in the analysis. For instance, page 523, “There was a significant effect for number of ideas expressed on each day, Wilks’s Lambda, F(3, 65) = 22.784, p = 0.048, mu^2 = 0.11 …”. It is not quite clear what those numbers mean; people will little educational background won’t be able to understand fully the analysis. Some tables would probably have been more appropriate for the readers to interpret the results.

As stated above, the authors were focused on the quantity of the ideas generated as well as the quality. In Figure 1, we can observe that on average, members of the nominal conditions yielded more ideas than the members of the group. However, the difference was not significant. Furthermore, the authors specifically analyzed the number of sentences or words per idea, where the difference between both conditions was also negligible. To be able to evaluate the quality of ideas generated, two raters were chosen to “grade” ideas with a scale going from 1 to 7. The average quality was calculated by averaging originality, feasibility, and effectiveness into a quality score for each idea. The results showed that the average quality of ideas generated in the nominal condition was significantly greater than those of the group conditions. Others factors were considered in the evaluation of quality including the number of good ideas generated and again, the members of the nominal conditions outperformed the members of the groups.

The conclusions made by the authors are reasonable; they claim that “electronic nominal brainstorming warrants further study as a cost-effective means to address industrially relevant problems.” They also mentioned the fact that the findings that individuals were more successful than a group suggests time and cost savings potential. This conclusion is exactly what the experimenters were aiming for, and the ANOVA analysis clearly bolstered the idea that nominal brainstorming is more efficient than group brainstorming.

The information presented in this article can clearly have direct implications for design and practice since it will provide cost and time savings in the companies. However, industries have different targets. Some may rely on idea quality rather than on quantity measures and while others adopt a different philosophy where quantity is more important.

The work was limited in the sense that the authors should have hired people from the same educational background to perform the experiment. We are told that the 69 employees work in Sandia National Laboratories but they might have different areas of study; some of them might be more capable than others. Also, the sample size for the number of participants could have been greater for better accuracy in the results. It would have been feasible to overcome such limitations by choosing more participants and finding them meticulously. Those limitations don’t diminish significantly the value of their work; the experimenters did a great job by analyzing multiple factors that have to be taken into account in order to come up with effective results.

Works Cited Page

Dornburg, C. C., S. M. Stevens, S. M. L. Hendrickson, and G. S. Davidson. "Improving Extreme-Scale Problem Solving: Assessing Electronic Brainstorming Effectiveness in an Industrial Setting." Human Factors: The Journal of the Human Factors and Ergonomics Society51.4 (2009): 519-27. Web.

...