Original publication:

Dron, J., Mitchell, R. & Siviter, P. 1998, 'From Answer Garden to Answer Jungle,' Education and Training, vol. 40, no. 8.

This paper describes an experiment making use of a newsgroup to artificially accelerate the evolution of a learning resource within a group of level one students of computing and information systems at the University of Brighton. The author discusses how evolutionary processes can be perceived in most learning environments, and how Internet Based Learning (IBL) systems can provide an effective ‘jungle’ to encourage rapid evolution so that resources adapt to the needs of learners. The experiment draws on the students’ own comments and reflections on the process. The conclusion drawn is that the newsgroup is an effective aid to learning and a useful but imperfect tool for accelerating evolution.

Darwinian notions of variation, reproduction and selection underpin higher education. We develop courses and learning material, we get feedback, discover what works, drop what fails, modify ideas and lessons which compete with others in our own and other universities. However, what constitutes success is a complicated issue where success may be determined by factors including cost, what makes life easier or the educators, marketing concerns, or which fits into the bureaucratic structures of educational establishments and the needs of faculties. Only part of the driving force for evolving better learning comes from the need to provide better learning experiences for students. Evolution in traditional educational systems is a slow and lethargic process, full of reinvented wheels, insufficient interbreeding and intermingling and resistance to change.

The exponential growth of the Internet has resulted in new ways of combining CAL and CMC approaches, which has been described as Internet based learning (IBL), a "model of teaching that combines the advantages of both prescriptive and exploratory learning by selecting appropriate materials and admitting a wide range of views of a subject" (Tait, 1998). Various workers in the field have touched on the capacity of such systems to adapt and change rapidly and responsively. Brad Cox writes, "what actually governs complex systems is rarely the industrial age’s notion of design at all. Rather, they evolve, shaped by an interaction in which system and environment minutely adjust to each other as biological organisms evolve within ecologies." (Cox, 1997). This idea is embodied by the Evolutionary Knowledge Base used in the JITOL project, where it is an explicit goal of the system to focus on "the progressive construction of appropriate and evolving knowledge bases, built by constantly integrating users’ requirements"(Boder, 1992). A slightly less sophisticated but more widespread mechanism comes out of work by Ackerman and Malone on the notion of Answer Gardens (Ackerman & Malone, 1990), where a hierarchically organised searchable database of commonly asked questions and answers grows organically out of questions posed by its users. Where there is not yet a satisfactory answer in the database, the question is posed to an appropriate expert or teacher who then adds the reply to the database. Of course, more than one expert can be involved and the whole thing can become a tool for capturing organisational memory, or in our case a repository of evolving course material.

The Answer Garden was the basic model chosen for the experiment conducted here with one major variation- I explicitly excluded any experts, particularly myself. By so doing I sought to pull the learning resource up by its own bootstraps, avoiding ‘skyhooks’ (some unjustified external agency) and substituting "cranes"- "a subprocess or special feature of a design process that can be demonstrated to permit the local speeding up of the basic, slow process of natural selection, and that can be demonstrated to be itself the predictable (or retrospectively explicable) product of the basic process." (Dennett, 1995). The intention was thus to provide a resource completely tailored to the students’ requirements, without prejudice, preconceptions or learning style imposed by me. In effect, the intention was to produce an Answer Jungle (wild and self-organising) rather than an Answer Garden.

All of the material for this experiment comes from the newsgroup itself and the unsolicited comments of the students in their follow-up reports. The newsgroup forms an effective archive of the process, the reports form useful reflections on that process. I seek to analyse the results in terms of learning effectiveness and for evidence of evolution.

The assignment

This assignment involved a group of around 100 students, all at level 1 (effectively first year BA/BSc students, with some part-timers at the same level) on a module called ‘PC Concepts’. The course seeks to introduce students (mostly for the first time) to the issues and technologies surrounding personal computers.

The exercise was a two-week discussion using Usenet News of the relative merits of PCs, Net Computers (NCs), Windows-based terminals (WBTs), WebPCs and thin clients in general, with the intention of encouraging reflection on issues of PC management by considering the competition. I chose this topic as it is a particularly rapidly developing area about which there is little useful literature and which causes strong feelings and antagonistic opinions amongst industry commentators, and is ideally suited to a Problem-based Learning (PBL) approach (Funderstanding, 1998). It was an assessed assignment, with thirty percent of the marks going to newsgroup contributions themselves and 70% to a summary/conclusion document in the form of a web page in which the students were free to summarise, collate or discuss any or all issue(s) raised in the newsgroup. By making part of the assessment based on the students' contributions, I intended that they would all take part in the process. This was due to some previous newsgroup experiments with other groups being less successful than I had hoped because those with most difficulties with the subject (and who would thus benefit most) had failed to contribute, apparently through fear or lack of understanding.

For several reasons I made it clear from the start that I would play no part in the discussion. In particular:

• I did not want to discourage less able users from contributing by setting the initial standard too high
• I did not want to get a biased report based on the premise of 'pleasing the teacher'
• I did not want to be a 'sky-hook'. Any evolution had to happen without external intervention

I had given an earlier tutorial on use of the newsgroup, explaining the concept and providing some practice.

The results

A variety of different approaches were taken by contributors to the newsgroup. Some leapt in with seeding comments and short pithy replies, others attempted essay-like epistles, whilst others asked questions and attempted to conduct surveys.

In quoting the students’ remarks, I have left spellings and grammar as originally presented (perhaps an indictment of the web-authoring tools used by the students?).

The subject itself was very difficult for first year first-semester students, requiring a lot of background reading and a level of understanding which would typically be tricky for such students. CAF5 makes a typical comment:

"it seems obvious that the topic of future alternatives to PCs has been a huge subject for the students to grasp in such a concentrated way. It has been like 0-60 in 10.8 nanoseconds! However, the students (including myself) have clearly enjoyed the challenge judging by the volume and the variety of contributions."

Similarly PLB states:

"The newsgroup itself was very good as a source of information, and for getting a wide range of opinions. I think the only flaw of the newsgroup is that a lot of people didn't know a lot of the advantages and disadvantages of the systems debated, or even what they were in some cases, perhaps due to a shortage of research. But as a forum for debate it was excellent."

Given the difficult nature of the subject, the overall quality of learning and understanding was exceptional. However, the downside was that several had difficulties distinguishing truth from half-truths and fiction. LJ33 remarked:

"Having scanned through the messages 'posted' on the Newsgroup --- I realised that there were a great number of people, like me, with very little knowledge on the subject in question - but there were a few prolific 'posters' who seemed to have a good technical understanding of what was going on. Unfortunately, despite the latter group, a lot of basic questions posed by students remained unanswered. Hence, I found it difficult to build up a solid picture from all the little snippets."

One student suggested to me afterwards that in future, students should be encouraged to cite their sources during such exercises. I suggested that it would have been a good use of the newsgroup to bring this to the attention of other contributors; it should not be necessary for me to limit the style of discussion, but there is clearly a need for a more solid grounding in effective use of news so that the participants can impose their own limits.

The newsgroup as a learning environment

It seems that a form of evolution took place- students with no knowledge created a body of knowledge, reproducing, refining and enhancing snippets of information. DJAP1 wrote:

"I found the newsgroup to be beneficial in both learning facts and gaining a wide variety of views that I had not considered. This is the first time that I have participated in a discussion of this sort and I was pleased to find how well it worked in providing fact and opinions and then prompting relevant discussion."

In a similar vein, EKF wrote:

"There are various opinions on this Newsgroup some of which I agree with and others that I do not. The main benefit of this group, to me, was how items have been discussed that I hadn’t realised or thought of."

TM63 echoed this:

"I felt that using the newsgroup for this excercise was a good idea as it was very usefull to see volumes of opinons on such things as NET PC's, web TV etc before forming one of my own."

Five of the group did not contribute at all, although a couple of those turned out to be lurkers, as they produced adequate summaries but no contributions. The bulk of the students contributed one or two messages, although one managed to post over 20 messages. One person from outside the group unexpectedly contributed three messages. The interloper apologised:

" Sorry to keep butting in on your group, but the discussion is so much better than on ours. "

He contributed some useful material with a slightly different and possibly better-informed perspective (he turned out to be a final-year student with research interests in this area, although this was not explicit in his contributions). I discovered no adverse reaction within the group to outsiders butting in and it raises a lot of interesting possibilities. From an evolutionary point of view, introducing new information to the 'gene-pool' is a powerful means of accelerating evolution. However, there is a danger that the better-informed might take over the group and alter its dynamic to the detriment of those starting further back.

Some students had difficulties with the range of opinions and potentially dubious facts expressed in the newsgroup. I had hoped that this would happen, as it is important for them to be able to come to grips with widely varying differences of opinion in a rapidly changing market, but some were overwhelmed. SJ73 wrote:

"Confused? Yes! The consoling factor being that I'm not the only one! Personally I've trawled extensively through Techweb, read every magazine article I could lay my hands on, talked to people and read our newsgroup of (at the time) 120 plus messages and the conclusion I have reached is…… I'm not sure! "

Despite the requirements of the assessment to keep to the general subject area, a separate thread developed on the future of the floppy disk. This may have been caused by the initiating student’s poor grasp of what was required at the early stage this started, but it was obviously interesting enough and relevant enough to a number of students who continued the discussion. SJM29 wrote:

"A lot of the discussion was on Web TV, etc., but sometimes the discussion went off at a slight tangent, talking about floppy disks for quite awhile. Although not very related to the main discussion points some very valid remarks were written."

I see this as evidence that mutation and variation can occur in unexpected ways, leading to speciation.

Some students were put off by insensitive comments by one student in particular, although other messages showed similar traits to a lesser degree and may have seemed intimidating to some. Such behaviour is not uncommon in any discussion (including more traditional forms) but it may be that the relatively anonymous nature of the newsgroup encourages a more relaxed attitude to personal attacks than might occur face-to-face. This is a two-edged sword, which has distinct benefits as well as disadvantages. The relaxation of social restrictions on behaviour also means that those who would otherwise be intimidated or shy may be more inclined to participate in this semi-anonymous fashion.

My supposition that the body of material would provide different perspectives catering for different learning styles proved to be accurate. A number of comments confirmed this. MM202 wrote:

"When we were given this assignment, I could not understand the topic we had to research. I was very confused, but found the entries in the newsgroup from my colleagues were excellent! It was very helpful easy to read and very clear."

Similar views were expressed in a number of different ways. For instance, LJH26 wrote:

"The discussions that have occurred as a result of the IS115 newsgroup, have been very productive. To begin with those students who had difficulty defining the topics involved have been able to seek the help of their peers. This has been beneficial to everybody mainly because the explanations that have been posted, have been the answerer’s own words, not jargon from a book or magazine. Plain English has been essential."

JPM15 reflects this sentiment:

"The discussion was good because it made it easier for people to understand what was going on, as it was in peoples own words. Also it showed peoples understanding"

Part of the efficacy of the procedure came from the simple device of expressing thoughts publicly, a learning method informed by Pask’s conversation theory, described in the TIP database thus- "The critical method of learning according to conversation theory is "teachback" in which one person teaches another what they have learned."(Kearsley, 1994). Although this may have arisen in more traditional forms of discussion and debate, the nature of the newsgroup meant that these thoughts were made concrete (adding to the body of knowledge) and those who may not have made an effective contribution in a traditional discussion were able to voice their thoughts in a considered manner. EE13 wrote:

"Only when I started to participate in the newsgroup my ideas about the subject began to clarify"

One student commented to me afterwards that she particularly liked the way in which the pressure was taken off for hand-in dates- that she was being assessed on her entire contribution over a period of time. She made a similar comment about the use of a web page to hand in work. A constant problem in our University (and I am sure this is true in others) is the massive amount of printing that occurs on and just before hand-in dates. All the students had to do for this assignment was to save their page under a specific name in their web-home directories. This also made it very difficult for them to hand work in late- the pages were automatically collected by a simple batch file at the allotted time.

Technical issues

Using the newsgroup proved technically challenging to many. Netscape News (version 3.0) was shown to be lacking in a number in its defaults of not showing previously read messages and limiting the number of messages you can see at once- some people were discouraged when they thought their messages had gone missing. There was a significant amount of discussion on the newsgroup involved in fixing this problem- the mechanism was self-repairing, evolving new pieces of learning material as new learning needs arose, but it again shows that there was a need for a more thorough introduction to the technology.

Understanding of the technology was particularly lacking in the use of news generally, especially in the use of threads. This was undoubtedly due to insufficient pre-orientation, but it meant that many threads failed to grow because answers were often posted as new messages, not as replies to the group. This problem noticeably improved over the course of the two-week assignment, indicating some evolution took place (good use of threads being recognised as a successful strategy).

Many students used the University’s own dial-in facilities to access the newsgroup from home, and several figured out how to save the newsgroup discussion on disk to peruse at their leisure. This was particularly valuable for the part-time students (around 10% of the cohort). However, due to the use of an internal or private newsgroup, those trying to access the newsgroup through their own Internet Service Providers could only manage to get to the University's news server by using Telnet to reach the main Unix machines then struggling with the deeply off-putting interface of Tin (which no-one liked).

From my own point of view as the assessor it was logistically difficult to mark the assignment, but by using two computers (one to view the newsgroup the other to view the web page) it seldom took more than 10-15 minutes per student, which is on a par with a standard written assignment. In future, I would like to involve the students more in assessing each other’s contributions, a process made considerably simpler by being on-line. It is also possible that they could be marked at least partly by their success or failure within an evolutionary environment.

A number of valuable lessons have come out of this exercise. Firstly, the newsgroup has proved to be a powerful tool in aiding learning, lending credence to Pask’s Conversation theory- the effectiveness of teachback and the resulting variation of styles catering for holist and serialist learners being clearly demonstrated. It would be impossible to have conducted this kind of discussion on this scale using traditional methods and the quality of the results combined with the comments of the students shows that the results were worthwhile.

Key benefits include:

• Variety of perspectives- different students could learn through different paths.
• No time restrictions- students could contribute to the discussion at any time (asynchronous).
• Little restriction on place of work (asyntopic).
• Quantity of responses- 100 students produced over 250 messages in two weeks.
• Quality of responses- after a slowish start, there was a highly perceptible improvement in the quality of the discussion.
• Lack of pressure- without the pressure of a particular hand-in date (although with the pressure of having to contribute something within a relatively short period) the students could concentrate on learning.

Key problems include:

• Coming to terms with the technology- use of newsreaders and learning the conventions of newsgroup use.
• Over-emotional reactions and postings
• Technical constraints (e.g. access from other ISPs)

There is an over-riding lesson that more orientation and preparation is needed so that the students can come to terms with the newsgroup environment prior to attempting an exercise of this sort. Allied to this, two weeks was not really long enough to allow sufficient growth- next time I shall conduct the experiment over a longer period.

On the effectiveness of the newsgroup in encouraging evolution

The newsgroup has been shown to be an imperfect mechanism for evolving learning. The newsgroup notably fails in providing adequate competition and in preferentially favouring the successful. As a form of speciation, the thread leaves a lot to be desired as it does not sufficiently embody values and preferences. The most interesting conclusions relate to what is implied for the new environments that will be designed specifically to encourage evolution. Significant questions leading to that environment that have been raised by this assignment include:

• Should the discussion be private (within a group) or public?
• How can cross-fertilisation be formalised and encouraged whilst making sure that the discussion is not stunted by inappropriate levels of discussion?
• How can peer-assessment be used effectively within this context? Might it be possible to judge the success of a student by the same criteria that make their work succeed or fail within the environment?
• What is the optimal length for contributions? Can this be discovered through the use of the same evolutionary mechanism that decides success or failure? For instance, where feedback is in the form of survival or extinction, will we see a natural move towards an optimal style or styles?
• How can we ensure speciation and diversity? We do not wish to see a single successful strategy, but a variety of niches filled in a manner suitable for all participants, no matter what their preferred learning style.
• How can less effective offerings be rendered extinct? More importantly, how can success be rewarded?

The next step

I am currently working on ways of identifying answers to these questions by designing a slightly modified newsgroup environment with the following features:

• votes on the usefulness of particular messages are registered as part of the subject lines of the original messages. (so that we should see preferences shaping the success or failure of particular threads and learning materials)
• a mechanism to weed out less useful contributions, automatically expiring them after enough negative votes.
• a simple self-imposed categorisation of the postings, making it easier to wade through the mass of messages. Ideally, these categories should themselves evolve according to the needs of the participants, perhaps contributing to speciation.

I will deal with some of the factors that inhibit use of the newsgroup, by simplifying access via a web browser and thus removing problems associated with access and newsreader-specific difficulties.

Cox, Brad, 1997, Evolving a Distributed Learning Community, in The Online Classroom in K12, ed. Berge and Colins, Hampton Press

Tait, Bill, 1997, Constructive Internet based learning, Active Learning Number 7, December 1997

Calvin, William H, 1997, The Six Essentials? Minimal Requirements for the Darwinian Bootstrapping of Quality, Journal of Memetics 1

Dennett, Daniel, 1995, Darwin’s Dangerous Idea, Penguin

Ackerman, M., & Malone, T. 1990 Answer Garden: a tool for growing organisational memory, Proceedings of the Conference on Office Information Systems, ACM Press

Boder, A., 1992, The Process of knowledge reification in human-human interaction, Journal of Computer Assisted Learning ( 8,3)

Funderstanding, 1998, About Learning, http://www.funderstanding.com/about_learning.html visited 2/4/98

Ackerman, Mark S. & McDonald, David W., 1996, Answer Garden 2: Merging Organizational Memory with Collaborative Help, Proceedings of the ACM Conference on Computer-Supported Cooperative Work (CSCW'96), November 1996, pp. 97-105, also online at http://www.ics.uci.edu/~ackerman/docs/cscw96.ag2/cscw96.ag2.html, visited April 5th 1998

Kearsley, Greg , 1994, The TIP database, http://www.lincoln.ac.nz/educ/tip/1.htm visited April 7th 1998

Pask, G., 1976, Conversation Theory- applications in education and epistemology, Elsevier.