This is a continuation of a study I introduced in Part 1. In this series of blogs I’m presenting the results of a
research project to measure the effectiveness of two mailing lists.

How many messages were on each thread?

Figure 1 shows the distribution of messages across all 28 threads. The
distribution follows a typical power law, with many low-count threads
at the left end (8 threads containing only one message, for instance)
and a few high-count ones on the right. The high-count threads were
almost always created by a high volume of irrelevant messages.
Correspondents went off on tangents such as whether a volunteer
free-software developer should respond to error reports from users.

Figure 1. Number of messages per thread

Figures 2 and 3 show a more informative breakdown: the number of
messages on resolved threads versus unresolved threads. Once again,
threads at the right of each chart are skewed by the preponderance of
irrelevant messages.

Figure 2. Number of messages on resolved threads	Figure 3. Number of messages on unresolved threads

One interesting result is the similarity between the charts; the
numbers of messages on resolved and unresolved threads are
comparable. I expected the volume of messages to be much higher on
unresolved threads. This is because I compared the search for answers
to a problem to searches in computer software, as well as to real-life
searches (such as the unsuccessful search I made in my basement this
morning for a hack-saw). The worst-case time is always taken by a
search that fails.

However, it seems that the mailing lists I examined are efficient. If
the list members can solve a problem, they provide the answer
quickly. If they can’t solve it, they abandon the effort. While this
is efficient, it’s possible that more threads would be resolved if
list members tried harder. This is particularly pertinent because 8
out of the 28 threads had only one message. In other words, nearly
one-third of all questions elicited absolutely no response.

One of my colleagues said that members of Linux-related lists used to
be more helpful during the early days of Linux, when mere use of the
operating system was considered exciting. He sensed that the
commercialization of Linux has caused some of the expert users to lose
interest in helping new users. But the lists remain quite
high-volume. Whatever the cause of the unanswered messages, the
results should be checked by further surveys on other mailing lists.

In the conclusion to this article, I’ll talk about whether a more
exploratory and experimental approach would benefit the users of the
list.

How long did it take to resolve questions?

Figure 4 shows the time it took to resolve each of the 14 resolved
threads. If a single message provided the answer, I measured the
resolution time from the time the question was posted till the time
the first correct answer was posted. The resolution times for a few
threads were less clear, because several people contributed ideas and
the original correspondent eventually responded that he or she had
solved the problem; I took this message as the resolution
time. Finally, some threads included multiple responses that seemed
valuable, but no resolution was reported; here I considered the thread
resolved but took the time of the last helpful response as the
resolution time.

Figure 4. Resolution time

Resolution times varied widely, and results at the high end are
probably not very meaningful because I had to guess at many
resolutions. The minimum resolution time was 8 minutes, the maximum
was 2 days, 10 hours, and the median was 9 hours, 45 minutes. (The
median is probably a more accurate average than the mean because the
threads with the longest response times may be measured incorrectly
for reasons I gave earlier.)

I have heard that resolution time is better on other lists that cover
narrower topics. The median in this case is pretty impressive for an
interactive forum that spans the world’s continents. But if the person
posing the question had documentation that answered the question, the
answer could probably be found in half an hour or so. Of course, the
questioner can perform other tasks in the time it takes to wait for
an answer, but conversely, his or her use of the mailing list requires
other people to spend time on the problem. So the results here suggest
that mailing lists are rather inefficient compared to high-quality
documentation.

(To be continued.)