Rebecca L. Dubowy, BA,
Charlotte Bell, MD, Keith Ruskin, MD, Rene Hagenouw1, MD
Department of
Anesthesiology, Yale University School of Medicine, New Haven, CT
1Erasmus University,
Rotterdam, The Netherlands
A
three-step methodology consisting of usability studies, site redesign and log
file analysis was used to improve user interactivity at an anesthesiology
educational web site. Average user session length increased by 27% after
application of this system.
Introduction
GASNet, the Global Anesthesiology
Server Network, was established 5 years ago by a group of
anesthesiologist-computer scientists with the goal of developing Internet
technology for use in medical education. As usage of the Internet has broadened
to include a vast audience with a diverse range of technical expertise,
accessibility and usability of the site's content has become an important
aspect of ongoing site development. As it is not feasible to study the
information-seeking behavior and user preferences of the entire GASNet audience, it became imperative to
develop a methodology to study usability in a small targeted group that could
then be applied to the larger audience.
A step-wise evaluation of site usability was performed
using a methodology that combines a qualitative
and quantitative analysis of the user interface.
Step I: A group of 5 medical
students with no prior experience with GASNet
performed a series of defined tasks on the web site. Technical errors were identified and acceptance or rejection of
design features was quantified at
the end of each task (usability studies).
These tasks required several different modes of information seeking
including using the search engine, using the Table of Contents (TOC) and navigating
from the top-level menu bar.
Step II: Technical errors
identified were corrected and areas of poor usability were redesigned.
Step III: Log file analysis
(WebTrends, www.webtrends.com) was performed to quantify user activity on the
web site including number of hits and user sessions, site entry and exit points
and number of individual page hits.
Statistical analysis is limited to report of frequency.
I.
Qualitative usability studies indicated that a) users most commonly started
navigation with the top-level menu bar; b) users preferred the TOC for
navigation once its link was identified by scrolling down on the home page; c)
more user errors were present when navigating with the search engine than with
the TOC.
II.
Changes made to the site over a 60-day period after the usability studies
included moving the TOC proximally on the home page and correcting technical
and design usability problems.
III.
Results of log file analysis (below) indicate a consistent increase in average
user session length (27%) and more frequent use of the TOC after site
improvements.
When
|
Avg# user hits/day |
Avg # user sessions/day |
Avg user session length |
Position of TOC * |
Position of Search * |
|
Pre-usability |
30,425 |
1913 |
11:39 |
6 |
3 |
|
60-90 days post |
30,695 |
1884 |
12:25 |
5 |
3 |
|
90-120 days post |
31,400 |
1872 |
14:51 |
4 |
3 |
*Position of TOC or search
in list of most requested pages.
Conclusions
Research
in information seeking behavior indicates that discrete sociological groups
with common interests and goals seek information in a similar fashion (1).
Therefore, it should be possible to project the results of a usability study
performed with small populations onto a larger virtual audience (i.e. the
Internet). This study indicates that usability studies on small groups can
improve user interactivity with Internet-based education systems.
Reference
Allen
TJ. Managing the Flow of Technology. The MIT Press, Cambridge, MA, 1995.