[Apologies for cross-posting]
CALL FOR PAPERS
INFORMATION PROCESSING & MANAGEMENT: An International
Journal
Special Topic Issue on
User-centred Evaluation of Information Retrieval Systems
http://www.cis.st
rath.ac.uk/~ir/ipm
Guest Editors
Pia Borlund, Royal School of Library and Information
Science, Aalborg,
Denmark.
Ian Ruthven, University of Strathclyde, Glasgow, United
Kingdom
Introduction to special issue topic
Evaluation has always been a strong element of Information
Retrieval
(IR) research, much of our focus being on how we evaluate IR
algorithms.
As a research field we have benefited greatly from
initiatives such as
Cranfield, TREC, CLEF and INEX that have added to our
knowledge of how
to create test collections, the reliability of system-based
evaluation
criteria and our understanding of how to interpret the
results of an
algorithmic evaluation. In contrast, evaluations whose main
focus is
the user experience of searching using IR systems have not
yet reached
the same level of maturity. Such evaluations are complex to
create and
assess due to the increased number of variables to
incorporate within
the study, the lack of standard tools available (for
example, test
collections) and the difficulty of selecting appropriate
evaluation
criteria for study.
In spite of the complicated nature of user-centred
evaluations, this
form of evaluation is necessary to understand the
effectiveness of
individual IR systems and user search interactions. The
growing
incorporation of users into the evaluation process reflects
the changing
nature of IR within society; for example, more and more
people have
access to IR systems through Internet search engines but
have little
training or guidance in how to use these systems
effectively. Similarly,
new types of search system such as recommender systems and
interactive
IR facilities are becoming available to wide groups of
end-users.
This special topic issue will address the current and future
position of
user-centred evaluation within IR and reflect the increased
interest in,
use of, and need for user-centred evaluations. The special
topic issue
will act as a focus for dissemination in best practice in
the area of
user-centred evaluations by increasing our understanding of
appropriate
methodologies, our awareness of the effectiveness of
evaluation measures
and in raising new research directions in the user side of
IR.
Submissions
Submissions for the special topic issue should be
high-quality
manuscripts on the theoretical, empirical and methodological
issues
surrounding user-centred evaluations. Submissions that
describe
evaluations will be welcome as long as they contribute to
our
understanding of how to perform a user evaluation.
Submissions should
describe original research and not be under consideration in
any other
forum. All submissions should be formatted according to the
Information
Processing & Management guidelines for papers,
references, and citations
and should be submitted using the electronic submission
site. Submitted
articles will be reviewed according to the Information
Processing &
Management reviewing criteria and appropriateness to the
special topic
issue. Full details of paper submission and reviewing are
available at
http://www.cis.st
rath.ac.uk/~ir/ipm
We particularly welcome submissions on, but not limited to,
the
following aspects of user evaluations:
• Methodologies for user-centred evaluations
• Evaluation measures for user-centred evaluations
• Theoretical issues in user-centred evaluations
• User-centred test collections
• Evaluations of user interfaces and interactive IR systems
• User models and evaluation
• Evaluations of non-traditional information access systems
Important dates
Submission of articles: Friday 7th April 2006
Notification to authors: Friday 30th June 2006
Final version of accepted papers: Friday 11th August 2006
All questions regarding submissions should be directed to
the
specialtopic issue Guest Editors: Pia Borlund (pb at db.dk)
and Ian
Ruthven (ir at cis.strath.ac.uk)
--
Ian Ruthven
Department of Computer and Information Sciences
University of Strathclyde
Glasgow G1 1XH
Email: Ian.Ruthven cis.strath.ac.uk
Tel: +44 141 548 3098
Fax: +44 141 548 4523
http://www.cis.strath
.ac.uk/~ir
Information Interaction in Context (IIiX) - http://www.db.dk/IIiX/
____
________________________________________
Asis-l mailing list
Asis-lasis.org
http://m
ail.asis.org/mailman/listinfo/asis-l
|