Dr. Ellen Voorhees from TREC at NIST.gov sent me their most recent Call For Participation. To help with its dissemination, I am posting it in its entirety, so it will reach out through  IRThoughts and Mi Islita.com  wider audience.

              CALL FOR PARTICIPATION

                    TEXT RETRIEVAL CONFERENCE (TREC)

                      February 2010 – November 2010

                          Conducted by:
      National Institute of Standards and Technology (NIST)

The Text Retrieval Conference (TREC) workshop series encourages
research in information retrieval and related applications by
providing a large test collection, uniform scoring procedures,
and a forum for organizations interested in comparing their
results.  Now in its nineteenth year, the conference has become
the major experimental effort in the field.  Participants in
the previous TREC conferences have examined a wide variety
of retrieval techniques and retrieval environments,
including cross-language retrieval, retrieval of web documents,
multimedia retrieval, and question answering.  Details about TREC
can be found at the TREC web site,  http://trec.nist.gov.

You are invited to participate in TREC 2010.  TREC 2010 will
consist of a set of tasks known as “tracks”.  Each track focuses
on a particular subproblem or variant of the retrieval task as
described below.  Organizations may choose to participate in any or
all of the tracks.  Training and test materials are available from
NIST for some tracks; other tracks will use special collections that
are available from other organizations for a fee.

Dissemination of TREC work and results other than in the (publicly
available) conference proceedings is welcomed, but the conditions of
participation specifically preclude any advertising claims based
on TREC results.  All retrieval results submitted to NIST are
published in the Proceedings and are archived on the TREC web site.
The workshop in November is open only to participating groups that
submit retrieval results for at least one track and to selected
government personnel from sponsoring agencies.

Schedule:
——–

  By February 18 — submit your application to participate in
        TREC 2010 as described below.  Submitting an application
        will add you to the active participants’ mailing list.
        On Feb 24, NIST will announce a new password for the “active
        participants” portion of the TREC web site.  Included
        in this portion of the web site is information regarding
        the permission forms needed to obtain the TREC document
        disks.

   Beginning March 2 — document disks used in some existing
        TREC collections distributed to participants who have
        returned the required forms.  Please note that no disks
        will be shipped before March 2.

   July–August  — results submission deadline for most tracks
        Specific deadlines for each track will be included in
        the track guidelines, which will be finalized in the spring.

   September 9  (estimated) — speaker proposals due at NIST.

   September 30 (estimated) — relevance judgments and individual
        evaluation scores due back to participants.

   Nov 16-19 — TREC 2010 conference at NIST in Gaithersburg, Md. USA

Task Description:
—————-

Below is a brief summary of the tasks.  Complete descriptions of
tasks performed in previous years are included in the Overview
papers in each of the TREC proceedings (in the Publications section
of the web site).

The exact definition of the tasks to be performed in each track for
TREC 2010 is still being formulated.  Track discussion takes place
on the track mailing list.  To be added to a track mailing list,
follow the instructions for contacting that mailing list as
given below.  For questions about the track, send mail to the
track coordinator (or post the question to the track mailing list
once you join).

TREC 2010 will contain seven tracks.  The blog, chemical IR,
entity, legal, relevance feedback, and web tracks will
continue from TREC 2009.  The million query track will be
incorporated into the web track.  TREC 2010 will also contain
a new “session” track.

Blog Track — The purpose of the blog track is to explore information
    seeking behavior in the blogosphere.

    Track coordinators: Craig Macdonald, Iadh Ounis, Ian Soboroff
                        trecblog-organisers@dcs.gla.ac.uk
    Mailing list:  send a mail message to listproc@nist.gov
        such that the body consists of the line
        subscribe trec-blog <FirstName> <LastName>

Chemical IR Track — The goal of the chemical IR track is to develop
    and evaluate technology for large scale search in chemical
    documents including academic papers and patents to better
    meet the needs of professional searchers: specifically patent
    searchers and chemists.

    Track co-ordinators: John Tait, john.tait@ir-facility.org
                         Jimmy Huang, jhuang@yorku.ca
                         Jianhan Zhu, j.zhu@adastral.ucl.ac.uk
                         Mhai Lupu, m.lupu@ir-facility.org

    Track Web Page: http://www.ir-facility.org/the_irf/trec_chem.htm
    Mailing List: follow the link on the web page to join the list

Entity Track — The overall aim of this track is to perform
    entity-related search on Web data.  These search tasks
    (such as finding entities and properties of entities) address
    common information needs that are not that well modeled as
    ad hoc document search.

    Track coordinators: Krisztian Balog, k.balog@uva.nl
                        Paul Thomas, Paul.Thomas@csiro.au
                        Arjen P. de Vries, arjen@acm.org
                        Thijs Westerveld, thijs.westerveld@teezir.nl
    Track web page: http://ilps.science.uva.nl/trec-entity/
    Mailing list: visit http://groups.google.com/group/trec-entity
        to apply for membership.

Legal Track — The goal of the legal track is to develop search technology
    that meets the needs of lawyers to engage in effective discovery
    in digital document collections.

    Track coordinators: Gord Cormack, gvcormac@uwaterloo.ca
                        Maura Grossman, MRGrossman@wlrk.com
                        Bruce Hedin, bhedin@h5.com
                        Doug Oard, oard@umd.edu
    Track web page: http://trec-legal.umiacs.umd.edu
    Mailing list: contact oard@umd.edu to be added to the list.

Relevance Feedback Track — The goal of the relevance feedback track
   is to provide a framework for exploring the effects of different
   factors on the success of relevance feedback.

   Track coordinators: Chris Buckley, cabuckley@sabir.com
                       Matt Lease, ml@ischool.utexas.edu
                       Mark Smucker, msmucker@engmail.uwaterloo.ca
   Track web page:  http://groups.google.com/group/trec-relfeed
   Mailing list: follow the instructions given on the track web page
                 to join the email list

Session Track —  The Session track has two primary goals: (1) to test
        whether systems can improve their performance for a given query
        by using a previous query (and search results from the search
        session), and (2) to evaluate system performance over an entire
        query session instead of a single query.

    Track coordinators: Ben Carterette, carteret@cis.udel.edu
                        Paul Clough, p.d.clough@sheffield.ac.uk
                        Evangelos Kanoulas, ekanou@ccs.neu.edu
                        Mark Sanderson, m.sanderson@sheffield.ac.uk

    Track web page: http://ir.cis.udel.edu/sessions
    Mailing list: Use the link given on the track web page to
                  join the email list

Web Track —  The Web track explores Web-specific retrieval tasks,
    including diversity and efficiency tasks, over collections of
    up to 1 billion Web pages.
    Track coordinators: Nick Craswell, nickcr@microsoft.com
                        Charles Clarke, claclark@plg.uwaterloo.ca
    Mailing list:  send a mail message to listproc@nist.gov
        such that the body consists of the line
        subscribe trec-web <FirstName> <LastName>

Conference Format
—————–

The conference itself will be used as a forum both for presentation
of results (including failure analyses and system comparisons),
and for more lengthy system presentations describing retrieval
techniques used, experiments run using the data, and other issues
of interest to researchers in information retrieval.  As there
is a limited amount of time for these presentations, the TREC
program committee will determine which groups are asked to speak
and which groups will present in a poster session.  Groups that
are interested in having a speaking slot during the workshop
should submit a 200-300 word abstract in September describing
the experiments they performed.  The program committee will use
these abstracts to select speakers.

Data
—-
Many of the existing TREC English collections (documents, topics,
and relevance judgments) are available for training purposes and
may also be used in some of the tracks.  Parts of the training
collection (Disks 1-3) were assembled from Linguistic Data
Consortium (LDC) text, and a signed User Agreement will be required
from all participants.  The documents are an assorted collection
of newspapers, newswire, journals, and technical abstracts.
A second agreement is needed for disks (4-5).

All documents are typical of those seen in a real-world situation
(i.e. there will not be arcane vocabulary, but there may be
missing pieces of text or typographical errors).  For most tracks,
the relevance judgments against which each system’s output will be
scored will be made by experienced relevance assessors based on the
output of all TREC participants using a pooled relevance methodology.
See the Overview paper in the TREC-8 proceedings (on the TREC
web site) for a detailed discussion of pooling.
  
  
Application details:
——————–
  
Organizations wishing to participate in TREC 2010 should respond
to this call for participation by submitting an application.
Participants in previous TRECs who wish to participate
in TREC 2010 must submit a new application.

To apply, follow the instructions at
     http://ir.nist.gov/trecsubmit.open/application.html
to submit an online application.  The application system
will send an acknowledgement to the email address
supplied in the form once it has processed the form.

Any questions about conference participation should be sent
to the general TREC email address, trec@nist.gov .

Advertisements