Evaluation of filters on private email collections will be carried out by the organizers. Participants who wish to take part in this task must implement the TREC spam-filter evaluation interface and submit their filters for evaluation. To preserve privacy, filters will not be allowed to use network resources for this task. All filters submitted for this task will also be evaluated on a copy of the live challenge e-mail stream. This task was conceived as a continuation of the TREC spam track effort, and we encourage TREC participants to submit existing systems for the evaluation.
In both tasks, filters will be required to provide a Ham/Spam classification for each message and an optional numeric score reflecting the filter's confidence in the prediction. Filters will be evaluated using two measures which combine the percentage of spam blocked and its false positive rate:
The challenge will be organized within the framework of the Fifth Annual Conference on Email and Anti-Spam (CEAS). Participants will be required to provide a short (minimum 1 page) description of their system(s), to be included in the CEAS proceedings. Conference registration is not required for participation. Participants are however strongly encouraged to attend CEAS, which will be held in Mountain View, California, on August 21st and August 22nd, 2008. A Spam Challenge poster session will be held at the conference venue. Selected participants will also be given the opportunity for oral presentation of their systems.
Competition details, rules, scope and limitations will be announced at http://www.ceas.cc/challenge and the competition mailing list. To join the competition mailing list, fill out the form at http://eight.pairlist.net/mailman/listinfo/ceas-challenge or send an email request to information@ceas.cc.