Time: 11am-12:30pm, August 1, Thursday
Place: Room 6495, CUNY Graduate Center (365 5th Ave, between 34str. & 35str)

Speaker:  Xiaoqiang Luo(xql@google.com), Senior Research Scientist, Google Inc.

Title:  Two Decades of Coreference Resolution Evaluation

A high quality coreference resolution system are useful in many
language-related applications such as information extraction,
question answering, and spoken dialog systems. The community have made
tremendous progress in last two decades in improving the accuracy
of coreference resolution systems; however, the answer to one basic
question remains elusive: how to measure the
quality of a coreference system? In this talk, I will review
the evaluation metrics proposed in last 2 decades, and highlight
the difficulties when devising a goodness criterion that can rank coreference
systems in agreement with human judgments, and that are tampering-proof (
i.e., a player cannot game the metric). I will show
that some recent developments, where misinformation
is not only being spread in the community, but also being put into
practice, have started to hinder the progress of this sub-field,
and the community urgently need to reach a consensus on a common evaluation
metric based on a few guiding-principles.

Speaker bio:

Xiaoqiang Luo is a senior research scientist of Google Inc. Prior to joining
Google in April, 2013, he was a Research Staff Member at IBM T.J. Watson
Research Center. He received his Ph.D and Master in electrical engineering
from The John Hopkins University in 1999 and 1995, respectively, and
Bachelor degree in electrical engineering from University of Science and
Technology of China in 1990.

Dr. Luo has worked in a broad area of human language technologies: speech
recognition and spoken dialog system, information extraction, coreference
resolution, statistical syntactic and semantic parsing of natural languages,
information retrieval and question answering. He is a major contributor to
IBM's successes in many research projects sponsored by DARPA (DARPA
Communicator, ACE, GALE/distillation and BOLT).

Dr. Luo served as co-chair of the NLP professional interest group (PIC) at
IBM. T.J. Watson Research Center in 2013. From 2007 to 2010, he was
an Associate Editor from ACM Transactions on Asian Language Information
Processing (TALIP). He received the IBM Research Division Award in 2011
for his contributions to IBM's statistical information and relation
extraction (SIRE) technology, IBM's prestigious Outstanding Technical
Achievement Award (OTAA) in 2007, ThinkPlace Bravo Award in 2006,
and numerous IBM Invention Achievement Awards.