Time: 11am-12:30pm, August 1, Thursday Place: Room 6495, CUNY Graduate Center (365 5th Ave, between 34str. & 35str) Speaker: Xiaoqiang Luo(email@example.com), Senior Research Scientist, Google Inc. Title: Two Decades of Coreference Resolution Evaluation Abstract: A high quality coreference resolution system are useful in many language-related applications such as information extraction, question answering, and spoken dialog systems. The community have made tremendous progress in last two decades in improving the accuracy of coreference resolution systems; however, the answer to one basic question remains elusive: how to measure the quality of a coreference system? In this talk, I will review the evaluation metrics proposed in last 2 decades, and highlight the difficulties when devising a goodness criterion that can rank coreference systems in agreement with human judgments, and that are tampering-proof ( i.e., a player cannot game the metric). I will show that some recent developments, where misinformation is not only being spread in the community, but also being put into practice, have started to hinder the progress of this sub-field, and the community urgently need to reach a consensus on a common evaluation metric based on a few guiding-principles. Speaker bio: Xiaoqiang Luo is a senior research scientist of Google Inc. Prior to joining Google in April, 2013, he was a Research Staff Member at IBM T.J. Watson Research Center. He received his Ph.D and Master in electrical engineering from The John Hopkins University in 1999 and 1995, respectively, and Bachelor degree in electrical engineering from University of Science and Technology of China in 1990. Dr. Luo has worked in a broad area of human language technologies: speech recognition and spoken dialog system, information extraction, coreference resolution, statistical syntactic and semantic parsing of natural languages, information retrieval and question answering. He is a major contributor to IBM's successes in many research projects sponsored by DARPA (DARPA Communicator, ACE, GALE/distillation and BOLT). Dr. Luo served as co-chair of the NLP professional interest group (PIC) at IBM. T.J. Watson Research Center in 2013. From 2007 to 2010, he was an Associate Editor from ACM Transactions on Asian Language Information Processing (TALIP). He received the IBM Research Division Award in 2011 for his contributions to IBM's statistical information and relation extraction (SIRE) technology, IBM's prestigious Outstanding Technical Achievement Award (OTAA) in 2007, ThinkPlace Bravo Award in 2006, and numerous IBM Invention Achievement Awards.