Polina Kuznetsova, Stony Brook University

Date: Fri Nov 8, 2013 
Time: 2:15pm 
Venue: Graduate Center C-415A (basement level)

Title: Generation of Natural Language Descriptions for Images


Generation of human-like and relevant descriptions for images is a challenging
task. We present an approach for collective generation of natural image
descriptions. Our method exploits preexisting human written captions in order to
obtain plausible output. We cast the generation process as an Integer Linear
Programming (ILP) task. The proposed ILP formulation combines parts of the
associated preexisting captions into a new description for the target image.

However, extraneous information present in human text can hurt the relevance of
the generated descriptions. Thus, we introduce a new task of image caption
generalization. We exploit Dynamic Programming (DP) to remove information which
is less likely to be visually verifiable. In order to filter grammatically
incorrect solutions we enhance DP with typed dependency constraints. Resulting
generalized captions are potentially more applicable to the generation of image

Short Bio:

Polina Kuznetsova is a PhD Candidate in Computer Science at Stony Brook
University. She received her BS in Computer Science and Engineering at Moscow
Engineering Physics Institute and her MS in Computer Science at SUNY New Paltz.
Her research primarily focuses on Natural Language Generation, in particular,
Image Description Generation.

More info at: http://www.cs.sunysb.edu/~pkuznetsova/