Skip to main content

Expression Recognition with Deep Neural Nets (with Barbara Mones in CSE)

facial_expressionWe have developed a system called DeepExpr, a novel expression transfer approach from humans to multiple stylized characters. We first train two Convolutional Neural Networks to recognize the expression of humans and stylized characters independently. Then we utilize a transfer learning technique to learn the mapping from humans to characters to create a shared embedding feature space. This embedding also allows human expression-based image retrieval and character expression-based image retrieval. We use our perceptual model to retrieve character expressions corresponding to humans. We have evaluated our method on a set of retrieval tasks on our collected stylized character dataset of expressions. The ranking order predicted by the proposed features is highly correlated with the ranking order provided by a facial expression expert and Mechanical Turk experiments.


Research Areas