The human face is an emotional âhot spotâ. New-born babies seem to gravitate spontaneously towards the face of the caretaker. The human face is an emotionally expressive display that is more than the sum of its parts. The face forms a total configuration that manifests a personâs humanity in a way especially engaging to another person. We humans seem to be hard-wired to interact with faces as the location for emotional expression â and the lack of expression. Freud famously said that âbetrayal oozes at every pore.â Though Freudâs quip is not about his patient Doraâs facial expression as such, his slogan applies to the face in ways he would have appreciated and which were being explored by Darwin (1871) when Freud was still only fifteen years old. This is where the game gets interesting.
Innovations in computing hardware power, social networking, neural networks, and pattern recognition, are advancing the automated understanding of the expression of human facial emotions. Enter Affectiva (www.Affectiva.com), which, as the saying goes, is disrupting the disrupters. Affectiva was founded in 2009 by Rana el Kaliouby and Rosalind W. Picard, scientists at the MIT Media Lab. In a conversation with Daniel McDuff, Ph.D., principal scientist at Affectiva (www.affectiva.com), I had an opportunity to learn how innovations in the computer-mediated assessment of the emotions are being implemented and brought to market in a variety of applications in advertising and media measurement (Affectivaâs current chosen market), law enforcement, and engaging diseases of empathy. One innovation that Dr McDuff brought to Affectiva from his work at the MIT Media Lab was the use of webcams to collect facial data from persons providing informed consent. This has enabled Affectiva to build a Big Data database of facial expressions to power the processing of its software algorithms. Simply stated, the output of the software process is as assessment of the individualâs emotional experience along a number of dimensions and variables. The devil is in the details. Layers of technology go into Affectivaâs Affdex system to capture and feed information to its algorithms. The system tracks and processes the texture of the human face, performing a complicated mapping to minute facial muscle movements described by the Facial Action Coding System (FACS), resulting in inferences about the categorization and intensity of emotional engagement, valence, and related nuances of affect. As with any software system, issues of ease of use, accessibility and flexibility of the human-machine interface, scalability, maintainability, and end-to-end system integration are front and center. This is where Affectiva seems to have stolen a march on the competition with the use of small computer-based webcams to capture data that is then stored to a Big Data backend. Webcams are pervasive. The potential amount of data is formidable. I have been known to say: âWe donât need more data; we need expanded empathy.â However, sometimes we need both. This seems to be one of those occasions. The deep background to Affectivaâs work is to be found in Paul Ekman (1992) and his colleague Wallace Friesen, who themselves relied on the researches of Charles Darwin (1871) and Silvan S. Tomkins (1961, 1962, 1992, 1993) and Duchenne de Boulogne (1862). In an enormous research effort lasting some eight years, Ekman led a team that coded some 5000 detailed movements of muscles in the face that are activated, in many cases involuntarily, in the arousal of some seven basic emotions. This Facial Action Coding Scheme becomes the basis for software automation. Whatâs so innovative about that? Well, anyone can try to fake a smile, pretending to be happy when one is really miserable. But what one cannot fake is activation of the âsmile musclesâ around oneâs eyes, which are only engaged by an authentic and positive emotion that expresses oneâs sincere delight and which remain uninvolved in an insincere baring of oneâs teeth. Unlike oneâs lips, which can be voluntarily displayed in a grimace, the muscles around the eyes are not subject to voluntary control. Hence, the opportunity exists for âbetrayal to ooze out at every poreâ. Moreover, the activation of such a muscle can occur and vanish in a fraction of a second. It moves rapidly across the face at a speed that lies beneath the threshold of oneâs ability to see it without significantly slowing down the digital recording and playing it back. Yet the emotion occurs, however briefly. It lives. Such a detailed, minute muscle activation is called a âmicro expressionâ. Micro expressions are hypothesized to be the basis for the enigmatic smile of Leonardo da Vinciâs painting of the Mona Lisa. In our time, Ekmanâs facial action coding scheme of micro expressions becomes the basis for detecting deceit in the marketplace, politics and marriage (which, incidentally, is the subtitle of Ekmanâs Telling Lies (1992)). In some cases, the marketplace hype is justified, and, in this case, caused me to chase the ârumor of empathyâ. Ekman is on record as saying that he is skeptical about empathy, and I do not aim to change that here. In any case, Ekman identifies âduping delightâ as the micro expression of happiness of the liar at having âput one overâ on the teacher with the deception of being believed that the âdog really did eat the homeworkâ. Or in law enforcement, the micro expression of contempt on the otherwise emotionless face of the would-be terrorist at striking back at the ârunning dogs of western imperialismâ. What would the detection of such micro expressions be if not a subtle example of empathy or, more precisely, empathic receptivity? Before Ekman â and even before Darwin â the philosopher David Hume wrote of a âdelicacy of sympathyâ (1741), in detecting an impression of which another person was unaware. The word âempathyâ had not yet been invented. Close enough. Take aways include: No market, no mission: Affectiva has traction in the advertising and marketing verticals. While all the usual disclaimers apply, and I have not âtest drivenâ the Affdex system, Affectiva seems to be well on its way to integrating its facial recognition algorithm(s) in a comprehensive end-to-end automated process that incudes a user friendly frontend (webcam) and big data backend. The intellectual property is relevant, and patent the algorithms. But absent an integrated, usable approach, it is going to be an idle wheel that does not move any other part of the business process. While a strong start is no guarantee of long term success, Affectiva has innovative technology and a compelling message that resonates with corporate needs to spend money on advertising that delivers demonstrable bang for the buck.
An implied definition of empathy: An account of empathy exists here based on micro expressions, which is what inspired my interest. Though the debate about the relevance of empathy continues, empathy is hypothesized to be at the basis of the human ability (1) to relate emotionally to other persons like oneself (2) to experience other persons as intentional agents (3) to attribute a mind such as oneâs own â as in âmindednessâ â to other persons like oneself. Empathy is not reducible to emotional contagion, shared-joint attention or mindedness; but at least the first two are input for further empathic understanding, empathic interpretation, and empathic responsiveness that enriches a personâs relations with other human beings. Absent empathy, people cease to matter to the person lacking empathy, though people may be useful in certain means-ends way of providing services. In disorders of empathy, one or more of these mechanisms has misfired or is hypothesized to be missing. The individual âon the autism spectrumâ seems to be unaware of the emotions, intentions or mindedness of other people. The subsequent breakdowns in human development, education, and day-to-day functioning are debilitating in the extreme and can even be life threatening. People and computer systems can produce similar results and output using profoundly different means and methods. Though it is improbable that the Affdex software arrives at its conclusions about what people are experiencing emotionally in the same way that the human brain arrives at its results, the possibilities for comparison are significant. A comparison between the steps of human brain-based empathy and the artificial empath implemented in software may reveal what can go wrong and suggest meaningful interventions. Finally, no substitute exists for an expert clinical differential diagnosis by an informed human â though hope springs eternal in the matter of eliminating focus groups in marketing â but the use of software to detect and analyze micro expressions â or their absence â can be a significant check and balance in the diagnostic process. Empathy is still on the short list of those things where humans enjoy a decisive advantage over automated systems. Still, it is sobering the way the boundary keeps getting pushed around as humans are beaten by computing systems in chess, natural language processing in Jeopardy, and now challenged by decoding facial emotions. In any case, the combination of human judgment and a software system acting as a kind of âco-pilotâ to the decision-making human is a compelling partnership. In summary, the rumor of empathy at Affectiva is confirmed. Empathy lives at Affectiva. I hasten to add that âa rumor of empathyâ is my turn of phrase â my spin â and not a description employed by Affectiva, though I suggest that the distinction is an apt one.
Relevant URLs
www.Affectiva.com
