Comments

You must log in or register to comment.

masque wrote (edited )

Early in the article, we get this comment about problems with previous work:

One common task is to arm people with a small, preselected set of emotion words (such as ‘anger’ or ‘sadness’) and to ask them to label posed, disembodied, contextless faces (such as a person smiling) with the word that they think best describes the emotion on each face. This method, when compared with others, has been consistently shown to inflate support for the universality hypothesis.

But then, when describing the AI-based approach:

The DNN learnt from human evaluators (‘raters’), who annotated the facial movements contained in each video clip by choosing from a set of English words describing emotion that were provided by the authors.

The AI here is basically just being used to produce approximate human annotations for a larger, more-diverse set of videos compared to previous studies. It'll share most of the same methodological concerns as previous works (plus introducing some noise due to classification error in the trained model), although the larger data set & ability to consider a wide range of contexts is probably a meaningful contribution? But the AI is not being fundamentally more "objective" than previous studies, or anything like that.

4

existential1 OP wrote

Hard agree. A lot of science articles are guilty of implying black-box AI when it isn't the case of what happened methodologically.

3

masque wrote (edited )

The more I think about it, the more it seems like "AI weighs in..." is just a straight-up inappropriate title.

I mean, the experiment is basically just

  • annotate a massive number of videos according to perceived facial expression and context
  • do statistical analysis to check whether perceived facial expression is appropriately correlated with emotionally-charged contexts (toys etc.), and whether these correlations are preserved across cultures.

The fact that the annotation was carried out by a DNN mimicking humans, instead of directly by humans, is irrelevant to the interpretation of the results (except that it introduces more noise) and it shouldn't be mentioned in the headline at all.

The actual paper is titled "Sixteen facial expressions occur in similar contexts worldwide" which is a much more appropriate title.

4

existential1 OP wrote

Yeah, that's the difference between "scientific article" and "scientific reporting" i guess.

2