Chi-squared: which characteristics do really matter?

Posted: July 22, 2020 | Author: | Filed under: review | Tags: , , , , , , , | Leave a comment

In our little stylometric experiments, we compared different manga in terms of their hiragana frequencies. While we were able to say how similar or different the comics are to each other, it’s hard to tell in what way precisely they are different, i.e. which hiragana differed vastly in quantity and which were more or less the same. Intuitively, we ought to be able to answer this by looking at how much the hiragana counts differ from the average, but it would be good to have a more exact measure of what it means to differ “vastly” or to be “more or less” the same. If we could identify those hiragana in which the manga are hardly any different, we could ignore them in future experiments, which would be a relief since we’re otherwise stuck with as many as ~50 hiragana to keep track of.

Enter chi-squared (also called chi-square, or χ²), which is perhaps the most widespread of several statistical tests for this purpose. I first learned about it during my Master’s, but either I forgot about it or I had never really understood it in the first place. But now that I’ve looked it up again, I found it’s actually quite simple: the idea is to not only calculate the difference between the actual (observed) and the “average” (expected) value, but to square the result and then divide it by the expected value. The squaring has the effect to make large differences stand out more, while the division makes different chi-squared values comparable.

So, the formula would be:

(observed – expected)² / expected

[You might have seen this formula with a sum sign at the beginning: when you perform a “chi-squared test”, you take the sum of all calculated values and look it up in a table to determine whether your experiment is random or not (see below). In our case, it definitely isn’t.]

Let’s take the hiragana で de as an example. In our first 100-character sample from Katsuhiro Ōtomo’s Akira (A1), で de occurred 8 times (see the chart here). In the second Akira sample (A2), it is found 3 times. In the two manga samples from Morning magazine, Miko Yasu’s Hakozume (M1) and Rito Asami’s Ichikei no karasu (M2), で de is found 6 and 7 times, respectively. Overall, there are 24 で de in those four manga samples. The sum of all hiragana in these manga samples is 435 (so it turns out I took slightly more than 100 hiragana for each sample; don’t ask me why), which means that on average, で de should occur with a frequency of 24/435 = 0.0552. In other words, roughly every 19th hiragana in any of the four manga should be a で de. For the first of the two Akira samples, A1, which consists of 112 hiragana in total, the expected value for で de is 112 * 0.0552 = 6.18, i.e. we expect to find 6 or 7 で de in A1.

There actually are 8 で de in A1. That’s a difference of 8 – 6.18 = 1.82. Squared and divided by the expected value of 6.18, this results in a chi-squared value of 0.536.

Compare this to the frequency of で de in the other Akira sample, A2, where it occurs only 3 times, i.e. much less than one would have thought. Given a hiragana total of 106 for A2, we get an expected value of 106 * 0.0552 = 5.85. Accordingly, chi-squared for で/A2 is (3 – 5.85)² / 5.85 = 1.39.

However, our aim was to compare different hiragana, so let’s also calculate the chi-squared values for し shi, which occurs 7 times in A1, 1 time in A2, and 6 times in the other two manga, so the total for し shi is 14. Chi-squared for し in A1 is (7 – (14/435)*112)² / ((14/435)*112) = 3.199 and chi-squared for し in A2 is (1 – (14/435)*106)² / (14/435)*106 = 1.705.

As you can see, the chi-squared values for し shi are higher than for で de, which means that the former hiragana contributes more to the overall difference between A1 and A2 than the latter. In other words, the usage of で de throughout Akira is close to the average, thus comparatively unremarkable and perhaps not the most relevant stylometric property.

Here’s a chart of the chi-squared values for all 51 hiragana characters that occur in the four manga samples (click to enlarge):

A dialogue excerpt from Hakozume by Miko Yasu which illustrates the above-average frequency of the hiragana character え e. With regard to our little example corpus, か ka and ん n are relatively frequent too.

One can easily see several spikes at the hiragana え e, ん n, と to and ず zu, though more important than the individual values are the sums, which are also high for お o and こ ko. These 6 hiragana alone contribute roughly 70% towards the overall sum of chi-squares! If our corpus was of a sufficient size (which it is definitely not), we could focus on these 6 hiragana in further experiments, as difference in hiragana usage among manga would be most likely connected to them.

In contrast, hiragana like び bi, く ku and に ni, with chi-square values close to zero, seem to have very little explanatory power over stylometric differences; their usage differs hardly among the four manga in question.

Of course, chi-squared can not only be applied to character counts in stylometry, but also to anything else that is countable. For instance, I recently mentioned the 1:1 gender ratio as a potential criterion for corpus building. One possible null hypothesis would be that good (or popular) comics are equally likely to be authored by men or by women. If we look at the 60 people who authored the top 10 comics from each of the last four years’ best-of lists (only counting the first-mentioned author when there are more than 3), we end up with 41 men and 19 women. This distribution isn’t quite the 30:30 we might have expected, but can it still be said to be roughly equal?

To answer this with the help of chi-squared, we calculate the two chi-squared values, one for male authors:

(41 – 30)² / 30 = 4.033

and one for female authors:

(19 – 30)² / 30 = 4.033

Now we add those two numbers together and look up the result in a table like this one. We need to use the first row as we have 1 “degree of freedom” in our essentially binary variable. There, our chi-squared sum of 8.07 lies between the p=0.01 and the p=0.001 column, meaning that the null hypothesis can be rejected with high confidence. In other words, the deviation of our sample from a 30:30 gender ratio is statistically significant. Of course, what exactly this gender bias means and where it comes from is another question.

In case all of this didn’t make any sense to you, there are many online tutorials on chi-squared which perhaps explain it better, among which I recommend this video by Paul Andersen on YouTube.

Judith Butler’s gender performativity – in comics?

Posted: March 27, 2018 | Author: | Filed under: review | Tags: , , , , , , , , , , | 1 Comment

Judith Butler’s article “Performative Acts of Gender Constitution: An Essay in Phenomenology and Feminist Theory” from 1988 (Theatre Journal 40.4, pp. 519-531) is, of course, a classic. But when it gets reduced to the buzzword of ‘gender performativity’, there’s a danger of missing all the implications of this concept, so it’s still worth reading the original article. Building primarily on Simone de Beauvoir and Maurice Merleau-Ponty, Butler’s basis in this essay is the sex/gender distinction: a gender isn’t something one is born with – it is constituted during one’s lifetime. Furthermore, Butler says, the gender binary (the idea that there are exactly two genders, male and female) is likewise artificial. As is the prevalent “system of compulsory heterosexuality” in which “‘attraction’ to the opposing sex/gender” is seen as “natural”. (The term ‘heteronormativity’ isn’t used by Butler yet.)

Butler’s groundbreaking idea is that, as the title already says, gender is consituted through acts of performance. What is a performative act? Butler doesn’t explicitly define that, but hints at a wide range of possibilities: “acts, gestures, the visual body, the clothed body, the various physical attributes usually associated with gender”. Her only example is a “transvestite” who is not further described. This transvestite challenges the binary male/female distinction by combining traditionally female clothes with physical features usually seen as male, or vice versa.

This seems to be a crucial point that often gets overlooked: performative acts aren’t only ways of speaking, dressing and moving, but “the gendered body acts its part in a culturally restricted corporeal space”. While these corporeal restrictions can be overcome to some extent through clothes, make-up and surgery, it’s important to always consider physical attributes as potential acts of gender constitution. Comics might be interesting objects to analyse from this perspective because they’re good at rendering visual performance acts but not so great at depicting movement or sound (e.g. the tone of one’s voice – though this can be used for humorous effect, as I’ve noted here).

Frank Miller’s comics have frequently been accused of misogyny and sexism, but his Rōnin (DC, 1983-84, colours by Lynn Varley) has been lauded for a strong female main character, Casey McKenna. Casey isn’t exactly a feminist role model either, but that’s not the point here – let’s look at how her gender is performed.

As head of security of a futuristic corporation, Casey wears a gender-neutral uniform that isn’t much different from those of her male subordinates. (At least for most of the first half of the comic – things get weird after that, including her clothes.) It isn’t so tight as to reveal much of her physical features as female either. Only when she stands next to them, the shape of her body appears female, with a waist-to-hip ratio usually associated with women, and the subtle curve of her chest. The pose she strikes, hip slightly thrust, might also be read as female. Her face is a combination of traditionally male (bushy eyebrows) and female (long eyelashes) features; others such as the angular jaw and full lips are more of a stylistic peculiarity of Miller’s applied to male and female characters alike. Thus Casey’s gender is indeed somewhat complex. It needs to be, if 1980s superhero comic readers were to accept her as both tough (a traditionally male trait) and sexy (traditionally female).

The gender of two other characters in Rōnin is interesting because they are not human (three if we count the rōnin himself, but… it’s complicated; if you’ve read the comic you’ll have noticed I’m trying to avoid spoilers this time): the demon Agat, and the Artificial Intelligence Virgo. Agat’s ‘natural’ form is roughly human, but without genitals. His (Agat is referred to with male pronouns) overly muscular, broad-shouldered body can be read as male though, and consequently he later assumes the form and identity of male human character.

Virgo, on the other hand – referred to with female pronouns and even jokingly called a “lady” (and, later, a “bitch”) – is “the sentient computer who commands every function of [the corporate complex,] Aquarius”. She interacts with people both through voice alone and through an audiovisual screen signal on which a blurred, ‘scanlined’ head of a maybe elderly, perhaps female person is visible, possibly wearing earrings. This character is reminiscent of the AI “Mother” in the film Alien (released only four years before Rōnin), but it’s unclear whether Virgo was programmed by humans to look that way – as a powerful and wilful AI, she might have chosen her screen persona herself.

All that being said, we haven’t even talked about the characters’ words and actions as performative acts yet. A scene highly relevant to this matter, for example, is at the end of the comic when Casey accuses the rōnin of unmanly behaviour. But I’ll leave this analysis to others.

Butler’s concept of gender performativity doesn’t explain everything about gender and sexuality, but reading comics with her theory in mind helps us to ask important questions in this context: which characters can be identified as male, female, or something else? (And which can’t?) On which criteria (i.e. performative acts) are these identifications based? Do these acts conform to a traditional gender attribution in a character or are they conflicting? Is the gender attribution of a character stable or does it change in the course of a story? Is all this likely to be the creators’ intention, or a product of their subconscious, or a reading “against the grain”? If the gender configuration in a comic is presumably intentional, what is its function in the story? And is Frank Miller really a sexist?

Index to all “[theory] – in comics?” posts on this weblog

Mako-Mori-testing Archie and JLA

Posted: February 28, 2016 | Author: | Filed under: review | Tags: , , , , , , , , , , , , , , | 1 Comment

The other day I learned through the Comix-Scholars mailing list of the “Mako Mori test”. Devised as an alternative to the Bechdel test and named after a Pacific Rim character, it works like this (quoted from The Daily Dot):

The Mako Mori test is passed if the movie has:

a) at least one female character;

b) who gets her own narrative arc;

c) that is not about supporting a man’s story.

If the Bechdel test isn’t always easy to put into practice, the Mako Mori test is almost unusable. How do we define a narrative arc? How do we define “supporting a man’s story”?

Let’s say a narrative arc in a film is similar to a dramatic arc in drama theory, in that the story builds up to a conflict involving the protagonist (which must be female to pass the Mako Mori test) and then this conflict is resolved. This is already a much simpler definition of an arc than e.g. Gustav Freytag’s five-part model, so it should be easy to find stories – maybe even in comics – that match the Mako Mori test criteria, right?

While a comic may pass the Bechdel test in its first panel, it takes a lot more panels until one can say whether the Mako Mori test is passed. This makes it difficult to apply the Mako Mori test to ongoing serialised comics such as most webcomics, newspaper strips, and monthly comic books. Then again, nowadays comic book writers usually “write for the trade”, so that 6 (give or take 1-2) consecutive issues can be collected in a trade paperback that stands well on its own, in the sense that it contains a complete, self-contained story arc.

There are two ongoing comic book series I’m currently reading that have just reached their sixth issue, so let’s see how they fare in the Mako Mori test.

In the recently re-launched Archie by Mark Waid there are two candidates for female characters with their own narrative arc, Betty Cooper and Veronica Lodge. It’s difficult to place Betty in a narrative structure because of Waid’s unusual storytelling: the big crisis – Archie’s and Betty’s breakup – has already happened in the past, and the comic is about how everyone deals with the aftermath. From the beginning, Betty doesn’t want to get back together with Archie, but still has feelings for him and doesn’t approve of his new relationship with Veronica. Throughout the six issues, nothing happens that changes Betty’s attitude, so I don’t see a narrative arc here. I’m not saying a character needs to undergo drastic changes to constitute a narrative arc, but there’s simply no turning point or climax that the events involving Betty build up to.

Betty's situation summed up in two panels. Art by Fiona Staples

Betty’s situation summed up in two panels. Art by Fiona Staples

Likewise, there’s no arc structure around Veronica: she faces some challenges when she comes to Riverdale, but a new status quo (dating Archie and more or less fitting in with her other schoolmates) is quickly established. At the end of the sixth issue, a conflict involving her father is set up, but we’ll have to wait and see if this conflict qualifies as a climax of her narrative arc.

Even if Betty and/or Veronica got their own narrative arcs, one would be hard-pressed to argue that these arcs do not support a man’s story and thus fulfil the third Mako Mori criterion. Archie is first and foremost the story of its title character (who even sometimes acts as first-person narrator) and everything that happens is related to him.

JLA by Bryan Hitch is technically already at issue #7, but #5 was a filler issue completely unrelated to the actual story. In the beginning, the Justice League of America fights a supervillain called Parasite, only to be scattered across place/time/dimensions. Then the main plot begins with the Kryptonian god Rao coming to Earth. Meanwhile, Wonder Woman – the eternal token woman in the Justice League, if you will – finds herself on Olympus, which lies in ruins and has been abandoned by the other gods. Somehow Aquaman ends up on Olympus too, and the two fight off an attack by Rao’s prophets. Wonder Woman then rebuilds Olympus and arms Aquaman and herself with the weapons of the gods.

Soon she will be joined by a male companion, though.

Soon she will be joined by a male companion, though.

It’s too early to say how the final conflict will be resolved and who Wonder Woman is actually up against. Wonder Woman thinks it’s not Rao but “something else”, “something that terrified [the other gods]” (#7). But I still wouldn’t rule out the possibility that Rao and Superman are going to be at the center of this conflict, and if Wonder Woman should merely come to Superman’s aid, her narrative arc would be supporting a man’s story.

However, even though (or precisely because) Wonder Woman doesn’t get much on-panel time in this comic, her portion of the story runs parallel to the others and thus forms something like an independent narrative arc. If we had to pick a “winner”, the “Mako Mori award” would surprisingly go to JLA rather than Archie. Another outcome of this little exercise might be that the Mako Mori test isn’t that great a sexism detector.

Just for the record: Archie passes the Bechdel test in issue #2 or #3, whereas JLA doesn’t pass it at all…

Sexy-lamp-testing Rick Remender

Posted: December 14, 2013 | Author: | Filed under: review | Tags: , , , , , , , , , , , , , | Leave a comment

Continuing from the previous post, let’s turn to a gender bias test that some people believe to be superior to the Bechdel Test. In an interview last year, writer Kelly Sue DeConnick (Captain Marvel) said,

Nevermind the Bechdel test, try this: if you can replace your female character with a sexy lamp and the story still basically works, maybe you need another draft. They have to be protagonists, not devices.

This seems even more difficult to put into practice than the Bechdel Test. Is there a scholarly sound way to determine if a story “works”? Anyway, I’m going to try this test with two recent comic books written by Rick Remender. Mind you, that selection doesn’t mean I think Rick Remender is a sexist writer or anything. It’s just that he’s writing a lot of comic books at the moment, and by pure coincidence I happened to have read two of them, Black Science #1 and Uncanny Avengers #14. And who knows, maybe this comparison will reveal something about different attitudes towards gender issues at Image and Marvel, respectively.

The science fiction story Black Science (art by Matteo Scalera and Dean White, published by Image) starts with dimension-travelling scientist Grant McKay running away from fish monsters. He is accompanied by a sexy lamp in a space suit, and his internal monologue is addressed at another sexy lamp. Weird, but not that important for the story. His flight leads him to the den of some frog men, who have captured and enslaved a sexy lamp. McKay frees that lamp and returns her to the fish men, whereupon they become less hostile. There are some more sexy lamps towards the end of the issue, but they are not that significant.

Black Science #1 with sexy lamp added

Black Science #1: frog men ogling their stolen sexy lamp. Makes sense to me.

 

Overall, the story works almost as well with sexy lamps instead of female characters. The “damsel in distress” motif at work here is almost as objectifying as turning her into a lamp.

Uncanny Avengers #14 (pencils by Steve McNiven, inks by John Dell, colours by Laura Martin, published by Marvel) is part of a somewhat convoluted story. The gist is that one sexy lamp with magical powers wants to perform a ritual to defeat the two major supervillains of this story (one of which is a sexy lamp), while two other superheroes (again, one of them a sexy lamp) try to stop her because they think the ritual will help the villains. Of course, this conflict is resolved by means of a lot of fighty-fighty, in the course of which one sexy lamp kills another, only to be killed in turn by one of the supervillains.

panel from Uncanny Avengers #14 with sexy lamps added

Uncanny Avengers #14: clash of the sexy lamps. Makes no sense at all.

Clearly, this fighting and killing makes much more sense when done by the Scarlet Witch and Rogue, rather than by some sexy lamps. Therefore, Uncanny Avengers #14 passes the Sexy Lamp Test, whereas Black Science #1 fails.

Does that mean Uncanny Avengers is less gender biased then Black Science? Not necessarily. The problem with the Sexy Lamp Test is, it “rewards” comics with female characters who say and do a lot, but it doesn’t judge what they say and do. Despite their importance to the story, the female characters in Uncanny Avengers are “lazily” written – all women in this comic book could just as well be men (and vice versa) and nothing would change (except for Wonder Man and Scarlet Witch becoming a gay couple). These female superheroes are just male superheroes with breasts. On the other hand, the femaleness of the enslaved fish woman in Black Science reveals the society of the frog men as patriarchic, and thus at least serves a purpose within the story.

Therefore, I don’t think the Sexy Lamp Test is better at detecting gender bias than the Bechdel Test. They just point out different aspects of gender bias (in speech vs. in narrative function), so maybe they are best used in combination.

Bechdel-testing Japan Inc. and Mai, the Psychic Girl

Posted: December 5, 2013 | Author: | Filed under: review | Tags: , , , , , , , , , , | 4 Comments

The other day, Forrest Helvie posted a nice summary of the “Bechdel Test” and the “Sexy Lamp Test” at sequart.org. Both of these tests are used to gauge gender bias in comics, even though they were not originally intended for that purpose by their inventors, Alison Bechdel and Kelly Sue DeConnick, respectively. Due to both Bechdel and DeConnick being comic creators, rather than theorists, it’s not surprising that in their original form, both tests are under-theorised (i.e. no explanation is given why they should work at all) as well as under-operationalised (i.e. there are many different possible ways to apply them). This in turn has led a lot of people to perceive these tests as simple and consequently apply them to a lot of different things.

Rather than debate their flaws and merits, I’ll just join in and see how they work for some comics I’ve been reading lately. I’ll start with the Bechdel Test (reserving the Sexy Lamp Test for another blog post), which consists of three parts:

A work (originally a film) passes the test if

1. It has at least two women in it
2. Who talk to each other
3. About something besides a man.

One way of putting this into practice is to design a ranking system in which the parts of the test add up, e.g. on the Bechdel Test Movie List which uses different icons for four ranks from “Fewer than two women in this movie” up to “There are two or more women in this movie and they talk to each other about something other than a man”. In contrast, the website Bechdel Testing Comics simply labels comic book issues as either “Failed” or “Passed”.

I’m going to try something else that I think is better suited for comics of different lengths: I’ll look for the first page of a comic on which the dialogue of two women about something besides a man occurs. Let’s start with a manga that I’ve already offhandedly accused of sexism: Shōtarō Ishinomori’s Japan Inc. (マンガ日本経済入門 / Manga Nihon Keizai Nyūmon). Originally published from 1986-88 in three volumes, the first volume was translated into English in 1988, which is the edition I’ll use here.

  • Japan Inc. starts off well. The first female character appears on the second page already (p. 4 in the University of California Press edition), and she even has a speaking part and is a named character: television reporter Kathy White.
  • Soon afterwards, on p. 11, a second woman appears on panel – the first (and only) recurring female character, Miss Amamiya, who works in the same company as the two (male) protagonists, Kudo and Tsugawa.
  • After that, women do have occasional appearances, but hardly ever do we get to see two women in the same scene. On p. 71 there are two female office workers on the same panel, albeit not talking to each other.
  • Finally, on p. 178, two women talk to each other – the aforementioned Miss Amamiya and Mrs. Ueda, the elderly mother of another co-worker. However, their entire short conversation revolves around Mrs. Ueda’s son.

And that’s it! On 313 pages, there’s not a single instance of two women talking about something besides a man. (The second volume of Japan Inc. is a different story with a different cast of characters.)

At around the same time (1987), Mai, the Psychic Girl (舞 / Mai) by Kazuya Kudō and Ryōichi Ikegami was published in English. Its protagonist is a young woman, so surely this manga does better at the Bechdel Test?

  • Sure enough, the eponymous protagonist appears on the very first panel of the comic, and on p. 17 we see Mai talking to her female classmates, Yumiko and Rie. Their conversation goes on for seven pages, but its main subject are men. Mai herself notices that: “You guys are always talking about boys. Can’t you talk about something else?” (p. 21).
  • Then, on p. 14 in issue #2 (or the 50th page of the comic series as a whole), Mai calls Yumiko on the telephone, and this time their conversation revolves not around boys but Mai’s whereabouts.

Thus we can say Mai, the Psychic Girl has passed the Bechdel Test, whereas Japan Inc. has failed.

The Bechdel Test has often been criticised for not being able to detect sexism and misogyny in every instance, but as this little comparison hopefully shows, it’s a good way to get started on talking about gender bias problems in comics.