This summer, I did a study on political pessimism in Swedish political discourse for Dagens Nyheter. It was a collaboration between Jussi Karlgren at Gavagai, Lovisa Bergström at Dagens Nyheter and myself where we made an attempt to quantify tonality in political speeches.
The editorial staff at Dagens Nyheter were initially interested in understanding change in tonality in the party leader’s speeches from Politician’s Week in Almedalen. These occur once a year and are best charactarized as an address to the nation. The speeches are, however, rarely longer than 45 minutes so at eight speeches per year, the data was limited. Therefore, we decided to expand the scope. We also included the party leader debates in the parliament because of their similarity to the Almedalen speeches in tone. Lovisa Bergström collected the data (she even transcribed some speeches!).
But we didn’t just want to model positive and negative emotions in the speeches; we were interested in nuances. To find frequently occurring themes in the speeches, I performed a discourse analysis of the language of the party leaders. Some examples of the final themes — or concepts — are “enthusiasm”, “anger” and “competence”. For each concept, a domain-specific list of representative n-grams were compiled using the results from the discourse analysis and expanded using the automatic synonym extraction from Gavagai Explorer. Gavagai Explorer is a tool for qualitative analysis of text data provided by Gavagai. It has multiple features, including tonality (or sentiment) analysis and concept modeling. Because Gavagai Explorer is based on a Random Indexing word-space model, it allows for synonym extraction by finding n-grams that occur in a similar context to a given n-gram. That is, all concepts (or themes) were accompanied by a domain-specific (political) list of representative n-grams. The finished concepts were plugged into the Gavagai Explorer tonality analysis and the speeches were evaluated against these.
We made a lot of findings and I am not going to account for them all here. In conclusion, we found that all party leaders are increasingly pessimistic in their speeches — despite a flourishing economy and low unemployment rates. (Read the articles below if you’re interested in the details.)
The study resulted in a number of articles:
- “Pessimismen breder ut sig när partierna talar” (“Pessimism unfolds in political speeches”): a long piece with lots of visualization. It explains the method and accounts for the results. A very short summary: the closer the election, the more pessimistic the speeches.
- “Partierna anstränger sig för att bekräfta att de ser samhällsproblemen” (“The political parties are making an effort to confirm that they see the problems in society”): an excellent analysis by Karin Eriksson where she speculates about the results: “[…] de [partiledarna] tar ogärna risken att utmålas som naiva eller godtrogna, och retoriken blir därefter.” (“[…] they [the party leaders] are unwilling to risk being depicted as naive or gullible, and it shows.”)
- “Låt inte mörkret skymma vägen framåt” (“Don’t let the darkness conceal the road ahead”): an editorial piece.
- The results were also referenced in Alexandra Urisman Otto’s excellent story on the Prime Minister Stefan Löfven.
- After the election, the original article was cited in a Washington Post story on the Swedish election.
Unfortunately, the nuances that we were initially interested in were largely lost. When modeling underlying concepts in a limited data set, the concepts need to be very broad for the signal to be sufficient. A concept that is very prevalent in one speech, might be completely absent in another. This makes the results difficult to interpret (at best!). We found that the most comprehensible results were found when we merged all positive concepts and compared them with all negative concepts. In a research paper, I might have submitted other findings, but when your results are being published in the newspaper, clarity is key.