A Broken Compass: Sentiment Analysis in the Tweetstream


About a year ago, I became acutely aware that my consumption of Twitter was becoming increasingly entangled with elevated levels of anxiety. Twitter has long been a wonderful way to connect with cool people I can’t always see in person, so I didn’t appreciate how my experience with it was souring.

I began searching for ways to understand what was happening, so I could think about how to make it better. I could very reasonably blame my physiological reactions on my own bad habits, thought loops, and behavior patterns. However, I didn’t feel that I could confidently address that problem in the short-term (therapy can only do so much so fast), so I looked for another place to intervene.

The answer, I realized, was in the people I follow. Who I follow dictates what tweets and retweets end up in my feed. It even strongly influences the less obvious content of my feed — what “the algorithm” thinks I want to see.

It seemed reasonable to hypothesize that the quality of my feed depends greatly on the quality of the content authored (tweeted) and curated (retweeted) by those I follow. So I decided I might begin to actively shape my feed by following, unfollowing, and muting different accounts.

I realized, however, that I couldn’t perform these actions in a consistent way. My feed-shaping was ad-hoc and based on luck — whatever content showed up when I was looking would be evaluated — so I decided to find a more systematic approach.

How could I decide more quickly and deliberately who to follow, who to unfollow, and who to mute? First, I would need to understand what I wanted more of and less of.

I started playing with the Twitter API to pull down a list of people I followed, so I could at least open up all their profiles quickly and look through what they’d been tweeting. I was following hundreds of accounts, so it turned into quite the mess. I started playing with some other uses for the Twitter API, when I had a bit of a eureka moment…

What if I could automatically analyze the tweets and retweets of all the people I follow in order to create a short-list of accounts to examine?

IBM Watson had made a sentiment analysis API available, and that just might do the trick. I signed up for an account with free credits and began to try a few things out. With a few hours’ work, I had a giant pile of sentiment-analyzed tweets and retweets for all the accounts I followed. I aggregated the data, and visualized it as distributions according to percentiles of various sentiment dimensions, such as “Cheerfulness.”

As you can see, I was following accounts that skewed towards Depression, Sympathy, Artistic Interests, Liberalism, Self-Consciousness, and Imagination. And they were very much NOT Cheerful.

It became clear how I could use this information as a sort of broken compass. If I wanted more Cheerfulness and less Depression, for example, I could unfollow or mute accounts that ranked disproportionately low along those two dimensions. The distributions ought to change accordingly.

But I wouldn’t just perform those actions on those accounts automatically… that would be to accept a tyranny of algorithms. I needed to have choice.

I decided to start with Cheerfulness and shared the idea with Venkatesh Rao. He was very supportive.

Jokes aside, he did take interest, even introducing me to Renee DiResta, who has done awesome, thought-provoking work about social media and its impact on our lives.

I began my first self-intervention, generating a list of accounts at the top and bottom percentiles as shown below. Could I really increase the Cheerfulness in my Twitter feed by adjusting who I followed?

The answer, in short, was “yes.” (Blue is before. Orange is after.)

This result might seem unremarkable, yet it was incredibly encouraging as an example of having successfully “reshaped” (though perhaps in a very small way) the distribution of Cheerfulness in my feed!

This change also resulted in a few correlating changes to Anxiety, Anger, and Depression dimensions (less of each). There were unexpected effects, however, such as less Imagination:

Though unexpected, it was not unwelcome — as long as I remained informed about how adjustments in following or unfollowing different accounts resulted in the tradeoffs along these dimensions, I could use this “broken compass” to take a more active role in shaping my feed.

As I shared the idea more widely, I ran similar exercises with willing participants, even going as far as to create a set of Frequently Asked Questions, such as:

  • What are you trying to accomplish here?
    • The intention is to give you more control over the content of your Twitter feed through a fundamental mechanism: Who you follow.
  • Are these personality models accurate?
  • So what’s the point if the models aren’t exactly accurate?
    • Well, accuracy that’s better than “good enough” isn’t necessary, because the models only tell you where to focus your attention and a “good enough” reason for doing so. If IBM Watson Personality Insights ranks an account a certain way along a certain dimension, it’s still up to you to decide what to do.
  • What mindset should I take with this method?
    • There are no silver bullets.
    • This is an ongoing, iterative process.
    • Think, “IBM Watson ranked this account this way on this dimension. Do I agree with that? Is it a reasonable hypothesis that unfollowing/following this account will bring about the desired change?”

I considered building out a full service, but I realized the economics of the API calls just weren’t going to work out in my favor. Maybe some ideas are best left alone, for now. I hope one day to see a greater balance of power between social media giants like Twitter and its user base, and this project seemed like an interesting step in that direction. It felt good to explore an interesting edge of this problem space.

Two final points to significantly increase the quality of your feed…

First, an important lesson I learned about the “retweet” function is that it is not designed to engender a sufficiently careful curatorial approach to boosting content. In other words, it’s very easy to RT something, so we do it all the time, with little thought applied, amplifying the noise in our feeds. As a general policy, I recommend disabling retweets for all accounts you follow on Twitter (make exceptions as appropriate). Your feed quality will thank you!

Second, there is exactly one account I will endorse as overwhelmingly positive, and that is Visakan Veerasamy. He is a cheerful tweet genius who never ceases to brighten my day.

Thanks for reading!

,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: