Dave's Adventures in Business Intelligence

I was checking the weather this morning and noticed that weather.com now offers a social media component to their web site. It seems that if I am so inclined, I can see what other folks in my area are saying about the weather. Without doing much, well, any research I am guessing that they’re simply looking at the location information that can optionally be provided on tweets and then scanning for certain weather-related keywords.

Here’s a screen shot of the fail I noticed. Have a look at some of the tweets.

How many of them are about the weather versus something else?

First I see a person from Garland (not far away from me) who is tweeting what appears to be various national headlines, including one about the real estate situation in Florida. Apparently there are “clouds on the horizon.” Does that have anything to do with weather in my area? 😆 No, but it does have a key word “clouds” included.

I like the next example even more. We certainly have clouds here in Texas, but I can’t remember the last time we had a blizzard. Yet someone from Lewisville, Texas, just a few miles up the road from me, is tweeting his disappointment about being left out of (again I’m assuming) a beta program for the game company Blizzard Entertainment and their next incarnation of the Diablo game series. Yes, there is a weather-related keyword in that tweet, but would it not make sense to tie key words to geographical areas? The odds of having a blizzard in Texas (the weather kind, at least) are slim.

This is part of what makes text analytics so difficult. Business Objects purchased a company several years ago (Inxight) that delivers text analytics; this product is now a part of the Data Services product line. It would be interesting to see if they have a feature that would allow me to tie geo-location services to keywords so that I could discount tweets mentioning blizzards in Texas, or hurricanes in Alaska.

Ultimately it comes down to context. When I read those tweets, I can immediately see that they’re not really talking about the weather, even if they do have weather-related keywords. Apparently it’s still challenging for software to do the same thing. Then again, it’s hard enough to predict the weather correctly, so maybe I can forgive them a few errant tweets. 😉