Dave's Adventures in Business Intelligence

It was brought to my attention recently that someone has taken content from my blog and re-posted it to blogspot. I have used the contact form available on their blog to request the removal of my content and have gathered the information I need to file a DMCA complaint if that is not successful.

I don’t have advertisements on this site, and I don’t ask for donations. Everything I have done over the years remains in place and available for the broader user community. Why someone would think they should (or could) take what I have created it and post it as their own work is beyond me. 😕 Bottom line: if you find my content posted elsewhere in the Internet (quotes are fine, references are fine, copying is not) please let me know. Thanks.

With “reporting tools” we have become extremely good at letting people see things they already know. At some companies it can be very difficult to break out of that mode. I can’t tell you how many consulting engagements that I had where I was simply asked to take something they already had and recreate it in a newer tool. While possible, work like that was hardly exciting.

In an unrelated note, FEMA, or the Federal Emergency Management Agency, apparently likes waffles.

What do these two concepts have to do with each other?

FEMA realized that Waffle House was an excellent indicator of whether electricity was available in an area, and therefore could be considered an indicator of an area that they would have to assist. If a Waffle House was open, the area had electricity and passable roads. If the Waffle House was closed, maybe not so much. Even better, Waffle House locations were sprinkled all along the Gulf Coast as well as the Eastern seaboard; both hurricane-sensitive areas. You can read the full article (originally published in Popular Science) at the link below.

What I found interesting about the article was the fact that something that on the surface would seem to be completely unrelated becomes interesting when viewed through the proper perspective. I have been looking within my current company, trying to find something similar, and perhaps I am too close to things but so far I have not been successful.

What about you? Have you found your Waffle House?

Related Links

• How Waffle House Became a Disaster Indicator for FEMA
• What is FEMA?

I was reading about the FBI versus Apple issue earlier today and came across this graphic:

Pie charts are viewed with anything from disdain to downright hatred at times. I don’t want to open a debate on that. But you would think for something as simple as this, they would have managed to get it right…

I emailed the author of the original article and hopefully they’ll get the picture fixed.

Those three little words: “Working as designed.” I hate them.

I don’t know why it took me so long to experience this, as I found this topic on BOB that brought the issue to light years ago. That being said, here’s what’s going on…

In Universe Designer (UDT) a designer can change the state of an object from visible to hidden without impacting a report. I use this technique to create objects that have a specific purpose but keep them out of general circulation. For example, I will often create a class called “Report Objects” that contains objects with special exception handling or a unique purpose. That class is visible in the development environment only, which means my report developers have access to this class. Before the universe is migrated to the QA (and production) environments that entire class is hidden.

This allows me to create report-specific objects without risking an ad-hoc report using one of them incorrectly. It’s a great technique.

A secondary reason why I use the “hide” option is to retire old objects. I don’t want to break reports, but I don’t want any new reports to be created with these objects. Hiding them means existing reports continue to work.

In Information Design Tool neither of these strategies will work. Once an object is hidden, any report that includes that object fails. 👿 Based on information from SAP, that’s the expected behavior. They acknowledge that it’s a change in behavior, but so far I have not found the reason for the change. The bottom line is that it’s “working as designed” and will not be fixed.

Keep this in mind when you consider converting your UNV universes to UNX, as if you’re using the hidden object trick for any of the reasons I outlined above (or others that I may not have considered) that technique will fail in IDT.

Florida State University won the 2014 college football national championship game this past January in a thrilling come-back victory over upstart Auburn. Did big data help?

It turns out that a few years ago FSU started using GPS devices on their players during practice. Each device collects around 1,000 data points every second. The devices are downloaded and analyzed by a special group of coaches and analysts who report back to the head coach. Right now the team is not allowed to tag players during games, so they only use them during practice. Did they help?

One of the things that always amuses me is how often reports — based on big data or otherwise — reveal something we should already know. Two years ago the FSU coaching staff was getting frustrated with the lack of game-day performance from one of their star receivers. It turns out that because he was so good, the coaches were constantly asking him to run extra routes during practice to demonstrate to the other less experienced receivers on the team. By the time the game day on Saturday arrived, he had run much farther than anyone else, so of course he was tired!

Greene was Florida State’s most refined receiver, so when Fisher would grow agitated with poor routes or dropped balls by other players, he would ask Greene to illustrate the proper form. Again and again, Greene would run a route or catch a pass, and his workload mounted. The GPS device offered clear-cut data that showed Greene was simply doing too much.

After tagging the players with the data collection devices the coaching staff was able to recognize this (duh) and make sure he got the proper rest during the week. The player responded with a career best year last year during the FSU championship run.

Related Links

• FSU Uses GPS Technology To Win National Championship

Author Note: This blog post was originally a guest entry at The Decision Factor, a site that appears to have ceased publication. I have reproduced it here. It was fun to write, and I hate to see good content go missing. The original post was published in 2012 but I believe the content is still relevant today.

In my first Star Trek post, I explored two lessons learned from Captain Kirk’s leadership skills and how we could apply them to business intelligence. Today, I’ll cover three more lessons.

Be Part of the Away Team

Captain Picard of Star Trek: The Next Generation rarely joined the away team, leaving that role to his Number One, Commander Riker. Kirk operated with far more freedom. Whenever there was an opportunity to beam down to a new planet, he was there. His leadership style demanded that he lead from the front.

I think the corollary with business intelligence work here is obvious: the success or failure of our systems depends far more on establishing correct requirements than on the actual implementation. By being part of the away team, we can be directly involved. Lack of user involvement and poor requirements gathering are generally two of the top reasons projects of any kind fail, and business intelligence is not immune to this problem.

Play Poker, not Chess

In Star Trek, there were numerous times when Captain Kirk and his crew were in a bad spot. In one episode, The Corbomite Maneuver, the Enterprise and her crew were experiencing first contact with a new alien race and it wasn’t going well. At one point, Spock had to tell Kirk that they were out of options and the alien had backed them into a corner from which there was no escape. In chess terms, checkmate. Spock, as a character, was the very embodiment of chess: cold, logical, and process driven. It was natural that he would see things this way. But Kirk was different.

Capt. Kirk: There must be SOMETHING to do. Something I’ve overlooked.
Mr. Spock: Chess: When one is outmatched the game is over. Checkmate.
Capt. Kirk: Is that your BEST recommendation?
Mr. Spock: I’m s… I regret that I can find no other logical alternative.
Capt. Kirk: Not chess, Mr. Spock. Poker!

Kirk took inspiration from the game of poker and bluffed his way out of the situation. How can this possibly apply to business intelligence? We don’t want to lie to our business partners, do we?

Perhaps we do – but in a good way.

I often come into contact with people or business processes that have been rigidly followed for many years. To the mind of the many, there’s no other way to do that particular process. In order to convince them otherwise, sometimes we have to bluff. I like to call this a demo. 😉 In order to make a difference, we have to get past that initial resistance; and sometimes a well-placed bluff demonstration helps us along that road.

Blow Up the Enterprise

Sometimes there’s no way around it. A system that’s well known, trusted, and loved just isn’t cutting it any more. The original Enterprise made it through several years of TV shows and a couple of movies before ultimately meeting her end. As much it pained Kirk to let her go, he realized that in order to move forward, the Enterprise had to be sacrificed. (I may have shed a tear or two myself.)

Legacy systems can often inspire the same attachment, but at some point we have to blow them up so we can move on. “It’s always been done that way” is a poor reason to keep a system around when there are so many new and exciting technologies that have come to market recently. Don’t be afraid to blow up the Enterprise to move forward.

But I think the overwhelming lesson to be learned from Star Trek is don’t wear a red shirt to work.

Author Note: This blog post was originally a guest entry at The Decision Factor, a site that appears to have ceased publication. I have reproduced it here. It was fun to write, and I hate to see good content go missing. The original post was published in 2012 but I believe the content is still relevant today.

While the touch-panel displays on Star Trek: The Next Generation certainly foreshadowed the iPad, the focus of my discussion today is around another blog: “Five Leadership Lessons from James T. Kirk,” published by Forbes Magazine.

While Captain Kirk had his flaws—making his character all the more interesting—he certainly was the unquestioned leader of the Enterprise. I enjoyed reading the article and think the specific lessons outlined also apply to business intelligence.

I’ll share the first two with you today and save the others for a future post.

Never Stop Learning

When Kirk ran the Enterprise, the Federation was much younger. Their mission was to seek out and explore, encounter new civilizations, and learn from them. The opening monologue said it all:

Its five-year mission: to explore strange new worlds, to seek out new life and new civilizations, to boldly go where no man has gone before.

Sounds like good advice for business intelligence professionals too, right?

In order to deliver effective results, we have to first understand the business. Whether we work for a large company or a small start-up, knowing the business (and the data) is the first step towards providing business value. It doesn’t help to design solutions in a vacuum that the business won’t understand, won’t use, and doesn’t need.

As a business intelligence professional, I look forward to those times when I get sent to the far reaches of our enterprise (see what I did there? 😉 ), so that I can directly observe business processes in action. The lessons learned might not provide an immediate benefit. It might be months or even years, before I can put something I have learned to good use. What’s important is that I do interact with the business and learn from that experience.

Have Advisors with Different Worldviews

Captain Kirk was blessed with a diverse crew, not only on the bridge of the Enterprise but throughout the ship. This was no accident. James Roddenberry, the visionary behind what ultimately became one of the biggest TV and movie franchises of all time, intended this to show how a diverse crew of different ethnicities, genders, and planetary origins could act together in towards a common goal. Spock and Dr. McCoy could hardly have been more different, yet each was able to provide valuable input to Captain Kirk’s decisions. They each supported his role as their captain and leader in their own way, often in direct opposition with each other.

In the business intelligence role, it’s just as important for us to seek out and explore diversity. Before designing a system, I need to talk to everyone who might use the system and gather their input. Getting only one side of the story could lead to incomplete requirements, which in turn leads to wasted time and money. In this day and age (much less the 23rd century), we can’t afford to do much of either.

Stay tuned for more business intelligence lessons from Star Trek

In a few weeks I’ll return with the final three leadership lessons from Star Trek that have implications for business intelligence professionals. In the meantime, I wish you success in exploring the world of business intelligence.

Author Note: This post was originally a guest post published on “The Decision Factor” blog. Since that site seems to have disappeared I am re-posting it here. The original publication date was November 1, 2012, but I believe the content is still relevant.

What’s the purpose of a business intelligence (BI) dashboard? It’s not just to look sexy. The primary purpose of a dashboard is to convey information. A secondary purpose is to inspire a behavioral change based on the information that’s being conveyed. Nobody wants to be “red” on a dashboard reviewed by executives, so they’ll change their behavior in order to get into the “green” area.

But humans are a creative species. What happens if their behavior changes in an unanticipated way?
Continue reading “Lessons in Business Intelligence: Be Careful What You Wish For”

Stephen Few weighed in on what is the proper definition of big data yesterday, and it’s an interesting read. If you don’t want to click through, I will summarize in one sentence: “Big data is nothing special, it’s just data.” Obviously Stephen’s opinions have not stopped (and won’t stop) people from using the term.

Next up on my blog reading list this morning was a trip to FiveThirtyEight.com. The headline article this morning was titled, “The Story Behind the Worst Movie on IMDB.” I’m guessing that IMDB doesn’t really qualify as “big data” as they have “only” 2.8 million titles in their database. 🙂 But the story wasn’t about big data, it was about the worst movie in the database as determined by public rankings. I would have expected the soundly panned “Battlefield Earth” (and it was one of the worst with an overall rating of 2.4), the unfortunate Halle Berry stinker “Catwoman” (3.3) or perhaps even the Paris Hilton vehicle “The Hottie and the Nottie” (which I’m somewhat ashamed to admit I even knew about and brings in a lowly rating of 1.8).

It turns out the worst rated movie was not any of these, but instead was a Bollywood production called “Gunday” which has a rating of 1.4. Over 91% of the posted ratings are one star! What happened? Was the movie really that terrible?

For the full story, click through to the story on fivethirtyeight. In summary: an entire country decided they didn’t like the movie and decided to do something about it.

…the movement has since become an online alliance of bloggers focused on protecting Bangladesh’s history and promoting the country’s image. That includes protesting “Gunday,” because of the film’s reference to the Bangladesh Liberation War as the Indo-Pak war. In its first 11 minutes, the movie claims that India alone defeated Pakistan, and implies that an independent Bangladesh was simply a result of the fight.

What happens when an entire country decides that a movie is bad? The movie becomes perceived as historically bad. More from the article:

For Paris Hilton’s “The Hottie & the Nottie” — currently rated second-worst of all time — to take over IMDb’s bottom spot, the next 41,000 voters would have to give it a 1.

Last year I wrote a blog post titled Is External Data Always Good?. This is one more example of how social media / crowd-sourced data can be skewed by a concentrated effort. Is “Gunday” really the worst movie of all time? Probably not. Most professional critics were not nearly as harsh, especially when compared to Paris Hilton’s effort. One user reviewed Paris’s acting by saying, “Paris Hilton’s acting made me lose braincells.” The reviews on IMDB were not spammers; they were unique individuals. They just happened to be part of a focused effort to trash a movie they perceived as historically inaccurate. (Please note: I am not making any assessment as to the accuracy of the film. I am far from an expert in that area so I’m neither endorsing nor rejecting the movie.)

Ultimately I think the article from FiveThirtyEight wraps it up the best.

Crowdsourcing can be a tremendously powerful way to get a consensus understanding of the world. Because the sample size is so large, there’s an assumption that whatever it yields is robust and true. But even with oversight, aggregated rankings are subject to unforeseen biases. Crowds are always big — but they’re not always wise. Sometimes it’s impossible to control which crowds are being sourced.

Big data is just data. But you still have to understand where it’s coming from in order to benefit from it.

The actual origin of the concept of a “red herring” is unknown, but that doesn’t stop it from causing grief while trying to diagnose a performance issue. If you are not familiar with the concept, a red herring is something that initially appears to be relevant but ultimately is proved to have nothing to do with the actual issue. It’s a popular technique for mystery novels… and in tech support calls.

Case in point: Today I had to help someone who was wondering why their report took over thirty seconds to display a prompt window when there was only one prompt in the document. Clearly it was a prompt issue, right? Or something related to the list of values definition for that object? Continue reading “Big Universe + Security Profiles = Slow Query Generation”