Dave's Adventures in Business Intelligence

Yesterday was a busy day! I flew in to Vegas on Monday night, and arrived too late to visit the registration booth. I met Michael Welter and he watched me eat while we talked about plans for the conference. Later I got a chance to visit with Steve Krandel as well.

I got my registration taken care of the next morning with minimal fuss, had a quick breakfast, and then made my way over to the keynote speech which was going to be given by Steve Lucas and Timo Elliott. It was the first time I remembered hearing of them speaking together, and they confirmed that during the keynote itself. Unfortunately I was going to have to leave the keynote early, as Michael and I had to go review the configuration of the laptops in the room for our training class at 9AM.

The topic on the keynote was big data, which is probably not a surprise. SAP has been using the big data concept to push HANA (to his credit, Steve waited until about 10 minutes into the talk before mentioning that product name 😉 ). He also took a more unique approach to demonstrate the growth of big data. In many sessions like this the presenter has focused on a graph that shows the amount of data being generated every week / day / hour / minute of our lives. That’s all well and good, but it doesn’t mean that I have to process all of that data. Who is really going to analyze all of the bytes of every photo ever uploaded to Flikr for example? Instead of doing that, Steve showed a graph based on research from LinkedIn and other sources showing the documented growth of big data jobs. These would be jobs with titles like Data Scientist and so on, and there was a clear growth in this area over the past few years along with a major spike in 2010. Interesting approach.

Steve then introduced the typical “Three V’s” of big data, those being Velocity, Volume, and Variety. At this point Timo stepped in and suggested that a fourth “V was equally important, that being Validity. It doesn’t really matter how fast your data comes in, how much data comes in, or how different the data is if it’s not valid. That’s a fair point, and also serves as a selling point for the data services from SAP which come with various data validation tools. I don’t think that point was lost on the audience. 😉

Steve then went on to show a list of a variety of open source tools used to ingest, process, store, and then analyze big data. There were quite a few tools that I had heard of before (Hadoop, Voldemort, others) but also some tools that I had not yet heard of. Many of those tools are in use in large enterprises today.

However, these open source tools – as great as some of them are – were not designed to integrate with each other. Now we’re starting to see where this talk is going as Steve shows a slide that shows how SAP provides tools for the same four steps (ingest, process, store, and analyze) for handling big data sources. This is about when HANA made her first appearance, I think.

Now that HANA was on the table, Steve was able to start talking about real-time processing of big data. He used an analogy that seemed appropriate at the time but later as I started thinking about it, maybe not so much. He showed a picture of a person looking at traffic on a very busy street. That person wanted to cross the street, and was taking in all of the data points including traffic flow patterns, weather, sounds, smells… basically a huge amount of data was being collected as the person was standing on the side of the road.

Steve then said, would you make your decision to cross the road if your data was five minutes old? What about 60 seconds old? Even five seconds?

The point, obviously, was you needed up to the second, or actual real-time data in order to make that decision. That’s a fair point in a life or death situation like crossing a very busy street, but I’m not sure it works when talking about big data quite as much. For example, what about the volume of the data flow (one of the three V’s mentioned earlier)? If I have been looking at historical data (say for the past 24 hours) then I have some idea of what the traffic flow patterns are going to be. There is not nearly as much traffic at 3AM as there is at 9AM, and thats useful information too. But I am going to maintain that there are times when real-time is important (my speedometer in my car) and times when it’s not as important (fuel gauge). As long as I know my fuel gauge is updated every fifteen minutes that’s accurate enough for me to determine when I need to stop for gas. But I don’t want to have to explain to a traffic policeman that I didn’t really know how fast I was going through that school zone since I had just exited the highway… 😆

The point is, real-time data has a place, but I don’t think that all data has to be real-time data in order to be valuable. (Is that the fifth V?)

At this point I had to leave the keynote, but I know they were webcasting it so perhaps it’s available online for me to go back to later.

Michael and I found that our virtual machine image was working fine, so I think we’re prepared for our training session (which will come later today.) I delivered my first talk of the week about Web Intelligence variables (my first time ever on that subject! :lol:) which seemed to go well. Later I attended a session given by Timo during which he shared a lot of statistics and case studies about analytics, including one about new paint on big boats that was designed to be too slippery for marine life to attach to but had the pleasant side effect of making the boats more fuel efficient as well. It was also good to catch up with Timo again; he says he has now been with Business Objects for twenty years. 😯

At the end of the day I participated in an Ask the Experts session. I shared a table with Tammy Powlas, and we had conference attendees drop by with their questions. The session ran for an hour and a half, and our table always had people there, so I assume we were popular. In fact the last group of folks I talked to went beyond the scheduled closing time and they started to turn out the lights! 😆 Eric Vallo did a drive-by but didn’t stop to chat as the table was full.

So what are my thoughts on the event so far?

Everything has been running smoothly and the folks here seem to appreciate the content. WIS made a specific effort to include a lot more “classic” Business Objects content this year so I’m seeing a lot of familiar faces. (Haven’t mentioned Jamie yet, but he’s here, and I expect to see Chris later today. Also had dinner with Alan Mayer (founder of Integra Solutions and now Solid Ground Technologies) last night.) My schedule has kept me fairly busy, but I hope to attend more sessions in the coming days, specifically around mobility. I’ll be back with details.

First post from a new laptop. Browser doesn’t have spell check, so hope everything came out okay…

I missed the keynote on the first day, but I made sure I made it on day two. The four pillars of the conference were cloud, big data, mobility, and social. I get the first three; they’re very similar to the Business Objects themes of cloud, mobility, and in-memory for the past few years. The social aspect bothers me a bit, and not just because I’m not a huge fan of Facebook. They showed off a number of Facebook apps (Usher, Wisdom, and Emma are three names that I remember as I’m finishing up this post). The apps themselves are okay, but I really don’t think they should be the focus of the company. Michael Saylor is apparently very big into the social scene, and he’s betting big that the company efforts in this area are going to pay off. I keep thinking back to what the core competencies are, and how does social fit? Not so well, in my opinion, but as the next few years play out I guess we’ll see how that strategy works out. I recently read another article on CNN that was musing about the future of Facebook, titled, “Will Facebook Be The Next Yahoo?” that sums it up nicely, I think.

Mobility was another subject that was emphasized during the keynote. Various studies were quoted that suggested that within a few years almost every customer will have at least one mobile app, and by 2015 mobile development projects will outweigh desktop projects by a ratio of four to one. I get that, but so does SAP. I believe it was Steve Lucas (or perhaps Vishal Sikka) who said at a recent event that “Mobile is the new desktop.” I found that most folks at this event believe that MicroStrategy is currently ahead of SAP’s mobility offerings. (That’s probably not a surprise, given the venue.) The primary advantage that they have over SAP is the fact that everything has grown organically, so integration is already there. SAP is still trying to work to get Web Intelligence and Xcelsius and other content delivered to the iPad in a seamless fashion. They’re making progress, but it’s not like MicroStrategy is waiting for them to catch up either.

Both vendors (SAP and MicroStrategy) are making a lot of noise about the cloud. I can see where this would be very attractive, especially for a smaller or mid-sized company. The ability to spin up a project with minimal (or no) investment in hardware would save money but more importantly allow faster time to delivery for the project. They did a good job of spinning that advantage during the keynote, but I don’t think it needs that much “spin” to show the advantages. There wasn’t anything super exciting about this that I haven’t also seen from SAP.

After the keynote we had another nice lunch (again, the food and service at this event reminded me of the way things used to work at Business Objects events) and then it was time for some track sessions. Overall I found this event did not have very many deep-dive technical sessions, and I really missed the labs that are available at Business Objects events.

I attended one session on data “mashups” that reminded me a lot of the features provided on the bi.ondemand.com cloud site. I did pick up a few new sites that are interesting sources for public data. For example, we looked at garbage collection statistics from NYCOpenData. 😆 MicroStrategy offers direct interfaces to any URL-enabled data source but also Twitter, FourSquare, and of course Facebook. Another site that provides open data is InfoChimps.

I went to a few other sessions on day 2 and more on the morning of day 3, but nothing that I really want to cover in detail. As I’ve mentioned (more than once) I felt that at least for me this event did not offer enough deep-dive technical content. Even when a talk was interesting (see the notes about the LinkedIn talk from day 1) they often didn’t offer much about MicroStrategy.

But I did get a t-shirt. 😛

I have had a couple of people ask me (offline) if I’m leaving Business Objects and working with MicroStrategy now. Rest assured the answer is no, I am not. But I’m excited about the opportunity to get my hands on another tool, especially when both tools have strengths. Ultimately it’s about getting data to end users in a format that they can use to improve the business, so why not have more than one choice? In fact it has been over a year since I renamed my blog from “Adventures in Business Objects” to “Adventures in Business Intelligence” in order to be able to talk about different ideas like this. I’m nowhere near as fluent in MSTR as I am in BOBJ, but I hope to fix that over the coming years. We’ll see how that goes. 😎

I’m at my first MicroStrategy conference this week. It’s interesting to see what is different and what’s the same compared to the Business Objects conferences that I normally attend. For one thing, they don’t hand out survey forms at the sessions. For a BI / data company, that seems surprising. The food here (which I know is a big reason why folks come to conferences, yes? 😉 ) is like the food used to be at Business Objects conferences several years ago, that is to say we sit down for lunch and they bring around plates. Yesterday was chicken, pasta, green beans, and cheesecake for dessert. It was quite good.

That being said, try to find a bottle of water, or even a water cooler, anywhere around the place. It’s nearly impossible. I finally found some water late yesterday afternoon, after asking a number of different conference folks and getting blank looks or, “I think I saw some over there, somewhere…” comments. The wireless has been good, although it ironically dropped out (at least for me) in the room I went to for the mobility track. 😆 That track is all the way on the end of the hotel, so perhaps they need another access point.

Not only are they not handing out survey forms during the sessions, but they’re not scanning our badges as we enter the rooms. Again, for a data company I find that surprising.

But what about the content? Yesterday I attended three sessions. The first was a joint session between Teradata and eBay. The Teradata folks talked about their generic concepts for “big data” and how to let analysts make the best use of it. The eBay gentleman then talked about some specifics around how they work with their large data sources (petabytes of data). It was interesting but I didn’t see a lot of MicroStrategy stuff, just big data stuff. Next I went to a session delivered by LinkedIn. I found it to be more interesting because in this case they talked about data quality issues that I can certainly related to. BOB is nowhere near as big as LinkedIn (they have 135MM users at this point) but we still have consistency issues. For example, the presenter asked the audience how many ways we thought the job title of “Software Engineer” appeared in their database. The majority of the guesses were very low compared to the actual value of over 6,000. They have over 8,000 different iterations of the company name IBM! 😯 As you can imagine, searching is a big challenge for LinkedIn. As I said, the talk was interesting, but at the end the presenter had not talked about MicroStrategy or shown a single product during the entire talk! In fact that was the first question from the audience, “How are you using MicroStrategy in your environment?”

The last session I attended (the aforementioned mobility track) was given by a presenter from Lowes Hardware. Lowes is a big user of MicroStrategy products. (In fact their former CIO is now apparently in charge of the cloud for MicroStrategy.) He was by far the most engaging presenter and he powered through his session even after the failure of the audio equipment in the room. Lowes has purchased over 40,000 iPhones and has apparently bet big on that hardware platform along with the mobile products from MicroStrategy. He gave a great example… every store manager used to spend a few minutes each morning jotting down some notes from a sales report in order to have that information with him or her at any point throughout the day. Just a few minutes a day, but it was something they did essentially every day. The replaced the report delivery (and hand notations) with a mobile app and eliminated those few minutes. It doesn’t sound like much, does it? They estimated that the savings (I assume based on average pay for store managers) at only $6.84. I think I have that number correct, if I’m wrong it’s not by much. When that savings was multiplied by the number of managers across all stores, and then multiplied again by the number of days in a year, the total productivity savings came out to $4.3 million dollars. Per year. Talk about a quick return on investment, yes?

The app was cool, but I wanted to know more about how it was built, what tools were used, and what the process looked like. So far the sessions I’ve attended have been very light on specifics, so I hope I pick better sessions today. Will let you know tomorrow.

I did not do much blogging at the conference this year, primarily because I was always doing something! I had two presentations on Monday, followed by an interview session with ASUG News on Tuesday morning, followed by my participation in the Leadership 2.0 session Tuesday afternoon, the BI 4.0 launch celebration Tuesday night, a usability study for an interesting new potential product this morning, and my flight home. 😯 Over the next few days I will attempt to turn my notes taking at the various sessions into blog posts, as well as upload my two sessions for download both here and on the conference web site.

I would like to take this opportunity to thank each and every one of you that took the time to come up to me at the conference and say how much you appreciated either BOB or my blog. As long as I have been doing this sort of stuff (on BOB we will be celebrating our 10-year anniversary next year) hearing from people never ever gets old. Thank you for taking the time to let me know that you have been helped in some fashion.

Are you attending the SAP Business Objects user conference in Orlando next week? If so, look for an email soon inviting you to a special lunchtime panel discussion on social media, influence, and leadership called “Leadership 2.0” and featuring folks such as Cindi Howson, Lisa Leslie, Timo Elliott, and … somehow… me. 🙂 Reservations will be required as space is limited to the first 100 people to sign up. I think they’re planning on sending out an email to all folks registered for the conference, but you can get a jump on reserving your spot by going to the reservation page.

The panel will run from 12:30 to 2:30pm on Tuesday, October 11^th and will include a lunch. The Twitter hash tag for the event will be #L2dot0.

Added Information
If you’re not attending the conference, you can still participate. There is a dial-in number where you can listen, and you can post questions via the hashtag mentioned above. Here’s the call information:

Participant Access Instructions – Dial in 5-10 minutes prior to start time using the Participant Phone Number and Participant Passcode.
Participant Passcode: 690937

Toll-Free Phone Number (United States/Canada): 877-702-9054
Toll Phone Number (International): 913-312-1229

2011 has flown by, and before I realize it I’ll be heading down to Orlando for the SAP Business Objects User Conference (#SBOUC hash tag). This year I have two sessions, both on Monday. For the first session I have the honor of opening up the “Building a Data Warehouse” series that was suggested by Werner Daehn. When he first mentioned the concept to me my first thought was, “what a great idea!” I’m really glad it came to fruition. What is it, exactly? I am participating in the first ever “group topic” at SBOUC. Werner wanted to show the complete development cycle, from database design all the way through to report delivery. He came up with the initial concept and outline and posted it on BOB, and then various folks submitted talks to fill in the various slots. Werner has as couple of slots, Michael Welter did the universe design, Dell Stinnett will bring Crystal into the mix, and Simon To will be presenting the Web Intelligence session. Here’s the full schedule as I have it today:

• Monday 9:30 AM – 10:30 PM Dave Building a Data Warehouse: Data Modeling
• Monday 10:45 AM – 11:45 AM Werner Building a Data Warehouse: Getting the data into the DWH database
• Monday 1:30 PM – 2:30 PM Michael Building a Data Warehouse: Building a Universe
• Tuesday 9:30 AM – 10:30 AM Dell 0106 Building a Data Warehouse: Intro to SAP Crystal Reports
• Tuesday 1:30 PM – 2:30 PM Simon 2008 Building a Data Warehouse: Report Development in SAP BusinessObjects Web Intelligence
• Wednesday 9:15 AM – 10:15 AM Werner 8012 Building a Data Warehouse: Data Quality is key for BI
• Wednesday 10:30 AM – 11:30 AM Werner 8013 Building a Data Warehouse: Enhance the DWH with external data

I won’t get to attend all of the sessions in this series, because immediately after I open with the data modeling talk I will have to jog over to another room to present my second talk titled “Universe Designer Essentials.” I’m all done with my presentations after Monday.

It’s hard to believe that I attended my first Business Objects conference in 1995 (spoke for the first time in 1996) and here we are 17 years later. I must have been 17 when I attended my first one. 😉

Can’t raise the budget to get to the SAP BusinessObjects user conference next month? Or maybe you’re already booked, but think your finance department would love it if you got a rebate for your conference costs? All you have to do is mention a few hash tags in a tweet, and you could get your expenses covered (read the fine print for details). Contest void where prohibited. Batteries not included. Your mileage may vary.

As a follow-up from the SAP TechEd opening keynote on gamification and the power of online gamers:

Gamers solve molecular puzzle that stumped scientists

Scientists had long been puzzled by the molecular structure of a protein-cutting enzyme classified as a “retroviral proteases” and found in an AIDS-like virus afflicting rhesus monkeys. The enzyme helps the virus spread, and it could hold a secret about how AIDS and other diseases are transmitted.

The scientists uploaded the viral structures to an online game called “Foldit” which allows game players to collaborate on solving various three dimensional problems. The results?

Sure, that might not sound like much fun to the average Joe. But gamers on Foldit solved the puzzle in less than 10 days.

More detailed article at MSNBC.com

I published an abridged version of my keynote review already. This post will contain more details on a number of different subjects, but still almost in one-liner format.

In no particular order… Continue reading “SAP TechEd Keynote – Final Review”

We had a great start to the conference! Dr. Jane McGonigal took the stage to tell us how we can do so much more if only everything were like a game. That’s perhaps an over-simplification, but I’m going to run with it for a moment. The concept of gamification is basically this: if you put an obstacle in front of someone and tell them they have to do it, they’re likely to whine and complain. Trust me on this one, I have two boys (ages 8 and 10) and I can vouch for the whining and complaining part. 😉 On the other hand, if you give them the same obstacle but frame it as a game, they’ll willingly go along with actions that they would not normally do… and have fun doing it. As one example she mentioned the “dance pad” games. Many people do not like to dance, and would rather do almost anything else when out in public. But turn it into a game that starts out simple and builds in complexity and they’ll willingly give it a try.

Golf was another example she used. Golf really is quite an unusual way to spend time. Suppose that golf didn’t exist today, and someone came up with the idea of putting a little ball into a small hole. Weird idea, right? The most obvious solution is to pick up the ball, walk over, and drop it in the hole. Of course that’s not how golf works. They make it far harder by starting from a long way away, and using various different sticks to hit the ball towards the hole. Pretty ridiculous, right? I can only imagine what the first conversation was like as folks were inventing golf… it probably did not go as Robin Williams imagines. (Warning: includes strong language, definitely not safe for work without headphones.)

Here’s a quote from one of her slides:

Games are unnecessary obstacles we volunteer to tackle

Golf certainly fits that description. With apologies to golf fans everywhere, it’s certainly not a necessary obstacle. The same could be said for many sports. Yet today golf is a multi-billion dollar industry. Folks spend hundreds or thousands of dollars buying equipment, spend hours out of their day, just trying to get better at putting a little ball into a little hole. What are they getting out of it? Continue reading “SAP TechEd 2011 – Gamification Keynote”