What startups with ‘massive amounts of clue’ will drive (Low Latency) Big Data?

Brenda concludes with:

While the discussion on what is, and isn’t, big data is only getting started, the value of unlocking business insights from data piles is well understood. Dion’s quote of Tim O’Reilly brings it all home:
Companies that have massive amounts of data without massive amounts of clue are going to be displaced by startups that have less data but more clue.”

h30565.www3.hp.com

It seems that the definitions and opinions on what is, and isn’t, big data, could fill a terabyte or two on their own.  As a business-technology practitioner, I’m not in the market space definition business.  But, I am a…

—–

The problem with these types of articles, you have people who are ‘talking heads’ who don’t get their hands dirty, nor do they seem like the type to have actually spent any real time thinking about the problems that they are trying to discuss.

There’s a simple reason why you’ll see start ups eating the big guy’s lunches. They lack the existing infrastructure. The larger the in place infrastructure, the more resistance to change and the slower that they will be to adopt to newer technology.

The other issue is what sort of information can you get from the types of data. The larger the company the more focused they are on their existing operations and if they are a public company, their focus is on making money and not spending money on R&D. A startup has no real existing flow of revenue to focus on so that they can take risks and focus on what they think will be the next big thing.

Its these talking heads that lack the clue, not the large corporations.

—-

Love the O’Reilly quote!

—–

I define “big data” as the data the size of that is bigger than an organization can processes. The data that an organization can process is just, well, data.

—–

There are no big data problems that money cannot solve. The problem with Big Data is that there are a lot of companies who don’t understand the value of the information they possess because all they know is data…

This is very true but beginning to change. The emergence of ‘data scientist’ and quantitative analysis roles in domains typically not considered traditional consumers of ‘data science’ is changing this.

Skybox Imaging, for example, a Palo Alto based startup have designed and built a micro-satellite. They plan to launch many of these micro-satellites into orbit, take regular high resolution imagery of the earth, perform feature and image extraction, apply domain metadata and … monetize the resulting feeds.

PeopleBrowsr is another good ‘down to earth’ example. They’re effectively monetizing sentiment extracted from social networks.

Two good examples of big data. In both cases, they’re capitalizing on freely available or publically accessible data in order to provide value added services. So Tim O’Reilly’s quote is prescient, no?

So to me big data is about mining value from data that, here-to-fore, has been underutilized, for fun, profit and more effective data driven business efficiencies.

Perhaps Big Value Data would be a better term. It’s not a size thing. It would be unfortunate if big data was reduced to a volumetric or processing arms race. So
Tim O’Reilly’s quote strikes a chord yet again. It’s about value. And value is clue-oriented.

“The emergence of ‘data scientist’ and quantitative analysis roles in domains typically not considered traditional consumers of ‘data science’ is changing this. ”

No it isn’t! And it isn’t new either… In fact, there is nothing new about this.

We have had “big data” and the need for “low latency” for the last 40 years! Nothing about this is new and Information Engineering has been around for the same number of decades. What has happened is; our industry has become reliant on “tools techs” rather than the true scientists behind most of the larger product sets.

You see this in people’s titles – if it is preceded by the name of a tool and followed by a role – those are tool techs (eg, SAP Architect … or Oracle DBA). We used to have real Architects and real DBA’s (non tools specific) – the fact we now call them “data scientists” doesn’t mean it is new. These people are information engineers and many of them are still around looking further into a very deeply understood problem of managing and harvesting information.

I like Forresters approach, makes sense.

I also want to add – Google isn’t a ‘start up’ or a new thinker – they are merely another very large company with a massive amount of data that needs to be cleverly engineered to make sense given their domain of interest. The “Niagara Falls” of data probably still belongs to the banks who keep detailed records on every single transaction they’ve ever made.

It’s called Information Engineering and it is in the realm of DAMA. Been around forever, is well understood, is heavily studied in academia – but is not tools-centric or tools specific. All of your startups Michael – are struggling to produce even more tools into a very mature market. Darach – your examples of big data in fact, are not that big and much of it is already publicly available.

But don’t fool yourself into believing that startups with little or no experience will do any better than those that have studied this problem for decades.

 

 

NOTE I now post my TRADING ALERTS into my personal FACEBOOK ACCOUNT and TWITTER. Don't worry as I don't post stupid cat videos or what I eat!