Trend #3: From Schema-Constrained to Idea-Constrained, The Real Big Data Opportunity

From my many travels and conversations in 2013, I've synthesized four primary trends that I believe

will make 2014 a transformative year for reporting and analytics.  Think of this as my travel log and diary distilled into a short, easily readable series of blog posts.  I invite you to follow this short series through my first and second installments and now my third below. Your comments and ideas will surely help shape not only my perspective on the future of reporting and analytics, but Jaspersoft's product priorities as well.  I look forward to the on-going dialog and I thank our customers and partners for their business and partnership, which mean everything to us.

In the past (and too often today), we collected just the data that we could afford to store and for which we had a clear, known use.  In this sense, we were hard-wired to winnow the data down to its most obvious and practical subset; thus, we were (and are) schema-constrained. By this I mean that today we must know, in advance, the uses for data as they are being captured.  This mindset leaves little-to-no room for future, latent value that may exist within a data set.  In physics, we recognize that energy has an immediate value (kinetic) and a future value (latent).  Why should data be any different?  

As costs have declined and the power of technology has increased exponentially, we now have the ability to store and use ALL of the data, not just some of the data. But, we may not always know the value of this data while it is being captured.  That’s okay.  The latent value of data will become more obvious each year and the technology now exists for this to be the norm.  In this sense, the real big data opportunity is based on the scale of our ideas to put data to work, finding new correlation and value where it previously was not discernible.  

Unlocking this new value becomes easier as the world is increasingly digitized; that is, we now regularly put very new data types to work:  geo-position / location, sensor updates, click streams, videos and pictures, documents and forms, etc.  Just a few years ago, almost none of this would have been considered “data”. Commonly using all these new data types and searching for correlations that can positively impact business will shift the primary constraint to the quality and quantity of our ideas.  

Perhaps my favorite example of latent data use is the world’s first consumer light field camera from Lytro.  The Lytro camera captures and processes the entire light field (11 million rays of light, to be precise).  This allows every shot to be filtered and viewed from any perspective, after the shot, allowing the user to later uncover what the best focus, angle, and zoom, might yield.  Lytro refers to the result as a “living picture” with a 3-dimensional feel.  What a beautiful use of big, latent data.

In 2014, we’ll move from being schema-constrained to idea-constrained, more often finding the real value in Big Data.