Big Data is a Big Deal

Businesses understand that better information delivered to more decision makers leads to more and better business decisions, but even such as simple understanding is made complex by a variety of factors.

• Data is fragmented among siloed data repositories, access is limited and automated data delivery of the right information to the right person at the right time is a rarity.
• Business intelligence (BI) and analytics do exist in many companies, but are isolated, narrowly focused and delivered to a privileged few. Most companies are drowning in data but starved for information.
• The transformation of business data into business intelligence is a costly and technical process. Data is stored in many data siloes, and getting it through the ETL (Extract Transform Load) process and into data visualization tools is slow and costly.
• The challenges are being exacerbated by the growth and variety of data types. Data increasingly no longer resides in nicely formatted relational databases on company servers. Instead, 80 percent of the world’s information is now unstructured, and unstructured data is growing 15 times faster than structured data.

The increasing rise of data volumes, velocities and varieties have defined the concept of Big Data, and more importantly a new opportunity to better transform data from raw form into business intelligence.

Big Data is filling a void by delivering information faster and at a lower cost. Unlike conventional business intelligence or analytics solutions which incur a rigid and technical process to convert data into a common format for subsequent analysis, Big Data normally leaves the data in its native form and instead provides flexible search, access or synchronization tools to bring data types together for analysis when needed. This process removes much of the ETL procedures and reduces or eliminates the technical requirements to normalize, cleanse and stage the information with underlying metadata before making information available for analysis, interrogation or modeling.

Big Data has emerged as a response to the challenge of accessing and synchronizing more and different types of data across disparate sources in order to achieve holistic views, uncover patters, detect anomalies or deliver insights which solve problems.

Making Big Data Little

But evolving voluminous and complex data into a valuable information asset isn’t easy. In fact, in my experience, I think the critical success factor that challenges most is the ability to filter big data into little data.

Big Data becomes helpful when it is advanced from its raw form into relevant and contextual information that benefits employees in specific decision making use cases. Little data is the specific and actionable information gleaned from the volumes of unorganized data that lends itself to aid decisions with measurable results.

To get from big data to little data requires a data driven culture, data stewards (generally analysts), data governance, a technology platform, and presentation or analysis tools.

For many organizations, better decision making requires a cultural shift which expects data-driven, fact-based decisions, and does not accept unsupported or gut-feel conclusions. Executive sponsors need to champion an internal emphasis on optimizing business performance through quantitative measurements. “Show me the data” is the mandate when executives are being asked to approve decisions or recommendations.

Finding the right data stewards to define data governance and implement data management processes can be a challenge. Complex analytics have historically been relegated to statisticians, analysts, PhDs, data scientists or other highly cerebral thinkers. But such titles are not within the org charts of most companies, and integrating these roles with line of business managers to solve business problems can be a challenge. New technologies and a new breed of data stewards are finding that a mix of technical and business skills, whether from a single person or members of a tightly aligned team, are producing successful results.

And interestingly, these roles may or may not report to IT. In fact, with increasing frequency, analysts are operating in a more decentralized environment closely aligned with departmental or business functions. According to Forrester’s James Kobielus, data analytics teams are usually organized by business function or placed directly within a business unit. Kobielus advises that developing, testing and maintaining complex analytical data models requires strong domain and business knowledge, a requirement that doesn‘t easily lend itself to centrally controlled analytics.

Technology is clearly required to capitalize on the growing volumes of data inside and outside the company, and while Hadoop has become a commonly discussed go-to data engine, a complete platform solution beyond simple compute and storage is required to apply business rules, enrich data, enforce integrity checks, uphold governance policies, implement workflow processes and withstand regulatory or privacy concerns.

The last mile in making big data small is often the biggest challenge. Delivering little data in context with a business use case and delivered to decision makers in a way that insights are easily consumed and acted upon is a tall order. To aid the challenge, data insights should be tailored for specific roles and included in the applications, devices and channels where decision makers spend their time, and in a way where the insights are aligned or joined with their existing presentation technologies. For example, rather than requiring a separate endpoint for big data presentations, it’s far more effective to include big data insights with existing decision support systems, or within existing business systems such as CRM, ERP and HCM software applications.

Big Data Use Cases

Big Data objectives are often born from difficult business challenges. In fact, no Big Data project should commence without specific business use cases for which the technology will deliver predictable results and forecasted ROI. Without a specific business imperative, big data will simply contribute big costs to a budget that cannot afford bloat or cancelled projects.

Here are some big data use cases to stimulate your thinking.

Applying big data solutions to online customer sentiment is an increasingly popular use case. For example, brands are using social monitoring tools to find disgruntled customers (their own or their competitors), correlate the social ID to determine the customer’s identity, link that identity to customer demographics (with something as simple as finance.google.com) to determine if the unhappy customer is a potential fit and then deliver a response. Sometimes the response is to engage, other times to aid with (proprietary or third party) remarkable content or possibly to insert the ailing customer into a relevant nurture campaign.

Companies are also harvesting social data and appending it to CRM or ERP contact records in order to identify the likes and dislikes of customers, segment them pursuant to social criteria, engage them socially and possibly extend messaging to their social spheres. And knowing that customers are generally more receptive to outreach during happy times, brands are monitoring social profiles and triggering their messages to occur on a beautiful day (weather data), after a promotion (LinkedIn update), following a return from vacation (Facebook update), during a company stock surge (based on any stock chart) or even when a prospect’s favorite sports team is victorious.

Other companies are tapping into big data for operational improvements. When I managed the global CRM deployment for the U.S. Department of Commerce, I was amazed at the volume of import/export data this federal bureau makes publicly available. It’s not difficult to make the link between this data and a business’s planning for the next geo location.

SHL, a division of the Corporate Executive Board, has acquired the data and predictive employee information on about 25 million employees globally, including 40 percent of the Fortune 500, in order to harness big data for improved talent acquisition and talent management purposes. These HR analytics permit business leaders to compare and benchmark talent in their companies across staff, groups and even other companies.

There are many sources of data to consider, much of it publicly accessible, such as using weather or meteorological data to plan operational asset management tasks, using census data to plan target markets or sales territorial realignments, using geospatial data to improve supply chain logistics, using social listening to get out in front of product defects or minimize product recalls, using crowdsourced data to develop better products or tapping into social networks to understand product trends in real time.

The challenge is coming up with the creative thought to align big data sources with operational drivers that correlate to business outcomes. However, this is a challenge that’s being aided by new and evolving data curators and brokers.

Big Data has evolved from an interesting concept to a proven capability. Although Big Data is frequently overhyped, business leaders should set aside their natural skepticism long enough to evaluate how this disruptive technology may aid their tough business problems. When big data is made small, and delivered timely and in context to users who make that data actionable—and make more informed decisions—big data becomes a valuable information asset that can be leveraged with measurable results.

Comments (11)	— Comments for this page are closed —

	Anonymous
	not so sure i agree of the data explosion that you and others use as a starting point

		Chuck Schaeffer
		From the beginning of time until 2003, humanity created 5 Exabyte’s of information. In the last 12 hours, the world has created 5 Exabyte’s of information. The volume of data expansion continues in an exponential order or magnitude, and in that data are the answers to many problems. The challenge is sifting through the data to get the right information to the right people at the right time, in an automated manner. Early tools are now available to begin capitalizing on this opportunity, and I believe those that take advantage early will benefit early, and further create the culture and know-how to build a learning environment that will apply learning in a way that leapfrogs competitors.

	Frank James
	I agree with the potential power of big data, but I’ve seen projects fail because they weren’t aligned with existing business processes, they failed to tackle the truly difficult business problems, they failed to recognize that data quality still matters with big data and they were unable to measure business outcomes.

		Chuck Schaeffer
		You bring up a good point. Applying thoughtful analysis to understand the inter-relationships among data sets with subsequent actions and outcomes offers a unique opportunity to develop models with predictive insights. However, be cautious in analyzing data without a focus on downstream results and outcomes or you risk falling into the analysis paralysis trap or analyzing data without producing a measurable value to the company.

	Mimi Sasen
	You made a subtle point that I think must be emphasized with big data. Big data is not sloppy data or mismanaged data or data without integrity.

	CFO4EVER
	I think the quantum leap in big data will come as natural language processing (NLP) and sentiment analysis continue to improve to the point where text, audio and video can be harnessed to solve business problems.

	KerryJ
	Revenues can be protected or increased when Big Data is used to detect changes in customer activities and preferences in near real time.

	Denise Johnson
	I like your SHL example. Smart HR leaders have found that winning the war on talent has to less to do with the applicant measures that are readily available (skills, education, tenure) and more to do with personal successes that are discovered apart from the candidates resume.

	Alan Walker
	Customers generally share their opinions (or rants) of a product or service online before contacting the company’s contact center. With sentiment tracking tools, businesses can get an early read on customer complaints, far faster than relying on contact centers, and make a proactive outreach to remedy the problem. The outreach alone will do wonders in winning over upset customers.

	R Chapman
	I'm interested in big data but trying to figure out where all this is heading. What's the future of big data?

		Chuck Schaeffer
		That’s an interesting question but somewhat like asking my 9 year old soon what he will name his first son? So much will change, and so much learning will happen, that today’s prediction will be based on things we’ve not yet thought about and a slew of factors not yet in play. But with said, here’s some of my speculation. First, I believe Big Data will evolve from rich repositories to cloud services to big data platforms tightly integrated with application platforms (such as ERP, SCM, HCM and CRM). Cloud services, online ecosystems and Big Data brokers will evolve to simplify data sourcing and the composition of building blocks (front end visualization analytics with back end data repositories) for devices (i.e. mobile) and channels (i.e. web) offered by vertical market, geo and other market segmentation. I also suspect these data management platforms will evolve to include industry accepted schemas which promote integration and portability, and declarative languages to promote development standards, object classes and orientation, and code reuse. Second, I believe companies will recognize trying to store the increasing volume and types of data is not a practical or wise business decision, and data storage will give way to more cloud services whereby companies subscribe to data only when they need it. Finally, I suspect the technologies will evolve to the point where Big Data will morph with traditional enterprise data warehousing (EDW) technologies in a way that achieves Big Data’s higher flexibility and lower cost of data management and EDW’s higher data quality and advanced analytics.

Share This Article

Big Data is frequently overhyped, but business leaders should set aside their natural skepticism long enough to evaluate how this technology can aid their tough business problems. When big data is made small, and delivered timely and in context to users who make that data actionable—and make more informed decisions—big data becomes a valuable information asset that can be leveraged with measurable results.

Related Articles