Big Data Balancing Act

May 22, 2013

Article ImageAs businesses try to make sense of big data, I'm reminded of how contradictory forces can bring balance to a situation in order to move it forward. In my opinion, there is an over-emphasis on the technology side of big data, and not enough focus on the business side.

Lots of new data scientists will be needed in the years ahead to interpret all of the data we are generating today. However, we also need a new set of talents and strategies that allow us to act upon these insights. To truly exploit big data, the interdependency between technology and business must be addressed.  

Here are four areas worth considering:

Define the business problem before diving into the technology.

Research conducted by AIIM found that 70% of business and technology professionals have in mind a killer application for big data that could be used for competitive advantage, running the business smoothly or detecting non-compliance. But given technical hurdles and missing expertise, a better approach is to start with a defined business problem and use big data to harness skills, tackling the killer app.

To implement a big data killer app requires business acumen in addition to process-driven, statistical analysis - expertise a company has in its personnel but usually not all in one person. Data scientists can't be expected to do it alone. A combination of data science, business, IT, and analytical skill sets is needed. As data sets grow larger they have more potential, but also become less reliable without the appropriate level of expert analysis from a practical business as well as technical perspective. The killer app is out there, but should wait until the real business problems are tackled first.

Michael Schrage of MIT's Sloan School's Center for Digital Intelligence says, "The business needs to move from being data-driven to being algorithmically aware." Businesses need to break down internal barriers to create a team that can focus on what and how to achieve new goals in order to manage exploding data volumes. With all eyes on realistic problems you want to solve, they're more likely to be solved expediently.

Understand which people can deliver big data results for your business.

Data scientists need multi-skilled information professionals. These are people who work across disciplines such as social media, content management, and governance.

While the scientist is firmly grounded in the data and technology, the information professional can approach the big data issues as an entrepreneur would, creating a balance that leads to innovation and discovery. At the heart of this balance is an understanding of how the content assets of an organization can be optimized to meet the customer needs identified by analytics.

Move from big data experimentation to experiment-driven big data.

The advantage of big data is speed and the ability to respond to change, or even the ability to predict change. Success hinges on a process that comfortably supports the concept of hypothesis testing as a basis to formalize big data analysis and allow integration into business processes.

This new way of working - generating lots of hypotheses and testing as many as possible - may seem counterintuitive to traditional data management practices, but lots of experiments running quickly will uncover both the successes and failures needed to move closer to an answer. Ruling out an idea or area of investigation as not relevant to the particular business goal is completely acceptable and allows the focus to quickly shift toward solutions.

In many ways, big data should operate the way managers look at the world, but through a much bigger lens. Take a global sales manager looking to smooth revenues from quarter to quarter. Big data provides the opportunity to quickly test various hypotheses on the regional level as well as the macro level. Analysis that maps to business problems often yields the best insights.

Strike a balance between analysis and action.

When it comes to big data, one size does not fit all. The goal is to deliver the right insight at the right time as well as recognize new opportunities that pertain to your business should they arise.  Analysis of large amounts of data can lead to varying results without hypothesis-driven analysis that is grounded in business realities. 

The benefit of insight needs to be more than simply a collection of interesting results. The goal of producing insight that can be turned into action is hard to achieve without the clarity of a measurable business benefit from the start of a big data project.

In an era where companies are looking for big outcomes, balancing business problems with technology solutions is the key to harnessing the power of big data.

(Big Data image courtesy of Shutterstock.)