The State of Big Data 2018

Article ImageBig Data, that corpus of global digital information characterized by velocity, variety, and volume—with contributions from just about every being and machine on the planet—has achieved such a scope and speed of growth that any attempt to quantify it is outdated as soon as it’s measured. If, in the last year, Amazon sold 636 items per second on Amazon Prime Day, YouTube saw 300 hours of video uploaded by users every minute, and Google handled 3.5 billion searches per day, then count on 2018 to bring more of the same.

If 2011 was the year that “data” began answering to “Big Data,” Tamara Dull, director of emerging technologies for SAS’ Best Practices team, says that in 2017, “IoT ripped the ‘big’ right off Big Data’s face.” Dull says, “The story isn’t as much about Big Data, but rather, the Big Data technologies that allow organizations to store and process all kinds of data—structured, semi-structured, and unstructured—at a fraction of the cost and time of traditional technologies.”

That change in emphasis means a permanent shift to figuring out the best way to pull out what’s needed for the people who need it—and doing so in a way that supports their decision making without requiring a data-science degree. It means integrating disparate datasets to create more personalized end-user experiences. As 5G wireless moves closer to fruition, there will be implications for when, where, and how Big Data is accessed by an increasingly mobile userbase. And while pushing data to the edges for faster decision making has its appeal, concerns around data security and privacy abound, especially after a year when data breaches dominated headlines and set consumers on edge.

With Big Data growth getting a boost thanks to such things as your organization’s customer relationship management (CRM), the U.S. president’s Twitter account, and your mother-in-law’s smart thermostat, there’s no time to lose in tackling these challenges.


The Big Data Year in Review

The growth of the Internet of Things (IoT) in commercial and industrial usage is one of the main drivers of Big Data’s expansion in uncharted directions. Stan Lequin, VP of consulting services for Insight, a technology provider of hardware, software, and service solutions to business and government clients, says, “We work with companies that have been around for 200 years, who are now able to pull client endpoint data and analyze it for the first time,” thanks to IoT. “It’s made predictive maintenance and preventative maintenance easier and created new as-a-service possibilities that are entirely new revenue streams.”

Dull sees the IoT having an impact across the board, with manufacturing, transportation, utilities, and healthcare leading the way. She says, “Any organization, regardless of size or shape, can now ask questions like, What does ‘data-driven’ look like for us? What stories are locked inside our data? And can we make money with our data?”

Who gets to dive into the data to answer those questions has changed. “The democratization of Big Data means that we are seeing data push out to the edges,” says Lequin. “Business intelligence for the masses has really picked up over the past year.” He credits an increase in data accessibility within organizations, easier access to external data sources thanks to APIs, and tools that put visualization and analytical capability into the hands of decision makers.

With regard to security concerns in 2017, Dull points out that the devices networked in the IoT don’t necessarily come with robust built-in security, and that puts the onus on consumers. “We can no longer assume that a manufacturer or an app developer is going to do everything they can to make our experience safe and secure. It is now our responsibility as consumers to become amateur security geeks and privacy freaks.” The upside? IoT manufacturers that create safe, secure devices will have a competitive edge.


A Look Ahead at Big Data

Lequin believes that for Big Data in 2018, all the key capabilities are in place, but their adoption and evolution will speed up. “There will be more accessibility; it will be easier to plug in both internal and external data sources—and the database tools with which it’s all done will be easier to work with,” he says. Dull says there’s an urgency to getting the IoT’s contribution to Big Data right. She says, “I’ve been keeping my eye on three ‘S’ developments for IoT: security, standards, and skills. If these three areas don’t get addressed properly—and sooner rather than later—then it’s game over for IoT.”

Personalization will continue to gain importance, according to Craig Smith, CEO and founder of Trinity Insight, an optimization agency that assists ecommerce brands with managing data, digital marketing, and user-experience efforts. “A key part of Big Data is activation in the customer journey,” says Smith. He cites the example of a shopper buying a pair of children’s cleats in-store and providing an email address during the checkout process. Effective personalization might come in the form of an emailed offer for a complimentary ebook on football, a Facebook sidebar ad for football helmets, and a football equipment catalog in the mailbox 12 months later. Due to the complexity of integrating datasets such as account information, web analytics, and search behaviors, Smith says that type of implementation isn’t mainstream yet. “But it will be par for the course in 5 years.”

With the growing importance of mobile usage, all eyes are on 5G, the next-gen network system that will be characterized by higher speeds and capacity and lower latency than existing cellular systems—or at least they should be. “We’re shocked at how much people don’t know about 5G,” says Lequin. “Our expectation is that 5G will be a part of every client conversation we have in late 2019 and 2020. We know it will create a lot more accessibility.”

Finally, Lequin points to one challenge that pre-dates Big Data by approximately 200,000 years: an aversion to change. “Organizations are averse to change; it’s hard to think of as-a-service revenue streams that bring in revenue monthly rather than all at once.” It’s why Lequin says that to get the most value from their Big Data, organizations need one thing above all: blue-sky visionaries who are internal champions and can guide that transformation from analog to digital.  

Related Articles

There may be no tool more important to the daily life of web users, but more overlooked, than the WCM system. Ask the average web surfer what powers the content he or she consumes, and he or she would likely just look at you dumbfounded. But businesses and publishers know how important WCM systems are to getting their messages out to the masses. While WCM systems may be the foundation much of the web is built on, the industry is changing shape all the time.
Despite the significant uptick in consumer use of mobile technology, there's still a big gap between those marketers who have effectively embraced mobile to provide an engaging user experience and those who have a long way to go. While experts have been predicting a big bump in the use of more sophisticated technology to deliver better mobile experiences through augmented reality (AR), mixed reality, and virtual reality (VR), for instance, those predictions have yet to be fully borne out. Instead, what we saw in 2017 was continued growth in the app marketplaces, the rapid advent of chatbots, and a growing focus on video (specifically live video), to engage audiences.
It's January 2017, and you've turned over the digital calendar page on a new year. Was your resolution to walk an extra 1,000 steps a week this year, to be tracked by your Apple Watch, shared on Facebook for extra accountability, and added to your Lifetick goal-tracking web app? Congratulations—you've both added to and exemplified the state of Big Data in 2017.
Consider, for a moment, the latest staggering stats. Worldwide, people watch an average of 5 hours and 45 minutes of online video weekly—a 34% increase from 2016, based on the results of Limelight Network's "The State of Online Video 2017" consumer survey. By 2021, a million minutes of video content will cross global IP networks every second, according to Cisco; at that rate, it would take you more than 5 million years to watch all the video crossing the network each month. And within 3 years, IP video traffic will account for 82% of all consumer internet traffic, per Cisco.
Another year is gone, and both B2C and B2B marketers still continue to be increasingly focused on content marketing. Despite new technology that may take the content production world by storm, movement has been somewhat slow among the masses of marketers in terms of really leveraging things such as artificial intelligence (AI) and machine learning, despite the predictions of experts in last year's Sourcebook that content would become far more interactive and sophisticated as tools such as IBM's Watson came into play.
In an increasingly high-tech world in which artificial intelligence (AI) and machine learning are rapidly accelerating our understanding of customers and paving innovative pathways to reaching them more intuitively, digital marketers have to ask themselves a logical question: Are the robots coming for my job too? No, not yet. Those tasked with reaching more eyeballs and raising brand awareness online can likely bolster their job security by staying ahead of important trends and carefully watching industry developments.