This content is part of the Essential Guide: Maximizing and managing big data with SOA middleware
Manage Learn to apply best practices and optimize your operations.

More organizations take on big data initiative

More than 40% of organizations involved in big data initiatives expect to see an ROI of more than 25%.

More than 50% of businesses were engaged in big data initiatives last year, according to a Tata Consultancy Services (TCS) Ltd. survey of 1,217 big data and analytics, IT management, and sales and marketing professionals. Yet, while many organizations are striving to make better use of big data, most encounter problems in the notorious three Vs: volume, variety and velocity, said Dr. Satya Ramaswamy, TCS vice president. Overcoming these hurdles calls for restructuring workgroups, as well making as other technology and organizational adjustments.

big data return on investment (ROI) chart
Study results: 2012 expected company big data ROI.

Of the companies that undertook big data in 2012, just over 40% predict seeing a return on investment (ROI) of more than 25%, data indicates. Ramaswamy believes such findings indicate more organizations could be giving the technology the green light down the road.

Conquering big data initiative roadblocks

After the survey, the TCS research team was surprised to learn that four of the top five challenges reported were structural, not technical, said Ramaswamy. The top-rated challenge was business units' failure to share information across the organizational silos. Another hurdle is building trust between data scientists and IT managers.

Eliminating separate departments and creating cross-functional teams can address the collaboration challenges. "One of the best practices we saw is companies setting up independent units that are different from traditional units to drive the big data initiative," said Ramaswamy.

Main big data challenges
Study results: Top big data challenges.

Forrester Research Inc. principal analyst Mike Gualtieri advises that two core practices are musts to drive structural change:

Having a data platform for big data analytics is crucial because data silos have proliferated. "Even the data warehouse has become just one more silo," said Gualtieri.

The difference between business intelligence (BI) and predictive analytics is vast. Traditional BI applications are used to determine which data is important, whereas predictive analytics uses machine learning algorithms to obtain the same information. "That's why it is important to store and analyze all your data," Gualtieri said. "You don't know upfront what is the signal from the noise."

Breaking down technical barriers

Handling the three Vs was cited as the top technical issue, largely because analysis calls for considering all the data, not just what is immediately important. "One challenge for the velocity of big data is to capture and store it as fast as it comes at you," said Gualtieri. "Streaming technology is a critical component of a big data platform." Use scale-out architectures and tools like Hadoop to manage fast-growing, large quantities of data, he advised.

Clearly, one thing I see [is that] the adoption of big data is going to become universal.
Dr. Satya Ramaswamyvice president, TCS

Choosing the appropriate big data platform is critical to overcoming technical issues, Ramaswamy noted. One simple, yet sometimes overlooked, tactic for selecting platforms is for architects to make sure they clearly understand their organization's priority on big data.

"Architects should investigate all specific areas outlined by the enterprise architecture for the big data platform fitment," said Ramaswamy. "Smaller proof of concepts/technology in the specific areas could be done to make sure each and every one of the guidelines is met."

Other challenges and good news

Staffing and budgets are related challenges for the IT managers and big data/analytics pros surveyed. Getting top management to invest in big data initiatives, particularly in training, is a top problem for IT management. Analytics pros have trouble finding and hiring data scientists capable of managing large amounts of structure and unstructured data and providing creative analysis and action.

Both groups need to work together to inform top management that investing in big data practices provides competitive advantages and, as stated above, gains strong ROI, says Ramaswamy. Also, he noted, companies that have created and funded big data analytics centers of excellence have been successful in attracting and retaining data scientists.

The TCS survey showed that almost all respondents who did not have a big data program said they will have one by 2015. "Clearly, one thing I see [is that] the adoption of big data is going to become universal," said Ramaswamy. The obvious conclusion is that companies that don't invest in big data processes will be in the minority in the near future and, thusly, lose competitive advantage.

Follow us on Twitter at @SearchSOA and like us on Facebook !

Dig Deeper on Application development planning

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.