Big data has been a buzzword in the industry for years, but we are starting to see hard evidence of both enterprises that have been successful with technologies, such as Hadoop, and those that are facing challenges. One of the distinguishing elements of successful enterprises is the holistic approach with which they have approached big data management, governance and security.

There is clear value to using technologies, such as Hadoop, to build next generation "data lakes" for collecting, preparing, and analyzing greater volumes and types of data. Enterprises are augmenting their traditional data warehousing architectures to include Hadoop both as a more efficient and scalable preparation stage and to offload less frequently used data for easily access archives. But as more enterprise data moves into Hadoop for collection, preparation, and analysis, concerns around security can arise. Compliance-sensitive industries, such as healthcare or financial services, or any other consumer-driven industry, such as retail or CPG, are legally obligated to ensure strict controls on the use of data. These controls must also apply to new data platforms, such as Hadoop.

The risks of data breaches are very clear. Most conservatively, data breaches lead to a loss in customer trust and revenue shortfalls from consumer churn. More drastically, we hear of enterprise executives who face fines or incarceration for failing to meet government mandated compliance policies for the collection and access of sensitive consumer information. The risks of data breaches are all the more complicated by a trend toward more autonomous and self-service access to data by an increasingly information-driven workforce. In a world where data is becoming progressively more strategic to the enterprise, organizations have the opportunity to either manage data as an asset or face the risk of it becoming a liability.

Successful enterprises are taking a holistic approach to big data security by evolving beyond traditional perimeter and endpoint-based security approaches and addressing the security of data itself at multiple levels.

1) Authentication. Most Hadoop distributions have native Kerberos-based systems for controlling access to data. Enterprises are enabling Kerberos-based
access controls and using data preparation technologies that fully integrate with these control systems.