Unlock Safety & Savings: Mastering a Secure, Cost-Effective Cloud Data Lake

Abstract: 

Have you ever experienced a surge in your cloud data lake expenses? Is this surge indicating a malicious activity or a legitimate operation? Data lakes have become a cornerstone of the digital age, prized for their flexibility and cost-effectiveness. Yet, as they expand, they bring forth challenges in security, access control, cost management, and monitoring. The stakes are high: unauthorized access can lead to data breaches, while even legitimate users can inadvertently drive up costs.

With the growth in usage comes far more complexity. The size of data, together with the number of objects, are growing rapidly. A growing number of users, both human and application, are performing constant operations on the data lake. The large number of operations makes access and cost control a hard and ongoing task. Monitoring is also a complex task, since there are many access options, and all should be monitored.

Traditional monitoring methods often fall short. Tracking object store access can be overwhelming, with a single query generating thousands of log records. Monitoring at the query engine level demands a unique solution for each engine, adding complexity.

Join us as we unveil two novel techniques for data lake monitoring, leveraging both object store logs and query engine logs. Dive deep into our aggregation strategies and discover how anomaly detection can be applied to this consolidated data. We'll explain how enhanced access control mechanisms can fortify your data lake's security, mitigating the risk of data leaks and data corruption. Additionally, we'll shed light on how to harness these insights to minimize the attack surface, identify and fix cost anomalies and system glitches.

Embark on this journey with us and uncover the secrets to optimizing the security and cost-efficiency of your data lake operations.

Bio: 

Johnathan Azaria is a tech lead in data science @ Imperva, specializing in AI-driven security algorithms and digital protection.

Open Data Science

 

 

 

Open Data Science
One Broadway
Cambridge, MA 02142
info@odsc.com

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Youtube
Consent to display content from - Youtube
Vimeo
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google