Security Data Lake Concept laptop Giacomo Lanzi

What is it for? Hadoop Security Data Lake (SDL)

Estimated reading time: 5 minutes

New cybersecurity threats continue to emerge every day and hackers develop new intrusion techniques to access sensitive data and breach IT systems. This is why it is necessary to collaborate with high-level experts who keep track of new developments in the field of IT security. With the birth and continuous evolution of Big Data, the concept of Data Lake and Security Data Lake has also established itself.

For a company, it is expensive to hire a team that deals exclusively with the internal security of a system, which is why many turn to professionals, using a Security Operations Center as a Service (SOCaaS) This service, offered by SOD, also includes an SDL. Let us now try to understand what it is and what their importance and convenience is.

Security Data Lake: what they are

Security Data Lake Concept Big Data

Un Data Lake è un archivio che include grandi quantità di dati, strutturati e non, che non sono stati ancora elaborati per uno scopo specifico. These have a simple architecture to store data. Each item is assigned a unique identifier and then tagged with a set of metadata.

When a business question arises, data scientists can query the Data Lake in order to discover data that could answer the question. Since the Data Lakes are sources that will store sensitive company information, it is necessary to protect them with effective security measures, however the external data ecosystem that feeds the Data Lakes is very dynamic and new problems could regularly arise that undermine its security.

Users authorized to access Data Lakes, for example, could explore and enrich its resources, consequently also increasing the risk of violation. If this were to occur, the consequences could be catastrophic for a company: violation of employee privacy, regulatory information or compromise of business-critical information.

A Security Data Lake, on the other hand, is more focused on security. It offers the possibility of acquiring data from many security tools, analyzing them to extract important information, mapping the fields following a common pattern.

The data contained in an SDL

There are countless different varieties of data, in different formats, JSON, XML, PCAP and more. A Security Data Lake supports all these types of data, ensuring a more accurate and efficient analysis process. Many companies leverage Big Data to develop machine learning-based threat detection systems. An example, for this eventuality, is the UEBA system integrated with the SOCaaS offered by SOD.

A Security Data Lake allows you to easily access data, making it available, also offering the opportunity for real-time analysis.

Apache Hadoop

It is a set of Open Source programs that allows applications to work and manage an enormous amount of data. The goal is to solve problems that involve high amounts of information and computation.

Apache Hadoop includes HDFS, YARN, and MapReduce. When we talk about Hadoop, therefore, we are referring to all those tools capable of interfacing and integrating with this technology. The role of Hadoop is essential because with them it is possible to store and process data at a very low cost compared to other tools. Furthermore, it is possible to do this on a large scale. An ideal solution, therefore, for managing an SDL.

Security Data Lake Concept laptop

Hadoop Distributed File System (HDFS): is one of the main components of Apache Hadoop, it provides access to application data without having to worry about defining schemas in advance.

Yet Another Resource Negotiator (YARN): It is used to manage computing resources in clusters, giving the possibility of using them to program user applications. It is responsible for managing the allocation of resources throughout the Hadoop ecosystem.

MapReduce: is a tool with which processing logic can be transferred, thus helping developers write applications capable of manipulating large amounts of information in a single manageable dataset.

What advantages does Hadoop offer?

It is important to use Hadoop because with it You can leverage clusters of multiple computers to analyze large amounts of information rather than using a single large computer. The advantage, compared to relational databases and data warehouse, lies in Hadoop’s ability to manage big data in a fast and flexible way.

Other advantages

Fault tolerance: Data is replicated across a cluster, so it can be easily recovered in the event of disk or node errors or malfunctions.

Costs: Hadoop is a much cheaper solution than other systems. It provides compute and storage on affordable hardware.

Strong community support: Hadoop is currently a project supported by an active community of developers who introduce updates, improvements and ideas, making it an attractive product for many companies.

Conclusions

In this article we learned the differences between a Data Lake and a Security Data Lake, clarifying the importance of using these tools in order to guarantee the correct integrity of the IT systems present in a company.

Collecting infrastructure data is only the first step for efficient analysis and the resulting security offered by monitoring, essential for a SOCaaS. Ask us how these technologies can help you manage your company’s cyber security.

For doubts and clarifications, we are always ready to answer all your questions.

Useful links:

Share


RSS

More Articles…

Categories …

Tags

RSS darkreading

RSS Full Disclosure

  • Business Logic Flaw and Username Enumeration in spa-cartcmsv1.9.0.6 June 16, 2024
    Posted by Andrey Stoykov on Jun 15# Exploit Title: Business Logic Flaw and Username Enumeration in spa-cartcmsv1.9.0.6 # Date: 6/2024 # Exploit Author: Andrey Stoykov # Version: 1.9.0.6 # Tested on: Ubuntu 22.04 # Blog: https://msecureltd.blogspot.com/2024/04/friday-fun-pentest-series-5-spa.html Description - It was found that the application suffers from business logic flaw - Additionally the application is vulnerable […]
  • APPLE-SA-06-10-2024-1 visionOS 1.2 June 12, 2024
    Posted by Apple Product Security via Fulldisclosure on Jun 11APPLE-SA-06-10-2024-1 visionOS 1.2 visionOS 1.2 addresses the following issues. Information about the security content is also available at https://support.apple.com/HT214108. Apple maintains a Security Releases page at https://support.apple.com/HT201222 which lists recent software updates with security advisories. CoreMedia Available for: Apple Vision Pro Impact: An app may be […]
  • CyberDanube Security Research 20240604-0 | Multiple Vulnerabilities in utnserver Pro/ProMAX/INU-100 June 9, 2024
    Posted by Thomas Weber via Fulldisclosure on Jun 09CyberDanube Security Research 20240604-0 ------------------------------------------------------------------------------- title| Multiple Vulnerabilities product| SEH utnserver Pro/ProMAX / INU-100 vulnerable version| 20.1.22 fixed version| 20.1.28 CVE number| CVE-2024-5420, CVE-2024-5421, CVE-2024-5422 impact| High homepage| https://www.seh-technology.com/...
  • SEC Consult SA-20240606-0 :: Multiple critical vulnerabilities in Kiuwan SAST on-premise (KOP) & cloud/SaaS & Kiuwan Local Analyzer (KLA) June 9, 2024
    Posted by SEC Consult Vulnerability Lab via Fulldisclosure on Jun 09SEC Consult Vulnerability Lab Security Advisory < 20240606-0 > ======================================================================= title: Multiple critical vulnerabilities product: Kiuwan SAST on-premise (KOP) & cloud/SaaS Kiuwan Local Analyzer (KLA) vulnerable version: Kiuwan SAST
  • Blind SQL Injection - fengofficev3.11.1.2 June 9, 2024
    Posted by Andrey Stoykov on Jun 09# Exploit Title: FengOffice - Blind SQL Injection # Date: 06/2024 # Exploit Author: Andrey Stoykov # Version: 3.11.1.2 # Tested on: Ubuntu 22.04 # Blog: https://msecureltd.blogspot.com/2024/05/friday-fun-pentest-series-6.html Steps to Reproduce: 1. Login to application 2. Click on "Workspaces" 3. Copy full URL 4. Paste the HTTP GET request into […]
  • Trojan.Win32.DarkGateLoader (multi variants) / Arbitrary Code Execution June 9, 2024
    Posted by malvuln on Jun 09Discovery / credits: Malvuln (John Page aka hyp3rlinx) (c) 2024 Original source: https://malvuln.com/advisory/afe012ed0d96abfe869b9e26ea375824.txt Contact: malvuln13 () gmail com Media: x.com/malvuln Threat: Trojan.Win32.DarkGateLoader (multi variants) Vulnerability: Arbitrary Code Execution Description: Multiple variants of this malware look for and execute x32-bit "urlmon.dll" PE file in its current directory. Therefore, we can...
  • SQL Injection Vulnerability in Boelter Blue System Management (version 1.3) June 9, 2024
    Posted by InfoSec-DB via Fulldisclosure on Jun 09Exploit Title: SQL Injection Vulnerability in Boelter Blue System Management (version 1.3) Google Dork: inurl:"Powered by Boelter Blue" Date: 2024-06-04 Exploit Author: CBKB (DeadlyData, R4d1x) Vendor Homepage: https://www.boelterblue.com Software Link: https://play.google.com/store/apps/details?id=com.anchor5digital.anchor5adminapp&hl=en_US Version: 1.3 Tested on: Linux Debian 9 (stretch), Apache 2.4.25, MySQL >= 5.0.12 CVE:...
  • CyberDanube Security Research 20240528-0 | Multiple Vulnerabilities in ORing IAP-420 May 30, 2024
    Posted by Thomas Weber via Fulldisclosure on May 29CyberDanube Security Research 20240528-0 ------------------------------------------------------------------------------- title| Multiple Vulnerabilities product| ORing IAP-420 vulnerable version| 2.01e fixed version| - CVE number| CVE-2024-5410, CVE-2024-5411 impact| High homepage| https://oringnet.com/ found| 2024-01-19 by| T. Weber...
  • HNS-2024-06 - HN Security Advisory - Multiple vulnerabilities in Eclipse ThreadX May 30, 2024
    Posted by Marco Ivaldi on May 29Hi, Please find attached a security advisory that describes multiple vulnerabilities we discovered in Eclipse ThreadX (aka Azure RTOS). * Title: Multiple vulnerabilities in Eclipse ThreadX * OS: Eclipse ThreadX < 6.4.0 * Author: Marco Ivaldi * Date: 2024-05-28 * CVE IDs and severity: * CVE-2024-2214 - High - […]
  • SEC Consult SA-20240527-0 :: Multiple vulnerabilities in HAWKI didactic interface May 28, 2024
    Posted by SEC Consult Vulnerability Lab via Fulldisclosure on May 27 SEC Consult Vulnerability Lab Security Advisory < 20240527-0 > ======================================================================= title: Multiple vulnerabilities product: HAWKI (Interaction Design Team at the University of Applied Sciences and Arts in Hildesheim/Germany) vulnerable version: 1.0.0-beta.1, versions before commit 146967f     fixed version: Github commit 146967f...

Customers

Newsletter

{subscription_form_1}