Deep-dive into Azure Sentinel – Part 2 – Data Collection and Processing
From our experience in deploying various SIEM platform, we would rank Azure sentinel number one when it comes to the variety of data collection options it provides. Virtually any log sources and type of data can be ingested in the Azure Sentinel with the different options it provides.
Sentinel comes inbuilt with Data connectors for almost all mainstream System, networking and security solutions. There are around 60+ connectors prebuilt along with default parsers for CEF and Syslog
Types of Supported Data Connectors:
Service to service integration: Integrating any Azure Cloud Services like, Azure Active Directory, o365, all Azure Security solutions can be done in just two clicks to open the Connector page and click the Connect Button. As its already part of the same platform, it automatically detects and configures the parameters.
External APIs: Sentinel can connect to External APIs like AWS CloudTrail using APIs and Secret to collect logs and Matrices.
Azure sentinel Agent: Azure Sentinel features an enterprise class log analytics agent, that can be installed on Windows or Linux Hosts. It can read Windows Events, Linux Logs, Any custom Log files and metrics from hosts. The agent can be configured to Listen for syslog or CEF based messages and forwarded to Log Analytics Workspace with the help of Linux syslog Server applications like syslog-ng or rsyslog.
The Agents can be centrally managed from Azure Log Analytics workspace. Configuration for all the log sources and parsing are configured right from the Log Analytics Workspace UI
LogStash and FluentD Support: This is probably the best part about Sentinel Log Collection. LogStash and Fluentd are considered to be the best Log collectors across all SIEM platforms that supports almost all kind log Inputs ever required by a security Analyst and they natively support output to Azure Sentinel.
LogStash (https://www.elastic.co/logstash) used to be the Go-to log collection agent for Elasticsearch until Filebeat was introduced. LogStash is opensource and support a huge list if Input collection methods. It supports a variety (https://www.elastic.co/guide/en/logstash/current/input-plugins.html) of Input methods that includes by not limited to, Syslog, Application logs, ODBC, Message Queues, HTTP, almost all Cloud Services, SNMP, File and Flow data. Once collected by LogStash, the data can be parsed, transformed and forwarded right in to Sentinel.
Fluentd (https://www.fluentd.org/) is another Opensource Log collector that compete with Logstash in case of data input (https://www.fluentd.org/datasources).
Support for both these tools makes Sentinel virtually integrate any log sources in an IT/OT network.