Microsoft Azure - Cloud Log Collection Configuration

Introduction

Microsoft Azure, often referred to as Azure is a cloud computing platform run by Microsoft. It offers access, management, and the development of applications and services through global data centers. It also provides a range of capabilities, including software as a service (SaaS), platform as a service (PaaS), and infrastructure as a service (IaaS).

Snare Central is offering a convenient way to collect logs generated by a variety of Azure cloud services and store them in the Snare Central Archive for reporting, analysis and compliance.

This setup guide will cover the basic required setup for the SNARE - Azure cloud log collection to work. Security related setup, charges you may incur, and other intricacies related to Microsoft Azure will not be covered in detail in this guide.

Overview

All applications, whether on-premises or in the cloud, must include logging since it aids in security implementation and debugging. Azure provides services in order collect cloud platform logs to ensure optimal application performance.

Snare Central can be configured to collect activity and resource or diagnostic logs from Log Analytics API.

Snare Central needs to request authentication keys from Microsoft Entra ID in order to connect to the Log Analytics API. Once authentication is accepted and the required API permission(s) were setup, Snare Central will be able query the target activity and diagnostic logs using the same API.

About Azure platform logs

Below are the types of logs that can be collected from Azure (Snare Central supports the collection some log types, see details here for the supported log types).

Microsoft Entra logs

Microsoft Entra logs contain the history of sign-in activity and an audit trail of changes made in Microsoft Entra ID for a particular tenant.
Types of activity logs in Microsoft Entra ID: Sign-in logs and Audit logs

Activity logs

It provides insight into the operations on each Azure resource and use to determine what, who, and when for any write operations (PUT, POST, DELETE) taken on the resources in your subscription.
There's a single activity log for each Azure subscription.

Resource logs

Resource logs provide an insight into operations that were performed within an Azure resource, known as the data plane. Examples include getting a secret from a key vault or making a request to a database.
Log contents vary according to the Azure service and resource type.
Logs aren't collected until they're routed to a destination, to be enabled and configured via Diagnostic settings. While some Azure resource may have some special type of logs (e.g. Azure NSG - Flow logs)

Snare Central and Log Analytics API communication

In order for the Snare Central to properly communicate and collect Azure logs using Log Analytics API, these things need to be created and setup first on Azure environment:

Register Snare Central in Microsoft Entra ID.
Setting up application Access control (IAM)
Create Log Analytics workspace.
Export activity and diagnostic logs towards Log Analytics workspace.

Register Snare Central in Microsoft Entra ID

To allow Snare Central to access the Log Analytics API, Snare Central must be registered in Microsoft Entra ID formerly known as Azure Active Directory (AD). This allows the Snare Central to establish an identity and specify the needed permission levels for the API access.

The Log Analytics API uses Microsoft Entra ID to provide authentication services that you can use to setup the necessary permission rights for Snare Central to access them.

3 important steps when registering an application:

Step 1: App registration

Create a dedicated application for Snare Central inside Microsoft Entra ID.
Follow steps 1~5 on this user guide, last output screen should look like this:
Target output:
Application (client) ID - Generated by Microsoft Entra ID, Snare Central will use this value when requesting consent from tenant admins and when requesting app-only tokens from Microsoft Entra ID. Make sure to save this value, it will be used during Snare Central’s Azure Cloud log collection configuration.

Step 2: Key or client secret generation

Generate the necessary client secret that will be used by Snare Central’s authentication towards Log Analytics API.
Follow steps 6~10 on the same user guide, last output screen should look like this:
Target output:
Client Secret: Make sure to copy and save the text in the “Value” column for the generated credential. Microsoft Entra ID only displays this value at the time of its generation, it will be masked after that. Also, it will be used during Snare Central’s Azure Cloud log collection configuration.
Note: A user is allowed to create and use multiple client credentials.

Step 3: Setting up APIs permissions

Configure and setup the required permissions for Log Analytics API connection and interaction with Snare Central.
Follow steps 2~7 on this user guide, step 1 is not needed it was already done during App registration and Key or client secret generation.
Last output screen should look like this:
Target output: Permission is set Data.Read, Type is Delegated and Admin consent required is set to No.

Setting up application Access control (IAM)

To access resources under your subscription, you must assign a role to the application (previously created during App registration). Since Snare Central was associated with the said application, the role assigned to the application will be the basis of Snare Central’s access control towards the Azure subscription.

Without this step, Snare Central will not be able to proceed with its connection and collection towards Azure and you may encounter error: The provided credentials have insufficient access to perform the requested operation if not properly done.

Setting up application’s Access Control

Setting up application’s Access Control

Configure the required access control by following the steps below.
Go to the Azure portral then search for Subscriptions.
Select and click the target subscription, at the subscription’s main page, go and click the Access Control (IAM).
Go to Role assignments, then click Add role assignment.
Then select Reader then Next, click the Select Member then search for the name of the application (previously created during App registration) then click Select.
Click Review + assign and wait for change on Role Assignment to reflect.
Target output: Previously created application should have a Reader access control for all resources under your subscription.

Create Log Analytics workspace

When Azure collects logs and data, the information is stored in a workspace. A workspace has a unique workspace ID and resource ID. After you've created a workspace, you can configure the Azure resource(s) to send the activity logs and diagnostic logs to the created workspace.

Snare Central can query and collect logs from the created workspace using the Log Analytics API.

Creating a Log Analytic workspace

Creating a workspace

Note(s):
- To create a Log Analytics workspace, you need an Azure account with an active subscription.
- A user may opt to skip this step if the user already has a Log Analytics workspace.
Follow steps 1~7 on this user guide.
Target output: Workspace resource and unique GUID assign for it - The unique GUID assigned to the workspace will be used during Azure cloud log configuration on Snare Central.

Export activity and diagnostic logs towards Log Analytics workspace.

Export activity and diagnostic logs.

Export activity logs towards Log Analytics workspace

This setting allows the activity logs to be dumped into a target log analytics workspace for Snare Central to query and collect those logs using the Log Analytics API.
Follow steps below for the required settings:
- Go to the Azure portral then search for Monitor.
- At Monitor page, look for Activity log then click it, then click Export Activity Logs.
- Click Add diagnostic setting, then fill up the necessary info, then select the target Log Categories and select Send to Log Analytics workspace and select the target Log analytics workspace (which was previously created) then Save.
- Wait for 1-2 minutes for the settings to reflect on Azure side.
Target output: All Azure activity logs is expected to be dumped into the selected Log Analytics workspace and Snare Central should be able to collect those activity logs.

Export diagnostic logs towards Log Analytics workspace

This setting allows a resource specific log(s) to be dumped into a target log analytics workspace for Snare Central to query and collect those logs using the Log Analytics API.
Follow steps below for the required settings:
- Go to the Azure portral then search for the target resource, example: Firewall.
- Click the target resource and go its Diagnostic Settings
- Click Add diagnostic setting, then fill up the necessary info, then select the target Log Categories and select Send to Log Analytics workspace, choose Azure diagnostics and select the target Log analytics workspace (which was previously created) then Save.
Target output: All logs for the selected Azure resource is expected to be dumped into the selected Log Analytics workspace and Snare Central should be able to collect those activity logs.
Note: User needs to repeat the above steps for all the resource that needs log collection.

Setting Up Snare Central - Azure Cloud Log Collection

Starting from Snare Central v8.6.0, Azure Cloud Log Collection functionality will be available as long as you have the proper license for it.

This capability requires a license with either Office 365 Logs Collection (IA_CLOUD_O365) or Cloud Logs Collection (IA_CLOUD) license features.

This guide will help you setup up your Snare Central and start collecting supported Azure activity and resource logs in no-time by simply using the intuitive Cloud Log Collection Configuration Web UI of Snare Central.

For more info about the supported Azure Log types, see: Supported Azure Log Types.

Step by Step Guide for Setting Up Snare Central - Azure Cloud Log Collection

Step 1. Go to Snare Central and navigate to System > Administrative Tools > Cloud Log Collection Configuration.

Step 2. Select Azure Cloud and Click ADD CLOUD COLLECTION button.

Step 3. Input all the necessary Azure Cloud Collection Configuration Information and click Test Connection Button to check if the configuration is correct and can properly connect to the Log Analytics API.

Name: Any name to easily identify this Azure Cloud Log Collector.
Enabled: Can be toggled ON/OFF. This will determine if the Azure Cloud Collector will be enabled and start log collection (This can also be toggled ON/OFF easily later after setup).
Supported Logs: "Identifies the target log type/s to be collected collector, see supported log types here.
Fetch Interval: Log collection interval (in millisecond) for each log collection request, should not be less than 120000 or greater than 900000.
Domain: Fetch the Primary domain value on the main page of the Azure portal site.
Tenant ID: Fetch the Tenant ID value on the same Azure portal site.
Organization ID: The organization or company name or you can fetch the Name value on the same Azure portal site.
Subscription ID: Fetch the user’s subscription ID using this user guide.
Workspace ID: Fetch the Workspace ID that was generated during Log Analytics workspace creation process above.
Client ID: Fetch the Application (client) ID that was generated during App registration process above.
Client Secret: Fetch the “Value” that was generated during “Key or client secret generation” process.
Additional Notes
- Optional field that you may use to Note any related information to this Azure Cloud Log Collector
- Test Connection can be used to check whether the input configuration parameters were correct or not.

Step 4. Click ADD button, then you should be able to see the added Azure Cloud Log Collector under the Azure Cloud Collection List.

Updating/Deleting - Azure Cloud Log Collection Configuration

If you want to update or delete an existing Azure Cloud - Log Collector that were previously configured, you can simply use the Snare Central’s Cloud Log Collection Configuration Web UI and follow the simple steps below.

Step by Step Guide for Updating Snare Central - Azure Cloud Log Collection

Step 1. Go to Snare Central and navigate to System > Administrative Tools > Cloud Log Collection Configuration.

Step 2. Select Azure Cloud and Click the Azure Cloud Log Collector that you want to update.
Collector details panel will open on the right-hand side.
Click the Edit icon on the top right of the details panel.

Step 3. In the Edit screen, you can update the configuration and optionally do a Test Connection to check if the updated configuration can successfully connect to the Log Analytics API, then simply click the SAVE button to save the updated configuration.

Step by Step Guide for Deleting Snare Central - Azure Cloud Log Collection

Step 1. Go to Snare Central and navigate to System > Administrative Tools > Cloud Log Collection Configuration.

Step 2. Select Azure Cloud and Click the Azure Cloud Collector that you want to delete.
Collector details panel will open on the right-hand side.

Click the Delete icon on the top right of the details panel.