Azure - Cloud Log Collection Configuration

Owned by Svetlana Sheptiy
Last updated: Mar 04, 2024
18 min read

Introduction

Microsoft Azure, often referred to as Azure is a cloud computing platform run by Microsoft. It offers access, management, and the development of applications and services through global data centers. It also provides a range of capabilities, including software as a service (SaaS), platform as a service (PaaS), and infrastructure as a service (IaaS).

Snare Central is offering a convenient way to collect logs generated by a variety of Azure cloud services and store them in the Snare Central Archive for reporting, analysis and compliance.

This setup guide will cover the basic required setup for the SNARE - Azure cloud log collection to work. Security related setup, charges you may incur, and other intricacies related to Azure will not be covered in detail in this guide.

 

Overview

All applications, whether on-premises or in the cloud, must include logging since it aids in security implementation and debugging. Azure provides services to collect cloud platform logs to ensure optimal application performance.

Snare Central can be configured to collect activity and resource or diagnostic logs from Azure using Log Analytics API.

image-20240115-092102.png
Azure Cloud Log Collection Overview

Snare Central needs to request authentication keys from Microsoft Entra ID in order to connect to the Log Analytics API. Once authentication is accepted, required API permissions are granted, and access control is setup, Snare Central will be able to query the target activity and diagnostic logs using the same API.

 

About Azure platform logs

Below are the types of logs that can be collected from Azure (Snare Central supports the collection some log types, see details here for the supported log types).

Microsoft Entra logs

  • Microsoft Entra logs contain the history of sign-in activity and an audit trail of changes made in Microsoft Entra ID for a particular tenant.

  • Types of activity logs in Microsoft Entra ID: Sign-in logs and Audit logs

Activity logs

  • It provides insight into the operations on each Azure resource and use to determine what, who, and when for any write operations (PUT, POST, DELETE) taken on the resources in your subscription.

  • There's a single activity log for each Azure subscription.

Resource logs

  • Resource logs provide an insight into operations that were performed within an Azure resource, known as the data plane. Examples include getting a secret from a key vault or making a request to a database.

  • Log contents vary according to the Azure service and resource type.

  • Logs aren't collected until they're routed to a destination, to be enabled and configured via Diagnostic settings. While some Azure resource may have some special type of logs (e.g. Azure NSG - Flow logs)

 

Snare Central and Log Analytics API communication

In order for Snare Central to properly communicate to and collect Azure logs from the Log Analytics API, the following steps need to be completed in Azure environment:

  • Register Snare Central in Microsoft Entra ID.

  • Create Log Analytics workspace

  • Set up the workspace Access Control (IAM)

  • Export activity and diagnostic logs to a Log Analytics workspace.

Register Snare Central in Microsoft Entra ID

To allow Snare Central to access the Log Analytics API, Snare Central must be registered in Microsoft Entra ID formerly known as Azure Active Directory (AD). This allows the Snare Central to establish an identity and specify the needed permission levels for the API access.

The Log Analytics API uses Microsoft Entra ID to provide authentication services that you can use to setup the necessary permission rights for Snare Central to access them.

Step 1: App registration

  • Create a dedicated application for Snare Central inside Microsoft Entra ID.

  • Follow steps 1~5 in this Azure user guide, last output screen should look like this:

    az-app-reg.png
    Azure - App registrations page

  • Target output:
    Application (client) ID - Generated by Microsoft Entra ID, Snare Central will use this value when requesting consent from tenant admins and when requesting app-only tokens from Microsoft Entra ID. Make sure to save this value, it will be used during Snare Central’s Azure Cloud log collection configuration.

 

Step 2: Key or client secret generation

  • Generate the necessary client secret that will be used by Snare Central’s authentication towards Log Analytics API.

  • Follow steps 6~10 in the same Azure user guide, last output screen should look like this:

  • Target output:
    Client Secret: Make sure to copy and save the text in the “Value” column for the generated credential. Microsoft Entra ID only displays this value at the time of its generation, it will be masked after that. Also, it will be used during Snare Central’s Azure Cloud log collection configuration.
    Note: A user is allowed to create and use multiple client credentials.

 

Step 3: Setting up APIs permissions

  • Configure and setup the required permissions for Log Analytics API connection and interaction with Snare Central.

  • Follow steps 2~7 in this Azure user guide, step 1 is not needed, since it was already done during App registration and Key or client secret generation.
    Last output screen should look like this:

  • Target output: Permission is set Data.Read, Type is Delegated and Admin consent required is set to No.

Create Log Analytics workspace

When Azure collects logs and data, the information is stored in a workspace. A workspace has a unique workspace ID and resource ID. After you've created a workspace, you can configure the Azure resource(s) to send the activity logs and diagnostic logs to the created workspace.

Snare Central can query and collect logs from the created workspace using the Log Analytics API.

Creating a workspace

  • Note(s):

    • To create a Log Analytics workspace, you need an Azure account with an active subscription.

    • A user may opt to skip this step if the user already has a Log Analytics workspace.

  • Follow steps 1~7 in this user guide.

  • Target output: Workspace resource and unique GUID assigned for it - The unique GUID assigned to the workspace will be used during Azure cloud log configuration on Snare Central.

Set up the workspace Access Control (IAM)

Setting up Access Control (IAM)

  • Note: Without this setup, Snare Central will not be able to proceed with its connection and collection towards Azure, and Snare Central will encounter an error "The provided credentials have insufficient access to perform the requested operation" if not properly done.

  • Configure the required access control by following the steps below.

  • Go to the Azure portral then search for Log Analytics workspace.

  • Select and click the previously created Log Analytics workspace, at the Log Analytics workspace’s main page, go and click the Access Control (IAM).

  • Click Add Role assignment then select Reader role then click Members.

  • At Members page, click + Select Members then search for the name of the application (which was previously created during App registration) then click Select.

  • Click Review + assign and wait for the modification on Role Assignment to reflect on Role Assignments list.

  • To check the configured Role Assignment, go to the Access Control (IAM) page, then Role Assignments, look for the configured Role Assignment, it should be present on the list under Reader role.

Export activity and diagnostic logs to a Log Analytics workspace.

Export activity logs to a Log Analytics workspace

  • This setting allows the activity logs to be dumped into a target log analytics workspace for Snare Central to query and collect those logs using the Log Analytics API.

  • Follow steps below for the required settings.

  • Go to the Azure portal then search for Monitor.

     

  • On Monitor page, click Activity log, then click Export Activity Logs.

  • Click Add diagnostic setting, fill the necessary details, then select the target Log Categories and select Send to Log Analytics workspace and select the target Log analytics workspace (which was previously created) then Save.

  • Wait for 1-2 minutes for the settings to reflect on Azure side.

  • Target output: All Azure activity logs are expected to be dumped into the selected Log Analytics workspace and Snare Central should be able to collect those activity logs.

 

Export diagnostic logs to the Log Analytics workspace

  • This setting allows a resource specific log(s) to be dumped into a target log analytics workspace for Snare Central to query and collect those logs using the Log Analytics API.

  • Follow steps below for the required settings.

  • Go to the Azure portral then search for the target resource, example: Firewall.

     

  • Click the target resource and go to its Diagnostic Settings.

  • Click Add diagnostic setting, fill the necessary details, then select the target Log Categories and select Send to Log Analytics workspace, choose Azure diagnostics and select the target Log analytics workspace (which was previously created) then Save.

     

  • Target output: All logs for the selected Azure resource are expected to be dumped into the selected Log Analytics workspace and Snare Central should be able to collect those activity logs.

  • Note: User needs to repeat the above steps for all the resources that need log collection.

 

Setting Up Snare Central - Azure Cloud Log Collection

Starting from Snare Central v8.6.0, Azure Cloud Log Collection functionality is available as long as you have the proper license for it.

This capability requires a license with either Office 365 Logs Collection (IA_CLOUD_O365) or Cloud Logs Collection (IA_CLOUD) license features.

This guide will help you setup up your Snare Central and start collecting supported Azure activity and resource logs in no-time by simply using the intuitive Cloud Log Collection Configuration Web UI of Snare Central.

For more info about the supported Azure Log types, see: Supported Azure Log Types.

Step 1. Go to Snare Central and navigate to System > Administrative Tools > Cloud Log Collection Configuration.

Step 2. Select Azure Cloud and Click ADD CLOUD COLLECTION button.

Step 3. Input all the necessary Azure Cloud Collection Configuration Information and click Test Connection Button to check if the configuration is correct and can properly connect to the Log Analytics API.

  • Name: Any name to easily identify this Azure Cloud Log Collector.

  • Enabled: Can be toggled ON/OFF. This will determine if the Azure Cloud Collector will be enabled and start log collection (This can also be toggled ON/OFF easily later after setup).

  • Supported Logs: Identifies the target log type/s to be collected collector, see supported log types here.

  • Fetch Interval: Log collection interval (in millisecond) for each log collection request, should not be less than 120000 or greater than 900000.

  • Domain: Fetch the Primary domain value on the main page of the Azure portal site.

  • Tenant ID: Fetch the Tenant ID value on the same Azure portal site.

  • Organization ID: The organization or company name or you can fetch the Name value on the same Azure portal site.

  • Subscription ID: Fetch the user’s subscription ID using this user guide.

  • Workspace ID: Fetch the Workspace ID that was generated during Log Analytics workspace creation.

  • Client ID: Fetch the Application (client) ID that was generated during App registration.

  • Client Secret: Fetch the “Value” that was generated during Key or client secret generation.

  • Additional Notes

    • For optional field(s) and information about the collector you may use to Note part of the Azure Cloud Log Collector.

    • Test Connection can be used to check whether the input configuration parameters are correct or not.

Step 4. Click ADD button, then you should be able to see the added Azure Cloud Log Collector under the Azure Cloud Collection List.

 

 

Updating/Deleting - Azure Cloud Log Collection Configuration

If you want to update or delete an existing Azure Cloud - Log Collector that were previously configured, you can simply use the Snare Central’s Cloud Log Collection Configuration Web UI and follow the simple steps below.

 

Troubleshooting Guide

This guide will be your resource for resolving common issues and challenges that you may encounter with Azure - Cloud Log Collection.

  1. Azure Cloud icon is gray in System > Administrative Tools > Cloud Log Collection Configuration Web UI.

  1. Azure Cloud Log Collector icon is gray, and the Status is Not Running (Disabled by configuration)

  1. Azure Cloud Log Collector icon is red, and the Status is Not Running (Could not generate access token for ….)

  1. Azure Cloud Log Collector icon is red, and the Status is Not Running (The provided credentials have insufficient access …)

  1. Azure Cloud Log Collector icon is red, and the Status is Not Running (The workspace could not be found for logtype ...)

  1. Azure Cloud Log Collector icon is red, and the Status is Not Running (Cannot connect to microsoft API, please check the IP configuration …)

  1. Azure Cloud Log Collector icon is red, and the Status is Not Running (Cannot connect to proxy server, ….)

  1. Azure Cloud Log Collector icon is red, and the Status is Not Running (Invalid proxy credentials...)

  1. Azure Cloud Log Collector icon is red, and the Status is Not Running (Invalid proxy type…)