/
Collection subsystem

Collection subsystem

The Snare Central collection subsystem is a robust group of services that are capable of retrieving data from a variety of different sources, and a range of protocols.


The following services listen on the Snare Central network interfaces, and receive audit and event log data.

Snare Agent Logs: Port 6161, TCP & UDP

This is the default reception port for all Snare Agent log data.
Log data that is sent to this port, is generally assumed to be formatted in a certain way, in particular:

  • Tab delimited data
  • The following fields in order:
    • System name
    • Log Type (eg: SolarisBSM)


By requiring this format, the Snare Central collection system on port 6161 can significantly reduce the amount of scanning that needs to be performed on each and every input line; leading to a commensurate increase in event collection per second (EPS) rates.

Although the Snare Central operates a syslog collection capability, log data from Snare and Epilog agents should always be sent to this port. Sending the data to the Syslog port, will result in significant EPS rate drops, and may also result in logs that are not filed correctly in Snare's data store; leading to Windows event log data that are filed in the "Generic Log" group, and therefore cannot be used with Windows-related report templates.

Syslog: Port 514, TCP & UDP

The Snare Central operates a syslog receiver on both TCP and UDP protocols. Many devices use syslog to distribute log data to a central collection point. Network devices such as routers and firewalls often choose this log distribution method.

It is rare that data arriving via syslog provides consistent information that identifies the log source. Usually, Snare will have to scan each incoming event, line-by-line, and pattern match against a series of potential log format templates in order to match a particular event to a log format such as PIX or perhaps Unix SUDO log data. As such, the speed of collection through syslog, is approximately 20% lower than that of the primary collection port.

SNMPTrap: Port 162, UDP

SNMP traps are used for logging in a number of older network-related appliances. The Snare Central can receive snmptrap data, and make it available for analysis from within the Snare Central interface.

MIB (management information base) files are not supported by the Snare Central. SNMPTrap messages that rely on MIB files for decoding content, will still be processed - but may be presented with the original numeric content included, rather than the translated/enhanced text-based content. FTOKENs, as described in the documentation above, can be implemented to provide translations for often-monitored events.

Browser Collection: Port 6162, TCP

Different browsers, implement slightly different default security profiles for extensions/add-ons. Whilst the Snare for Firefox browser add-on can make a TCP connection to the default Snare Central collection port on 6161, Google Chrome will only allow extensions to connect to a HTTP-compatible server, using the HTTP protocol.

As such, the Snare Central operates a HTTP simulator on TCP port 6162, and accepts encoded data sent from the Snare for Chrome browser extension.

TLS Server: Port 6163

Several newer Snare agents are capable of sending data over a TLS encrypted connection. The Snare Central TLS Server port can receive such data, and integrate the data into the normal Snare Central collection framework.

TLS_AUTH Server: Port 6164

Several newer Snare agents are capable of sending data over a TLS_AUTH encrypted connection. This TLS_AUTH is an extension of TLS protocol where an agent is authenticated before any log data is received from it. For authentication purposes, a same TLS Authentication Key must be configured in Snare Central and Snare agents. A default TLS Authentication Key is set to during installation and it is strongly recommended to update it. A valid TLS Authentication Key must be between 8-4096 characters and allowed characters include A-Za-z0-9~!@$%^*\()_+=`-

Performance

Prior to Snare Central 7.4

The EPS collection rates for earlier versions of Snare Central, are significantly dependent on the underlying hardware. In particular, single-core CPU speed is a reliable indicator of the system's ability to collect data.

On a reasonably modern, workstation-quality system (i7 or equivalent), the TCP server can sustainably collect approximately 16,000 events per second. The UDP collection server can collect events in burst mode, of upwards of 80,000 events per second, without losing data due to CPU limitations, as long as the average EPS rates over the course of an hour, remains at or below the 16,000 EPS threshold.

Higher performance server-quality CPUs, will receive a boost in EPS figures commensurate with their single-core processing capabilities. The Snare Central uses multiple cores where available, to segregate the collection of different log sources (eg: TCP collection is on one core, and UDP is on another), so actual collection rates may be significantly higher for organisations that funnel data to the Snare Central using several different protocols (eg: TCP for servers that have high reliability requirements, UDP for workstations where guaranteed delivery of audit data is less of a concern, syslog for network devices).

Snare Central 7.4.4+

More recent versions of Snare Central include a significantly upgraded collection subsystem. The new collection software can sustain significantly higher events-per-second rates, particularly on multi-CPU systems, due to several factors, including:

  • Migration to a compiled collection infrastructure, rather than a JIT-based scripting language
  • Changing the structure of the software, to utilise resources on multiple CPU cores concurrently.

Sustained collection rates should jump by a factor of 10 or more, on similar hardware, when compared to pre-7.4 servers.

It should be noted, that there are two potential configuration options that may decrease effective EPS rates:

  1. If the "Threat Intelligence" capability has been activated in Snare Central, EPS rates for the collection subsystem will be locked to the maximum ElasticSearch ingest rates.
    1. ElasticSearch ingest rates are significantly lower than Snare Central collection capabilities - generally 50x to 100x slower depending on hardware configuration.
  2. If any objectives have been configured to include a 'Realtime Alert' output component, the Snare Central collection subsystem will send data to a slower fallback mode for realtime delivery. This fallback mode is roughly commensurate with the speed of the previous (Pre 7.4) collection system, and will use a significantly higher proportion of system CPU.


Related content

Collection subsystem
Collection subsystem
More like this
Snare Central Server and the Solaris Agent
Snare Central Server and the Solaris Agent
More like this
Release Notes for Snare Windows Agent v5.3.0
Release Notes for Snare Windows Agent v5.3.0
More like this