Installing Illuminate 2.0
  • 01 Feb 2022
  • 7 Minutes to read
  • Dark
    Light

Installing Illuminate 2.0

  • Dark
    Light

Version 2.1

Graylog Illuminate 2.1.x on Graylog Enterprise is the most current version available. Contact your sales representative for information on obtaining the Graylog Illuminate release file.

This guide assumes you have at least Graylog Illuminate 2.0.0 on Graylog Enterprise. It does not cover upgrading Illuminate from versions 1.x to 2.x. Refer to Upgrading to Illuminate 2.0 before starting this guide.

Currently, you can download and install these versions:

  • Illuminate 2.0.x
  • Illuminate 2.1.0

Prerequisites

Prior to installation, you must have a Graylog Enterprise server, at least version 4.2.0, with a current Enterprise license.

Installing Graylog Illuminate 2.x.x

Extract the Release File

The Graylog Illuminate 2.x.x release file can be extracted using a zip-format compatible archive tool. The contents differ based on version:

Graylog Illuminate Version Directory Name Archive File
2.0.0 ../graylog_illuminate.v2_0_0 illuminate-bundle.v2_0_0.zip
2.0.1 ../graylog_illuminate.v2_0_1 illuminate-bundle.v2_0_1.zip
2.1.0 ../graylog_illuminate.v2_1_0 bundle.v2_1_0.zip

The version you install can be extracted into a directory named graylog_illuminate.v2_x_x. Inside this directory, there will be a release notes file, the Illuminate ”bundle” archive file named <archive_file_name>.v2_x_x.zip, and a Spotlights directory that contains all of the Graylog Illuminate Spotlight content pack files you have obtained.

Here’s an example of the file structure on an Ubuntu distribution:

Install_illuminate_2x_1

The Illuminate bundle archive file contains all of the technology packs that perform the identification, parsing and normalization, categorization, and enrichment of supported event log messages.

The Spotlights directory contains Graylog content pack files that provide the dashboards, saved searches, and event definitions for Graylog.

Upload Graylog Illuminate

  1. Log in to your Graylog Enterprise server with an account that has administrator privileges. Click on the Enterprise menu and select Illuminate.
    Install_Illuminate_2

  2. The Illuminate Installer page will open.
    Install_Illuminate_3

  3. Click to select, or drag and drop the Illuminate bundle ZIP file — e.g., bundle.v2_1_0.zip — located in the extracted directory — e.g., ./graylog_illuminate.v2_1_0.

  4. If you receive the error Failed to read bundle metadata 'bundle.toml' file. Please try again., verify that you have selected the correct archive file.

  5. After the Illuminate bundle file has been uploaded, a list of available technology packs and add-ons will appear. Activating these packs will be covered in the following sections.
    Install_Illuminate_4

Activate Graylog Illuminate Technology Packs and Add-ons

About Add-ons

The two available optional add-ons with Illuminate are “GIM Enforcement” and “Geolocation and AS Enrichment.” For more information about GIM enforcement, please read about GIM Enforcement at the end of this article.

In 2.1, two technology packs support geolocation and ASN enrichments:

  • one supporting MaxMind city and ASN databases
  • and another supporting IPinfo city and ASN databases.
Warning

We recommend that you install only one of these packs with its respective database files. If you install both packs, with no files your instance will receive error messages about missing database files.

Enabling MaxMind Geolocation and ASN Enrichment

The “Geolocation and AS Enrichment Add-on for MaxMind Databases
Geolocation and ASN Enrichment” requires that two files be installed on every Graylog Enterprise node in your cluster:

  • The MaxMind City database in MMDB format with the filename GeoLite2-City.mmdb
  • The MaxMind ASN database in MMDB format with the filename GeoLite2-ASN.mmdb

These files must be placed in the directory /etc/graylog/server for the enrichment to function.

Enabling IPinfo Geolocation and ASN Enrichment

Note

As of version 2.1, you can enable IPinfo services on Graylog Illuminate.

The “Geolocation and AS Enrichment Add-on for IPinfo Databases” requires that two files be installed on every Graylog Enterprise node in your cluster:

  • The IPinfo City database in MMDB format with the filename standard_location.mmdb
  • The IPinfo ASN database in MMDB format with the filename asn.mmdb

These files must be placed in the directory /etc/graylog/server for the enrichment to function.

Select Technology Packs and Add-ons

  1. From the list of available content, select the technology packs and optional add-ons you want to install.

Install_Illuminate_5

  1. After verifying your selection(s), click on Enable Selected in the upper right corner.

Install_Illuminate_6

  1. The Illuminate Installer will enable the selected packs, and the Installer page will indicate which packs have been enabled. There may be a slight delay while Graylog enables the selected add-ons and technology packs.

Install_Illuminate_7

Illuminate is now operational and processing messages, but the installation is not yet complete.

Upload the Spotlight Content Packs

Graylog Illuminate includes Spotlight content packs that provide dashboards, saved searches, and event definitions to assist with analyzing your log data. These are installed using the content pack system as detailed in the previous section. (Please note that not every technology pack has an accompanying Spotlight.)

There are also ”Core” and “Event Definitions” Spotlights available.

Core Spotlight

The Core Spotlight includes dashboards that are not associated with any specific technology but instead provide some insights into authentication in the Enterprise and a few investigative dashboards for researching user accounts and devices in your log data.

Event Definitions Spotlight

The Event Definitions content pack provides event definitions that can be used to generate alerts. Configuring the event definitions will be covered later in these instructions.

To install the "Event Definitions" Spotlight:

  1. Click on the System drop-down menu and select Content Packs.

Install_Illuminate_8

  1. Click on the Upload button on the upper right hand corner of the page. You will be presented with an Upload Content Pack window.

Install_Illuminate_9

  1. Click Browse and navigate to the directory — e.g. graylog_illuminate.v2_0_0— then the subdirectory ../spotlights. You will see a list of available Spotlight content pack files.

Install_Illuminate_10

  1. Select one you wish to upload, click Open, and then Upload. Repeat this process until you have uploaded all of the content packs you intend to install.

  2. The "Event Definitions" content pack is in the file with the filename that begins with illuminate_events_. It may not be necessary to upload this file. Event Definitions are not released with every content pack. If you are upgrading from a release which provided event definitions to a version which did not release any newer versions then you can skip the “Configure Event Definitions” process. Review the table below to determine if there are additional event definitions that you can configure based on which versions you have installed and which you are upgrading to.

Illuminate Version Event Definitions File Name Event Definitions Version
2.0.0 illuminate_events_2021_10_16.json 4
2.0.1
2.1.0

Install the Spotlight Content Packs

After all of the Spotlight content packs have been activated and uploaded, they can now be installed.

  1. To the right of the Content Packs page and at the top of the content packs list is a Filter text box. Enter Illuminate in this box and click the Filter button. You will see a list of the Spotlight content packs that you have uploaded.

Install_Illuminate_11

  1. Click on the title of a content pack entry to install it, and a page will open providing some additional details about the Spotlight. On the left of the page is a list of available and installed versions. Click on the Actions button and select Install from the list.

Install_Illuminate_12

  1. Click on the Content Packs button on the upper right corner of the page to return to the list of uploaded content packs. Repeat this process to install all of the desired Spotlight content packs.

Configure Event Definitions

If you have installed the “Event Definitions” content pack, then these event definitions will be installed on your system; however, they are disabled by default. The event definitions provided with Illuminate 2.0.0 begin with revision 4. This revision contains updated rule definitions that were provided in previous Spotlight versions.

  1. To enable event definitions click on the Alerts menu item, then click on the Event Definitions button on the upper right-hand side of the Alerts page.

Install_Illuminate_13

  1. Note the Illuminate event definitions will have titles that all begin with “Illuminate” and are all disabled. To enable an alert, click on the More button to the right of the event definition to activate and click on the Enable button.

Install_Illuminate_14

Event definitions can and likely should be customized to match your environment. This includes adjusting thresholds, search schedules, and notifications. It is important to note that these customizations will be lost if the content pack is ever uninstalled.

GIM Enforcement

About GIM Enforcement

One of the changes included with Illuminate 2.0 is the ability to enable and disable GIM Enforcement on demand.

What is GIM?

GIM, short for Graylog Information Model, is how we ensure known types of messages that have been properly categorized will have the necessary fields required for processing.

Why Enable GIM Enforcement?

GIM Enforcement will ensure that all events that have been categorized and intended to be available for search and aggregation, even if the message has been parsed incorrectly. This can be due to log format changes between versions of a product or unexpected data in the message that the parsing logic did not account for.

For example, all logon events should have the field user_name. With GIM Enforcement enabled, any message that has been categorized but is missing one of these required fields will have a default value assigned, and the field gim_error field will be added indicating that the message is incomplete. This will ensure that searches, which look for logon messages by user_name, will include these messages in related search results and aggregations.

Without GIM enforcement messages may not be included in search results or aggregations if they have been improperly parsed or if they are malformed in some way.

Note

Enabling GIM Enforcement will incur additional computational costs.


Was this article helpful?