Graylog lets you extract and visualize geolocation information from IP addresses in your logs. In this section, you'll learn how to configure the geolocation processor, which has been updated in the Graylog 4.3 release. Additionally, you'll learn how to create a map with the extracted geo-information.
Setup the Processor
Graylog ships with geolocation capabilities by default but additional configuration is still required. This section explains how to configure the functionality in detail.
Download the Database
To start, download a geolocation database. As of version 4.3, both MaxMind and IPInfo databases are supported by Graylog.
The next step is to store the geolocation database on all servers running Graylog. Make sure you grant the right permissions to the file so the user running Graylog can read the database.
Configure the Processor
You will need to configure Graylog to start using the geolocation database to resolve IPs in your logs. To ensure the correct order of operations occurs, verify that the configuration of the Illuminate Processor is executed before the GeoIP Resolver. Navigate to System > Configurations and locate the "Message Processors Configuration" setting at the top right.
Ensure that the Illuminate Processor is queued before the GeoIP Resolver, as shown. If this is not the case, then select Update to reorder your processing steps.
Then, on the same page, locate the geolocation configuration under the Plugins section.
Here, select "Update" to configure your plugin utilizing either MaxMind or IPInfo. In the configuration dialog, you need to select "Enable Geo-location processor" and enter the paths to both the city and ASN databases you use. Once you are all set, click on save to store the configuration changes.
Enforce Graylog Schema Option
If Schema enforcement is disabled: all IP fields that are not reserved IP addresses will be processed and have the following fields added with the field name as a prefix
-
_geolocation -
_country_code -
_city_name
An example of the generated fields for the source_ip field might read:
-
source_ip_city_name: Vienna -
source_ip_country_code: AT -
source_ip_geolocation: 48.20849, 16.37208
If Schema enforcement is enabled: only the following GIM schema fields that are not reserved IP addressed will be processed:
-
destination_ip -
destination_nat_ip -
event_observer_ip -
host_ip -
network_forwarded_ip -
source_ip -
source_nat_ip
An example of the generated fields for the source_ip field might read:
-
source_as_number: AS1853 -
source_as_organization: ACONET -
source_geo_city: Vienna -
source_geo_coordinates: 48.20849, 16.37208 -
source_geo_country_iso: AT -
source_geo_name: Vienna, AT -
source_geo_region: Vienna -
source_go_timezome: Europe/Vienna
Storing Geolocation DB Files in AWS S3
Prior to Graylog 5.0 users needed to manually move geolocation database files to the correct location on disk on each node in their cluster. Graylog 5.0 introduces the ability to pull geolocation database files from AWS S3 buckets. A new configuration value for Pull files from S3 bucket has been added at the bottom of the configuration page. Enabling this feature allows for an S3 bucket URL to be added to the Path configuration values.
When enabled a service will run every refresh interval and poll the files in the S3 buckets provided. If those files have been updated since the last poll, then the new files will be pulled down onto each node. This service relies on the DefaultCredentialsProvider for credentials to the S3 buckets and will not use any configuration values that may or may not be set in the Graylog AWS Plugin configuration settings.
The geolocation database files retrieved from S3 are stored in the Graylog data_dir directory under the geolocation subdirectory. To change where these files are downloaded, set geo_ip_processor_s3_download_location to the desired location on disk in your Graylog server configuration file.
If the pull files from S3 bucket option is left disabled, all Graylog nodes will continue to operate as they did before 5.0, reading the files from the path on disk and requiring manual updating to those files for updates.
Visualize Geolocations in a Map
Graylog can display maps from
geolocation stored in any field, as long as the geo-points are using the
latitude,longitude format.
Display a Map in the Search Results Page
On any search result page, you can expand the field you want to use to draw a map in the search sidebar. Click the Create button (+) in the left sidebar and select aggregation under the Generic menu.
This will generate an empty aggregation widget. Click Edit and enter your information. Select World Map as the Visualization Type. You will then see a map with all the different points stored in that field.
You may click Update preview to take a peek at your map and make any changes before you click Update widget.
Hint: Adding a metric will affect the size of the dot on the map. If there is no metric defined, every dot will have the same radius.
For additional fields used in Graylog related to different sources of geo coordinates please view the Graylog Schema.
Add a Map to a Dashboard
You can add the map visualization into any dashboards as you do with other widgets. Once you displayed a map in the search result page:
-
Click on the three dots in the upper right corner.
-
Select Export to Dashboard.
-
You may then rename, edit and save the new Dashboard.
