Upgrading to Graylog 3.0.x

Elasticsearch Version Requirements

Graylog 3.0 drops support for Elasticsearch versions before 5.6.x. That means you have to upgrade Elasticsearch to at least version 5.6.13 before upgrading Graylog to version 3.0. Make sure to read the Elasticsearch upgrade guides before doing that.

Simplified HTTP interface configuration

Graylog used to have a lot of different settings regarding the various HTTP interfaces it provides, namely the Graylog REST API and the Graylog web interface.

This mostly originates from the fact that Graylog used to consist of two components before Graylog 2.0.0, a server component and a separate web interface.

The changes in this release finally merge the HTTP listeners for the Graylog REST API and web interface into a single HTTP listener, which should make the initial configuration of Graylog simpler and reduce errors caused by conflicting settings.

The path of the Graylog REST API is now hard-coded to /api, so if you’re still using the legacy URI on port 12900/tcp or have been using a custom path (via the rest_listen_uri or rest_transport_uri settings), you’ll have to update the URI used to access the Graylog REST API.

This might also affect your Graylog Collector Sidecars. Make sure to check each collector_sidecar.yml and update the server_url accordingly.

If you are using a reverse proxy in front of Graylog (like nginx) and configured it to set the X-Graylog-Server-URL HTTP header, you have to remove the api/ suffix because that is now the default. (as mentioned above)

Example:

# This nginx setting in Graylog <3.0 ...
header_upstream X-Graylog-Server-URL http://{host}/api

# ... needs to be changed to the following with Graylog 3.0
header_upstream X-Graylog-Server-URL http://{host}/

For a more detailed description of the new HTTP settings, please consult the annotated Graylog configuration file.

Overview of removed Graylog REST API settings:

Removed Setting

New Setting

Default

rest_listen_uri

http_bind_address

127.0.0.1:9000

rest_transport_uri

http_publish_uri

http://$http_bind_address/

web_endpoint_uri

http_external_uri

$http_publish_uri

rest_enable_cors

http_enable_cors

true

rest_enable_gzip

http_enable_gzip

true

rest_max_header_size

http_max_header_size

8192

rest_max_initial_line_length

http_max_initial_line_length

4096

rest_thread_pool_size

http_thread_pool_size

16

rest_enable_tls

http_enable_tls

false

rest_tls_cert_file

http_tls_cert_file

Empty

rest_tls_key_file

http_tls_key_file

Empty

rest_tls_key_password

http_tls_key_password

Empty

Overview of removed Graylog web interface settings:

Removed Setting

New Setting

Default

web_enable

None

web_listen_uri

http_bind_address

127.0.0.1:9000

web_enable_cors

http_enable_cors

true

web_enable_gzip

http_enable_gzip

true

web_max_header_size

http_max_header_size

8192

web_max_initial_line_length

http_max_initial_line_length

4096

web_thread_pool_size

http_thread_pool_size

16

web_enable_tls

http_enable_tls

false

web_tls_cert_file

http_tls_cert_file

Empty

web_tls_key_file

http_tls_key_file

Empty

web_tls_key_password

http_tls_key_password

Empty

Plugins merged into the Graylog server

Starting with Graylog 3.0.0, the following official plugins were merged into the Graylog server:

That means these plugins are not available as separate plugins anymore. If you manually update your Graylog installation (without using operating system packages), make sure to remove all old plugin files from the plugin_dir folder.

The old issues in these repositories are still available for reference but new issues should only be created in the Graylog server issue tracker.

The following HTTP API paths changed due to the plugin merge:

Old Path

New Path

/plugins/org.graylog.plugins.map/mapdata

/search/mapdata

/plugins/org.graylog.plugins.pipelineprocessor/system/pipelines/pipeline

/system/pipelines/pipeline

/plugins/org.graylog.plugins.pipelineprocessor/system/pipelines/pipeline/parse

/system/pipelines/pipeline/parse

/plugins/org.graylog.plugins.pipelineprocessor/system/pipelines/rule

/system/pipelines/rule

/plugins/org.graylog.plugins.pipelineprocessor/system/pipelines/rule/functions

/system/pipelines/rule/functions

/plugins/org.graylog.plugins.pipelineprocessor/system/pipelines/rule/multiple

/system/pipelines/rule/multiple

/plugins/org.graylog.plugins.pipelineprocessor/system/pipelines/rule/parse

/system/pipelines/rule/parse

/plugins/org.graylog.plugins.pipelineprocessor/system/pipelines/connections

/system/pipelines/connections

/plugins/org.graylog.plugins.pipelineprocessor/system/pipelines/connections/to_stream

/system/pipelines/connections/to_stream

/plugins/org.graylog.plugins.pipelineprocessor/system/pipelines/connections/to_pipeline

/system/pipelines/connections/to_pipeline

/plugins/org.graylog.plugins.pipelineprocessor/system/pipelines/simulate

/system/pipelines/simulate

New “bin_dir” and “data_dir” configuration parameters

We introduced two new configuration parameters related to file system paths.

  • bin_dir config option points to the directory that contains scripts like graylogctl.

  • data_dir option configures the base directory for Graylog server state.

Please check the updated default graylog.conf configuration file for required changes to your existing file.

Removed support for Drools-based filters

For a long time, Graylog allowed to use Drools to filter messages. Unfortunately, using Drools to perform complex filter logic came with a performance penalty and wasn’t as flexible as we would have liked it to be.

Starting with Graylog 3.0.0, the support for Drools-based message filters has been removed from Graylog. The rules_file configuration setting has been removed accordingly.

We recommend migrating the Drools-based logic to Processing Pipelines.

Drools-based blacklist

Graylog provided undocumented blacklist-functionality based on Drools. This blacklist could only be modified via the Graylog REST API on the /filters/blacklist resource.

If you’ve been using this functionality, you’ll have to migrate these blacklist rules to the Processing Pipelines.

To check if you’re using the Drools-based blacklist in Graylog prior to version 3.0.0, you can run the following command:

# curl -u admin:password -H 'Accept: application/json' 'http://graylog.example.com/api/filters/blacklist?pretty=true'

String-based blacklist rule

Old blacklist rule:

{
   "id" : "54e300001234123412340001",
   "type" : "string",
   "name" : "String Blacklist",
   "description" : "Drop messages based on case-insensitive string comparison",
   "fieldName" : "custom_field",
   "pattern" : "EXAMPLE pattern",
   "creator_user_id" : "admin",
   "created_at" : "2018-04-04T12:00:00.000Z"
}

New pipeline rule:

rule "string-blacklist"
when
  has_field("custom_field") &&
  lowercase(to_string($message.custom_field)) == "example pattern"
then
  drop_message();
end

See also:

Regex-based blacklist rule

Old blacklist rule:

{
   "id" : "54e300001234123412340002",
   "type" : "regex",
   "name" : "Regex Blacklist",
   "description" : "Drop messages based on regular expression",
   "fieldName" : "custom_field",
   "pattern" : "^EXAMPLE.*",
   "creator_user_id" : "admin",
   "created_at" : "2018-04-04T12:00:00.000Z"
}

New pipeline rule:

rule "regex-blacklist"
when
  has_field("custom_field") &&
  regex("^EXAMPLE.*", to_string($message.custom_field)).matches == true
then
  drop_message();
end

See also:

IP Range-based blacklist rule

Old blacklist rule:

{
   "id" : "54e300001234123412340003",
   "type" : "iprange",
   "name" : "IP Blacklist",
   "description" : "Drop messages based on IP address",
   "fieldName" : "custom_field",
   "pattern" : "192.168.0.0/16",
   "creator_user_id" : "admin",
   "created_at" : "2018-04-04T12:00:00.000Z"
}

New pipeline rule:

rule "ip-blacklist"
when
  has_field("custom_field") &&
  cidr_match("192.168.0.0/16", to_ip($message.custom_field))
then
  drop_message();
end

See also:

Changed metrics name for stream rules

The name of the metrics for stream rules have been changed to include the stream ID which helps identifying the actual stream they are related to.

Old metric name:

org.graylog2.plugin.streams.StreamRule.${stream-rule-id}.executionTime

New metric name:

org.graylog2.plugin.streams.Stream.${stream-id}.StreamRule.${stream-rule-id}.executionTime

Email alarm callback default settings

The defaults of the configuration settings for the email alarm callback with regard to encrypted connections have been changed.

Setting

Old default

New default

transport_email_use_tls

false

true

transport_email_use_ssl

true

false

Furthermore, it’s not possible anymore to enable both settings (SMTP with STARTTLS and SMTP over SSL) at the same time because this led to errors at runtime when Graylog tried to upgrade the connection to TLS with STARTTLS in an already existing SMTPS connection.

Most SMTP services prefer SMTP with STARTTLS to provide an encrypted connection.

Collector Sidecar is deprecated

Graylog 3.0 comes with a new Sidecar implementation. We still support the old Collector Sidecars, which can be found in the System / Collectors (legacy) menu entry. For more information check the Sidecar documentation and the Upgrade guide.

Legacy Content Packs

The implementation of content packs where fundamentally reworked. Parameters were added and checks implemented to give the user better usability. This rework did come with the cost that old content packs might not work any longer and stop the new content packs from loading. If the content packs page does not finish loading we recommend to remove the old content packs from your MongoDB. For that, please connect to your MongoDB shell and remove all content packs with the following command:

> db.content_packs.deleteMany({})

This command will only remove the content packs, it will not remove the installed configurations.

Elasticsearch 6 changes

There is a breaking change in Elasticsearch 6 that may affect some queries on your searches and dashboards:

Before Elasticsearch 6, queries for keyword fields were split by whitespaces and combined with OR operators resulting, for example, in type:(ssh login) and type:(ssh OR login) being equivalent. This is no longer the case in Elasticsearch 6 and now those queries are different: the former looking for the ssh login value, the second for either ssh or login values.

Please ensure to look for those queries in your Graylog setup before upgrading to Elasticsearch 6 and add the OR operators where needed.