Upgrading to Graylog 2.0.x

Elasticsearch 2.x

The embedded Elasticsearch node being used by Graylog has been upgraded to Elasticsearch 2.x which includes some breaking changes. Graylog 2.x does not work with Elasticsearch 1.x anymore and cannot communicate with existing Elasticsearch 1.x clusters.

Please see Breaking changes in Elasticsearch 2.x for details.

The blog article Key points to be aware of when upgrading from Elasticsearch 1.x to 2.x also contains interesting information about the upgrade path from Elasticsearch 1.x to 2.x.

Multicast Discovery

Multicast discovery has been removed from Elasticsearch 2.x (although it is still provided as an Elasticsearch plugin for now).

To reflect this change, the elasticsearch_discovery_zen_ping_unicast_hosts now has to contain the address of at least one Elasticsearch node in the cluster which Graylog can connect to.

Default network host

The network interface which Elasticsearch binds to (elasticsearch_network_host) has been changed to localhost (i. e. 127.0.0.1 or ::1); see Network changes/Bind to localhost.

If Elasticsearch is not running on the same machine, elasticsearch_network_host must be set to a host name or an IP address which can be accessed by the other Elasticsearch nodes in the cluster.

Index range types

Note

This step needs to be performed before the update to Elasticsearch 2.x!

Some Graylog versions stored meta information about indices in elasticsearch, alongside the messages themselves. Since Elasticsearch 2.0 having multiple types with conflicting mappings is no longer possible, which means that the index_range type must be removed before upgrading to Elasticsearch 2.x.

Find out if your setup is affected by running (replace $elasticsearch with the address of one of your Elasticsearch nodes) curl -XGET $elasticsearch:9200/_all/_mapping/index_range; echo

If the output is {} you are not affected and can skip this step.

Otherwise, you need to delete the index_range type, Graylog does not use it anymore.

As Graylog sets older indices to read-only, first we need to remove the write block on those indices. Since we’ll be working with Elasticsearch’s JSON output, we recommend installing the jq utility which should be available on all popular package managers or directly at GitHub.

for i in `curl -s -XGET $elasticsearch:9200/_all/_mapping/index_range | jq -r "keys[]"`; do
    echo -n "Updating index $i: "
    echo -n "curl -XPUT $elasticsearch:9200/$i/_settings -d '{\"index.blocks.read_only\":false, \"index.blocks.write\":false}' : "
    curl -XPUT $elasticsearch:9200/$i/_settings -d '{"index.blocks.read_only":false, "index.blocks.write":false}'
    echo
done

The output for each of the curl commands should be {"acknowledged":true}. Next we have to delete the index_range mapping. We can perform this via the next command.

Note

We strongly recommend to perform this on a single index before running this bulk command. This operation can be expensive to perform if you have a lot of affected indices.

for i in `curl -s -XGET $elasticsearch:9200/_all/_mapping/index_range | jq -r "keys[]"`; do
    echo -n "Updating index $i: "
    curl -XDELETE $elasticsearch:9200/$i/index_range
    echo
done

It is not strictly necessary to set the indices back to read only, but if you prefer to do that, note the index names and commands during the first step and change the false into true.

Graylog Index Template

Graylog applies a custom index template to ensure that the indexed messages adhere to a specific schema.

Unfortunately the index template being used by Graylog 1.x is incompatible with Elasticsearch 2.x and has to be removed prior to upgrading.

In order to delete the index template the following curl command has to be issued against on of the Elasticsearch nodes:

curl -X DELETE http://localhost:9200/_template/graylog-internal

Graylog will automatically create the new index template on the next startup.

Dots in field names

One of the most important breaking changes in Elasticsearch 2.x is that field names may not contain dots anymore.

Using the Elasticsearch Migration Plugin might help to highlight some potential pitfalls if an existing Elasticsearch 1.x cluster should be upgraded to Elasticsearch 2.x.

MongoDB

Graylog 2.x requires MongoDB 2.4 or newer. We recommend using MongoDB 3.x and the WiredTiger storage engine.

When upgrading from MongoDB 2.0 or 2.2 to a supported version, make sure to read the Release Notes for the particular version.

Log4j 2 migration

Graylog switched its logging backend from Log4j 1.2 to Log4j 2.

Please refer to the Log4j Migration Guide for information on how to update your existing logging configuration.

Dead Letters feature removed

The Dead Letters feature, which stored messages that couldn’t be indexed into Elasticsearch for various reasons, has been removed.

This feature has been disabled by default. If you have enabled the feature the configuration file, please check the dead_letters_enabled collection in MongoDB and remove it afterwards.

Removed configuration settings

Index Retention and Rotation Settings

In 2.0.0 the index rotation and retention settings have been moved from the Graylog server config file to the database and are now configurable via the web interface.

The old settings from the graylog.conf or /etc/graylog/server/server.conf will be migrated to the database.

Warning

When you upgrade from a 1.x version and you modified any rotation/retention settings, please make sure you KEEP your old settings in the config file so the migration process will add your old settings to the database! Otherwise the retention process will use the default settings and might remove a lot of indices.

Overview

Some settings, which have been deprecated in previous versions, have finally been removed from the Graylog configuration file.

Removed configuration settings

Setting name

Replacement

mongodb_host

mongodb_uri

mongodb_port

mongodb_uri

mongodb_database

mongodb_uri

mongodb_useauth

mongodb_uri

mongodb_user

mongodb_uri

mongodb_password

mongodb_uri

elasticsearch_node_name

elasticsearch_node_name_prefix

collector_expiration_threshold

(moved to collector plugin)

collector_inactive_threshold

(moved to collector plugin)

rotation_strategy

UI in web interface (System/Indices)

retention_strategy

UI in web interface (System/Indices)

elasticsearch_max_docs_per_index

UI in web interface (System/Indices)

elasticsearch_max_size_per_index

UI in web interface (System/Indices)

elasticsearch_max_time_per_index

UI in web interface (System/Indices)

elasticsearch_max_number_of_indices

UI in web interface (System/Indices)

dead_letters_enabled

None

Changed configuration defaults

For better consistency, the defaults of some configuration settings have been changed after the project has been renamed from Graylog2 to Graylog.

Configuration defaults

Setting name

Old default

New default

elasticsearch_cluster_name

graylog2

graylog

elasticsearch_node_name

graylog2-server

graylog-server

elasticsearch_index_prefix

graylog2

graylog

elasticsearch_discovery_zen_ping_unicast_hosts

empty

127.0.0.1:9300

elasticsearch_discovery_zen_ping_multicast_enabled

true

false

mongodb_uri

mongodb://127.0.0.1/graylog2

mongodb://localhost/graylog

Changed prefixes for configuration override

In the past it was possible to override configuration settings in Graylog using environment variables or Java system properties with a specific prefix.

For better consistency, these prefixes have been changed after the project has been renamed from Graylog2 to Graylog.

Configuration override prefixes

Override

Old prefix

New prefix

Example

Environment variables

GRAYLOG2_

GRAYLOG_

GRAYLOG_IS_MASTER

System properties

graylog2.

graylog.

graylog.is_master

REST API Changes

The output ID key for the list of outputs in the /streams/* endpoints has been changed from _id to id.

 {
   "id": "564f47c41ec8fe7d920ef561",
   "creator_user_id": "admin",
   "outputs": [
     {
       "id": "56d6f2cce45e0e52d1e4b9cb", // ==> Changed from `_id` to `id`
       "title": "GELF Output",
       "type": "org.graylog2.outputs.GelfOutput",
       "creator_user_id": "admin",
       "created_at": "2016-03-02T14:03:56.686Z",
       "configuration": {
         "hostname": "127.0.0.1",
         "protocol": "TCP",
         "connect_timeout": 1000,
         "reconnect_delay": 500,
         "port": 12202,
         "tcp_no_delay": false,
         "tcp_keep_alive": false,
         "tls_trust_cert_chain": "",
         "tls_verification_enabled": false
       },
       "content_pack": null
     }
   ],
   "matching_type": "AND",
   "description": "All incoming messages",
   "created_at": "2015-11-20T16:18:12.416Z",
   "disabled": false,
   "rules": [],
   "alert_conditions": [],
   "title": "ALL",
   "content_pack": null
 }

Web Interface Config Changes

The web interface has been integrated into the Graylog server and was rewritten in React. Therefore configuring it has changed fundamentally since the last version(s). Please consult Web interface for details.

Please take note that the application.context configuration parameter present in Graylog 1.x (and earlier) is not existing anymore. The web interface can currently only be served without a path prefix.