Elasticsearch snapshot and restore

Elasticsearch snapshot is the backup of the one or more indexes in the elasticsearch cluster.

Backup

Create a snapshot directory

mkdir -p /esdata/esbackup

Give elasticsearch the permission to write on these directories

chown -R elasticsearch:elasticsearch /esdata/esbackup

Add the path to the elasticsearch config file

vim /etc/elasticsearch/elasticsearch.yml

path.repo: ["/esdata/esbackup"]

Restart the elasticsearch services

service elasticsearch restart

make sure the elasticsearch service is in active state.

Create the repository, creating the repository backup

curl -XPUT -H "Content-Type: application/json;charset=UTF-8" 'http://localhost:9200/_snapshot/backup' -d '{
  "type": "fs",
  "settings": {
     "location": "/esdata/esbackup",
     "compress": true
  }
}'

Check the repository

curl -XGET http://localhost:9200/_snapshot/_all?pretty
{
  "backup" : {
    "type" : "fs",
    "settings" : {
      "compress" : "true",
      "location" : "/esdata/esbackup"
    }
  }
}

Create a snapshot by name backup2, the important thing is there it backs up all the indexes

curl -XPUT  "http://localhost:9200/_snapshot/backup/mybackup2?wait_for_completion=true&pretty=true" -H "Content-Type: application/json;charset=UTF-8"
{
  "snapshot" : {
    "snapshot" : "mybackup2",
    "uuid" : "5X60QHBNRhOddJOgVI7j9Q",
    "version_id" : 7060299,
    "version" : "7.6.2",
    "indices" : [
      "course",
      "blog",
      "test"
    ],
    "include_global_state" : true,
    "state" : "SUCCESS",
    "start_time" : "2020-04-19T17:08:58.181Z",
    "start_time_in_millis" : 1587316138181,
    "end_time" : "2020-04-19T17:08:58.381Z",
    "end_time_in_millis" : 1587316138381,
    "duration_in_millis" : 200,
    "failures" : [ ],
    "shards" : {
      "total" : 10,
      "failed" : 0,
      "successful" : 10
    }
  }
}

Now lets create a snapshot by name backup3, where were going to take two indexes blog & course

curl -XPUT  "http://localhost:9200/_snapshot/backup/mybackup3?wait_for_completion=true&pretty=true" -H "Content-Type: application/json;charset=UTF-8" -d '{
    "indices": "blog,course",
    "ignore_unavailable": true,
    "include_global_state": false
}'
{
  "snapshot" : {
    "snapshot" : "mybackup3",
    "uuid" : "e4dLWT8FRYu7WP-6kJWozg",
    "version_id" : 7060299,
    "version" : "7.6.2",
    "indices" : [
      "course",
      "blog"
    ],
    "include_global_state" : false,
    "state" : "SUCCESS",
    "start_time" : "2020-04-19T17:14:34.237Z",
    "start_time_in_millis" : 1587316474237,
    "end_time" : "2020-04-19T17:14:34.237Z",
    "end_time_in_millis" : 1587316474237,
    "duration_in_millis" : 0,
    "failures" : [ ],
    "shards" : {
      "total" : 2,
      "failed" : 0,
      "successful" : 2
    }
  }
}

Restore

Create a snapshot directory

mkdir /esdata

On Source Elasticsearch instance go to the directory /esdata and create a tar of the esbackup directory, and copy the tar to the destination server at /esdata location using scp or whatever

tar -xvf esbackup.tar esbackup

scp esbackup.tar userid@dest_ip:~/esdata/

On destination server, untar the esbackup.tar at /esdata directory, and the directory esbackup is created with the data

tar -xvf esbackup.tar

On destination give elasticsearch user permission to write on these directories

chown -R elasticsearch:elasticsearch /esdata/esbackup

Add the path to the elasticsearch config file

vim /etc/elasticsearch/elasticsearch.yml

path.repo: ["/esdata/esbackup"]

Restart the elasticsearch services

service elasticsearch restart

make sure the elasticsearch service is in active state.

Create the repository, creating the repository esrestore

curl -XPUT -H "Content-Type: application/json;charset=UTF-8" 'http://localhost:9200/_snapshot/esrestore' -d '{
  "type": "fs",
  "settings": {
     "location": "/esdata/esbackup",
     "compress": true
  }
}'

Verify the repository, the esrestore repo is created, as per output below

curl -XGET 'http://localhost:9200/_snapshot/_all?pretty'
{
  "esrestore" : {
    "type" : "fs",
    "settings" : {
      "compress" : "true",
      "location" : "/esdata/esbackup"
    }
  }
}

Restoring the backup for single index

curl  -XPOST -H "Content-Type: application/json;charset=UTF-8" "http://localhost:9200/_snapshot/esrestore/mybackup/_restore?wait_for_completion=true&pretty=true" -d '{
  "indices": "test",
  "ignore_unavailable": true,
  "include_global_state": true
}'
{
  "snapshot" : {
    "snapshot" : "mybackup",
    "indices" : [
      "test"
    ],
    "shards" : {
      "total" : 1,
      "failed" : 0,
      "successful" : 1
    }
  }
}

Restoring the backup for multiple indexes

curl  -XPOST -H "Content-Type: application/json;charset=UTF-8" "http://localhost:9200/_snapshot/esrestore/mybackup/_restore?wait_for_completion=true&pretty=true" -d '{
  "indices": "blog,course",
  "ignore_unavailable": true,
  "include_global_state": false,
  "rename_pattern": "index_(.+)",
  "rename_replacement": "restored_index_$1"
}'
{
  "snapshot" : {
    "snapshot" : "mybackup",
    "indices" : [
      "course",
      "blog"
    ],
    "shards" : {
      "total" : 2,
      "failed" : 0,
      "successful" : 2
    }
  }
}

Restoring the backup for all indexes is simple, just send request without any data to the above request.

References

Snapshot and restore | Elasticsearch Reference [7.6] | Elastic