索引别名deflector的异常处理

官方推荐处理步骤

Stop all Graylog nodes
(OPTIONAL) If you want to keep the already ingested messages, reindex them into the Elasticsearch index with the greatest number, e. g. graylog_23 if you want to fix the deflector graylog_deflector, via the Elasticsearch Reindex API.
Delete the graylog_deflector index via the Elasticsearch Delete Index API.
Add action.auto_create_index: false to the configuration files of all Elasticsearch nodes in your cluster and restart these Elasticsearch nodes, see Elasticsearch Index API - Automatic Index Creation and Creating an Index for details.
Start the Graylog master node.
Manually rotate the active write index of the index set on the System / Indices / Index Set page in the Maintenance dropdown menu.
(OPTIONAL) Start all remaining Graylog slave nodes.

实际可行操作

一般出现索引别名异常时，一般是Graylog和Elasticsearch的连接出现异常或者Elasticsearch集群本身异常，导致别名和实际索引挂载异常导致。
所以处理起来，也是按照这个思路，首先关闭es的自动创建索引，以防删除挂载别名异常索引后，自动又新建一个，陷入循环；其次，删除异常的别名对应索引；然后手动进行索引轮换，查看无问题；最后打开索引的自动创建，Graylog恢复正常。

1. 禁用自动创建索引：

   curl -s -XPUT -u elastic 'http://127.0.0.1:9200/_cluster/settings' -d '{"transient":{"action.auto_create_index":"false"}}'
   

2. 删除最新索引名称或者通过http://127.0.0.1:9200/_cat/aliases查看别名对应的索引名称：

   curl -XDELETE -u elastic 'http://10.10.238.178:9200/INDEX_NAME'
   

3. 手动轮换索引（66d435e168e94f0cb77cd14e索引集id）：

   curl -u admin:admin -XPOST -H 'X-Requested-By: XMLHttpRequest' 'http://127.0.0.1/api/cluster/deflector/66d435e168e94f0cb77cd14e/cycle'
   

4. 开启自动创建索引：

   curl -s -XPUT -u elastic 'http://127.0.0.1:9200/_cluster/settings' -d '{"transient":{"action.auto_create_index":"true"}}'
   

Elasticsearch的索引分片数限制解除

报错

Validation Failed: 1: this action would add [1] total shards, but this cluster currently has [6000]/[6000] maximum shards open.

解除

curl -XPUT http://localhost:9200/_cluster/settings
-u elastic:password
-H “Content-Type: application/json”
-d ‘{“transient”:{“cluster”:{“max_shards_per_node”:10000}}}’

优化

Elasticsearch推荐分片大小在30-50G之间，一定要人为进行干预优化，太多的小分片，也不利于集群故障后的分片速度，也会影响一定的查询和写入性能。