找出mongodb的jumbo块并进行分裂

news2024/12/27 9:33:36

https://www.cnblogs.com/abclife/p/15968628.html

根据这篇文档中的脚本，在我们自己的环境中跑了下，第一次跑的结果如下：

运行完上面跑出的split脚本后，还是存在jumbo块，第二次跑出的结果：

从上面结果可以看出，chunks的数量增加了，但jumbo chunks的数量没有减少很多。

也和阿里云后台沟通，jumbo chunk不能分裂的原因：

因为大部分jumbo chunk是相同shard key组成的，所以无法继续分割。

不知道有没有通过这种方法分割成功的案例。

具体运行脚本如下：

1、使用以下脚本找出未被检测到的Jumbo块，并生成相关命令：

var allChunkInfo = function(ns){

    var chunks = db.getSiblingDB("config").chunks.find({"ns" : ns}).sort({min:1}).noCursorTimeout(); //this will return all chunks for the ns ordered by min

    var totalChunks = 0;

    var totalJumbo = 0;

    var totalSize = 0;

    var totalEmpty = 0;



    chunks.forEach(

        function printChunkInfo(chunk) {

          var db1 = db.getSiblingDB(chunk.ns.split(".")[0]) // get the database we will be running the command against later

          var key = db.getSiblingDB("config").collections.findOne({_id:chunk.ns}).key; // will need this for the dataSize call



          // dataSize returns the info we need on the data, but using the estimate option to use counts is less intensive

          var dataSizeResult = db1.runCommand({datasize:chunk.ns, keyPattern:key, min:chunk.min, max:chunk.max, estimate:false});



          if(dataSizeResult.size > 67108864) {

            totalJumbo++;

            print('sh.splitFind("' + chunk.ns.toString() + '", ' + JSON.stringify(chunk.min) + ')' + ' // '+ chunk.shard + '    ' + Math.round(dataSizeResult.size/1024/1024) + ' MB    ' + dataSizeResult.numObjects );

          }

          totalSize += dataSizeResult.size;

          totalChunks++;



          if (dataSizeResult.size == 0) { totalEmpty++ }; //count empty chunks for summary

          }

    )

    print("***********Summary Chunk Information***********");

    print("Total Chunks: "+totalChunks);

    print("Total Jumbo Chunks: "+totalJumbo);

    print("Average Chunk Size (Mbytes): "+(totalSize/totalChunks/1024/1024));

    print("Empty Chunks: "+totalEmpty);

    print("Average Chunk Size (non-empty): "+(totalSize/(totalChunks-totalEmpty)/1024/1024));

}

2、脚本必须通过mongos router进行调用：

mongos> allChunkInfo("db.test_col")

3、生成如下结果

sh.splitFind("db.test_col", {"_id":"jhxT2neuI5fB4o4KBIASK1"}) // shard-1    222 MB    7970

sh.splitFind("db.test_col", {"_id":"zrAESqSZjnpnMI23oh5JZD"}) // shard-2    226 MB    7988

sh.splitFind("db.test_col", {"_id":"SgkCkfSDrY789e9nD4crk9"}) // shard-1    218 MB    7986

sh.splitFind("db.test_col", {"_id":"X5MKEH4j32OhmAhY7LGPMm"}) // shard-1    238 MB    8338

...

***********Summary Chunk Information***********

Total Chunks: 5047

Total Jumbo Chunks: 120

Average Chunk Size (Mbytes): 19.29779934868946

Empty Chunks: 1107

Average Chunk Size (non-empty): 24.719795257064895