Sharding-JDBC笔记04-分库分表实战

文章目录

前言
一、需求描述
二、数据库设计
三、环境说明
四、环境准备
- 4.1.mysql主从同步(windows)
- 4.2.初始化数据库
五、实现步骤
- 5.1 搭建maven工程
- - 引入maven依赖
- 5.2 实体类
- 5.3 dao层
- 5.4 服务类
- 5.5 测试类
- - 总结
- 5.6 查询商品
- - Dao
  - Service
  - 单元测试
  - 输出
  - 小结
- 5.7 统计商品
- - Dao
  - 单元测试
  - - 统计商品总数
    - 分组统计商品
  - 小结
六、总结
- 为什么分库分表？
七、附 SQL支持说明
- Sharding-JDBC支持的SQL
- Sharding-JDBC不支持的SQL
- DISTINCT支持情况详细说明
- - Sharding-JDBC支持的SQL
  - Sharding-JDBC不支持的SQL

前言

Sharding-JDBC定位为轻量级Java框架，在Java的JDBC层提供的额外服务。它使用客户端直连数据库，以jar包形式提供服务，无需额外部署和依赖，可理解为增强版的JDBC驱动，完全兼容JDBC和各种ORM框架。

Sharding-JDBC的核心功能为数据分片和读写分离，通过Sharding-JDBC，应用可以透明的使用jdbc访问已经分库分表、读写分离的多个数据源，而不用关心数据源的数量以及数据如何分布。

一、需求描述

电商平台商品列表展示，每个列表项中除了包含商品基本信息、商品描述信息之外，还包括了商品所属的店铺信息，如下：
在这里插入图片描述
本案例实现功能如下：

添加商品
商品分页查询
商品统计

二、数据库设计

数据库设计如下，其中商品与店铺信息之间进行了垂直分库，分为了PRODUCT_DB(商品库)和STORE_DB(店铺库)；商品信息还进行了垂直分表，分为了商品基本信息(product_info)和商品描述信息(product_descript)，地理区域信息(region)作为公共表，冗余在两库中：
在这里插入图片描述
考虑到商品信息的数据增长性，对PRODUCT_DB(商品库)进行了水平分库，分片键使用店铺id，分片策略为店铺ID%2 + 1，因此商品描述信息对所属店铺ID进行了冗余；
对商品基本信息(product_info)和商品描述信息(product_descript)进行水平分表，分片键使用商品id，分片策略为商品ID%2 + 1,并将为这两个表设置为绑定表，避免笛卡尔积join；
为避免主键冲突，ID生成策略采用雪花算法来生成全局唯一ID，最终数据库设计为下图：
在这里插入图片描述
要求使用读写分离来提升性能，可用性。

三、环境说明

操作系统：Win10
数据库：MySQL-5.7.25
JDK：64位 jdk1.8.0_201
应用框架：spring-boot-2.1.3.RELEASE，Mybatis3.5.0
Sharding-JDBC：sharding-jdbc-spring-boot-starter-4.0.0-RC1

四、环境准备

4.1.mysql主从同步(windows)

参考读写分离章节，对以下库进行主从同步配置：
或参考文章：Windows环境下搭建MySQL主从同步实现读写分离
主库

[mysql]
# 设置mysql客户端默认字符集
default-character-set=utf8
[mysqld]
# 设置3308端口
port = 3308
server_id = 1    # 主库和从库需要不一致，分别配一个唯一的ID编号
log_bin=master-bin    # 二进制文件存放路径,存放在根目录data文件夹下
#设置需要同步的数据库
binlog_do_db=user_db
binlog_do_db=store_db
binlog_do_db=product_db_1
binlog_do_db=product_db_2
#屏蔽系统库同步
binlog_ignore_db=mysql
binlog_ignore_db=information_schema
binlog_ignore_db=performance_schema
# 设置mysql的安装目录
basedir=E:\\tool\\mysql-5.7.37-winx64
# 设置mysql数据库的数据的存放目录（自动生成，不然可能报错）
datadir=E:\\tool\\mysql-5.7.37-winx64\\data
# 允许最大连接数
max_connections=10000
# 允许最大连接人数
max_user_connections=1000
# 服务端使用的字符集默认为8比特编码的latin1字符集
character-set-server=utf8
# 创建新表时将使用的默认存储引擎
default-storage-engine=INNODB
 
# 连接时间一年
wait_timeout=31536000
interactive_timeout=31536000

从库

[mysql]
# 设置mysql客户端默认字符集
default-character-set=utf8
[mysqld]
# 设置3309端口
port = 3309
server_id = 2    # 主库和从库需要不一致，分别配一个唯一的ID编号
log_bin=master-bin    # 二进制文件存放路径,存放在根目录data文件夹下
#设置需要同步的数据库
replicate_wild_do_table=user_db.%
replicate_wild_do_table=store_db.%
replicate_wild_do_table=product_db_1.%
replicate_wild_do_table=product_db_2.%
#屏蔽系统库同步
replicate_wild_ignore_table=mysql.%
replicate_wild_ignore_table=information_schema.%
replicate_wild_ignore_table=performance_schema.%
# 设置mysql的安装目录
basedir=E:\\tool\\mysql-5.7.37-winx64-s1
# 设置mysql数据库的数据的存放目录（自动生成，不然可能报错）
datadir=E:\\tool\\mysql-5.7.37-winx64-s1\\data
# 允许最大连接数
max_connections=10000
# 允许最大连接人数
max_user_connections=1000
# 服务端使用的字符集默认为8比特编码的latin1字符集
character-set-server=utf8
# 创建新表时将使用的默认存储引擎
default-storage-engine=INNODB
 
# 连接时间一年
wait_timeout=31536000
interactive_timeout=31536000

4.2.初始化数据库

创建store_db数据库，并执行以下脚本创建表：

DROP TABLE IF EXISTS `region`;
CREATE TABLE `region` (
`id` bigint(20) NOT NULL COMMENT 'id',
`region_code` varchar(50) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT
'地理区域编码',
`region_name` varchar(100) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL
COMMENT '地理区域名称',
`level` tinyint(1) NULL DEFAULT NULL COMMENT '地理区域级别(省、市、县)',
`parent_region_code` varchar(50) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL
COMMENT '上级地理区域编码',
PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8 COLLATE = utf8_general_ci ROW_FORMAT = Dynamic;

INSERT INTO `region` VALUES (1, '110000', '北京', 0, NULL);
INSERT INTO `region` VALUES (2, '410000', '河南省', 0, NULL);
INSERT INTO `region` VALUES (3, '110100', '北京市', 1, '110000');
INSERT INTO `region` VALUES (4, '410100', '郑州市', 1, '410000');

DROP TABLE IF EXISTS `store_info`;
CREATE TABLE `store_info` (
`id` bigint(20) NOT NULL COMMENT 'id',
`store_name` varchar(100) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT
'店铺名称',
`reputation` int(11) NULL DEFAULT NULL COMMENT '信誉等级',
`region_code` varchar(50) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT
'店铺所在地',
PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8 COLLATE = utf8_general_ci ROW_FORMAT = Dynamic;

INSERT INTO `store_info` VALUES (1, 'XX零食店', 4, '110100');
INSERT INTO `store_info` VALUES (2, 'XX饮品店', 3, '410100');

创建product_db_1、product_db_2数据库，并分别对两库执行以下脚本创建表：

DROP TABLE IF EXISTS `product_descript_1`;
CREATE TABLE `product_descript_1` (
    `id` bigint(20) NOT NULL COMMENT 'id',
    `product_info_id` bigint(20) NULL DEFAULT NULL COMMENT '所属商品id',
    `descript` longtext CHARACTER SET utf8 COLLATE utf8_general_ci NULL COMMENT '商品描述',
    `store_info_id` bigint(20) NULL DEFAULT NULL COMMENT '所属店铺id',
    PRIMARY KEY (`id`) USING BTREE,
    INDEX `FK_Reference_2`(`product_info_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8 COLLATE = utf8_general_ci ROW_FORMAT = Dynamic;

DROP TABLE IF EXISTS `product_descript_2`;
CREATE TABLE `product_descript_2` (
`id` bigint(20) NOT NULL COMMENT 'id',
`product_info_id` bigint(20) NULL DEFAULT NULL COMMENT '所属商品id',
`descript` longtext CHARACTER SET utf8 COLLATE utf8_general_ci NULL COMMENT '商品描述',
`store_info_id` bigint(20) NULL DEFAULT NULL COMMENT '所属店铺id',
PRIMARY KEY (`id`) USING BTREE,
INDEX `FK_Reference_2`(`product_info_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8 COLLATE = utf8_general_ci ROW_FORMAT = Dynamic;

DROP TABLE IF EXISTS `product_info_1`;
CREATE TABLE `product_info_1` (
`product_info_id` bigint(20) NOT NULL COMMENT 'id',
`store_info_id` bigint(20) NULL DEFAULT NULL COMMENT '所属店铺id',
`product_name` varchar(100) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL
COMMENT '商品名称',
`spec` varchar(50) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT '规
格',
`region_code` varchar(50) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT
'产地',
`price` decimal(10, 0) NULL DEFAULT NULL COMMENT '商品价格',
`image_url` varchar(100) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT
'商品图片',
PRIMARY KEY (`product_info_id`) USING BTREE,
INDEX `FK_Reference_1`(`store_info_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8 COLLATE = utf8_general_ci ROW_FORMAT = Dynamic;

DROP TABLE IF EXISTS `product_info_2`;
CREATE TABLE `product_info_2` (
`product_info_id` bigint(20) NOT NULL COMMENT 'id',
`store_info_id` bigint(20) NULL DEFAULT NULL COMMENT '所属店铺id',
`product_name` varchar(100) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL
COMMENT '商品名称',
`spec` varchar(50) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT '规
格',
`region_code` varchar(50) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT
'产地',
`price` decimal(10, 0) NULL DEFAULT NULL COMMENT '商品价格',
`image_url` varchar(100) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT
'商品图片',
PRIMARY KEY (`product_info_id`) USING BTREE,
INDEX `FK_Reference_1`(`store_info_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8 COLLATE = utf8_general_ci ROW_FORMAT = Dynamic;

DROP TABLE IF EXISTS `region`;
CREATE TABLE `region` (
    `id` bigint(20) NOT NULL COMMENT 'id',
    `region_code` varchar(50) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT
    '地理区域编码',
    `region_name` varchar(100) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL
    COMMENT '地理区域名称',
    `level` tinyint(1) NULL DEFAULT NULL COMMENT '地理区域级别(省、市、县)',
    `parent_region_code` varchar(50) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL
    COMMENT '上级地理区域编码',
    PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8 COLLATE = utf8_general_ci ROW_FORMAT = Dynamic;

INSERT INTO `region` VALUES (1, '110000', '北京', 0, NULL);
INSERT INTO `region` VALUES (2, '410000', '河南省', 0, NULL);
INSERT INTO `region` VALUES (3, '110100', '北京市', 1, '110000');
INSERT INTO `region` VALUES (4, '410100', '郑州市', 1, '410000');

五、实现步骤

5.1 搭建maven工程

引入maven依赖

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.1.3.RELEASE</version>
</parent>

<dependencies>
            <dependency>
                <groupId>org.projectlombok</groupId>
                <artifactId>lombok</artifactId>
                <version>1.18.0</version>
            </dependency>


            <dependency>
                <groupId>javax.interceptor</groupId>
                <artifactId>javax.interceptor-api</artifactId>
                <version>1.2</version>
            </dependency>


            <dependency>
                <groupId>mysql</groupId>
                <artifactId>mysql-connector-java</artifactId>
                <version>5.1.47</version>
            </dependency>

            <dependency>
                <groupId>org.mybatis.spring.boot</groupId>
                <artifactId>mybatis-spring-boot-starter</artifactId>
                <version>2.0.0</version>
            </dependency>

            <dependency>
                <groupId>com.alibaba</groupId>
                <artifactId>druid-spring-boot-starter</artifactId>
                <version>1.1.16</version>
            </dependency>


            <dependency>
                <groupId>org.apache.shardingsphere</groupId>
                <artifactId>sharding-jdbc-spring-boot-starter</artifactId>
                <version>4.0.0-RC1</version>
            </dependency>


            <dependency>
                <groupId>com.baomidou</groupId>
                <artifactId>mybatis-plus-boot-starter</artifactId>
                <version>3.1.0</version>
            </dependency>

            <dependency>
                <groupId>com.baomidou</groupId>
                <artifactId>mybatis-plus-generator</artifactId>
                <version>3.1.0</version>
            </dependency>

            <dependency>
                <groupId>org.mybatis</groupId>
                <artifactId>mybatis-typehandlers-jsr310</artifactId>
                <version>1.0.2</version>
            </dependency>
        </dependencies>

5.2 实体类

@Data
public class ProductDescript {
    private Long id;
    /**
     * 所属商品id
     */
    private Long productInfoId;

    /**
     * 商品描述
     */
    private String descript;

    /**
     * 所属店铺id
     */
    private Long storeInfoId;
}

@Data
public class ProductInfo {
    private Long productInfoId;
    /**
     * 所属店铺id
     */
    private Long storeInfoId;

    /**
     * 商品名称
     */
    private String productName;

    /**
     * 规格
     */
    private String spec;

    /**
     * 产地
     */
    private String regionCode;

    /**
     * 商品价格
     */
    private BigDecimal price;

    /**
     * 商品图片
     */
    private String imageUrl;

    关联信息/

    /**
     * 商品描述
     */
    private String descript;

    /**
     * 产地名称
     */
    private String placeOfOrigin;

    /**
     * 店铺名称
     */
    private String storeName;

    /**
     * 店铺信誉等级
     */
    private int reputation;

    /**
     * 店铺所在地名称
     */
    private String storeRegionName;
}

@Data
public class region {
    private Long id;
    /**
     * 地理区域编码
     */
    private String regionCode;

    /**
     * 地理区域名称
     */
    private String regionName;

    /**
     * 地理区域级别(省、市、县)
     */
    private int level;

    /**
     * 上级地理区域编码
     */
    private String parentRegionCode;
}

@Data
public class StoreInfo {
    private Long id;
    /**
     * 店铺名称
     */
    private String storeName;

    /**
     * 信誉等级
     */
    private int reputation;

    /**
     * 店铺所在地
     */
    private String regionCode;

    /**
     * 店铺所在地名称
     */
    private String regionName;
}

5.3 dao层

@Mapper
@Component
public interface ProductDao {

    //添加商品基本信息
    @Insert("insert into product_info(store_info_id,product_name,spec,region_code,price) " +
            " values (#{storeInfoId},#{productName},#{spec},#{regionCode},#{price})")
    //@Options注解配置生成主键并映射到productInfoId属性
    //作用：插入数据后，mybatis返回创建主键，并映射到productInfoId属性中
    @Options(useGeneratedKeys = true,keyProperty = "productInfoId",keyColumn = "product_info_id")
    int insertProductInfo(ProductInfo productInfo);

    //添加商品描述信息
    @Insert("insert into product_descript(product_info_id,descript,store_info_id) " +
            " value(#{productInfoId},#{descript},#{storeInfoId})")
    @Options(useGeneratedKeys = true,keyProperty = "id",keyColumn = "id")
    int insertProductDescript(ProductDescript productDescript);

    //查询商品列表
    @Select("select i.*,d.descript,r.region_name placeOfOrigin " +
            "from product_info i " +
            "join product_descript d on i.product_info_id = d.product_info_id " +
            "join region r on i.region_code = r.region_code " +
            "order by product_info_id desc " +
            "limit #{start},#{pageSize}")
    List<ProductInfo> selectProductList(@Param("start")int start, @Param("pageSize") int pageSize);

    //商品总数
    @Select("select count(1) from product_info")
    int selectCount();

    //商品分组统计
    @Select("select t.region_code,count(1) as num " +
            "from product_info t " +
            "group by t.region_code having num > 1 " +
            "order by region_code ")
    List<Map> selectProductGroupList();

}

5.4 服务类

import com.itheima.shopping.entity.ProductInfo;
import java.util.List;

public interface ProductService {
    //添加商品
    public void createProduct(ProductInfo product);

    //查询商品
    public List<ProductInfo> queryProduct(int page, int pageSize);
}

实现类

import com.itheima.shopping.dao.ProductDao;
import com.itheima.shopping.entity.ProductDescript;
import com.itheima.shopping.entity.ProductInfo;
import com.itheima.shopping.service.ProductService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import java.util.List;

@Service
public class ProductServiceImpl implements ProductService {

    @Autowired
    ProductDao productDao;


    //添加商品
    @Override
    @Transactional
    public void createProduct(ProductInfo productInfo) {
        ProductDescript productDescript =new ProductDescript();
        //设置商品描述 信息
        productDescript.setDescript(productInfo.getDescript());
        //调用dao向商品信息表
        productDao.insertProductInfo(productInfo);
        //将商品信息id设置到productDescript
        productDescript.setProductInfoId(productInfo.getProductInfoId());
        //设置店铺id
        productDescript.setStoreInfoId(productInfo.getStoreInfoId());
        //向商品描述信息表插入数据
        productDao.insertProductDescript(productDescript);
    }

    @Override
    public List<ProductInfo> queryProduct(int page, int pageSize) {
        int start = (page - 1) * pageSize;
        return productDao.selectProductList(start,pageSize);
    }
}

5.5 测试类

向数据库中添加商品

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

import java.math.BigDecimal;
import java.util.List;
import java.util.Map;

/**
 * @author qf
 * @version 1.0
 **/
@RunWith(SpringRunner.class)
@SpringBootTest(classes = ShoppingBootstrap.class)
public class ShardingTest {

    @Autowired
    ProductService productService;

    @Autowired
    ProductDao productDao;

    //添加商品
    @Test
    public void testCreateProduct(){
        for (int i=0;i<10;i++){
            ProductInfo productInfo = new ProductInfo();
            //注意：分库根据storeInfoId的奇偶为分片策略
            productInfo.setStoreInfoId(2L);//店铺id

            productInfo.setProductName("Java编程思想"+i);//商品名称
            productInfo.setSpec("大号");
            productInfo.setPrice(new BigDecimal(60));
            productInfo.setRegionCode("110100");
            productInfo.setDescript("Java编程思想不错！！！"+i);//商品描述
            productService.createProduct(productInfo);
        }
    }
}

这里使用了sharding-jdbc所提供的全局主键生成方式之一雪花算法，来生成全局业务唯一主键。
输出

2024-09-25 22:46:13.294  INFO 6372 --- [           main] ShardingSphere-SQL                       : Logic SQL: insert into product_info(store_info_id,product_name,spec,region_code,price)  values (?,?,?,?,?)
2024-09-25 22:46:13.294  INFO 6372 --- [           main] ShardingSphere-SQL                       : SQLStatement: InsertStatement(super=DMLStatement(super=AbstractSQLStatement(type=DML, tables=Tables(tables=[Table(name=product_info, alias=Optional.absent())]), routeConditions=Conditions(orCondition=OrCondition(andConditions=[AndCondition(conditions=[Condition(column=Column(name=store_info_id, tableName=product_info), operator=EQUAL, compareOperator=null, positionValueMap={}, positionIndexMap={0=0})])])), encryptConditions=Conditions(orCondition=OrCondition(andConditions=[])), sqlTokens=[TableToken(tableName=product_info, quoteCharacter=NONE, schemaNameLength=0), SQLToken(startIndex=24)], parametersIndex=5, logicSQL=insert into product_info(store_info_id,product_name,spec,region_code,price)  values (?,?,?,?,?)), deleteStatement=false, updateTableAlias={}, updateColumnValues={}, whereStartIndex=0, whereStopIndex=0, whereParameterStartIndex=0, whereParameterEndIndex=0), columnNames=[store_info_id, product_name, spec, region_code, price], values=[InsertValue(columnValues=[org.apache.shardingsphere.core.parse.old.parser.expression.SQLPlaceholderExpression@7df3da0b, org.apache.shardingsphere.core.parse.old.parser.expression.SQLPlaceholderExpression@4e5d5ac1, org.apache.shardingsphere.core.parse.old.parser.expression.SQLPlaceholderExpression@70777a65, org.apache.shardingsphere.core.parse.old.parser.expression.SQLPlaceholderExpression@236fd411, org.apache.shardingsphere.core.parse.old.parser.expression.SQLPlaceholderExpression@402f61f5])])
2024-09-25 22:46:13.295  INFO 6372 --- [           main] ShardingSphere-SQL                       : Actual SQL: m1 ::: insert into product_info_2 (store_info_id, product_name, spec, region_code, price, product_info_id) VALUES (?, ?, ?, ?, ?, ?) ::: [2, Java编程思想0, 大号, 110100, 60, 1045832812957007873]
2024-09-25 22:46:13.322  INFO 6372 --- [           main] ShardingSphere-SQL                       : Rule Type: sharding
2024-09-25 22:46:13.322  INFO 6372 --- [           main] ShardingSphere-SQL                       : Logic SQL: insert into product_descript(product_info_id,descript,store_info_id)  value(?,?,?)
2024-09-25 22:46:13.322  INFO 6372 --- [           main] ShardingSphere-SQL                       : SQLStatement: InsertStatement(super=DMLStatement(super=AbstractSQLStatement(type=DML, tables=Tables(tables=[Table(name=product_descript, alias=Optional.absent())]), routeConditions=Conditions(orCondition=OrCondition(andConditions=[AndCondition(conditions=[Condition(column=Column(name=product_info_id, tableName=product_descript), operator=EQUAL, compareOperator=null, positionValueMap={}, positionIndexMap={0=0}), Condition(column=Column(name=store_info_id, tableName=product_descript), operator=EQUAL, compareOperator=null, positionValueMap={}, positionIndexMap={0=2})])])), encryptConditions=Conditions(orCondition=OrCondition(andConditions=[])), sqlTokens=[TableToken(tableName=product_descript, quoteCharacter=NONE, schemaNameLength=0), SQLToken(startIndex=28)], parametersIndex=3, logicSQL=insert into product_descript(product_info_id,descript,store_info_id)  value(?,?,?)), deleteStatement=false, updateTableAlias={}, updateColumnValues={}, whereStartIndex=0, whereStopIndex=0, whereParameterStartIndex=0, whereParameterEndIndex=0), columnNames=[product_info_id, descript, store_info_id], values=[InsertValue(columnValues=[org.apache.shardingsphere.core.parse.old.parser.expression.SQLPlaceholderExpression@4c2bea52, org.apache.shardingsphere.core.parse.old.parser.expression.SQLPlaceholderExpression@11015ca0, org.apache.shardingsphere.core.parse.old.parser.expression.SQLPlaceholderExpression@6ebbc06])])
2024-09-25 22:46:13.322  INFO 6372 --- [           main] ShardingSphere-SQL                       : Actual SQL: m1 ::: insert into product_descript_2 (product_info_id, descript, store_info_id, id) VALUES (?, ?, ?, ?) ::: [1045832812957007873, Java编程思想不错！！！0, 2, 1045832813284163585]
2024-09-25 22:46:13.330  INFO 6372 --- [           main] ShardingSphere-SQL                       : Rule Type: sharding
2024-09-25 22:46:13.330  INFO 6372 --- [           main] ShardingSphere-SQL                       : Logic SQL: insert into product_info(store_info_id,product_name,spec,region_code,price)  values (?,?,?,?,?)
2024-09-25 22:46:13.330  INFO 6372 --- [           main] ShardingSphere-SQL                       : SQLStatement: InsertStatement(super=DMLStatement(super=AbstractSQLStatement(type=DML, tables=Tables(tables=[Table(name=product_info, alias=Optional.absent())]), routeConditions=Conditions(orCondition=OrCondition(andConditions=[AndCondition(conditions=[Condition(column=Column(name=store_info_id, tableName=product_info), operator=EQUAL, compareOperator=null, positionValueMap={}, positionIndexMap={0=0})])])), encryptConditions=Conditions(orCondition=OrCondition(andConditions=[])), sqlTokens=[TableToken(tableName=product_info, quoteCharacter=NONE, schemaNameLength=0), SQLToken(startIndex=24)], parametersIndex=5, logicSQL=insert into product_info(store_info_id,product_name,spec,region_code,price)  values (?,?,?,?,?)), deleteStatement=false, updateTableAlias={}, updateColumnValues={}, whereStartIndex=0, whereStopIndex=0, whereParameterStartIndex=0, whereParameterEndIndex=0), columnNames=[store_info_id, product_name, spec, region_code, price], values=[InsertValue(columnValues=[org.apache.shardingsphere.core.parse.old.parser.expression.SQLPlaceholderExpression@7df3da0b, org.apache.shardingsphere.core.parse.old.parser.expression.SQLPlaceholderExpression@4e5d5ac1, org.apache.shardingsphere.core.parse.old.parser.expression.SQLPlaceholderExpression@70777a65, org.apache.shardingsphere.core.parse.old.parser.expression.SQLPlaceholderExpression@236fd411, org.apache.shardingsphere.core.parse.old.parser.expression.SQLPlaceholderExpression@402f61f5])])
2024-09-25 22:46:13.330  INFO 6372 --- [           main] ShardingSphere-SQL                       : Actual SQL: m1 ::: insert into product_info_1 (store_info_id, product_name, spec, region_code, price, product_info_id) VALUES (?, ?, ?, ?, ?, ?) ::: [2, Java编程思想1, 大号, 110100, 60, 1045832813321912320]
2024-09-25 22:46:13.333  INFO 6372 --- [           main] ShardingSphere-SQL                       : Rule Type: sharding
2024-09-25 22:46:13.333  INFO 6372 --- [           main] ShardingSphere-SQL                       : Logic SQL: insert into product_descript(product_info_id,descript,store_info_id)  value(?,?,?)
2024-09-25 22:46:13.333  INFO 6372 --- [           main] ShardingSphere-SQL                       : SQLStatement: InsertStatement(super=DMLStatement(super=AbstractSQLStatement(type=DML, tables=Tables(tables=[Table(name=product_descript, alias=Optional.absent())]), routeConditions=Conditions(orCondition=OrCondition(andConditions=[AndCondition(conditions=[Condition(column=Column(name=product_info_id, tableName=product_descript), operator=EQUAL, compareOperator=null, positionValueMap={}, positionIndexMap={0=0}), Condition(column=Column(name=store_info_id, tableName=product_descript), operator=EQUAL, compareOperator=null, positionValueMap={}, positionIndexMap={0=2})])])), encryptConditions=Conditions(orCondition=OrCondition(andConditions=[])), sqlTokens=[TableToken(tableName=product_descript, quoteCharacter=NONE, schemaNameLength=0), SQLToken(startIndex=28)], parametersIndex=3, logicSQL=insert into product_descript(product_info_id,descript,store_info_id)  value(?,?,?)), deleteStatement=false, updateTableAlias={}, updateColumnValues={}, whereStartIndex=0, whereStopIndex=0, whereParameterStartIndex=0, whereParameterEndIndex=0), columnNames=[product_info_id, descript, store_info_id], values=[InsertValue(columnValues=[org.apache.shardingsphere.core.parse.old.parser.expression.SQLPlaceholderExpression@4c2bea52, org.apache.shardingsphere.core.parse.old.parser.expression.SQLPlaceholderExpression@11015ca0, org.apache.shardingsphere.core.parse.old.parser.expression.SQLPlaceholderExpression@6ebbc06])])
2024-09-25 22:46:13.333  INFO 6372 --- [           main] ShardingSphere-SQL                       : Actual SQL: m1 ::: insert into product_descript_1 (product_info_id, descript, store_info_id, id) VALUES (?, ?, ?, ?) ::: [1045832813321912320, Java编程思想不错！！！1, 2, 1045832813330300928]
2024-09-25 22:46:13.336  INFO 6372 --- [           main] ShardingSphere-SQL                       : Rule Type: sharding
...

可以看到storeInfoId(商铺id)为偶数时，向m1库插入数据。
同时根据在配置文件中通过雪花算法生成的主键productInfoId的奇偶来插入到product_info_1或product_info_2中

总结

通过添加商品接口新增商品进行分库验证，store_info_id为偶数的数据在product_db_1库，为奇数的数据在product_db_2库。
通过添加商品接口新增商品进行分表验证，product_id为偶数的数据在product_info_1表、product_descript_1表，为奇数的数据在product_info_2表、product_descript_2表。

5.6 查询商品

Dao

在ProductDao中定义商品查询方法：

//查询商品列表
@Select("select i.*,d.descript,r.region_name placeOfOrigin " +
        "from product_info i " +
        "join product_descript d on i.product_info_id = d.product_info_id " +
        "join region r on i.region_code = r.region_code " +
        "order by product_info_id desc " +
        "limit #{start},#{pageSize}")
List<ProductInfo> selectProductList(@Param("start")int start, @Param("pageSize") int pageSize);

Service

@Override
public List<ProductInfo> queryProduct(int page, int pageSize) {
    int start = (page - 1) * pageSize;
    return productDao.selectProductList(start,pageSize);
}

单元测试

分页查询

@Test
public void testQueryProduct(){
    List<ProductInfo> productInfos = productService.queryProduct(1, 10);
    System.out.println(productInfos);
}

输出

2024-09-25 23:14:22.909  INFO 4820 --- [           main] ShardingSphere-SQL                       : Rule Type: sharding
2024-09-25 23:14:22.909  INFO 4820 --- [           main] ShardingSphere-SQL                       : Logic SQL: select i.*,d.descript,r.region_name placeOfOrigin from product_info i join product_descript d on i.product_info_id = d.product_info_id join region r on i.region_code = r.region_code order by product_info_id desc limit ?,?
2024-09-25 23:14:22.909  INFO 4820 --- [           main] ShardingSphere-SQL                       : SQLStatement: SelectStatement(super=DQLStatement(super=AbstractSQLStatement(type=DQL, tables=Tables(tables=[Table(name=product_info, alias=Optional.of(i)), Table(name=product_descript, alias=Optional.of(d)), Table(name=region, alias=Optional.of(r))]), routeConditions=Conditions(orCondition=OrCondition(andConditions=[])), encryptConditions=Conditions(orCondition=OrCondition(andConditions=[])), sqlTokens=[TableToken(tableName=product_info, quoteCharacter=NONE, schemaNameLength=0), TableToken(tableName=product_descript, quoteCharacter=NONE, schemaNameLength=0), TableToken(tableName=region, quoteCharacter=NONE, schemaNameLength=0)], parametersIndex=2, logicSQL=select i.*,d.descript,r.region_name placeOfOrigin from product_info i join product_descript d on i.product_info_id = d.product_info_id join region r on i.region_code = r.region_code order by product_info_id desc limit ?,?)), containStar=true, firstSelectItemStartIndex=7, selectListStopIndex=48, groupByLastIndex=0, items=[StarSelectItem(owner=Optional.of(i)), CommonSelectItem(expression=d.descript, alias=Optional.absent()), CommonSelectItem(expression=r.region_name, alias=Optional.of(placeOfOrigin))], groupByItems=[], orderByItems=[OrderItem(owner=Optional.absent(), name=Optional.of(product_info_id), orderDirection=DESC, nullOrderDirection=ASC, index=-1, expression=null, alias=Optional.absent())], limit=Limit(offset=LimitValue(value=-1, index=0, boundOpened=false), rowCount=LimitValue(value=-1, index=1, boundOpened=false)), subqueryStatement=null, subqueryStatements=[], subqueryConditions=[])
2024-09-25 23:14:22.910  INFO 4820 --- [           main] ShardingSphere-SQL                       : Actual SQL: s1 ::: select i.*,d.descript,r.region_name placeOfOrigin from product_info_1 i join product_descript_1 d on i.product_info_id = d.product_info_id join region r on i.region_code = r.region_code order by product_info_id desc limit ?,? ::: [0, 10]
2024-09-25 23:14:22.910  INFO 4820 --- [           main] ShardingSphere-SQL                       : Actual SQL: s1 ::: select i.*,d.descript,r.region_name placeOfOrigin from product_info_2 i join product_descript_2 d on i.product_info_id = d.product_info_id join region r on i.region_code = r.region_code order by product_info_id desc limit ?,? ::: [0, 10]
2024-09-25 23:14:22.910  INFO 4820 --- [           main] ShardingSphere-SQL                       : Actual SQL: s2 ::: select i.*,d.descript,r.region_name placeOfOrigin from product_info_1 i join product_descript_1 d on i.product_info_id = d.product_info_id join region r on i.region_code = r.region_code order by product_info_id desc limit ?,? ::: [0, 10]
2024-09-25 23:14:22.910  INFO 4820 --- [           main] ShardingSphere-SQL                       : Actual SQL: s2 ::: select i.*,d.descript,r.region_name placeOfOrigin from product_info_2 i join product_descript_2 d on i.product_info_id = d.product_info_id join region r on i.region_code = r.region_code order by product_info_id desc limit ?,? ::: [0, 10]
[ProductInfo(productInfoId=1045832813535821824, storeInfoId=2, productName=Java编程思想9, spec=大号, regionCode=110100, price=60, imageUrl=null, descript=Java编程思想不错！！！9, placeOfOrigin=北京市, storeName=null, reputation=0, storeRegionName=null), ProductInfo(productInfoId=1045832813506461697, storeInfoId=2, productName=Java编程思想8, spec=大号, regionCode=110100, price=60, imageUrl=null, descript=Java编程思想不错！！！8, placeOfOrigin=北京市, storeName=null, reputation=0, storeRegionName=null), ProductInfo(productInfoId=1045832813481295872, storeInfoId=2, productName=Java编程思想7, spec=大号, regionCode=110100, price=60, imageUrl=null, descript=Java编程思想不错！！！7, placeOfOrigin=北京市, storeName=null, reputation=0, storeRegionName=null), ProductInfo(productInfoId=1045832813451935745, storeInfoId=2, productName=Java编程思想6, spec=大号, regionCode=110100, price=60, imageUrl=null, descript=Java编程思想不错！！！6, placeOfOrigin=北京市, storeName=null, reputation=0, storeRegionName=null), ProductInfo(productInfoId=1045832813426769920, storeInfoId=2, productName=Java编程思想5, spec=大号, regionCode=110100, price=60, imageUrl=null, descript=Java编程思想不错！！！5, placeOfOrigin=北京市, storeName=null, reputation=0, storeRegionName=null), ProductInfo(productInfoId=1045832813397409793, storeInfoId=2, productName=Java编程思想4, spec=大号, regionCode=110100, price=60, imageUrl=null, descript=Java编程思想不错！！！4, placeOfOrigin=北京市, storeName=null, reputation=0, storeRegionName=null), ProductInfo(productInfoId=1045832813368049664, storeInfoId=2, productName=Java编程思想3, spec=大号, regionCode=110100, price=60, imageUrl=null, descript=Java编程思想不错！！！3, placeOfOrigin=北京市, storeName=null, reputation=0, storeRegionName=null), ProductInfo(productInfoId=1045832813342883841, storeInfoId=2, productName=Java编程思想2, spec=大号, regionCode=110100, price=60, imageUrl=null, descript=Java编程思想不错！！！2, placeOfOrigin=北京市, storeName=null, reputation=0, storeRegionName=null), ProductInfo(productInfoId=1045832813321912320, storeInfoId=2, productName=Java编程思想1, spec=大号, regionCode=110100, price=60, imageUrl=null, descript=Java编程思想不错！！！1, placeOfOrigin=北京市, storeName=null, reputation=0, storeRegionName=null), ProductInfo(productInfoId=1045832812957007873, storeInfoId=2, productName=Java编程思想0, spec=大号, regionCode=110100, price=60, imageUrl=null, descript=Java编程思想不错！！！0, placeOfOrigin=北京市, storeName=null, reputation=0, storeRegionName=null)]

通过查询商品列表接口，能够查询到所有分片的商品信息，关联的地理区域，店铺信息。

当分页不是从第一页开始

@Test
public void testQueryProduct2(){
    List<ProductInfo> productInfos = productService.queryProduct(2, 2);
    System.out.println(productInfos);
}

输出

2024-09-25 23:15:18.262  INFO 21204 --- [           main] ShardingSphere-SQL                       : Rule Type: sharding
2024-09-25 23:15:18.262  INFO 21204 --- [           main] ShardingSphere-SQL                       : Logic SQL: select i.*,d.descript,r.region_name placeOfOrigin from product_info i join product_descript d on i.product_info_id = d.product_info_id join region r on i.region_code = r.region_code order by product_info_id desc limit ?,?
2024-09-25 23:15:18.262  INFO 21204 --- [           main] ShardingSphere-SQL                       : SQLStatement: SelectStatement(super=DQLStatement(super=AbstractSQLStatement(type=DQL, tables=Tables(tables=[Table(name=product_info, alias=Optional.of(i)), Table(name=product_descript, alias=Optional.of(d)), Table(name=region, alias=Optional.of(r))]), routeConditions=Conditions(orCondition=OrCondition(andConditions=[])), encryptConditions=Conditions(orCondition=OrCondition(andConditions=[])), sqlTokens=[TableToken(tableName=product_info, quoteCharacter=NONE, schemaNameLength=0), TableToken(tableName=product_descript, quoteCharacter=NONE, schemaNameLength=0), TableToken(tableName=region, quoteCharacter=NONE, schemaNameLength=0)], parametersIndex=2, logicSQL=select i.*,d.descript,r.region_name placeOfOrigin from product_info i join product_descript d on i.product_info_id = d.product_info_id join region r on i.region_code = r.region_code order by product_info_id desc limit ?,?)), containStar=true, firstSelectItemStartIndex=7, selectListStopIndex=48, groupByLastIndex=0, items=[StarSelectItem(owner=Optional.of(i)), CommonSelectItem(expression=d.descript, alias=Optional.absent()), CommonSelectItem(expression=r.region_name, alias=Optional.of(placeOfOrigin))], groupByItems=[], orderByItems=[OrderItem(owner=Optional.absent(), name=Optional.of(product_info_id), orderDirection=DESC, nullOrderDirection=ASC, index=-1, expression=null, alias=Optional.absent())], limit=Limit(offset=LimitValue(value=-1, index=0, boundOpened=false), rowCount=LimitValue(value=-1, index=1, boundOpened=false)), subqueryStatement=null, subqueryStatements=[], subqueryConditions=[])
2024-09-25 23:15:18.263  INFO 21204 --- [           main] ShardingSphere-SQL                       : Actual SQL: s1 ::: select i.*,d.descript,r.region_name placeOfOrigin from product_info_1 i join product_descript_1 d on i.product_info_id = d.product_info_id join region r on i.region_code = r.region_code order by product_info_id desc limit ?,? ::: [0, 4]
2024-09-25 23:15:18.263  INFO 21204 --- [           main] ShardingSphere-SQL                       : Actual SQL: s1 ::: select i.*,d.descript,r.region_name placeOfOrigin from product_info_2 i join product_descript_2 d on i.product_info_id = d.product_info_id join region r on i.region_code = r.region_code order by product_info_id desc limit ?,? ::: [0, 4]
2024-09-25 23:15:18.263  INFO 21204 --- [           main] ShardingSphere-SQL                       : Actual SQL: s2 ::: select i.*,d.descript,r.region_name placeOfOrigin from product_info_1 i join product_descript_1 d on i.product_info_id = d.product_info_id join region r on i.region_code = r.region_code order by product_info_id desc limit ?,? ::: [0, 4]
2024-09-25 23:15:18.263  INFO 21204 --- [           main] ShardingSphere-SQL                       : Actual SQL: s2 ::: select i.*,d.descript,r.region_name placeOfOrigin from product_info_2 i join product_descript_2 d on i.product_info_id = d.product_info_id join region r on i.region_code = r.region_code order by product_info_id desc limit ?,? ::: [0, 4]
[ProductInfo(productInfoId=1045832813481295872, storeInfoId=2, productName=Java编程思想7, spec=大号, regionCode=110100, price=60, imageUrl=null, descript=Java编程思想不错！！！7, placeOfOrigin=北京市, storeName=null, reputation=0, storeRegionName=null), ProductInfo(productInfoId=1045832813451935745, storeInfoId=2, productName=Java编程思想6, spec=大号, regionCode=110100, price=60, imageUrl=null, descript=Java编程思想不错！！！6, placeOfOrigin=北京市, storeName=null, reputation=0, storeRegionName=null)]

可以发现Sharding-jdbc改写了sql，本来应该为limit ?,? ::: [2, 4]改写成了limit ?,? ::: [0, 4]
但其实Sharding-jdbc大部分情况都通过流式归并获取数据结果集，不会出现大量无意义的数据加载至内存中，造成内存溢出风险。

小结

分页查询是业务中最常见的场景，Sharding-jdbc支持常用关系数据库的分页查询，不过Sharding-jdbc的分页功能比较容易让使用者误解，用户通常认为分页归并会占用大量内存。
在分布式的场景中，将 LIMIT 10000000 , 10改写为 LIMIT 0, 10000010 ，才能保证其数据的正确性。用户非常容易产生ShardingSphere会将大量无意义的数据加载至内存中，造成内存溢出风险的错觉。 其实大部分情况都通过流式归并获取数据结果集，因此ShardingSphere会通过结果集的next方法将无需取出的数据全部跳过，并不会将其存入内存。
但同时需要注意的是，由于排序的需要，大量的数据仍然需要传输到Sharding-Jdbc的内存空间。因此，采用LIMIT这种方式分页，并非最佳实践。由于LIMIT并不能通过索引查询数据，因此如果可以保证ID的连续性(使用一个数据库表单独维护主键id)，通过ID进行分页是比较好的解决方案，例如：

SELECT * FROM t_order WHERE id > 100000 AND id <= 100010 ORDER BY id;

或通过记录上次查询结果的最后一条记录的ID进行下一页的查询，例如：

SELECT * FROM t_order WHERE id > 10000000 LIMIT 10;

排序功能是由Sharding-jdbc的排序归并来完成，由于在SQL中存在 ORDER BY 语句，因此每个数据结果集自身是有序的，因此只需要将数据结果集当前游标指向的数据值进行排序即可。这相当于对多个有序的数组进行排序，归并排序是最适合此场景的排序算法。

5.7 统计商品

本小节实现商品总数统计，商品分组统计

Dao

在ProductDao中定义：

//商品总数
@Select("select count(1) from product_info")
int selectCount();

//商品分组统计
@Select("select t.region_code,count(1) as num " +
        "from product_info t " +
        "group by t.region_code having num > 1 " +
        "order by region_code ")
List<Map> selectProductGroupList();

单元测试

统计商品总数

@Test
public void testSelectCount(){
    int i = productDao.selectCount();
    System.out.println(i);
}

输出

2024-09-25 23:26:04.018  INFO 10560 --- [           main] ShardingSphere-SQL                       : Rule Type: sharding
2024-09-25 23:26:04.019  INFO 10560 --- [           main] ShardingSphere-SQL                       : Logic SQL: select count(1) from product_info
2024-09-25 23:26:04.019  INFO 10560 --- [           main] ShardingSphere-SQL                       : SQLStatement: SelectStatement(super=DQLStatement(super=AbstractSQLStatement(type=DQL, tables=Tables(tables=[Table(name=product_info, alias=Optional.absent())]), routeConditions=Conditions(orCondition=OrCondition(andConditions=[])), encryptConditions=Conditions(orCondition=OrCondition(andConditions=[])), sqlTokens=[TableToken(tableName=product_info, quoteCharacter=NONE, schemaNameLength=0)], parametersIndex=0, logicSQL=select count(1) from product_info)), containStar=false, firstSelectItemStartIndex=7, selectListStopIndex=14, groupByLastIndex=0, items=[AggregationSelectItem(type=COUNT, innerExpression=(1), alias=Optional.absent(), derivedAggregationSelectItems=[], index=-1)], groupByItems=[], orderByItems=[], limit=null, subqueryStatement=null, subqueryStatements=[], subqueryConditions=[])
2024-09-25 23:26:04.019  INFO 10560 --- [           main] ShardingSphere-SQL                       : Actual SQL: s1 ::: select count(1) from product_info_1
2024-09-25 23:26:04.019  INFO 10560 --- [           main] ShardingSphere-SQL                       : Actual SQL: s1 ::: select count(1) from product_info_2
2024-09-25 23:26:04.019  INFO 10560 --- [           main] ShardingSphere-SQL                       : Actual SQL: s2 ::: select count(1) from product_info_1
2024-09-25 23:26:04.019  INFO 10560 --- [           main] ShardingSphere-SQL                       : Actual SQL: s2 ::: select count(1) from product_info_2
10

分组统计商品

@Test
public void testSelectProductGroupList(){
    List<Map> maps = productDao.selectProductGroupList();
    System.out.println(maps);
}

输出

2024-09-25 23:32:56.538  INFO 20588 --- [           main] ShardingSphere-SQL                       : Rule Type: sharding
2024-09-25 23:32:56.538  INFO 20588 --- [           main] ShardingSphere-SQL                       : Logic SQL: select t.region_code,count(1) as num from product_info t group by t.region_code having num > 1 order by region_code
2024-09-25 23:32:56.538  INFO 20588 --- [           main] ShardingSphere-SQL                       : SQLStatement: SelectStatement(super=DQLStatement(super=AbstractSQLStatement(type=DQL, tables=Tables(tables=[Table(name=product_info, alias=Optional.of(t))]), routeConditions=Conditions(orCondition=OrCondition(andConditions=[])), encryptConditions=Conditions(orCondition=OrCondition(andConditions=[])), sqlTokens=[ItemsToken(isFirstOfItemsSpecial=false, items=[region_code AS ORDER_BY_DERIVED_0 ]), TableToken(tableName=product_info, quoteCharacter=NONE, schemaNameLength=0)], parametersIndex=0, logicSQL=select t.region_code,count(1) as num from product_info t group by t.region_code having num > 1 order by region_code)), containStar=false, firstSelectItemStartIndex=7, selectListStopIndex=35, groupByLastIndex=78, items=[CommonSelectItem(expression=t.region_code, alias=Optional.absent()), AggregationSelectItem(type=COUNT, innerExpression=(1), alias=Optional.of(num), derivedAggregationSelectItems=[], index=-1)], groupByItems=[OrderItem(owner=Optional.of(t), name=Optional.of(region_code), orderDirection=ASC, nullOrderDirection=ASC, index=-1, expression=null, alias=Optional.absent())], orderByItems=[OrderItem(owner=Optional.absent(), name=Optional.of(region_code), orderDirection=ASC, nullOrderDirection=ASC, index=-1, expression=null, alias=Optional.of(ORDER_BY_DERIVED_0))], limit=null, subqueryStatement=null, subqueryStatements=[], subqueryConditions=[])
2024-09-25 23:32:56.538  INFO 20588 --- [           main] ShardingSphere-SQL                       : Actual SQL: s1 ::: select t.region_code,count(1) as num , region_code AS ORDER_BY_DERIVED_0 from product_info_1 t group by t.region_code having num > 1 order by region_code
2024-09-25 23:32:56.538  INFO 20588 --- [           main] ShardingSphere-SQL                       : Actual SQL: s1 ::: select t.region_code,count(1) as num , region_code AS ORDER_BY_DERIVED_0 from product_info_2 t group by t.region_code having num > 1 order by region_code
2024-09-25 23:32:56.538  INFO 20588 --- [           main] ShardingSphere-SQL                       : Actual SQL: s2 ::: select t.region_code,count(1) as num , region_code AS ORDER_BY_DERIVED_0 from product_info_1 t group by t.region_code having num > 1 order by region_code
2024-09-25 23:32:56.539  INFO 20588 --- [           main] ShardingSphere-SQL                       : Actual SQL: s2 ::: select t.region_code,count(1) as num , region_code AS ORDER_BY_DERIVED_0 from product_info_2 t group by t.region_code having num > 1 order by region_code
[{num=5, region_code=110100}, {num=5, region_code=410000}]

注意：sql中写的是通过region_code分组后筛选出大于1的

小结

分组统计
分组统计也是业务中常见的场景，分组功能的实现由Sharding-jdbc分组归并完成。分组归并的情况最为复杂，它分为流式分组归并和内存分组归并。 流式分组归并要求SQL的排序项与分组项的字段必须保持一致，否则只能通过内存归并才能保证其数据的正确性。
举例说明，假设根据科目分片，表结构中包含考生的姓名（为了简单起见，不考虑重名的情况）和分数。通过SQL获取每位考生的总分，可通过如下SQL：

SELECT name, SUM(score) FROM t_score GROUP BY name ORDER BY name;

在分组项与排序项完全一致的情况下，取得的数据是连续的，分组所需的数据全数存在于各个数据结果集的当前游标所指向的数据值，因此可以采用流式归并。如下图所示。
在这里插入图片描述
进行归并时，逻辑与排序归并类似。下图展现了进行next调用的时候，流式分组归并是如何进行的。

通过图中我们可以看到，当进行第一次next调用时，排在队列首位的t_score_java将会被弹出队列，并且将分组值同为“Jetty”的其他结果集中的数据一同弹出队列。在获取了所有的姓名为“Jetty”的同学的分数之后，进行累加操作，那么，在第一次next调用结束后，取出的结果集是“Jetty”的分数总和。
与此同时，所有的数据结果集中的游标都将下移至数据值“Jetty”的下一个不同的数据值，并且根据数据结果集当前游标指向的值进行重排序。因此，包含名字顺着第二位的“John”的相关数据结果集则排在的队列的前列。

六、总结

为什么分库分表？

分库分表就是为了解决由于数据量过大而导致数据库性能降低的问题，将原来独立的数据库拆分成若干数据库组成，将数据大表拆分成若干数据表组成，使得单一数据库、单一数据表的数据量变小，从而达到提升数据库性能的目的。

分库分表方式：垂直分表、垂直分库、水平分库、水平分表
分库分表带来问题：由于数据分散在多个数据库，服务器导致了事务一致性问题、跨节点join问题、跨节点分页、排序、函数，主键需要全局唯一，公共表。
Sharding-JDBC基础概念：逻辑表，真实表，数据节点，绑定表，广播表，分片键，分片算法，分片策略，主键生成策略
Sharding-JDBC核心功能：数据分片，读写分离
Sharding-JDBC执行流程： SQL解析 => 查询优化 => SQL路由 => SQL改写 => SQL执行 => 结果归并

最佳实践：
系统在设计之初就应该对业务数据的耦合松紧进行考量，从而进行垂直分库、垂直分表，使数据层架构清晰明了。
若非必要，无需进行水平切分，应先从缓存技术着手降低对数据库的访问压力。如果缓存使用过后，数据库访问量还是非常大，可以考虑数据库读、写分离原则。若当前数据库压力依然大，且业务数据持续增长无法估量，最后可考虑水平分库、分表，单表拆分数据控制在1000万以内。