前言
本文将会使用 SpringCloud Gateway 网关组件配合 Nacos 实现灰度发布（金丝雀发布）
环境搭建
创建子模块服务提供者 provider，网关模块 gateway
父项目
pom.xml 配置
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.example</groupId>
<artifactId>spring-gateway-demo</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>pom</packaging>
<name>spring-gateway-demo</name>
<description>spring-gateway-demo</description>
<properties>
<java.version>11</java.version>
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
<maven.compiler.plugin>3.8.1</maven.compiler.plugin>
<spring-boot.version>2.3.7.RELEASE</spring-boot.version>
<spring-cloud-alibaba.version>2.2.2.RELEASE</spring-cloud-alibaba.version>
<spring-cloud.version>Hoxton.SR9</spring-cloud.version>
<spring-cloud-starter-alibaba-nacos-config>2.2.0.RELEASE</spring-cloud-starter-alibaba-nacos-config>
</properties>
<modules>
<module>provider</module>
<module>gateway</module>
</modules>
<dependencies>
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-starter-alibaba-nacos-config</artifactId>
</dependency>
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-dependencies</artifactId>
<version>${spring-boot.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-alibaba-dependencies</artifactId>
<version>${spring-cloud-alibaba.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
</project>
服务提供者 provider
这里我们计划引入 nacos, 所以先创建一个 nacos 配置文件 dataId 为 provider.properties, 这里用默认的命名空间 public, 默认分组 DEFAULT_GROUP
version=2
provider 的 pom 配置依赖
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
application.yml
server:
port: 9001
spring:
application:
name: provider
cloud:
nacos:
config:
server-addr: 127.0.0.1:8848
discovery:
server-addr: 127.0.0.1:8848
启动类上添加 @EnableDiscoveryClient 注解
@EnableDiscoveryClient
@SpringBootApplication
public class ProviderApplication {
public static void main(String[] args) {
SpringApplication.run(ProviderApplication.class, args);
}
}
然后添加测试 controller
@RefreshScope
@RestController
@RequestMapping("/test")
public class TestController {
@Autowired
private Environment env;
@Value("${version:0}")
private String version;
/**
* http://localhost:9001/test/port
* @return
*/
@GetMapping("/port")
public Object port() {
return String.format("port=%s, version=%s", env.getProperty("local.server.port"), version);
}
}
注意，这里配置 nacos 的时候需要配置下面两个文件 provider.properties 和 provider，然后实际配置最终 nacos 是采用的 provider 文件，否则后端控制台就会持续输出 400 错误，可能是新版本问题，其他版本暂时不清楚（后面网关配置也是同理）
后端控制台输出，也可以看出需要两个
[fixed-localhost_8848] [subscribe] provider.properties+DEFAULT_GROUP
[fixed-localhost_8848] [add-listener] ok, tenant=, dataId=provider.properties, group=DEFAULT_GROUP, cnt=1
[fixed-localhost_8848] [subscribe] provider+DEFAULT_GROUP
[fixed-localhost_8848] [add-listener] ok, tenant=, dataId=provider, group=DEFAULT_GROUP, cnt=1
Nacos 版本：2.3.0-BETA
网关 gateway
gateway 服务的 pom 依赖配置如下:
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
</dependencies>
application.yml
# 应用服务 WEB 访问端口
server:
port: 9000
# 应用名称
spring:
application:
name: gateway
cloud:
nacos:
config:
server-addr: 127.0.0.1:8848
discovery:
server-addr: 127.0.0.1:8848
gateway:
routes: # http://127.0.0.1:9000/actuator/gateway/routes
- id: provider # 路由 ID，保持唯一
uri: lb://provider # uri指目标服务地址，lb代表从注册中心获取服务
predicates:
- Path=/provider/** # http://127.0.0.1:9000/provider/port 会转发到 http://localhost:9001/provider/port, 和预期不符合, 需要StripPrefix来处理
filters:
- StripPrefix=1 # StripPrefix=1就代表截取路径的个数为1, 这样请求 http://127.0.0.1:9000/provider/test/port 会转发到 http://localhost:9001/test/port
management:
endpoint:
gateway:
enabled: true
endpoints:
web:
exposure:
include: gateway
同样启动类上添加 @EnableDiscoveryClient 注解
查看所有路由：/actuator/gateway/routes
查看指定路由（GET）：/actuator/gateway/routes/{id}
查看全局过滤器：/actuator/gateway/globalfilters
查看路由过滤器：/actuator/gateway/routefilters
POST 方式刷新路由缓存：/actuator/gateway/refresh
测试
curl http://127.0.0.1:9001/test/port
port=9001, version=2
curl http://127.0.0.1:9000/provider/test/port
port=9001, version=2
动态路由
实现动态路由有两种方式，一个是改写 RouteDefinitionRepository（实测失败），一个是基于 nacos 的监听器给 RouteDefinitionRepository 动态更新值。实现逻辑大同小异
Spring Cloud Gateway 中加载路由信息分别由以下几个类负责
1、PropertiesRouteDefinitionLocator：从配置文件中读取路由信息 (如 YML、Properties 等)
2、RouteDefinitionRepository：从存储器中读取路由信息 (如内存、配置中心、Redis、MySQL 等)
3、DiscoveryClientRouteDefinitionLocator：从注册中心中读取路由信息(如 Nacos、Eurka、Zookeeper 等)
下面使用 RouteDefinitionRepository 配置动态路由
gateway-router.json
[{
"id": "provider",
"predicates": [{
"name": "Path",
"args": {
"_genkey_0": "/provider/**"
}
}],
"filters": [{
"name": "StripPrefix",
"args": {
"_genkey_0": "1"
}
}],
"uri": "lb://provider",
"order": 0
}]
NacosRouteDefinitionRepository 配置类
@Component
public class NacosRouteDefinitionRepository implements RouteDefinitionRepository, ApplicationEventPublisherAware {
private static final Logger log = LoggerFactory.getLogger(NacosRouteDefinitionRepository.class);
@Autowired
private NacosConfigManager nacosConfigManager;
// 更新路由信息需要的
private ApplicationEventPublisher applicationEventPublisher;
private String dataId = "gateway-router.json";
private String group = "DEFAULT_GROUP";
@Value("${spring.cloud.nacos.config.server-addr}")
private String serverAddr;
private ObjectMapper objectMapper = new ObjectMapper();
@PostConstruct
public void dynamicRouteByNacosListener() {
try {
nacosConfigManager.getConfigService().addListener(dataId, group, new Listener() {
public void receiveConfigInfo(String configInfo) {
log.info("自动更新配置...\r\n{}", configInfo);
applicationEventPublisher.publishEvent(new RefreshRoutesEvent(this));
}
public Executor getExecutor() {
return null;
}
});
} catch (NacosException e) {
e.printStackTrace();
}
}
@Override
public Flux<RouteDefinition> getRouteDefinitions() {
try {
String configInfo = nacosConfigManager.getConfigService().getConfig(dataId, group, 5000);
List<RouteDefinition> gatewayRouteDefinitions = objectMapper.readValue(configInfo, new TypeReference<List<RouteDefinition>>() {
});
return Flux.fromIterable(gatewayRouteDefinitions);
} catch (NacosException e) {
e.printStackTrace();
} catch (JsonMappingException e) {
e.printStackTrace();
} catch (JsonProcessingException e) {
e.printStackTrace();
}
return Flux.fromIterable(Lists.newArrayList());
}
@Override
public Mono<Void> save(Mono<RouteDefinition> route) {
return null;
}
@Override
public Mono<Void> delete(Mono<String> routeId) {
return null;
}
@Override
public void setApplicationEventPublisher(ApplicationEventPublisher applicationEventPublisher) {
this.applicationEventPublisher = applicationEventPublisher;
}
}
然后重启网关，访问 http://127.0.0.1:9000/actuator/gateway/routes ，查看是否生效
[
{
"predicate": "Paths: [/provider/**], match trailing slash: true",
"route_id": "provider",
"filters": [
"[[StripPrefix parts = 1], order = 1]"
],
"uri": "lb://provider",
"order": 0
}
]
灰度发布
首先需要明白灰度的场景, 因为有不同版本的服务需要共存, 所以新的节点升级的时候必然代码及配置会存在差别, 所以我们根据这种差别来判断服务版本是新版本还是线上稳定版本。这里我们用 prod 和 gray 来标识 2 个版本。
实现的整体思路：
- 编写带版本号的灰度路由(负载均衡策略)
- 编写自定义 filter
- nacos 服务配置需要灰度发布的服务的元数据信息以及权重(在服务 jar 中配置)
注意, 应该先修改 nacos 配置实现动态路由, 然后再升级灰度节点. 本案例只是简单示例灰度原理。
下面进行网关配置
依赖配置
首先排除掉默认的 ribbon 依赖
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-ribbon</artifactId>
</exclusion>
</exclusions>
</dependency>
引入官方新的负载均衡包
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-loadbalancer</artifactId>
</dependency>
负载均衡策略
public class VersionGrayLoadBalancer implements ReactorServiceInstanceLoadBalancer {
private ObjectProvider<ServiceInstanceListSupplier> serviceInstanceListSupplierProvider;
private String serviceId;
private final AtomicInteger position;
public VersionGrayLoadBalancer(ObjectProvider<ServiceInstanceListSupplier> serviceInstanceListSupplierProvider, String serviceId) {
this(serviceInstanceListSupplierProvider, serviceId, new Random().nextInt(1000));
}
public VersionGrayLoadBalancer(ObjectProvider<ServiceInstanceListSupplier> serviceInstanceListSupplierProvider, String serviceId, int seedPosition) {
this.serviceId = serviceId;
this.serviceInstanceListSupplierProvider = serviceInstanceListSupplierProvider;
this.position = new AtomicInteger(seedPosition);
}
@Override
public Mono<Response<ServiceInstance>> choose(Request request) {
HttpHeaders headers = (HttpHeaders) request.getContext();
ServiceInstanceListSupplier supplier = this.serviceInstanceListSupplierProvider.getIfAvailable(NoopServiceInstanceListSupplier::new);
return ((Flux) supplier.get()).next().map(list -> processInstanceResponse((List<ServiceInstance>) list, headers));
}
private Response<ServiceInstance> processInstanceResponse(List<ServiceInstance> instances, HttpHeaders headers) {
if (instances.isEmpty()) {
return new EmptyResponse();
} else {
String reqVersion = headers.getFirst("version");
if (StringUtils.isEmpty(reqVersion)) {
return processRibbonInstanceResponse(instances);
}
List<ServiceInstance> serviceInstances = instances.stream()
.filter(instance -> reqVersion.equals(instance.getMetadata().get("version")))
.collect(Collectors.toList());
if (serviceInstances.size() > 0) {
return processRibbonInstanceResponse(serviceInstances);
} else {
return processRibbonInstanceResponse(instances);
}
}
}
/**
* 负载均衡器
* 参考 org.springframework.cloud.loadbalancer.core.RoundRobinLoadBalancer#getInstanceResponse
*
* @author javadaily
*/
private Response<ServiceInstance> processRibbonInstanceResponse(List<ServiceInstance> instances) {
int pos = Math.abs(this.position.incrementAndGet());
ServiceInstance instance = instances.get(pos % instances.size());
return new DefaultResponse(instance);
}
}
过滤器加载负载均衡
public class GrayReactiveLoadBalancerClientFilter implements GlobalFilter, Ordered {
private static final Log log = LogFactory.getLog(ReactiveLoadBalancerClientFilter.class);
private static final int LOAD_BALANCER_CLIENT_FILTER_ORDER = 10150;
private final LoadBalancerClientFactory clientFactory;
private LoadBalancerProperties properties;
public GrayReactiveLoadBalancerClientFilter(LoadBalancerClientFactory clientFactory, LoadBalancerProperties properties) {
this.clientFactory = clientFactory;
this.properties = properties;
}
@Override
public int getOrder() {
return LOAD_BALANCER_CLIENT_FILTER_ORDER;
}
@Override
public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
URI url = exchange.getAttribute(ServerWebExchangeUtils.GATEWAY_REQUEST_URL_ATTR);
String schemePrefix = exchange.getAttribute(ServerWebExchangeUtils.GATEWAY_SCHEME_PREFIX_ATTR);
if (url != null && ("grayLb".equals(url.getScheme()) || "grayLb".equals(schemePrefix))) {
ServerWebExchangeUtils.addOriginalRequestUrl(exchange, url);
if (log.isTraceEnabled()) {
log.trace(ReactiveLoadBalancerClientFilter.class.getSimpleName() + " url before: " + url);
}
return this.choose(exchange).doOnNext((response) -> {
if (!response.hasServer()) {
throw NotFoundException.create(this.properties.isUse404(), "Unable to find instance for " + url.getHost());
} else {
URI uri = exchange.getRequest().getURI();
String overrideScheme = null;
if (schemePrefix != null) {
overrideScheme = url.getScheme();
}
DelegatingServiceInstance serviceInstance = new DelegatingServiceInstance((ServiceInstance) response.getServer(), overrideScheme);
URI requestUrl = this.reconstructURI(serviceInstance, uri);
if (log.isTraceEnabled()) {
log.trace("LoadBalancerClientFilter url chosen: " + requestUrl);
}
exchange.getAttributes().put(ServerWebExchangeUtils.GATEWAY_REQUEST_URL_ATTR, requestUrl);
}
}).then(chain.filter(exchange));
} else {
return chain.filter(exchange);
}
}
protected URI reconstructURI(ServiceInstance serviceInstance, URI original) {
return LoadBalancerUriTools.reconstructURI(serviceInstance, original);
}
private Mono<Response<ServiceInstance>> choose(ServerWebExchange exchange) {
URI uri = (URI) exchange.getAttribute(ServerWebExchangeUtils.GATEWAY_REQUEST_URL_ATTR);
VersionGrayLoadBalancer loadBalancer = new VersionGrayLoadBalancer(clientFactory.getLazyProvider(uri.getHost(), ServiceInstanceListSupplier.class), uri.getHost());
if (loadBalancer == null) {
throw new NotFoundException("No loadbalancer available for " + uri.getHost());
} else {
return loadBalancer.choose(this.createRequest(exchange));
}
}
private Request createRequest(ServerWebExchange exchange) {
HttpHeaders headers = exchange.getRequest().getHeaders();
Request<HttpHeaders> request = new DefaultRequest<>(headers);
return request;
}
}
注入过滤器
@Configuration
public class GrayGatewayReactiveLoadBalancerClientAutoConfiguration {
@Bean
@ConditionalOnMissingBean({GrayReactiveLoadBalancerClientFilter.class})
public GrayReactiveLoadBalancerClientFilter grayReactiveLoadBalancerClientFilter(LoadBalancerClientFactory clientFactory, LoadBalancerProperties properties) {
return new GrayReactiveLoadBalancerClientFilter(clientFactory, properties);
}
}
发布灰度服务
生产环境配置文件 application-prod.yml
server:
port: 9002
spring:
application:
name: provider
cloud:
nacos:
config:
server-addr: 127.0.0.1:8848
discovery:
metadata:
version: prod
server-addr: 127.0.0.1:8848
灰度环境配置文件 application-gray.yml
server:
port: 9003
spring:
application:
name: provider
cloud:
nacos:
config:
server-addr: 127.0.0.1:8848
discovery:
metadata:
version: gray
server-addr: 127.0.0.1:8848
idea 启动参数指定配置文件
同时注意配置两个环境的 nacos 文件（prod version: 4，gray version: 5）
测试
然后分别启动三个服务：9000 端口网关 gateway 服务，9002 端口生产环境 provider-prod 服务，9003 端口灰度环境 provider-gray 服务
E:\Nacos\nacos>curl http://127.0.0.1:9000/provider/test/port
port=9003, version=5
E:\Nacos\nacos>curl -X GET -H "version:prod" http://127.0.0.1:9000/provider/test/port
port=9003, version=5
E:\Nacos\nacos>curl -X GET -H "version:gray" http://127.0.0.1:9000/provider/test/port
port=9002, version=4
仓库地址
如果对于文章中代码有疑问，可以直接查看下方作者仓库
仓库地址：ReturnTmp/spring-gateway-demo: 网关配置 + 灰度发布 + 配置中心 示例仓库 (github.com)
参考链接
- Spring Cloud Gateway+nacos灰度发布 - 掘金 (juejin.cn)
- SpringCloud gateway Actuator - shigp1 - 博客园 (cnblogs.com)
- Spring Cloud Gateway监控 周立的博客
- 前端灰度发布落地方案-腾讯云开发者社区-腾讯云 (tencent.com)
- 后端 - 灰度发布架构设计！值得品鉴
本文由博客一文多发平台 OpenWrite 发布！
