重写 Nacos 服务发现：多个服务器如何跨命名空间，访问公共服务？

一、问题背景

在开发某个公共应用时，笔者发现该公共应用的数据是所有测试环境（假设存在 dev/dev2/dev3）通用的。

这就意味着只需部署一个应用，就能满足所有测试环境的需求；也意味着所有测试环境都需要调用该公共应用，而不同测试环境的应用注册在不同的 Nacos 命名空间。

二、两种解决方案

如果所有测试环境都需要调用该公共应用，有两种可行的方案。第一种，将该公共服务同时注册到不同的测试环境所对应的命名空间中。

第二种，将公共应用注册到单独的命名空间，不同的测试环境能够跨命名空间访问该应用。

三、详细的问题解决过程

先行交代笔者的版本号配置。Nacos 客户端版本号为 NACOS 1.4.1；Java 项目的 Nacos 版本号如下。

最初想法是将该公共应用同时注册到多个命名空间下。在查找资料的过程中，团队成员在 GitHub 上发现了一篇类似问题的博客分享：Registration Center: Can services in different namespaces be called from each other? #1176。

01 注册多个命名空间

从该博客中，我们看到其他程序员朋友也遇到了类似的公共服务的需求。在本篇文章中，笔者将进一步分享实现思路以及示例代码。

说明：以下代码内容来自用户 chuntaojun 的分享。

shareNamespace={namespaceId[:group]},{namespaceId[:group]}

@RunWith(SpringRunner.class)
@SpringBootTest(classes = NamingApp.class, properties = {"server.servlet.context-path=/nacos"},
    webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class SelectServiceInShareNamespace_ITCase {

    private NamingService naming1;
    private NamingService naming2;
    @LocalServerPort
    private int port;
    @Before
    public void init() throws Exception{
        NamingBase.prepareServer(port);
        if (naming1 == null) {
            Properties properties = new Properties();
            properties.setProperty(PropertyKeyConst.SERVER_ADDR, "127.0.0.1"+":"+port);
            properties.setProperty(PropertyKeyConst.SHARE_NAMESPACE, "57425802-3058-4507-9a73-3229b9f00a36");
            naming1 = NamingFactory.createNamingService(properties);

            Properties properties2 = new Properties();
            properties2.setProperty(PropertyKeyConst.SERVER_ADDR, "127.0.0.1"+":"+port);
            properties2.setProperty(PropertyKeyConst.NAMESPACE, "57425802-3058-4507-9a73-3229b9f00a36");
            naming2 = NamingFactory.createNamingService(properties2);
        }
        while (true) {
            if (!"UP".equals(naming1.getServerStatus())) {
                Thread.sleep(1000L);
                continue;
            }
            break;
        }
    }

    @Test
    public void testSelectInstanceInShareNamespaceNoGroup() throws NacosException, InterruptedException {
        String service1 = randomDomainName();
        String service2 = randomDomainName();
        naming1.registerInstance(service1, "127.0.0.1", 90);
        naming2.registerInstance(service2, "127.0.0.2", 90);

        Thread.sleep(1000);

        List<Instance> instances = naming1.getAllInstances(service2);
        Assert.assertEquals(1, instances.size());
        Assert.assertEquals(service2, NamingUtils.getServiceName(instances.get(0).getServiceName()));
    }

    @Test
    public void testSelectInstanceInShareNamespaceWithGroup() throws NacosException, InterruptedException {
        String service1 = randomDomainName();
        String service2 = randomDomainName();
        naming2.registerInstance(service1, groupName, "127.0.0.1", 90);
        naming3.registerInstance(service2, "127.0.0.2", 90);

        Thread.sleep(1000);

        List<Instance> instances = naming3.getAllInstances(service1);
        Assert.assertEquals(1, instances.size());
        Assert.assertEquals(service1, NamingUtils.getServiceName(instances.get(0).getServiceName()));
        Assert.assertEquals(groupName, NamingUtils.getServiceName(NamingUtils.getGroupName(instances.get(0).getServiceName())));
    }

}

进一步考虑后发现该解决方案可能不太契合当前遇到的问题。公司目前的开发测试环境有很多个，并且不确定以后会不会继续增加。

如果每增加一个环境，都需要修改一次公共服务的配置，并且重启一次公共服务，着实太麻烦了。倒不如反其道而行，让其他的服务器实现跨命名空间访问公共服务。

02 跨命名空间访问

针对实际问题查找资料时，我们找到了类似的参考分享《重写 Nacos 服务发现逻辑动态修改远程服务IP地址》。

跟着博客思路看代码，笔者了解到服务发现的主要相关类是 NacosNamingService, NacosDiscoveryProperties, NacosDiscoveryAutoConfiguration。

然后，笔者将博客的示例代码复制过来，试着进行如下调试：

@Slf4j
@Configuration
@ConditionalOnNacosDiscoveryEnabled
@ConditionalOnProperty(
        name = {"spring.profiles.active"},
        havingValue = "dev"
)
@AutoConfigureBefore({NacosDiscoveryClientAutoConfiguration.class})
public class DevEnvironmentNacosDiscoveryClient {

    @Bean
    @ConditionalOnMissingBean
    public NacosDiscoveryProperties nacosProperties() {
        return new DevEnvironmentNacosDiscoveryProperties();
    }

    static class DevEnvironmentNacosDiscoveryProperties extends NacosDiscoveryProperties {

        private NamingService namingService;

        @Override
        public NamingService namingServiceInstance() {
            if (null != this.namingService) {
                return this.namingService;
            } else {
                Properties properties = new Properties();
                properties.put("serverAddr", super.getServerAddr());
                properties.put("namespace", super.getNamespace());
                properties.put("com.alibaba.nacos.naming.log.filename", super.getLogName());
                if (super.getEndpoint().contains(":")) {
                    int index = super.getEndpoint().indexOf(":");
                    properties.put("endpoint", super.getEndpoint().substring(0, index));
                    properties.put("endpointPort", super.getEndpoint().substring(index + 1));
                } else {
                    properties.put("endpoint", super.getEndpoint());
                }

                properties.put("accessKey", super.getAccessKey());
                properties.put("secretKey", super.getSecretKey());
                properties.put("clusterName", super.getClusterName());
                properties.put("namingLoadCacheAtStart", super.getNamingLoadCacheAtStart());

                try {
                    this.namingService = new DevEnvironmentNacosNamingService(properties);
                } catch (Exception var3) {
                    log.error("create naming service error!properties={},e=,", this, var3);
                    return null;
                }

                return this.namingService;
            }
        }

    }

    static class DevEnvironmentNacosNamingService extends NacosNamingService {

        public DevEnvironmentNacosNamingService(Properties properties) {
            super(properties);
        }

        @Override
        public List<Instance> selectInstances(String serviceName, List<String> clusters, boolean healthy) throws NacosException {
            List<Instance> instances = super.selectInstances(serviceName, clusters, healthy);
            instances.stream().forEach(instance -> instance.setIp("10.101.232.24"));
            return instances;
        }
    }

}

调试后发现博客提供的代码并不能满足笔者的需求，还得进一步深入探索。

但幸运的是，调试过程发现 Nacos 服务发现的关键类是 com.alibaba.cloud.nacos.discovery.NacosServiceDiscovery，其中的关键方法是 getInstances() 和 getServices()，即「返回指定服务 ID 的所有服务实例」和「获取所有服务的名称」。

也就是说，对 getInstances() 方法进行重写肯定能实现本次目标——跨命名空间访问公共服务。

/**
 * Return all instances for the given service.
 * @param serviceId id of service
 * @return list of instances
 * @throws NacosException nacosException
 */
public List<ServiceInstance> getInstances(String serviceId) throws NacosException {
        String group = discoveryProperties.getGroup();
        List<Instance> instances = discoveryProperties.namingServiceInstance()
                        .selectInstances(serviceId, group, true);
        return hostToServiceInstanceList(instances, serviceId);
}

/**
 * Return the names of all services.
 * @return list of service names
 * @throws NacosException nacosException
 */
public List<String> getServices() throws NacosException {
        String group = discoveryProperties.getGroup();
        ListView<String> services = discoveryProperties.namingServiceInstance()
                        .getServicesOfServer(1, Integer.MAX_VALUE, group);
        return services.getData();
}

03 最终解决思路及代码示例

具体的解决方案思路大致如下：

生成一个共享配置类NacosShareProperties，用来配置共享公共服务的 namespace 和 group；
重写配置类 NacosDiscoveryProperties （新：NacosDiscoveryPropertiesV2），将新增的共享配置类作为属性放进该配置类，后续会用到；
重写服务发现类 NacosServiceDiscovery （新：NacosServiceDiscoveryV2），这是最关键的逻辑；
重写自动配置类 NacosDiscoveryAutoConfiguration，将自定义相关类比 Nacos 原生类更早的注入容器。

最终代码中用到了一些工具类，可以自行补充完整。

/**
 * <pre>
 *  @description: 共享nacos属性
 *  @author: rookie0peng
 *  @date: 2022/8/29 15:22
 *  </pre>
 */
@Configuration
@ConfigurationProperties(prefix = "nacos.share")
public class NacosShareProperties {

    private final Map<String, Set<String>> NAMESPACE_TO_GROUP_NAME_MAP = new ConcurrentHashMap<>();

    /**
     * 共享nacos实体列表
     */
    private List<NacosShareEntity> entities;

    public List<NacosShareEntity> getEntities() {
        return entities;
    }

    public void setEntities(List<NacosShareEntity> entities) {
        this.entities = entities;
    }

    public Map<String, Set<String>> getNamespaceGroupMap() {
        safeStream(entities).filter(entity -> nonNull(entity) && nonNull(entity.getNamespace()))
                .forEach(entity -> {
                    Set<String> groupNames = NAMESPACE_TO_GROUP_NAME_MAP.computeIfAbsent(entity.getNamespace(), k -> new HashSet<>());
                    if (nonNull(entity.getGroupNames()))
                        groupNames.addAll(entity.getGroupNames());
                });
        return new HashMap<>(NAMESPACE_TO_GROUP_NAME_MAP);
    }

    @Override
    public String toString() {
        return "NacosShareProperties{" +
                "entities=" + entities +
                '}';
    }

    /**
     * 共享nacos实体
     */
    public static final class NacosShareEntity {

        /**
         * 命名空间
         */
        private String namespace;

        /**
         * 分组
         */
        private List<String> groupNames;

        public String getNamespace() {
            return namespace;
        }

        public void setNamespace(String namespace) {
            this.namespace = namespace;
        }

        public List<String> getGroupNames() {
            return groupNames;
        }

        public void setGroupNames(List<String> groupNames) {
            this.groupNames = groupNames;
        }

        @Override
        public String toString() {
            return "NacosShareEntity{" +
                    "namespace='" + namespace + '\'' +
                    ", groupNames=" + groupNames +
                    '}';
        }
    }
}

/**
 * @description: naocs服务发现属性重写
 * @author: rookie0peng
 * @date: 2022/8/30 1:19
 */
public class NacosDiscoveryPropertiesV2 extends NacosDiscoveryProperties {

    private static final Logger log = LoggerFactory.getLogger(NacosDiscoveryPropertiesV2.class);

    private final NacosShareProperties nacosShareProperties;

    private static final Map<String, NamingService> NAMESPACE_TO_NAMING_SERVICE_MAP = new ConcurrentHashMap<>();

    public NacosDiscoveryPropertiesV2(NacosShareProperties nacosShareProperties) {
        super();
        this.nacosShareProperties = nacosShareProperties;
    }

    public Map<String, NamingService> shareNamingServiceInstances() {
        if (!NAMESPACE_TO_NAMING_SERVICE_MAP.isEmpty()) {
            return new HashMap<>(NAMESPACE_TO_NAMING_SERVICE_MAP);
        }
        List<NacosShareProperties.NacosShareEntity> entities = Optional.ofNullable(nacosShareProperties)
                .map(NacosShareProperties::getEntities).orElse(Collections.emptyList());
        entities.stream().filter(entity -> nonNull(entity) && nonNull(entity.getNamespace()))
                .filter(PredicateUtil.distinctByKey(NacosShareProperties.NacosShareEntity::getNamespace))
                .forEach(entity -> {
                    try {
                        NamingService namingService = NacosFactory.createNamingService(getNacosProperties(entity.getNamespace()));
                        if (namingService != null) {
                            NAMESPACE_TO_NAMING_SERVICE_MAP.put(entity.getNamespace(), namingService);
                        }
                    } catch (Exception e) {
                        log.error("create naming service error! properties={}, e=", this, e);
                    }
                });
        return new HashMap<>(NAMESPACE_TO_NAMING_SERVICE_MAP);
    }

    private Properties getNacosProperties(String namespace) {
        Properties properties = new Properties();
        properties.put(SERVER_ADDR, getServerAddr());
        properties.put(USERNAME, Objects.toString(getUsername(), ""));
        properties.put(PASSWORD, Objects.toString(getPassword(), ""));
        properties.put(NAMESPACE, namespace);
        properties.put(UtilAndComs.NACOS_NAMING_LOG_NAME, getLogName());
        String endpoint = getEndpoint();
        if (endpoint.contains(":")) {
            int index = endpoint.indexOf(":");
            properties.put(ENDPOINT, endpoint.substring(0, index));
            properties.put(ENDPOINT_PORT, endpoint.substring(index + 1));
        }
        else {
            properties.put(ENDPOINT, endpoint);
        }

        properties.put(ACCESS_KEY, getAccessKey());
        properties.put(SECRET_KEY, getSecretKey());
        properties.put(CLUSTER_NAME, getClusterName());
        properties.put(NAMING_LOAD_CACHE_AT_START, getNamingLoadCacheAtStart());

//        enrichNacosDiscoveryProperties(properties);
        return properties;
    }
}

/**
 * @description: naocs服务发现重写
 * @author: rookie0peng
 * @date: 2022/8/30 1:10
 */
public class NacosServiceDiscoveryV2 extends NacosServiceDiscovery {

    private final NacosDiscoveryPropertiesV2 discoveryProperties;

    private final NacosShareProperties nacosShareProperties;

    private final NacosServiceManager nacosServiceManager;

    public NacosServiceDiscoveryV2(NacosDiscoveryPropertiesV2 discoveryProperties, NacosShareProperties nacosShareProperties, NacosServiceManager nacosServiceManager) {
        super(discoveryProperties, nacosServiceManager);
        this.discoveryProperties = discoveryProperties;
        this.nacosShareProperties = nacosShareProperties;
        this.nacosServiceManager = nacosServiceManager;
    }

    /**
     * Return all instances for the given service.
     * @param serviceId id of service
     * @return list of instances
     * @throws NacosException nacosException
     */
    public List<ServiceInstance> getInstances(String serviceId) throws NacosException {
        String group = discoveryProperties.getGroup();
        List<Instance> instances = discoveryProperties.namingServiceInstance()
                .selectInstances(serviceId, group, true);
        if (isEmpty(instances)) {
            Map<String, Set<String>> namespaceGroupMap = nacosShareProperties.getNamespaceGroupMap();
            Map<String, NamingService> namespace2NamingServiceMap = discoveryProperties.shareNamingServiceInstances();
            for (Map.Entry<String, NamingService> entry : namespace2NamingServiceMap.entrySet()) {
                String namespace;
                NamingService namingService;
                if (isNull(namespace = entry.getKey()) || isNull(namingService = entry.getValue()))
                    continue;
                Set<String> groupNames = namespaceGroupMap.get(namespace);
                List<Instance> shareInstances;
                if (isEmpty(groupNames)) {
                    shareInstances = namingService.selectInstances(serviceId, group, true);
                    if (nonEmpty(shareInstances))
                        break;
                } else {
                    shareInstances = new ArrayList<>();
                    for (String groupName : groupNames) {
                        List<Instance> subShareInstances = namingService.selectInstances(serviceId, groupName, true);
                        if (nonEmpty(subShareInstances)) {
                            shareInstances.addAll(subShareInstances);
                        }
                    }
                }
                if (nonEmpty(shareInstances)) {
                    instances = shareInstances;
                    break;
                }
            }
        }
        return hostToServiceInstanceList(instances, serviceId);
    }

    /**
     * Return the names of all services.
     * @return list of service names
     * @throws NacosException nacosException
     */
    public List<String> getServices() throws NacosException {
        String group = discoveryProperties.getGroup();
        ListView<String> services = discoveryProperties.namingServiceInstance()
                .getServicesOfServer(1, Integer.MAX_VALUE, group);
        return services.getData();
    }

    public static List<ServiceInstance> hostToServiceInstanceList(
            List<Instance> instances, String serviceId) {
        List<ServiceInstance> result = new ArrayList<>(instances.size());
        for (Instance instance : instances) {
            ServiceInstance serviceInstance = hostToServiceInstance(instance, serviceId);
            if (serviceInstance != null) {
                result.add(serviceInstance);
            }
        }
        return result;
    }

    public static ServiceInstance hostToServiceInstance(Instance instance,
                                                        String serviceId) {
        if (instance == null || !instance.isEnabled() || !instance.isHealthy()) {
            return null;
        }
        NacosServiceInstance nacosServiceInstance = new NacosServiceInstance();
        nacosServiceInstance.setHost(instance.getIp());
        nacosServiceInstance.setPort(instance.getPort());
        nacosServiceInstance.setServiceId(serviceId);

        Map<String, String> metadata = new HashMap<>();
        metadata.put("nacos.instanceId", instance.getInstanceId());
        metadata.put("nacos.weight", instance.getWeight() + "");
        metadata.put("nacos.healthy", instance.isHealthy() + "");
        metadata.put("nacos.cluster", instance.getClusterName() + "");
        metadata.putAll(instance.getMetadata());
        nacosServiceInstance.setMetadata(metadata);

        if (metadata.containsKey("secure")) {
            boolean secure = Boolean.parseBoolean(metadata.get("secure"));
            nacosServiceInstance.setSecure(secure);
        }
        return nacosServiceInstance;
    }

    private NamingService namingService() {
        return nacosServiceManager
                .getNamingService(discoveryProperties.getNacosProperties());
    }
}

/**
 * @description: 重写nacos服务发现的自动配置
 * @author: rookie0peng
 * @date: 2022/8/30 1:08
 */
@Configuration(proxyBeanMethods = false)
@ConditionalOnDiscoveryEnabled
@ConditionalOnNacosDiscoveryEnabled
@AutoConfigureBefore({NacosDiscoveryAutoConfiguration.class})
public class NacosDiscoveryAutoConfigurationV2 {

    @Bean
    @ConditionalOnMissingBean
    public NacosDiscoveryPropertiesV2 nacosProperties(NacosShareProperties nacosShareProperties) {
        return new NacosDiscoveryPropertiesV2(nacosShareProperties);
    }

    @Bean
    @ConditionalOnMissingBean
    public NacosServiceDiscovery nacosServiceDiscovery(
            NacosDiscoveryPropertiesV2 discoveryPropertiesV2, NacosShareProperties nacosShareProperties, NacosServiceManager nacosServiceManager
    ) {
        return new NacosServiceDiscoveryV2(discoveryPropertiesV2, nacosShareProperties, nacosServiceManager);
    }
}

本以为问题到这就结束了，但最后自测时发现程序根本不走 Nacos 的服务发现逻辑，而是执行 Ribbon 的负载均衡逻辑com.netflix.loadbalancer.AbstractLoadBalancerRule。

不过实现类是 com.alibaba.cloud.nacos.ribbon.NacosRule，继续基于 NacosRule 重写负载均衡。

/**
 * @description: 共享nacos命名空间规则
 * @author: rookie0peng
 * @date: 2022/8/31 2:04
 */
public class ShareNacosNamespaceRule extends AbstractLoadBalancerRule {

    private static final Logger LOGGER = LoggerFactory.getLogger(ShareNacosNamespaceRule.class);

    @Autowired
    private NacosDiscoveryPropertiesV2 nacosDiscoveryPropertiesV2;
    @Autowired
    private NacosShareProperties nacosShareProperties;

    /**
     * 重写choose方法
     *
     * @param key
     * @return
     */
    @SneakyThrows
    @Override
    public Server choose(Object key) {
        try {
            String clusterName = this.nacosDiscoveryPropertiesV2.getClusterName();
            DynamicServerListLoadBalancer loadBalancer = (DynamicServerListLoadBalancer) getLoadBalancer();
            String name = loadBalancer.getName();

            NamingService namingService = nacosDiscoveryPropertiesV2
                    .namingServiceInstance();
            List<Instance> instances = namingService.selectInstances(name, true);
            if (CollectionUtils.isEmpty(instances)) {
                LOGGER.warn("no instance in service {}, then to get share service's instance", name);
                List<Instance> shareNamingService = this.getShareNamingService(name);
                if (nonEmpty(shareNamingService))
                    instances = shareNamingService;
                else
                    return null;
            }
            List<Instance> instancesToChoose = instances;
            if (org.apache.commons.lang3.StringUtils.isNotBlank(clusterName)) {
                List<Instance> sameClusterInstances = instances.stream()
                        .filter(instance -> Objects.equals(clusterName,
                                instance.getClusterName()))
                        .collect(Collectors.toList());
                if (!CollectionUtils.isEmpty(sameClusterInstances)) {
                    instancesToChoose = sameClusterInstances;
                }
                else {
                    LOGGER.warn(
                            "A cross-cluster call occurs，name = {}, clusterName = {}, instance = {}",
                            name, clusterName, instances);
                }
            }

            Instance instance = ExtendBalancer.getHostByRandomWeight2(instancesToChoose);

            return new NacosServer(instance);
        }
        catch (Exception e) {
            LOGGER.warn("NacosRule error", e);
            return null;
        }
    }


    @Override
    public void initWithNiwsConfig(IClientConfig iClientConfig) {

    }

    private List<Instance> getShareNamingService(String serviceId) throws NacosException {
        List<Instance> instances = Collections.emptyList();
        Map<String, Set<String>> namespaceGroupMap = nacosShareProperties.getNamespaceGroupMap();
        Map<String, NamingService> namespace2NamingServiceMap = nacosDiscoveryPropertiesV2.shareNamingServiceInstances();
        for (Map.Entry<String, NamingService> entry : namespace2NamingServiceMap.entrySet()) {
            String namespace;
            NamingService namingService;
            if (isNull(namespace = entry.getKey()) || isNull(namingService = entry.getValue()))
                continue;
            Set<String> groupNames = namespaceGroupMap.get(namespace);
            List<Instance> shareInstances;
            if (isEmpty(groupNames)) {
                shareInstances = namingService.selectInstances(serviceId, true);
                if (nonEmpty(shareInstances))
                    break;
            } else {
                shareInstances = new ArrayList<>();
                for (String groupName : groupNames) {
                    List<Instance> subShareInstances = namingService.selectInstances(serviceId, groupName, true);
                    if (nonEmpty(subShareInstances)) {
                        shareInstances.addAll(subShareInstances);
                    }
                }
            }
            if (nonEmpty(shareInstances)) {
                instances = shareInstances;
                break;
            }
        }
        return instances;
    }
}

至此问题得以解决。

在 Nacos 上配置好共享 namespace 和 group 后，就能够进行跨命名空间访问了。

# nacos共享命名空间配置 示例
nacos.share.entities[0].namespace=e6ed2017-3ed6-4d9b-824a-db626424fc7b
nacos.share.entities[0].groupNames[0]=DEFAULT_GROUP
# 指定服务使用共享的负载均衡规则，service-id是注册到nacos上的服务id，ShareNacosNamespaceRule需要写全限定名
service-id.ribbon.NFLoadBalancerRuleClassName=***.***.***.ShareNacosNamespaceRule

注意：如果 Java 项目的 nacos discovery 版本用的是 2021.1，则不需要重写 Ribbon 的负载均衡类，因为该版本的 Nacos 不依赖 Ribbon。

2.2.1.RELEASE 版本的 nacos discovery 依赖 Ribbon.

2021.1 版本的 nacos discovery 不依赖 Ribbon。

四、总结

为了达到共享命名空间的预期，构思、查找资料、实现逻辑、调试，前后一共花费 4 天时间。成就感满满的同时，笔者也发现该功能仍存在共享服务缓存等可优化空间，留待后续实现。

五、参考文献

[1] Registration Center: Can services in different namespaces be called from each other? [EB/OL]. https://github.com/alibaba/nacos/issues/1176, 2019-05-07/2022-11-29.

[2] 重写Nacos服务发现逻辑动态修改远程服务IP地址 [EB/OL]. https://www.cnblogs.com/changxy-codest/p/14632574.html, 2021-04-08/2022-11-29.