java 整合 Elastic 8.

news2024/11/26 2:55:50

1. 准备工作

使用docker 快速搭建的环境,官网docker-compose 方式搭建的集群

设置了密码登录 elastic elastic

需要给jdk 导入证书

找到 证书对应目录,复制到桌面。主要导入下面2个证书,执行如下命令

keytool -importcert -alias "修改成你的证书名" -keystore "D:\Program Files\Java\jdk-18\lib\security\cacerts" -file "你的es 证书地址"

导入证书(需要导入2个如下)

keytool -importcert -alias es01 -keystore "D:\Program Files\Java\jdk-18\lib\security\cacerts" -file "E:\桌面\certs\certs\es01\es01.crt"
keytool -importcert -alias mycert -keystore "D:\Program Files\Java\jdk-18\lib\security\cacerts" -file "E:\桌面\certs\ca\ca.crt"

下面就可以进行连接了

2、使用httpClient

插入pom

<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.13</version>
</dependency>

创建一个重试handler

public class MyRequestRetryHandler implements HttpRequestRetryHandler {
    
    public boolean retryRequest(
            IOException exception,
            int executionCount,
            HttpContext context) {
        if (executionCount >= 3) {
            // Do not retry if over max retry count
            return false;
        }
        if (exception instanceof InterruptedIOException) {
            // Timeout
            return false;
        }
        if (exception instanceof UnknownHostException) {
            // Unknown host
            return false;
        }
        if (exception instanceof ConnectTimeoutException) {
            // Connection refused
            return false;
        }
        if (exception instanceof SSLException) {
            // SSL handshake exception
            return false;
        }
        HttpClientContext clientContext = HttpClientContext.adapt(context);
        HttpRequest request = clientContext.getRequest();
        return !(request instanceof HttpEntityEnclosingRequest);
    }
}

kibana 准备数据

# 插入数据
PUT /test_index/_doc/1
{
  "name":"zhangsan",
  "age":"12"
}
 # 查询数据
 GET /test_index/_doc/1

安全登录认证测试

public class AppWithSecurity {

    private static String wsUrl = "https://localhost:9200";

    public static void main(String[] args) throws Exception {

        CloseableHttpClient client = HttpClients.custom().setRetryHandler(new MyRequestRetryHandler()).build();
        HttpGet method = new HttpGet(wsUrl +"/test_index/_doc/1");
        // Execute the method.

        HttpHost targetHost = new HttpHost("localhost", 9200, "https");


        CredentialsProvider credsProvider = new BasicCredentialsProvider();

        credsProvider.setCredentials(new AuthScope(targetHost.getHostName(), targetHost.getPort()),
                new UsernamePasswordCredentials("elastic", "elastic"));
        // 创建认证实例

        AuthCache authCache = new BasicAuthCache();

        // Generate BASIC scheme object and add it to local auth cache

        BasicScheme basicAuth = new BasicScheme();
        authCache.put(targetHost, basicAuth);

        // Add AuthCache to the execution context
        HttpClientContext context = HttpClientContext.create();
        context.setCredentialsProvider(credsProvider);

        method.addHeader("Accept-Encoding", "gzip");

        try {
            CloseableHttpResponse response = client.execute(method, context);

            if (response.getStatusLine().getStatusCode() != HttpStatus.SC_OK) {
                System.err.println("Method failed: " + response.getStatusLine());
            } else {
                HttpEntity entity = response.getEntity();
                String responseBody = EntityUtils.toString(entity);
                System.out.println(responseBody);
            }

        } catch (IOException e) {
            System.err.println("Fatal transport error: " + e.getMessage());
            e.printStackTrace();
        } finally {
            // Release the connection.
            method.releaseConnection();
        }


    }
}

image-20230711164024548

不使用安全认证

可以使用如下代码

public class App {
	//不使用安全认证不用https 请求,使用http
    private static String wsUrl = "http://127.0.0.1:9200";


    public static void main(String[] args) throws Exception {

        CloseableHttpClient client = HttpClients.custom()
                .setRetryHandler(new MyRequestRetryHandler()).build();
        HttpGet method = new HttpGet(wsUrl + "/test_index/_doc/1");
        // Execute the method.


        try {

            CloseableHttpResponse response = client.execute(method);

            if (response.getStatusLine().getStatusCode() != HttpStatus.SC_OK) {
                System.err.println("Method failed: " + response.getStatusLine());
            } else {
                HttpEntity entity = response.getEntity();
                String responseBody = EntityUtils.toString(entity);
                System.out.println(responseBody);
            }

        } catch (IOException e) {
            System.err.println("Fatal transport error: " + e.getMessage());
            e.printStackTrace();
        } finally {
            // Release the connection.
            method.releaseConnection();
        }
    }
}

3. 官方 Java Api

证书,操作如上1,不能省

[Elasticsearch Java API Client 8.8] |弹性的

启动kibana

image-20230711174255778

image-20230711175539705

引入依赖

<dependency>
    <groupId>co.elastic.clients</groupId>
    <artifactId>elasticsearch-java</artifactId>
    <version>8.3.3</version>
</dependency>

<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>2.12.3</version>
</dependency>
<dependency>
    <groupId>jakarta.json</groupId>
    <artifactId>jakarta.json-api</artifactId>
    <version>2.0.1</version>
</dependency>

<dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.12</version>
</dependency>
<dependency>
    <groupId>org.projectlombok</groupId>
    <artifactId>lombok</artifactId>
    <version>1.18.22</version>

</dependency>
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-api</artifactId>
    <version>2.0.5</version>
</dependency>
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-simple</artifactId>
    <version>2.0.5</version>
</dependency>

创建客户端

public ElasticsearchClient esClient(){
    // URL and API key
    String serverUrl = "https://localhost:9200";
    //官网提示如何获取apikey https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/getting-started-java.html
    String apiKey = "cHdCUlJJa0JSTzdJbVI1dHhKQ2o6cWp3VWVOTmxUamV3bW9pZE9MRXRFQQ==";

    // Create the low-level client
    RestClient restClient = RestClient
            .builder(HttpHost.create(serverUrl))
            .setDefaultHeaders(new Header[]{
                    new BasicHeader("Authorization", "ApiKey " + apiKey)
            })
            .build();

    // Create the transport with a Jackson mapper
    ElasticsearchTransport transport = new RestClientTransport(
            restClient, new JacksonJsonpMapper());

    // And create the API client
    return new ElasticsearchClient(transport);

}

Crud 测试完整代码

@Data
@AllArgsConstructor
@NoArgsConstructor
public class Product {

    private String id;
    private String name;
    private Double sku;
    
}

根据官网写的,个别方法有坑,不知道是不是版本问题,我用的8.3.3,下面是自己改好的

@Slf4j(topic = "logger")
public class Es8ApI {


    private static final Logger logger = LoggerFactory.getLogger(Es8ApI.class);
    public static ElasticsearchClient getEsClient(){
        // URL and API key
        String serverUrl = "https://localhost:9200";
        //官网提示如何获取apikey https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/getting-started-java.html
        String apiKey = "cndCc1JJa0JSTzdJbVI1dFhwQng6WlNCbVZtUXNTdG1WZExVQlExVFV2dw==";

        // Create the low-level client
        RestClient restClient = RestClient
                .builder(HttpHost.create(serverUrl))
                .setDefaultHeaders(new Header[]{
                        new BasicHeader("Authorization", "ApiKey " + apiKey)
                })
                .build();

        // Create the transport with a Jackson mapper
        ElasticsearchTransport transport = new RestClientTransport(
                restClient, new JacksonJsonpMapper());

        // And create the API client
        return new ElasticsearchClient(transport);

    }
    //获取客户端
    private static final ElasticsearchClient esClient = getEsClient();
    @Test
    public  void create() throws IOException {


        //创建索引
        CreateIndexResponse response = esClient.indices().create(c -> c
                .index("products")
        );
        log.info(response.toString());


    }
    @Test
    public void indexDoc() throws IOException {
        //插入文档
        Product product = new Product("bk-1", "City bike", 123.0);

        CreateResponse response = esClient.create(
                builder -> builder
                .index("products")
                .id(product.getId())
                .document(product)
        );

        log.info("Indexed with version " + response.version());
    }
    @Test
    public void getDoc() throws IOException {
        GetResponse<Product> response = esClient.get(
                builder -> builder
                        .index("products")
                        .id("bk-1"),
                Product.class
        );
        log.info(response.toString());

        if (response.found()) {
            Product product = response.source();
            logger.info("Product name " + product.getName());
        } else {
            logger.info ("Product not found");
        }

    }
    @Test
    public void searchDoc() throws IOException {
        String searchText = "bike";

        SearchResponse<Product> response = esClient.search(
                builder -> builder
                        .index("products")
                        .query(q -> q
                                .match(t -> t
                                        .field("name")
                                        .query(searchText)
                                )
                        ),
                Product.class
        );
        log.info(response.toString());


    }

    @Test
    public void updateDoc() throws IOException {

        Product product = new Product("bk-1", "Big bike", 127.0);

        esClient.update(builder ->builder.index("products")
                .id("bk-1")
                //官网坑人,坑小白
                .doc(product)
                ,Product.class);

    }

    @Test
    public void deleteDoc() throws IOException {
        esClient.delete(d -> d.index("products").id("bk-1"));
    }

    @Test

    public void deleteIndex() throws IOException {
        esClient.indices().delete(c -> c
                .index("products")
        );
    }
}

更多操作,请查看官网

4.springboot 整合 ES8

经过进一步的学习。重写client 配置类。有2种获取client 的方法,一种是账号密码连接,一直是使用apikey

配置文件application.yml

spring:
  elasticsearch:
    rest:
      # 是否启用es
      enable: true
      # 多个IP逗号隔开
      hosts: 127.0.0.1:9200
      # 账号密码登录
      username: elastic
      password: elastic
      # http 授权证书
      crtName: ca.crt
      # 自定义apikey
      apikey: cndCc1JJa0JSTzdJbVI1dFhwQng6WlNCbVZtUXNTdG1WZExVQlExVFV2dw==

ElasticSearchConfig配置类

/**
 * es8的Java客户端配置
 * author:Geng
 */
@Configuration
@Slf4j
public class ElasticSearchConfig {

    @Value("${spring.elasticsearch.rest.enable}")
    private boolean enable;



    @Value("${spring.elasticsearch.rest.hosts}")
    private String hosts;


    @Value("${spring.elasticsearch.rest.username}")
    private String userName;
    @Value("${spring.elasticsearch.rest.password}")
    private String passWord;
    @Value("${spring.elasticsearch.rest.crtName}")
    private String tempCrtName;
    @Value("${spring.elasticsearch.rest.apikey}")
    private String apikey;

    private static String crtName;



    @PostConstruct
    private void init() {
        crtName = tempCrtName;
    }

    /**
     * 解析配置的字符串,转为HttpHost对象数组
     * @return
     */
    private HttpHost[] toHttpHost() {
        if (!StringUtils.hasLength(hosts)) {
            throw new RuntimeException("invalid elasticsearch configuration");
        }

        String[] hostArray = hosts.split(",");
        HttpHost[] httpHosts = new HttpHost[hostArray.length];
        HttpHost httpHost;
        for (int i = 0; i < hostArray.length; i++) {
            String[] strings = hostArray[i].split(":");
            httpHost = new HttpHost(strings[0], Integer.parseInt(strings[1]), "https");
            httpHosts[i] = httpHost;
        }

        return httpHosts;
    }


    @Bean
    @Qualifier("clientByPasswd")
    public ElasticsearchClient clientByPasswd() throws Exception {
        ElasticsearchTransport transport = getElasticsearchTransport(userName, passWord,toHttpHost() );
        return new ElasticsearchClient(transport);
    }

    /**
     * apikey 方式访问
     * @return
     */
    @Bean
    @Qualifier("apiKeyBean")
    public  ElasticsearchClient esClientByAPIKEy(){

        // 2.自签证书的设置,并且还包含了账号密码
        RestClientBuilder.HttpClientConfigCallback callback = httpAsyncClientBuilder -> httpAsyncClientBuilder
                .setSSLContext(buildSSLContext())
                .setSSLHostnameVerifier(NoopHostnameVerifier.INSTANCE);

        // Create the low-level client
        RestClient restClient = RestClient
                .builder(toHttpHost())
                .setDefaultHeaders(new Header[]{
                        new BasicHeader("Authorization", "ApiKey " + this.apikey)
                })
                .setHttpClientConfigCallback(callback)
                .build();

        // Create the transport with a Jackson mapper
        ElasticsearchTransport transport = new RestClientTransport(
                restClient, new JacksonJsonpMapper());

        // And create the API client
        return new ElasticsearchClient(transport);

    }

    /**
     * http 证书认证
     * @return
     */
    private static SSLContext buildSSLContext() {
        ClassPathResource resource = new ClassPathResource(crtName);
        SSLContext sslContext = null;
        try {
            CertificateFactory factory = CertificateFactory.getInstance("X.509");
            Certificate trustedCa;
            try (InputStream is = resource.getInputStream()) {
                trustedCa = factory.generateCertificate(is);
            }
            KeyStore trustStore = KeyStore.getInstance("pkcs12");
            trustStore.load(null, null);
            trustStore.setCertificateEntry("ca", trustedCa);
            SSLContextBuilder sslContextBuilder = SSLContexts.custom()
                    .loadTrustMaterial(trustStore, null);
            sslContext = sslContextBuilder.build();
        } catch (CertificateException | IOException | KeyStoreException | NoSuchAlgorithmException |
                 KeyManagementException e) {
            log.error("ES连接认证失败", e);
        }

        return sslContext;
    }

    private static ElasticsearchTransport getElasticsearchTransport(String username, String passwd, HttpHost... hosts ) {
        // 1.账号密码的配置
        final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
        credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, passwd));

        // 2.自签证书的设置,并且还包含了账号密码
        RestClientBuilder.HttpClientConfigCallback callback = httpAsyncClientBuilder -> httpAsyncClientBuilder
                .setSSLContext(buildSSLContext())
                .setSSLHostnameVerifier(NoopHostnameVerifier.INSTANCE)
                .setDefaultCredentialsProvider(credentialsProvider);

        //3. 用builder创建RestClient对象
        RestClient client = RestClient
                .builder( hosts)
                .setHttpClientConfigCallback(callback)
                .build();
        return new RestClientTransport(client, new JacksonJsonpMapper());
    }

}

由于2种连接方式都使用了https 认证,需要把es 自定义证书,复制到resources 文件下,命名为ca.crt

这种方式,就不需要在本机jdk 中添加http证书了。代码中验证了

@Service
public class EsService {
    private static final Logger logger = LoggerFactory.getLogger(EsService.class);

    @Autowired
    @Qualifier("apiKeyBean")
    private ElasticsearchClient esClient;

    public String addIndex(String name) throws IOException {
        //创建索引
        CreateIndexResponse response = esClient.indices().create(c -> c
                .index(name)
        );
        logger.info(response.toString());
        return response.toString();
    }

}

修改@Qualifier("apiKeyBean")切换2种方式检测是否可行

@RestController
public class EsController {
    @Autowired
    private EsService esService;
    @GetMapping("/test/{index}")
    public String test(@PathVariable("index")String index) throws IOException {

        return esService.addIndex(index);

    }
}

访问 http://localhost/test/ada 后面字段随便输,不能输入相同的,输入相同的会报错

结果

image-20230711231233564

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.coloradmin.cn/o/742938.html

如若内容造成侵权/违法违规/事实不符,请联系多彩编程网进行投诉反馈,一经查实,立即删除!

相关文章

[数字图像处理]第八章 图像压缩

文章目录 第八章 图像压缩引言8.1 基础知识8.1.1 编码冗余8.1.2 空间冗余和时间冗余8.1.3 不相关的信息8.1.4 图像信息的度量山农第一定理 8.1.5 保真度准则8.1.6 图像压缩模型编码或压缩过程解码或解压缩过程 8.2 一些基本的压缩方法8.2.1 霍夫曼编码8.2.2 Golomb编码8.2.3 算…

开启visual studio,git for windows 疯狂占用内存的解决

1、开启visual studio&#xff0c;git for windows 疯狂占用内存的解决 最近开启visual studio写代码的时候&#xff0c;IDE总是会莫名奇妙的卡住然后闪退&#xff0c;今天打开任务管理器看了下原因&#xff0c;发现是visual studio所占磁盘内存不断疯涨&#xff0c;每秒几十M…

基于单片机语音识别智能家居系统的设计与实现

功能介绍 以STM32单片机作为主控系统&#xff1b;液晶显示当前环境温湿度&#xff0c;用电器开关状态通过语音模块识别设定的语音&#xff1b;DHT11进行环境温湿度采集&#xff1b;通过语音播报模块报当前温湿度&#xff0c;智能回复通过语音识别可以打开灯&#xff0c;窗帘&am…

LangChain大型语言模型(LLM)应用开发(三):QA over Documents

LangChain是一个基于大语言模型&#xff08;如ChatGPT&#xff09;用于构建端到端语言模型应用的 Python 框架。它提供了一套工具、组件和接口&#xff0c;可简化创建由大型语言模型 (LLM) 和聊天模型提供支持的应用程序的过程。LangChain 可以轻松管理与语言模型的交互&#x…

Jenkins 创建一个 job , 用于单独执行脚本

目录 1.首先,在Jenkins中创建一个新的job 2.之后&#xff0c;会进入配置页面&#xff0c;在配置页面进行配置。 2.1.找到【Build Steps】在下&#xff0c;拉菜单中选择「シェルの実行」 &#xff08;Shell的运行&#xff09; 2.2.之后&#xff0c;会出现シェルスクリプト …

文本挖掘 day2 文本挖掘的研究趋势:期刊语义网络和主路径分析(2020年)

期刊语义网络和主路径分析 文本挖掘的研究趋势&#xff1a;期刊语义网络和主路径分析&#xff08;2020年&#xff09;1. 简介2. 理论背景2.1 文本挖掘2.2 网络分析2.3. 主路径分析2.4. 文献分析 3. 文本挖掘研究的分析和分类3.1. 分析范围3.2. 文本挖掘研究的学术领域 4. 选定期…

红帽回应对开源的承诺

导读红帽上周宣布了限制源代码访问性的政策&#xff0c;称其企业发行版 RHEL (Red Hat Enterprise Linux) 相关源码仅通过 CentOS Stream 公开&#xff0c;付费客户和合作伙伴可通过 Red Hat Customer Portal 访问到源代码。 红帽上周宣布了限制源代码访问性的政策&#xff0c…

spring springmvc springboot mybatis

Spring框架中的单例bean是线程安全的吗? 面试官&#xff1a;什么是AOP 候选人&#xff1a; aop是面向切面编程&#xff0c;在spring中用于将那些与业务无关&#xff0c;但却对多个对象产 生影响的公共行为和逻辑&#xff0c;抽取公共模块复用&#xff0c;降低耦合&#xff0c…

【专题速递】传输网络优化(长专题)

// RTC如何助力远程医疗&#xff1f;5G的差异化体现在哪&#xff1f;SMT如何将远程做到极致&#xff1f;7月29日LiveVideoStackCon2023上海站传输网络优化专场&#xff0c;为您解答。 传输网络优化 为应对互联网日益增长的加速需求、复杂的网络环境以及多种多样的视频业务&am…

第12讲:剖析 Trace 在 SkyWalking 中的落地实现方案(上)

SkyWalking 中 Trace 的相关概念以及实现类与 OpenTracing 中的概念基本类似&#xff0c;像 Trace、Span、Tags、Logs 等核心概念&#xff0c;在 SkyWalking Agent 中都有对应实现&#xff0c;只是在细微实现上略有区别的&#xff0c;其中最重要的是&#xff1a; SkyWalking 的…

Redis——redis的java客户端+(Jedis和SpringDataRedis)

客户端对比 一个Spring Data Redis底层可以兼容前两个 Jedis快速入门 直接使用命令名作为方法名就是Jedis好学的原因 第一步 &#xff1a; 创建一个maven项目并导入如下依赖&#xff0c;除了redis的以来还有一个junit5的依赖 <!--redis的依赖--><dependency>&l…

如何使用3D转换工具HOOPS Exchange与LibConverter进行流缓存导出?

如果您正在使用HOOPS Communicator&#xff0c;您可能想在生成流缓存模型之前利用HOOPS Exchange的高级功能和转换选项。 申请HOOPS试用 HOOPS中文网 如何使用 如您所知&#xff0c;LibConverter是HOOPS Communicator软件包中包含的一个简单的API&#xff0c;conver…

【GeoDa实用技巧100例】003:GeoDa连接并打开多种格式的数据源

GeoDa支持 shapefile、地理数据库、GeoJSON、MapInfo、GML、KML 以及 GDAL 库支持的其他矢量数据格式。该程序还将表格格式&#xff08;.csv、.dbf、.xls、.ods&#xff09;中的坐标转换为这些空间数据格式之一&#xff0c;并在不同文件格式之间转换数据。 文章目录 一、连接文…

深入了解 CSS 变量,让 CSS 创造更多可能!

日常开发中所说的 CSS 变量&#xff0c;实际上是 CSS 的变量函数 var() 与 CSS 自定义属性的统称。 CSS 变量带来的提升: 使得开发和维护成本更低了&#xff0c;如让整个网站的换肤变得更容易&#xff1b;改变了在图形交互效果的实现中 JavaScript 的占据比重&#xff0c;使得…

EasyUI Pagination 分页的两种做法小结

EasyUI Pagination 分页的两种做法小结 EasyUI 的 datagrid 支持服务器端分页&#xff0c;但是官方的资料比较少&#xff0c;以下总结了两种 datagrid 的服务器端分页机制&#xff0c;一种是datagrid默认机制&#xff0c;另一种是利用 Ajax 获取数据并填充 Datagrid&#xff0c…

矩形面积 (力扣)数学推理 JAVA

给你 二维 平面上两个 由直线构成且边与坐标轴平行/垂直 的矩形&#xff0c;请你计算并返回两个矩形覆盖的总面积。 每个矩形由其 左下 顶点和 右上 顶点坐标表示&#xff1a; 第一个矩形由其左下顶点 (ax1, ay1) 和右上顶点 (ax2, ay2) 定义。 第二个矩形由其左下顶点 (bx1, b…

系统驱动作业

作业1 myled_time.c #include <linux/init.h> #include <linux/module.h> #include<linux/of.h> #include<linux/gpio.h> #include<linux/of_gpio.h> #include<linux/timer.h>struct device_node *dnode; unsigned int gpiono; struct t…

Redis — 不仅仅是缓存

1*qIy3PMmEWNcD9Czh_21C8g.png Redis是一种快速、开源的内存键值&#xff08;NoSQL&#xff09;数据库&#xff0c;远远超越了缓存的功能。Redis使用RAM进行操作&#xff0c;提供亚毫秒级的响应时间&#xff0c;支持每秒数百万次请求。Redis主要用于缓存&#xff0c;但它也可以…

【CSS】文字渐变色

CSS设置文字渐变色 background-image: -webkit-linear-gradient(bottom,red,#fd8403,yellow); -webkit-background-clip: text; // 设置背景作用域 -webkit-text-fill-color: transparent;

docker数据卷权限管理--理论和验证

一、Docker容器中用户权限管理 Linux系统的权限管理是由uid和gid负责&#xff0c;Linux系统会检查创建进程的uid和gid&#xff0c;以确定它是否有足够的权限修改文件&#xff0c;而非是通过用户名和用户组来确认。 同样&#xff0c;在docker容器中主机上运行的所有容器共享同一…