微服务实用篇6-分布式搜索elasticsearch篇2

今天我们继续学习分布式搜索引擎elasticsearch，今天主要学习四个模块，分别为DSL查询文档，搜索结果处理，RestClient查询文档，还有最好演示一个旅游案例。下面开始今天的学习吧。

一、DSL查询文档

1.1、DSL查询分类

1.2、DSL查询之复合查询function_score

1.3、DSL查询之复合查询boolean

二、ES搜索结果处理

2.1、搜索结果处理-排序

2.2、搜索结果处理-分页

2.3、搜索结果处理-高亮

三、RestClient查询文档

3.1、快速入门

3.2、RestClient查询文档-match、term、range、bool

3.3、RestClient搜索结果处理-排序和分页

3.4、RestClient搜索结果处理-高亮显示

四、旅游案例

4.1、搜索与分页

4.2、条件过滤

4.3、附近的酒店距离升序排序

一、DSL查询文档

1.1、DSL查询分类

常见的DSL查询分为如下几类：查询所有、全文查询、精确查询、地理查询、复合查询。


#查询所有
GET /hotel/_search
{
  "query":{ 
    "match_all": {}
  }
}

#全文检索-match单字段查询
GET /hotel/_search
{
  "query": {
    "match": {
      "name": "上海"
    }
  }
}

#全文检索-multi_match多字段查询
GET /hotel/_search
{
  "query": {
    "multi_match": {
      "query": "上海如家外滩"
      , "fields": ["brand","name","business"]
    }
  }
}

#精确查询-term查询
GET /hotel/_search
{
  "query": {
    "term": {
      "city.keyword": {
       "value": "上海"
       }
    }
  }
}

#精确查询-range查询
GET /hotel/_search
{
  "query": {
    "range": {
      "price": {
       "gt": 100
       , "lt": 10000
       }
    }
  }
}

#distance查询
GET /hotel/_search
{
  "query": {
    "geo_distance": {
      "distance": "30km",
      "location": "31, 121"
    }
  }
}

1.2、DSL查询之复合查询function_score

function_score的查询方式可以修改文档的相关性分，根据新的得分进行排序，主要包含四个部分，分别为：原始查询条件、过滤条件、算分函数、加权模式。

我们看下面的一个案例，给名为如家的酒店排名设计靠前一点，具体如下：

我们查询所有名称包含“外滩”的酒店名，然后使用算分函数对brand为如家的进行过滤，将其排名靠前一些，如下所示。

#function-score查询
GET /hotel/_search
{
  "query": {
    "function_score": {
      "query": {
        "match": {
          "name": "外滩"
        }
      }
      , "functions": [
        {
          "filter": {
            "term": {
              "brand.keyword": "如家"
            }
          },
          "weight": 12
        }
      ],
      "boost_mode": "sum"
    }
  }
}

1.3、DSL查询之复合查询boolean

boolean查询和function_score都是复合查询，但是boolean查询是不参与算分的，只返回是否满足，must是必须满足、should是满足其中之一、must_not是必须不满足、filter是过滤。

我们看下面的案例，首先是must必须包含如家，价格大于400取反，用filter过滤在坐标范围的10km内的酒店，具体如下：

二、ES搜索结果处理

2.1、搜索结果处理-排序

我们先看第一个案例，使用sort对酒店先按照评价进行降序，再按照价格进行升序排序。

#sort排序
GET /hotel/_search
{
  "query": {
    "match_all": {} 
    },
    "sort":[{
      "score": "desc"
    },
    {
      "price": "asc"
    }]
  }

下面演示按地理位置进行升序排序，具体如下：

#sort排序
GET /hotel/_search
{
  "query": {
    "match_all": {} 
    },
    "sort":[
      {
     "_geo_distance": {
       "location": {
         "lat": 22.507276,
         "lon": 113.931251
       } ,
       "order": "asc"
       , "unit": "km"
     }
    }
    ]
}

2.2、搜索结果处理-分页

ES通过修改from和size的参数来控制返回分页的结果，具体如下：按价格进行升序排序，然后返回10条数据，如下所示：

因为ES是分布式搜索，因此会面临深度分页的问题，他需要把所有的如下1000条数据都查询并聚合后重新排序，再从中截取10条文档数据，这样的话，可能会导致搜索页数过深，对内存和CPU的消耗也很大。

ES分页一般有如下三种分页方式，目前用的比较多的还是from+size的搜索分页方式。

2.3、搜索结果处理-高亮

高亮：即对搜索的结果进行关键字突出显示，给高亮的字段加标签即可。

将名称为如家的字段进行高亮显示，具体如下，标签可以省略，自动默认em标签：

GET /hotel/_search
{
  "query": {
    "match": {
      "name": "如家"
    }
  },
  "highlight": {
    "fields": {
      "name": {}
    }
  }
}

三、RestClient查询文档

3.1、快速入门

Match_all查询：主要包括两部分，DSL请求的发送，以及Json结果的解析，具体如下：


    @Test
    void testMatchAll() throws IOException {
        //1.创建request对象
        SearchRequest searchRequest = new SearchRequest("hotel") ;
        //2.准备DSL
        searchRequest.source().query(QueryBuilders.matchAllQuery()) ;
        //3.发送请求
        SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT) ;
        //4.解析响应
        SearchHits searchHits = response.getHits() ;
        //获取总条数
        long total = searchHits.getTotalHits().value ;
        System.out.println("一共搜索到了" + total + "条数据！");
        //获取文档数组
        SearchHit [] searchHits1 = searchHits.getHits() ;
        for(SearchHit searchHits2 : searchHits1){
            String json = searchHits2.getSourceAsString() ;
            //反序列化
            HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class) ;
            System.out.println(hotelDoc);
        }
    }

3.2、RestClient查询文档-match、term、range、bool

下面演示mtach查询，查询name中包含“如家”的酒店名称，具体如下，其中我把解析响应单独抽取出来作为一个方法，直接调用方法进行解析即可，不用重复撰写。


    @Test
    void testMatch() throws IOException {
        //1.创建request对象
        SearchRequest searchRequest = new SearchRequest("hotel") ;
        //2.准备DSL
        searchRequest.source().query(QueryBuilders.matchQuery("name","如家")) ;
        //3.发送请求
        SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT) ;
        handleResponse(response);
    }

    private void handleResponse(SearchResponse response) {
        //4.解析响应
        SearchHits searchHits = response.getHits() ;
        //获取总条数
        long total = searchHits.getTotalHits().value ;
        System.out.println("一共搜索到了" + total + "条数据！");
        //获取文档数组
        SearchHit[] searchHits1 = searchHits.getHits() ;
        for(SearchHit searchHits2 : searchHits1){
            String json = searchHits2.getSourceAsString() ;
            //反序列化
            HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class) ;
            System.out.println(hotelDoc);
        }
    }

下面演示bool查询，使用了must和filter，同时其中包含term精准查询和range查询，具体如下：

    @Test
    void testAll() throws IOException {
        //1.创建request对象
        SearchRequest searchRequest = new SearchRequest("hotel") ;
        //2.准备DSL
        BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery() ;
        //添加term
        boolQueryBuilder.must(QueryBuilders.termQuery("city.keyword","上海")) ;
        //添加range
        boolQueryBuilder.filter(QueryBuilders.rangeQuery("price").lte(500)) ;
        searchRequest.source().query(boolQueryBuilder) ;
        //3.发送请求
        SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT) ;
        handleResponse(response);
    }

3.3、RestClient搜索结果处理-排序和分页

定义页码和页面大小，每次将查询到的对象按照price升序排序，然后进行分页，每5个一页，具体如下所示：

    @Test
    void testPageAndSort() throws IOException {
        int page = 1 , pageSize = 5 ;

        //1.创建request对象
        SearchRequest searchRequest = new SearchRequest("hotel") ;
        //2.准备DSL
        searchRequest.source().query(QueryBuilders.matchAllQuery()) ;
        //排序
        searchRequest.source().sort("price", SortOrder.ASC) ;
        //分页
        searchRequest.source().from((page-1)*pageSize).size(pageSize) ;
        //3.发送请求
        SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT) ;
        //4.解析响应
        handleResponse(response);
    }

3.4、RestClient搜索结果处理-高亮显示

将查询的结果进行高亮显示，查询到name为如家的，进行高亮显示，对处理的结果进行覆盖，然后就可以形成高亮显示了。

 @Test
    void testHighlight() throws IOException {

        //1.创建request对象
        SearchRequest searchRequest = new SearchRequest("hotel") ;
        //2.准备DSL
        searchRequest.source().query(QueryBuilders.matchQuery("name","如家")) ;
        //高亮显示
        searchRequest.source().highlighter(new HighlightBuilder().field("name").requireFieldMatch(false)) ;
        //3.发送请求
        SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT) ;
        //4.解析响应
        handleResponse1(response);
    }

    private void handleResponse1(SearchResponse response) {
        SearchHits searchHits = response.getHits() ;
        //获取总条数
        long total = searchHits.getTotalHits().value ;
        System.out.println("一共搜索到了" + total + "条数据！");
        //获取文档数组
        SearchHit[] searchHits1 = searchHits.getHits() ;
        for(SearchHit searchHits2 : searchHits1){
            String json = searchHits2.getSourceAsString() ;
            //反序列化
            HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class) ;
            //获取高亮的结果
            Map<String, HighlightField> highlightFields = searchHits2.getHighlightFields();
            if(!CollectionUtils.isEmpty(highlightFields)){
                HighlightField highlightField = highlightFields.get("name") ;
                if(highlightField != null){
                    String name = highlightField.getFragments()[0].string() ;
                    hotelDoc.setName(name);
                }
            }
            System.out.println(hotelDoc);
        }
    }

四、旅游案例

4.1、搜索与分页

实现旅游网站的酒店搜索功能，完成关键字搜索和分页，具体包括下面的三个步骤：

定义实体类如下：


import lombok.Data;

@Data
public class RequestParams {
    private String key ;
    private Integer page ;
    private Integer size ;
    private String sortBy ;
    
}


import java.util.List;

@Data
public class PageResult {
    private Long total ;
    private List<HotelDoc> hotels ;

    public PageResult() {
    }

    public PageResult(Long total, List<HotelDoc> hotels) {
        this.total = total;
        this.hotels = hotels;
    }
}

定义表现层controller，接收前端页面请求，调用业务层方法实现查找和分页。

import cn.itcast.hotel.pojo.PageResult;
import cn.itcast.hotel.pojo.RequestParams;
import cn.itcast.hotel.service.IHotelService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/hotel")
public class HotelController {

    @Autowired
    private IHotelService iHotelService ;

    @PostMapping("/list")
    public PageResult search(@RequestBody RequestParams params){
        return iHotelService.search(params) ;
    }

}

在业务层定义接口和实现类，如下，在实现类中完成查询和分页业务，将解析得到的结果进行封装后返回。其中注入的client在启动类中注解为bean，变成配置类交给spring管理。


import cn.itcast.hotel.mapper.HotelMapper;
import cn.itcast.hotel.pojo.Hotel;
import cn.itcast.hotel.pojo.HotelDoc;
import cn.itcast.hotel.pojo.PageResult;
import cn.itcast.hotel.pojo.RequestParams;
import cn.itcast.hotel.service.IHotelService;
import com.alibaba.fastjson.JSON;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.sort.SortOrder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

@Service
public class HotelService extends ServiceImpl<HotelMapper, Hotel> implements IHotelService {

    @Autowired
    private RestHighLevelClient client ;

    @Override
    public PageResult search(RequestParams params) {

        try {
            //1.创建request对象
            SearchRequest searchRequest = new SearchRequest("hotel");
            //2.准备DSL
            String key = params.getKey();
            if (key == null || "".equals(key)) {
                searchRequest.source().query(QueryBuilders.matchAllQuery());
            } else {
                searchRequest.source().query(QueryBuilders.matchQuery("name", key));
            }
            //分页
            int page = params.getPage();
            int pageSize = params.getSize();
            searchRequest.source().from((page - 1) * pageSize).size(pageSize);
            //3.发送请求
            SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
            //4.解析响应
            return handleResponse(response);
        }catch (IOException e){
            throw new RuntimeException(e) ;
        }

    }

    private PageResult handleResponse(SearchResponse response) {
        //4.解析响应
        SearchHits searchHits = response.getHits() ;
        //获取总条数
        long total = searchHits.getTotalHits().value ;
        //获取文档数组
        SearchHit[] searchHits1 = searchHits.getHits() ;
        //遍历
        List<HotelDoc> list = new ArrayList<>() ;
        for(SearchHit searchHits2 : searchHits1){
            String json = searchHits2.getSourceAsString() ;
            //反序列化
            HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class) ;
            list.add(hotelDoc) ;
        }
        return new PageResult(total, list) ;
    }
}

4.2、条件过滤

添加品牌、价格、星级、城市等条件过滤功能，具体实现步骤如下：

修改实体类，添加一些参数，如下：


import lombok.Data;

@Data
public class RequestParams {
    private String key ;
    private Integer page ;
    private Integer size ;
    private String sortBy ;
    private String city ;
    private String brand ;
    private String starName ;
    private Integer minPrice ;
    private Integer maxPrice ;

}

然后在业务层的实现方法中进行条件过滤即可，具体如下：


import cn.itcast.hotel.mapper.HotelMapper;
import cn.itcast.hotel.pojo.Hotel;
import cn.itcast.hotel.pojo.HotelDoc;
import cn.itcast.hotel.pojo.PageResult;
import cn.itcast.hotel.pojo.RequestParams;
import cn.itcast.hotel.service.IHotelService;
import com.alibaba.fastjson.JSON;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.sort.SortOrder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

@Service
public class HotelService extends ServiceImpl<HotelMapper, Hotel> implements IHotelService {

    @Autowired
    private RestHighLevelClient client ;

    @Override
    public PageResult search(RequestParams params) {

        try {
            //1.创建request对象
            SearchRequest searchRequest = new SearchRequest("hotel");
            buildBasicQuery(params, searchRequest);
            //分页
            int page = params.getPage();
            int pageSize = params.getSize();
            searchRequest.source().from((page - 1) * pageSize).size(pageSize);
            //3.发送请求
            SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
            //4.解析响应
            return handleResponse(response);
        }catch (IOException e){
            throw new RuntimeException(e) ;
        }

    }

    private void buildBasicQuery(RequestParams params, SearchRequest searchRequest) {
        //2.关键字搜索
        BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery() ;
        String key = params.getKey();
        if (key == null || "".equals(key)) {
            boolQueryBuilder.must(QueryBuilders.matchAllQuery());
        } else {
            boolQueryBuilder.must(QueryBuilders.matchQuery("name", key));
        }
        //城市条件搜索
        if(params.getCity() != null && !"".equals(params.getCity())){
            boolQueryBuilder.filter(QueryBuilders.termQuery("city.keyword", params.getCity())) ;
        }
        //品牌条件搜索
        if(params.getBrand() != null && !"".equals(params.getBrand())){
            boolQueryBuilder.filter(QueryBuilders.termQuery("brand.keyword", params.getBrand())) ;
        }
        //星级条件搜索
        if(params.getStarName() != null && !"".equals(params.getStarName())){
            boolQueryBuilder.filter(QueryBuilders.termQuery("starName.keyword", params.getStarName())) ;
        }
        //价格条件搜索
        if(params.getMinPrice() != null && params.getMaxPrice() != null){
            boolQueryBuilder.filter(QueryBuilders.rangeQuery("price").
                    gte(params.getMinPrice()).lte(params.getMaxPrice())) ;
        }

        searchRequest.source().query(boolQueryBuilder) ;

    }

    private PageResult handleResponse(SearchResponse response) {
        //4.解析响应
        SearchHits searchHits = response.getHits() ;
        //获取总条数
        long total = searchHits.getTotalHits().value ;
        //获取文档数组
        SearchHit[] searchHits1 = searchHits.getHits() ;
        //遍历
        List<HotelDoc> list = new ArrayList<>() ;
        for(SearchHit searchHits2 : searchHits1){
            String json = searchHits2.getSourceAsString() ;
            //反序列化
            HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class) ;
            list.add(hotelDoc) ;
        }
        return new PageResult(total, list) ;
    }
}

4.3、附近的酒店距离升序排序

获取前端的位置数据，根据计算得到的位置升序排序，找出最近的酒店位置，如下：

import cn.itcast.hotel.mapper.HotelMapper;
import cn.itcast.hotel.pojo.Hotel;
import cn.itcast.hotel.pojo.HotelDoc;
import cn.itcast.hotel.pojo.PageResult;
import cn.itcast.hotel.pojo.RequestParams;
import cn.itcast.hotel.service.IHotelService;
import com.alibaba.fastjson.JSON;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.common.geo.GeoPoint;
import org.elasticsearch.common.unit.DistanceUnit;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.sort.SortBuilder;
import org.elasticsearch.search.sort.SortBuilders;
import org.elasticsearch.search.sort.SortOrder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

@Service
public class HotelService extends ServiceImpl<HotelMapper, Hotel> implements IHotelService {

    @Autowired
    private RestHighLevelClient client ;

    @Override
    public PageResult search(RequestParams params) {

        try {
            //1.创建request对象
            SearchRequest searchRequest = new SearchRequest("hotel");
            buildBasicQuery(params, searchRequest);
            //分页
            int page = params.getPage();
            int pageSize = params.getSize();
            searchRequest.source().from((page - 1) * pageSize).size(pageSize);
            //排序
            String location = params.getLocation() ;
            if(location != null && !"".equals(location)){
                searchRequest.source().sort(SortBuilders.
                        geoDistanceSort("location", new GeoPoint(location)).
                        order(SortOrder.ASC).unit(DistanceUnit.KILOMETERS)) ;
            }
            //3.发送请求
            SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
            //4.解析响应
            return handleResponse(response);
        }catch (IOException e){
            throw new RuntimeException(e) ;
        }

    }

    private void buildBasicQuery(RequestParams params, SearchRequest searchRequest) {
        //2.关键字搜索
        BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery() ;
        String key = params.getKey();
        if (key == null || "".equals(key)) {
            boolQueryBuilder.must(QueryBuilders.matchAllQuery());
        } else {
            boolQueryBuilder.must(QueryBuilders.matchQuery("name", key));
        }
        //城市条件搜索
        if(params.getCity() != null && !"".equals(params.getCity())){
            boolQueryBuilder.filter(QueryBuilders.termQuery("city.keyword", params.getCity())) ;
        }
        //品牌条件搜索
        if(params.getBrand() != null && !"".equals(params.getBrand())){
            boolQueryBuilder.filter(QueryBuilders.termQuery("brand.keyword", params.getBrand())) ;
        }
        //星级条件搜索
        if(params.getStarName() != null && !"".equals(params.getStarName())){
            boolQueryBuilder.filter(QueryBuilders.termQuery("starName.keyword", params.getStarName())) ;
        }
        //价格条件搜索
        if(params.getMinPrice() != null && params.getMaxPrice() != null){
            boolQueryBuilder.filter(QueryBuilders.rangeQuery("price").
                    gte(params.getMinPrice()).lte(params.getMaxPrice())) ;
        }

        searchRequest.source().query(boolQueryBuilder) ;

    }

    private PageResult handleResponse(SearchResponse response) {
        //4.解析响应
        SearchHits searchHits = response.getHits() ;
        //获取总条数
        long total = searchHits.getTotalHits().value ;
        //获取文档数组
        SearchHit[] searchHits1 = searchHits.getHits() ;
        //遍历
        List<HotelDoc> list = new ArrayList<>() ;
        for(SearchHit searchHits2 : searchHits1){
            String json = searchHits2.getSourceAsString() ;
            //反序列化
            HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class) ;
            //获取排序值
            Object [] objects = searchHits2.getSortValues() ;
            if(objects.length > 0){
                Object value = objects[0] ;
                hotelDoc.setDistance(value);
            }
            list.add(hotelDoc) ;
        }
        return new PageResult(total, list) ;
    }
}