网站导航
> 文档中心 > go语言实战es，高亮全文检索关键词等！

go语言实战es，高亮全文检索关键词等！

网友: 文档中心 2022-03-16 13:54:42

Golang实战ES

  • 一、ES的安装
    • 下载elasticSearch7.7.0
    • docker-compose启动elasticSearch7.7.0
    • 安装中文分词器IK
    • 注意事项
  • 二、ES的简单的应用
    • 查询
      • 简单查询
      • 复合条件查询
  • 三、ES的在go中实战项目运用
    • 安装
    • 连接
    • 查询索引
    • 添加
    • 查询
    • 更新
    • 删除
    • 返回搜索词高亮处理

一、ES的安装

下载elasticSearch7.7.0

docker pull elasticsearch:7.7.0 //下拉镜像docker images//查看镜像//创建所需文件mkdir -p /mydata/elasticsearch/configmkdir -p /mydata/elasticsearch/dataecho "http.host: 0.0.0.0">>/mydata/elasticsearch/config/elasticsearch.yml//文件夹赋权chmod -R 777 elasticsearch///命令启动docker run --name elasticsearch -p 9200:9200 \ -p 9300:9300 \ -e "discovery.type=single-node" \ -e ES_JAVA_OPTS="-Xms64m -Xmx128m" \  -v /mydata/elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml \ -v /mydata/elasticsearch/data:/usr/share/elasticsearch/data \ -v /mydata/elasticsearch/plugins:/usr/share/elasticsearch/plugins \ -d elasticsearch:7.7.0  //参数说明 --name elasticsearch：将容器命名为 elasticsearch-p 9200:9200：将容器的9200端口映射到宿主机9200端口-p 9300:9300：将容器的9300端口映射到宿主机9300端口，目的是集群互相通信-e "discovery.type=single-node"：单例模式-e ES_JAVA_OPTS="-Xms64m -Xmx128m"：配置内存大小-v /mydata/elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml：将配置文件挂载到宿主机-v /mydata/elasticsearch/data:/usr/share/elasticsearch/data：将数据文件夹挂载到宿主机-v /mydata/elasticsearch/plugins:/usr/share/elasticsearch/plugins：将插件目录挂载到宿主机(需重启)-d elasticsearch:7.7.0：后台运行容器，并返回容器ID

docker-compose启动elasticSearch7.7.0

//docker-compose启动version: '2'services:  elasticsearch:    container_name: elasticsearch    image: elasticsearch:7.7.0    ports:      - "9200:9200"    volumes:      - /mydata/elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml      - /mydata/elasticsearch/data:/usr/share/elasticsearch/data      - /mydata/elasticsearch/plugins:/usr/share/elasticsearch/plugins    environment:      - "ES_JAVA_OPTS=-Xms64m -Xmx128m"      - "discovery.type=single-node"      - "COMPOSE_PROJECT_NAME=elasticsearch-server"    restart: always//查看容器docker ps -a//验证是否成功curl http://192.168.0.50:9200///返回信息{  "name" : "550eca5cf3b2",  "cluster_name" : "elasticsearch",  "cluster_uuid" : "1mP9IJU3TUy_C2DZMfKGBg",  "version" : {    "number" : "7.7.0",    "build_flavor" : "default",    "build_type" : "docker",    "build_hash" : "81a1e9eda8e6183f5237786246f6dced26a10eaf",    "build_date" : "2020-05-12T02:01:37.602180Z",    "build_snapshot" : false,    "lucene_version" : "8.5.1",    "minimum_wire_compatibility_version" : "6.8.0",    "minimum_index_compatibility_version" : "6.0.0-beta1"  },  "tagline" : "You Know, for Search"}//Docker启动容器自启docker update elasticsearch --restart=always

安装中文分词器IK

1.在线安装//进入容器docker exec -it elasticsearch /bin/bash//在线下载并安装./bin/elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.5.4/elasticsearch-analysis-ik-6.5.4.zip//进入plugins可以看到IK分词器已经安装成功目录中有analysis-ik安装成功2.离线安装//将IK分词器上传到/tmp目录中elasticsearch-analysis-ik-6.5.4.zip//将压缩包移动到容器里面docker cp /tmp/elasticsearch-analysis-ik-6.5.4.zip elasticsearch:/usr/share/elasticsearch/plugins//进入容器docker exec -it elasticsearch /bin/bash//创建目录mkdir /usr/share/elasticsearch/plugins/ik//将文件压缩包移动到ik中mv /usr/share/elasticsearch/plugins/elasticsearch-analysis-ik-6.5.4.zip /usr/share/elasticsearch/plugins/ik//进入目录cd /usr/share/elasticsearch/plugins/ik//解压unzip elasticsearch-analysis-ik-6.5.4.zip//删除压缩包rm -rf elasticsearch-analysis-ik-6.5.4.zip//退出并重启镜像exitdocker restart elasticsearch

注意事项

* 注意 安装中文分词器版本一定要和ES的版本号对应，不然会报错。

报错信息如下：Exception in thread "main" java.lang.IllegalArgumentException: Plugin [analysis-ik] was built for Elasticsearch version 7.0.0 but version 7.7.0 is runningat org.elasticsearch.plugins.PluginsService.verifyCompatibility(PluginsService.java:346)at org.elasticsearch.plugins.InstallPluginCommand.loadPluginInfo(InstallPluginCommand.java:814)at org.elasticsearch.plugins.InstallPluginCommand.installPlugin(InstallPluginCommand.java:869)at org.elasticsearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:254)at org.elasticsearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:224)at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127)at org.elasticsearch.cli.MultiCommand.execute(MultiCommand.java:91)at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127)at org.elasticsearch.cli.Command.main(Command.java:90)at org.elasticsearch.plugins.PluginCli.main(PluginCli.java:47)

查看对应IK版本号:https://github.com/medcl/elasticsearch-analysis-ik/releases

安装完成重启ESdocker restart elasticsearchIK分词使用请求地址POST：http://127.0.0.1/_analyze?pretty{"analyzer": "ik_smart","text": "中华人民共和国国歌"}//返回内容{"tokens": [    { "token": "中华人民共和国", "start_offset": 0, "end_offset": 7, "type": "CN_WORD", "position": 0    },    { "token": "国歌", "start_offset": 7, "end_offset": 9, "type": "CN_WORD", "position": 1    }]}ik_max_word：最细粒度的拆分ik_smart： 最粗粒度的拆分

二、ES的简单的应用

查询

简单查询

Query Context 处理判断文档是否满足查询条件以外，ES还会计算一个_score 来标识匹配程度，判断目标文档与查询条件匹配的有多好。 常用查询 针对文本类型数据 字段级别的查询 针对结构化数据，如数字，日期。

Post请求 127.0.0.1/book/_search//模糊查询{    "query":{ "match":{     "author":"瓦力" }    }} //准确查询{    "query":{ "match_parase":{     "author":"瓦力" }    }} //多字段的匹配查询(查询autor和title包含瓦力的){    "query":{ "multi_match":{     "query":"瓦力",     "fields":["author", "title"] }    }} //查询瓦力和很好或则是python{    "query":{ "query_string":{     "query":"(瓦力 AND 很好）OR python", }    }} //查询瓦力和很好或则是python（在author和title字段中）{    "query":{ "query_string":{     "query":"(瓦力 AND 很好）OR python",      "fields":["author", "title"] }    }}//字段级别的查询，查询字数等于1000的图书{    "query":{ "term":{     "word_count": 1000 }    }}//字段级别的查询，查询字数在1000-2000的图书有哪些,//now表示现在时间，gte表示大于等于，gt表示大于{    "query":{ "range":{     "word_count":{  "gte":1000,  "lte":2000     } }    }}

Filter Context 子条件查询 在查询过程中，只判断该文档是否满足条件，只有Yes或No 进行数据过滤，会加到缓存中，比query要快

//查询字数在一千字的书籍{    "query": { "bool": {     "filter": {  "term": {      "word_count": 1000  }     } }    }}

复合条件查询

//POST 127.0.0.1/_search//全文查询，但是给出的评分不是一样的{    "query": { "match": {     "title": "ElasticSeach" }    }}//固定评分查询，结果评分全部是1，不支持match{    "query": { "constant_score": {     "filter": {  "match": {      "title": "ElasticSearch"  }     },     "boost": 2  //指定分数查询， }    }}//must必须查询条件{    "query": { bool": {     "should": [     {  "match": {      "author": "瓦力"  }     },     {  "match":{      "title": "ElasticSearch"  }     }     ],     "filter": [  {      "term":{   "word_count": 1000      }  }     ]     } }    }}//must_not.不满足条件，筛选处作者没有瓦力的条件{    "query": { "must_not": {     "term": {  "author": "瓦力"     } }    }}

三、ES的在go中实战项目运用

安装

1.根据版本安装依赖（注意go mod 安装和直接安装方法不一样，下面是go mod 安装）"github.com/olivere/elastic/v7"

连接

//连接客户端client, err := elastic.NewClient(elastic.SetURL(esUrl), elastic.SetSniff(false))if err != nil {// Handle errorfmt.Println("出错了")panic(err)}// Ping the Elasticsearch server to get e.g. the version number// ping通服务端，并获得服务端的es版本,本实例的es版本为version 7.6.1info, code, err := client.Ping(esUrl).Do(ctx)if err != nil {// Handle errorpanic(err)}fmt.Println("开始打印版本号")fmt.Printf("Elasticsearch returned with code>: %d and version %s\n", code, info.Version.Number)// 获取版本号的直接APIesVersion, err := client.ElasticsearchVersion(esUrl)if err != nil {panic(err)}fmt.Printf("es的版本为%s\n", esVersion)

查询索引

//连接客户端client, err := elastic.NewClient(elastic.SetURL(esUrl), elastic.SetSniff(false))if err != nil {// Handle errorfmt.Println("出错了")panic(err)}// Ping the Elasticsearch server to get e.g. the version number// ping通服务端，并获得服务端的es版本,本实例的es版本为version 7.6.1info, code, err := client.Ping(esUrl).Do(ctx)if err != nil {// Handle errorpanic(err)}fmt.Println("开始打印版本号")fmt.Printf("Elasticsearch returned with code>: %d and version %s\n", code, info.Version.Number)// 获取版本号的直接APIesVersion, err := client.ElasticsearchVersion(esUrl)if err != nil {panic(err)}fmt.Printf("es的版本为%s\n", esVersion)

添加

//连接客户端client, err := elastic.NewClient(elastic.SetURL(esUrl), elastic.SetSniff(false))if err != nil {// Handle errorfmt.Println("出错了")panic(err)}// Ping the Elasticsearch server to get e.g. the version number// ping通服务端，并获得服务端的es版本,本实例的es版本为version 7.6.1info, code, err := client.Ping(esUrl).Do(ctx)if err != nil {// Handle errorpanic(err)}fmt.Println("开始打印版本号")fmt.Printf("Elasticsearch returned with code>: %d and version %s\n", code, info.Version.Number)// 获取版本号的直接APIesVersion, err := client.ElasticsearchVersion(esUrl)if err != nil {panic(err)}fmt.Printf("es的版本为%s\n", esVersion)

查询

5.查询get1, err := client.Get().Index("user").Id("1").Do(ctx)if err != nil{panic(err)}if get1.Found{fmt.Printf("Got document %s in version %d from index %s, type %s\n", get1.Id, get1.Version, get1.Index, get1.Type)}//将保存的内容保存到磁盘_, err = client.Flush().Index("user").Do(ctx)if err != nil {panic(err)}// 按"term"搜索Search with a term querytermQuery := elastic.NewTermQuery("name", "mike")searchResult, err := client.Search().Index("user").   // 搜索的索引"user"Query(termQuery).   // specify the querySort("age", true). //按字段"age"排序，升序排列From(0).Size(10).   // 分页，单页显示10条Pretty(true).// pretty print request and response JSON以json的形式返回信息Do(ctx)      // 执行if err != nil {// Handle errorpanic(err)}fmt.Printf("Query took %d milliseconds\n", searchResult.TookInMillis)// 按"term"搜索Search with a term querytermQuery := elastic.NewTermQuery("name", "mike")searchResult, err := client.Search().Index("user").   // 搜索的索引"user"Query(termQuery).   // specify the querySort("age", true). //按字段"age"排序，升序排列From(0).Size(10).   // 分页，单页显示10条Pretty(true).// pretty print request and response JSON以json的形式返回信息Do(ctx)      // 执行if err != nil {// Handle errorpanic(err)}fmt.Printf("Query took %d milliseconds\n", searchResult.TookInMillis)var user User//Each是一个简便函数，此函数忽略了错误输出for _, item1 := range searchResult.Each(reflect.TypeOf(user)) {if u, ok := item1.(User); ok {fmt.Printf("Person by %s,age:%d,married:%t,Sex:%s\n", u.Name, u.Age, u.Married,u.Sex) //Person by bob,age:23,married:false,Sex:male}// 搜索文档方法2// 使用hits，获得更详细的输出结果if searchResult.Hits.TotalHits.Value >0{fmt.Printf("找到的数据总数是 %d \n", searchResult.Hits.TotalHits.Value)for _,hits := range searchResult.Hits.Hits{u :=User{}err := json.Unmarshal([]byte(hits.Source), &u)if err != nil{fmt.Println("反序列化失败",err)}fmt.Printf("User by %s,age:%d,married:%t,Sex:%s\n", u.Name, u.Age, u.Married,u.Sex)}}else {fmt.Println("没有搜到用户")}

更新

// 更新文档 updateupdate, err := client.Update().Index("user").Id("1").Script(elastic.NewScriptInline("ctx._source.age += params.num").Lang("painless").Param("num", 1)).//Upsert(map[string]interface{}{"created": "2020-06-17"}). // 插入未初始化的字段valueDo(ctx)if err != nil {// Handle errorpanic(err)}fmt.Printf("New version of user %q is now %d\n", update.Id, update.Version) // 更新方法2update,err := client.Update().Index("user").Id("1").Script(elastic.NewScriptInline("ctx._source.created=params.date").Lang("painless").Param("date","2020-06-17")).Do(ctx)termQuery := elastic.NewTermQuery("name", "bob")update,err = client.UpdateByQuery("user").Query(termQuery).Script(elastic.NewScriptInline("ctx._source.age += params.num").Lang("painless").Param("num", 1)).Do(ctx)if err != nil{panic(err)}fmt.Printf("New version of user %q is now %d\n", update.Id, update.Version)fmt.Println(update)

删除

//删除文档termQuery := elastic.NewTermQuery("name", "mike")_, err = client.DeleteByQuery().Index("user"). // search in index "user"Query(termQuery). // specify the queryDo(ctx)if err != nil {// Handle errorpanic(err)}

返回搜索词高亮处理

func QueryHeight(search dto.SearchActileDto,client *elastic.Client) (hightmap []bo.SearchActileBo,err error){var data []bo.SearchActileBotextFild:=elastic.NewHighlighterField("content")//idFild:=elastic.NewHighlighterField("id")//fild.PreTags("")//fild.PostTags("")titleFild:=elastic.NewHighlighterField("title")titleFild.FragmentSize(350)higthFild :=elastic.NewHighlight()higthFild.Fields(titleFild)higthFild.Fields(textFild)//higthFild.Fields(idFild)//两个里面必须有//query :=elastic.NewBoolQuery().Must(elastic.NewMatchPhraseQuery("title","css"),elastic.NewMatchPhraseQuery("text","css"))query :=elastic.NewBoolQuery().Should(elastic.NewMatchPhraseQuery("title",search.SearchWord),elastic.NewMatchPhraseQuery("content",search.SearchWord))res, err := client.Search("article").Query(query).Highlight(higthFild).Size(int(search.Size)).From(int((search.Current - 1) * search.Size)).Do(context.Background())if err != nil {return nil,err}//fmt.Println(res.Hits.Hits)for _, hit := range res.Hits.Hits {aa :=hit.Highlightvar search bo.SearchActileBosearch.ContentId = hit.Idvar qq dto.Articlejson.Unmarshal(hit.Source,&qq)search.CreatTime = fmt.Sprintf("%v",qq.CreateTime)search.CreateID = qq.CreateByif aa["title"] == nil{search.Title=qq.Title}else{search.Title=aa["title"][0]}if aa["content"] == nil{search.Content=qq.Content}else{search.Content = aa["content"][0]}data = append(data, search)}return data,nil}

欢迎私信我交流更多ES用法

网络标签：

相关问题