ES 学习笔记

致敬kuangshen

ES安装

Linux集群安装

1.解压安装包

#以7.13.2为例tar zxvf elasticsearch-7.13.2-linux-x86_64.tar.gz

2.配置es用户

useradd espasswd es

3.更改用户和用户组

es默认不以root用户启动

# 安装节点chown -R es:es /opt/moudle/（es-cluster/）

8.修改elasticsearch.yml文件

cluster.name: elasticsearch# 配置的集群名称，默认是elasticsearch，es服务会通过广播方式自动连接在同一网段下的es服务，通过多播方式进行通信，同一网段下可以有多个集群，通过集群名称这个属性来区分不同的集群node.name: node-1# 当前配置所在机器的节点名node.master: true#指定该节点是否有资格被选举成为node（注意这里只是设置成有资格),默认truenode.data: true# 指定该节点是否存储索引数据，默认为true。network.host: hadoop1http.cors.allow-origin: "*"http.cors.enabled: truehttp.port: 9200# 设置对外服务的http端口，默认为9200。transport.tcp.port: 9300# 设置节点之间交互的tcp端口，默认是9300。press: true# 设置是否压缩tcp传输时的数据，默认为false，不压缩。http.max_content_length: 100mb# 设置内容的最大容量，默认100mb#gateway.type: HDFS# gateway的类型，默认为local即为本地文件系统，可以设置为本地文件系统，分布式文件系统，hadoop的HDFS，和amazon的s3服务器等。discovery.seed_hosts: ["hadoop1:9300","hadoop2:9300","hadoop3:9300"]# 在启动此节点时传递要执行发现的主机的初始列表cluster.initial_master_nodes: ["node-1"]# 使用初始的一组符合主节点条件的节点引导集群

5.修改其他文件

5.1 增加以下配置 /etc/security/limits.conf

es soft nofile 65536es hard nofile 65536

5.2 增加以下配置 /etc/security/limits.d/20-nproc.conf

es soft nofile 65536es hard nofile 65536

5.3 增加以下配置 /etc/sysctl.conf 加入

vm.max_map_count=655360

8.分发文件到各个节点。

8.切换es用户、启动

su es

切换到es安装目录，启动 nohup bin/elasticsearch &

ES Header 安装

/mobz/elasticsearch-head

1.windows安装nodejs

2.下载上面的zip文件

3.切换到es-head安装目录npm install

4.elasticsearch.yml 里面加入

用notepad++编辑的话，需要将编码格式改成UTF-8-BOM

http.cors.enabled: truehttp.cors.allow-origin: "*"

5.启动命令为：npm run start

ES分词器安装

地址：/medcl/elasticsearch-analysis-ik/releases

1.下载zip包

2.plugin目录下创建ik文件夹

3.ik目录下解压zip包

8.重启es、kibana

5.ik分词器的使用

GET _analyze{"analyzer": "ik_smart","text": ["我爱我的祖国，我爱我的国家"]}GET _analyze{"analyzer": "ik_max_word","text": ["我爱我的祖国，我爱我的国家"]}##下面为默认的自带分词器GET _analyze{"analyzer": "standard","text": ["我爱我的祖国，我爱我的国家"]}#keyword不会被分开GET _analyze{"analyzer": "keyword","text": ["我爱我的祖国，我爱我的国家"]}

ES使用

ES原理以及定义

1.索引（Index）

ES将数据存储于一个或多个索引中。类比传统的关系型数据库领域来说，索引相当于SQL中的一个数据库，或者一个数据存储方案(schema)。索引由其名称(必须为全小写字符)进行标识。一个ES集群中可以按需创建任意数目的索引。

建立索引

1.1方式1

POST /people/humanbeing?pretty=true

1.2方式2

PUT /shoes/product/1{"name" : "NB 鞋子","desc" : "特别好的鞋子","price" : 530,"producer" : "NB producer","tags": [ "实用", "美观" ]}

问题如下：

{"error" : {"root_cause" : [{"type" : "illegal_argument_exception","reason" : "Rejecting mapping update to [shoes] as the final mapping would have more than 1 type: [product, design]"}],"type" : "illegal_argument_exception","reason" : "Rejecting mapping update to [shoes] as the final mapping would have more than 1 type: [product, design]"},"status" : 800}

一个索引只能有一个类型，就是一个index只能一个type。

2.类型（Type）

类型是索引内部的逻辑分区(category/partition)，一个索引内部可定义一个或多个类型(type)。类比传统的关系型数据库领域来说，类型相当于“表”。

3.文档（Document）

文档是索引和搜索的原子单位，它是包含了一个或多个域（Field）的容器，每个域拥有一个名字及一个或多个值，有多个值的域通常称为“多值域”，文档基于JSON格式进行表示。每个文档可以存储不同的域集，但同一类型下的文档至应该有某种程度上的相似之处。

实践练习一

1.查询其中的一条数据

GET /shoes/product/1

2.修改部分信息

POST /shoes/product/1/_update{"doc":{"name":"NB 新款鞋子"}}

3.查询所有

GET /shoes/product/_search

4.创建索引的另外一种方式

curl -X POST "182.20.188.158:9200/person/male?pretty" -H 'Content-Type: application/json' -d'{ "name": {"type": "text"},"country": {"type": "keyword"},"age": {"type": "integer"},"date": {"type": "date"} }'

5.创建索引设置映射

curl -H 'Content-Type: application/json' -XPUT "http://182.20.188.155:9200/book/?pretty" -d '{"settings": {"refresh_interval": "20s","number_of_shards": 1,"number_of_replicas": 0},"mappings": {"properties": {"title": {"type": "text"},"author": {"type": "keyword"},"word_count": {"type": "integer"},"publish_date": {"type": "date", "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"}"type": {"type": "text"}}}}'

6.删除数据

DELETE /shoes/product/1

7.查询所有数据

GET /shoes/product/_search

或者

GET /shoes/product/_search{"query": {"match_all": {} }}

8.包含条件查询

8.1 按照价格排列

GET /shoes/product/_search{"query" : {"match" : {"name" : "NB"}},"sort": [ {"price": "desc" }]}

8.2 查询结果只包含name和price

GET /shoes/product/_search{"query": {"match_all": {}},"_source":["name","price"]}

Bool查询现在包括四种子句，must，filter,should,must_not。

8.3 查询name中含有nb的商品

GET /shoes/product/_search{"query":{"bool":{"must":{"match":{"name": "NB"}}}}}

8.4 查询name中含有nb的商品，同时价格高于800

GET /shoes/product/_search{"query":{"bool":{"must":{"match":{"name": "NB"}},"filter":{"range": {"price": {"gt": 800 }}}}}}

注意：bool查询中包含的有must,must_not,filter,should

8.5 包含词语查询

GET /shoes/product/_search{"query":{"match": {"producer": "NB producer"} }}

以上查询，会将NB和producer分开，然后只要producer中包含这两个词语中的一个就会显示出来。上面显示包含nb或者producer的5条信息。

严格查询，不分词，只显示包含的唯一一条

GET /shoes/product/_search{"query":{"match_phrase": {"producer": "NB producer"}}}

9.分页查询

//from 就是页码 size是每页数量GET /shoes/product/_search{"query": {"match_all": {} },"from": 0,"size": 1}

实践练习二

1.新建索引

PUT /test_es/

2.查看索引列表

GET /_cat/indices?v&pretty

3.修改索引

更改索引名称

POST _reindex{"source": {"index": "cmcc"},"dest": {"index": "cmccc"}}

4.删除索引

DELETE /cmccc

5.插入数据

PUT /cmcc/_doc/1{"name":"张三丰","age":20,"location":"南京市","province":"江苏省","country":"中国"}

PUT /test_es/_doc/1{"name":"zhangsan"}

6.查询单条数据

GET cmcc/_doc/1

7.插入多条数据

注意只能以一条数据一行的形式，不能包含换行

POST _bulk{"index":{"_index":"cmcc"}}{"name":"法外狂徒张三","age":20,"location":"南京市","province":"江苏省","country":"中国"}{"index":{"_index":"cmcc"}}{"name":"王五","age":21,"location":"长沙市","province":"湖南省","country":"中国"}{"index":{"_index":"cmcc"}}{"name":"王力宏","age":22,"location":"台北市","province":"台湾省","country":"中国"}{"index":{"_index":"cmcc"}}{"name":"杨过","age":23,"location":"海口市","province":"海南省","country":"中国"}{"index":{"_index":"cmcc"}}{"name":"辛弃疾","age":28,"location":"无锡市","province":"江苏省","country":"中国"}{"index":{"_index":"cmcc"}}{"name":"范仲淹","age":32,"location":"广州市","province":"广东省","country":"中国"}{"index":{"_index":"cmcc"}}{"name":"刘德华","age":29,"location":"新界","province":"香港","country":"中国"}

8.查询

8.1 简单查询一

GET cmcc/_search?q=name:"张三"

8.2 简单查询二

与上条的表达意思相同

GET cmcc/_search{"query": {"match": {"name": "张三"} }}

8.3 限定展示内容

如果只想显示部分内容的话，可以在query之后加上source过滤

GET cmcc/_search{"query": {"match": {"name": "张三"}},"_source":["name","age","location"]}

8.4布尔查询

GET cmcc/_search{"query":{"bool":{"must":[{"match":{"location":"广州市"}}],"must_not":[{"match":{"age":30}}]}}}

8.5精确查询

GET cmcc/_search{"query":{"bool":{"must":[{"match":{"location":"广州市"}},{"match":{"age":30}}]}}}

8.6精确查询+排序

GET cmcc/_search{"query":{"range":{{"age":{"gte":20,"lte":35}}}}，"sort":[{"age":{"order":"desc"}}]}

8.7复合查询

GET cmcc/_search{"size":0,"aggs":{"age":{"range":{"field":"age","ranges":[{"from":20,"to":23},{"from":23,"to":25},{"from":25,"to":30}]}}}}

8.8关键字查询

这条请求是查询属性"province"的统计结果，这里是统计5条数据，并显示其中2条，并在198行"field"后输入属性名，并在其后添加 .keyword，查询结果如下

GET cmcc/_search{"size": 2,"aggs": {"province": {"terms": {"field": "location.keyword","size": 5}}}}

8.9 排序

GET cmcc/_search{"query": {"match": {"name": "张三"}},"sort": [{"age": {"order": "desc"}}]}

8.10or查询

GET cmcc/_search{"query": {"bool": {"should": [{"match":{"name":"法外狂徒"}},{"match": {"age": 20}}]}}}

8.11not查询

GET cmcc/_search{"query": {"bool": {"must_not": [{"match": {"age": 20}}]}}}

8.12filter

GET cmcc/_search{"query": {"bool": {"must": [{"match":{"name":"张三"}}],"filter":{"range": {"age": {"lt": 10}}} }}}

8.13term

GET /test_db/_doc/_search{"query":{"term":{"desc": "chengchen要努力"}}}

8.14term2

GET test_db/_search{"query":{"bool":{"should":[{"term":{"t1":"22"}},{"term":{"t1":"33"}}]}}}

8.15高亮操作

GET cmcc/_search{"query": {"match": {"name": "张三"}},"highlight": {"fields": {"name": {}}}}

8.16高亮操作二-自定义

GET cmcc/_search{"query": {"match": {"name": "张三"}},"highlight": {"pre_tags": "<p class='key', style='color:red'>","post_tags": "</p>", "fields": {"name": {}}}}

9.更新数据

POST test_es/_doc/1{"doc":{"name":"张三丰"}}

ElasticSearch–Springboot

1.创建项目

2.修改es版本–pom.xml

<properties><java.version>11</java.version><!-- 下面的es的版本与机器安装的版本一致--> <elasticsearch.version>7.14.1</elasticsearch.version></properties>

3.java代码

@Testvoid createIndex() throws IOException {CreateIndexRequest createIndexRequest = new CreateIndexRequest("test_es");CreateIndexResponse createIndexResponse = restHighLevelClient.indices().create(createIndexRequest, RequestOptions.DEFAULT);System.out.println(createIndexResponse);}@Testvoid getIndex() throws IOException {GetIndexRequest getIndexRequest = new GetIndexRequest("test_es");boolean exists = restHighLevelClient.indices().exists(getIndexRequest, RequestOptions.DEFAULT);System.out.println(exists);}@Testvoid deleteIndex() throws IOException {DeleteIndexRequest deleteIndexRequest = new DeleteIndexRequest("test_es");AcknowledgedResponse delete = restHighLevelClient.indices().delete(deleteIndexRequest, RequestOptions.DEFAULT);boolean acknowledged = delete.isAcknowledged();System.out.println(acknowledged);}@Testvoid createDocument() throws IOException {User user = new User("张三", 2);IndexRequest indexRequest = new IndexRequest("test_es");indexRequest.id("1");// indexRequest.timeout("1s");IndexRequest request = indexRequest.source(JSON.toJSONString(user), XContentType.JSON);IndexResponse indexResponse = restHighLevelClient.index(request, RequestOptions.DEFAULT);String string = indexResponse.toString();RestStatus status = indexResponse.status();System.out.println(string);System.out.println(status);}@Testvoid existDocument() throws IOException {//GetIndexRequest indexRequest = new GetIndexRequest("test_es");//创建请求？？？GetRequest test_es = new GetRequest("test_es","1");boolean exists = restHighLevelClient.exists(test_es, RequestOptions.DEFAULT);System.out.println(exists);}@Testvoid getDocument() throws IOException {GetRequest getRequest = new GetRequest("test_es","1");GetResponse getResponse = restHighLevelClient.get(getRequest, RequestOptions.DEFAULT);String sourceAsString = getResponse.getSourceAsString();System.out.println(sourceAsString);System.out.println(getResponse);}@Testvoid updateDocument() throws IOException {UpdateRequest updateRequest = new UpdateRequest("test_es","1");User user= new User("张三丰",18);UpdateRequest doc = updateRequest.doc(JSON.toJSONString(user),XContentType.JSON);UpdateResponse updateResponse = restHighLevelClient.update(doc, RequestOptions.DEFAULT);RestStatus status = updateResponse.status();System.out.println(status);}@Testvoid deleteDocument() throws IOException {DeleteRequest deleteRequest = new DeleteRequest("test_es","1");DeleteResponse delete = restHighLevelClient.delete(deleteRequest, RequestOptions.DEFAULT);RestStatus status = delete.status();System.out.println(status);}//批量插入@Testvoid bulkCreateDocument() throws IOException {BulkRequest bulkRequest = new BulkRequest();List<User> users = new ArrayList<>();for (int i = 0; i < 10; i++) {users.add(new User("zhangsan"+i,i+1));}for (int i = 0; i < users.size(); i++) {bulkRequest.add(new IndexRequest("test_es").id(""+i).source(JSON.toJSONString(users.get(i)),XContentType.JSON));}BulkResponse responses = restHighLevelClient.bulk(bulkRequest, RequestOptions.DEFAULT);boolean flag = responses.hasFailures();System.out.println(flag);}//查询@Testvoid searchEs() throws IOException {SearchRequest searchRequest = new SearchRequest("test_es");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("name", "zhangsan1");//将需要查询的内容加入构建器当中searchSourceBuilder.query(termQueryBuilder);searchRequest.source(searchSourceBuilder);SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);SearchHits hits = searchResponse.getHits();String string = JSON.toJSONString(hits);System.out.println(string);SearchHit[] searchHits = hits.getHits();for (SearchHit searchHit:searchHits) {Map<String, Object> sourceAsMap = searchHit.getSourceAsMap();System.out.println(sourceAsMap);}}