elasticsearch6.5.0(es6.5)加密xpack,java客户端访问xpack集群SSL
- xpack已经在es6.3之后默认集成，本文包含：
- 1、es集群使用xpack加密，达到访问集群需要用户密码登录的效果。
- 2、采用SSL加密通信，提供https访问方式
- 3、java客户端如何连接加密后的es集群(9200,9300端口)httpclient方式，restClient高低版本方式都有包含
- 4、使用kibana界面化工具连接操作加密后的es集群
开启xpack
-
elasticsearch.yml 中加上：
xpack.security.enabled: true注：
:后空一个空格，如果是es集群请务必全部添加此配置 -
确保集群健康访问
_xpack/license/start_trial?acknowledge=true开启xpack功能，这时候集群会立马提示您输入用户名密码的弹窗进行登录，别急还需要设置一下
注:(如果只需要加密功能，可以省略此步骤，以便永久免费使用，官方对于基本版不收费)此开启的是试用版，xpack的破解，请自行搜索，我文末也会提供一个6.5.0的破解包，替换一下就可以用了 -
设置es集群用户名密码
#自动生成(二选一) elasticsearch-setup-passwords auto #手动生成(推荐) elasticsearch-setup-passwords interactive此处最容易出现错误，请保证您的集群是健康的并且全部开启了xpack功能，且配置一样，如果生成失败，请检查自己的集群配置，如果设置密码失败的可以试试这个博文的顺序ELK6.6.0 Xpack
-
为Elasticearch集群创建一个证书颁发机构
PKCS#12格式生成(代码推荐)：elasticsearch-certutil ca PEM格式格式生成(kibana推荐)：elasticsearch-certutil ca --pem生成过程会提示输入密码，作用是访问证书的安全性，可以不设置
-
为es集群生成证书和秘钥
PKCS#12格式生成：elasticsearch-certutil cert --ca elastic-stack-ca.p12 PEM格式格式生成：elasticsearch-certutil cert --pem同上会提示输入密码，略过
pkcs12证书里面包含了秘钥所以只有一个文件，pem会有独立的证书和key
请把生成的证书elastic-certificates.p12拷贝到集群中的其他节点，否则加密失效 -
TLS/SSL加密Transport通信
xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.keystore.path: certselastic-certificates.p12 xpack.security.transport.ssl.truststore.path: certselastic-certificates.p12 -
加密https访问
xpack.security.http.ssl.enabled: true xpack.security.http.ssl.keystore.path: certselastic-certificates.p12 xpack.security.http.ssl.truststore.path: certselastic-certificates.p12
按照上面的步骤集群就可以实现加密访问，下面是使用java访问加密后的es集群
给出pom
repositories>
repository>
id>elasticsearch-releasesid>
url>https://artifacts.elastic.co/mavenurl>
releases>
enabled>trueenabled>
releases>
snapshots>
enabled>falseenabled>
snapshots>
repository>
repositories>
dependency>
groupId>org.elasticsearch.plugingroupId>
artifactId>transport-netty4-clientartifactId>
version>${es.version}version>
dependency>
dependency>
groupId>org.elasticsearchgroupId>
artifactId>elasticsearchartifactId>
version>${es.version}version>
dependency>
dependency>
groupId>org.elasticsearch.clientgroupId>
artifactId>transportartifactId>
version>${es.version}version>
dependency>
dependency>
groupId>org.elasticsearch.clientgroupId>
artifactId>x-pack-transportartifactId>
version>${es.version}version>
dependency>
dependency>
groupId>org.elasticsearch.clientgroupId>
artifactId>elasticsearch-rest-clientartifactId>
version>6.5.0version>
dependency>
dependency>
groupId>org.apache.httpcomponentsgroupId>
artifactId>httpclientartifactId>
version>4.5.2version>
dependency>
HttpClient方式
package com.**.**.configuration;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.client.HttpClient;
import org.apache.http.config.Registry;
import org.apache.http.config.RegistryBuilder;
import org.apache.http.conn.socket.ConnectionSocketFactory;
import org.apache.http.conn.socket.PlainConnectionSocketFactory;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
import org.apache.http.ssl.SSLContextBuilder;
import org.apache.http.ssl.SSLContexts;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSession;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.security.KeyStore;
/**
* Created by Majg on 2019-06-04
**/
@Configuration
public class ESHttpClientConfig {
@Bean(name = "EsHttpClient")
public HttpClient getHttpClient() throws Exception {
KeyStore truststore = KeyStore.getInstance("PKCS12");
// File file = ResourceUtils.getFile(ResourceUtils.CLASSPATH_URL_PREFIX + "certs/elastic-certificates.p12");
// //生产环境，打包方式不同，获取的是jar内的环境
// String absolutePath = file.getAbsolutePath();
String absolutePath = "C:\certs\elastic-certificates.p12";
try (InputStream is = Files.newInputStream(Paths.get(absolutePath))) {
truststore.load(is, "".toCharArray());
}
SSLContextBuilder sslBuilder = SSLContexts.custom().loadTrustMaterial(truststore, new TrustSelfSignedStrategy());
// 获取证书
SSLContext sslcontext = sslBuilder.build();
// 以上证书有效，但是证书签名是instance,下方有设置跳过签名验证
final CredentialsProvider credentialsProvider =
new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY,
new UsernamePasswordCredentials("elastic", "密码"));
//创建自定义的httpclient对象
CloseableHttpClient client = HttpClients.custom()
.setDefaultCredentialsProvider(credentialsProvider)
//跳过证书签名
.setSSLHostnameVerifier(new HostnameVerifier() {
public boolean verify(String hostname, SSLSession session) {
return true;
}
})
.setSSLContext(sslcontext)
.build();
return client;
}
// // 跳过证书签名
// private HostnameVerifier getTrustedVerifier() {
// if (TRUSTED_VERIFIER == null)
// TRUSTED_VERIFIER = new HostnameVerifier() {
//
// public boolean verify(String hostname, SSLSession session) {
// return true;
// }
// };
// return TRUSTED_VERIFIER;
// }
}
restClient方式(本人觉得最好用，也是官方推荐的)
import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.config.RequestConfig.Builder;
import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
import org.apache.http.ssl.SSLContexts;
import org.elasticsearch.client.Node;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestClientBuilder;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSession;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.security.KeyManagementException;
import java.security.KeyStore;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.security.cert.CertificateException;
/**
* Created by JGMa on 2019-06-03
**/
@Configuration
public class ESRestConfig {
private static Logger logger = LoggerFactory.getLogger(ESRestConfig.class);
private int connectTimeOut = 1000;
private int socketTimeOut = 30000;
private int maxConnectNum = 100;
private int maxConnectPerRoute = 100;
private int connectionRequestTimeOut = 5000;
@Value("${elasticsearch.cluster-nodes}")
private String clusterNodes;
@Value("${is_SSL_es}")
private Boolean is_SSL_es;
@Value("${elasticsearch.certPath:certs\elastic-certificates.p12}")
private String certPath;
@Bean("restHighLevelClient")
public RestClient RestClient() {
RestClient restClient = null;
String scheme = null;
if (is_SSL_es) {
scheme = "https";
} else {
scheme = "http";
}
// 配置hostName
String[] split = clusterNodes.split(",");
int length = split.length;
HttpHost[] https = new HttpHost[length];
for (int i = 0; i length; i++) {
String[] split1 = split[i].split(":");
https[i] = new HttpHost(split1[0], Integer.valueOf(split1[1]), scheme);
}
RestClientBuilder builder = RestClient.builder(https)
.setFailureListener(new RestClient.FailureListener() {
@Override
public void onFailure(Node node) {
HttpHost host = node.getHost();
logger.error("连接ES节点失败，host:{}", host);
}
});
//连接数配置
setMutiConnectConfig(builder);
//超时配置
setRequestTimeOutConfig(builder);
restClient = builder.build();
return restClient;
}
/**
* 异步httpclient的连接延时配置
*/
public void setRequestTimeOutConfig(RestClientBuilder builder) {
builder.setRequestConfigCallback(new RestClientBuilder.RequestConfigCallback() {
@Override
public Builder customizeRequestConfig(RequestConfig.Builder requestConfigBuilder) {
requestConfigBuilder.setConnectTimeout(connectTimeOut);
requestConfigBuilder.setSocketTimeout(socketTimeOut);
requestConfigBuilder.setConnectionRequestTimeout(connectionRequestTimeOut);
return requestConfigBuilder;
}
});
}
/**
* 异步httpclient的连接数配置
*/
public void setMutiConnectConfig(RestClientBuilder builder) {
builder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
@Override
public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
if (is_SSL_es) {
try {
KeyStore truststore = KeyStore.getInstance("PKCS12");
truststore.load(Files.newInputStream(Paths.get(certPath)), "".toCharArray());
SSLContext sslcontext = SSLContexts.custom().loadTrustMaterial(truststore, new TrustSelfSignedStrategy()).build();
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials("elastic", "123456"));
httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
httpClientBuilder.setSSLHostnameVerifier(new HostnameVerifier() {
public boolean verify(String hostname, SSLSession session) {
return true;
}
});
httpClientBuilder.setSSLContext(sslcontext);
httpClientBuilder.setMaxConnTotal(maxConnectNum);
httpClientBuilder.setMaxConnPerRoute(maxConnectPerRoute);
} catch (KeyStoreException | IOException | CertificateException | NoSuchAlgorithmException | KeyManagementException e) {
logger.error("配置ES加密集群连接数错误！-", e);
e.printStackTrace();
}
} else {
httpClientBuilder.setMaxConnTotal(maxConnectNum);
httpClientBuilder.setMaxConnPerRoute(maxConnectPerRoute);
}
return httpClientBuilder;
}
});
}
}
transport方式(一般代码写入会使用这个使用9300端口，但是官方不推荐并且会在8之后移除此)
import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.transport.TransportAddress;
import org.elasticsearch.xpack.client.PreBuiltXPackTransportClient;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.elasticsearch.core.ElasticsearchTemplate;
import java.io.FileNotFoundException;
import java.net.InetAddress;
import java.net.UnknownHostException;
/**
* Created by JGMa on 2019-06-03
**/
@Configuration
public class ESTransportConfig {
@Value("${elasticsearch.cluster_name:elasticsearch}")
private String clusterName;
@Value("${elasticsearch.transport_es_nodes:127.0.0.1:9300}")
private String clusterNodes;
@Value("${is_SSL_es}")
private Boolean is_ssl_es;
@Value("${elasticsearch.certPath:certs\elastic-certificates.p12}")
private String certPath;
@Bean(name = "sslTransportClient")
public TransportClient getTransportClient() throws FileNotFoundException {
try {
PreBuiltXPackTransportClient packTransportClient = new PreBuiltXPackTransportClient(settings());
String[] split = clusterNodes.split(",");
for (String s : split) {
String[] split1 = s.split(":");
int port = Integer.parseInt(split1[1]);
packTransportClient.addTransportAddress(new TransportAddress(InetAddress.getByName(split1[0]), port));
}
return packTransportClient;
} catch (UnknownHostException e) {
e.printStackTrace();
return null;
}
}
private Settings settings() throws FileNotFoundException {
if (is_ssl_es) {
Settings.Builder builder = Settings.builder();
builder.put("cluster.name", clusterName);
builder.put("xpack.security.user", "elastic:123456");
builder.put("xpack.security.enabled", true);
builder.put("xpack.security.transport.ssl.keystore.path", certPath);
builder.put("xpack.security.transport.ssl.truststore.path", certPath);
builder.put("xpack.security.transport.ssl.verification_mode", "certificate");
builder.put("xpack.security.transport.ssl.enabled", true);
builder.put("thread_pool.search.size", 10);//增加线程池个数，暂时设为10
return builder.build();
} else {
Settings.Builder builder = Settings.builder();
return builder.build();
}
}
@Bean(name = "elasticsearchTemplate")
public ElasticsearchTemplate getElasticSearchTemplate() throws Exception {
return new ElasticsearchTemplate(getTransportClient());
}
}
上方给出了elasticsearchTemplate访问xpack的集成，springboot使用方便
kibana集成xpack
使用xpack加密集群之后，es-header就不可以使用了，不过kibana要更好用
如上文所说PEM格式格式生成后后出现一个压缩包，正常的里面会有如下目录
instance
ca
然后下载响应的kibana版本配置yml如下
# 用于所有查询的ElasticSearch实例的url,7.2的kibana是.host
elasticsearch.url: "https://localhost:9200"
# es集群设置的xpack密码
elasticsearch.username: "elastic"
elasticsearch.password: "123456"
# 提供PEM格式SSL证书和密钥文件路径的可选设置。
# 这些文件确认您的弹性搜索后端使用相同的密钥文件。
elasticsearch.ssl.certificate: D:\certs\certificate-bundle\instance\instance.crt
elasticsearch.ssl.key: D:\certs\certificate-bundle\instance\instance.key
# 可选设置，用于为证书指定到PEM文件的路径
# 您的ElasticSearch实例的权限。
elasticsearch.ssl.certificateAuthorities: [ "D:\certs\certificate-bundle\ca\ca.crt" ]
# 若要忽略SSL证书的有效性，请将此设置的值更改为'none'
elasticsearch.ssl.verificationMode: none
访问就输入用户名密码就可以了，如果是服务器，请填写kibana的ip配置文件里面有
xpack破解
-
破解思路：需要修改es的源码
-
下面这个是我已经破解了的针对es6.5.0的jar包
- 检查x-pack 的许可证状态：
_xpack/license
- 先关闭x-park 功能：xpack.security.enabled: false
- elasticsearch-6.5.0modulesx-pack-core 替换 ，重启服务
- 使用post访问
_xpack/license注册许可证
- 检查x-pack 的许可证状态：
全文结束，如果解决了您的问题，别忘了点赞，收藏~ 撒~
文章来源于互联网:elasticsearch6.5.0(es6.5)加密xpack,java客户端访问xpack集群SSL
文章评论