1. 首页
  2. docker
  3. 正文

elasticsearch6.5.0(es6.5)加密xpack,java客户端访问xpack集群SSL

2021年12月15日 383点热度 0条评论

elasticsearch6.5.0(es6.5)加密xpack,java客户端访问xpack集群SSL

  • xpack已经在es6.3之后默认集成，本文包含：
  • 1、es集群使用xpack加密，达到访问集群需要用户密码登录的效果。
  • 2、采用SSL加密通信，提供https访问方式
  • 3、java客户端如何连接加密后的es集群(9200,9300端口)httpclient方式，restClient高低版本方式都有包含
  • 4、使用kibana界面化工具连接操作加密后的es集群
开启xpack

  • elasticsearch.yml 中加上： xpack.security.enabled: true

    注：:后空一个空格，如果是es集群请务必全部添加此配置

  • 确保集群健康访问_xpack/license/start_trial?acknowledge=true

    开启xpack功能，这时候集群会立马提示您输入用户名密码的弹窗进行登录，别急还需要设置一下
    注:(如果只需要加密功能，可以省略此步骤，以便永久免费使用，官方对于基本版不收费)此开启的是试用版，xpack的破解，请自行搜索，我文末也会提供一个6.5.0的破解包，替换一下就可以用了

  • 设置es集群用户名密码

    #自动生成(二选一) elasticsearch-setup-passwords auto    
#手动生成(推荐) 	 elasticsearch-setup-passwords interactive

    此处最容易出现错误，请保证您的集群是健康的并且全部开启了xpack功能，且配置一样，如果生成失败，请检查自己的集群配置，如果设置密码失败的可以试试这个博文的顺序ELK6.6.0 Xpack

  • 为Elasticearch集群创建一个证书颁发机构

    PKCS#12格式生成(代码推荐)：elasticsearch-certutil ca
PEM格式格式生成(kibana推荐)：elasticsearch-certutil ca --pem

    生成过程会提示输入密码，作用是访问证书的安全性，可以不设置

  • 为es集群生成证书和秘钥

    PKCS#12格式生成：elasticsearch-certutil cert --ca elastic-stack-ca.p12 
PEM格式格式生成：elasticsearch-certutil cert --pem

    同上会提示输入密码，略过
    pkcs12证书里面包含了秘钥所以只有一个文件，pem会有独立的证书和key
    请把生成的证书elastic-certificates.p12拷贝到集群中的其他节点，否则加密失效

  • TLS/SSL加密Transport通信

     xpack.security.transport.ssl.enabled: true  
 xpack.security.transport.ssl.verification_mode: certificate
 xpack.security.transport.ssl.keystore.path: certselastic-certificates.p12
 xpack.security.transport.ssl.truststore.path: certselastic-certificates.p12

  • 加密https访问

    xpack.security.http.ssl.enabled: true 
xpack.security.http.ssl.keystore.path: certselastic-certificates.p12
xpack.security.http.ssl.truststore.path: certselastic-certificates.p12
按照上面的步骤集群就可以实现加密访问，下面是使用java访问加密后的es集群
给出pom
repositories>
      
      repository>
          id>elasticsearch-releasesid>
          url>https://artifacts.elastic.co/mavenurl>
          releases>
              enabled>trueenabled>
          releases>
          snapshots>
              enabled>falseenabled>
          snapshots>
      repository>
  repositories>
  
        
        dependency>
            groupId>org.elasticsearch.plugingroupId>
            artifactId>transport-netty4-clientartifactId>
            version>${es.version}version>
        dependency>

        dependency>
            groupId>org.elasticsearchgroupId>
            artifactId>elasticsearchartifactId>
            version>${es.version}version>
        dependency>
        dependency>
            groupId>org.elasticsearch.clientgroupId>
            artifactId>transportartifactId>
            version>${es.version}version>
        dependency>
        dependency>
            groupId>org.elasticsearch.clientgroupId>
            artifactId>x-pack-transportartifactId>
            version>${es.version}version>
        dependency>
        
        dependency>
            groupId>org.elasticsearch.clientgroupId>
            artifactId>elasticsearch-rest-clientartifactId>
            version>6.5.0version>
        dependency>

        
        
        dependency>
            groupId>org.apache.httpcomponentsgroupId>
            artifactId>httpclientartifactId>
            version>4.5.2version>
        dependency>
HttpClient方式
package com.**.**.configuration;

import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.client.HttpClient;
import org.apache.http.config.Registry;
import org.apache.http.config.RegistryBuilder;
import org.apache.http.conn.socket.ConnectionSocketFactory;
import org.apache.http.conn.socket.PlainConnectionSocketFactory;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
import org.apache.http.ssl.SSLContextBuilder;
import org.apache.http.ssl.SSLContexts;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSession;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.security.KeyStore;

/**
 * Created by Majg on 2019-06-04
 **/
@Configuration
public class ESHttpClientConfig {

    @Bean(name = "EsHttpClient")
    public HttpClient getHttpClient() throws Exception {
        KeyStore truststore = KeyStore.getInstance("PKCS12");

//        File file = ResourceUtils.getFile(ResourceUtils.CLASSPATH_URL_PREFIX + "certs/elastic-certificates.p12");
//        //生产环境，打包方式不同，获取的是jar内的环境
//        String absolutePath = file.getAbsolutePath();
        String absolutePath = "C:\certs\elastic-certificates.p12";

        try (InputStream is = Files.newInputStream(Paths.get(absolutePath))) {
            truststore.load(is, "".toCharArray());
        }
        SSLContextBuilder sslBuilder = SSLContexts.custom().loadTrustMaterial(truststore, new TrustSelfSignedStrategy());

        // 获取证书
        SSLContext sslcontext = sslBuilder.build();

        // 以上证书有效，但是证书签名是instance,下方有设置跳过签名验证
        final CredentialsProvider credentialsProvider =
                new BasicCredentialsProvider();
        credentialsProvider.setCredentials(AuthScope.ANY,
                new UsernamePasswordCredentials("elastic", "密码"));

        //创建自定义的httpclient对象
        CloseableHttpClient client = HttpClients.custom()
                .setDefaultCredentialsProvider(credentialsProvider)
                //跳过证书签名
                .setSSLHostnameVerifier(new HostnameVerifier() {
                    public boolean verify(String hostname, SSLSession session) {
                        return true;
                    }
                })
                .setSSLContext(sslcontext)
                .build();
        return client;
    }

//    // 跳过证书签名
//    private HostnameVerifier getTrustedVerifier() {
//        if (TRUSTED_VERIFIER == null)
//            TRUSTED_VERIFIER = new HostnameVerifier() {
//
//                public boolean verify(String hostname, SSLSession session) {
//                    return true;
//                }
//            };
//        return TRUSTED_VERIFIER;
//    }

}
restClient方式(本人觉得最好用，也是官方推荐的)
import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.config.RequestConfig.Builder;
import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
import org.apache.http.ssl.SSLContexts;
import org.elasticsearch.client.Node;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestClientBuilder;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSession;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.security.KeyManagementException;
import java.security.KeyStore;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.security.cert.CertificateException;

/**
 * Created by JGMa on 2019-06-03
 **/
@Configuration
public class ESRestConfig {

    private static Logger logger = LoggerFactory.getLogger(ESRestConfig.class);
    private int connectTimeOut = 1000;
    private int socketTimeOut = 30000;
    private int maxConnectNum = 100;
    private int maxConnectPerRoute = 100;
    private int connectionRequestTimeOut = 5000;

    @Value("${elasticsearch.cluster-nodes}")
    private String clusterNodes;

    @Value("${is_SSL_es}")
    private Boolean is_SSL_es;

    @Value("${elasticsearch.certPath:certs\elastic-certificates.p12}")
    private String certPath;

    @Bean("restHighLevelClient")
    public RestClient RestClient() {
        RestClient restClient = null;
        String scheme = null;
        if (is_SSL_es) {
            scheme = "https";
        } else {
            scheme = "http";
        }

        // 配置hostName
        String[] split = clusterNodes.split(",");

        int length = split.length;
        HttpHost[] https = new HttpHost[length];

        for (int i = 0; i  length; i++) {

            String[] split1 = split[i].split(":");
            https[i] = new HttpHost(split1[0], Integer.valueOf(split1[1]), scheme);

        }

        RestClientBuilder builder = RestClient.builder(https)
                .setFailureListener(new RestClient.FailureListener() {
                    @Override
                    public void onFailure(Node node) {
                        HttpHost host = node.getHost();
                        logger.error("连接ES节点失败，host:{}", host);
                    }
                });
        //连接数配置
        setMutiConnectConfig(builder);
        //超时配置
        setRequestTimeOutConfig(builder);

        restClient = builder.build();
        return restClient;
    }

    /**
     * 异步httpclient的连接延时配置
     */
    public void setRequestTimeOutConfig(RestClientBuilder builder) {
        builder.setRequestConfigCallback(new RestClientBuilder.RequestConfigCallback() {
            @Override
            public Builder customizeRequestConfig(RequestConfig.Builder requestConfigBuilder) {
                requestConfigBuilder.setConnectTimeout(connectTimeOut);
                requestConfigBuilder.setSocketTimeout(socketTimeOut);
                requestConfigBuilder.setConnectionRequestTimeout(connectionRequestTimeOut);
                return requestConfigBuilder;
            }
        });
    }

    /**
     * 异步httpclient的连接数配置
     */
    public void setMutiConnectConfig(RestClientBuilder builder) {
        builder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
            @Override
            public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
                if (is_SSL_es) {
                    try {
                        KeyStore truststore = KeyStore.getInstance("PKCS12");

                        truststore.load(Files.newInputStream(Paths.get(certPath)), "".toCharArray());

                        SSLContext sslcontext = SSLContexts.custom().loadTrustMaterial(truststore, new TrustSelfSignedStrategy()).build();
                        CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
                        credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials("elastic", "123456"));

                        httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
                        httpClientBuilder.setSSLHostnameVerifier(new HostnameVerifier() {
                            public boolean verify(String hostname, SSLSession session) {
                                return true;
                            }
                        });
                        httpClientBuilder.setSSLContext(sslcontext);
                        httpClientBuilder.setMaxConnTotal(maxConnectNum);
                        httpClientBuilder.setMaxConnPerRoute(maxConnectPerRoute);
                    } catch (KeyStoreException | IOException | CertificateException | NoSuchAlgorithmException | KeyManagementException e) {
                        logger.error("配置ES加密集群连接数错误！-", e);
                        e.printStackTrace();
                    }
                } else {
                    httpClientBuilder.setMaxConnTotal(maxConnectNum);
                    httpClientBuilder.setMaxConnPerRoute(maxConnectPerRoute);
                }
                return httpClientBuilder;
            }
        });
    }

}
transport方式(一般代码写入会使用这个使用9300端口，但是官方不推荐并且会在8之后移除此)

import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.transport.TransportAddress;
import org.elasticsearch.xpack.client.PreBuiltXPackTransportClient;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.elasticsearch.core.ElasticsearchTemplate;

import java.io.FileNotFoundException;
import java.net.InetAddress;
import java.net.UnknownHostException;

/**
 * Created by JGMa on 2019-06-03
 **/
@Configuration
public class ESTransportConfig {
    @Value("${elasticsearch.cluster_name:elasticsearch}")
    private String clusterName;

    @Value("${elasticsearch.transport_es_nodes:127.0.0.1:9300}")
    private String clusterNodes;

    @Value("${is_SSL_es}")
    private Boolean is_ssl_es;

    @Value("${elasticsearch.certPath:certs\elastic-certificates.p12}")
    private String certPath;

    @Bean(name = "sslTransportClient")
    public TransportClient getTransportClient() throws FileNotFoundException {
        try {
            PreBuiltXPackTransportClient packTransportClient = new PreBuiltXPackTransportClient(settings());
            String[] split = clusterNodes.split(",");
            for (String s : split) {
                String[] split1 = s.split(":");
                int port = Integer.parseInt(split1[1]);
                packTransportClient.addTransportAddress(new TransportAddress(InetAddress.getByName(split1[0]), port));
            }

            return packTransportClient;
        } catch (UnknownHostException e) {
            e.printStackTrace();
            return null;
        }
    }

    private Settings settings() throws FileNotFoundException {
        if (is_ssl_es) {
            Settings.Builder builder = Settings.builder();
            builder.put("cluster.name", clusterName);
            builder.put("xpack.security.user", "elastic:123456");
            builder.put("xpack.security.enabled", true);
            builder.put("xpack.security.transport.ssl.keystore.path", certPath);
            builder.put("xpack.security.transport.ssl.truststore.path", certPath);
            builder.put("xpack.security.transport.ssl.verification_mode", "certificate");
            builder.put("xpack.security.transport.ssl.enabled", true);
            builder.put("thread_pool.search.size", 10);//增加线程池个数，暂时设为10
            return builder.build();
        } else {
            Settings.Builder builder = Settings.builder();
            return builder.build();

        }
    }

    @Bean(name = "elasticsearchTemplate")
    public ElasticsearchTemplate getElasticSearchTemplate() throws Exception {
        return new ElasticsearchTemplate(getTransportClient());
    }
}

上方给出了elasticsearchTemplate访问xpack的集成，springboot使用方便

kibana集成xpack

使用xpack加密集群之后，es-header就不可以使用了，不过kibana要更好用

如上文所说PEM格式格式生成后后出现一个压缩包，正常的里面会有如下目录
instance
%title插图%num
ca
%title插图%num
然后下载响应的kibana版本配置yml如下

# 用于所有查询的ElasticSearch实例的url,7.2的kibana是.host
elasticsearch.url: "https://localhost:9200"

# es集群设置的xpack密码
elasticsearch.username: "elastic"
elasticsearch.password: "123456"

# 提供PEM格式SSL证书和密钥文件路径的可选设置。 
# 这些文件确认您的弹性搜索后端使用相同的密钥文件。
elasticsearch.ssl.certificate: D:\certs\certificate-bundle\instance\instance.crt
elasticsearch.ssl.key: D:\certs\certificate-bundle\instance\instance.key

# 可选设置，用于为证书指定到PEM文件的路径 
# 您的ElasticSearch实例的权限。 
elasticsearch.ssl.certificateAuthorities: [ "D:\certs\certificate-bundle\ca\ca.crt" ]

# 若要忽略SSL证书的有效性，请将此设置的值更改为'none'
elasticsearch.ssl.verificationMode: none

访问就输入用户名密码就可以了，如果是服务器，请填写kibana的ip配置文件里面有

xpack破解

  • 破解思路：需要修改es的源码

  • 下面这个是我已经破解了的针对es6.5.0的jar包

    • 检查x-pack 的许可证状态：_xpack/license
      %title插图%num
    • 先关闭x-park 功能：xpack.security.enabled: false
    • elasticsearch-6.5.0modulesx-pack-core 替换 ，重启服务
    • 使用post访问_xpack/license 注册许可证
      %title插图%num

全文结束，如果解决了您的问题，别忘了点赞，收藏~ 撒~

文章来源于互联网:elasticsearch6.5.0(es6.5)加密xpack,java客户端访问xpack集群SSL

微海报分享
本作品采用 知识共享署名 4.0 国际许可协议 进行许可
标签： 暂无
最后更新：2021年12月15日

harry

这个人很懒，什么都没留下

文章评论

关注
Sora的第一波受害者出现了。 龙年第一个赚钱机会，是AI知识付费？ 人工智能文生视频大模型Sora,颠覆性的革命? 与 Java 17 相比，Java 21 究竟有多快？ 宝塔部署Nuxt3项目 最佳实践 | 即时通信IM接入AI服务，高效构建智能聊天能力
ElasticSearch、ES-head、Kibana最新版安装教程 人工智能文生视频大模型Sora,颠覆性的革命? linux redis安装及安装遇到的问题及解决方案-记录 几个内网端口映射服务网站 “我不是不在乎钱，我只是不在乎这点钱。” Mac OS 下三种修改Hosts文件的方法
标签聚合
Elasticsearch 橘子CPS联盟 cps spring 外卖cps 橘子cps 设计模式 springcloud 2020
分类

COPYRIGHT © 2023 Harry技术. ALL RIGHTS RESERVED.

豫ICP备2021018072号-1