ELK5.5安装和配置

kibana5和之前的3版本差距是很大的，提供了一些非常不错功能，比如登陆验证和其他组建插件等，直接进入安装：但是x-pack不是无偿的。结构如下：

安装包下载

https://artifacts.elastic.co/downloads/logstash/logstash-5.5.1.rpm
https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.5.1.rpm
https://artifacts.elastic.co/downloads/kibana/kibana-5.5.1-x86_64.rpm
https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-5.5.1-x86_64.rpm

安装 elasticsearch

[root@linuxea-Node49 ~/elk]# yum install elasticsearch -y

1，安装x-pack

这个插件如果反复安装的话需要删除/etc/elasticsearch/x-pack/

[root@linuxea-Node49 ~/elk]# /usr/share/elasticsearch/bin/elasticsearch-plugin install x-pack
-> Downloading x-pack from elastic
[=================================================] 100%   
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.io.FilePermission \.pipe* read,write
* java.lang.RuntimePermission accessClassInPackage.com.sun.activation.registries
* java.lang.RuntimePermission getClassLoader
* java.lang.RuntimePermission setContextClassLoader
* java.lang.RuntimePermission setFactory
* java.security.SecurityPermission createPolicy.JavaPolicy
* java.security.SecurityPermission getPolicy
* java.security.SecurityPermission putProviderProperty.BC
* java.security.SecurityPermission setPolicy
* java.util.PropertyPermission * read,write
* java.util.PropertyPermission sun.nio.ch.bugLevel write
* javax.net.ssl.SSLPermission setHostnameVerifier
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@        WARNING: plugin forks a native controller        @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
This plugin launches a native controller that is not subject to the Java
security manager nor to system call filters.

Continue with installation? [y/N]y
-> Installed x-pack
[root@linuxea-Node49 ~/elk]# 

2，修改配置文件

修改networkhost

[root@linuxea-Node49 ~/elk]# sed -i 's/#network.host: 192.168.0.1/network.host: 0.0.0.0/g' /etc/elasticsearch/elasticsearch.yml
[root@linuxea-Node49 ~/elk]# sed -i 's/#cluster.name: my-application/cluster.name: linuxea-app/g' /etc/elasticsearch/elasticsearch.yml
[root@linuxea-Node49 ~/elk]# mkdir /elk/logs && chown elasticsearch.elasticsearch -R /elk/
[root@linuxea-Node49 ~/elk]# sed -i 's@#path.logs: /path/to/logs@path.logs: /elk/logs@g' /etc/elasticsearch/elasticsearch.yml 
[root@linuxea-Node49 ~/elk]# systemctl restart elasticsearch.service

配置文件示例

[root@linuxea-Node49 /etc/logstash]# cat /etc/elasticsearch/elasticsearch.yml 
cluster.name: linuxea-app
node.name: master
path.data: /elk/data
path.logs: /elk/logs
bootstrap.system_call_filter: false
bootstrap.memory_lock: false
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-headers: Authorization
action.auto_create_index: .security,.monitoring*,.watches,.triggered_watches,.watcher-history*,.ml*,logstash-*
network.host: 0.0.0.0
http.port: 9200
node.master: true
node.data: true
discovery.zen.ping.unicast.hosts: ["10.0.1.49"]
#discovery.zen.minimum_master_nodes: 
#xpack.security.audit.enabled: true
#xpack.security.authc.accept_default_password: false
[root@linuxea-Node49 /etc/logstash]# 

3，配置登录认证

配置elastic 密码,需要输入密码:changeme，返回为空说明正确

[root@linuxea-Node49 /data/logs]# curl -u elastic -XPUT '127.0.0.1:9200/_xpack/security/user/elastic/_password?pretty' -H 'Content-Type: application/json' -d'
{
  "password": "linuxea"
}
'
Enter host password for user 'elastic':  输入：changeme

可以省略

[root@linuxea-Node49 ~/elk]# /usr/share/elasticsearch/bin/x-pack/syskeygen 
Storing generated key in [/etc/elasticsearch/x-pack/system_key]...
Ensure the generated key can be read by the user that Elasticsearch runs as, permissions are set to owner read/write only

修改权限

[root@linuxea-Node49 ~/elk]# chmod 400 /etc/elasticsearch/x-pack/system_key 
[root@linuxea-Node49 ~/elk]# chown elasticsearch.elasticsearch /etc/elasticsearch/x-pack/system_key
[root@linuxea-Node49 ~/elk]# echo "xpack.security.audit.enabled: true" >> /etc/elasticsearch/elasticsearch.yml 

看日志：

install kibana

[root@linuxea-Node49 ~/elk]# yum install kibana -y

配置文件示例

[root@linuxea-Node49 /etc/logstash]# egrep -v "^#|^$" /etc/kibana/kibana.yml
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.url: "http://10.10.240.20:2900"
elasticsearch.username: "elastic"
elasticsearch.password: "linuxea"
tilemap.url: 'http://webrd02.is.autonavi.com/appmaptile?lang=zh_cn&size=1&scale=1&style=7&x=&y={y}&z={z}'
[root@linuxea-Node49 /etc/logstash]# 

安装插件

[root@linuxea-Node49 /elk/logs]# /usr/share/kibana/bin/kibana-plugin install x-pack
Attempting to transfer from x-pack
Attempting to transfer from https://artifacts.elastic.co/downloads/kibana-plugins/x-pack/x-pack-5.5.1.zip
Transferring 119276972 bytes....................
Transfer complete
Retrieving metadata from plugin archive
Extracting plugin archive
Extraction complete
Optimizing and caching browser bundles...
Plugin installation complete
[root@linuxea-Node49 /elk/logs]# 

配置文件解释：https://www.elastic.co/guide/en/kibana/5.5/settings.html

[root@linuxea-Node49 /elk/logs]# sed -i 's/#server.host: "localhost"/server.host: "0.0.0.0"/g' /etc/kibana/kibana.yml
[root@linuxea-Node49 /elk/logs]# sed -i 's/#elasticsearch.username: "user"/elasticsearch.username: "elastic"/g'  /etc/kibana/kibana.yml
[root@linuxea-Node49 /elk/logs]# sed -i 's/#elasticsearch.password: "pass"/elasticsearch.password: "linuxea"/g'  /etc/kibana/kibana.yml
[root@linuxea-Node49 /elk/logs]# sed -i 's@#elasticsearch.url: "http://localhost:9200"@elasticsearch.url: "http://127.0.0.1:9200"@g' /etc/kibana/kibana.yml
[root@linuxea-Node49 /var/log]# sed -i 's/#server.port: 5601/server.port: 5601/g' /etc/kibana/kibana.yml 

配置kibana密码,需要输入密码:linuxea

[root@linuxea-Node49 /data/logs]# curl -u elastic -XPUT '127.0.0.1:9200/_xpack/security/user/kibana/_password?pretty' -H 'Content-Type: application/json' -d'
{
  "password": "linuxea"
}
'
Enter host password for user 'elastic':

install logstach

直接yum即可配置文件示例

[root@DS-VM-Node49 /etc/logstash]# cat logstash.yml 
node.name: node1
path.data: /elk/logstash/data
path.config: /etc/logstash/conf.d
log.level: info
path.logs: /elk/logstash/logs

安装x-pack

[root@linuxea-Node49 /elk/elasticsearch-head]# /usr/share/logstash/bin/logstash-plugin install x-pack
Downloading file: https://artifacts.elastic.co/downloads/logstash-plugins/x-pack/x-pack-5.5.1.zip
Downloading [=============================================================] 100%
Installing file: /tmp/studtmp-2d494e1b2f721643348e5d8787188f1234f43369beb164da7a73bc94b899/x-pack-5.5.1.zip
Install successful 
[root@linuxea-Node49 /etc/logstash]# sed -i 's/#log.level: info/log.level: info/g' /etc/logstash/logstash.yml

这里需要密码改一下

[root@linuxea-Node49 /data/logs]# curl -u elastic -XPUT '127.0.0.1:9200/_xpack/security/user/logstash_system/_password?pretty' -H 'Content-Type: application/json' -d'
{
  "password": "linuxea"
}
'
Enter host password for user 'elastic':

redis链接,redis已经部署好了，当然，这里使用的docker

[root@linuxea-Node49 /etc/logstash/conf.d]# cat redis_input.conf 
input {
    redis {
        host => "10.10.0.98"
        port => "6379"
        key => "filebeat"
        data_type => "list"
        password => "OTdmOWI4ZTM4NTY1M2M4OTZh"
        threads => 20
    }
}
output {
    elasticsearch {
        hosts => ["127.0.0.1:9200"]
        index => "logstash-nginx-error-%{+YYYY.MM.dd}"
        user => "elastic"
        password => "linuxea"
    }
    stdout {codec => rubydebug}
}

安装模块：这些模块后面会用到

[root@linuxea-Node49 /etc/logstash/conf.d]# /usr/share/elasticsearch/bin/elasticsearch-plugin install ingest-geoip
-> Downloading ingest-geoip from elastic
[=================================================] 100%   
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission accessDeclaredMembers
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
-> Installed ingest-geoip
[root@linuxea-Node49 /etc/logstash/conf.d]# /usr/share/elasticsearch/bin/elasticsearch-plugin install ingest-user-agent
-> Downloading ingest-user-agent from elastic
[=================================================] 100%   
-> Installed ingest-user-agent
[root@linuxea-Node49 /etc/logstash/conf.d]# 

到此elk安装完成