开发必知必会的几套方案

镜像docker pull docker.elastic.co/elasticsearch/elasticsearch:7.10.0启动docker run --name localEs -d -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.10.0
配置docker exec -it localEs bashcd configvi elasticsearch.yml//配置http.cors.enabled: truehttp.cors.allow-origin: "*"http.cors.allow-headers: Authorizationxpack.security.enabled: truexpack.security.transport.ssl.enabled: true//配置密码-重启容器elasticsearch-setup-passwords interactive
//安装kibinadocker pull docker.elastic.co/kibana/kibana:7.10.0docker run --name kibana -d --link localEs:elasticsearch -p 5601:5601 docker.elastic.co/kibana/kibana:7.10.0//配置docker exec -it kibana bashcd configvi kibana.yml//配置server.name: kibanaserver.host: "0.0.0.0"elasticsearch.hosts: [ "http://{IpAddress}:9200" ]//docker inspect es查看es容器内部的ip地址monitoring.ui.container.elasticsearch.enabled: trueelasticsearch.username: "elastic"elasticsearch.password: "zmx111222"i18n.locale: "zh-CN"
重启容器

1.下载 https://www.elastic.co/downloads/logstashhttps://artifacts.elastic.co/downloads/logstash/logstash-7.13.1-linux-x86_64.tar.gz
2.配置编辑 /opt/logstash/logstash-7.13.1/config/logstash.yml配置文件修改如下内容：node.name: aliyun2core    #设置节点名称，一般写主机名path.data: /opt/logstash/plugin-data    #创建logstash 和插件使用的持久化目录config.reload.automatic: true    #开启配置文件自动加载config.reload.interval: 10    #定义配置文件重载时间周期http.host: "47.96.74.110"    #定义访问主机名，一般为域名或IP
3.可以直接读取日志、也可以从插件读取-例如Filebeat、Jdbc（logstash 有非常多插件https://www.elastic.co/guide/en/logstash/current/index.html
3.1安装logstash-input-jdbc 和logstash-input-beats-master 插件（暂缓测试）./bin/logstash-plugin install  logstash-input-jdbc./logstash-plugin install logstash-input-beats
3.2 codec插件 codec解决日志换行的问题在logstash读入的时候，通过codec编码解析日志为相应格式，从logstash输出的时候，通过codec解码成相应格式。
3.3 grok插件 支持以正则表达式的方式提取日志信息用于将非结构化数据解析为结构化和可查询的数据。https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html

4.启动nohup ./logstash -f ../config/logstash.conf &

1.下载https://www.elastic.co/cn/downloads/beats/filebeatcurl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.13.1-linux-x86_64.tar.gztar xzvf filebeat-7.13.1-linux-x86_64.tar.gzcd filebeat-7.13.1-linux-x86_64
2.编辑配置修改 filebeat.yml 以设置连接信息：output.elasticsearch:  hosts: ["<es_url>"]  username: "elastic"  password: "<password>"setup.kibana:  host: "<kibana_url>"  monitoring.enabled: true   #开启服务监控-需要  Elasticsearch 配合开启xpack monitoring =》编辑输入log日志路径：/opt/nifi/nifi-1.13.2/logs/nifi-app.log3.启用和配置 apache 模块./filebeat modules enable apache在 modules.d/apache.yml 文件中修改设置。
4.启动 Filebeat./filebeat setup./filebeat -e
nohup ./filebeat -e -c filebeat.yml -d "publish" & > nohup.out
提示：1.fileBeat可以从日志流直接获取日志信息 2.可以输出到logstash中

下载：https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-7.13.1-linux-x86_64.tar.gz配置：修改 /etc/metricbeat/metricbeat.yml 以设置连接信息：output.elasticsearch:  hosts: ["<es_url>"]  username: "elastic"  password: "<password>"setup.kibana:  host: "<kibana_url>"  启用和配置 system 模块（其他模板差不多流程）sudo ./metricbeat modules enable system
配置系统服务在 /etc/metricbeat/modules.d/system.yml 文件中修改设置。
启动服务sudo metricbeat setupsudo service metricbeat start
nohup ./metricbeat -e -c metricbeat.yml -d "publish" & > nohup.out
kibana上查看数据

启动node-exporter  =》仅仅提取系统指标docker run -d -p 9100:9100 \  -v "/proc:/host/proc:ro" \  -v "/sys:/host/sys:ro" \  -v "/:/rootfs:ro" \  --net="host" \  prom/node-exporter
启动prometheus  =》elt转移、存储指标mkdir /opt/prometheuscd /opt/prometheus/vim prometheus.yml
配置global:  scrape_interval:     60s  evaluation_interval: 60s scrape_configs:  - job_name: prometheus    static_configs:      - targets: ['localhost:9090']        labels:          instance: prometheus   - job_name: linux    static_configs:      - targets: ['ip:9100']        labels:          instance: localhost           启动docker docker run  -d \  -p 9090:9090 \  -v /opt/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml  \  prom/prometheus         

展示：docker run -d --name=grafana -p 3000:3000 grafana/grafana

docker run -d -p 8083:8083 -p 8086:8086   --name  my_influxdb    influxdb:latest