首先介绍下ElastiFlow自家文档:
ElastiFlow是用来干什么得?
ElastiFlow ™统一流收集器使用 IPFIX、Netflow 和 sFlow 接收、解码、转换、规范化、转换和丰富从网络设备和应用程序发送得网络流记录和遥测数据。生成得记录可以发送到各种 Elasticsearch 发行版和服务,包括:
弹性搜索弹性云面向 Elasticsearch 得 Open DistroAWS 弹性搜索服务对各厂商得支持(华为可以使用sFlow)
前置条件:
1、一台centos7机器
2、已安装docker和docker-compose
3、不要使用root账号,root可能起不来
安装:
一、创建一个docker-compose.yml文件,并且写入一下内容
version: '3'services: elastiflow-elasticsearch: image: docker.elastic.co/elasticsearch/elasticsearch:7.8.1 container_name: elastiflow-elasticsearch restart: 'no' ulimits: memlock: soft: -1 hard: -1 nofile: soft: 131072 hard: 131072 nproc: 8192 fsize: -1 network_mode: host volumes: # 使用如下命令创建数据目录 # mkdir /qyt_elastiflow && chown -R 1000:1000 /qyt_elastiflow - /qyt_elastiflow:/usr/share/elasticsearch/data environment: # JVM Heap size # - this should be at least 2GB for simple testing, receiving only a few flows per second. # - for production environments upto 31GB is recommended. ES_JAVA_OPTS: '-Xms4g -Xmx4g' cluster.name: elastiflow bootstrap.memory_lock: 'true' network.host: 0.0.0.0 http.port: 9200 discovery.type: 'single-node' indices.query.bool.max_clause_count: 8192 search.max_buckets: 250000 action.destructive_requires_name: 'true' elastiflow-kibana: image: docker.elastic.co/kibana/kibana:7.8.1 container_name: elastiflow-kibana restart: 'no' depends_on: - elastiflow-elasticsearch network_mode: host environment: SERVER_HOST: 0.0.0.0 SERVER_PORT: 5601 SERVER_MAXPAYLOADBYTES: 8388608 ELASTICSEARCH_HOSTS: "感谢分享127.0.0.1:9200" ELASTICSEARCH_REQUESTTIMEOUT: 132000 ELASTICSEARCH_SHARDTIMEOUT: 120000 KIBANA_DEFAULTAPP发布者会员账号: "dashboard/653cf1e0-2fd2-11e7-99ed-49759aed30f5" KIBANA_AUTOCOMPLETETIMEOUT: 3000 KIBANA_AUTOCOMPLETETERMINATEAFTER: 2500000 LOGGING_DEST: stdout LOGGING_QUIET: 'false' elastiflow-logstash: image: robcowart/elastiflow-logstash:4.0.1 container_name: elastiflow-logstash restart: 'no' depends_on: - elastiflow-elasticsearch network_mode: host environment: # JVM Heap size - this MUST be at least 3GB (4GB preferred) LS_JAVA_OPTS: '-Xms4g -Xmx4g' # ElastiFlow global configuration ELASTIFLOW_AGENT_发布者会员账号: elastiflow ELASTIFLOW_GEOIP_CACHE_SIZE: 16384 ELASTIFLOW_GEOIP_LOOKUP: 'true' ELASTIFLOW_ASN_LOOKUP: 'true' ELASTIFLOW_OUI_LOOKUP: 'false' ELASTIFLOW_POPULATE_LOGS: 'true' ELASTIFLOW_KEEP_ORIG_data: 'true' ELASTIFLOW_DEFAULT_APP发布者会员账号_SRCTYPE: '__UNKNOWN' # Name resolution option ELASTIFLOW_RESOLVE_IP2HOST: 'false' ELASTIFLOW_NAMESERVER: '127.0.0.1' ELASTIFLOW_DNS_HIT_CACHE_SIZE: 25000 ELASTIFLOW_DNS_HIT_CACHE_TTL: 900 ELASTIFLOW_DNS_FAILED_CACHE_SIZE: 75000 ELASTIFLOW_DNS_FAILED_CACHE_TTL: 3600 ELASTIFLOW_ES_HOST: '127.0.0.1:9200' #ELASTIFLOW_ES_USER: 'elastic' #ELASTIFLOW_ES_PASSWD: 'changeme' ELASTIFLOW_NETFLOW_IPV4_PORT: 2055 ELASTIFLOW_NETFLOW_UDP_WORKERS: 2 ELASTIFLOW_NETFLOW_UDP_QUEUE_SIZE: 4096 ELASTIFLOW_NETFLOW_UDP_RCV_BUFF: 33554432 ELASTIFLOW_SFLOW_IPV4_PORT: 6343 ELASTIFLOW_SFLOW_UDP_WORKERS: 2 ELASTIFLOW_SFLOW_UDP_QUEUE_SIZE: 4096 ELASTIFLOW_SFLOW_UDP_RCV_BUFF: 33554432 ELASTIFLOW_IPFIX_UDP_IPV4_PORT: 4739 ELASTIFLOW_IPFIX_UDP_WORKERS: 2 ELASTIFLOW_IPFIX_UDP_QUEUE_SIZE: 4096 ELASTIFLOW_IPFIX_UDP_RCV_BUFF: 33554432
二、部署容器
docker-compose up -d
三、等待一段时间后、既可以在浏览器中进入图形化界面
联动设备测试:
目得:
收集设备得sFlow信息,并用图形化呈现
实验设备:
华为S5720,完成开局设置,并且与172.21.100.153(服务器)通信
实验步骤
一、完成S5720得设备配置
# 系统视图下
sflow agent ip 172.21.254.18
sflow collector 2 ip 172.21.100.153 description 172.21.254.18
# 接口视图下
sflow flow-sampling rate 4000
sflow flow-sampling collector 2
]sflow counter-sampling interval 120
sflow counter-sampling collector 2
查看sflow配置信息
二、 创建Kibana索引模式
引入elastiflow.kibana.7.8.x.ndjson,代码在下方
导入完成后,可以在界面中看到多个dashboards信息。
三、 查看自己dashboard信息
输出如下
参考资料:
视频教程:感谢分享特别bilibili感谢原创分享者/video/BV1HK4y1p7a8
代码地址:感谢分享gitee感谢原创分享者/qytang/ElasticFlow
自家文档:感谢分享docs.elastiflow感谢原创分享者/docs/