JMX采集Kafka指标
## 使用JMX采集Kafka指标
从 JMX-Exporter 提供的样例配置目录，下载 kafka.yml，然后导出 KAFKA_OPTS 环境变量即可。
下载 jmx_prometheus_javaagent 和 kafka.yaml 。
```sh
wget https://raw.githubusercontent.com/prometheus/jmx_exporter/master/example_configs/kafka-2_0_0.yml
wget https://repo1.maven.org/maven2/io/prometheus/jmx/jmx_prometheus_javaagent/0.20.0/jmx_prometheus_javaagent-0.20.0.jar
```
Kafka配置参数如下：
```sh
export JMX_PORT="9995"
export KAFKA_OPTS="-javaagent:/path/jmx_prometheus_javaagent-0.20.0.jar=9996:/etc/jmx-exporter/kafka-2_0_0.yml"
```
配置方式有如下几种：
修改bin/kafka-run-class.sh脚本,在开始运行的最上方加入以上配置。
修改bin/kafka-server-start.sh脚本，在开始运行的最上方加入以上配置。
```sh
vim bin/kafka-server-start.sh
if [ $# -lt 1 ];
then
echo "USAGE: $0 [-daemon] server.properties [--override property=value]*"
exit 1
fi
base_dir=$(dirname $0)
if [ "x$KAFKA_LOG4J_OPTS" = "x" ]; then
export KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:$base_dir/../config/log4j.properties"
fi
if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
fi
EXTRA_ARGS=${EXTRA_ARGS-'-name kafkaServer -loggc'}
# 插入到此*******************************************************
export JMX_PORT="9995"
export KAFKA_OPTS="-javaagent:/mnt/kafka/monitor/jmx_prometheus_javaagent-0.20.0.jar=9996:/mnt/kafka/monitor/kafka-2_0_0.yml"
COMMAND=$1
case $COMMAND in
-daemon)
EXTRA_ARGS="-daemon "$EXTRA_ARGS
shift
;;
*)
;;
esac
exec $base_dir/kafka-run-class.sh $EXTRA_ARGS kafka.Kafka "$@"
```
修改后重启kafka,可以看到指标即为成功
```
[root@node-104 bin]# curl -s 172.24.61.104:9996/metrics |head -10
# HELP kafka_coordinator_transaction_transactionmarkerchannelmanager_unknowndestinationqueuesize Attribute exposed for management kafka.coordinator.transaction:name=UnknownDestinationQueueSize,type=TransactionMarkerChannelManager,attribute=Value
# TYPE kafka_coordinator_transaction_transactionmarkerchannelmanager_unknowndestinationqueuesize gauge
kafka_coordinator_transaction_transactionmarkerchannelmanager_unknowndestinationqueuesize 0.0
# HELP kafka_controller_controllerstats_topicuncleanleaderelectionenablerateandtimems Attribute exposed for management kafka.controller:name=TopicUncleanLeaderElectionEnableRateAndTimeMs,type=ControllerStats,attribute=50thPercentile
# TYPE kafka_controller_controllerstats_topicuncleanleaderelectionenablerateandtimems gauge
kafka_controller_controllerstats_topicuncleanleaderelectionenablerateandtimems{quantile="0.50",} 0.0
kafka_controller_controllerstats_topicuncleanleaderelectionenablerateandtimems{quantile="0.75",} 0.0
kafka_controller_controllerstats_topicuncleanleaderelectionenablerateandtimems{quantile="0.95",} 0.0
kafka_controller_controllerstats_topicuncleanleaderelectionenablerateandtimems{quantile="0.98",} 0.0
kafka_controller_controllerstats_topicuncleanleaderelectionenablerateandtimems{quantile="0.99",} 0.0
```
将metrics添加到prometheus-operator采集项
```yaml
kind: Service
apiVersion: v1
metadata:
name: kafka-jmx #job名
namespace: default
labels:
app: kafka-jmx
spec:
ports:
- port: 9996
name: metrics
protocol: TCP
targetPort: 9996
---
kind: Endpoints
apiVersion: v1
metadata:
name: kafka-jmx
namespace: default
subsets:
- addresses:
- ip: 172.24.61.103
- ip: 172.24.61.104
- ip: 172.24.61.102
ports:
- port: 9996
name: metrics
protocol: TCP
---
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: kafka-jmx-nodeexporter
namespace: default
labels:
app: kafka-jmx
spec:
endpoints:
- port: metrics
interval: 30s
namespaceSelector:
matchNames:
- default
selector:
matchLabels:
app: kafka-jmx
```
kubectl apply -f kafka-jmx.yaml
添加后等待片刻去prometheus查看是否添加成功
导入大屏模板即可