Monitoring HDFS NameNode
HertzBeat monitors metrics for HDFS NameNode nodes.
Protocol Used: HTTP
Pre-Monitoring Actions
Ensure that you have obtained the JMX monitoring port for the HDFS NameNode.
Configuration Parameters
| Parameter Name | Parameter Description |
|---|---|
| Target Host | The IPv4, IPv6, or domain name of the target being monitored. Exclude protocol headers. |
| Port | The monitoring port number of the HDFS NameNode, default is 50070. |
| Query Timeout | Timeout for querying the HDFS NameNode, in milliseconds, default is 6000 milliseconds. |
| Metrics Collection Interval | Time interval for collecting monitoring data, in seconds, minimum interval is 30 seconds. |
| Probe Before Monitoring | Whether to probe and check the availability of monitoring before adding it. |
| Description/Remarks | Additional description and remarks for this monitoring. |
Collected Metrics
Metric Set: FSNamesystem
| Metric Name | Metric Unit | Metric Description |
|---|---|---|
| CapacityTotal | Total cluster storage capacity | |
| CapacityTotalGB | GB | Total cluster storage capacity |
| CapacityUsed | Used cluster storage capacity | |
| CapacityUsedGB | GB | Used cluster storage capacity |
| CapacityRemaining | Remaining cluster storage capacity | |
| CapacityRemainingGB | GB | Remaining cluster storage capacity |
| CapacityUsedNonDFS | Non-HDFS usage of cluster capacity | |
| TotalLoad | Total client connections in the cluster | |
| FilesTotal | Total number of files in the cluster | |
| BlocksTotal | Total number of BLOCKs | |
| PendingReplicationBlocks | Number of blocks awaiting replication | |
| UnderReplicatedBlocks | Number of blocks with insufficient replicas | |
| CorruptBlocks | Number of corrupt blocks | |
| ScheduledReplicationBlocks | Number of blocks scheduled for replication | |
| PendingDeletionBlocks | Number of blocks awaiting deletion | |
| ExcessBlocks | Number of excess blocks | |
| PostponedMisreplicatedBlocks | Number of misreplicated blocks postponed for processing | |
| NumLiveDataNodes | Number of live data nodes in the cluster | |
| NumDeadDataNodes | Number of data nodes marked as dead | |
| NumDecomLiveDataNodes | Number of decommissioned live nodes | |
| NumDecomDeadDataNodes | Number of decommissioned dead nodes | |
| NumDecommissioningDataNodes | Number of nodes currently being decommissioned | |
| TransactionsSinceLastCheckpoint | Number of transactions since the last checkpoint | |
| LastCheckpointTime | Time of the last checkpoint | |
| PendingDataNodeMessageCount | Number of DATANODE requests queued in the standby namenode |
Metric Set: RPC
| Metric Name | Metric Unit | Metric Description |
|---|---|---|
| ReceivedBytes | Data receiving rate | |
| SentBytes | Data sending rate | |
| RpcQueueTimeNumOps | RPC call rate |
Metric Set: runtime
| Metric Name | Metric Unit | Metric Description |
|---|---|---|
| StartTime | Start time |
Metric Set: JvmMetrics
| Metric Name | Metric Unit | Metric Description |
|---|---|---|
| MemNonHeapUsedM | MB | Current usage of NonHeapMemory by JVM |
| MemNonHeapCommittedM | MB | Committed NonHeapMemory by JVM |
| MemHeapUsedM | MB | Current usage of HeapMemory by JVM |
| MemHeapCommittedM | MB | Committed HeapMemory by JVM |
| MemHeapMaxM | MB | Maximum HeapMemory configured for JVM |
| MemMaxM | MB | Maximum memory that can be used by JVM |
| GcCountParNew | Count | Number of ParNew GC events |
| GcTimeMillisParNew | Milliseconds | Time spent in ParNew GC |
| GcCountConcurrentMarkSweep | Count | Number of ConcurrentMarkSweep GC events |
| GcTimeMillisConcurrentMarkSweep | Milliseconds | Time spent in ConcurrentMarkSweep GC |
| GcCount | Count | Total number of GC events |
| GcTimeMillis | Milliseconds | Total time spent in GC events |
| ThreadsRunnable | Count | Number of threads in RUNNABLE state |
| ThreadsBlocked | Count | Number of threads in BLOCKED state |
| ThreadsWaiting | Count | Number of threads in WAITING state |
| ThreadsTimedWaiting | Count | Number of threads in TIMED WAITING state |