New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add examples of Apache Ignite 2.6.0 basic JMX metrics #319

Open

Mmuzaf wants to merge 9 commits into prometheus:main from Mmuzaf:add-ignite-examples

Mmuzaf commented Sep 13, 2018

@brian-brazil , @skonto

Hi Brian, Stavros

Can you review my changes?
I've added basic JMX metrics for Apache Ignite https://ignite.apache.org/ project for version 2.6.0. Hope they will help somebody to use this tool.

Kind regards,
Maxim Muzafarov

Mmuzaf force-pushed the add-ignite-examples branch from 7ed010f to 7a1cf49 Compare

September 13, 2018 14:02

brian-brazil reviewed

View reviewed changes

Contributor

brian-brazil left a comment

Do you have example output?

example_configs/ignite_2.6.0.yml Outdated

    
            @@ -0,0 +1,95 @@
          
              # Licensed to the Apache Software Foundation (ASF) under one or more

Contributor

brian-brazil Sep 13, 2018

We don't put this in these files.

Author

Mmuzaf Sep 13, 2018

Removed.

example_configs/ignite_2.6.0.yml

    
              lowercaseOutputLabelNames: true

              lowercaseOutputName: true

              rules:

              # Apache Ignite provide runtime information on a cluster.  Keep in mind that there will be a certain

Contributor

brian-brazil Sep 13, 2018

It'd be better to get the data from the individual processes directly, rather than adding jitter and lag.

Author

Mmuzaf Sep 13, 2018

Brian, Could you, please, calrify? In general, I agree that it's better to share individual processes metrics. But I would like to have an batch of heap and non-heap metrics for the whole cluster.

example_configs/ignite_2.6.0.yml

    
                name: ignite_cluster_heap_$1

                help: Ignite cluster amount of heap memory in bytes

                labels:

                  attr: $3

Contributor

brian-brazil Sep 13, 2018

Metric names should be in the name

Author

Mmuzaf Sep 13, 2018

Agree. Added.

Contributor

brian-brazil Sep 18, 2018

This wasn't done, all these attrs should be in the metric name.

example_configs/ignite_2.6.0.yml Outdated

    
              #

              # see https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cluster/ClusterMetrics.html

              - pattern: "^org.apache<clsLdr=(.+), group=Kernal, name=ClusterMetrics(.+)><>(HeapMemory.*):"

                name: ignite_cluster_heap_$1

Contributor

brian-brazil Sep 13, 2018

clsLdr sounds like a label - does it vary as the leader changes?

Author

Mmuzaf Sep 13, 2018

Brian, it remains the same for single cluster but can be changed if e.g. we will run another. What should I do with this? Is it valid name metric name?

Contributor

brian-brazil Sep 18, 2018

You should drop it.

example_configs/ignite_2.6.0.yml

    
              # see https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cluster/ClusterMetrics.html

              - pattern: "^org.apache<clsLdr=(.+), group=Kernal, name=ClusterMetrics(.+)><>(HeapMemory.*):"

                name: ignite_cluster_heap_$1

                help: Ignite cluster amount of heap memory in bytes

Contributor

brian-brazil Sep 13, 2018

If you know it's in bytes, append _bytes to the name

Author

Mmuzaf Sep 13, 2018

Agree. Added. JVM metrics used as an example.

example_configs/ignite_2.6.0.yml

    
                help: Ignite durable memory pages currently loaded in RAM

                labels:

                  attr: $3

              - pattern: "^org.apache<clsLdr=(.+), group=DataRegionMetrics, name=(.+)><>(Off[Hh]eap.*|MaxSize|PhysicalMemorySize|TotalAllocatedSize):"

Contributor

brian-brazil Sep 13, 2018

Are these the usual JVM memory metrics?

Author

Mmuzaf Sep 13, 2018

Brian, no these metrics are collected through the whole cluster on specific memory region. They are not bounded to JVMs.

example_configs/ignite_2.6.0.yml Outdated

    
                help: Ignite cluster job detalization

                labels:

                  attr: $3

                type: COUNTER

Contributor

brian-brazil Sep 13, 2018

Counters should end in _total

Author

Mmuzaf Sep 13, 2018

Fixed.

Mmuzaf force-pushed the add-ignite-examples branch from e11f5c2 to 98f203b Compare

September 13, 2018 16:40

Author

Mmuzaf commented Sep 13, 2018 •

edited

Example,

# HELP ignite_18b4aac2_metastorememplc_pages Ignite durable memory pages currently loaded in RAM
# TYPE ignite_18b4aac2_metastorememplc_pages gauge
ignite_18b4aac2_metastorememplc_pages{attr="DirtyPages",} 0.0
ignite_18b4aac2_metastorememplc_pages{attr="PhysicalMemoryPages",} 0.0
ignite_18b4aac2_metastorememplc_pages{attr="TotalAllocatedPages",} 0.0
ignite_18b4aac2_metastorememplc_pages{attr="AllocationRate",} 0.0
ignite_18b4aac2_metastorememplc_pages{attr="CheckpointBufferPages",} 0.0
# HELP ignite_18b4aac2_cacheclustermetricsmxbeanimpl_test_cache_operations_total Ignite cache "test-cache" operations metrics
# TYPE ignite_18b4aac2_cacheclustermetricsmxbeanimpl_test_cache_operations_total counter
ignite_18b4aac2_cacheclustermetricsmxbeanimpl_test_cache_operations_total{attr="CacheGets",cache="\"test-cache\"",} 0.0
ignite_18b4aac2_cacheclustermetricsmxbeanimpl_test_cache_operations_total{attr="CacheRemovals",cache="\"test-cache\"",} 0.0
ignite_18b4aac2_cacheclustermetricsmxbeanimpl_test_cache_operations_total{attr="CachePuts",cache="\"test-cache\"",} 0.0
ignite_18b4aac2_cacheclustermetricsmxbeanimpl_test_cache_operations_total{attr="CacheMisses",cache="\"test-cache\"",} 0.0
ignite_18b4aac2_cacheclustermetricsmxbeanimpl_test_cache_operations_total{attr="CacheHits",cache="\"test-cache\"",} 0.0
ignite_18b4aac2_cacheclustermetricsmxbeanimpl_test_cache_operations_total{attr="CacheHitPercentage",cache="\"test-cache\"",} 0.0
ignite_18b4aac2_cacheclustermetricsmxbeanimpl_test_cache_operations_total{attr="CacheTxRollbacks",cache="\"test-cache\"",} 0.0
ignite_18b4aac2_cacheclustermetricsmxbeanimpl_test_cache_operations_total{attr="CacheTxCommits",cache="\"test-cache\"",} 0.0
ignite_18b4aac2_cacheclustermetricsmxbeanimpl_test_cache_operations_total{attr="CacheMissPercentage",cache="\"test-cache\"",} 0.0
ignite_18b4aac2_cacheclustermetricsmxbeanimpl_test_cache_operations_total{attr="CacheEvictions",cache="\"test-cache\"",} 0.0
# HELP ignite_18b4aac2_cacheclustermetricsmxbeanimpl_test_cache_rebalance_rate Ignite cache "test-cache" rebalancing rate
# TYPE ignite_18b4aac2_cacheclustermetricsmxbeanimpl_test_cache_rebalance_rate gauge
ignite_18b4aac2_cacheclustermetricsmxbeanimpl_test_cache_rebalance_rate{attr="RebalancingBytesRate",cache="\"test-cache\"",} 0.0
ignite_18b4aac2_cacheclustermetricsmxbeanimpl_test_cache_rebalance_rate{attr="RebalancingKeysRate",cache="\"test-cache\"",} 0.0
# HELP ignite_18b4aac2_clustermetricsmxbeanimpl_heap_bytes Ignite cluster amount of heap memory in bytes
# TYPE ignite_18b4aac2_clustermetricsmxbeanimpl_heap_bytes gauge
ignite_18b4aac2_clustermetricsmxbeanimpl_heap_bytes{attr="HeapMemoryMaximum",} 1.029177344E9
ignite_18b4aac2_clustermetricsmxbeanimpl_heap_bytes{attr="HeapMemoryInitialized",} 1.073741824E9
ignite_18b4aac2_clustermetricsmxbeanimpl_heap_bytes{attr="HeapMemoryUsed",} 2.49779008E8
ignite_18b4aac2_clustermetricsmxbeanimpl_heap_bytes{attr="HeapMemoryCommitted",} 1.029177344E9
ignite_18b4aac2_clustermetricsmxbeanimpl_heap_bytes{attr="HeapMemoryTotal",} 1.029177344E9
# HELP ignite_18b4aac2_dpl_mem_plc_pages_bytes Ignite durable memory pages size metrics in bytes
# TYPE ignite_18b4aac2_dpl_mem_plc_pages_bytes gauge
ignite_18b4aac2_dpl_mem_plc_pages_bytes{attr="MaxSize",} 600.0
ignite_18b4aac2_dpl_mem_plc_pages_bytes{attr="OffheapUsedSize",} 0.0
ignite_18b4aac2_dpl_mem_plc_pages_bytes{attr="OffHeapSize",} 0.0
ignite_18b4aac2_dpl_mem_plc_pages_bytes{attr="PhysicalMemorySize",} 0.0
ignite_18b4aac2_dpl_mem_plc_pages_bytes{attr="TotalAllocatedSize",} 0.0
# HELP ignite_18b4aac2_dpl_mem_plc_pages Ignite durable memory pages currently loaded in RAM
# TYPE ignite_18b4aac2_dpl_mem_plc_pages gauge
ignite_18b4aac2_dpl_mem_plc_pages{attr="DirtyPages",} 0.0
ignite_18b4aac2_dpl_mem_plc_pages{attr="PhysicalMemoryPages",} 0.0
ignite_18b4aac2_dpl_mem_plc_pages{attr="TotalAllocatedPages",} 0.0
ignite_18b4aac2_dpl_mem_plc_pages{attr="AllocationRate",} 0.0
ignite_18b4aac2_dpl_mem_plc_pages{attr="CheckpointBufferPages",} 0.0
# HELP ignite_18b4aac2_datastoragemetrics_pds_wal Ignite WAL metrics
# TYPE ignite_18b4aac2_datastoragemetrics_pds_wal untyped
ignite_18b4aac2_datastoragemetrics_pds_wal{attr="WalFsyncTimeAverage",} 0.0
ignite_18b4aac2_datastoragemetrics_pds_wal{attr="WalWritingRate",} 0.0
ignite_18b4aac2_datastoragemetrics_pds_wal{attr="WalLoggingRate",} 0.0
ignite_18b4aac2_datastoragemetrics_pds_wal{attr="WalArchiveSegments",} 0.0
ignite_18b4aac2_datastoragemetrics_pds_wal{attr="WalLastRollOverTime",} 0.0
ignite_18b4aac2_datastoragemetrics_pds_wal{attr="WalBuffPollSpinsRate",} 0.0
ignite_18b4aac2_datastoragemetrics_pds_wal{attr="WalTotalSize",} 6.7108864E7
# HELP ignite_18b4aac2_cacheclustermetricsmxbeanimpl_test_cache_rebalance_left Ignite cache "test-cache" rebalancing data left
# TYPE ignite_18b4aac2_cacheclustermetricsmxbeanimpl_test_cache_rebalance_left gauge
ignite_18b4aac2_cacheclustermetricsmxbeanimpl_test_cache_rebalance_left{attr="RebalanceClearingPartitionsLeft",cache="\"test-cache\"",} 0.0
ignite_18b4aac2_cacheclustermetricsmxbeanimpl_test_cache_rebalance_left{attr="KeysToRebalanceLeft",cache="\"test-cache\"",} 0.0
# HELP ignite_18b4aac2_sysmemplc_pages_bytes Ignite durable memory pages size metrics in bytes
# TYPE ignite_18b4aac2_sysmemplc_pages_bytes gauge
ignite_18b4aac2_sysmemplc_pages_bytes{attr="MaxSize",} 100.0
ignite_18b4aac2_sysmemplc_pages_bytes{attr="OffheapUsedSize",} 0.0
ignite_18b4aac2_sysmemplc_pages_bytes{attr="OffHeapSize",} 0.0
ignite_18b4aac2_sysmemplc_pages_bytes{attr="PhysicalMemorySize",} 0.0
ignite_18b4aac2_sysmemplc_pages_bytes{attr="TotalAllocatedSize",} 0.0
# HELP ignite_18b4aac2_metastorememplc_pages_bytes Ignite durable memory pages size metrics in bytes
# TYPE ignite_18b4aac2_metastorememplc_pages_bytes gauge
ignite_18b4aac2_metastorememplc_pages_bytes{attr="MaxSize",} 100.0
ignite_18b4aac2_metastorememplc_pages_bytes{attr="OffheapUsedSize",} 0.0
ignite_18b4aac2_metastorememplc_pages_bytes{attr="OffHeapSize",} 0.0
ignite_18b4aac2_metastorememplc_pages_bytes{attr="PhysicalMemorySize",} 0.0
ignite_18b4aac2_metastorememplc_pages_bytes{attr="TotalAllocatedSize",} 0.0
# HELP ignite_18b4aac2_clustermetricsmxbeanimpl_job Ignite cluster job detalization
# TYPE ignite_18b4aac2_clustermetricsmxbeanimpl_job untyped
ignite_18b4aac2_clustermetricsmxbeanimpl_job{attr="TotalRejectedJobs",} 0.0
ignite_18b4aac2_clustermetricsmxbeanimpl_job{attr="AverageWaitingJobs",} 0.0
ignite_18b4aac2_clustermetricsmxbeanimpl_job{attr="TotalExecutedJobs",} 0.0
ignite_18b4aac2_clustermetricsmxbeanimpl_job{attr="MaximumActiveJobs",} 0.0
ignite_18b4aac2_clustermetricsmxbeanimpl_job{attr="MaximumWaitingJobs",} 0.0
ignite_18b4aac2_clustermetricsmxbeanimpl_job{attr="AverageCancelledJobs",} 0.0
ignite_18b4aac2_clustermetricsmxbeanimpl_job{attr="MaximumRejectedJobs",} 0.0
ignite_18b4aac2_clustermetricsmxbeanimpl_job{attr="CurrentRejectedJobs",} 0.0
ignite_18b4aac2_clustermetricsmxbeanimpl_job{attr="CurrentActiveJobs",} 0.0
ignite_18b4aac2_clustermetricsmxbeanimpl_job{attr="CurrentWaitingJobs",} 0.0
ignite_18b4aac2_clustermetricsmxbeanimpl_job{attr="AverageRejectedJobs",} 0.0
ignite_18b4aac2_clustermetricsmxbeanimpl_job{attr="TotalCancelledJobs",} 0.0
ignite_18b4aac2_clustermetricsmxbeanimpl_job{attr="CurrentCancelledJobs",} 0.0
ignite_18b4aac2_clustermetricsmxbeanimpl_job{attr="AverageActiveJobs",} 0.0
ignite_18b4aac2_clustermetricsmxbeanimpl_job{attr="MaximumCancelledJobs",} 0.0
# HELP ignite_18b4aac2_datastoragemetrics_pds_checkpoint Ignite persistence last checkpoint metrics
# TYPE ignite_18b4aac2_datastoragemetrics_pds_checkpoint gauge
ignite_18b4aac2_datastoragemetrics_pds_checkpoint{attr="LastCheckpointPagesWriteDuration",} 0.0
ignite_18b4aac2_datastoragemetrics_pds_checkpoint{attr="LastCheckpointCopiedOnWritePagesNumber",} 0.0
ignite_18b4aac2_datastoragemetrics_pds_checkpoint{attr="LastCheckpointLockWaitDuration",} 0.0
ignite_18b4aac2_datastoragemetrics_pds_checkpoint{attr="LastCheckpointTotalPagesNumber",} 0.0
ignite_18b4aac2_datastoragemetrics_pds_checkpoint{attr="LastCheckpointFsyncDuration",} 0.0
ignite_18b4aac2_datastoragemetrics_pds_checkpoint{attr="LastCheckpointMarkDuration",} 0.0
ignite_18b4aac2_datastoragemetrics_pds_checkpoint{attr="LastCheckpointDataPagesNumber",} 0.0
ignite_18b4aac2_datastoragemetrics_pds_checkpoint{attr="LastCheckpointDuration",} 0.0
# HELP ignite_18b4aac2_clustermetricsmxbeanimpl_nonheap_bytes Ignite cluster amount of non-heap memory in bytes
# TYPE ignite_18b4aac2_clustermetricsmxbeanimpl_nonheap_bytes gauge
ignite_18b4aac2_clustermetricsmxbeanimpl_nonheap_bytes{attr="NonHeapMemoryInitialized",} 2555904.0
ignite_18b4aac2_clustermetricsmxbeanimpl_nonheap_bytes{attr="NonHeapMemoryCommitted",} 6.5888256E7
ignite_18b4aac2_clustermetricsmxbeanimpl_nonheap_bytes{attr="NonHeapMemoryMaximum",} 1.59383552E9
ignite_18b4aac2_clustermetricsmxbeanimpl_nonheap_bytes{attr="NonHeapMemoryUsed",} 6.3833256E7
ignite_18b4aac2_clustermetricsmxbeanimpl_nonheap_bytes{attr="NonHeapMemoryTotal",} 1.59383552E9
# HELP ignite_18b4aac2_sysmemplc_pages Ignite durable memory pages currently loaded in RAM
# TYPE ignite_18b4aac2_sysmemplc_pages gauge
ignite_18b4aac2_sysmemplc_pages{attr="DirtyPages",} 0.0
ignite_18b4aac2_sysmemplc_pages{attr="PhysicalMemoryPages",} 0.0
ignite_18b4aac2_sysmemplc_pages{attr="TotalAllocatedPages",} 0.0
ignite_18b4aac2_sysmemplc_pages{attr="AllocationRate",} 0.0
ignite_18b4aac2_sysmemplc_pages{attr="CheckpointBufferPages",} 0.0

Mmuzaf force-pushed the add-ignite-examples branch from c2b0681 to f74d309 Compare

September 13, 2018 16:51

Mmuzaf added 9 commits

September 17, 2018 18:45


          Add Apache Ingnite basic JMX metrics configuration

e45c5b2

Signed-off-by: Maxim Muzafarov <maxmuzaf@gmail.com>


          Add Apache Ingnite basic JMX metrics configuration

7f63d60

Signed-off-by: Maxim Muzafarov <maxmuzaf@gmail.com>


          Update metrics after review

743003d

Signed-off-by: Maxim Muzafarov <maxmuzaf@gmail.com>


          Update metrics after review 2

2d57c67

Signed-off-by: Maxim Muzafarov <maxmuzaf@gmail.com>


          Update metrics after review 3

3b94b8f

Signed-off-by: Maxim Muzafarov <maxmuzaf@gmail.com>


          Update metrics after review 4

665e0cf

Signed-off-by: Maxim Muzafarov <maxmuzaf@gmail.com>


          Update metrics after review 5

464634b

Signed-off-by: Maxim Muzafarov <maxmuzaf@gmail.com>


          Update metrics after review 6

3f5789e

Signed-off-by: Maxim Muzafarov <maxmuzaf@gmail.com>


          Update metrics after review 7

6cc565e

Signed-off-by: Maxim Muzafarov <maxmuzaf@gmail.com>

Mmuzaf force-pushed the add-ignite-examples branch from a5cf8b2 to 6cc565e Compare

September 17, 2018 15:45

Author

Mmuzaf commented Sep 17, 2018

@brian-brazil
I've updated metrics according to Metric and label naming and Instrumentation articles. Also, I've updated the metric description (some of them disabled by default).

Contributor

brian-brazil commented Nov 14, 2019

clsLdr is still in there, did you forget to push?

Author

Mmuzaf commented Nov 15, 2019

@brian-brazil yes, I'll update PR shortly.

patsevanton commented Apr 27, 2022

any news?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment