From e18ce574a0741a18425c2afe33312372f757f2f2 Mon Sep 17 00:00:00 2001 From: momo-jun Date: Thu, 4 Aug 2022 21:41:44 +0800 Subject: [PATCH] Merge 2.8.1-specific docs from #11509, #11512, #11739, #11810, and #11563 --- .../version-2.8.x/client-libraries-cpp.md | 7 ++----- .../cookbooks-retention-expiry.md | 13 +++++++++++-- .../version-2.8.x/functions-cli.md | 18 +++++++++--------- .../version-2.8.x/reference-configuration.md | 1 + .../version-2.8.x/reference-metrics.md | 9 +++++++++ .../version-2.8.x/reference-pulsar-admin.md | 8 +++++--- 6 files changed, 37 insertions(+), 19 deletions(-) diff --git a/site2/website/versioned_docs/version-2.8.x/client-libraries-cpp.md b/site2/website/versioned_docs/version-2.8.x/client-libraries-cpp.md index deebe13bada6d..858345f0bcf38 100644 --- a/site2/website/versioned_docs/version-2.8.x/client-libraries-cpp.md +++ b/site2/website/versioned_docs/version-2.8.x/client-libraries-cpp.md @@ -235,11 +235,8 @@ $ brew install openssl $ export OPENSSL_INCLUDE_DIR=/usr/local/opt/openssl/include/ $ export OPENSSL_ROOT_DIR=/usr/local/opt/openssl/ -# Protocol Buffers installation -$ brew tap homebrew/versions -$ brew install protobuf260 -$ brew install boost -$ brew install log4cxx +$ brew install protobuf boost boost-python log4cxx +# If you are using python3, you need to install boost-python3 # Google Test installation $ git clone https://github.com/google/googletest.git diff --git a/site2/website/versioned_docs/version-2.8.x/cookbooks-retention-expiry.md b/site2/website/versioned_docs/version-2.8.x/cookbooks-retention-expiry.md index af4be6bb2d9c6..81fc4f342be33 100644 --- a/site2/website/versioned_docs/version-2.8.x/cookbooks-retention-expiry.md +++ b/site2/website/versioned_docs/version-2.8.x/cookbooks-retention-expiry.md @@ -216,7 +216,7 @@ You can set a size and/or time threshold and backlog retention policy for all of #### pulsar-admin -Use the [`set-backlog-quota`](reference-pulsar-admin.md#namespaces) subcommand and specify a namespace, a size limit using the `-l`/`--limit` flag, and a retention policy using the `-p`/`--policy` flag. +Use the [`set-backlog-quota`](reference-pulsar-admin.md#namespaces) subcommand and specify a namespace, a size limit using the `-l`/`--limit` flag and the `-lt`/`--limitTime` flag (for 2.8.1 and later versions only) to limit backlog, a retention policy using the `-p`/`--policy` flag, and a policy type using `-t`/`--type` (for 2.8.1 and later versions only, defaults to `destination_storage`). ##### Example @@ -224,11 +224,20 @@ Use the [`set-backlog-quota`](reference-pulsar-admin.md#namespaces) subcommand a $ pulsar-admin namespaces set-backlog-quota my-tenant/my-ns \ --limit 2G \ - --limitTime 36000 \ --policy producer_request_hold ``` +```shell + +$ pulsar-admin namespaces set-backlog-quota my-tenant/my-ns/my-topic \ + --limitTime 3600 \ + --policy producer_request_hold \ + --type message_age + +``` + + #### REST API {@inject: endpoint|POST|/admin/v2/namespaces/:tenant/:namespace/backlogQuota|operation/getBacklogQuotaMap?version=@pulsar:version_number@} diff --git a/site2/website/versioned_docs/version-2.8.x/functions-cli.md b/site2/website/versioned_docs/version-2.8.x/functions-cli.md index ba7b7fbb9561e..dda6590c2d348 100644 --- a/site2/website/versioned_docs/version-2.8.x/functions-cli.md +++ b/site2/website/versioned_docs/version-2.8.x/functions-cli.md @@ -25,10 +25,10 @@ dead-letter-topic | The topic where all messages that were not processed success disk | The disk in bytes that need to be allocated per function instance (applicable only to docker runtime). | | fqfn | The Fully Qualified Function Name (FQFN) for the function. | | function-config-file | The path to a YAML config file specifying the configuration of a Pulsar Function. | | -go | Path to the main Go executable binary for the function (if the function is written in Go). | | +go | Path to the main Go executable binary for the function (if the function is written in Go). Since 2.8.1 and later versions, it also supports URL path [http/https/file (file protocol assumes that file already exists on worker host)/function (package URL from packages management service)] where worker can download the package from.| | hostname-verification-enabled | Enable hostname verification. | false inputs | The input topic or topics of a Pulsar Function (multiple topics can be specified as a comma-separated list). | | -jar | Path to the jar file for the function (if the function is written in Java). It also supports URL-path [http/https/file (file protocol assumes that file already exists on worker host)] from which worker can download the package. | | +jar | Path to the jar file for the function (if the function is written in Java). It also supports URL-path [http/https/file (file protocol assumes that file already exists on worker host)/function (package URL from packages management service, only available in 2.8.1 and later versions)] where worker can download the package from. | | instance-id-offset | Start the instanceIds from this offset. | 0 log-topic | The topic to which the logs a Pulsar Function are produced. | | max-message-retries | How many times should we try to process a message before giving up. | | @@ -38,7 +38,7 @@ output | The output topic of a Pulsar Function (If none is specified, no output output-serde-classname | The SerDe class to be used for messages output by the function. | | parallelism | The parallelism factor of a Pulsar Function (i.e. the number of function instances to run). | | processing-guarantees | The processing guarantees (delivery semantics) applied to the function. Available values: [ATLEAST_ONCE, ATMOST_ONCE, EFFECTIVELY_ONCE]. | ATLEAST_ONCE -py | Path to the main Python file/Python Wheel file for the function (if the function is written in Python). | | +py | Path to the main Python file/Python Wheel file for the function (if the function is written in Python). Since 2.8.1 and later versions, it also supports URL path [http/https/file (file protocol assumes that file already exists on worker host)/function (package URL from packages management service)] where worker can download the package from. | | ram | The ram in bytes that need to be allocated per function instance (applicable only to process/docker runtime). | | retain-ordering | Function consumes and processes messages in order. | | schema-type | The builtin schema type or custom schema class name to be used for messages output by the function. | | @@ -72,9 +72,9 @@ dead-letter-topic | The topic where all messages that were not processed success disk | The disk in bytes that need to be allocated per function instance (applicable only to docker runtime). | | fqfn | The Fully Qualified Function Name (FQFN) for the function. | | function-config-file | The path to a YAML config file specifying the configuration of a Pulsar Function. | | -go | Path to the main Go executable binary for the function (if the function is written in Go). | | +go | Path to the main Go executable binary for the function (if the function is written in Go). Since 2.8.1 and later versions, it also supports URL path [http/https/file (file protocol assumes that file already exists on worker host)/function (package URL from packages management service)] where worker can download the package from.| | inputs | The input topic or topics of a Pulsar Function (multiple topics can be specified as a comma-separated list). | | -jar | Path to the jar file for the function (if the function is written in Java). It also supports URL-path [http/https/file (file protocol assumes that file already exists on worker host)] from which worker can download the package. | | +jar | Path to the jar file for the function (if the function is written in Java). It also supports URL-path [http/https/file (file protocol assumes that file already exists on worker host)/function (package URL from packages management service, only available in 2.8.1 and later versions)] where worker can download the package from. | | log-topic | The topic to which the logs of a Pulsar Function are produced. | | max-message-retries | How many times should we try to process a message before giving up. | | name | The name of a Pulsar Function. | | @@ -83,7 +83,7 @@ output | The output topic of a Pulsar Function (If none is specified, no output output-serde-classname | The SerDe class to be used for messages output by the function. | | parallelism | The parallelism factor of a Pulsar Function (i.e. the number of function instances to run). | | processing-guarantees | The processing guarantees (delivery semantics) applied to the function. Available values: [ATLEAST_ONCE, ATMOST_ONCE, EFFECTIVELY_ONCE]. | ATLEAST_ONCE -py | Path to the main Python file/Python Wheel file for the function (if the function is written in Python). | | +py | Path to the main Python file/Python Wheel file for the function (if the function is written in Python). Since 2.8.1 and later versions, it also supports URL path [http/https/file (file protocol assumes that file already exists on worker host)/function (package URL from packages management service)] where worker can download the package from.| | ram | The ram in bytes that need to be allocated per function instance (applicable only to process/docker runtime). | | retain-ordering | Function consumes and processes messages in order. | | schema-type | The builtin schema type or custom schema class name to be used for messages output by the function. | | @@ -124,9 +124,9 @@ dead-letter-topic | The topic where all messages that were not processed success disk | The disk in bytes that need to be allocated per function instance (applicable only to docker runtime). | | fqfn | The Fully Qualified Function Name (FQFN) for the function. | | function-config-file | The path to a YAML config file specifying the configuration of a Pulsar Function. | | -go | Path to the main Go executable binary for the function (if the function is written in Go). | | +go | Path to the main Go executable binary for the function (if the function is written in Go). Since 2.8.1 and later versions, it also supports URL path [http/https/file (file protocol assumes that file already exists on worker host)/function (package URL from packages management service)] where worker can download the package from. | | inputs | The input topic or topics of a Pulsar Function (multiple topics can be specified as a comma-separated list). | | -jar | Path to the jar file for the function (if the function is written in Java). It also supports URL-path [http/https/file (file protocol assumes that file already exists on worker host)] from which worker can download the package. | | +jar | Path to the jar file for the function (if the function is written in Java). It also supports URL path [http/https/file (file protocol assumes that file already exists on worker host)/function (package URL from packages management service, only available in 2.8.1 and later versions)] where worker can download the package from. | | log-topic | The topic to which the logs of a Pulsar Function are produced. | | max-message-retries | How many times should we try to process a message before giving up. | | name | The name of a Pulsar Function. | | @@ -135,7 +135,7 @@ output | The output topic of a Pulsar Function (If none is specified, no output output-serde-classname | The SerDe class to be used for messages output by the function. | | parallelism | The parallelism factor of a Pulsar Function (i.e. the number of function instances to run). | | processing-guarantees | The processing guarantees (delivery semantics) applied to the function. Available values: [ATLEAST_ONCE, ATMOST_ONCE, EFFECTIVELY_ONCE]. | ATLEAST_ONCE -py | Path to the main Python file/Python Wheel file for the function (if the function is written in Python). | | +py | Path to the main Python file/Python Wheel file for the function (if the function is written in Python). Since 2.8.1 and later versions, it also supports URL path [http/https/file (file protocol assumes that file already exists on worker host)/function (package URL from packages management service)] where worker can download the package from.| | ram | The ram in bytes that need to be allocated per function instance (applicable only to process/docker runtime). | | retain-ordering | Function consumes and processes messages in order. | | schema-type | The builtin schema type or custom schema class name to be used for messages output by the function. | | diff --git a/site2/website/versioned_docs/version-2.8.x/reference-configuration.md b/site2/website/versioned_docs/version-2.8.x/reference-configuration.md index ae79fc83ddda0..86511e36c8c37 100644 --- a/site2/website/versioned_docs/version-2.8.x/reference-configuration.md +++ b/site2/website/versioned_docs/version-2.8.x/reference-configuration.md @@ -356,6 +356,7 @@ brokerServiceCompactionThresholdInBytes|If the estimated backlog size is greater | maxTopicsPerNamespace | The maximum number of persistent topics that can be created in the namespace. When the number of topics reaches this threshold, the broker rejects the request of creating a new topic, including the auto-created topics by the producer or consumer, until the number of connected consumers decreases. The default value 0 disables the check. | 0 | |subscriptionTypesEnabled| Enable all subscription types, which are exclusive, shared, failover, and key_shared. | Exclusive, Shared, Failover, Key_Shared | |narExtractionDirectory | The extraction directory of the nar package.
Available for Protocol Handler, Additional Servlets, Offloaders, Broker Interceptor. | System.getProperty("java.io.tmpdir") | +| managedLedgerInfoCompressionType | Compression type of `ManagedLedgerInfo`.
**Note:** This configuration is only available in 2.8.1 and later versions.
Available values are `NONE`, `LZ4`, `ZLIB`, `ZSTD`, and `SNAPPY`.
If the value is `NONE` or invalid, the `managedLedgerInfo` is not compressed.
**Note** that after enabling this configuration, if you want to degrade broker, you need to change the value to `NONE` and ensure all ledger metadata are saved without compression before starting to degrade. | NONE | ## Client diff --git a/site2/website/versioned_docs/version-2.8.x/reference-metrics.md b/site2/website/versioned_docs/version-2.8.x/reference-metrics.md index 24a67d4b81da2..c3ffc2fabecb8 100644 --- a/site2/website/versioned_docs/version-2.8.x/reference-metrics.md +++ b/site2/website/versioned_docs/version-2.8.x/reference-metrics.md @@ -181,6 +181,15 @@ All the topic metrics are labelled with the following labels: | pulsar_in_messages_total | Counter | The total number of messages received for this topic | | pulsar_out_bytes_total | Counter | The total number of bytes read from this topic | | pulsar_out_messages_total | Counter | The total number of messages read from this topic | +| pulsar_compaction_removed_event_count | Gauge | The removed event count of compaction.
**Note:** This metric is only available in 2.8.1 and later versions. | +| pulsar_compaction_succeed_count | Gauge | The succeed count of compaction.
**Note:** This metric is only available in 2.8.1 and later versions. | +| pulsar_compaction_failed_count | Gauge | The failed count of compaction.
**Note:** This metric is only available in 2.8.1 and later versions. | +| pulsar_compaction_duration_time_in_mills | Gauge | The duration time of compaction.
**Note:** This metric is only available in 2.8.1 and later versions. | +| pulsar_compaction_read_throughput | Gauge | The read throughput of compaction.
**Note:** This metric is only available in 2.8.1 and later versions. | +| pulsar_compaction_write_throughput | Gauge | The write throughput of compaction.
**Note:** This metric is only available in 2.8.1 and later versions. | +| pulsar_compaction_latency_le_* | Histogram | The compaction latency with given quantile.
**Note:** This metric is only available in 2.8.1 and later versions.
Available thresholds:
| +| pulsar_compaction_compacted_entries_count | Gauge | The compacted entries count.
**Note:** This metric is only available in 2.8.1 and later versions. | +| pulsar_compaction_compacted_entries_size |Gauge | The compacted entries size.
**Note:** This metric is only available in 2.8.1 and later versions. | #### Replication metrics diff --git a/site2/website/versioned_docs/version-2.8.x/reference-pulsar-admin.md b/site2/website/versioned_docs/version-2.8.x/reference-pulsar-admin.md index e41afd6f11bbf..832a03e71afbd 100644 --- a/site2/website/versioned_docs/version-2.8.x/reference-pulsar-admin.md +++ b/site2/website/versioned_docs/version-2.8.x/reference-pulsar-admin.md @@ -5,9 +5,11 @@ sidebar_label: "Pulsar Admin CLI" original_id: reference-pulsar-admin --- -> **Important** -> -> This page is deprecated and not updated anymore. For the latest and complete information about `pulsar-admin`, including commands, flags, descriptions, and more, see [pulsar-admin doc](https://pulsar.apache.org/tools/pulsar-admin/). +:::tip + +This page is deprecated and not updated anymore. For the latest and complete information about `pulsar-admin`, including commands, flags, descriptions, and more, see [pulsar-admin doc](https://pulsar.apache.org/tools/pulsar-admin/). + +::: The `pulsar-admin` tool enables you to manage Pulsar installations, including clusters, brokers, namespaces, tenants, and more.