大数据系列之:安装pulsar详细步骤
- 一、Pulsar版本和jdk对应关系
- 二、安装JDK
- 三、设置和激活jdk环境变量
- 四、下载和解压Pulsar
- 五、查看Pulsar目录
- 六、启动Pulsar standalone cluster
- 七、创建Kafka Topic
- 八、往Topic写入数据
- 九、消费pulsar的Topic
一、Pulsar版本和jdk对应关系
点击下面的链接查看Pulsar版本和jdk对应关系:
- https://github.com/apache/pulsar/blob/master/README.md#pulsar-runtime-java-version-recommendation
二、安装JDK
点击链接下载JDK:
- https://adoptium.net/temurin/releases/?variant=openjdk17
tar -zxvf OpenJDK17U-jdk_x64_linux_hotspot_17.0.6_10.tar.gz
三、设置和激活jdk环境变量
设置环境变量
vim /etc/profile
export JAVA_HOME="/data/src/jdk17"
PATH=$PATH:$JAVA_HOME/bin
激活环境变量
source /etc/profile
四、下载和解压Pulsar
下载Pulsar
wget https://archive.apache.org/dist/pulsar/pulsar-2.11.0/apache-pulsar-2.11.0-bin.tar.gz
解压Pulsar
tar -zxvf apache-pulsar-2.11.0-bin.tar.gz
五、查看Pulsar目录
cd apache-pulsar-2.11.0
ls
bin conf data examples instances lib LICENSE logs NOTICE packages-storage README
六、启动Pulsar standalone cluster
bin/pulsar standalone
2023-02-10T15:19:17,880+0800 [main] INFO org.apache.pulsar.functions.worker.FunctionRuntimeManager - Initializing secrets provider configurator org.apache.pulsar.functions.secretsproviderconfigurator.DefaultSecretsProviderConfigurator with configs: null
2023-02-10T15:19:17,885+0800 [pulsar-web-48-6] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [10/Feb/2023:15:19:17 +0800] "GET /admin/v2/persistent/public/functions/coordinate/subscriptions HTTP/1.1" 200 16 "-" "Pulsar-Java-v2.11.0" 19
2023-02-10T15:19:17,888+0800 [pulsar-io-19-3] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://public/functions/__change_events][reader-6bed90fa36] Successfully getLastMessageId -1:-1
2023-02-10T15:19:17,890+0800 [main] INFO org.apache.pulsar.functions.runtime.process.ProcessRuntimeFactory - Java instance jar location is not defined, using the location defined in system environment : /data/src/apache-pulsar-2.11.0/instances/java-instance.jar
2023-02-10T15:19:17,890+0800 [main] INFO org.apache.pulsar.functions.runtime.process.ProcessRuntimeFactory - Python instance file location is not defined using the location defined in system environment : /data/src/apache-pulsar-2.11.0/instances/python-instance/python_instance_main.py
2023-02-10T15:19:17,891+0800 [main] INFO org.apache.pulsar.functions.runtime.process.ProcessRuntimeFactory - Extra dependencies location is not defined using the location defined in system environment : /data/src/apache-pulsar-2.11.0/instances/deps
2023-02-10T15:19:17,894+0800 [main] INFO org.apache.pulsar.functions.worker.PulsarWorkerService - /** Start Leader Service **/
2023-02-10T15:19:17,898+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConnectionPool - [[id: 0xa5a16e3a, L:/127.0.0.1:10729 - R:localhost/127.0.0.1:6650]] Connected to server
2023-02-10T15:19:17,899+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.ServerCnx - New connection from /127.0.0.1:10729
2023-02-10T15:19:17,915+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerStatsRecorderImpl - Starting Pulsar consumer status recorder with config: {"topicNames":["persistent://public/functions/coordinate"],"topicsPattern":null,"subscriptionName":"participants","subscriptionType":"Failover","subscriptionProperties":null,"subscriptionMode":"Durable","receiverQueueSize":1000,"acknowledgementsGroupTimeMicros":100000,"negativeAckRedeliveryDelayMicros":60000000,"maxTotalReceiverQueueSizeAcrossPartitions":50000,"consumerName":"c-standalone-fw-localhost-8080:localhost:8080","ackTimeoutMillis":0,"tickDurationMillis":1000,"priorityLevel":0,"maxPendingChunkedMessage":10,"autoAckOldestChunkedMessageOnQueueFull":false,"expireTimeOfIncompleteChunkedMessageMillis":60000,"cryptoFailureAction":"FAIL","properties":{"id":"c-standalone-fw-localhost-8080:localhost:8080"},"readCompacted":false,"subscriptionInitialPosition":"Latest","patternAutoDiscoveryPeriod":60,"regexSubscriptionMode":"PersistentOnly","deadLetterPolicy":null,"retryEnable":false,"autoUpdatePartitions":true,"autoUpdatePartitionsIntervalSeconds":60,"replicateSubscriptionState":false,"resetIncludeHead":false,"keySharedPolicy":null,"batchIndexAckEnabled":false,"ackReceiptEnabled":false,"poolMessages":false,"startPaused":false,"autoScaledReceiverQueueSizeEnabled":false,"topicConfigurations":[],"maxPendingChuckedMessage":10}
2023-02-10T15:19:17,919+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerStatsRecorderImpl - Pulsar client config: {"serviceUrl":"pulsar://localhost:6650","authPluginClassName":null,"authParams":null,"authParamMap":null,"operationTimeoutMs":30000,"lookupTimeoutMs":30000,"statsIntervalSeconds":60,"numIoThreads":1,"numListenerThreads":1,"connectionsPerBroker":1,"connectionMaxIdleSeconds":180,"useTcpNoDelay":true,"useTls":false,"tlsKeyFilePath":null,"tlsCertificateFilePath":null,"tlsTrustCertsFilePath":null,"tlsAllowInsecureConnection":false,"tlsHostnameVerificationEnable":false,"concurrentLookupRequest":5000,"maxLookupRequest":50000,"maxLookupRedirects":20,"maxNumberOfRejectedRequestPerConnection":50,"keepAliveIntervalSeconds":30,"connectionTimeoutMs":10000,"requestTimeoutMs":60000,"initialBackoffIntervalNanos":100000000,"maxBackoffIntervalNanos":60000000000,"enableBusyWait":false,"listenerName":null,"useKeyStoreTls":false,"sslProvider":null,"tlsKeyStoreType":"JKS","tlsKeyStorePath":null,"tlsKeyStorePassword":null,"tlsTrustStoreType":"JKS","tlsTrustStorePath":null,"tlsTrustStorePassword":null,"tlsCiphers":[],"tlsProtocols":[],"memoryLimitBytes":0,"proxyServiceUrl":null,"proxyProtocol":null,"enableTransaction":false,"dnsLookupBindAddress":null,"dnsLookupBindPort":0,"socks5ProxyAddress":null,"socks5ProxyUsername":null,"socks5ProxyPassword":null}
2023-02-10T15:19:17,925+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://public/functions/coordinate][participants] Subscribing to topic on cnx [id: 0xa5a16e3a, L:/127.0.0.1:10729 - R:localhost/127.0.0.1:6650], consumerId 0
2023-02-10T15:19:17,926+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:10729] Subscribing on topic persistent://public/functions/coordinate / participants
2023-02-10T15:19:17,928+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:10729] Created subscription on topic persistent://public/functions/coordinate / participants
2023-02-10T15:19:17,928+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://public/functions/coordinate][participants] Subscribed to topic on localhost/127.0.0.1:6650 -- consumer: 0
2023-02-10T15:19:17,929+0800 [main] INFO org.apache.pulsar.functions.worker.PulsarWorkerService - /** Initializing Metadata Manager **/
2023-02-10T15:19:17,935+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerStatsRecorderImpl - Starting Pulsar consumer status recorder with config: {"topicNames":["persistent://public/functions/metadata"],"topicsPattern":null,"subscriptionName":"c-standalone-fw-localhost-8080-function-metadata-tailer-reader-c67f7ce6a3","subscriptionType":"Exclusive","subscriptionProperties":null,"subscriptionMode":"NonDurable","receiverQueueSize":1000,"acknowledgementsGroupTimeMicros":100000,"negativeAckRedeliveryDelayMicros":60000000,"maxTotalReceiverQueueSizeAcrossPartitions":50000,"consumerName":"c-standalone-fw-localhost-8080-function-metadata-tailer","ackTimeoutMillis":0,"tickDurationMillis":1000,"priorityLevel":0,"maxPendingChunkedMessage":10,"autoAckOldestChunkedMessageOnQueueFull":false,"expireTimeOfIncompleteChunkedMessageMillis":60000,"cryptoFailureAction":"FAIL","properties":{},"readCompacted":false,"subscriptionInitialPosition":"Latest","patternAutoDiscoveryPeriod":60,"regexSubscriptionMode":"PersistentOnly","deadLetterPolicy":null,"retryEnable":false,"autoUpdatePartitions":true,"autoUpdatePartitionsIntervalSeconds":60,"replicateSubscriptionState":false,"resetIncludeHead":false,"keySharedPolicy":null,"batchIndexAckEnabled":false,"ackReceiptEnabled":false,"poolMessages":false,"startPaused":false,"autoScaledReceiverQueueSizeEnabled":false,"topicConfigurations":[],"maxPendingChuckedMessage":10}
2023-02-10T15:19:17,939+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerStatsRecorderImpl - Pulsar client config: {"serviceUrl":"pulsar://localhost:6650","authPluginClassName":null,"authParams":null,"authParamMap":null,"operationTimeoutMs":30000,"lookupTimeoutMs":30000,"statsIntervalSeconds":60,"numIoThreads":1,"numListenerThreads":1,"connectionsPerBroker":1,"connectionMaxIdleSeconds":180,"useTcpNoDelay":true,"useTls":false,"tlsKeyFilePath":null,"tlsCertificateFilePath":null,"tlsTrustCertsFilePath":null,"tlsAllowInsecureConnection":false,"tlsHostnameVerificationEnable":false,"concurrentLookupRequest":5000,"maxLookupRequest":50000,"maxLookupRedirects":20,"maxNumberOfRejectedRequestPerConnection":50,"keepAliveIntervalSeconds":30,"connectionTimeoutMs":10000,"requestTimeoutMs":60000,"initialBackoffIntervalNanos":100000000,"maxBackoffIntervalNanos":60000000000,"enableBusyWait":false,"listenerName":null,"useKeyStoreTls":false,"sslProvider":null,"tlsKeyStoreType":"JKS","tlsKeyStorePath":null,"tlsKeyStorePassword":null,"tlsTrustStoreType":"JKS","tlsTrustStorePath":null,"tlsTrustStorePassword":null,"tlsCiphers":[],"tlsProtocols":[],"memoryLimitBytes":0,"proxyServiceUrl":null,"proxyProtocol":null,"enableTransaction":false,"dnsLookupBindAddress":null,"dnsLookupBindPort":0,"socks5ProxyAddress":null,"socks5ProxyUsername":null,"socks5ProxyPassword":null}
2023-02-10T15:19:17,941+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://public/functions/metadata][c-standalone-fw-localhost-8080-function-metadata-tailer-reader-c67f7ce6a3] Subscribing to topic on cnx [id: 0xa5a16e3a, L:/127.0.0.1:10729 - R:localhost/127.0.0.1:6650], consumerId 1
2023-02-10T15:19:17,942+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:10729] Subscribing on topic persistent://public/functions/metadata / c-standalone-fw-localhost-8080-function-metadata-tailer-reader-c67f7ce6a3
2023-02-10T15:19:17,942+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://public/functions/metadata][c-standalone-fw-localhost-8080-function-metadata-tailer-reader-c67f7ce6a3] Creating non-durable subscription at msg id -1:-1:-1:-1 - {}
2023-02-10T15:19:17,942+0800 [pulsar-io-19-3] INFO org.apache.bookkeeper.mledger.impl.NonDurableCursorImpl - [public/functions/persistent/metadata] Created non-durable cursor read-position=18:0 mark-delete-position=18:-1
2023-02-10T15:19:17,942+0800 [pulsar-io-19-3] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/functions/persistent/metadata] Opened new cursor: NonDurableCursorImpl{ledger=public/functions/persistent/metadata, ackPos=18:-1, readPos=18:0}
2023-02-10T15:19:17,943+0800 [pulsar-io-19-3] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/functions/persistent/metadata-c-standalone-fw-localhost-8080-function-metadata-tailer-reader-c67f7ce6a3] Rewind from 18:0 to 18:0
2023-02-10T15:19:17,943+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:10729] Created subscription on topic persistent://public/functions/metadata / c-standalone-fw-localhost-8080-function-metadata-tailer-reader-c67f7ce6a3
2023-02-10T15:19:17,943+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://public/functions/metadata][c-standalone-fw-localhost-8080-function-metadata-tailer-reader-c67f7ce6a3] Subscribed to topic on localhost/127.0.0.1:6650 -- consumer: 1
2023-02-10T15:19:17,944+0800 [main] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://public/functions/metadata][c-standalone-fw-localhost-8080-function-metadata-tailer-reader-c67f7ce6a3] Get topic last message Id
2023-02-10T15:19:17,945+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://public/functions/metadata][c-standalone-fw-localhost-8080-function-metadata-tailer-reader-c67f7ce6a3] Successfully getLastMessageId -1:-1
2023-02-10T15:19:17,949+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:10729] Closing consumer: consumerId=1
2023-02-10T15:19:17,949+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.AbstractDispatcherSingleActiveConsumer - Removing consumer Consumer{subscription=PersistentSubscription{topic=persistent://public/functions/metadata, name=c-standalone-fw-localhost-8080-function-metadata-tailer-reader-c67f7ce6a3}, consumerId=1, consumerName=c-standalone-fw-localhost-8080-function-metadata-tailer, address=/127.0.0.1:10729}
2023-02-10T15:19:17,950+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.persistent.PersistentSubscription - [persistent://public/functions/metadata][c-standalone-fw-localhost-8080-function-metadata-tailer-reader-c67f7ce6a3] Successfully closed subscription [NonDurableCursorImpl{ledger=public/functions/persistent/metadata, ackPos=18:-1, readPos=18:0}]
2023-02-10T15:19:17,951+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.persistent.PersistentSubscription - [persistent://public/functions/metadata][c-standalone-fw-localhost-8080-function-metadata-tailer-reader-c67f7ce6a3] Successfully closed dispatcher for reader
2023-02-10T15:19:17,952+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:10729] Closed consumer, consumerId=1
2023-02-10T15:19:17,952+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://public/functions/metadata] [c-standalone-fw-localhost-8080-function-metadata-tailer-reader-c67f7ce6a3] Closed consumer
2023-02-10T15:19:17,954+0800 [main] INFO org.apache.pulsar.functions.worker.FunctionMetaDataManager - FunctionMetaData Manager initialization complete
2023-02-10T15:19:17,954+0800 [main] INFO org.apache.pulsar.functions.worker.PulsarWorkerService - /** Initializing Runtime Manager **/
2023-02-10T15:19:17,961+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerStatsRecorderImpl - Starting Pulsar consumer status recorder with config: {"topicNames":["persistent://public/functions/assignments"],"topicsPattern":null,"subscriptionName":"c-standalone-fw-localhost-8080-function-assignment-initialize-reader-a8bc01f6ce","subscriptionType":"Exclusive","subscriptionProperties":null,"subscriptionMode":"NonDurable","receiverQueueSize":1000,"acknowledgementsGroupTimeMicros":100000,"negativeAckRedeliveryDelayMicros":60000000,"maxTotalReceiverQueueSizeAcrossPartitions":50000,"consumerName":"c-standalone-fw-localhost-8080-function-assignment-initialize","ackTimeoutMillis":0,"tickDurationMillis":1000,"priorityLevel":0,"maxPendingChunkedMessage":10,"autoAckOldestChunkedMessageOnQueueFull":false,"expireTimeOfIncompleteChunkedMessageMillis":60000,"cryptoFailureAction":"FAIL","properties":{},"readCompacted":true,"subscriptionInitialPosition":"Latest","patternAutoDiscoveryPeriod":60,"regexSubscriptionMode":"PersistentOnly","deadLetterPolicy":null,"retryEnable":false,"autoUpdatePartitions":true,"autoUpdatePartitionsIntervalSeconds":60,"replicateSubscriptionState":false,"resetIncludeHead":false,"keySharedPolicy":null,"batchIndexAckEnabled":false,"ackReceiptEnabled":false,"poolMessages":false,"startPaused":false,"autoScaledReceiverQueueSizeEnabled":false,"topicConfigurations":[],"maxPendingChuckedMessage":10}
2023-02-10T15:19:17,964+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerStatsRecorderImpl - Pulsar client config: {"serviceUrl":"pulsar://localhost:6650","authPluginClassName":null,"authParams":null,"authParamMap":null,"operationTimeoutMs":30000,"lookupTimeoutMs":30000,"statsIntervalSeconds":60,"numIoThreads":1,"numListenerThreads":1,"connectionsPerBroker":1,"connectionMaxIdleSeconds":180,"useTcpNoDelay":true,"useTls":false,"tlsKeyFilePath":null,"tlsCertificateFilePath":null,"tlsTrustCertsFilePath":null,"tlsAllowInsecureConnection":false,"tlsHostnameVerificationEnable":false,"concurrentLookupRequest":5000,"maxLookupRequest":50000,"maxLookupRedirects":20,"maxNumberOfRejectedRequestPerConnection":50,"keepAliveIntervalSeconds":30,"connectionTimeoutMs":10000,"requestTimeoutMs":60000,"initialBackoffIntervalNanos":100000000,"maxBackoffIntervalNanos":60000000000,"enableBusyWait":false,"listenerName":null,"useKeyStoreTls":false,"sslProvider":null,"tlsKeyStoreType":"JKS","tlsKeyStorePath":null,"tlsKeyStorePassword":null,"tlsTrustStoreType":"JKS","tlsTrustStorePath":null,"tlsTrustStorePassword":null,"tlsCiphers":[],"tlsProtocols":[],"memoryLimitBytes":0,"proxyServiceUrl":null,"proxyProtocol":null,"enableTransaction":false,"dnsLookupBindAddress":null,"dnsLookupBindPort":0,"socks5ProxyAddress":null,"socks5ProxyUsername":null,"socks5ProxyPassword":null}
2023-02-10T15:19:17,965+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://public/functions/assignments][c-standalone-fw-localhost-8080-function-assignment-initialize-reader-a8bc01f6ce] Subscribing to topic on cnx [id: 0xa5a16e3a, L:/127.0.0.1:10729 - R:localhost/127.0.0.1:6650], consumerId 2
2023-02-10T15:19:17,966+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:10729] Subscribing on topic persistent://public/functions/assignments / c-standalone-fw-localhost-8080-function-assignment-initialize-reader-a8bc01f6ce
2023-02-10T15:19:17,967+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://public/functions/assignments][c-standalone-fw-localhost-8080-function-assignment-initialize-reader-a8bc01f6ce] Creating non-durable subscription at msg id -1:-1:-1:-1 - {}
2023-02-10T15:19:17,967+0800 [pulsar-io-19-3] INFO org.apache.bookkeeper.mledger.impl.NonDurableCursorImpl - [public/functions/persistent/assignments] Created non-durable cursor read-position=15:0 mark-delete-position=15:-1
2023-02-10T15:19:17,967+0800 [pulsar-io-19-3] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/functions/persistent/assignments] Opened new cursor: NonDurableCursorImpl{ledger=public/functions/persistent/assignments, ackPos=15:-1, readPos=15:0}
2023-02-10T15:19:17,967+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:10729] Created subscription on topic persistent://public/functions/assignments / c-standalone-fw-localhost-8080-function-assignment-initialize-reader-a8bc01f6ce
2023-02-10T15:19:17,968+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://public/functions/assignments][c-standalone-fw-localhost-8080-function-assignment-initialize-reader-a8bc01f6ce] Subscribed to topic on localhost/127.0.0.1:6650 -- consumer: 2
2023-02-10T15:19:17,969+0800 [main] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://public/functions/assignments][c-standalone-fw-localhost-8080-function-assignment-initialize-reader-a8bc01f6ce] Get topic last message Id
2023-02-10T15:19:17,970+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://public/functions/assignments][c-standalone-fw-localhost-8080-function-assignment-initialize-reader-a8bc01f6ce] Successfully getLastMessageId -1:-1
2023-02-10T15:19:17,970+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:10729] Closing consumer: consumerId=2
2023-02-10T15:19:17,970+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.AbstractDispatcherSingleActiveConsumer - Removing consumer Consumer{subscription=PersistentSubscription{topic=persistent://public/functions/assignments, name=c-standalone-fw-localhost-8080-function-assignment-initialize-reader-a8bc01f6ce}, consumerId=2, consumerName=c-standalone-fw-localhost-8080-function-assignment-initialize, address=/127.0.0.1:10729}
2023-02-10T15:19:17,970+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.persistent.PersistentSubscription - [persistent://public/functions/assignments][c-standalone-fw-localhost-8080-function-assignment-initialize-reader-a8bc01f6ce] Successfully closed subscription [NonDurableCursorImpl{ledger=public/functions/persistent/assignments, ackPos=15:-1, readPos=15:0}]
2023-02-10T15:19:17,971+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.persistent.PersistentSubscription - [persistent://public/functions/assignments][c-standalone-fw-localhost-8080-function-assignment-initialize-reader-a8bc01f6ce] Successfully closed dispatcher for reader
2023-02-10T15:19:17,971+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:10729] Closed consumer, consumerId=2
2023-02-10T15:19:17,971+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://public/functions/assignments] [c-standalone-fw-localhost-8080-function-assignment-initialize-reader-a8bc01f6ce] Closed consumer
2023-02-10T15:19:17,972+0800 [main] INFO org.apache.pulsar.functions.worker.PulsarWorkerService - /** Starting Function Assignment Tailer **/
2023-02-10T15:19:17,972+0800 [main] INFO org.apache.pulsar.functions.worker.FunctionAssignmentTailer - Assignment tailer will start reading from message id -1:-1:-1
2023-02-10T15:19:17,977+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerStatsRecorderImpl - Starting Pulsar consumer status recorder with config: {"topicNames":["persistent://public/functions/assignments"],"topicsPattern":null,"subscriptionName":"c-standalone-fw-localhost-8080-function-assignment-tailer-reader-f1cd2d43c3","subscriptionType":"Exclusive","subscriptionProperties":null,"subscriptionMode":"NonDurable","receiverQueueSize":1000,"acknowledgementsGroupTimeMicros":100000,"negativeAckRedeliveryDelayMicros":60000000,"maxTotalReceiverQueueSizeAcrossPartitions":50000,"consumerName":"c-standalone-fw-localhost-8080-function-assignment-tailer","ackTimeoutMillis":0,"tickDurationMillis":1000,"priorityLevel":0,"maxPendingChunkedMessage":10,"autoAckOldestChunkedMessageOnQueueFull":false,"expireTimeOfIncompleteChunkedMessageMillis":60000,"cryptoFailureAction":"FAIL","properties":{},"readCompacted":true,"subscriptionInitialPosition":"Latest","patternAutoDiscoveryPeriod":60,"regexSubscriptionMode":"PersistentOnly","deadLetterPolicy":null,"retryEnable":false,"autoUpdatePartitions":true,"autoUpdatePartitionsIntervalSeconds":60,"replicateSubscriptionState":false,"resetIncludeHead":false,"keySharedPolicy":null,"batchIndexAckEnabled":false,"ackReceiptEnabled":false,"poolMessages":false,"startPaused":false,"autoScaledReceiverQueueSizeEnabled":false,"topicConfigurations":[],"maxPendingChuckedMessage":10}
2023-02-10T15:19:17,980+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerStatsRecorderImpl - Pulsar client config: {"serviceUrl":"pulsar://localhost:6650","authPluginClassName":null,"authParams":null,"authParamMap":null,"operationTimeoutMs":30000,"lookupTimeoutMs":30000,"statsIntervalSeconds":60,"numIoThreads":1,"numListenerThreads":1,"connectionsPerBroker":1,"connectionMaxIdleSeconds":180,"useTcpNoDelay":true,"useTls":false,"tlsKeyFilePath":null,"tlsCertificateFilePath":null,"tlsTrustCertsFilePath":null,"tlsAllowInsecureConnection":false,"tlsHostnameVerificationEnable":false,"concurrentLookupRequest":5000,"maxLookupRequest":50000,"maxLookupRedirects":20,"maxNumberOfRejectedRequestPerConnection":50,"keepAliveIntervalSeconds":30,"connectionTimeoutMs":10000,"requestTimeoutMs":60000,"initialBackoffIntervalNanos":100000000,"maxBackoffIntervalNanos":60000000000,"enableBusyWait":false,"listenerName":null,"useKeyStoreTls":false,"sslProvider":null,"tlsKeyStoreType":"JKS","tlsKeyStorePath":null,"tlsKeyStorePassword":null,"tlsTrustStoreType":"JKS","tlsTrustStorePath":null,"tlsTrustStorePassword":null,"tlsCiphers":[],"tlsProtocols":[],"memoryLimitBytes":0,"proxyServiceUrl":null,"proxyProtocol":null,"enableTransaction":false,"dnsLookupBindAddress":null,"dnsLookupBindPort":0,"socks5ProxyAddress":null,"socks5ProxyUsername":null,"socks5ProxyPassword":null}
2023-02-10T15:19:17,981+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://public/functions/assignments][c-standalone-fw-localhost-8080-function-assignment-tailer-reader-f1cd2d43c3] Subscribing to topic on cnx [id: 0xa5a16e3a, L:/127.0.0.1:10729 - R:localhost/127.0.0.1:6650], consumerId 3
2023-02-10T15:19:17,983+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:10729] Subscribing on topic persistent://public/functions/assignments / c-standalone-fw-localhost-8080-function-assignment-tailer-reader-f1cd2d43c3
2023-02-10T15:19:17,983+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://public/functions/assignments][c-standalone-fw-localhost-8080-function-assignment-tailer-reader-f1cd2d43c3] Creating non-durable subscription at msg id -1:-1:-1:-1 - {}
2023-02-10T15:19:17,983+0800 [pulsar-io-19-3] INFO org.apache.bookkeeper.mledger.impl.NonDurableCursorImpl - [public/functions/persistent/assignments] Created non-durable cursor read-position=15:0 mark-delete-position=15:-1
2023-02-10T15:19:17,983+0800 [pulsar-io-19-3] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/functions/persistent/assignments] Opened new cursor: NonDurableCursorImpl{ledger=public/functions/persistent/assignments, ackPos=15:-1, readPos=15:0}
2023-02-10T15:19:17,984+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:10729] Created subscription on topic persistent://public/functions/assignments / c-standalone-fw-localhost-8080-function-assignment-tailer-reader-f1cd2d43c3
2023-02-10T15:19:17,984+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://public/functions/assignments][c-standalone-fw-localhost-8080-function-assignment-tailer-reader-f1cd2d43c3] Subscribed to topic on localhost/127.0.0.1:6650 -- consumer: 3
2023-02-10T15:19:17,987+0800 [main] INFO org.apache.pulsar.functions.worker.PulsarWorkerService - /** Starting Metadata Manager **/
2023-02-10T15:19:17,992+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerStatsRecorderImpl - Starting Pulsar consumer status recorder with config: {"topicNames":["persistent://public/functions/metadata"],"topicsPattern":null,"subscriptionName":"c-standalone-fw-localhost-8080-function-metadata-tailer-reader-63c4334116","subscriptionType":"Exclusive","subscriptionProperties":null,"subscriptionMode":"NonDurable","receiverQueueSize":1000,"acknowledgementsGroupTimeMicros":100000,"negativeAckRedeliveryDelayMicros":60000000,"maxTotalReceiverQueueSizeAcrossPartitions":50000,"consumerName":"c-standalone-fw-localhost-8080-function-metadata-tailer","ackTimeoutMillis":0,"tickDurationMillis":1000,"priorityLevel":0,"maxPendingChunkedMessage":10,"autoAckOldestChunkedMessageOnQueueFull":false,"expireTimeOfIncompleteChunkedMessageMillis":60000,"cryptoFailureAction":"FAIL","properties":{},"readCompacted":false,"subscriptionInitialPosition":"Latest","patternAutoDiscoveryPeriod":60,"regexSubscriptionMode":"PersistentOnly","deadLetterPolicy":null,"retryEnable":false,"autoUpdatePartitions":true,"autoUpdatePartitionsIntervalSeconds":60,"replicateSubscriptionState":false,"resetIncludeHead":false,"keySharedPolicy":null,"batchIndexAckEnabled":false,"ackReceiptEnabled":false,"poolMessages":false,"startPaused":false,"autoScaledReceiverQueueSizeEnabled":false,"topicConfigurations":[],"maxPendingChuckedMessage":10}
2023-02-10T15:19:17,994+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerStatsRecorderImpl - Pulsar client config: {"serviceUrl":"pulsar://localhost:6650","authPluginClassName":null,"authParams":null,"authParamMap":null,"operationTimeoutMs":30000,"lookupTimeoutMs":30000,"statsIntervalSeconds":60,"numIoThreads":1,"numListenerThreads":1,"connectionsPerBroker":1,"connectionMaxIdleSeconds":180,"useTcpNoDelay":true,"useTls":false,"tlsKeyFilePath":null,"tlsCertificateFilePath":null,"tlsTrustCertsFilePath":null,"tlsAllowInsecureConnection":false,"tlsHostnameVerificationEnable":false,"concurrentLookupRequest":5000,"maxLookupRequest":50000,"maxLookupRedirects":20,"maxNumberOfRejectedRequestPerConnection":50,"keepAliveIntervalSeconds":30,"connectionTimeoutMs":10000,"requestTimeoutMs":60000,"initialBackoffIntervalNanos":100000000,"maxBackoffIntervalNanos":60000000000,"enableBusyWait":false,"listenerName":null,"useKeyStoreTls":false,"sslProvider":null,"tlsKeyStoreType":"JKS","tlsKeyStorePath":null,"tlsKeyStorePassword":null,"tlsTrustStoreType":"JKS","tlsTrustStorePath":null,"tlsTrustStorePassword":null,"tlsCiphers":[],"tlsProtocols":[],"memoryLimitBytes":0,"proxyServiceUrl":null,"proxyProtocol":null,"enableTransaction":false,"dnsLookupBindAddress":null,"dnsLookupBindPort":0,"socks5ProxyAddress":null,"socks5ProxyUsername":null,"socks5ProxyPassword":null}
2023-02-10T15:19:17,996+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://public/functions/metadata][c-standalone-fw-localhost-8080-function-metadata-tailer-reader-63c4334116] Subscribing to topic on cnx [id: 0xa5a16e3a, L:/127.0.0.1:10729 - R:localhost/127.0.0.1:6650], consumerId 4
2023-02-10T15:19:17,996+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:10729] Subscribing on topic persistent://public/functions/metadata / c-standalone-fw-localhost-8080-function-metadata-tailer-reader-63c4334116
2023-02-10T15:19:17,997+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://public/functions/metadata][c-standalone-fw-localhost-8080-function-metadata-tailer-reader-63c4334116] Creating non-durable subscription at msg id -1:-1:-1:-1 - {}
2023-02-10T15:19:17,997+0800 [pulsar-io-19-3] INFO org.apache.bookkeeper.mledger.impl.NonDurableCursorImpl - [public/functions/persistent/metadata] Created non-durable cursor read-position=18:0 mark-delete-position=18:-1
2023-02-10T15:19:17,997+0800 [pulsar-io-19-3] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/functions/persistent/metadata] Opened new cursor: NonDurableCursorImpl{ledger=public/functions/persistent/metadata, ackPos=18:-1, readPos=18:0}
2023-02-10T15:19:17,997+0800 [pulsar-io-19-3] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/functions/persistent/metadata-c-standalone-fw-localhost-8080-function-metadata-tailer-reader-63c4334116] Rewind from 18:0 to 18:0
2023-02-10T15:19:17,997+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:10729] Created subscription on topic persistent://public/functions/metadata / c-standalone-fw-localhost-8080-function-metadata-tailer-reader-63c4334116
2023-02-10T15:19:17,998+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://public/functions/metadata][c-standalone-fw-localhost-8080-function-metadata-tailer-reader-63c4334116] Subscribed to topic on localhost/127.0.0.1:6650 -- consumer: 4
2023-02-10T15:19:18,003+0800 [main] INFO org.apache.pulsar.functions.worker.FunctionMetaDataManager - MetaData Manager Tailer started
2023-02-10T15:19:18,007+0800 [main] INFO org.apache.pulsar.functions.worker.PulsarWorkerService - /** Starting Cluster Service Coordinator **/
2023-02-10T15:19:18,007+0800 [main] INFO org.apache.pulsar.functions.worker.ClusterServiceCoordinator - /** Starting cluster service coordinator **/
2023-02-10T15:19:18,011+0800 [main] INFO org.apache.pulsar.functions.worker.PulsarWorkerService - /** Started worker id=c-standalone-fw-localhost-8080 **/
2023-02-10T15:19:18,011+0800 [main] INFO org.apache.pulsar.broker.PulsarService - Function worker service started
2023-02-10T15:19:18,014+0800 [main] INFO org.apache.pulsar.packages.management.storage.filesystem.FileSystemPackagesStorage - Packages management filesystem storage initialized on packages-storage
2023-02-10T15:19:18,036+0800 [main] INFO org.apache.pulsar.broker.PulsarService - messaging service is ready, bootstrap_seconds=4
2023-02-10T15:19:18,036+0800 [main] INFO org.apache.pulsar.broker.PulsarService - messaging service is ready, bootstrap service port = 8080, broker url= pulsar://localhost:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@32c8d67
2023-02-10T15:19:18,042+0800 [main] INFO org.apache.pulsar.broker.PulsarService - created admin with url http://localhost:8080
2023-02-10T15:19:18,086+0800 [pulsar-web-48-7] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [10/Feb/2023:15:19:18 +0800] "GET /admin/v2/clusters HTTP/1.1" 200 14 "-" "Pulsar-Java-v2.11.0" 7
2023-02-10T15:19:18,094+0800 [pulsar-web-48-8] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [10/Feb/2023:15:19:18 +0800] "GET /admin/v2/tenants HTTP/1.1" 200 19 "-" "Pulsar-Java-v2.11.0" 4
2023-02-10T15:19:18,108+0800 [pulsar-web-48-5] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [10/Feb/2023:15:19:18 +0800] "GET /admin/v2/namespaces/public HTTP/1.1" 200 37 "-" "Pulsar-Java-v2.11.0" 12
2023-02-10T15:19:18,112+0800 [pulsar-web-48-1] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [10/Feb/2023:15:19:18 +0800] "GET /admin/v2/clusters HTTP/1.1" 200 14 "-" "Pulsar-Java-v2.11.0" 2
2023-02-10T15:19:18,119+0800 [pulsar-web-48-4] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [10/Feb/2023:15:19:18 +0800] "GET /admin/v2/tenants HTTP/1.1" 200 19 "-" "Pulsar-Java-v2.11.0" 2
2023-02-10T15:19:18,126+0800 [pulsar-web-48-1] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [10/Feb/2023:15:19:18 +0800] "GET /admin/v2/namespaces/pulsar HTTP/1.1" 200 17 "-" "Pulsar-Java-v2.11.0" 4
2023-02-10T15:19:18,928+0800 [pulsar-io-19-4] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/functions/persistent/coordinate-participants] Rewind from 17:0 to 17:0
2023-02-10T15:19:18,930+0800 [pulsar-external-listener-59-1] INFO org.apache.pulsar.functions.worker.LeaderService - Worker c-standalone-fw-localhost-8080:localhost:8080 became the leader.
2023-02-10T15:19:18,949+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - Starting Pulsar producer perf with config: {"topicName":"persistent://public/functions/assignments","producerName":"c-standalone-fw-localhost-8080-scheduler-manager","sendTimeoutMs":30000,"blockIfQueueFull":true,"maxPendingMessages":0,"maxPendingMessagesAcrossPartitions":0,"messageRoutingMode":"RoundRobinPartition","hashingScheme":"JavaStringHash","cryptoFailureAction":"FAIL","batchingMaxPublishDelayMicros":1000,"batchingPartitionSwitchFrequencyByPublishDelay":10,"batchingMaxMessages":1000,"batchingMaxBytes":131072,"batchingEnabled":false,"chunkingEnabled":false,"chunkMaxMessageSize":-1,"compressionType":"LZ4","initialSequenceId":null,"autoUpdatePartitions":true,"autoUpdatePartitionsIntervalSeconds":60,"multiSchema":true,"accessMode":"Exclusive","lazyStartPartitionedProducers":false,"properties":{},"initialSubscriptionName":null}
2023-02-10T15:19:18,951+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - Pulsar client config: {"serviceUrl":"pulsar://localhost:6650","authPluginClassName":null,"authParams":null,"authParamMap":null,"operationTimeoutMs":30000,"lookupTimeoutMs":30000,"statsIntervalSeconds":60,"numIoThreads":1,"numListenerThreads":1,"connectionsPerBroker":1,"connectionMaxIdleSeconds":180,"useTcpNoDelay":true,"useTls":false,"tlsKeyFilePath":null,"tlsCertificateFilePath":null,"tlsTrustCertsFilePath":null,"tlsAllowInsecureConnection":false,"tlsHostnameVerificationEnable":false,"concurrentLookupRequest":5000,"maxLookupRequest":50000,"maxLookupRedirects":20,"maxNumberOfRejectedRequestPerConnection":50,"keepAliveIntervalSeconds":30,"connectionTimeoutMs":10000,"requestTimeoutMs":60000,"initialBackoffIntervalNanos":100000000,"maxBackoffIntervalNanos":60000000000,"enableBusyWait":false,"listenerName":null,"useKeyStoreTls":false,"sslProvider":null,"tlsKeyStoreType":"JKS","tlsKeyStorePath":null,"tlsKeyStorePassword":null,"tlsTrustStoreType":"JKS","tlsTrustStorePath":null,"tlsTrustStorePassword":null,"tlsCiphers":[],"tlsProtocols":[],"memoryLimitBytes":0,"proxyServiceUrl":null,"proxyProtocol":null,"enableTransaction":false,"dnsLookupBindAddress":null,"dnsLookupBindPort":0,"socks5ProxyAddress":null,"socks5ProxyUsername":null,"socks5ProxyPassword":null}
2023-02-10T15:19:18,953+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ProducerImpl - [persistent://public/functions/assignments] [c-standalone-fw-localhost-8080-scheduler-manager] Creating producer on cnx [id: 0xa5a16e3a, L:/127.0.0.1:10729 - R:localhost/127.0.0.1:6650]
2023-02-10T15:19:18,970+0800 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://public/functions/assignments] Updated topic epoch to 1
2023-02-10T15:19:18,972+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:10729] Created new producer: Producer{topic=PersistentTopic{topic=persistent://public/functions/assignments}, client=/127.0.0.1:10729, producerName=c-standalone-fw-localhost-8080-scheduler-manager, producerId=0}
2023-02-10T15:19:18,977+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ProducerImpl - [persistent://public/functions/assignments] [c-standalone-fw-localhost-8080-scheduler-manager] Created producer on cnx [id: 0xa5a16e3a, L:/127.0.0.1:10729 - R:localhost/127.0.0.1:6650]
2023-02-10T15:19:18,977+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ProducerImpl - [persistent://public/functions/assignments] [c-standalone-fw-localhost-8080-scheduler-manager] Producer epoch is Optional[1]
2023-02-10T15:19:18,981+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - Starting Pulsar producer perf with config: {"topicName":"persistent://public/functions/metadata","producerName":"c-standalone-fw-localhost-8080-leader","sendTimeoutMs":30000,"blockIfQueueFull":true,"maxPendingMessages":0,"maxPendingMessagesAcrossPartitions":0,"messageRoutingMode":"RoundRobinPartition","hashingScheme":"JavaStringHash","cryptoFailureAction":"FAIL","batchingMaxPublishDelayMicros":1000,"batchingPartitionSwitchFrequencyByPublishDelay":10,"batchingMaxMessages":1000,"batchingMaxBytes":131072,"batchingEnabled":false,"chunkingEnabled":false,"chunkMaxMessageSize":-1,"compressionType":"LZ4","initialSequenceId":null,"autoUpdatePartitions":true,"autoUpdatePartitionsIntervalSeconds":60,"multiSchema":true,"accessMode":"Exclusive","lazyStartPartitionedProducers":false,"properties":{},"initialSubscriptionName":null}
2023-02-10T15:19:18,983+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - Pulsar client config: {"serviceUrl":"pulsar://localhost:6650","authPluginClassName":null,"authParams":null,"authParamMap":null,"operationTimeoutMs":30000,"lookupTimeoutMs":30000,"statsIntervalSeconds":60,"numIoThreads":1,"numListenerThreads":1,"connectionsPerBroker":1,"connectionMaxIdleSeconds":180,"useTcpNoDelay":true,"useTls":false,"tlsKeyFilePath":null,"tlsCertificateFilePath":null,"tlsTrustCertsFilePath":null,"tlsAllowInsecureConnection":false,"tlsHostnameVerificationEnable":false,"concurrentLookupRequest":5000,"maxLookupRequest":50000,"maxLookupRedirects":20,"maxNumberOfRejectedRequestPerConnection":50,"keepAliveIntervalSeconds":30,"connectionTimeoutMs":10000,"requestTimeoutMs":60000,"initialBackoffIntervalNanos":100000000,"maxBackoffIntervalNanos":60000000000,"enableBusyWait":false,"listenerName":null,"useKeyStoreTls":false,"sslProvider":null,"tlsKeyStoreType":"JKS","tlsKeyStorePath":null,"tlsKeyStorePassword":null,"tlsTrustStoreType":"JKS","tlsTrustStorePath":null,"tlsTrustStorePassword":null,"tlsCiphers":[],"tlsProtocols":[],"memoryLimitBytes":0,"proxyServiceUrl":null,"proxyProtocol":null,"enableTransaction":false,"dnsLookupBindAddress":null,"dnsLookupBindPort":0,"socks5ProxyAddress":null,"socks5ProxyUsername":null,"socks5ProxyPassword":null}
2023-02-10T15:19:18,985+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ProducerImpl - [persistent://public/functions/metadata] [c-standalone-fw-localhost-8080-leader] Creating producer on cnx [id: 0xa5a16e3a, L:/127.0.0.1:10729 - R:localhost/127.0.0.1:6650]
2023-02-10T15:19:18,990+0800 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://public/functions/metadata] Updated topic epoch to 1
2023-02-10T15:19:18,990+0800 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:10729] Created new producer: Producer{topic=PersistentTopic{topic=persistent://public/functions/metadata}, client=/127.0.0.1:10729, producerName=c-standalone-fw-localhost-8080-leader, producerId=1}
2023-02-10T15:19:18,990+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ProducerImpl - [persistent://public/functions/metadata] [c-standalone-fw-localhost-8080-leader] Created producer on cnx [id: 0xa5a16e3a, L:/127.0.0.1:10729 - R:localhost/127.0.0.1:6650]
2023-02-10T15:19:18,990+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ProducerImpl - [persistent://public/functions/metadata] [c-standalone-fw-localhost-8080-leader] Producer epoch is Optional[1]
2023-02-10T15:19:18,991+0800 [pulsar-external-listener-59-1] INFO org.apache.pulsar.functions.worker.SchedulerManager - Initializing scheduler manager
2023-02-10T15:19:19,989+0800 [assignment-tailer-thread] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://public/functions/assignments][c-standalone-fw-localhost-8080-function-assignment-tailer-reader-f1cd2d43c3] Get topic last message Id
2023-02-10T15:19:19,990+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://public/functions/assignments][c-standalone-fw-localhost-8080-function-assignment-tailer-reader-f1cd2d43c3] Successfully getLastMessageId -1:-1
2023-02-10T15:19:19,991+0800 [assignment-tailer-thread] INFO org.apache.pulsar.functions.worker.FunctionAssignmentTailer - assignment tailer thread exiting
2023-02-10T15:19:19,991+0800 [pulsar-external-listener-59-1] INFO org.apache.pulsar.functions.worker.FunctionAssignmentTailer - Closing function assignment tailer
2023-02-10T15:19:19,991+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:10729] Closing consumer: consumerId=3
2023-02-10T15:19:19,991+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.AbstractDispatcherSingleActiveConsumer - Removing consumer Consumer{subscription=PersistentSubscription{topic=persistent://public/functions/assignments, name=c-standalone-fw-localhost-8080-function-assignment-tailer-reader-f1cd2d43c3}, consumerId=3, consumerName=c-standalone-fw-localhost-8080-function-assignment-tailer, address=/127.0.0.1:10729}
2023-02-10T15:19:19,992+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.persistent.PersistentSubscription - [persistent://public/functions/assignments][c-standalone-fw-localhost-8080-function-assignment-tailer-reader-f1cd2d43c3] Successfully closed subscription [NonDurableCursorImpl{ledger=public/functions/persistent/assignments, ackPos=15:-1, readPos=15:0}]
2023-02-10T15:19:19,992+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.persistent.PersistentSubscription - [persistent://public/functions/assignments][c-standalone-fw-localhost-8080-function-assignment-tailer-reader-f1cd2d43c3] Successfully closed dispatcher for reader
2023-02-10T15:19:19,992+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:10729] Closed consumer, consumerId=3
2023-02-10T15:19:19,992+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://public/functions/assignments] [c-standalone-fw-localhost-8080-function-assignment-tailer-reader-f1cd2d43c3] Closed consumer
2023-02-10T15:19:19,992+0800 [pulsar-external-listener-59-1] INFO org.apache.pulsar.functions.worker.FunctionMetaDataManager - FunctionMetaDataManager becoming leader by creating exclusive producer
2023-02-10T15:19:20,004+0800 [function-metadata-tailer-thread] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://public/functions/metadata][c-standalone-fw-localhost-8080-function-metadata-tailer-reader-63c4334116] Get topic last message Id
2023-02-10T15:19:20,005+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://public/functions/metadata][c-standalone-fw-localhost-8080-function-metadata-tailer-reader-63c4334116] Successfully getLastMessageId -1:-1
2023-02-10T15:19:20,005+0800 [function-metadata-tailer-thread] INFO org.apache.pulsar.functions.worker.FunctionMetaDataTopicTailer - metadata tailer thread exiting
2023-02-10T15:19:20,005+0800 [pulsar-external-listener-59-1] INFO org.apache.pulsar.functions.worker.FunctionMetaDataTopicTailer - Stopping function metadata tailer
2023-02-10T15:19:20,005+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:10729] Closing consumer: consumerId=4
2023-02-10T15:19:20,005+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.AbstractDispatcherSingleActiveConsumer - Removing consumer Consumer{subscription=PersistentSubscription{topic=persistent://public/functions/metadata, name=c-standalone-fw-localhost-8080-function-metadata-tailer-reader-63c4334116}, consumerId=4, consumerName=c-standalone-fw-localhost-8080-function-metadata-tailer, address=/127.0.0.1:10729}
2023-02-10T15:19:20,006+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.persistent.PersistentSubscription - [persistent://public/functions/metadata][c-standalone-fw-localhost-8080-function-metadata-tailer-reader-63c4334116] Successfully closed subscription [NonDurableCursorImpl{ledger=public/functions/persistent/metadata, ackPos=18:-1, readPos=18:0}]
2023-02-10T15:19:20,006+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.persistent.PersistentSubscription - [persistent://public/functions/metadata][c-standalone-fw-localhost-8080-function-metadata-tailer-reader-63c4334116] Successfully closed dispatcher for reader
2023-02-10T15:19:20,006+0800 [pulsar-io-19-3] INFO org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:10729] Closed consumer, consumerId=4
2023-02-10T15:19:20,006+0800 [pulsar-client-io-58-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://public/functions/metadata] [c-standalone-fw-localhost-8080-function-metadata-tailer-reader-63c4334116] Closed consumer
2023-02-10T15:19:20,006+0800 [pulsar-external-listener-59-1] INFO org.apache.pulsar.functions.worker.FunctionMetaDataTopicTailer - Stopped function metadata tailer
2023-02-10T15:19:20,006+0800 [pulsar-external-listener-59-1] INFO org.apache.pulsar.functions.worker.FunctionMetaDataManager - FunctionMetaDataManager done becoming leader
2023-02-10T15:19:20,043+0800 [ForkJoinPool.commonPool-worker-3] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [10/Feb/2023:15:19:20 +0800] "GET /admin/v2/persistent/public/functions/coordinate/stats?getPreciseBacklog=false&subscriptionBacklogSize=false&getEarliestTimeInBacklog=false HTTP/1.1" 200 2410 "-" "Pulsar-Java-v2.11.0" 28
2023-02-10T15:19:20,056+0800 [worker-scheduler-0] INFO org.apache.pulsar.functions.worker.SchedulerManager - Schedule summary - execution time: 0.043905637 sec | total unassigned: 0 | stats: {"Added": 0, "Updated": 0, "removed": 0}
{
"c-standalone-fw-localhost-8080" : {
"originalNumAssignments" : 0,
"finalNumAssignments" : 0,
"instancesAdded" : 0,
"instancesRemoved" : 0,
"instancesUpdated" : 0,
"alive" : true
}
}
2023-02-10T15:19:48,014+0800 [ForkJoinPool.commonPool-worker-3] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [10/Feb/2023:15:19:48 +0800] "GET /admin/v2/persistent/public/functions/coordinate/stats?getPreciseBacklog=false&subscriptionBacklogSize=false&getEarliestTimeInBacklog=false HTTP/1.1" 200 2410 "-" "Pulsar-Java-v2.11.0" 4
2023-02-10T15:19:48,021+0800 [ForkJoinPool.commonPool-worker-3] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [10/Feb/2023:15:19:48 +0800] "GET /admin/v2/persistent/public/functions/coordinate/stats?getPreciseBacklog=false&subscriptionBacklogSize=false&getEarliestTimeInBacklog=false HTTP/1.1" 200 2410 "-" "Pulsar-Java-v2.11.0" 3
2023-02-10T15:20:18,013+0800 [ForkJoinPool.commonPool-worker-3] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [10/Feb/2023:15:20:18 +0800] "GET /admin/v2/persistent/public/functions/coordinate/stats?getPreciseBacklog=false&subscriptionBacklogSize=false&getEarliestTimeInBacklog=false HTTP/1.1" 200 2410 "-" "Pulsar-Java-v2.11.0" 3
2023-02-10T15:20:18,020+0800 [ForkJoinPool.commonPool-worker-3] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [10/Feb/2023:15:20:18 +0800] "GET /admin/v2/persistent/public/functions/coordinate/stats?getPreciseBacklog=false&subscriptionBacklogSize=false&getEarliestTimeInBacklog=false HTTP/1.1" 200 2410 "-" "Pulsar-Java-v2.11.0" 3
2023-02-10T15:20:18,025+0800 [ForkJoinPool.commonPool-worker-3] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [10/Feb/2023:15:20:18 +0800] "GET /admin/v2/persistent/public/functions/coordinate/stats?getPreciseBacklog=false&subscriptionBacklogSize=false&getEarliestTimeInBacklog=false HTTP/1.1" 200 2410 "-" "Pulsar-Java-v2.11.0" 2
2023-02-10T15:20:48,014+0800 [ForkJoinPool.commonPool-worker-3] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [10/Feb/2023:15:20:48 +0800] "GET /admin/v2/persistent/public/functions/coordinate/stats?getPreciseBacklog=false&subscriptionBacklogSize=false&getEarliestTimeInBacklog=false HTTP/1.1" 200 2410 "-" "Pulsar-Java-v2.11.0" 3
2023-02-10T15:20:48,019+0800 [ForkJoinPool.commonPool-worker-3] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [10/Feb/2023:15:20:48 +0800] "GET /admin/v2/persistent/public/functions/coordinate/stats?getPreciseBacklog=false&subscriptionBacklogSize=false&getEarliestTimeInBacklog=false HTTP/1.1" 200 2410 "-" "Pulsar-Java-v2.11.0" 3
七、创建Kafka Topic
bin/pulsar-admin topics create persistent://public/default/my-topic
(root)[t-qcbj5-hadoop-database-002]{/data/src/apache-pulsar-2.11.0}<test>$ bin/pulsar-client produce my-topic --messages 'Hello Pulsar!'
2023-02-10T15:07:22,363+0800 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ConnectionPool - [[id: 0x0fe4db59, L:/127.0.0.1:9978 - R:localhost/127.0.0.1:6650]] Connected to server
2023-02-10T15:07:22,489+0800 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - Starting Pulsar producer perf with config: {"topicName":"my-topic","producerName":null,"sendTimeoutMs":30000,"blockIfQueueFull":false,"maxPendingMessages":1000,"maxPendingMessagesAcrossPartitions":50000,"messageRoutingMode":"RoundRobinPartition","hashingScheme":"JavaStringHash","cryptoFailureAction":"FAIL","batchingMaxPublishDelayMicros":1000,"batchingPartitionSwitchFrequencyByPublishDelay":10,"batchingMaxMessages":1000,"batchingMaxBytes":131072,"batchingEnabled":true,"chunkingEnabled":false,"chunkMaxMessageSize":-1,"compressionType":"NONE","initialSequenceId":null,"autoUpdatePartitions":true,"autoUpdatePartitionsIntervalSeconds":60,"multiSchema":true,"accessMode":"Shared","lazyStartPartitionedProducers":false,"properties":{},"initialSubscriptionName":null}
2023-02-10T15:07:22,521+0800 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - Pulsar client config: {"serviceUrl":"pulsar://localhost:6650/","authPluginClassName":null,"authParams":null,"authParamMap":null,"operationTimeoutMs":30000,"lookupTimeoutMs":30000,"statsIntervalSeconds":60,"numIoThreads":1,"numListenerThreads":1,"connectionsPerBroker":1,"connectionMaxIdleSeconds":180,"useTcpNoDelay":true,"useTls":false,"tlsKeyFilePath":"","tlsCertificateFilePath":"","tlsTrustCertsFilePath":"","tlsAllowInsecureConnection":false,"tlsHostnameVerificationEnable":false,"concurrentLookupRequest":5000,"maxLookupRequest":50000,"maxLookupRedirects":20,"maxNumberOfRejectedRequestPerConnection":50,"keepAliveIntervalSeconds":30,"connectionTimeoutMs":10000,"requestTimeoutMs":60000,"initialBackoffIntervalNanos":100000000,"maxBackoffIntervalNanos":60000000000,"enableBusyWait":false,"listenerName":null,"useKeyStoreTls":false,"sslProvider":null,"tlsKeyStoreType":"JKS","tlsKeyStorePath":"","tlsKeyStorePassword":"*****","tlsTrustStoreType":"JKS","tlsTrustStorePath":"","tlsTrustStorePassword":"*****","tlsCiphers":[],"tlsProtocols":[],"memoryLimitBytes":0,"proxyServiceUrl":null,"proxyProtocol":null,"enableTransaction":false,"dnsLookupBindAddress":null,"dnsLookupBindPort":0,"socks5ProxyAddress":null,"socks5ProxyUsername":null,"socks5ProxyPassword":null}
2023-02-10T15:07:22,536+0800 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ProducerImpl - [my-topic] [null] Creating producer on cnx [id: 0x0fe4db59, L:/127.0.0.1:9978 - R:localhost/127.0.0.1:6650]
2023-02-10T15:07:22,547+0800 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ProducerImpl - [my-topic] [standalone-0-1] Created producer on cnx [id: 0x0fe4db59, L:/127.0.0.1:9978 - R:localhost/127.0.0.1:6650]
2023-02-10T15:07:22,593+0800 [main] INFO org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - [my-topic] [standalone-0-1] Pending messages: 0 --- Publish throughput: 14.62 msg/s --- 0.00 Mbit/s --- Latency: med: 29.000 ms - 95pct: 29.000 ms - 99pct: 29.000 ms - 99.9pct: 29.000 ms - max: 29.000 ms --- BatchSize: med: 1.000 - 95pct: 1.000 - 99pct: 1.000 - 99.9pct: 1.000 - max: 1.000 --- MsgSize: med: 13.000 bytes - 95pct: 13.000 bytes - 99pct: 13.000 bytes - 99.9pct: 13.000 bytes - max: 13.000 bytes --- Ack received rate: 14.62 ack/s --- Failed messages: 0 --- Pending messages: 0
2023-02-10T15:07:22,598+0800 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ProducerImpl - [my-topic] [standalone-0-1] Closed Producer
2023-02-10T15:07:22,599+0800 [main] INFO org.apache.pulsar.client.impl.PulsarClientImpl - Client closing. URL: pulsar://localhost:6650/
2023-02-10T15:07:22,622+0800 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ClientCnx - [id: 0x0fe4db59, L:/127.0.0.1:9978 ! R:localhost/127.0.0.1:6650] Disconnected
2023-02-10T15:07:24,636+0800 [main] INFO org.apache.pulsar.client.cli.PulsarClientTool - 1 messages successfully produced
八、往Topic写入数据
bin/pulsar-client produce my-topic --messages 'Hello Pulsar!'
九、消费pulsar的Topic
bin/pulsar-client consume my-topic -s 'my-subscription' -p Earliest -n 0
2023-02-10T15:07:09,675+0800 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ConnectionPool - [[id: 0xd3503738, L:/127.0.0.1:9960 - R:localhost/127.0.0.1:6650]] Connected to server
2023-02-10T15:07:09,787+0800 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ConsumerStatsRecorderImpl - Starting Pulsar consumer status recorder with config: {"topicNames":["my-topic"],"topicsPattern":null,"subscriptionName":"my-subscription","subscriptionType":"Exclusive","subscriptionProperties":null,"subscriptionMode":"Durable","receiverQueueSize":1000,"acknowledgementsGroupTimeMicros":100000,"negativeAckRedeliveryDelayMicros":60000000,"maxTotalReceiverQueueSizeAcrossPartitions":50000,"consumerName":null,"ackTimeoutMillis":0,"tickDurationMillis":1000,"priorityLevel":0,"maxPendingChunkedMessage":10,"autoAckOldestChunkedMessageOnQueueFull":false,"expireTimeOfIncompleteChunkedMessageMillis":60000,"cryptoFailureAction":"FAIL","properties":{},"readCompacted":false,"subscriptionInitialPosition":"Earliest","patternAutoDiscoveryPeriod":60,"regexSubscriptionMode":"PersistentOnly","deadLetterPolicy":null,"retryEnable":false,"autoUpdatePartitions":true,"autoUpdatePartitionsIntervalSeconds":60,"replicateSubscriptionState":false,"resetIncludeHead":false,"keySharedPolicy":null,"batchIndexAckEnabled":false,"ackReceiptEnabled":false,"poolMessages":true,"startPaused":false,"autoScaledReceiverQueueSizeEnabled":false,"topicConfigurations":[],"maxPendingChuckedMessage":10}
2023-02-10T15:07:09,820+0800 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ConsumerStatsRecorderImpl - Pulsar client config: {"serviceUrl":"pulsar://localhost:6650/","authPluginClassName":null,"authParams":null,"authParamMap":null,"operationTimeoutMs":30000,"lookupTimeoutMs":30000,"statsIntervalSeconds":60,"numIoThreads":1,"numListenerThreads":1,"connectionsPerBroker":1,"connectionMaxIdleSeconds":180,"useTcpNoDelay":true,"useTls":false,"tlsKeyFilePath":"","tlsCertificateFilePath":"","tlsTrustCertsFilePath":"","tlsAllowInsecureConnection":false,"tlsHostnameVerificationEnable":false,"concurrentLookupRequest":5000,"maxLookupRequest":50000,"maxLookupRedirects":20,"maxNumberOfRejectedRequestPerConnection":50,"keepAliveIntervalSeconds":30,"connectionTimeoutMs":10000,"requestTimeoutMs":60000,"initialBackoffIntervalNanos":100000000,"maxBackoffIntervalNanos":60000000000,"enableBusyWait":false,"listenerName":null,"useKeyStoreTls":false,"sslProvider":null,"tlsKeyStoreType":"JKS","tlsKeyStorePath":"","tlsKeyStorePassword":"*****","tlsTrustStoreType":"JKS","tlsTrustStorePath":"","tlsTrustStorePassword":"*****","tlsCiphers":[],"tlsProtocols":[],"memoryLimitBytes":0,"proxyServiceUrl":null,"proxyProtocol":null,"enableTransaction":false,"dnsLookupBindAddress":null,"dnsLookupBindPort":0,"socks5ProxyAddress":null,"socks5ProxyUsername":null,"socks5ProxyPassword":null}
2023-02-10T15:07:09,839+0800 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [my-topic][my-subscription] Subscribing to topic on cnx [id: 0xd3503738, L:/127.0.0.1:9960 - R:localhost/127.0.0.1:6650], consumerId 0
2023-02-10T15:07:09,873+0800 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [my-topic][my-subscription] Subscribed to topic on localhost/127.0.0.1:6650 -- consumer: 0
----- got message -----
key:[null], properties:[], content:Hello Pulsar!
2023-02-10T15:08:09,828+0800 [pulsar-timer-6-1] INFO org.apache.pulsar.client.impl.ConsumerStatsRecorderImpl - [my-topic] [my-subscription] [4410a] Prefetched messages: 0 --- Consume throughput received: 0.02 msgs/s --- 0.00 Mbit/s --- Ack sent rate: 0.02 ack/s --- Failed messages: 0 --- batch messages: 0 ---Failed acks: 0
至此完成pulsar安装部署