1 2025-02-27 16:56:09 [NOTICE] [StoresManager.cpp:251 33456,4170311872] Create scheduler with file size 16MB, cache 10MB and flush limit 32MB, compressed 2, , read wait number 0, enable index true, read thread number 2, index arena size 256
2 2025-02-27 16:56:11 [NOTICE] [StoreCommandsProcessor.cpp:107 33456,4128315136] Get Request GET /start?subTopic=POSTGRESQL_np_6j2r94kphagw_6j2rad6e7olc-1-0&subId=0000000002&store.conf=%2Fhome%2Fds%2Fstore%2Fstore7102%2Fconf%2Fcrawler.conf&master=true HTTP/1.1
3 2025-02-27 16:56:11 [NOTICE] [StoreCommandsProcessor.cpp:112 33456,4128315136] Request params: master = true
4 2025-02-27 16:56:11 [NOTICE] [StoreCommandsProcessor.cpp:112 33456,4128315136] Request params: store.conf = /home/ds/store/store7102/conf/crawler.conf
5 2025-02-27 16:56:11 [NOTICE] [StoreCommandsProcessor.cpp:112 33456,4128315136] Request params: subId = 0000000002
6 2025-02-27 16:56:11 [NOTICE] [StoreCommandsProcessor.cpp:112 33456,4128315136] Request params: subTopic = POSTGRESQL_np_6j2r94kphagw_6j2rad6e7olc-1-0
7 2025-02-27 16:56:11 [NOTICE] [StoresManager.cpp:343 33456,4128315136] Starting store POSTGRESQL_np_6j2r94kphagw_6j2rad6e7olc-1-0.0000000002 with config /home/ds/store/store7102/conf/crawler.conf
8 
9 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:571 33456,4128315136] sectionnames size:13
10 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:587 33456,4128315136] load 0th sectionnames:global
11 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:config.version value:3
12 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:running.mode value:strict
13 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:577 33456,4128315136] Config of sectionnames:mysql2store is empty
14 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:577 33456,4128315136] Config of sectionnames:store2store is empty
15 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:577 33456,4128315136] Config of sectionnames:ob2store is empty
16 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:587 33456,4128315136] load 4th sectionnames:store
17 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:clearer.outdated value:86400
18 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:clearer.period value:3600
19 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:client.wait value:43200
20 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:connection.numLimit value:100
21 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:drcnetListenPort value:17002
22 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:listeningPort value:17001
23 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:reader value:on
24 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:repStatus value:master
25 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:storesManager.port value:7100
26 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:writer.threshold value:1
27 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:writer.type value:message
28 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:577 33456,4128315136] Config of sectionnames:liboblog is empty
29 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:577 33456,4128315136] Config of sectionnames:partition is empty
30 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:577 33456,4128315136] Config of sectionnames:hbase2store is empty
31 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:577 33456,4128315136] Config of sectionnames:oracle2store is empty
32 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:587 33456,4128315136] load 9th sectionnames:deliver2store
33 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:connect.deliver_proterties_path value:./kafka/config/connect-drcdeliver.properties
34 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:error.level value:WARN
35 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:master.timestamp value:1740645965
36 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:parallelism value:32
37 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:pipeline value:reset,read,parse,filter,consume
38 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:postgresql.database value:corehr1114
39 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:postgresql.host value:192.168.10.66
40 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:postgresql.password value:ch1qazXDR66
41 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:postgresql.port value:5432
42 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:postgresql.publication_name value:oms_np_6j2r94kphagw_1740646565138
43 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:postgresql.slot_name value:oms_np_6j2r94kphagw_1740646565138
44 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:postgresql.start_position value:
45 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:postgresql.username value:corehr
46 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:postgresql.whitelist value:[{"name":"public","func":"fn","all":false,"sub":[{"name":"payslip_info","func":"fn"}]}]
47 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:subId value:0000000002
48 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:subTopic value:POSTGRESQL_np_6j2r94kphagw_6j2rad6e7olc-1-0
49 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:subreader.bootstrup value:./kafka/xlog/bin/connect-xlog.sh
50 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:subreader.type value:postgresql
51 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:591 33456,4128315136] load config key:topic value:POSTGRESQL_np_6j2r94kphagw_6j2rad6e7olc
52 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:577 33456,4128315136] Config of sectionnames:unit is empty
53 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:577 33456,4128315136] Config of sectionnames:db2tostore is empty
54 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:577 33456,4128315136] Config of sectionnames:logproxy2store is empty
55 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:834 33456,4128315136] succeed to init record sent log with log.sent.record=0, log.sent.record.users=, log.sent.max.files=50
56 2025-02-27 16:56:11 [WARN] [StoreManager.cpp:680 33456,4128315136] Load Rule Library : library Name not found, FilterRule not work
57 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:1383 33456,4128315136] drcnet server buf size: 1048576
58 2025-02-27 16:56:11 [WARN] [StoreManager.cpp:1394 33456,4128315136] drcnet http listen port is 17002
59 2025-02-27 16:56:11 [NOTICE] [StoreManager.cpp:1487 33456,4128315136] Store Start use normal  mode
60 2025-02-27 16:56:11 [NOTICE] [StoresManager.cpp:414 33456,4128315136] Store POSTGRESQL_np_6j2r94kphagw_6j2rad6e7olc-1-0.0000000002 first start
61 2025-02-27 16:56:11 [NOTICE] [StoresManager.cpp:424 33456,4128315136] Store quota remains 9 out of 10 after start, 1 store instance in memory
62 2025-02-27 16:56:11 [NOTICE] [Reader.cpp:231 33456,3841955584] Drc Frame POSTGRESQL_np_6j2r94kphagw_6j2rad6e7olc-1-0.0000000002 started
63 2025-02-27 16:56:11 [NOTICE] [Reader.cpp:193 33456,3841955584] register module deliver2store ok
64 2025-02-27 16:56:11 [NOTICE] [DrcModRunner.cpp:166 33456,3825170176] pipeline diagnose off
65 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:88 33456,3825170176] get checkpoint info from queue is 0, 0, 0, 0
66 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:90 33456,3825170176] get checkpoint info from config is 0, 0, 1740645965
67 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:100 33456,3825170176] set deliver checkpoint: &mCurCheckpoint:[FileID: 0][FileOffset: 0][TimeStamp: 1740645965]
68 2025-02-27 16:56:11 [NOTICE] [subreadermanager.cpp:38 33456,3825170176] begin to start deliver2store's sub-reader: ./kafka/xlog/bin/connect-xlog.sh
69 2025-02-27 16:56:11 [NOTICE] [subreadermanager.cpp:156 33456,3825170176] sub reader type: postgresql
70 2025-02-27 16:56:11 [NOTICE] [subreadermanager.cpp:106 33456,3825170176] DELIVER_LISTEN_PORT is 43113
71 2025-02-27 16:56:11 [WARN] [subreadermanager.cpp:69 33456,3825170176] skipped property clearer.outdated=86400 when building deliver2store config, property name must starts with postgresql
72 2025-02-27 16:56:11 [WARN] [subreadermanager.cpp:69 33456,3825170176] skipped property clearer.period=3600 when building deliver2store config, property name must starts with postgresql
73 2025-02-27 16:56:11 [WARN] [subreadermanager.cpp:69 33456,3825170176] skipped property client.wait=43200 when building deliver2store config, property name must starts with postgresql
74 2025-02-27 16:56:11 [WARN] [subreadermanager.cpp:69 33456,3825170176] skipped property connect.deliver_proterties_path=./kafka/config/connect-drcdeliver.properties when building deliver2store config, property name must starts with postgresql
75 2025-02-27 16:56:11 [WARN] [subreadermanager.cpp:69 33456,3825170176] skipped property connection.numLimit=100 when building deliver2store config, property name must starts with postgresql
76 2025-02-27 16:56:11 [WARN] [subreadermanager.cpp:69 33456,3825170176] skipped property drcnetListenPort=17002 when building deliver2store config, property name must starts with postgresql
77 2025-02-27 16:56:11 [WARN] [subreadermanager.cpp:69 33456,3825170176] skipped property error.level=WARN when building deliver2store config, property name must starts with postgresql
78 2025-02-27 16:56:11 [WARN] [subreadermanager.cpp:69 33456,3825170176] skipped property listeningPort=17001 when building deliver2store config, property name must starts with postgresql
79 2025-02-27 16:56:11 [WARN] [subreadermanager.cpp:69 33456,3825170176] skipped property master.binlog= when building deliver2store config, property name must starts with postgresql
80 2025-02-27 16:56:11 [WARN] [subreadermanager.cpp:69 33456,3825170176] skipped property master.offset= when building deliver2store config, property name must starts with postgresql
81 2025-02-27 16:56:11 [WARN] [subreadermanager.cpp:69 33456,3825170176] skipped property master.timestamp=1740645965 when building deliver2store config, property name must starts with postgresql
82 2025-02-27 16:56:11 [WARN] [subreadermanager.cpp:69 33456,3825170176] skipped property network.listen_port=43113 when building deliver2store config, property name must starts with postgresql
83 2025-02-27 16:56:11 [WARN] [subreadermanager.cpp:69 33456,3825170176] skipped property parallelism=32 when building deliver2store config, property name must starts with postgresql
84 2025-02-27 16:56:11 [WARN] [subreadermanager.cpp:69 33456,3825170176] skipped property pipeline=reset,read,parse,filter,consume when building deliver2store config, property name must starts with postgresql
85 2025-02-27 16:56:11 [WARN] [subreadermanager.cpp:69 33456,3825170176] skipped property reader=on when building deliver2store config, property name must starts with postgresql
86 2025-02-27 16:56:11 [WARN] [subreadermanager.cpp:69 33456,3825170176] skipped property repStatus=master when building deliver2store config, property name must starts with postgresql
87 2025-02-27 16:56:11 [WARN] [subreadermanager.cpp:69 33456,3825170176] skipped property stores.listen.port=7102 when building deliver2store config, property name must starts with postgresql
88 2025-02-27 16:56:11 [WARN] [subreadermanager.cpp:69 33456,3825170176] skipped property storesManager.port=7102 when building deliver2store config, property name must starts with postgresql
89 2025-02-27 16:56:11 [WARN] [subreadermanager.cpp:69 33456,3825170176] skipped property subreader.bootstrup=./kafka/xlog/bin/connect-xlog.sh when building deliver2store config, property name must starts with postgresql
90 2025-02-27 16:56:11 [WARN] [subreadermanager.cpp:69 33456,3825170176] skipped property subreader.jvm_param= when building deliver2store config, property name must starts with postgresql
91 2025-02-27 16:56:11 [WARN] [subreadermanager.cpp:69 33456,3825170176] skipped property subreader.profile_path= when building deliver2store config, property name must starts with postgresql
92 2025-02-27 16:56:11 [WARN] [subreadermanager.cpp:69 33456,3825170176] skipped property subreader.type=postgresql when building deliver2store config, property name must starts with postgresql
93 2025-02-27 16:56:11 [WARN] [subreadermanager.cpp:69 33456,3825170176] skipped property writer.threshold=1 when building deliver2store config, property name must starts with postgresql
94 2025-02-27 16:56:11 [WARN] [subreadermanager.cpp:69 33456,3825170176] skipped property writer.type=message when building deliver2store config, property name must starts with postgresql
95 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:169 33456,3825170176] deliver2store initModule ok
96 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
97 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
98 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
99 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
100 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
101 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
102 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
103 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
104 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
105 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
106 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
107 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
108 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
109 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
110 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
111 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
112 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
113 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
114 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
115 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
116 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
117 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
118 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
119 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
120 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
121 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
122 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
123 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
124 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
125 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
126 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
127 2025-02-27 16:56:11 [NOTICE] [deliver2store.cpp:195 33456,3825170176] deliver2store initDrcPack ok
128 2025-02-27 16:56:12 [WARN] [stores.cpp:95 33456,4170311872] reset sigHandle for signal [SIGTERM]
129 2025-02-27 16:56:12 [WARN] [stores.cpp:101 33456,4170311872] reset sigHandle for signal [SIGINT]