OB复赛

连接到ECS

image-20211122133121670

  1. 首先下载Open VPN，然后使用附件的certs配置来配置open vpn，然后连接。下面是配置文件夹内的内容，基本把附件certs中的文件复制过去就好了。

    image-20211122133321979

  2. 然后打开finalshell，连接到对应的ip。

172.16.0.12 BtfQDDzWoYvCmiCXhm6f 172.16.0.67 23neMCCn5WhLbc4s6DWt

密码修改为：zhou19981209

使用VS code插件remote development来开发

参考 vscode连接远程服务器+SFTP同步本地文件 其实可以不用安装sftp，因为在vs code中可以直接上传本地文件和更改服务器文件

为了不用每次都输入密码，我们可以把公钥传输到远程服务器：

image-20211123153137099

在authorized_keys中加入很多行公钥。然后把.ssh和authorized_keys的访问权限更新为至少可读。

vscode通过跳板机连接远程服务器

如何使用源码进行测试

查看系统架构：

1
uname -a

在执行sudo yum install rpm*时，出现了错误：

image-20211123163724686

我直接跳过有依赖问题的包了，sudo yum install rpm* --skip-broken。

image-20211123165808857

image-20211123175221148

采用 sudo -E。简单来说，就是加上-E选项后，用户可以在sudo执行时保留当前用户已存在的环境变量，不会被sudo重置，另外，如果用户对于指定的环境变量没有权限，则会报错。

安装sysbench 1.0.20是可以的。直接在sysbench 上参考二进制的安装方法，推荐的那个。

如何多人在同一个服务器上协作

在同一台电脑上添加多个ssh key - 简书 (jianshu.com)

调试oceanbase

gdb

我们调试的时候要在oceanbase的根目录下调试，然后设置断点的时候要指定函数所在的文件，例如：

1
break ./src/sql/parser/ob_parser.cpp:ObParser::parse

我们要把build_debug/src/observer/observer作为~/zhouhuahui/ob-advanced-data/bin/observer的软链接，才能使用gdb调试成功，或者直接在gdb下使用：

1
dir /home/test/zhouhuahui/Github/oceanbase

定位到源文件也是可以的。

vscode attach

如何在vs code上调试：参考 如何debug OceanBase。但是这种不够，我是是使用attach的方式来调试，写了下面的launch.json

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "observer",
            "type": "cppdbg",
            "request": "attach",
            "processId": "${command:pickProcess}",
            "program": "/home/test/zhouhuahui/ob-advanced-data/bin/observer",
            // "args": ["-r", "127.0.0.1:2882:2881", "-o", "__min_full_resource_pool_memory=268435456,memory_limit=8G,datafile_size=10G,enable_syslog_recycle=True,enable_syslog_wf=True,max_syslog_file_count=4,system_memory=4G,cpu_count=16", "-z", "zone1", "-p", "2881", "-P", "2882", "-n", "obcluster", "-c", 1, "-d", "/home/test/zhouhuahui/ob-advanced-data/store", "-i", "lo", "-l", "INFO"],
            "stopAtEntry": true,
            "cwd": "/home/test/zhouhuahui/Github/oceanbase/",
            "environment": [],
            "externalConsole": false,
            "MIMode": "gdb",
            "setupCommands": [
                {
                    "description": "Enable pretty-printing for gdb",
                    "text": "-enable-pretty-printing",
                    "ignoreFailures": true
                }
            ]
        }
    ]
}

在点击调试之后，在DEBUG CONSOLE界面输入：

1
-exec dir /home/test/zhouhuahui/Github/oceanbase

就可以正确定位到源文件的位置了。

vscode launch

如果还要使用launch的方式，要在程序的参数加一个”-N”参数，来指明程序在前台运行，（因为默认程序在后台运行），如果不加的话，会出先调试程序刚打开却退出的现象。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
    
        {
            "name": "observer2",
            "type": "cppdbg",
            "request": "launch",
            "program": "/home/test/zhouhuahui/Github/oceanbase/build_debug/src/observer/observer",
            "args": ["-N", "-r", "127.0.0.1:2982:2981", "-o", "__min_full_resource_pool_memory=268435456,memory_limit=8G,datafile_size=10G,enable_syslog_recycle=True,enable_syslog_wf=True,max_syslog_file_count=4,system_memory=4G,cpu_count=16", "-z", "zone1", "-p", "2981", "-P", "2982", "-n", "obcluster", "-c", "1", "-d", "/home/test/zhouhuahui/Github/oceanbase/build_debug/ob-advanced-data/store", "-i", "lo", "-l", "INFO"],
            "stopAtEntry": true,
            "cwd": "/home/test/zhouhuahui/Github/oceanbase",
            "environment": [],
            "externalConsole": false,
            "MIMode": "gdb",
            "setupCommands": [
                {
                    "description": "Enable pretty-printing for gdb",
                    "text": "-enable-pretty-printing",
                    "ignoreFailures": true
                }
            ]
        }
    ]
}

100个gdb小技巧 (gitbooks.io)

跟踪sql的执行

1
obclient -uroot@test -h127.0.0.1 -P 2881 -c
1
2
3
4
5
6
7
8
9
10
use test;
create table t1(i int primary key, f float, c CHAR(60));
create table t2(i int primary key, f float, c CHAR(60));
create index t2_c on t2(c);
insert into t1 values(1, 1.5, "zhou");
insert into t1 values(2, 0.5, "hua");
insert into t1 values(3, 40.5, "hui");
insert into t2 values(1, 20.0, "hua");
insert into t2 values(2, 15.135, "hui");
insert into t2 values(3, 10.5, "fan");

理论教程

揭秘 OceanBase SQL 执行计划（一） (qq.com)

索引组织表-阿里云开发者社区 (aliyun.com)

OceanBase SQL 执行计划解读（二）──── 表连接和子查询_OceanBaseGFBK的博客-CSDN博客

两个表join时，谁作为外部表，谁作为内部表不好说，由过滤条件应用后的结果集大小来定。

OceanBase 存储引擎高级技术 - 知乎 (zhihu.com)

OceanBase的索引创建流程 - 知乎 (zhihu.com)

MySQL中的semi-join_lppl010_的专栏-CSDN博客 讲解了什么是semi-join。

OB有两种从表中获取行的方法，一个是TABLE_SCAN，一个是TABLE_GET，为了方便，就直接说scan和get了。scan表示直接从头到尾扫描OB的索引组织表，然后获取这些行；get表示通过索引的key来找到这些行或者行的key。使用scan还是get是由SQL中的where部分决定的，假如查询条件涉及的列是有索引的，就可以使用get，假如没有，就只能使用scan。

但是还有一个问题是：假如A,B表进行嵌套循环连接，那么谁作外表，谁作内表呢？根据A,B获取行的方法不同，分为三种情况来讨论：

  1. A使用scan，B也使用scan。那么需要预估A的结果集大小和B的结果集大小，哪个小就作为内表。
  2. A使用scan，B使用get。那么A作为外表，B作为内表。
  3. A使用get，B使用get。哪个结果集小就作为外表。

性能优化之Block Nested-Loop Join(BNL) - 云+社区 - 腾讯云 (tencent.com)

分析patch

reset和reuse分别在哪些地方被用到了，reset和reuse的具体语义是什么，为什么要这么用？

reuse:

  1. ObTableAccessContext.reuse
  2. ObTableScanStoreRowIterator.reuse_row_iters

分析群里给的patch的更改前后对比

更改前

1
2
3
4
5
6
7
8
namespace oceanbase {
namespace storage {
class ObIStoreRowFilter;
class ObMultipleMerge : public ObQueryRowIterator {
public:
  typedef common::ObSEArray<ObStoreRowIterator*, common::MAX_TABLE_CNT_IN_STORAGE> MergeIterators;

public:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
void ObMultipleGetMerge::reset_with_fuse_row_cache()
{
  prefetch_range_idx_ = 0;
  get_row_range_idx_ = 0;
  sstable_rowkeys_.reset();
  if (nullptr != rows_) {
    for (int64_t i = 0; i < prefetch_cnt_; ++i) {
      rows_[i].~ObQueryRowInfo();
    }
    rows_ = nullptr;
  }
  if (nullptr != handles_) {
    for (int64_t i = 0; i < prefetch_cnt_; ++i) {
      handles_[i].reset();
      handles_[i].~ObFuseRowValueHandle();
    }
    handles_ = nullptr;
  }
  prefetch_cnt_ = 0;
  reuse_iter_array();
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
void ObMultipleMerge::reset()
{
  ObStoreRowIterator* iter = NULL;
  for (int64_t i = 0; i < iters_.count(); ++i) {
    if (NULL != (iter = iters_.at(i))) {
      iter->~ObStoreRowIterator();
    }
  }
  padding_allocator_.reset();
  iters_.reset();
  access_param_ = NULL;
  access_ctx_ = NULL;
  tables_handle_.reset();
  nop_pos_.reset();
  row_stat_.reset();
  table_stat_.reset();
  scan_cnt_ = 0;
  filt_cnt_ = 0;
  need_padding_ = false;
  need_fill_default_ = false;
  need_fill_virtual_columns_ = false;
  need_output_row_with_nop_ = false;
  inited_ = false;
  range_idx_delta_ = 0;
  next_row_ = NULL;
  out_cols_projector_ = NULL;
  skip_refresh_table_ = false;
  read_memtable_only_ = false;
  is_tables_reset_ = false;
}

void ObMultipleMerge::reuse()
{
  reuse_iter_array();
  row_stat_.reset();
  table_stat_.reuse();
  range_idx_delta_ = 0;
  unprojected_row_.flag_ = -1;
  next_row_ = nullptr;
  skip_refresh_table_ = false;
  read_memtable_only_ = false;
}

void ObMultipleMerge::reuse_iter_array()
{
  ObStoreRowIterator* iter = NULL;
  for (int64_t i = 0; i < iters_.count(); ++i) {
    if (NULL != (iter = iters_.at(i))) {
      iter->~ObStoreRowIterator();
    }
  }
  iters_.reuse();
}


ObStoreRowIterator 只有空的reuse()函数，没有reset()函数。

ObMemtableScanIterator和ObMemtableMGetIterator的reuse()函数都是调用了自己的reset()函数。

ObSEArray.reset()函数最终调用了ObSEArrayImpl.destroy()函数。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
template <typename T, int64_t LOCAL_ARRAY_SIZE, typename BlockAllocatorT, bool auto_free>
void ObSEArrayImpl<T, LOCAL_ARRAY_SIZE, BlockAllocatorT, auto_free>::destroy()
{
  if (is_destructor_safe()) {
  } else {
    for (int64_t i = 0; i < count_; i++) {
      data_[i].~T();
    }
  }

  if (data_ != reinterpret_cast<T*>(local_data_buf_)) {
    internal_free_(data_);
    data_ = reinterpret_cast<T*>(local_data_buf_);
  }
  count_ = 0;
#ifndef DISABLE_SE_ARRAY
  capacity_ = LOCAL_ARRAY_SIZE;
#else
  if (typeid(BlockAllocatorT) != typeid(ObNullAllocator)) {
    capacity_ = 0;
  } else {
    capacity_ = LOCAL_ARRAY_SIZE;
  }
#endif
  max_print_count_ = DEFAULT_MAX_PRINT_COUNT;
  error_ = OB_SUCCESS;
}

// ....

void reuse()
{
  if (is_destructor_safe()) {
  } else {
    for (int64_t i = 0; i < count_; i++) {
      data_[i].~T();
    }
  }
  count_ = 0;
  error_ = OB_SUCCESS;
}

更改后

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
void ObMultipleGetMerge::reset_with_fuse_row_cache()
{
  prefetch_range_idx_ = 0;
  get_row_range_idx_ = 0;
  sstable_rowkeys_.reset();
  if (nullptr != rows_) {
    for (int64_t i = 0; i < prefetch_cnt_; ++i) {
      rows_[i].~ObQueryRowInfo();
    }
    rows_ = nullptr;
  }
  if (nullptr != handles_) {
    for (int64_t i = 0; i < prefetch_cnt_; ++i) {
      handles_[i].reset();
      handles_[i].~ObFuseRowValueHandle();
    }
    handles_ = nullptr;
  }
  prefetch_cnt_ = 0;
  reset_iter_array();
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
void ObMultipleMerge::reset()
{
  ObStoreRowIterator* iter = NULL;
  for (int64_t i = 0; i < iters_.count(); ++i) {
    if (NULL != (iter = iters_.at(i))) {
      iter->~ObStoreRowIterator();
      if (OB_NOT_NULL(access_ctx_->stmt_allocator_)) {
         access_ctx_->stmt_allocator_->free(iter);
       }
       iter = NULL;
    }
  }
  padding_allocator_.reset();
  iters_.reset();
  access_param_ = NULL;
  access_ctx_ = NULL;
  tables_handle_.reset();
  nop_pos_.reset();
  row_stat_.reset();
  table_stat_.reset();
  scan_cnt_ = 0;
  filt_cnt_ = 0;
  need_padding_ = false;
  need_fill_default_ = false;
  need_fill_virtual_columns_ = false;
  need_output_row_with_nop_ = false;
  inited_ = false;
  range_idx_delta_ = 0;
  next_row_ = NULL;
  out_cols_projector_ = NULL;
  skip_refresh_table_ = false;
  read_memtable_only_ = false;
  is_tables_reset_ = false;
}

// .....

void ObMultipleMerge::reuse_iter_array()
{
  ObStoreRowIterator *iter = NULL;
  for (int64_t i = 0; i < iters_.count(); ++i) {
    if (NULL != (iter = iters_.at(i))) {
      iter->reset(); 
      // 细心地同学会发现，在reuse接口中使用reset，是有点怪异的，并且肯定没有最大化提高效率。那么为什么不能直接使用reuse呢？还是要从allocator与stmt_allocator中考虑，可以继续挖掘。注意allocator中内存释放的时机。
    }
  }
}

// ....

void ObMultipleMerge::reset_iter_array()
{
 ObStoreRowIterator* iter = NULL;
 for (int64_t i = 0; i < iters_.count(); ++i) {
   if (NULL != (iter = iters_.at(i))) {
     iter->~ObStoreRowIterator();
     if (OB_NOT_NULL(access_ctx_->stmt_allocator_)) {
       access_ctx_->stmt_allocator_->free(iter);
     }
     iter = NULL;
   }
 }
 iters_.reuse();
}

如果按照简单的sql语句：select /*+ordered use_nl(A,B)*/ count(*) from t1 A, v1 B where A.c1 = B.c1 and A.c2 = B.c2，在rescan阶段将会执行ObTableScanStoreRowIterator::rescan() -> ... ObMultipleScanMerge::reuse() -> ObMultipleMerge::reuse()，

patch2(sysbench: 3850)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
diff --git a/etc/observer.config.bin b/etc/observer.config.bin
new file mode 100644
index 0000000..5e455cd
Binary files /dev/null and b/etc/observer.config.bin differ
diff --git a/etc2/observer.conf.bin b/etc2/observer.conf.bin
new file mode 100644
index 0000000..5e455cd
Binary files /dev/null and b/etc2/observer.conf.bin differ
diff --git a/etc3/observer.conf.bin b/etc3/observer.conf.bin
new file mode 100644
index 0000000..5e455cd
Binary files /dev/null and b/etc3/observer.conf.bin differ
diff --git a/src/storage/memtable/ob_memtable.cpp b/src/storage/memtable/ob_memtable.cpp
index 3a4d28f..ba7d295 100644
--- a/src/storage/memtable/ob_memtable.cpp
+++ b/src/storage/memtable/ob_memtable.cpp
@@ -1033,7 +1033,7 @@ int ObMemtable::get(const storage::ObTableIterParam& param, storage::ObTableAcce
     TRANS_LOG(WARN, "invalid argument, ", K(ret), K(param), K(context));
   } else if (OB_FAIL(context.store_ctx_->mem_ctx_->get_trans_status())) {
     TRANS_LOG(WARN, "trans already end", K(ret));
-  } else if (NULL == (get_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableGetIterator))) ||
+  } else if (NULL == (get_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableGetIterator))) ||
              NULL == (get_iter_ptr = new (get_iter_buffer) ObMemtableGetIterator())) {
     TRANS_LOG(WARN, "construct ObMemtableGetIterator fail");
     ret = OB_ALLOCATE_MEMORY_FAILED;
@@ -1082,7 +1082,7 @@ int ObMemtable::scan(const storage::ObTableIterParam& param, storage::ObTableAcc
   } else {
     if (param.is_multi_version_minor_merge_) {
       if (GCONF._enable_sparse_row) {
-        if (NULL == (scan_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableMultiVersionScanSparseIterator))) ||
+        if (NULL == (scan_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableMultiVersionScanSparseIterator))) ||
             NULL == (scan_iter_ptr = new (scan_iter_buffer) ObMemtableMultiVersionScanSparseIterator())) {
           TRANS_LOG(WARN,
               "construct ObMemtableMultiVersionScanSparseIterator fail",
@@ -1099,7 +1099,7 @@ int ObMemtable::scan(const storage::ObTableIterParam& param, storage::ObTableAcc
           TRANS_LOG(WARN, "scan iter init fail", "ret", ret, K(real_range), K(param), K(context));
         }
       } else {
-        if (NULL == (scan_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableMultiVersionScanIterator))) ||
+        if (NULL == (scan_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableMultiVersionScanIterator))) ||
             NULL == (scan_iter_ptr = new (scan_iter_buffer) ObMemtableMultiVersionScanIterator())) {
           TRANS_LOG(WARN,
               "construct ObMemtableScanIterator fail",
@@ -1117,7 +1117,7 @@ int ObMemtable::scan(const storage::ObTableIterParam& param, storage::ObTableAcc
         }
       }
     } else {
-      if (NULL == (scan_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableScanIterator))) ||
+      if (NULL == (scan_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableScanIterator))) ||
           NULL == (scan_iter_ptr = new (scan_iter_buffer) ObMemtableScanIterator())) {
         TRANS_LOG(WARN,
             "construct ObMemtableScanIterator fail",
@@ -1162,7 +1162,7 @@ int ObMemtable::multi_get(const storage::ObTableIterParam& param, storage::ObTab
     TRANS_LOG(WARN, "invalid argument, ", K(ret), K(param), K(context), K(rowkeys));
   } else if (OB_FAIL(context.store_ctx_->mem_ctx_->get_trans_status())) {
     TRANS_LOG(WARN, "trans already end", K(ret));
-  } else if (NULL == (mget_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableMGetIterator))) ||
+  } else if (NULL == (mget_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableMGetIterator))) ||
              NULL == (mget_iter_ptr = new (mget_iter_buffer) ObMemtableMGetIterator())) {
     TRANS_LOG(WARN,
         "construct ObMemtableMGetIterator fail",
@@ -1212,7 +1212,7 @@ int ObMemtable::multi_scan(const storage::ObTableIterParam& param, storage::ObTa
     TRANS_LOG(WARN, "invalid argument, ", K(ret), K(param), K(context), K(ranges));
   } else if (OB_FAIL(context.store_ctx_->mem_ctx_->get_trans_status())) {
     TRANS_LOG(WARN, "trans already end", K(ret));
-  } else if (NULL == (mscan_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableMScanIterator))) ||
+  } else if (NULL == (mscan_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableMScanIterator))) ||
              NULL == (mscan_iter_ptr = new (mscan_iter_buffer) ObMemtableMScanIterator())) {
     TRANS_LOG(WARN,
         "construct ObMemtableMScanIterator fail",
diff --git a/src/storage/memtable/ob_memtable_iterator.h b/src/storage/memtable/ob_memtable_iterator.h
index ef3a552..0601d87 100644
--- a/src/storage/memtable/ob_memtable_iterator.h
+++ b/src/storage/memtable/ob_memtable_iterator.h
@@ -100,6 +100,9 @@ public:
   {
     reset();
   }
+  virtual void reuse_try() override{
+    reset();
+  }
 
 private:
   // means GETITER
@@ -135,6 +138,9 @@ public:
   {
     reset();
   }
+  virtual void reuse_try() override{
+    reset();
+  }
   virtual int get_gap_end(int64_t& range_idx, const common::ObStoreRowkey*& gap_key, int64_t& gap_size) override
   {
     int ret = common::OB_SUCCESS;
@@ -216,6 +222,9 @@ public:
   {
     reset();
   }
+  virtual void reuse_try() override{
+    reset();
+  }
 
 public:
   static const int64_t ROW_ALLOCATOR_PAGE_SIZE = common::OB_MALLOC_NORMAL_BLOCK_SIZE;
@@ -254,6 +263,9 @@ public:
   {
     reset();
   }
+  virtual void reuse_try() override{
+    reset();
+  }
   virtual int skip_range(int64_t range_idx, const common::ObStoreRowkey* gap_key, const bool include_gap_key) override;
   virtual int get_gap_end(int64_t& range_idx, const common::ObStoreRowkey*& gap_key, int64_t& gap_size) override
   {
@@ -299,6 +311,9 @@ public:
   {
     reset();
   }
+  virtual void reuse_try() override{
+    reset();
+  }
   enum ScanState { SCAN_BEGIN, SCAN_UNCOMMITTED_ROW, SCAN_COMPACT_ROW, SCAN_MULTI_VERSION_ROW, SCAN_END };
   // the iteration process is divided into 2 phases:
   // iterating Complement SSTable / iterating Mini SSTable
diff --git a/src/storage/ob_i_store.h b/src/storage/ob_i_store.h
index e13283f..e2e5784 100644
--- a/src/storage/ob_i_store.h
+++ b/src/storage/ob_i_store.h
@@ -833,6 +833,10 @@ public:
   }
   virtual void reuse()
   {}
+  virtual void reset()
+  {}
+  virtual void reuse_try() 
+  {}
   virtual bool is_base_sstable_iter() const
   {
     return false;
diff --git a/src/storage/ob_multiple_get_merge.cpp b/src/storage/ob_multiple_get_merge.cpp
index ebfd26a..c568654 100644
--- a/src/storage/ob_multiple_get_merge.cpp
+++ b/src/storage/ob_multiple_get_merge.cpp
@@ -82,7 +82,7 @@ void ObMultipleGetMerge::reset_with_fuse_row_cache()
     handles_ = nullptr;
   }
   prefetch_cnt_ = 0;
-  reuse_iter_array();
+  reset_iter_array();
 }
 
 void ObMultipleGetMerge::reset()
diff --git a/src/storage/ob_multiple_merge.cpp b/src/storage/ob_multiple_merge.cpp
index 8a01f9d..6c59527 100644
--- a/src/storage/ob_multiple_merge.cpp
+++ b/src/storage/ob_multiple_merge.cpp
@@ -502,6 +502,10 @@ void ObMultipleMerge::reset()
   for (int64_t i = 0; i < iters_.count(); ++i) {
     if (NULL != (iter = iters_.at(i))) {
       iter->~ObStoreRowIterator();
+      if (OB_NOT_NULL(access_ctx_->stmt_allocator_)) {
+        access_ctx_->stmt_allocator_->free(iter);
+      }
+      iter = NULL;
     }
   }
   padding_allocator_.reset();
@@ -539,16 +543,29 @@ void ObMultipleMerge::reuse()
   read_memtable_only_ = false;
 }
 
-void ObMultipleMerge::reuse_iter_array()
+void ObMultipleMerge::reset_iter_array()
 {
   ObStoreRowIterator* iter = NULL;
   for (int64_t i = 0; i < iters_.count(); ++i) {
     if (NULL != (iter = iters_.at(i))) {
       iter->~ObStoreRowIterator();
+      if (OB_NOT_NULL(access_ctx_->stmt_allocator_)) {
+        access_ctx_->stmt_allocator_->free(iter);
+      }
+      iter = NULL;
     }
   }
   iters_.reuse();
 }
+void ObMultipleMerge::reuse_iter_array()
+{
+  ObStoreRowIterator *iter = NULL;
+  for (int64_t i = 0; i < iters_.count(); ++i) {
+    if (NULL != (iter = iters_.at(i))) {
+      iter->reuse_try(); 
+    }
+  }
+}
 
 int ObMultipleMerge::open()
 {
@@ -944,7 +961,7 @@ int ObMultipleMerge::refresh_table_on_demand()
   } else if (need_refresh) {
     if (OB_FAIL(save_curr_rowkey())) {
       STORAGE_LOG(WARN, "fail to save current rowkey", K(ret));
-    } else if (FALSE_IT(reuse_iter_array())) {
+    } else if (FALSE_IT(reset_iter_array())) {
     } else if (OB_FAIL(prepare_read_tables())) {
       STORAGE_LOG(WARN, "fail to prepare read tables", K(ret));
     } else if (OB_FAIL(reset_tables())) {
diff --git a/src/storage/ob_multiple_merge.h b/src/storage/ob_multiple_merge.h
index 12f8cdc..edc20f9 100644
--- a/src/storage/ob_multiple_merge.h
+++ b/src/storage/ob_multiple_merge.h
@@ -80,6 +80,7 @@ protected:
   const ObTableIterParam* get_actual_iter_param(const ObITable* table) const;
   int project_row(const ObStoreRow& unprojected_row, const common::ObIArray<int32_t>* projector,
       const int64_t range_idx_delta, ObStoreRow& projected_row);
+  void reset_iter_array();
   void reuse_iter_array();
   virtual int skip_to_range(const int64_t range_idx);
 
diff --git a/src/storage/ob_sstable.cpp b/src/storage/ob_sstable.cpp
index 13a3f0f..257c02d 100644
--- a/src/storage/ob_sstable.cpp
+++ b/src/storage/ob_sstable.cpp
@@ -1105,14 +1105,14 @@ int ObSSTable::get(const storage::ObTableIterParam& param, storage::ObTableAcces
     ObISSTableRowIterator* row_getter = NULL;
     if (is_multi_version_minor_sstable() && (context.is_multi_version_read(get_upper_trans_version()) ||
                                                 contain_uncommitted_row() || !meta_.has_compact_row_)) {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableMultiVersionRowGetter)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableMultiVersionRowGetter)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
         row_getter = new (buf) ObSSTableMultiVersionRowGetter();
       }
     } else {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowGetter)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowGetter)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
@@ -1163,14 +1163,14 @@ int ObSSTable::multi_get(const ObTableIterParam& param, ObTableAccessContext& co
       ObISSTableRowIterator* row_getter = NULL;
       if (is_multi_version_minor_sstable() && (context.is_multi_version_read(get_upper_trans_version()) ||
                                                   contain_uncommitted_row() || !meta_.has_compact_row_)) {
-        if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableMultiVersionRowMultiGetter)))) {
+        if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableMultiVersionRowMultiGetter)))) {
           ret = OB_ALLOCATE_MEMORY_FAILED;
           STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
         } else {
           row_getter = new (buf) ObSSTableMultiVersionRowMultiGetter();
         }
       } else {
-        if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowMultiGetter)))) {
+        if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowMultiGetter)))) {
           ret = OB_ALLOCATE_MEMORY_FAILED;
           STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
         } else {
@@ -1269,21 +1269,21 @@ int ObSSTable::scan(const ObTableIterParam& param, ObTableAccessContext& context
     void* buf = NULL;
     ObISSTableRowIterator* row_scanner = NULL;
     if (context.query_flag_.is_whole_macro_scan()) {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowWholeScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowWholeScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
         row_scanner = new (buf) ObSSTableRowWholeScanner();
       }
     } else if (is_multi_version_minor_sstable()) {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableMultiVersionRowScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableMultiVersionRowScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
         row_scanner = new (buf) ObSSTableMultiVersionRowScanner();
       }
     } else {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
@@ -1435,14 +1435,14 @@ int ObSSTable::multi_scan(const ObTableIterParam& param, ObTableAccessContext& c
     void* buf = NULL;
     ObISSTableRowIterator* row_scanner = NULL;
     if (is_multi_version_minor_sstable()) {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableMultiVersionRowMultiScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableMultiVersionRowMultiScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
         row_scanner = new (buf) ObSSTableMultiVersionRowMultiScanner();
       }
     } else {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowMultiScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowMultiScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
@@ -2930,7 +2930,7 @@ int ObSSTable::build_exist_iterator(const ObTableIterParam& iter_param, ObTableA
   } else {
     void* buf = NULL;
     ObSSTableRowExister* exister = NULL;
-    if (NULL == (buf = access_context.allocator_->alloc(sizeof(ObSSTableRowExister)))) {
+    if (NULL == (buf = access_context.stmt_allocator_->alloc(sizeof(ObSSTableRowExister)))) {
       ret = OB_ALLOCATE_MEMORY_FAILED;
       STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
     } else {
diff --git a/src/storage/ob_sstable_row_iterator.cpp b/src/storage/ob_sstable_row_iterator.cpp
index 6c261bf..62749f6 100644
--- a/src/storage/ob_sstable_row_iterator.cpp
+++ b/src/storage/ob_sstable_row_iterator.cpp
@@ -672,6 +672,59 @@ void ObSSTableRowIterator::reuse()
   prefetch_handle_depth_ = DEFAULT_PREFETCH_HANDLE_DEPTH;
   prefetch_micro_depth_ = DEFAULT_PREFETCH_MICRO_DEPTH;
 }
+void ObSSTableRowIterator::reuse_try(){
+  ObISSTableRowIterator::reset();
+  read_handles_.reset();
+  micro_handles_.reset();
+  sstable_micro_infos_.reset();
+
+  if (NULL != micro_exister_) {
+    micro_exister_->~ObMicroBlockRowExister();
+    micro_exister_ = NULL;
+  }
+  if (NULL != micro_getter_) {
+    micro_getter_->~ObMicroBlockRowGetter();
+    micro_getter_ = NULL;
+  }
+  if (NULL != micro_scanner_) {
+    micro_scanner_->~ObIMicroBlockRowScanner();
+    micro_scanner_ = NULL;
+  }
+  if (NULL != micro_lock_checker_) {
+    micro_lock_checker_->~ObMicroBlockRowLockChecker();
+    micro_lock_checker_ = NULL;
+  }
+
+  macro_block_iter_.reset();
+  iter_param_ = NULL;
+  access_ctx_ = NULL;
+  sstable_ = NULL;
+  query_range_ = NULL;
+  scan_step_ = 0;
+  is_opened_ = false;
+  is_base_ = false;
+  block_cache_ = NULL;
+  table_type_ = ObITable::MAJOR_SSTABLE;
+  sstable_snapshot_version_ = 0;
+  prefetch_handle_end_ = false;
+  prefetch_block_end_ = false;
+  cur_prefetch_handle_pos_ = 0;
+  cur_fetch_handle_pos_ = 0;
+  cur_read_handle_pos_ = 0;
+  cur_prefetch_micro_pos_ = 0;
+  cur_read_micro_pos_ = 0;
+  cur_micro_idx_ = -1;
+  cur_range_idx_ = -1;
+  io_micro_infos_.reset();
+  micro_info_iter_.reset();
+  block_index_handle_mgr_.reset();
+  block_handle_mgr_.reset();
+  table_store_stat_.reset();
+  skip_ctx_.reset();
+  storage_file_ = nullptr;
+  prefetch_handle_depth_ = DEFAULT_PREFETCH_HANDLE_DEPTH;
+  prefetch_micro_depth_ = DEFAULT_PREFETCH_MICRO_DEPTH;
+}
 
 int ObSSTableRowIterator::get_read_handle(const ObExtStoreRowkey& ext_rowkey, ObSSTableReadHandle& read_handle)
 {
diff --git a/src/storage/ob_sstable_row_iterator.h b/src/storage/ob_sstable_row_iterator.h
index dadb33b..2b46bf9 100644
--- a/src/storage/ob_sstable_row_iterator.h
+++ b/src/storage/ob_sstable_row_iterator.h
@@ -176,6 +176,9 @@ public:
   virtual int get_next_row(const ObStoreRow*& store_row);
   virtual void reset();
   virtual void reuse();
+  virtual void reuse_try(){
+    reset();
+  }
   OB_INLINE bool has_lob_column() const
   {
     return OB_NOT_NULL(lob_reader_);
@@ -321,6 +324,7 @@ public:
   virtual ~ObSSTableRowIterator();
   virtual void reset() override;
   virtual void reuse() override;
+  virtual void reuse_try() override;
   virtual int get_skip_range_ctx(
       ObSSTableReadHandle& read_handle, const int64_t cur_micro_idx, ObSSTableSkipRangeCtx*& skip_ctx);
   int get_row_iter_flag_impl(uint8_t& flag);

patch3

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
diff --git a/src/storage/blocksstable/ob_block_sstable_struct.cpp b/src/storage/blocksstable/ob_block_sstable_struct.cpp
index b455124b..3399b69c 100644
--- a/src/storage/blocksstable/ob_block_sstable_struct.cpp
+++ b/src/storage/blocksstable/ob_block_sstable_struct.cpp
@@ -4569,5 +4569,11 @@ int64_t ObFullMacroBlockMetaEntry::get_serialize_size() const
   return meta_.get_serialize_size() + schema_.get_serialize_size();
 }
 
+void ObSSTableMacroBlockId::reuse()
+{
+  // macro_block_id_.reset();
+  macro_block_id_in_files_ = -1;
+}
+
 }  // namespace blocksstable
 }  // namespace oceanbase
diff --git a/src/storage/blocksstable/ob_block_sstable_struct.h b/src/storage/blocksstable/ob_block_sstable_struct.h
index 42237e0c..a583f15c 100644
--- a/src/storage/blocksstable/ob_block_sstable_struct.h
+++ b/src/storage/blocksstable/ob_block_sstable_struct.h
@@ -859,6 +859,7 @@ struct ObSSTableMacroBlockId {
 
   OB_INLINE bool is_valid() const;
   OB_INLINE void reset();
+  void reuse();
   TO_STRING_KV(K_(macro_block_id), K_(macro_block_id_in_files));
 };
 
@@ -871,6 +872,11 @@ struct ObMacroBlockCtx final {
 
   OB_INLINE bool is_valid() const;
   OB_INLINE void reset();
+  OB_INLINE void reuse() {
+    // file_ctx_ = NULL;
+    // sstable_block_id_.reuse();
+    // sstable_ = nullptr;
+  }
 
   OB_INLINE const blocksstable::MacroBlockId& get_macro_block_id() const;
   TO_STRING_KV(K_(sstable_block_id), KP_(file_ctx));
diff --git a/src/storage/blocksstable/ob_micro_block_row_scanner.cpp b/src/storage/blocksstable/ob_micro_block_row_scanner.cpp
index d6fd2648..6114829b 100644
--- a/src/storage/blocksstable/ob_micro_block_row_scanner.cpp
+++ b/src/storage/blocksstable/ob_micro_block_row_scanner.cpp
@@ -154,6 +154,29 @@ void ObIMicroBlockRowScanner::reset()
   is_inited_ = false;
 }
 
+void ObIMicroBlockRowScanner::reuse()
+{
+  lob_reader_.reuse();
+  has_lob_column_ = false;
+  param_ = NULL;
+  context_ = NULL;
+  range_ = NULL;
+  sstable_ = NULL;
+  column_map_.reuse();
+  // if (OB_NOT_NULL(reader_)) {
+  //   reader_->reset();
+  // }
+  current_ = ObIMicroBlockReader::INVALID_ROW_INDEX;
+  start_ = ObIMicroBlockReader::INVALID_ROW_INDEX;
+  last_ = ObIMicroBlockReader::INVALID_ROW_INDEX;
+  reverse_scan_ = false;
+  is_left_border_ = false;
+  is_right_border_ = false;
+  step_ = 1;
+  // macro_id_.reset();
+  is_inited_ = false;
+}
+
 void ObIMicroBlockRowScanner::rescan()
 {
   range_ = NULL;
@@ -445,7 +468,7 @@ int ObMicroBlockRowScanner::init(const ObTableIterParam& param, ObTableAccessCon
     STORAGE_LOG(WARN, "fail to get projector", K(ret));
   } else if (OB_FAIL(param_->get_column_map(false /*is get*/, column_id_map))) {
     STORAGE_LOG(WARN, "fail to get column id map", K(ret));
-  } else if (OB_FAIL(column_map_.init(*context_->allocator_,
+  } else if (OB_FAIL(column_map_.init(*context_->stmt_allocator_,
                  param_->schema_version_,
                  param_->rowkey_cnt_,
                  0, /*store count*/
@@ -557,6 +580,11 @@ void ObMicroBlockRowScanner::reset()
   ObIMicroBlockRowScanner::reset();
 }
 
+void ObMicroBlockRowScanner::reuse()
+{
+  ObIMicroBlockRowScanner::reuse();
+}
+
 int ObMultiVersionMicroBlockRowScanner::init(
     const ObTableIterParam& param, ObTableAccessContext& context, const ObSSTable* sstable)
 {
@@ -573,7 +601,7 @@ int ObMultiVersionMicroBlockRowScanner::init(
     STORAGE_LOG(WARN, "fail to get projector", K(ret));
   } else if (OB_FAIL(param_->get_column_map(context.use_fuse_row_cache_, column_id_map))) {
     STORAGE_LOG(WARN, "fail to get column id map", K(ret));
-  } else if (OB_FAIL(column_map_.init(*context_->allocator_,
+  } else if (OB_FAIL(column_map_.init(*context_->stmt_allocator_,
                  param_->schema_version_,
                  param_->rowkey_cnt_,
                  0, /*store count*/
@@ -1358,7 +1386,7 @@ int ObMultiVersionMicroBlockMinorMergeRowScanner::init(
     // minor merge should contain 2
     if (OB_FAIL(build_minor_merge_out_cols(*param_, out_cols, expect_multi_version_col_cnt))) {
       STORAGE_LOG(WARN, "fail to build minor merge out columns", K(ret));
-    } else if (OB_FAIL(column_map_.init(*context_->allocator_,
+    } else if (OB_FAIL(column_map_.init(*context_->stmt_allocator_,
                    param_->schema_version_,
                    param_->rowkey_cnt_,
                    0, /*store count*/
diff --git a/src/storage/blocksstable/ob_micro_block_row_scanner.h b/src/storage/blocksstable/ob_micro_block_row_scanner.h
index 89ec845a..76d7587c 100644
--- a/src/storage/blocksstable/ob_micro_block_row_scanner.h
+++ b/src/storage/blocksstable/ob_micro_block_row_scanner.h
@@ -64,6 +64,7 @@ public:
   virtual int get_next_rows(const storage::ObStoreRow*& rows, int64_t& count);
   virtual void reset();
   virtual void rescan();
+  virtual void reuse();
   int alloc_row(ObIAllocator& allocator, const int64_t cell_cnt, storage::ObStoreRow& row);
   virtual int get_cur_micro_row_count(int64_t& row_count) const;
 
@@ -127,6 +128,7 @@ public:
   virtual int open(const MacroBlockId& macro_id, const ObFullMacroBlockMeta& macro_meta,
       const ObMicroBlockData& block_data, const bool is_left_border, const bool is_right_border) override;
   void reset() override;
+  void reuse() override;
 
 protected:
   virtual int inner_get_next_row(const storage::ObStoreRow*& row) override;
diff --git a/src/storage/blocksstable/ob_row_reader.cpp b/src/storage/blocksstable/ob_row_reader.cpp
index 32880944..b02f48d9 100644
--- a/src/storage/blocksstable/ob_row_reader.cpp
+++ b/src/storage/blocksstable/ob_row_reader.cpp
@@ -650,7 +650,9 @@ int ObFlatRowReader::compare_meta_rowkey(const common::ObStoreRowkey& rhs, const
         if (OB_FAIL(read_obj_no_meta(obj_meta, allocator_, obj))) {
           STORAGE_LOG(WARN, "row reader fail to read column.", K(ret), K(i));
         } else {
-          cmp_result = obj.compare(rhs.get_obj_ptr()[i], common::CS_TYPE_INVALID);
+          if (OB_FAIL(obj.compare(rhs.get_obj_ptr()[i], common::CS_TYPE_INVALID, cmp_result))) {
+            return OB_ERR_UNEXPECTED;
+          }
         }
       }  // end for
     }
diff --git a/src/storage/memtable/ob_memtable.cpp b/src/storage/memtable/ob_memtable.cpp
index 3a4d28f7..eec61ec2 100644
--- a/src/storage/memtable/ob_memtable.cpp
+++ b/src/storage/memtable/ob_memtable.cpp
@@ -927,7 +927,7 @@ int ObMemtable::get(const storage::ObTableIterParam& param, storage::ObTableAcce
     const ColumnMap* param_column_map = nullptr;
     if (nullptr == row.row_val_.cells_) {
       if (nullptr ==
-          (row.row_val_.cells_ = static_cast<ObObj*>(context.allocator_->alloc(sizeof(ObObj) * out_cols->count())))) {
+          (row.row_val_.cells_ = static_cast<ObObj*>(context.stmt_allocator_->alloc(sizeof(ObObj) * out_cols->count())))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         TRANS_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
@@ -940,11 +940,11 @@ int ObMemtable::get(const storage::ObTableIterParam& param, storage::ObTableAcce
       TRANS_LOG(WARN, "fail to get column map", K(ret));
     } else if (NULL == param_column_map) {
       void* buf = NULL;
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ColumnMap)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ColumnMap)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         TRANS_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
-        local_map = new (buf) ColumnMap(*context.allocator_);
+        local_map = new (buf) ColumnMap(*context.stmt_allocator_);
         if (OB_FAIL(local_map->init(*out_cols))) {
           TRANS_LOG(WARN, "Fail to build column map, ", K(ret));
         }
@@ -1033,7 +1033,7 @@ int ObMemtable::get(const storage::ObTableIterParam& param, storage::ObTableAcce
     TRANS_LOG(WARN, "invalid argument, ", K(ret), K(param), K(context));
   } else if (OB_FAIL(context.store_ctx_->mem_ctx_->get_trans_status())) {
     TRANS_LOG(WARN, "trans already end", K(ret));
-  } else if (NULL == (get_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableGetIterator))) ||
+  } else if (NULL == (get_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableGetIterator))) ||
              NULL == (get_iter_ptr = new (get_iter_buffer) ObMemtableGetIterator())) {
     TRANS_LOG(WARN, "construct ObMemtableGetIterator fail");
     ret = OB_ALLOCATE_MEMORY_FAILED;
@@ -1082,7 +1082,7 @@ int ObMemtable::scan(const storage::ObTableIterParam& param, storage::ObTableAcc
   } else {
     if (param.is_multi_version_minor_merge_) {
       if (GCONF._enable_sparse_row) {
-        if (NULL == (scan_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableMultiVersionScanSparseIterator))) ||
+        if (NULL == (scan_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableMultiVersionScanSparseIterator))) ||
             NULL == (scan_iter_ptr = new (scan_iter_buffer) ObMemtableMultiVersionScanSparseIterator())) {
           TRANS_LOG(WARN,
               "construct ObMemtableMultiVersionScanSparseIterator fail",
@@ -1099,7 +1099,7 @@ int ObMemtable::scan(const storage::ObTableIterParam& param, storage::ObTableAcc
           TRANS_LOG(WARN, "scan iter init fail", "ret", ret, K(real_range), K(param), K(context));
         }
       } else {
-        if (NULL == (scan_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableMultiVersionScanIterator))) ||
+        if (NULL == (scan_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableMultiVersionScanIterator))) ||
             NULL == (scan_iter_ptr = new (scan_iter_buffer) ObMemtableMultiVersionScanIterator())) {
           TRANS_LOG(WARN,
               "construct ObMemtableScanIterator fail",
@@ -1117,7 +1117,7 @@ int ObMemtable::scan(const storage::ObTableIterParam& param, storage::ObTableAcc
         }
       }
     } else {
-      if (NULL == (scan_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableScanIterator))) ||
+      if (NULL == (scan_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableScanIterator))) ||
           NULL == (scan_iter_ptr = new (scan_iter_buffer) ObMemtableScanIterator())) {
         TRANS_LOG(WARN,
             "construct ObMemtableScanIterator fail",
@@ -1162,7 +1162,7 @@ int ObMemtable::multi_get(const storage::ObTableIterParam& param, storage::ObTab
     TRANS_LOG(WARN, "invalid argument, ", K(ret), K(param), K(context), K(rowkeys));
   } else if (OB_FAIL(context.store_ctx_->mem_ctx_->get_trans_status())) {
     TRANS_LOG(WARN, "trans already end", K(ret));
-  } else if (NULL == (mget_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableMGetIterator))) ||
+  } else if (NULL == (mget_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableMGetIterator))) ||
              NULL == (mget_iter_ptr = new (mget_iter_buffer) ObMemtableMGetIterator())) {
     TRANS_LOG(WARN,
         "construct ObMemtableMGetIterator fail",
@@ -1212,7 +1212,7 @@ int ObMemtable::multi_scan(const storage::ObTableIterParam& param, storage::ObTa
     TRANS_LOG(WARN, "invalid argument, ", K(ret), K(param), K(context), K(ranges));
   } else if (OB_FAIL(context.store_ctx_->mem_ctx_->get_trans_status())) {
     TRANS_LOG(WARN, "trans already end", K(ret));
-  } else if (NULL == (mscan_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableMScanIterator))) ||
+  } else if (NULL == (mscan_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableMScanIterator))) ||
              NULL == (mscan_iter_ptr = new (mscan_iter_buffer) ObMemtableMScanIterator())) {
     TRANS_LOG(WARN,
         "construct ObMemtableMScanIterator fail",
diff --git a/src/storage/ob_micro_block_handle_mgr.cpp b/src/storage/ob_micro_block_handle_mgr.cpp
index 028a2018..39ea7cca 100644
--- a/src/storage/ob_micro_block_handle_mgr.cpp
+++ b/src/storage/ob_micro_block_handle_mgr.cpp
@@ -45,6 +45,17 @@ void ObMicroBlockDataHandle::reset()
   io_handle_.reset();
 }
 
+void ObMicroBlockDataHandle::reuse()
+{
+  block_state_ = ObSSTableMicroBlockState::UNKNOWN_STATE;
+  table_id_ = 0;
+  block_ctx_.reset();
+  block_index_ = -1;
+  micro_info_.reset();
+  cache_handle_.reset();
+  io_handle_.reset();
+}
+
 int ObMicroBlockDataHandle::get_block_data(
     ObMacroBlockReader& block_reader, ObStorageFile* storage_file, ObMicroBlockData& block_data)
 {
diff --git a/src/storage/ob_micro_block_handle_mgr.h b/src/storage/ob_micro_block_handle_mgr.h
index 37f6d005..1ff90688 100644
--- a/src/storage/ob_micro_block_handle_mgr.h
+++ b/src/storage/ob_micro_block_handle_mgr.h
@@ -30,6 +30,7 @@ struct ObMicroBlockDataHandle {
   ObMicroBlockDataHandle();
   virtual ~ObMicroBlockDataHandle();
   void reset();
+  void reuse();
   int get_block_data(blocksstable::ObMacroBlockReader& block_reader, blocksstable::ObStorageFile* storage_file,
       blocksstable::ObMicroBlockData& block_data);
   TO_STRING_KV(
diff --git a/src/storage/ob_multiple_merge.cpp b/src/storage/ob_multiple_merge.cpp
index 8a01f9de..20893d3a 100644
--- a/src/storage/ob_multiple_merge.cpp
+++ b/src/storage/ob_multiple_merge.cpp
@@ -485,7 +485,7 @@ int ObMultipleMerge::get_next_row(ObStoreRow*& row)
         access_ctx_->table_scan_stat_->row_cache_hit_cnt_ += access_ctx_->access_stat_.row_cache_hit_cnt_;
         access_ctx_->table_scan_stat_->row_cache_miss_cnt_ += access_ctx_->access_stat_.row_cache_miss_cnt_;
       }
-      report_table_store_stat();
+      // report_table_store_stat();
     }
     if (OB_SUCC(ret)) {
       if (NULL != access_ctx_->table_scan_stat_) {
diff --git a/src/storage/ob_multiple_merge.h b/src/storage/ob_multiple_merge.h
index 12f8cdc2..a5a049ea 100644
--- a/src/storage/ob_multiple_merge.h
+++ b/src/storage/ob_multiple_merge.h
@@ -80,7 +80,7 @@ protected:
   const ObTableIterParam* get_actual_iter_param(const ObITable* table) const;
   int project_row(const ObStoreRow& unprojected_row, const common::ObIArray<int32_t>* projector,
       const int64_t range_idx_delta, ObStoreRow& projected_row);
-  void reuse_iter_array();
+  virtual void reuse_iter_array();
   virtual int skip_to_range(const int64_t range_idx);
 
 private:
diff --git a/src/storage/ob_multiple_scan_merge.cpp b/src/storage/ob_multiple_scan_merge.cpp
index 958c335e..750f80a3 100644
--- a/src/storage/ob_multiple_scan_merge.cpp
+++ b/src/storage/ob_multiple_scan_merge.cpp
@@ -184,6 +184,16 @@ void ObMultipleScanMerge::reuse()
   return ObMultipleScanMergeImpl::reuse();
 }
 
+void ObMultipleScanMerge::reuse_iter_array()
+{
+  ObStoreRowIterator* iter = NULL;
+  for (int64_t i = 0; i < iters_.count(); ++i) {
+    if (NULL != (iter = iters_.at(i))) {
+      iter->reuse();
+    }
+  }
+}
+
 int ObMultipleScanMerge::inner_get_next_row(ObStoreRow& row)
 {
   int ret = OB_SUCCESS;
diff --git a/src/storage/ob_multiple_scan_merge.h b/src/storage/ob_multiple_scan_merge.h
index 455860eb..a3bb83be 100644
--- a/src/storage/ob_multiple_scan_merge.h
+++ b/src/storage/ob_multiple_scan_merge.h
@@ -28,6 +28,7 @@ public:
   int open(const common::ObExtStoreRange& range);
   virtual void reset() override;
   virtual void reuse() override;
+  virtual void reuse_iter_array() override;
   inline void set_iter_del_row(const bool iter_del_row)
   {
     iter_del_row_ = iter_del_row;
diff --git a/src/storage/ob_old_sstable.cpp b/src/storage/ob_old_sstable.cpp
index d41f0c42..6d4e639a 100644
--- a/src/storage/ob_old_sstable.cpp
+++ b/src/storage/ob_old_sstable.cpp
@@ -861,21 +861,21 @@ int ObOldSSTable::scan(const ObTableIterParam& param, ObTableAccessContext& cont
     void* buf = NULL;
     ObISSTableRowIterator* row_scanner = NULL;
     if (context.query_flag_.is_whole_macro_scan()) {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowWholeScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowWholeScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
         row_scanner = new (buf) ObSSTableRowWholeScanner();
       }
     } else if (is_multi_version_minor_sstable()) {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableMultiVersionRowScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableMultiVersionRowScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
         row_scanner = new (buf) ObSSTableMultiVersionRowScanner();
       }
     } else {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
diff --git a/src/storage/ob_sstable.cpp b/src/storage/ob_sstable.cpp
index 13a3f0fa..138d06a0 100644
--- a/src/storage/ob_sstable.cpp
+++ b/src/storage/ob_sstable.cpp
@@ -1269,21 +1269,21 @@ int ObSSTable::scan(const ObTableIterParam& param, ObTableAccessContext& context
     void* buf = NULL;
     ObISSTableRowIterator* row_scanner = NULL;
     if (context.query_flag_.is_whole_macro_scan()) {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowWholeScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowWholeScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
         row_scanner = new (buf) ObSSTableRowWholeScanner();
       }
     } else if (is_multi_version_minor_sstable()) {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableMultiVersionRowScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableMultiVersionRowScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
         row_scanner = new (buf) ObSSTableMultiVersionRowScanner();
       }
     } else {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
diff --git a/src/storage/ob_sstable_row_iterator.cpp b/src/storage/ob_sstable_row_iterator.cpp
index 6c261bfd..9d9e822d 100644
--- a/src/storage/ob_sstable_row_iterator.cpp
+++ b/src/storage/ob_sstable_row_iterator.cpp
@@ -413,6 +413,7 @@ ObSSTableRowIterator::ObSSTableRowIterator()
       micro_getter_(NULL),
       micro_lock_checker_(NULL),
       micro_scanner_(NULL),
+      micro_scanner_inited_(false),
       read_handles_(),
       prefetch_handle_end_(false),
       prefetch_block_end_(false),
@@ -469,13 +470,13 @@ int ObSSTableRowIterator::inner_open(
     STORAGE_LOG(WARN, "Unexpected error, ", K(ret), K_(read_handle_cnt), K_(micro_handle_cnt));
   } else if (OB_FAIL(init_handle_mgr(iter_param, access_ctx, query_range))) {
     STORAGE_LOG(WARN, "fail to init handle mgr", K(ret), K(iter_param), K(access_ctx));
-  } else if (OB_FAIL(read_handles_.reserve(*access_ctx.allocator_, read_handle_cnt_))) {
+  } else if (OB_FAIL(read_handles_.reserve(*access_ctx.stmt_allocator_, read_handle_cnt_))) {
     STORAGE_LOG(WARN, "failed to reserve read handles", K(ret), K_(read_handle_cnt));
-  } else if (OB_FAIL(micro_handles_.reserve(*access_ctx.allocator_, micro_handle_cnt_))) {
+  } else if (OB_FAIL(micro_handles_.reserve(*access_ctx.stmt_allocator_, micro_handle_cnt_))) {
     STORAGE_LOG(WARN, "failed to reserve micro handles", K(ret), K_(micro_handle_cnt));
-  } else if (OB_FAIL(sstable_micro_infos_.reserve(*access_ctx.allocator_, micro_handle_cnt_))) {
+  } else if (OB_FAIL(sstable_micro_infos_.reserve(*access_ctx.stmt_allocator_, micro_handle_cnt_))) {
     STORAGE_LOG(WARN, "failed to reserve sstable micro infos", K(ret), K_(micro_handle_cnt));
-  } else if (OB_FAIL(sorted_sstable_micro_infos_.reserve(*access_ctx.allocator_, micro_handle_cnt_))) {
+  } else if (OB_FAIL(sorted_sstable_micro_infos_.reserve(*access_ctx.stmt_allocator_, micro_handle_cnt_))) {
     STORAGE_LOG(WARN, "failed to reserve sorted sstable micro infos", K(ret), K_(micro_handle_cnt));
   } else {
     sstable_ = static_cast<ObSSTable*>(table);
@@ -643,8 +644,31 @@ void ObSSTableRowIterator::reset()
 void ObSSTableRowIterator::reuse()
 {
   ObISSTableRowIterator::reuse();
+  // NOTE: Seems unnecessary
+  // read_handles_.reset();
+  // NOTE: Seems unnecessary
+  // micro_handles_.reuse();
+  sstable_micro_infos_.reuse();
+  // NOTE: Seems unnecessary
+  // sorted_sstable_micro_infos_.reuse();
+
+  if (NULL != micro_exister_) {
+    micro_exister_->~ObMicroBlockRowExister();
+    micro_exister_ = NULL;
+  }
+  if (NULL != micro_getter_) {
+    micro_getter_->~ObMicroBlockRowGetter();
+    micro_getter_ = NULL;
+  }
   if (NULL != micro_scanner_) {
-    micro_scanner_->rescan();
+    micro_scanner_->reuse();
+    micro_scanner_inited_ = false;
+    // micro_scanner_->~ObIMicroBlockRowScanner();
+    // micro_scanner_ = NULL;
+  }
+  if (NULL != micro_lock_checker_) {
+    micro_lock_checker_->~ObMicroBlockRowLockChecker();
+    micro_lock_checker_ = NULL;
   }
 
   macro_block_iter_.reset();
@@ -664,8 +688,8 @@ void ObSSTableRowIterator::reuse()
   cur_range_idx_ = -1;
   io_micro_infos_.reuse();
   micro_info_iter_.reuse();
-  block_index_handle_mgr_.reset();
-  block_handle_mgr_.reset();
+  // block_index_handle_mgr_.reset();
+  // block_handle_mgr_.reset();
   table_store_stat_.reuse();
   skip_ctx_.reset();
   storage_file_ = nullptr;
@@ -1572,23 +1596,27 @@ int ObSSTableRowIterator::open_cur_micro_block(ObSSTableReadHandle& read_handle,
   if (NULL == micro_scanner_) {
     // alloc scanner
     if (!sstable_->is_multi_version_minor_sstable()) {
-      if (NULL == (buf = access_ctx_->allocator_->alloc(sizeof(ObMicroBlockRowScanner)))) {
+      if (NULL == (buf = access_ctx_->stmt_allocator_->alloc(sizeof(ObMicroBlockRowScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory for micro block scanner, ", K(ret));
       } else {
         micro_scanner_ = new (buf) ObMicroBlockRowScanner();
       }
     } else {
-      if (NULL == (buf = access_ctx_->allocator_->alloc(sizeof(ObMultiVersionMicroBlockRowScanner)))) {
+      if (NULL == (buf = access_ctx_->stmt_allocator_->alloc(sizeof(ObMultiVersionMicroBlockRowScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory for micro block scanner, ", K(ret));
       } else {
         micro_scanner_ = new (buf) ObMultiVersionMicroBlockRowScanner();
       }
     }
+  }
+  if (!micro_scanner_inited_) {
     if (OB_SUCC(ret)) {
       if (OB_FAIL(micro_scanner_->init(*iter_param_, *access_ctx_, sstable_))) {
         STORAGE_LOG(WARN, "Fail to init micro scanner, ", K(ret), K(read_handle));
+      } else {
+        micro_scanner_inited_ = true;
       }
     }
   }
@@ -1690,7 +1718,7 @@ int ObSSTableRowIterator::init_handle_mgr(
     ret = OB_ERR_UNEXPECTED;
     STORAGE_LOG(WARN, "range count should be greater than 0", K(ret), K(range_count));
   } else if (1 == range_count) {
-    is_multi = false;
+    is_multi = true; // NOTE: force enable cache of *_handle_mgr_ for nlj benchmark
     is_ordered = false;
   } else {
     is_multi = true;
@@ -1701,9 +1729,9 @@ int ObSSTableRowIterator::init_handle_mgr(
             range_count >= USE_HANDLE_CACHE_RANGE_COUNT_THRESHOLD);
   }
   if (OB_SUCC(ret)) {
-    if (OB_FAIL(block_handle_mgr_.init(is_multi, true, *access_ctx.allocator_))) {
+    if (!block_handle_mgr_.is_inited() && OB_FAIL(block_handle_mgr_.init(is_multi, true, *access_ctx.stmt_allocator_))) {
       STORAGE_LOG(WARN, "failed to init block handle mgr", K(ret), K(is_multi), K(is_ordered));
-    } else if (OB_FAIL(block_index_handle_mgr_.init(is_multi, is_ordered, *access_ctx.allocator_))) {
+    } else if (!block_index_handle_mgr_.is_inited() && OB_FAIL(block_index_handle_mgr_.init(is_multi, is_ordered, *access_ctx.stmt_allocator_))) {
       STORAGE_LOG(WARN, "failed to init block index handle mgr", K(ret), K(is_multi), K(is_ordered));
     }
   }
diff --git a/src/storage/ob_sstable_row_iterator.h b/src/storage/ob_sstable_row_iterator.h
index dadb33b8..25f137ff 100644
--- a/src/storage/ob_sstable_row_iterator.h
+++ b/src/storage/ob_sstable_row_iterator.h
@@ -104,6 +104,12 @@ struct ObSSTableMicroBlockInfo {
   blocksstable::ObMicroBlockInfo micro_info_;
   int64_t micro_idx_;
   bool is_skip_;
+  void reuse() {
+    macro_ctx_.reuse();
+    // micro_info_.reset();
+    micro_idx_ = -1;
+    is_skip_ = false;
+  }
 };
 
 class ObSSTableMicroBlockInfoCmp {
@@ -250,6 +256,11 @@ public:
     array_ = nullptr;
     capacity_ = 0;
   }
+  inline void reuse() {
+    for (int64_t i = 0; i < capacity_; ++i) {
+      array_[i].reuse();
+    }
+  }
   int reserve(common::ObArenaAllocator& allocator, const int64_t count)
   {
     int ret = common::OB_SUCCESS;
@@ -406,6 +417,7 @@ private:
   blocksstable::ObMicroBlockRowGetter* micro_getter_;
   blocksstable::ObMicroBlockRowLockChecker* micro_lock_checker_;
   blocksstable::ObIMicroBlockRowScanner* micro_scanner_;
+  bool micro_scanner_inited_;
   ReadHandleArray read_handles_;
   bool prefetch_handle_end_;
   bool prefetch_block_end_;

patch4

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
diff --git a/diff b/diff
index 3a4d28f..ba7d295 100644
--- a/src/storage/memtable/ob_memtable.cpp
+++ b/src/storage/memtable/ob_memtable.cpp
@@ -1033,7 +1033,7 @@ int ObMemtable::get(const storage::ObTableIterParam& param, storage::ObTableAcce
     TRANS_LOG(WARN, "invalid argument, ", K(ret), K(param), K(context));
   } else if (OB_FAIL(context.store_ctx_->mem_ctx_->get_trans_status())) {
     TRANS_LOG(WARN, "trans already end", K(ret));
-  } else if (NULL == (get_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableGetIterator))) ||
+  } else if (NULL == (get_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableGetIterator))) ||
              NULL == (get_iter_ptr = new (get_iter_buffer) ObMemtableGetIterator())) {
     TRANS_LOG(WARN, "construct ObMemtableGetIterator fail");
     ret = OB_ALLOCATE_MEMORY_FAILED;
@@ -1082,7 +1082,7 @@ int ObMemtable::scan(const storage::ObTableIterParam& param, storage::ObTableAcc
   } else {
     if (param.is_multi_version_minor_merge_) {
       if (GCONF._enable_sparse_row) {
-        if (NULL == (scan_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableMultiVersionScanSparseIterator))) ||
+        if (NULL == (scan_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableMultiVersionScanSparseIterator))) ||
             NULL == (scan_iter_ptr = new (scan_iter_buffer) ObMemtableMultiVersionScanSparseIterator())) {
           TRANS_LOG(WARN,
               "construct ObMemtableMultiVersionScanSparseIterator fail",
@@ -1099,7 +1099,7 @@ int ObMemtable::scan(const storage::ObTableIterParam& param, storage::ObTableAcc
           TRANS_LOG(WARN, "scan iter init fail", "ret", ret, K(real_range), K(param), K(context));
         }
       } else {
-        if (NULL == (scan_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableMultiVersionScanIterator))) ||
+        if (NULL == (scan_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableMultiVersionScanIterator))) ||
             NULL == (scan_iter_ptr = new (scan_iter_buffer) ObMemtableMultiVersionScanIterator())) {
           TRANS_LOG(WARN,
               "construct ObMemtableScanIterator fail",
@@ -1117,7 +1117,7 @@ int ObMemtable::scan(const storage::ObTableIterParam& param, storage::ObTableAcc
         }
       }
     } else {
-      if (NULL == (scan_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableScanIterator))) ||
+      if (NULL == (scan_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableScanIterator))) ||
           NULL == (scan_iter_ptr = new (scan_iter_buffer) ObMemtableScanIterator())) {
         TRANS_LOG(WARN,
             "construct ObMemtableScanIterator fail",
@@ -1162,7 +1162,7 @@ int ObMemtable::multi_get(const storage::ObTableIterParam& param, storage::ObTab
     TRANS_LOG(WARN, "invalid argument, ", K(ret), K(param), K(context), K(rowkeys));
   } else if (OB_FAIL(context.store_ctx_->mem_ctx_->get_trans_status())) {
     TRANS_LOG(WARN, "trans already end", K(ret));
-  } else if (NULL == (mget_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableMGetIterator))) ||
+  } else if (NULL == (mget_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableMGetIterator))) ||
              NULL == (mget_iter_ptr = new (mget_iter_buffer) ObMemtableMGetIterator())) {
     TRANS_LOG(WARN,
         "construct ObMemtableMGetIterator fail",
@@ -1212,7 +1212,7 @@ int ObMemtable::multi_scan(const storage::ObTableIterParam& param, storage::ObTa
     TRANS_LOG(WARN, "invalid argument, ", K(ret), K(param), K(context), K(ranges));
   } else if (OB_FAIL(context.store_ctx_->mem_ctx_->get_trans_status())) {
     TRANS_LOG(WARN, "trans already end", K(ret));
-  } else if (NULL == (mscan_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableMScanIterator))) ||
+  } else if (NULL == (mscan_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableMScanIterator))) ||
              NULL == (mscan_iter_ptr = new (mscan_iter_buffer) ObMemtableMScanIterator())) {
     TRANS_LOG(WARN,
         "construct ObMemtableMScanIterator fail",
diff --git a/src/storage/memtable/ob_memtable_iterator.h b/src/storage/memtable/ob_memtable_iterator.h
index ef3a552..0601d87 100644
--- a/src/storage/memtable/ob_memtable_iterator.h
+++ b/src/storage/memtable/ob_memtable_iterator.h
@@ -100,6 +100,9 @@ public:
   {
     reset();
   }
+  virtual void reuse_try() override{
+    reset();
+  }
 
 private:
   // means GETITER
@@ -135,6 +138,9 @@ public:
   {
     reset();
   }
+  virtual void reuse_try() override{
+    reset();
+  }
   virtual int get_gap_end(int64_t& range_idx, const common::ObStoreRowkey*& gap_key, int64_t& gap_size) override
   {
     int ret = common::OB_SUCCESS;
@@ -216,6 +222,9 @@ public:
   {
     reset();
   }
+  virtual void reuse_try() override{
+    reset();
+  }
 
 public:
   static const int64_t ROW_ALLOCATOR_PAGE_SIZE = common::OB_MALLOC_NORMAL_BLOCK_SIZE;
@@ -254,6 +263,9 @@ public:
   {
     reset();
   }
+  virtual void reuse_try() override{
+    reset();
+  }
   virtual int skip_range(int64_t range_idx, const common::ObStoreRowkey* gap_key, const bool include_gap_key) override;
   virtual int get_gap_end(int64_t& range_idx, const common::ObStoreRowkey*& gap_key, int64_t& gap_size) override
   {
@@ -299,6 +311,9 @@ public:
   {
     reset();
   }
+  virtual void reuse_try() override{
+    reset();
+  }
   enum ScanState { SCAN_BEGIN, SCAN_UNCOMMITTED_ROW, SCAN_COMPACT_ROW, SCAN_MULTI_VERSION_ROW, SCAN_END };
   // the iteration process is divided into 2 phases:
   // iterating Complement SSTable / iterating Mini SSTable
diff --git a/src/storage/ob_handle_mgr.h b/src/storage/ob_handle_mgr.h
index 45fdfcb..4cf6617 100644
--- a/src/storage/ob_handle_mgr.h
+++ b/src/storage/ob_handle_mgr.h
@@ -21,7 +21,7 @@ namespace storage {
 template <typename Handle, typename Key, int64_t N>
 class ObHandleMgr {
 public:
-  ObHandleMgr() : is_inited_(false), is_multi_(false), is_ordered_(false), last_handle_(NULL), handle_cache_(NULL)
+  ObHandleMgr() : is_inited_(false), is_multi_(false), is_ordered_(false), last_handle_(NULL), handle_cache_(NULL), stmt_allocator_(NULL)
   {}
   virtual ~ObHandleMgr()
   {
@@ -33,21 +33,50 @@ public:
     is_ordered_ = false;
     if (NULL != last_handle_) {
       last_handle_->~Handle();
+      if (stmt_allocator_ != NULL){
+        stmt_allocator_->free(last_handle_); 
+      }
       last_handle_ = NULL;
     }
     if (NULL != handle_cache_) {
       handle_cache_->~ObHandleCache();
+      if (stmt_allocator_ != NULL){
+        stmt_allocator_->free(handle_cache_); 
+      } 
       handle_cache_ = NULL;
     }
     is_inited_ = false;
+    stmt_allocator_ = NULL;
   }
   int init(const bool is_multi, const bool is_ordered, common::ObArenaAllocator& allocator)
   {
     int ret = common::OB_SUCCESS;
     void* buf = NULL;
     if (OB_UNLIKELY(is_inited_)) {
-      ret = common::OB_INIT_TWICE;
-      STORAGE_LOG(WARN, "handle mgr is inited twice", K(ret));
+      if (is_multi){
+        if (is_ordered) {
+          if (last_handle_ == NULL){
+            if (OB_ISNULL(buf = allocator.alloc(sizeof(Handle)))) {
+              ret = common::OB_ALLOCATE_MEMORY_FAILED;
+              STORAGE_LOG(WARN, "failed to allocate last handle");
+            } else {
+              last_handle_ = new (buf) Handle();
+            }
+          }
+        } else{
+          if (handle_cache_ == NULL){
+            if (OB_ISNULL(buf = allocator.alloc(sizeof(HandleCache)))) {
+              ret = common::OB_ALLOCATE_MEMORY_FAILED;
+              STORAGE_LOG(WARN, "failed to allocate last handle");
+            } else {
+              handle_cache_ = new (buf) HandleCache();
+            }
+          }
+          
+        }
+      }     
+//      ret = common::OB_INIT_TWICE;
+//      STORAGE_LOG(WARN, "handle mgr is inited twice", K(ret));
     } else if (is_multi) {
       if (is_ordered) {
         if (OB_ISNULL(buf = allocator.alloc(sizeof(Handle)))) {
@@ -69,6 +98,7 @@ public:
       is_multi_ = is_multi;
       is_ordered_ = is_ordered;
       is_inited_ = true;
+      stmt_allocator_ = &allocator;
     }
     return ret;
   }
@@ -85,6 +115,7 @@ protected:
   bool is_ordered_;
   Handle* last_handle_;
   HandleCache* handle_cache_;
+  common::ObArenaAllocator* stmt_allocator_;
 };
 
 }  // namespace storage
diff --git a/src/storage/ob_i_store.h b/src/storage/ob_i_store.h
index e13283f..e2e5784 100644
--- a/src/storage/ob_i_store.h
+++ b/src/storage/ob_i_store.h
@@ -833,6 +833,10 @@ public:
   }
   virtual void reuse()
   {}
+  virtual void reset()
+  {}
+  virtual void reuse_try() 
+  {}
   virtual bool is_base_sstable_iter() const
   {
     return false;
diff --git a/src/storage/ob_multiple_get_merge.cpp b/src/storage/ob_multiple_get_merge.cpp
index ebfd26a..c568654 100644
--- a/src/storage/ob_multiple_get_merge.cpp
+++ b/src/storage/ob_multiple_get_merge.cpp
@@ -82,7 +82,7 @@ void ObMultipleGetMerge::reset_with_fuse_row_cache()
     handles_ = nullptr;
   }
   prefetch_cnt_ = 0;
-  reuse_iter_array();
+  reset_iter_array();
 }
 
 void ObMultipleGetMerge::reset()
diff --git a/src/storage/ob_multiple_merge.cpp b/src/storage/ob_multiple_merge.cpp
index 8a01f9d..6c59527 100644
--- a/src/storage/ob_multiple_merge.cpp
+++ b/src/storage/ob_multiple_merge.cpp
@@ -502,6 +502,10 @@ void ObMultipleMerge::reset()
   for (int64_t i = 0; i < iters_.count(); ++i) {
     if (NULL != (iter = iters_.at(i))) {
       iter->~ObStoreRowIterator();
+      if (OB_NOT_NULL(access_ctx_->stmt_allocator_)) {
+        access_ctx_->stmt_allocator_->free(iter);
+      }
+      iter = NULL;
     }
   }
   padding_allocator_.reset();
@@ -539,16 +543,29 @@ void ObMultipleMerge::reuse()
   read_memtable_only_ = false;
 }
 
-void ObMultipleMerge::reuse_iter_array()
+void ObMultipleMerge::reset_iter_array()
 {
   ObStoreRowIterator* iter = NULL;
   for (int64_t i = 0; i < iters_.count(); ++i) {
     if (NULL != (iter = iters_.at(i))) {
       iter->~ObStoreRowIterator();
+      if (OB_NOT_NULL(access_ctx_->stmt_allocator_)) {
+        access_ctx_->stmt_allocator_->free(iter);
+      }
+      iter = NULL;
     }
   }
   iters_.reuse();
 }
+void ObMultipleMerge::reuse_iter_array()
+{
+  ObStoreRowIterator *iter = NULL;
+  for (int64_t i = 0; i < iters_.count(); ++i) {
+    if (NULL != (iter = iters_.at(i))) {
+      iter->reuse_try(); 
+    }
+  }
+}
 
 int ObMultipleMerge::open()
 {
@@ -944,7 +961,7 @@ int ObMultipleMerge::refresh_table_on_demand()
   } else if (need_refresh) {
     if (OB_FAIL(save_curr_rowkey())) {
       STORAGE_LOG(WARN, "fail to save current rowkey", K(ret));
-    } else if (FALSE_IT(reuse_iter_array())) {
+    } else if (FALSE_IT(reset_iter_array())) {
     } else if (OB_FAIL(prepare_read_tables())) {
       STORAGE_LOG(WARN, "fail to prepare read tables", K(ret));
     } else if (OB_FAIL(reset_tables())) {
diff --git a/src/storage/ob_multiple_merge.h b/src/storage/ob_multiple_merge.h
index 12f8cdc..edc20f9 100644
--- a/src/storage/ob_multiple_merge.h
+++ b/src/storage/ob_multiple_merge.h
@@ -80,6 +80,7 @@ protected:
   const ObTableIterParam* get_actual_iter_param(const ObITable* table) const;
   int project_row(const ObStoreRow& unprojected_row, const common::ObIArray<int32_t>* projector,
       const int64_t range_idx_delta, ObStoreRow& projected_row);
+  void reset_iter_array();
   void reuse_iter_array();
   virtual int skip_to_range(const int64_t range_idx);
 
diff --git a/src/storage/ob_sstable.cpp b/src/storage/ob_sstable.cpp
index 13a3f0f..257c02d 100644
--- a/src/storage/ob_sstable.cpp
+++ b/src/storage/ob_sstable.cpp
@@ -1105,14 +1105,14 @@ int ObSSTable::get(const storage::ObTableIterParam& param, storage::ObTableAcces
     ObISSTableRowIterator* row_getter = NULL;
     if (is_multi_version_minor_sstable() && (context.is_multi_version_read(get_upper_trans_version()) ||
                                                 contain_uncommitted_row() || !meta_.has_compact_row_)) {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableMultiVersionRowGetter)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableMultiVersionRowGetter)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
         row_getter = new (buf) ObSSTableMultiVersionRowGetter();
       }
     } else {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowGetter)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowGetter)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
@@ -1163,14 +1163,14 @@ int ObSSTable::multi_get(const ObTableIterParam& param, ObTableAccessContext& co
       ObISSTableRowIterator* row_getter = NULL;
       if (is_multi_version_minor_sstable() && (context.is_multi_version_read(get_upper_trans_version()) ||
                                                   contain_uncommitted_row() || !meta_.has_compact_row_)) {
-        if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableMultiVersionRowMultiGetter)))) {
+        if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableMultiVersionRowMultiGetter)))) {
           ret = OB_ALLOCATE_MEMORY_FAILED;
           STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
         } else {
           row_getter = new (buf) ObSSTableMultiVersionRowMultiGetter();
         }
       } else {
-        if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowMultiGetter)))) {
+        if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowMultiGetter)))) {
           ret = OB_ALLOCATE_MEMORY_FAILED;
           STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
         } else {
@@ -1269,21 +1269,21 @@ int ObSSTable::scan(const ObTableIterParam& param, ObTableAccessContext& context
     void* buf = NULL;
     ObISSTableRowIterator* row_scanner = NULL;
     if (context.query_flag_.is_whole_macro_scan()) {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowWholeScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowWholeScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
         row_scanner = new (buf) ObSSTableRowWholeScanner();
       }
     } else if (is_multi_version_minor_sstable()) {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableMultiVersionRowScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableMultiVersionRowScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
         row_scanner = new (buf) ObSSTableMultiVersionRowScanner();
       }
     } else {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
@@ -1435,14 +1435,14 @@ int ObSSTable::multi_scan(const ObTableIterParam& param, ObTableAccessContext& c
     void* buf = NULL;
     ObISSTableRowIterator* row_scanner = NULL;
     if (is_multi_version_minor_sstable()) {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableMultiVersionRowMultiScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableMultiVersionRowMultiScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
         row_scanner = new (buf) ObSSTableMultiVersionRowMultiScanner();
       }
     } else {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowMultiScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowMultiScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
@@ -2930,7 +2930,7 @@ int ObSSTable::build_exist_iterator(const ObTableIterParam& iter_param, ObTableA
   } else {
     void* buf = NULL;
     ObSSTableRowExister* exister = NULL;
-    if (NULL == (buf = access_context.allocator_->alloc(sizeof(ObSSTableRowExister)))) {
+    if (NULL == (buf = access_context.stmt_allocator_->alloc(sizeof(ObSSTableRowExister)))) {
       ret = OB_ALLOCATE_MEMORY_FAILED;
       STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
     } else {
diff --git a/src/storage/ob_sstable_row_iterator.cpp b/src/storage/ob_sstable_row_iterator.cpp
index 6c261bf..6170781 100644
--- a/src/storage/ob_sstable_row_iterator.cpp
+++ b/src/storage/ob_sstable_row_iterator.cpp
@@ -405,6 +405,7 @@ ObSSTableRowIterator::ObSSTableRowIterator()
       storage_file_(nullptr),
       is_opened_(false),
       is_base_(false),
+      is_rescan_(false),
       block_cache_(NULL),
       table_type_(ObITable::MAJOR_SSTABLE),
       sstable_snapshot_version_(0),
@@ -469,9 +470,9 @@ int ObSSTableRowIterator::inner_open(
     STORAGE_LOG(WARN, "Unexpected error, ", K(ret), K_(read_handle_cnt), K_(micro_handle_cnt));
   } else if (OB_FAIL(init_handle_mgr(iter_param, access_ctx, query_range))) {
     STORAGE_LOG(WARN, "fail to init handle mgr", K(ret), K(iter_param), K(access_ctx));
-  } else if (OB_FAIL(read_handles_.reserve(*access_ctx.allocator_, read_handle_cnt_))) {
+  } else if (OB_FAIL(read_handles_.reserve(*access_ctx.stmt_allocator_, read_handle_cnt_))) {
     STORAGE_LOG(WARN, "failed to reserve read handles", K(ret), K_(read_handle_cnt));
-  } else if (OB_FAIL(micro_handles_.reserve(*access_ctx.allocator_, micro_handle_cnt_))) {
+  } else if (OB_FAIL(micro_handles_.reserve(*access_ctx.stmt_allocator_, micro_handle_cnt_))) {
     STORAGE_LOG(WARN, "failed to reserve micro handles", K(ret), K_(micro_handle_cnt));
   } else if (OB_FAIL(sstable_micro_infos_.reserve(*access_ctx.allocator_, micro_handle_cnt_))) {
     STORAGE_LOG(WARN, "failed to reserve sstable micro infos", K(ret), K_(micro_handle_cnt));
@@ -608,7 +609,7 @@ void ObSSTableRowIterator::reset()
     micro_lock_checker_->~ObMicroBlockRowLockChecker();
     micro_lock_checker_ = NULL;
   }
-
+  
   macro_block_iter_.reset();
   iter_param_ = NULL;
   access_ctx_ = NULL;
@@ -636,6 +637,7 @@ void ObSSTableRowIterator::reset()
   table_store_stat_.reset();
   skip_ctx_.reset();
   storage_file_ = nullptr;
+  is_rescan_ = false;
   prefetch_handle_depth_ = DEFAULT_PREFETCH_HANDLE_DEPTH;
   prefetch_micro_depth_ = DEFAULT_PREFETCH_MICRO_DEPTH;
 }
@@ -669,6 +671,59 @@ void ObSSTableRowIterator::reuse()
   table_store_stat_.reuse();
   skip_ctx_.reset();
   storage_file_ = nullptr;
+  is_rescan_ = false;
+  prefetch_handle_depth_ = DEFAULT_PREFETCH_HANDLE_DEPTH;
+  prefetch_micro_depth_ = DEFAULT_PREFETCH_MICRO_DEPTH;
+}
+void ObSSTableRowIterator::reuse_try(){
+  ObISSTableRowIterator::reset();
+//  read_handles_.reset();
+//  micro_handles_.reset();
+  sstable_micro_infos_.reset();
+
+  if (NULL != micro_exister_) {
+    micro_exister_->~ObMicroBlockRowExister();
+    micro_exister_ = NULL;
+  }
+  if (NULL != micro_getter_) {
+    micro_getter_->~ObMicroBlockRowGetter();
+    micro_getter_ = NULL;
+  }
+  if (NULL != micro_scanner_) {
+    micro_scanner_->~ObIMicroBlockRowScanner();
+    micro_scanner_ = NULL;
+  }
+  if (NULL != micro_lock_checker_) {
+    micro_lock_checker_->~ObMicroBlockRowLockChecker();
+    micro_lock_checker_ = NULL;
+  }
+  
+  is_rescan_ = true;
+  macro_block_iter_.reset();
+  iter_param_ = NULL;
+  access_ctx_ = NULL;
+  sstable_ = NULL;
+  query_range_ = NULL;
+  scan_step_ = 0;
+  is_opened_ = false;
+  is_base_ = false;
+  block_cache_ = NULL;
+  table_type_ = ObITable::MAJOR_SSTABLE;
+  sstable_snapshot_version_ = 0;
+  prefetch_handle_end_ = false;
+  prefetch_block_end_ = false;
+  cur_prefetch_handle_pos_ = 0;
+  cur_fetch_handle_pos_ = 0;
+  cur_read_handle_pos_ = 0;
+  cur_prefetch_micro_pos_ = 0;
+  cur_read_micro_pos_ = 0;
+  cur_micro_idx_ = -1;
+  cur_range_idx_ = -1;
+  io_micro_infos_.reset();
+  micro_info_iter_.reset();
+  table_store_stat_.reset();
+  skip_ctx_.reset();
+  storage_file_ = nullptr;
   prefetch_handle_depth_ = DEFAULT_PREFETCH_HANDLE_DEPTH;
   prefetch_micro_depth_ = DEFAULT_PREFETCH_MICRO_DEPTH;
 }
@@ -1701,9 +1756,12 @@ int ObSSTableRowIterator::init_handle_mgr(
             range_count >= USE_HANDLE_CACHE_RANGE_COUNT_THRESHOLD);
   }
   if (OB_SUCC(ret)) {
-    if (OB_FAIL(block_handle_mgr_.init(is_multi, true, *access_ctx.allocator_))) {
+    if(is_rescan_ == true){
+      is_multi = true;
+    }
+    if (OB_FAIL(block_handle_mgr_.init(is_multi, true, *access_ctx.stmt_allocator_))) {
       STORAGE_LOG(WARN, "failed to init block handle mgr", K(ret), K(is_multi), K(is_ordered));
-    } else if (OB_FAIL(block_index_handle_mgr_.init(is_multi, is_ordered, *access_ctx.allocator_))) {
+    } else if (OB_FAIL(block_index_handle_mgr_.init(is_multi, is_ordered, *access_ctx.stmt_allocator_))) {
       STORAGE_LOG(WARN, "failed to init block index handle mgr", K(ret), K(is_multi), K(is_ordered));
     }
   }
diff --git a/src/storage/ob_sstable_row_iterator.h b/src/storage/ob_sstable_row_iterator.h
index dadb33b..bcb0f16 100644
--- a/src/storage/ob_sstable_row_iterator.h
+++ b/src/storage/ob_sstable_row_iterator.h
@@ -176,6 +176,9 @@ public:
   virtual int get_next_row(const ObStoreRow*& store_row);
   virtual void reset();
   virtual void reuse();
+  virtual void reuse_try(){
+    reset();
+  }
   OB_INLINE bool has_lob_column() const
   {
     return OB_NOT_NULL(lob_reader_);
@@ -236,7 +239,7 @@ public:
 template <typename T>
 class ObSimpleArray {
 public:
-  ObSimpleArray() : array_(nullptr), capacity_(0)
+  ObSimpleArray() : array_(nullptr), capacity_(0), stmt_allocator_(NULL) 
   {}
   ~ObSimpleArray()
   {
@@ -247,6 +250,9 @@ public:
     for (int64_t i = 0; i < capacity_; ++i) {
       array_[i].~T();
     }
+    if( NULL != stmt_allocator_ ){
+      stmt_allocator_->free(array_);
+    }
     array_ = nullptr;
     capacity_ = 0;
   }
@@ -262,6 +268,7 @@ public:
       } else {
         array_ = new (buf) T[count];
         capacity_ = count;
+        stmt_allocator_ = &allocator;
       }
     }
     return ret;
@@ -274,6 +281,7 @@ public:
 private:
   T* array_;
   int64_t capacity_;
+  common::ObArenaAllocator* stmt_allocator_;  
 };
 
 struct ObFastSkipChecker {
@@ -321,6 +329,7 @@ public:
   virtual ~ObSSTableRowIterator();
   virtual void reset() override;
   virtual void reuse() override;
+  virtual void reuse_try() override;
   virtual int get_skip_range_ctx(
       ObSSTableReadHandle& read_handle, const int64_t cur_micro_idx, ObSSTableSkipRangeCtx*& skip_ctx);
   int get_row_iter_flag_impl(uint8_t& flag);
@@ -398,6 +407,7 @@ protected:
 private:
   bool is_opened_;
   bool is_base_;
+  bool is_rescan_;
   blocksstable::ObIMicroBlockCache* block_cache_;
   ObITable::TableType table_type_;
   int64_t sstable_snapshot_version_;

patch4的主要思想就是缓存了微块(rescan场景下缓存了微块，然后在ObSSTableRowIterator的reuse()中标记这是rescan场景)；然后将read_hanldes_和micro_handles_重用了。

ObSSTableRowIterator等memtable或者sstable的iterator的reuse_try()函数是如何被调用的呢？因为每次rescan都会调用ObMultipleMerge的reuse()函数（其实它的子类的reuse()函数基本上也要调用这个函数），然后这个reuse()函数里面调用了ObMultipleMerge::reuse_iter_array()函数，然后我们修改了ObMultiMerge::reuse_iter_array()函数，使得它不是简单地reset所有iterator，而是reuse_try它们。

在ObSSTableRowIterator::reuse_try()函数中，我们取消reset read_handles_和reset micro_handles_ ，这样就可以再次用这两种handle了，并且在ObSimpleArray类中reset()函数中加入对所有元素的释放操作，这个操作比较重要。因为我们是通过stmt_allocator来给read_handles_和mico_handles_来申请空间，应该要找个合适的时机来释放这个空间。

我想我已经搞清楚了为什么使用HandleCache，也即在rescan场景下缓存微块，并且将read_handles_和micro_handles重用，可以提升性能。HandleCache是将这次scan右表需要的所有微块都缓存下来，micro_handles是将这些缓存的微块按照扫描顺序排好，micro_handles最里面有buf指针，指向微块数据，HandleCache也是的，这两个buf指针是相同的，也即HandleCache和micro_handles是对同一个微块数据的引用。当我们重用HandleCache和micro_handles时，下一次rescan就省了两部分工作：

  1. 不用重新从磁盘读取微块了，因为可能在上一个rescan微块已经缓存在内存中了。
  2. 不用重新给micro_handles和read_handles分配空间了。以前每次rescan都要给micro_handles和read_handles分配空间，但是这两个的分配器换成了stmt_allocator后，这两个的空间就不会在新的rescan处被回收，我们就可以复用这个空间。但是有一点需要注意
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
int reserve(common::ObArenaAllocator& allocator, const int64_t count)
{
  int ret = common::OB_SUCCESS;
  if (capacity_ < count) {
    void* buf = nullptr;
    reset();
    if (OB_ISNULL(buf = allocator.alloc(sizeof(T) * count))) {
      ret = common::OB_ALLOCATE_MEMORY_FAILED;
      STORAGE_LOG(WARN, "failed to allocate array", K(ret));
    } else {
      array_ = new (buf) T[count];
      capacity_ = count;
    }
  }
  return ret;
}

这是micro_handles和read_handles的空间分配函数，当我们不reset它们时，capacity_属性就不用清0，因此就不会走if路线了。

源码阅读

架构

image-20211210213832687

OB的block cache是微块级的，不建议缓存宏块。但是不懂是怎么通过range或者rowkey定位到宏块，又从宏块中找到对应的微块，反正最后是能够在一个rescan内，把所有微块给拿到内存中的。

OB的ObMicroBlockIndexHandle存储的是ObMicroBlockIndexCache，它是宏块中对微块的索引，通过它可以方便地在宏块中找到微块。

在ObHandleMgr::init()函数中，如果是is multi并且is ordered，说明这个不断传下来的rowkey或者range都是有序的，例如rowkey是1, 2, 4, 8, 20这样的，这样的话，我们就只需要对一个ObMicroBlockHandle进行缓存就好了，不需要HandleCache来缓存那么多的ObMicroBlockHandle。

从ObExecuteResult::get_next_row()函数读NLJ的流程

1
int ObExecuteResult::get_next_row(ObExecContext& ctx, const common::ObNewRow*& row)
1
2
3
4
5
6
7
8
ret = get_next_row();
// ...
const ObOpSpec& spec = static_engine_root_->get_spec();
// ...
if (OB_ISNULL(
                row_.cells_ = static_cast<ObObj*>(ctx.get_allocator().alloc(sizeof(ObObj) * spec.output_.count())))) {
// ...

我们可以看到get_next_row()函数应该是实际得到一行数据，结果保存在static_engine_root_.get_spec().output_中。

其实这里的static_engine_root_是ObOperator*类型的，是一个算子。

1
int ObExecuteResult::get_next_row()
1
2
3
4
5
6
7
8
9
10
11
while (OB_SUCC(ret) && !got_row) {
  if (OB_FAIL(static_engine_root_->get_next_row())) {
    if (OB_ITER_END == ret) {
      plan_ctx->inc_bind_array_idx();
      if (OB_FAIL(static_engine_root_->switch_iterator())) {
      }
    }
  }else {
    got_row = true;
  }
}

调用ObOperator::get_next_row()得到下一行数据，如果是OB_ITER_END，那就switch_iterator()，但是不明白这个函数是做什么的？

1
int ObOperator::get_next_row()
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
if (OB_FAIL(startup_filter(filtered)))
// ...
while (OB_SUCC(ret)) {
  if (OB_FAIL(inner_get_next_row())) {
  } else {
    if (!spec_.filters_.empty()) {
      bool filtered = false;
      if (OB_FAIL(filter_row(filtered))) {
        LOG_WARN("filter row failed", K(ret), "type", spec_.type_, "op", op_name());
      } else {
        if (filtered) {
          continue;
        }
      }
    }
  }
  break;
}
// ...
if (!got_first_row_) {
  op_monitor_info_.first_row_time_ = oceanbase::common::ObClockGenerator::getClock();
  ;
  got_first_row_ = true;
}

不懂startup_filter()函数是做什么的。当filtered为true时，表示过滤不成功。当还没有得到第一行记录的时候，记录一下当前的时间到op_monitor_info_中。

1
int ObNestedLoopJoinOp::inner_get_next_row()
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
if (OB_UNLIKELY(LEFT_SEMI_JOIN == MY_SPEC.join_type_ || LEFT_ANTI_JOIN == MY_SPEC.join_type_)) {
    if (OB_FAIL(join_row_with_semi_join())) {
    }
} else {
  output_row_produced_ = false;
  while (OB_SUCC(ret) && !output_row_produced_) {
    state_operation = this->ObNestedLoopJoinOp::state_operation_func_[state_];
    if (OB_ITER_END == (ret = (this->*state_operation)())) {
      func = FT_ITER_END;
      ret = OB_SUCCESS;
    } else if (OB_FAIL(ret)) {
      LOG_WARN("failed state operation", K(ret), K(state_));
    } else {
      func = FT_ITER_GOING;
    }
    if (OB_SUCC(ret)) {
      state_function = this->ObNestedLoopJoinOp::state_function_func_[state_][func];
      if (OB_FAIL((this->*state_function)()) && OB_ITER_END != ret) {
        LOG_WARN("failed state function", K(ret), K(state_), K(func));
      }
    }
  }  // while end
}
// ...

在构造ObNestLoopJoinOp的时候

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
ObNestedLoopJoinOp::ObNestedLoopJoinOp(ObExecContext& exec_ctx, const ObOpSpec& spec, ObOpInput* input)
    : ObBasicNestedLoopJoinOp(exec_ctx, spec, input),
      state_(JS_READ_LEFT),
      mem_context_(nullptr),
      is_left_end_(false),
      last_store_row_(),
      save_last_row_(false)
{
  state_operation_func_[JS_JOIN_END] = &ObNestedLoopJoinOp::join_end_operate;
  state_function_func_[JS_JOIN_END][FT_ITER_GOING] = NULL;
  state_function_func_[JS_JOIN_END][FT_ITER_END] = &ObNestedLoopJoinOp::join_end_func_end;

  state_operation_func_[JS_READ_LEFT] = &ObNestedLoopJoinOp::read_left_operate;
  state_function_func_[JS_READ_LEFT][FT_ITER_GOING] = &ObNestedLoopJoinOp::read_left_func_going;
  state_function_func_[JS_READ_LEFT][FT_ITER_END] = &ObNestedLoopJoinOp::read_left_func_end;

  state_operation_func_[JS_READ_RIGHT] = &ObNestedLoopJoinOp::read_right_operate;
  state_function_func_[JS_READ_RIGHT][FT_ITER_GOING] = &ObNestedLoopJoinOp::read_right_func_going;
  state_function_func_[JS_READ_RIGHT][FT_ITER_END] = &ObNestedLoopJoinOp::read_right_func_end;
}

join_row_with_semi_join()不知道干什么的？

在构造函数，state_初始化为JS_READ_LEFT。我们看到首先有一个while循环，所以肯定是在while循环中先read_left_operate()，然后再执行read_left_func_going()，在read_left_func_going()的最后，设置state_为JS_READ_RIGHT，然后就继续循环，读取右表中的一行read_right_operate()，然后就继续循环执行read_right_func_going()，这里会调用ObJoinOp::calc_other_conds()函数进行判断是否满足filter条件，如果满足，就设置output_row_produced_为true，这样就可以结束循环。所以这个循环的目的就是为了得到一行满足filter的数据，当然在上层，我们也进行了一个filter判断，我觉得这是多余的了。

这里state_是个很重要的属性，控制着NLJ的执行方向，是读左表，还是读右表，还是read_left_func_going()，具体这些执行怎么做的，后面具体说。

内存管理

ObArenaAllocator

我们先看page_arena.h这个文件中是怎么定义和实现ObArenaAllocator这个类的。

alloc

1
2
3
4
5
6
// class ObArenaAllocator final : public ObIAllocator
public:
  virtual void* alloc(const int64_t sz) override
  {
    return arena_.alloc_aligned(sz);
  }
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
// template <typename CharT = char, class PageAllocatorT = DefaultPageAllocator>
// class PageArena
/** allocate sz bytes */
  CharT* alloc_aligned(const int64_t sz, const int64_t alignment = 16)
  {
    ensure_cur_page();

    // common case
    CharT* ret = NULL;
    if (NULL != cur_page_ && sz > 0) {
      int64_t align_offset = get_align_offset(cur_page_->alloc_end_, alignment);
      int64_t adjusted_sz = sz + align_offset;

      if (adjusted_sz <= cur_page_->remain()) {
        ret = cur_page_->alloc(adjusted_sz) + align_offset;
        if (NULL != ret) {
          used_ += align_offset;
        }
      } else if (is_normal_overflow(sz)) {
        Page* new_page = extend_page(page_size_);
        if (NULL != new_page) {
          cur_page_ = new_page;
        }
        if (NULL != cur_page_) {
          ret = cur_page_->alloc(sz);
        }
      } else if (lookup_next_page(sz)) {
        if (NULL != cur_page_) {
          ret = cur_page_->alloc(sz);
        }
      } else {
        ret = alloc_big(sz);
      }

      if (NULL != ret) {
        used_ += sz;
      }
    }
    return ret;
  }

ObArenaAllocator每次是在一个page内进行内存分配的，如果在当前的page内可以分配，就分配，如果不能，就用ModulePageAllocator分配一个新的页，在这个页内进行分配。所有用ModulePageAllocator分配的页是用链表连接起来的。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
// template <typename CharT = char, class PageAllocatorT = DefaultPageAllocator>
// class PageArena
  Page* extend_page(const int64_t sz)
  {
    Page* page = cur_page_;
    if (NULL != page) {
      page = page->next_page_;
      if (NULL != page) {
        page->reuse();
      } else {
        page = alloc_new_page(sz);
      }
    }
  }

使用PageArena::extend_page()来扩展一个页。

1
2
3
4
5
6
7
8
9
// template <typename CharT = char, class PageAllocatorT = DefaultPageAllocator>
// class PageArena
  Page* alloc_new_page(const int64_t sz)
  {
    void* ptr = page_allocator_.alloc(sz);
      page = new (ptr) Page((char*)ptr + sz);
      total_ += sz;
      ++pages_;
  }

PageArena::alloc_new_page()调用了ModulePageAllocator::alloc()函数来分配一个内存页。这个alloc最终会调用到

1
void* ObMallocAllocator::alloc(const int64_t size, const oceanbase::lib::ObMemAttr& attr)
1
2
3
4
5
6
7
8
9
10
11
12
13
} else if (OB_UNLIKELY(inner_attr.tenant_id_ >= PRESERVED_TENANT_COUNT)) {
  const int64_t slot = inner_attr.tenant_id_ % PRESERVED_TENANT_COUNT;
  obsys::CRLockGuard guard(locks_[slot]);
  allocator = get_tenant_ctx_allocator(inner_attr.tenant_id_, inner_attr.ctx_id_);
  if (!OB_ISNULL(allocator)) {
    ptr = allocator->alloc(size, inner_attr);
  }
} else {
  allocator = allocators_[inner_attr.tenant_id_][inner_attr.ctx_id_];
  if (!OB_ISNULL(allocator)) {
    ptr = allocator->alloc(size, inner_attr);
  }
}

这里可以发现，我们要分配内存，就要使用allocator池中的一个allocator来分配，这里超过了我的知识范围，但是使用哪个allocator可能是和租户什么的有关。我看过一篇文章，解释过OB的租户的内存：OB 内存分配概述 (qq.com)

读左表的流程

ObNestLoopJoinOp::read_left_operate() 函数调用 ObJoinOp::get_next_left_row()函数。

1
2
3
4
5
6
7
8
9
int ObJoinOp::get_next_left_row()
{
  int ret = common::OB_SUCCESS;
  left_row_joined_ = false;
  if (OB_FAIL(left_->get_next_row()) && OB_ITER_END != ret) {
    LOG_WARN("fail to get next left row", K(ret));
  }
  return ret;
}

设置左表读出的row还没有join，然后调用left_的get_next_row()，它是ObTableScanOp动态类型。

1
2
3
4
5
6
7
8
9
10
11
int ObTableScanOp::inner_get_next_row() {
  else if (OB_FAIL(get_next_row_with_mode())) {
  } else {
    output_row_count_++;
    NG_TRACE_TIMES_WITH_TRACE_ID(1, cur_trace_id_, get_row);
    if (MY_SPEC.is_vt_mapping_ &&
        OB_FAIL(vt_result_converter_->convert_output_row(eval_ctx_, MY_SPEC.mapping_exprs_, MY_SPEC.storage_output_))) {
      LOG_WARN("failed to convert output row", K(ret));
    }
  }
}

get_next_row_with_mode()成功后，就让output_row_count_增加。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
int ObTableScanIterator::get_next_row(common::ObNewRow*& row) {
  if (OB_FAIL(row_iter_->get_next_row(row))) {
    if (OB_ITER_END != ret) {
      STORAGE_LOG(WARN, "fail to get next row", K(ret));
    } else {
      ret = OB_SUCCESS;
      while (OB_SUCC(ret)) {
        if (OB_FAIL(iter_.get_next_iter(row_iter_))) {
          if (OB_ITER_END != ret) {
            STORAGE_LOG(WARN, "fail to get next iter", K(ret));
          } else {
            STORAGE_LOG(DEBUG, "table scan iterator reaches end");
          }
        } else if (OB_FAIL(row_iter_->get_next_row(row))) {
          if (OB_ITER_END != ret) {
            STORAGE_LOG(WARN, "fail to get next row", K(ret));
          } else {
            STORAGE_LOG(DEBUG, "table scan row iterator reaches end");
            ret = OB_SUCCESS;
          }
        } else {
          break;
        }
      }
    }
    }
}

这里有个iter_和row_iter_属性，iter_是ObTableScanIterIterator对象，row_iter_是ObTableScanRangeArrayRowIterator对象。iter_可以产生多个row_iter_，当一个row_iter_遍历到结束后，就让iter_下一个row_iter_来继续遍历。结合多range和多rowkey的前提，可能是在ObTableScanIterator中多个ObTableScanRangeArrayRowIterator，它们由ObTableScanIterIterator来管理，并且和多range一一对应，比方说，根据主键来查表，where中主键的范围是key<7 或者 key > 100，那么这两个range就会对应两个ObTableScanRangeArrayRowIterator。（由于OB中的表是索引组织表，因此只要知道要查找的主键的范围，那么就可以根据索引轻松找到想要的元组记录在哪，这也是为什么OB有range的概念的原因，想一下，要是记录随便放在表文件中，有range有什么用呢，还是要一个页一个页地扫描）。

1
2
3
int ObTableScanStoreRowIterator::get_next_row(ObStoreRow*& cur_row) {
  OB_FAIL(main_iter_->get_next_row(cur_row));
}

这个函数就调用了get_next_row()。main_iter_是ObQueryRowIterator类型，动态类型估计是ObMultipleScanMerge。

1
int ObMultipleMerge::get_next_row(ObStoreRow*& row);

这个get_next_row()函数很长，里面大致有这几个过程：

  1. access_ctx_相关的操作
  2. refresh_table_on_demand()，在必要的时候刷新table，可能会执行一些reuse_iter_array, reset_tables等的操作
  3. inner_get_next_row()，得到一个没有投影的行
  4. check_row_in_current_range()，不知道check什么
  5. row_filter_->check() 判断这个行是否被过滤
  6. project_row() 将行进行投影
  7. 其他关于行的操作
  8. 最后是更新access_ctx_中的一些统计量。

NLJ的rescan流程

sql层的ObTableScanOp.cpp的inner_close()函数可以看到有table_allocator_的reuse()调用，这个时候就是把以前用allocator申请的空间给释放掉。

rescan的实现是在ObTableScanStoreRowIterator.cpp中的rescan()函数中可以看到，其实就是调用ObMultipleMerge::reuse_iter_array()把很多iterator给重用了，然后再调用open_iter()函数重新打开这些iterator，并没有涉及到数据的操作。

在ob_table_scan_op.cpp中的rt_rescan()函数中写了单机rescan的大致流程。

NLJ的右表读取流程

左表肯定是通过scan方式来得到数据，因为是要遍历左表的所有满足条件的行。

由于在我们的case中，右表是通过索引回表的方式得到数据的，因此是这样的流程：ObIndexMerge中先访问索引，得到每个rowkey，然后通过rowkey通过get的方式访问主表，来得到具体的行数据。这个流程实现在ObIndexMerge::get_next_row()函数中。

考虑src/storage/ob_handle_mgr.h/oceanbase::storage::ObHandleMgr::init()函数，它的部分调用链是：ObSSTableRowIterator::inner_open() -> ObSSTableRowIterator::init_handle_mgr() -> ObHandleMgr::init()。

我们可以发现这里面有一个HandleCache对象：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
} else if (is_multi) {
  if (is_ordered) {
    if (OB_ISNULL(buf = allocator.alloc(sizeof(Handle)))) {
      ret = common::OB_ALLOCATE_MEMORY_FAILED;
      STORAGE_LOG(WARN, "failed to allocate last handle");
    } else {
      last_handle_ = new (buf) Handle();
    }
  } else {
    if (OB_ISNULL(buf = allocator.alloc(sizeof(HandleCache)))) {
      ret = common::OB_ALLOCATE_MEMORY_FAILED;
      STORAGE_LOG(WARN, "failed to allocate last handle");
    } else {
      handle_cache_ = new (buf) HandleCache();
    }
  }
}

“当is_multi为true时，说明有多个range或者rowkey传下来，这样就可以走这个分支，然后就可以使用这个HandleCache。我们考虑rescan的场景，每次rescan都会有新的range下来，而且这个range和上次rescan的range是连续的，因此就相当于is_multi为true的情况，如果这个时候我们不使用HandleCache，就是不够优化的。”

考虑src/ob_sstable_row_iterator.cpp/prefetch_block()函数。这个函数就是预取micro block的，但是为什么要预取呢，因为OB内部取磁盘数据是异步执行的，我们可以边读A微块边从磁盘取B微块，当A微块读取完成之后，说不定B微块就读好了。

ObHandleMgr和预取没有太大关系，只是我们在预取之后的数据之上，加了层Handle的cache。

我们是先从cache中找我们需要的微块，

1
2
3
4
5
6
7
8
9
10
11
12
for (int64_t i = 0; OB_SUCC(ret) && i < sstable_micro_cnt; ++i) {
  const ObSSTableMicroBlockInfo& sstable_micro = sstable_micro_infos[i];
  ObMicroBlockDataHandle& micro_handle = micro_handles_[sstable_micro.micro_idx_ % micro_handle_cnt_];
  bool need_submit_io = false;
  if (OB_FAIL(block_handle_mgr_.get_micro_block_handle(iter_param_->table_id_,
          sstable_micro.macro_ctx_,
          storage_file_->get_file_id(),
          sstable_micro.micro_info_.offset_,
          sstable_micro.micro_info_.size_,
          sstable_micro.micro_info_.index_,
          micro_handle))) {
    // cache miss

如果没有找到，再从磁盘IO来找。因为每次找新的range或者rowkey会定位到一个微块，可能这个微块和上个读取微块相同，如果这时发现这个相同的微块在HandleCache中找到了，就很好。

1
2
// ob_micro_block_handle_mgr.h
class ObMicroBlockHandleMgr : public ObHandleMgr<ObMicroBlockDataHandle, blocksstable::ObMicroBlockCacheKey, 64>
1
2
3
// ob_sstable_row_iterator.h
typedef ObSimpleArray<ObMicroBlockDataHandle> BlockDataHandleArray;
BlockDataHandleArray micro_handles;
1
2
// ob_sstable_row_iterator.h
typedef ObSimpleArray<ObSSTableReadHandle> ReadHandleArray;

问题

rescan中的prefetch是怎么回事？

stmt_allocator申请的空间在何时释放的？

迭代器打开时做了什么事情？

ObTableScanOp的ObNewRowIterator result_属性在哪初始化的？

在ObTableScanOp::do_table_scan()函数中调用了

1
if (OB_FAIL(das->table_scan(scan_param_, ab_iters_))) {

这个das最终又调用了

1
ObPartitionService::table_scan(ObVTableScanParam& vparam, common::ObNewRowIterator*& result)

这里的result就是我们要的输出参数；然后又调用了

1
ObPartitionStorage::table_scan(ObTableScanParam& param, const int64_t data_max_schema_version, common::ObNewRowIterator*& result)

在这个函数里面有一条调用

1
if (OB_UNLIKELY(NULL == (iter = rp_alloc(ObTableScanIterator, ObTableScanIterator::LABEL)))) {

在什么层级下是线程安全的？

钉钉群消息

线程检查工具：https://www.jianshu.com/p/1f29ae9fceee

https://github.com/oceanbase/oceanbase/issues/488 这里面的issue，认领的话，在这里回复就好了

@王运来 来哥 发的机器不能访问外网吗？
export http_proxy=’http://172.16.0.232:8259‘
export https_proxy=$http_proxy

测试主机密码：6vqSJOonTr52LhzFmnUm

首发！OceanBase社区版入门教程开课啦！https://mp.weixin.qq.com/s/04YjSUsNoKtIRsC0OC394A

image-20211130094435977

@nauta ob支持执行请求时打开trace_log，打开方式有两种，一种是通过hint中的trace_log字段，这种方式只对携带hint的当前语句生效；另一种是通过session变量ob_enable_trace_log，这种方式对这个session的后续所有语句生效。 打开trace_log后，通过show trace可以拿到上一次的trace_log，从中可以获取trace_id。同时show trace还可以看到这条请求大致的性能统计。 使用示例如下， # 语句级 select /+ trace_log=on /c1 from t1 limit 2; show trace; # session级 set ob_enable_trace_log = ‘ON’; select count(*) from t1; show trace; last_trace_id 使用select last_trace_id();可以查看上一条语句执行的trace_id，然后在日志中grep查找相关信息。

引用

https://github.com/oceanbase/oceanbase/wiki/how_to_debug

展开全文 >>

LSM tree

LSM tree

LSM树详解 - 知乎 (zhihu.com)

这里需要关注一个重点，LSM树(Log-Structured-Merge-Tree)正如它的名字一样，LSM树会将所有的数据插入、修改、删除等操作记录(注意是操作记录)保存在内存之中，当此类操作达到一定的数据量后，再批量地顺序写入到磁盘当中。LSM树的特点是：采用顺序写，提高了写的性能。

因此当MemTable达到一定大小flush到持久化存储变成SSTable后，在不同的SSTable中，可能存在相同Key的记录，当然最新的那条记录才是准确的。这样设计的虽然大大提高了写性能，但同时也会带来一些问题：

在Compact策略上，主要介绍两种基本策略：size-tiered和leveled。

一文带你了解LSM Compaction - 知乎 (zhihu.com)

随着数据写入不断增加，Minor Freeze不断触发，转储数据不断增多，一次查询可能需要越来越多的IO操作，读取延时也在不断变大。而执行Minor Compaction会使得转储文件数基本稳定，进而IO Seek次数会比较稳定，延迟就会稳定在一定范围。然而，Minor Compaction操作重写文件会带来很大的带宽压力以及短时间IO压力。因此可以认为，Minor Compaction就是使用短时间的IO消耗以及带宽消耗换取后续查询的低延迟。

OceanBase 存储引擎高级技术 - 知乎 (zhihu.com) OceanBase的LSM-tree

最容易理解的LSM树—以示例讲解合并查找过程_jinking01的专栏-CSDN博客 这里用图示讲解了LSM-tree的合并查找过程，里面LSM-tree的实现是，memtable用AVL树实现，sstable用B-tree实现。虽然不知道OceanBase的sstable的实现过程是什么样的，不管是B-tree还是排序好的元组序列，其实查询效率都挺高的，也都可以称为索引组织表，如果是B-tree，肯定是索引组织表，如果是排序序列，用二分查找也不错。

展开全文 >>

Cpp语言

static

C++中的static - 知乎 (zhihu.com)

operator类型转换

C++中Operator类型强制转换成员函数_chenglinhust的专栏-CSDN博客_operator 类型转换

C++将析构函数定义成virtual的真正原因

C++将析构函数定义成virtual的真正原因_NEVER-CSDN博客_virtual 析构函数

显示调用析构函数

用户显式调用析构函数的时候，只是单纯执行析构函数内的语句，不会释放对象对应的堆内存，摧毁对象

c++类继承中的using声明

c++类继承中的using声明，派生类中用using声明改变基类成员的访问权限_lfw19891101的专栏-CSDN博客

展开全文 >>

Clion中写CMakeLists.txt

CMakeLists.txt

image-20211117205702367

这是我的简单的项目目录，针对它该如何写CMakeLists.txt呢。

1
2
3
4
cmake_minimum_required(VERSION 3.20)
project(test1)
include_directories(src)
add_executable(test1 src/hellow_world.cpp src/main.cpp)

因为我的.h文件都在源码下的src/目录里，所以就有代码include_directories(src)

因为有两个源文件hellow_world.cpp和main.cpp，而且想要生成的exe文件名称是test1，所以就有代码add_executable(test1 src/hellow_world.cpp src/main.cpp)

最后很多关于build和debug的文件都在cmake-build-debug文件夹里面。

展开全文 >>

LEX&YACC

简介

Lex & Yacc 是用来生成词法分析器和语法分析器的工具。

Lex （A Lexical Analyzer Generator）用于生成词法分析器，用于把输入分割成一个个有意义的词块（称为token）。

Yacc（Yet Another Compiler-Compiler）用于生成语法解析器，用于确定上述分隔好的token之间的关联

Lex

Lex结构

lex文件内容分为三个段分别为：定义段、规则段、用户子程序段

三个段用 %% 进行分隔:

1
2
3
4
5
6
7
8
9
10
11
12
13
/* 定义段 */
%{
...
%}
...

%%
    /* 规则段 */
...

%%
    /* 用户子程序段 */
...

第1段是声明段，包括：
1-C代码部分：include头文件、函数、类型等声明，这些声明会原样拷到生成的.c文件中。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
%{
#include<string.h>
#include<stdio.h>
#include "yacc_sql.tab.h"
extern int atoi();
extern double atof();
char * position = "";

#ifdef YYDEBUG
#define debug_printf  printf
#else
#define debug_printf(...)
#endif // YYDEBUG

#define RETURN_TOKEN(token) debug_printf("%s\n",#token);return token
%}

​ 2-状态声明，如%x COMMENT。

1
2
3
4
5
6
7
8
9
10
状态声明将会在第二段用到，例如：
若当读到//注释符时忽略该行输入则可以使用%x标记一个状态，当读到//符时切换到该状态，后续只能匹配到该状态相应的规则，读到换行符后再返回原状态。以达到忽略注释行的目的。
切换状态通过BEGIN实现,初始态为INITIAL
"//"    				{    //遇到串"//"，表明要开始一段注释，直到行尾
            			BEGIN COMMENT;//用BEGIN宏切换到注释状态，去过滤这段注释，下一次lex将只匹配前面带有<COMMENT>的正则式
        				}
        				
 <COMMENT>\n    {    //注释状态下的规则，只有当前切换到COMMENT状态才会去匹配
            			BEGIN INITIAL;//在注释状态下，当遇到换行符时，表明注释结束了，返回初始态
      				  }

​ 3-正则式定义，如ID [A-Za-z_]+[A-Za-z0-9_]*。

1
2
3
4
5
WS      [\ \t\b\f]
DIGIT   [0-9]+
ID      [A-Za-z_]+[A-Za-z0-9_]*
DOT     \.
QUOTE   [\'\"]

img

第2段是规则段，是lex文件的主体，包括每个规则是如何匹配的，以及匹配后要执行的C代码动作。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
{WS}                    /* ignore whitespace */;
[Ss][Ee][Ll][Ee][Cc][Tt]                 position=strdup(yytext);RETURN_TOKEN(SELECT);
[Ff][Rr][Oo][Mm]                      	 position=strdup(yytext);RETURN_TOKEN(FROM);
[Ww][Hh][Ee][Rr][Ee]                  	 position=strdup(yytext);RETURN_TOKEN(WHERE);
[Aa][Nn][Dd]                             position=strdup(yytext);RETURN_TOKEN(AND);
[Oo][Rr][Dd][Ee][Rr]										 position=strdup(yytext);RETURN_TOKEN(ORDER);
[Bb][Yy]                                 position=strdup(yytext);RETURN_TOKEN(BY);
{ID}                                             
{ yylval.string=strdup(yytext);position=strdup(yytext);RETURN_TOKEN(ID);}


lex匹配优先规则：
数据流进入后进行逐字符匹配，若加入下一个字符可以匹配到规则则压入字符，当达到最大匹配长度后若能匹配多个规则，则执行位置较前的规则。
如select * from t;
读第一个字符's'能匹配到id，但是加上后续字符'e'仍满足条件则继续压栈，直到栈里为select此时已是能匹配的最大长度（加入空格后不能匹配到规则），这时能匹配到两个规则
[Ss][Ee][Ll][Ee][Cc][Tt]                 position=strdup(yytext);RETURN_TOKEN(SELECT);
{ID}                                             
{ yylval.string=strdup(yytext);position=strdup(yytext);RETURN_TOKEN(ID);}
因为select字段的规则在前，因此不会匹配成id，而是匹配select规则返回SELECT TOKEN。

第3段是C函数定义段，如yywrap()的定义，这些C代码会原样拷到生成的.c文件中，该段内容可以为空。

1
2
3
4
5
6
7
8
9
yywrap()
当词法分析程序遇到文件结尾时，它调用例程yywrap()来找出下一步要做什么，如果返回0，扫描程序继续扫描，如果返回1，扫描程序就返回报告文件结尾需标记。
lex库中yywrap()的标准版本总是返回1，如果yywrap()返回指示有更多的输入0，那么它首先需要调整指向新的文件yyin,可能使用fopen()。

int yywrap(void)
{
    return 1;//返回1表示读取全部结束。如果要接着读其它文件，可以这里fopen该文件，文件指针赋给yyin，并返回0
}

lex文件中其他常见函数和变量

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
lex不仅返回相应的token，同时还会向yacc传递相应数据，如：
create table stu(id int,old int);
不仅需要返回create table int等关键字的token，同时还要把表名、列名等字符串传给yacc

yytext
每当词法分析程序匹配标记时，标记的文本就存储在以空字符结尾的字符串yytext中
每次匹配一个新的标记时，就要替换yytext的内容，如果yytext的内容还要使用，通过strdup()或者自己申请内存来保存字符串拷贝，从而使字符串的拷贝拷贝位于刚刚分配的内存中。
{ID}                                             
{ yylval.string=strdup(yytext);position=strdup(yytext);RETURN_TOKEN(ID);}

yylval变量
extern YYSTYPE yylval
lex将数据流中的数据赋给yylval变量相应字段，从而传递给yacc
YYSTYPE在yacc文件中定义结构，默认为int，后续在yacc部分详解

Yacc

简介

yacc(Yet Another Compiler Compiler)，是Unix/Linux上一个用来生成编译器的编译器（编译器代码生成器）。
使用巴克斯范式(BNF)定义语法，能处理上下文无关文法(context-free)。出现在每个产生式左边(left-hand side：lhs)的符号是非终端符号，出现在产生式右边(right-hand side：rhs)的符号有非终端符号和终端符号，但终端符号只出现在右端。

x
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
举个例子：
在中文语法里，一个句子一般由“主语”、“谓语”和“宾语”组成，主语可以是名词或者代词，谓语一般是动词，宾语可以使形容词，名词或者代词。那么“主语”、“谓语”和“宾语”就是非终止符，因为还可以继续由“名词”、“代词”、“动词”、“形容词”等替代。

例1. <句子> ::= <主语><谓语><宾语>

例2. <主语> ::= <名词>|<代词>

例3. <谓语>::=<动词>

例4. <宾语>::=<形容词>|<名词>|<代词>

例5. <代词>::=<我>

例6. <动词>::=<吃>

例7. <动词>::=<喜欢>

例8. <名词>::=<车>

例9. <名词>::=<肉>

如上，在::=左边的就是non-terminal非终止符，右边的就是replacement，可以是一系列的非终止符，如例1中的replacement便是后面例234左边的非终止符，也可以是终止符，如例56789的右边，找不到别的符号来进一步代替。
因此，终止符永远不会出现在左边。一旦我们看到了终止符，这个描述过程就结束了。

语法结构

yacc语法包括三部分：定义段、规则段和用户子例程段

1
2
3
4
5
6
7
8
9
...定义段...

%%

...规则段...

%%

...用户子例程段...

各部分由以两个百分号开头的行分开，尽管某一个部分可以为空，但是前两部分是必须的，第三部分和前面的百分号可以省略。

定义段：

1、C代码部分：include头文件、函数、类型等声明，这些声明会原样拷贝到生产的.c文件中

2、记号声明，如%token

3、类型声明，如%type

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
%{

#include "handler/handler_defs.h"
// #include "common/log/log.h" // TODO 包含C++中的头文件

extern char * position;
#include<stdio.h>
#include<stdlib.h>
#include<string.h>
extern int yylex(void);
....
....
int compOpT;
void yyerror(const char *str) {

	ssql->flag=0;
	whereleng=0; 
  fromleng=0;
  selectleng=0;
	valueleng=0;
  ssql->sstr.ins.nValues=0;
	ssql->sstr.errors=position;
	printf("parse sql failed. error=%s", str);
}

%}

yyerror会对无法解析的语句进行处理

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
//标识tokens
%token  SEMICOLON
        CREATE
        DROP
        TABLE
        TABLES
        INDEX
        SELECT
        DESC
        SHOW
        SYNC
        INSERT
.....
%union {
    struct _Attr *attr;
	  struct _Condition *condition1;
  	struct _Value *value1;
    char *string;
  //  char *ssss;
    int number;
    float floats;
}
%union字段定义了YYSTYPE的结构体类型，定义后lex文件中就可以使用yylval变量的相应部分进行传输数据

%token <number> NUMBER
%token <floats> FLOAT 
%token <string> ID
%token <string> SSS
%token <string>  STAR
这种方式可以把标志（token）绑定到YYSTYPE的某个域。这样在后面规则部分直接使用$x就可以得到token的值，而不需要使用$x.number/string这种方式



规则段：

规则段由语法规则和包括C代码的动作组成。

规则中目标或非终端符放在左边，后跟一个冒号（：），然后是产生式的右边，之后是对应的动作（用{}包含）。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
%%

commands:     //commands or sqls. parser starts here.
    /* empty */
    | commands command
    ;

command:
     select  
   | insert
   | update
   | delete
   | create_table
...
   ;
commands规则中的第一条为空非常重要，否则commands没有递归的出口

以select语句的解析为例
select:				/*  select 语句的语法解析树*/
    SELECT select_attr FROM ID rel_list where SEMICOLON {
      
            ssql->sstr.sel.relations[fromleng++]=$4;

            for(i=0;i<whereleng;i++){
              ssql->sstr.sel.conditions[i]=wherecondi[i];
            }

            ssql->flag=SCF_SELECT;//"select";
            ssql->sstr.sel.nSelAttrs=selectleng;
            ssql->sstr.sel.nRelations=fromleng;
            ssql->sstr.sel.nConditions=whereleng;

            //临时变量清零
            whereleng=0;
            fromleng=0;
            selectleng=0;
        }
        ;
        

select_attr:
    STAR {  // TODO 处理这里的内存泄露问题
         ssql->sstr.sel.selAttrs[selectleng].attrName="*";
         ssql->sstr.sel.selAttrs[selectleng++].relName=NULL;
       }
    | ID attr_list {
        ssql->sstr.sel.selAttrs[selectleng].attrName=$1;
        ssql->sstr.sel.selAttrs[selectleng++].relName=NULL;
      }
  	| ID DOT ID attr_list {
        ssql->sstr.sel.selAttrs[selectleng].attrName=$3;
        ssql->sstr.sel.selAttrs[selectleng++].relName=$1;
      }
    ;

attr_list:
    /* empty 递归出口*/
    | COMMA ID attr_list {
     	  ssql->sstr.sel.selAttrs[selectleng].relName = NULL;
        ssql->sstr.sel.selAttrs[selectleng++].attrName=$2;
      }
    | COMMA ID DOT ID attr_list {
        ssql->sstr.sel.selAttrs[selectleng].attrName=$4;
        ssql->sstr.sel.selAttrs[selectleng++].relName=$2;
  	  }
  	;
  	
rel_list:
    /* empty */
    | COMMA ID rel_list {	
				ssql->sstr.sel.relations[fromleng++]=$2;
		  }
    ;
    
where:
    /* empty */ 
    | WHERE condition condition_list {	
				wherecondi[whereleng++]=*$2;
			}
    ;
    
condition_list:
    /* empty */
    | AND condition condition_list {
				wherecondi[whereleng++]=*$2;
			}
    ;
condition:
    ID comOp value {
									$$ = ( Condition *)malloc(sizeof( Condition));
									$$->bLhsIsAttr = 1;
									$$->lhsAttr.relName = NULL;
									$$->lhsAttr.attrName = $1;
									$$->op = compOpT;
									$$->bRhsIsAttr = 0;
									$$->rhsAttr.relName = NULL;
									$$->rhsAttr.attrName = NULL;
									$$->rhsValue = *$3;

								    }
		|value comOp value {
									$$ = ( Condition *)malloc(sizeof( Condition));
									$$->bLhsIsAttr = 0;
									$$->lhsAttr.relName=NULL;
									$$->lhsAttr.attrName=NULL;
									$$->lhsValue = *$1;
									$$->op = compOpT;
									$$->bRhsIsAttr = 0;
									$$->rhsAttr.relName = NULL;
									$$->rhsAttr.attrName = NULL;
									$$->rhsValue = *$3;

								    }
		|ID comOp ID {
									$$=( Condition *)malloc(sizeof( Condition));
									$$->bLhsIsAttr = 1;
									$$->lhsAttr.relName=NULL;
									$$->lhsAttr.attrName=$1;
									$$->op=compOpT;
									$$->bRhsIsAttr = 1;
									$$->rhsAttr.relName=NULL;
									$$->rhsAttr.attrName=$3;

								    }
    |value comOp ID{
									$$=( Condition *)malloc(sizeof( Condition));
									$$->bLhsIsAttr = 0;
									$$->lhsAttr.relName=NULL;
									$$->lhsAttr.attrName=NULL;
									$$->lhsValue = *$1;
									$$->op=compOpT;
									
									$$->bRhsIsAttr = 1;
									$$->rhsAttr.relName=NULL;
									$$->rhsAttr.attrName=$3;
								
								    }
    |ID DOT ID comOp value{
									$$=( Condition *)malloc(sizeof( Condition));
									$$->bLhsIsAttr = 1;
									$$->lhsAttr.relName=$1;
									$$->lhsAttr.attrName=$3;
									$$->op=compOpT;
									$$->bRhsIsAttr = 0;   //属性值
									$$->rhsAttr.relName=NULL;
									$$->rhsAttr.attrName=NULL;
									$$->rhsValue=*$5;			
							
    											}
    |value comOp ID DOT ID{
									$$=( Condition *)malloc(sizeof( Condition));
									$$->bLhsIsAttr = 0;//属性值
									$$->lhsAttr.relName=NULL;
									$$->lhsAttr.attrName=NULL;
									$$->lhsValue = *$1;
									$$->op=compOpT;
									$$->bRhsIsAttr = 1;//属性
									$$->rhsAttr.relName = $3;
									$$->rhsAttr.attrName = $5;
									
    						}
    |ID DOT ID comOp ID DOT ID{
									$$=( Condition *)malloc(sizeof( Condition));
									$$->bLhsIsAttr = 1;		//属性
									$$->lhsAttr.relName=$1;
									$$->lhsAttr.attrName=$3;
									$$->op=compOpT;
									$$->bRhsIsAttr = 1;		//属性
									$$->rhsAttr.relName=$5;
									$$->rhsAttr.attrName=$7;								
    											}
    ;

comOp:
  	  EQ { compOpT = EQual; }
    | LT { compOpT = LEqual; }
    | GT { compOpT = GreatT; }
    | LE { compOpT = LessT; }
    | GE { compOpT = GEqual; }
    | NE { compOpT = NEqual; }
    ;

以下面这个select语句解析为例

1
2
3
4
5
6
select name,old from stu where old>10;

lex会将该句分解为以下token序列
SELECT ID COMMA ID FROM ID WHERE ID 

token序列进入yacc后会进行规则解析，然后触发相应动作，最终形成在后续使用的ssql结构体

展开全文 >>

20211010周报

e

科研

miniob

最近一星期我就看了mini-ob的源代码，看明白了：

  1. 底层的存储系统
  2. insert和select的流程

但是不懂的是：

  1. 如何实现多表的hash join

nvm

没有看

PG数据库

想了解wal的代码相关，但是心态浮躁，看不下去代码。

工作

只是练练leetcode，还需要做：

  1. 后端开发相关书籍
  2. 开源项目源码
  3. 面试题

体育

很久没有跑步，打球了。

生活

最近花钱过于频繁，以后一个月要考虑如何省钱，多和敏在一起，但是不要kaif。

每天坚持记日记，基本没有间断过，坚持早起，三餐都和敏吃饭。

社交

很久没有社交了，渐渐地不想关心华东师大和实验室的兄弟。

修养

很久没有看情商类和心理健康类课程了。

爱好

很久没有看书了。

总结

  1. 保持每天早睡，每天和敏吃饭，每天练leetcode，每天看miniob的代码。在这个基础上，还需要：
  2. 和组内人员讨论miniob的问题，比如多在群内说话，更新文章。
  3. 继续读redis书，学习技能。
  4. 可以在周四去打乒乓球或者羽毛球；每天在9点中开始跑半个小时。
  5. 如果敏没说特别想吃什么，那么自己就点半分的菜，吃面之类，因为比较省钱。
  6. 可以和瑜明，魏寒，魏星聊聊。
  7. 偶尔看一点情商类课程，看看论语，鲁迅。

展开全文 >>

latex

这篇文章参考了有哪些好的 LaTeX 编辑器？

安装

安装texlive

在清华大学开源镜像站搜索CTAN(latex软件包)，然后找到systems/texlive/Images目录，然后下载一个镜像。或者直接点击清华镜像texlive。

不得不说，这个镜像文件太大了，我以为latex就是一个轻量的东西。

下载完成之后，使用解压工具将这个iso文件解压到不含有中文的目录下面，然后安装时，也安装到不含有中文的目录下面，最好安装到D盘，因为它太大了。剩下的就按照安装程序走就好了，默认配置。

然后，将{安装目录}/bin/win32添加到系统PATH变量里面。

配置VS Code

假设已经安装好了VS Code。

LaTex Workshop

在VS Code侧边栏的extensions中搜索LaTex Workshop，然后安装这个插件。

在VS Code中按住快捷键CTRL+SHIFT+P，输入set，然后选择Preferences: Open Settings (JSON)，然后将下面的代码粘贴到settings.json文件的最外面一对花括号里。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
作者：黄盼
链接：https://www.zhihu.com/question/19954023/answer/676316395
来源：知乎
著作权归作者所有。商业转载请联系作者获得授权，非商业转载请注明出处。

"latex-workshop.latex.recipes": [{
        "name": "xelatex",
        "tools": [
            "xelatex"
        ]
    },
    {
        "name": "xelatex ->bibtex -> xelatex*2",
        "tools": [
            "xelatex",
            "bibtex",
            "xelatex",
            "xelatex"
        ]
    },
    {
        "name": "pdflatex",
        "tools": [
            "pdflatex"
        ]
    }
],
"latex-workshop.latex.tools": [{
        "name": "xelatex",
        "command": "xelatex",
        "args": [
            "-synctex=1",
            "-interaction=nonstopmode",
            "-file-line-error",
            "%DOC%"
        ]
    },
    {
        "name": "pdflatex",
        "command": "pdflatex",
        "args": [
            "-synctex=1",
            "-interaction=nonstopmode",
            "-file-line-error",
            "%DOC%"
        ]
    },
    {
        "name": "bibtex",
        "command": "bibtex",
        "args": [
            "%DOCFILE%"
        ]
    }
],
// 指定在PDF文件生成后自动清除辅助文件
"latex-workshop.latex.autoClean.run": "onBuilt",
// 解除每次自动保存都会编译 LaTex 文件的功能
"latex-workshop.latex.autoBuild.run": "never",

配置好了之后，就可以按CTRL+ALT+B编译latex文件了，按CTRL+ALT+V打开PDF文件。

latex-formatter

这是一款 VSCode 的 LaTex 代码格式化插件。

latex语法

数学符号

由于我使用latex主要是为了写数学作业，所以要了解很多数学语法，参考typora: 一些常用数学符号。然后在VS Code中使用快捷键SHIFT+ALT+F就可以自动排版了。

一份(不太)简短的 LaTeX2ε 介绍

在这里可以阅读这本latex入门书。但是这本书还是太长了，不适合快速入手。

插入python代码

参考在Latex中插入Python代码，需要下面的宏：

1
2
\usepackage{graphicx}
\usepackage{pythonhighlight}

代码块

1
2
3
\begin{python}
    % your python code
\end{python}

这个宏有个好处就是直接代码高亮，解决了over-full的问题，非常好用。

引用

展开全文 >>

tag:

    缺失模块。
    1、请确保node版本大于6.2
    2、在博客根目录（注意不是yilia根目录）执行以下命令：
    npm i hexo-generator-json-content --save

    3、在根目录_config.yml里添加配置： 

      jsonContent:
    meta: false
    pages: false
    posts:
      title: true
      date: true
      path: true
      text: false
      raw: false
      content: false
      slug: false
      updated: false
      comments: false
      link: false
      permalink: false
      excerpt: false
      categories: false
      tags: true

很惭愧<br><br>只做了一点微小的工作<br>谢谢大家