OB复赛

2021-12-17

连接到ECS

首先下载Open VPN，然后使用附件的certs配置来配置open vpn，然后连接。下面是配置文件夹内的内容，基本把附件certs中的文件复制过去就好了。
然后打开finalshell，连接到对应的ip。

172.16.0.12 BtfQDDzWoYvCmiCXhm6f 172.16.0.67 23neMCCn5WhLbc4s6DWt

密码修改为：zhou19981209

使用VS code插件remote development来开发

参考 vscode连接远程服务器+SFTP同步本地文件其实可以不用安装sftp，因为在vs code中可以直接上传本地文件和更改服务器文件

为了不用每次都输入密码，我们可以把公钥传输到远程服务器：

在authorized_keys中加入很多行公钥。然后把.ssh和authorized_keys的访问权限更新为至少可读。

vscode通过跳板机连接远程服务器

如何使用源码进行测试

查看系统架构：

uname -a

在执行sudo yum install rpm*时，出现了错误：

我直接跳过有依赖问题的包了，sudo yum install rpm* --skip-broken。

采用 sudo -E。简单来说，就是加上-E选项后，用户可以在sudo执行时保留当前用户已存在的环境变量，不会被sudo重置，另外，如果用户对于指定的环境变量没有权限，则会报错。

安装sysbench 1.0.20是可以的。直接在sysbench 上参考二进制的安装方法，推荐的那个。

如何多人在同一个服务器上协作

在同一台电脑上添加多个ssh key - 简书 (jianshu.com)

调试oceanbase

gdb

我们调试的时候要在oceanbase的根目录下调试，然后设置断点的时候要指定函数所在的文件，例如：

1	break ./src/sql/parser/ob_parser.cpp:ObParser::parse

我们要把build_debug/src/observer/observer作为~/zhouhuahui/ob-advanced-data/bin/observer的软链接，才能使用gdb调试成功，或者直接在gdb下使用：

1	dir /home/test/zhouhuahui/Github/oceanbase

定位到源文件也是可以的。

vscode attach

如何在vs code上调试：参考如何debug OceanBase。但是这种不够，我是是使用attach的方式来调试，写了下面的launch.json

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "observer",
            "type": "cppdbg",
            "request": "attach",
            "processId": "${command:pickProcess}",
            "program": "/home/test/zhouhuahui/ob-advanced-data/bin/observer",
            // "args": ["-r", "127.0.0.1:2882:2881", "-o", "__min_full_resource_pool_memory=268435456,memory_limit=8G,datafile_size=10G,enable_syslog_recycle=True,enable_syslog_wf=True,max_syslog_file_count=4,system_memory=4G,cpu_count=16", "-z", "zone1", "-p", "2881", "-P", "2882", "-n", "obcluster", "-c", 1, "-d", "/home/test/zhouhuahui/ob-advanced-data/store", "-i", "lo", "-l", "INFO"],
            "stopAtEntry": true,
            "cwd": "/home/test/zhouhuahui/Github/oceanbase/",
            "environment": [],
            "externalConsole": false,
            "MIMode": "gdb",
            "setupCommands": [
                {
                    "description": "Enable pretty-printing for gdb",
                    "text": "-enable-pretty-printing",
                    "ignoreFailures": true
                }
            ]
        }
    ]
}

在点击调试之后，在DEBUG CONSOLE界面输入：

1	-exec dir /home/test/zhouhuahui/Github/oceanbase

就可以正确定位到源文件的位置了。

vscode launch

如果还要使用launch的方式，要在程序的参数加一个”-N”参数，来指明程序在前台运行，（因为默认程序在后台运行），如果不加的话，会出先调试程序刚打开却退出的现象。

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
    
        {
            "name": "observer2",
            "type": "cppdbg",
            "request": "launch",
            "program": "/home/test/zhouhuahui/Github/oceanbase/build_debug/src/observer/observer",
            "args": ["-N", "-r", "127.0.0.1:2982:2981", "-o", "__min_full_resource_pool_memory=268435456,memory_limit=8G,datafile_size=10G,enable_syslog_recycle=True,enable_syslog_wf=True,max_syslog_file_count=4,system_memory=4G,cpu_count=16", "-z", "zone1", "-p", "2981", "-P", "2982", "-n", "obcluster", "-c", "1", "-d", "/home/test/zhouhuahui/Github/oceanbase/build_debug/ob-advanced-data/store", "-i", "lo", "-l", "INFO"],
            "stopAtEntry": true,
            "cwd": "/home/test/zhouhuahui/Github/oceanbase",
            "environment": [],
            "externalConsole": false,
            "MIMode": "gdb",
            "setupCommands": [
                {
                    "description": "Enable pretty-printing for gdb",
                    "text": "-enable-pretty-printing",
                    "ignoreFailures": true
                }
            ]
        }
    ]
}

100个gdb小技巧 (gitbooks.io)

跟踪sql的执行

1	obclient -uroot@test -h127.0.0.1 -P 2881 -c

use test;
create table t1(i int primary key, f float, c CHAR(60));
create table t2(i int primary key, f float, c CHAR(60));
create index t2_c on t2(c);
insert into t1 values(1, 1.5, "zhou");
insert into t1 values(2, 0.5, "hua");
insert into t1 values(3, 40.5, "hui");
insert into t2 values(1, 20.0, "hua");
insert into t2 values(2, 15.135, "hui");
insert into t2 values(3, 10.5, "fan");

理论教程

揭秘 OceanBase SQL 执行计划（一） (qq.com)

索引组织表-阿里云开发者社区 (aliyun.com)

OceanBase SQL 执行计划解读（二）──── 表连接和子查询_OceanBaseGFBK的博客-CSDN博客

两个表join时，谁作为外部表，谁作为内部表不好说，由过滤条件应用后的结果集大小来定。

OceanBase 存储引擎高级技术 - 知乎 (zhihu.com)

OceanBase的索引创建流程 - 知乎 (zhihu.com)

MySQL中的semi-join_lppl010_的专栏-CSDN博客讲解了什么是semi-join。

OB有两种从表中获取行的方法，一个是TABLE_SCAN，一个是TABLE_GET，为了方便，就直接说scan和get了。scan表示直接从头到尾扫描OB的索引组织表，然后获取这些行；get表示通过索引的key来找到这些行或者行的key。使用scan还是get是由SQL中的where部分决定的，假如查询条件涉及的列是有索引的，就可以使用get，假如没有，就只能使用scan。

但是还有一个问题是：假如A,B表进行嵌套循环连接，那么谁作外表，谁作内表呢？根据A,B获取行的方法不同，分为三种情况来讨论：

A使用scan，B也使用scan。那么需要预估A的结果集大小和B的结果集大小，哪个小就作为内表。
A使用scan，B使用get。那么A作为外表，B作为内表。
A使用get，B使用get。哪个结果集小就作为外表。

性能优化之Block Nested-Loop Join(BNL) - 云+社区 - 腾讯云 (tencent.com)

分析patch

reset和reuse分别在哪些地方被用到了，reset和reuse的具体语义是什么，为什么要这么用？

reuse:

ObTableAccessContext.reuse
ObTableScanStoreRowIterator.reuse_row_iters

分析群里给的patch的更改前后对比

更改前

namespace oceanbase {
namespace storage {
class ObIStoreRowFilter;
class ObMultipleMerge : public ObQueryRowIterator {
public:
  typedef common::ObSEArray<ObStoreRowIterator*, common::MAX_TABLE_CNT_IN_STORAGE> MergeIterators;

public:

void ObMultipleGetMerge::reset_with_fuse_row_cache()
{
  prefetch_range_idx_ = 0;
  get_row_range_idx_ = 0;
  sstable_rowkeys_.reset();
  if (nullptr != rows_) {
    for (int64_t i = 0; i < prefetch_cnt_; ++i) {
      rows_[i].~ObQueryRowInfo();
    }
    rows_ = nullptr;
  }
  if (nullptr != handles_) {
    for (int64_t i = 0; i < prefetch_cnt_; ++i) {
      handles_[i].reset();
      handles_[i].~ObFuseRowValueHandle();
    }
    handles_ = nullptr;
  }
  prefetch_cnt_ = 0;
  reuse_iter_array();
}

void ObMultipleMerge::reset()
{
  ObStoreRowIterator* iter = NULL;
  for (int64_t i = 0; i < iters_.count(); ++i) {
    if (NULL != (iter = iters_.at(i))) {
      iter->~ObStoreRowIterator();
    }
  }
  padding_allocator_.reset();
  iters_.reset();
  access_param_ = NULL;
  access_ctx_ = NULL;
  tables_handle_.reset();
  nop_pos_.reset();
  row_stat_.reset();
  table_stat_.reset();
  scan_cnt_ = 0;
  filt_cnt_ = 0;
  need_padding_ = false;
  need_fill_default_ = false;
  need_fill_virtual_columns_ = false;
  need_output_row_with_nop_ = false;
  inited_ = false;
  range_idx_delta_ = 0;
  next_row_ = NULL;
  out_cols_projector_ = NULL;
  skip_refresh_table_ = false;
  read_memtable_only_ = false;
  is_tables_reset_ = false;
}

void ObMultipleMerge::reuse()
{
  reuse_iter_array();
  row_stat_.reset();
  table_stat_.reuse();
  range_idx_delta_ = 0;
  unprojected_row_.flag_ = -1;
  next_row_ = nullptr;
  skip_refresh_table_ = false;
  read_memtable_only_ = false;
}

void ObMultipleMerge::reuse_iter_array()
{
  ObStoreRowIterator* iter = NULL;
  for (int64_t i = 0; i < iters_.count(); ++i) {
    if (NULL != (iter = iters_.at(i))) {
      iter->~ObStoreRowIterator();
    }
  }
  iters_.reuse();
}

ObStoreRowIterator 只有空的reuse()函数，没有reset()函数。

ObMemtableScanIterator和ObMemtableMGetIterator的reuse()函数都是调用了自己的reset()函数。

ObSEArray.reset()函数最终调用了ObSEArrayImpl.destroy()函数。

template <typename T, int64_t LOCAL_ARRAY_SIZE, typename BlockAllocatorT, bool auto_free>
void ObSEArrayImpl<T, LOCAL_ARRAY_SIZE, BlockAllocatorT, auto_free>::destroy()
{
  if (is_destructor_safe()) {
  } else {
    for (int64_t i = 0; i < count_; i++) {
      data_[i].~T();
    }
  }

  if (data_ != reinterpret_cast<T*>(local_data_buf_)) {
    internal_free_(data_);
    data_ = reinterpret_cast<T*>(local_data_buf_);
  }
  count_ = 0;
#ifndef DISABLE_SE_ARRAY
  capacity_ = LOCAL_ARRAY_SIZE;
#else
  if (typeid(BlockAllocatorT) != typeid(ObNullAllocator)) {
    capacity_ = 0;
  } else {
    capacity_ = LOCAL_ARRAY_SIZE;
  }
#endif
  max_print_count_ = DEFAULT_MAX_PRINT_COUNT;
  error_ = OB_SUCCESS;
}

// ....

void reuse()
{
  if (is_destructor_safe()) {
  } else {
    for (int64_t i = 0; i < count_; i++) {
      data_[i].~T();
    }
  }
  count_ = 0;
  error_ = OB_SUCCESS;
}

更改后

void ObMultipleGetMerge::reset_with_fuse_row_cache()
{
  prefetch_range_idx_ = 0;
  get_row_range_idx_ = 0;
  sstable_rowkeys_.reset();
  if (nullptr != rows_) {
    for (int64_t i = 0; i < prefetch_cnt_; ++i) {
      rows_[i].~ObQueryRowInfo();
    }
    rows_ = nullptr;
  }
  if (nullptr != handles_) {
    for (int64_t i = 0; i < prefetch_cnt_; ++i) {
      handles_[i].reset();
      handles_[i].~ObFuseRowValueHandle();
    }
    handles_ = nullptr;
  }
  prefetch_cnt_ = 0;
  reset_iter_array();
}

void ObMultipleMerge::reset()
{
  ObStoreRowIterator* iter = NULL;
  for (int64_t i = 0; i < iters_.count(); ++i) {
    if (NULL != (iter = iters_.at(i))) {
      iter->~ObStoreRowIterator();
      if (OB_NOT_NULL(access_ctx_->stmt_allocator_)) {
         access_ctx_->stmt_allocator_->free(iter);
       }
       iter = NULL;
    }
  }
  padding_allocator_.reset();
  iters_.reset();
  access_param_ = NULL;
  access_ctx_ = NULL;
  tables_handle_.reset();
  nop_pos_.reset();
  row_stat_.reset();
  table_stat_.reset();
  scan_cnt_ = 0;
  filt_cnt_ = 0;
  need_padding_ = false;
  need_fill_default_ = false;
  need_fill_virtual_columns_ = false;
  need_output_row_with_nop_ = false;
  inited_ = false;
  range_idx_delta_ = 0;
  next_row_ = NULL;
  out_cols_projector_ = NULL;
  skip_refresh_table_ = false;
  read_memtable_only_ = false;
  is_tables_reset_ = false;
}

// .....

void ObMultipleMerge::reuse_iter_array()
{
  ObStoreRowIterator *iter = NULL;
  for (int64_t i = 0; i < iters_.count(); ++i) {
    if (NULL != (iter = iters_.at(i))) {
      iter->reset(); 
      // 细心地同学会发现，在reuse接口中使用reset，是有点怪异的，并且肯定没有最大化提高效率。那么为什么不能直接使用reuse呢？还是要从allocator与stmt_allocator中考虑，可以继续挖掘。注意allocator中内存释放的时机。
    }
  }
}

// ....

void ObMultipleMerge::reset_iter_array()
{
 ObStoreRowIterator* iter = NULL;
 for (int64_t i = 0; i < iters_.count(); ++i) {
   if (NULL != (iter = iters_.at(i))) {
     iter->~ObStoreRowIterator();
     if (OB_NOT_NULL(access_ctx_->stmt_allocator_)) {
       access_ctx_->stmt_allocator_->free(iter);
     }
     iter = NULL;
   }
 }
 iters_.reuse();
}

如果按照简单的sql语句：select /*+ordered use_nl(A,B)*/ count(*) from t1 A, v1 B where A.c1 = B.c1 and A.c2 = B.c2，在rescan阶段将会执行ObTableScanStoreRowIterator::rescan() -> ... ObMultipleScanMerge::reuse() -> ObMultipleMerge::reuse()，

patch2(sysbench: 3850)

diff --git a/etc/observer.config.bin b/etc/observer.config.bin
new file mode 100644
index 0000000..5e455cd
Binary files /dev/null and b/etc/observer.config.bin differ
diff --git a/etc2/observer.conf.bin b/etc2/observer.conf.bin
new file mode 100644
index 0000000..5e455cd
Binary files /dev/null and b/etc2/observer.conf.bin differ
diff --git a/etc3/observer.conf.bin b/etc3/observer.conf.bin
new file mode 100644
index 0000000..5e455cd
Binary files /dev/null and b/etc3/observer.conf.bin differ
diff --git a/src/storage/memtable/ob_memtable.cpp b/src/storage/memtable/ob_memtable.cpp
index 3a4d28f..ba7d295 100644
--- a/src/storage/memtable/ob_memtable.cpp
+++ b/src/storage/memtable/ob_memtable.cpp
@@ -1033,7 +1033,7 @@ int ObMemtable::get(const storage::ObTableIterParam& param, storage::ObTableAcce
     TRANS_LOG(WARN, "invalid argument, ", K(ret), K(param), K(context));
   } else if (OB_FAIL(context.store_ctx_->mem_ctx_->get_trans_status())) {
     TRANS_LOG(WARN, "trans already end", K(ret));
-  } else if (NULL == (get_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableGetIterator))) ||
+  } else if (NULL == (get_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableGetIterator))) ||
              NULL == (get_iter_ptr = new (get_iter_buffer) ObMemtableGetIterator())) {
     TRANS_LOG(WARN, "construct ObMemtableGetIterator fail");
     ret = OB_ALLOCATE_MEMORY_FAILED;
@@ -1082,7 +1082,7 @@ int ObMemtable::scan(const storage::ObTableIterParam& param, storage::ObTableAcc
   } else {
     if (param.is_multi_version_minor_merge_) {
       if (GCONF._enable_sparse_row) {
-        if (NULL == (scan_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableMultiVersionScanSparseIterator))) ||
+        if (NULL == (scan_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableMultiVersionScanSparseIterator))) ||
             NULL == (scan_iter_ptr = new (scan_iter_buffer) ObMemtableMultiVersionScanSparseIterator())) {
           TRANS_LOG(WARN,
               "construct ObMemtableMultiVersionScanSparseIterator fail",
@@ -1099,7 +1099,7 @@ int ObMemtable::scan(const storage::ObTableIterParam& param, storage::ObTableAcc
           TRANS_LOG(WARN, "scan iter init fail", "ret", ret, K(real_range), K(param), K(context));
         }
       } else {
-        if (NULL == (scan_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableMultiVersionScanIterator))) ||
+        if (NULL == (scan_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableMultiVersionScanIterator))) ||
             NULL == (scan_iter_ptr = new (scan_iter_buffer) ObMemtableMultiVersionScanIterator())) {
           TRANS_LOG(WARN,
               "construct ObMemtableScanIterator fail",
@@ -1117,7 +1117,7 @@ int ObMemtable::scan(const storage::ObTableIterParam& param, storage::ObTableAcc
         }
       }
     } else {
-      if (NULL == (scan_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableScanIterator))) ||
+      if (NULL == (scan_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableScanIterator))) ||
           NULL == (scan_iter_ptr = new (scan_iter_buffer) ObMemtableScanIterator())) {
         TRANS_LOG(WARN,
             "construct ObMemtableScanIterator fail",
@@ -1162,7 +1162,7 @@ int ObMemtable::multi_get(const storage::ObTableIterParam& param, storage::ObTab
     TRANS_LOG(WARN, "invalid argument, ", K(ret), K(param), K(context), K(rowkeys));
   } else if (OB_FAIL(context.store_ctx_->mem_ctx_->get_trans_status())) {
     TRANS_LOG(WARN, "trans already end", K(ret));
-  } else if (NULL == (mget_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableMGetIterator))) ||
+  } else if (NULL == (mget_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableMGetIterator))) ||
              NULL == (mget_iter_ptr = new (mget_iter_buffer) ObMemtableMGetIterator())) {
     TRANS_LOG(WARN,
         "construct ObMemtableMGetIterator fail",
@@ -1212,7 +1212,7 @@ int ObMemtable::multi_scan(const storage::ObTableIterParam& param, storage::ObTa
     TRANS_LOG(WARN, "invalid argument, ", K(ret), K(param), K(context), K(ranges));
   } else if (OB_FAIL(context.store_ctx_->mem_ctx_->get_trans_status())) {
     TRANS_LOG(WARN, "trans already end", K(ret));
-  } else if (NULL == (mscan_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableMScanIterator))) ||
+  } else if (NULL == (mscan_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableMScanIterator))) ||
              NULL == (mscan_iter_ptr = new (mscan_iter_buffer) ObMemtableMScanIterator())) {
     TRANS_LOG(WARN,
         "construct ObMemtableMScanIterator fail",
diff --git a/src/storage/memtable/ob_memtable_iterator.h b/src/storage/memtable/ob_memtable_iterator.h
index ef3a552..0601d87 100644
--- a/src/storage/memtable/ob_memtable_iterator.h
+++ b/src/storage/memtable/ob_memtable_iterator.h
@@ -100,6 +100,9 @@ public:
   {
     reset();
   }
+  virtual void reuse_try() override{
+    reset();
+  }
 
 private:
   // means GETITER
@@ -135,6 +138,9 @@ public:
   {
     reset();
   }
+  virtual void reuse_try() override{
+    reset();
+  }
   virtual int get_gap_end(int64_t& range_idx, const common::ObStoreRowkey*& gap_key, int64_t& gap_size) override
   {
     int ret = common::OB_SUCCESS;
@@ -216,6 +222,9 @@ public:
   {
     reset();
   }
+  virtual void reuse_try() override{
+    reset();
+  }
 
 public:
   static const int64_t ROW_ALLOCATOR_PAGE_SIZE = common::OB_MALLOC_NORMAL_BLOCK_SIZE;
@@ -254,6 +263,9 @@ public:
   {
     reset();
   }
+  virtual void reuse_try() override{
+    reset();
+  }
   virtual int skip_range(int64_t range_idx, const common::ObStoreRowkey* gap_key, const bool include_gap_key) override;
   virtual int get_gap_end(int64_t& range_idx, const common::ObStoreRowkey*& gap_key, int64_t& gap_size) override
   {
@@ -299,6 +311,9 @@ public:
   {
     reset();
   }
+  virtual void reuse_try() override{
+    reset();
+  }
   enum ScanState { SCAN_BEGIN, SCAN_UNCOMMITTED_ROW, SCAN_COMPACT_ROW, SCAN_MULTI_VERSION_ROW, SCAN_END };
   // the iteration process is divided into 2 phases:
   // iterating Complement SSTable / iterating Mini SSTable
diff --git a/src/storage/ob_i_store.h b/src/storage/ob_i_store.h
index e13283f..e2e5784 100644
--- a/src/storage/ob_i_store.h
+++ b/src/storage/ob_i_store.h
@@ -833,6 +833,10 @@ public:
   }
   virtual void reuse()
   {}
+  virtual void reset()
+  {}
+  virtual void reuse_try() 
+  {}
   virtual bool is_base_sstable_iter() const
   {
     return false;
diff --git a/src/storage/ob_multiple_get_merge.cpp b/src/storage/ob_multiple_get_merge.cpp
index ebfd26a..c568654 100644
--- a/src/storage/ob_multiple_get_merge.cpp
+++ b/src/storage/ob_multiple_get_merge.cpp
@@ -82,7 +82,7 @@ void ObMultipleGetMerge::reset_with_fuse_row_cache()
     handles_ = nullptr;
   }
   prefetch_cnt_ = 0;
-  reuse_iter_array();
+  reset_iter_array();
 }
 
 void ObMultipleGetMerge::reset()
diff --git a/src/storage/ob_multiple_merge.cpp b/src/storage/ob_multiple_merge.cpp
index 8a01f9d..6c59527 100644
--- a/src/storage/ob_multiple_merge.cpp
+++ b/src/storage/ob_multiple_merge.cpp
@@ -502,6 +502,10 @@ void ObMultipleMerge::reset()
   for (int64_t i = 0; i < iters_.count(); ++i) {
     if (NULL != (iter = iters_.at(i))) {
       iter->~ObStoreRowIterator();
+      if (OB_NOT_NULL(access_ctx_->stmt_allocator_)) {
+        access_ctx_->stmt_allocator_->free(iter);
+      }
+      iter = NULL;
     }
   }
   padding_allocator_.reset();
@@ -539,16 +543,29 @@ void ObMultipleMerge::reuse()
   read_memtable_only_ = false;
 }
 
-void ObMultipleMerge::reuse_iter_array()
+void ObMultipleMerge::reset_iter_array()
 {
   ObStoreRowIterator* iter = NULL;
   for (int64_t i = 0; i < iters_.count(); ++i) {
     if (NULL != (iter = iters_.at(i))) {
       iter->~ObStoreRowIterator();
+      if (OB_NOT_NULL(access_ctx_->stmt_allocator_)) {
+        access_ctx_->stmt_allocator_->free(iter);
+      }
+      iter = NULL;
     }
   }
   iters_.reuse();
 }
+void ObMultipleMerge::reuse_iter_array()
+{
+  ObStoreRowIterator *iter = NULL;
+  for (int64_t i = 0; i < iters_.count(); ++i) {
+    if (NULL != (iter = iters_.at(i))) {
+      iter->reuse_try(); 
+    }
+  }
+}
 
 int ObMultipleMerge::open()
 {
@@ -944,7 +961,7 @@ int ObMultipleMerge::refresh_table_on_demand()
   } else if (need_refresh) {
     if (OB_FAIL(save_curr_rowkey())) {
       STORAGE_LOG(WARN, "fail to save current rowkey", K(ret));
-    } else if (FALSE_IT(reuse_iter_array())) {
+    } else if (FALSE_IT(reset_iter_array())) {
     } else if (OB_FAIL(prepare_read_tables())) {
       STORAGE_LOG(WARN, "fail to prepare read tables", K(ret));
     } else if (OB_FAIL(reset_tables())) {
diff --git a/src/storage/ob_multiple_merge.h b/src/storage/ob_multiple_merge.h
index 12f8cdc..edc20f9 100644
--- a/src/storage/ob_multiple_merge.h
+++ b/src/storage/ob_multiple_merge.h
@@ -80,6 +80,7 @@ protected:
   const ObTableIterParam* get_actual_iter_param(const ObITable* table) const;
   int project_row(const ObStoreRow& unprojected_row, const common::ObIArray<int32_t>* projector,
       const int64_t range_idx_delta, ObStoreRow& projected_row);
+  void reset_iter_array();
   void reuse_iter_array();
   virtual int skip_to_range(const int64_t range_idx);
 
diff --git a/src/storage/ob_sstable.cpp b/src/storage/ob_sstable.cpp
index 13a3f0f..257c02d 100644
--- a/src/storage/ob_sstable.cpp
+++ b/src/storage/ob_sstable.cpp
@@ -1105,14 +1105,14 @@ int ObSSTable::get(const storage::ObTableIterParam& param, storage::ObTableAcces
     ObISSTableRowIterator* row_getter = NULL;
     if (is_multi_version_minor_sstable() && (context.is_multi_version_read(get_upper_trans_version()) ||
                                                 contain_uncommitted_row() || !meta_.has_compact_row_)) {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableMultiVersionRowGetter)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableMultiVersionRowGetter)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
         row_getter = new (buf) ObSSTableMultiVersionRowGetter();
       }
     } else {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowGetter)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowGetter)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
@@ -1163,14 +1163,14 @@ int ObSSTable::multi_get(const ObTableIterParam& param, ObTableAccessContext& co
       ObISSTableRowIterator* row_getter = NULL;
       if (is_multi_version_minor_sstable() && (context.is_multi_version_read(get_upper_trans_version()) ||
                                                   contain_uncommitted_row() || !meta_.has_compact_row_)) {
-        if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableMultiVersionRowMultiGetter)))) {
+        if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableMultiVersionRowMultiGetter)))) {
           ret = OB_ALLOCATE_MEMORY_FAILED;
           STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
         } else {
           row_getter = new (buf) ObSSTableMultiVersionRowMultiGetter();
         }
       } else {
-        if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowMultiGetter)))) {
+        if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowMultiGetter)))) {
           ret = OB_ALLOCATE_MEMORY_FAILED;
           STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
         } else {
@@ -1269,21 +1269,21 @@ int ObSSTable::scan(const ObTableIterParam& param, ObTableAccessContext& context
     void* buf = NULL;
     ObISSTableRowIterator* row_scanner = NULL;
     if (context.query_flag_.is_whole_macro_scan()) {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowWholeScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowWholeScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
         row_scanner = new (buf) ObSSTableRowWholeScanner();
       }
     } else if (is_multi_version_minor_sstable()) {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableMultiVersionRowScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableMultiVersionRowScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
         row_scanner = new (buf) ObSSTableMultiVersionRowScanner();
       }
     } else {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
@@ -1435,14 +1435,14 @@ int ObSSTable::multi_scan(const ObTableIterParam& param, ObTableAccessContext& c
     void* buf = NULL;
     ObISSTableRowIterator* row_scanner = NULL;
     if (is_multi_version_minor_sstable()) {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableMultiVersionRowMultiScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableMultiVersionRowMultiScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
         row_scanner = new (buf) ObSSTableMultiVersionRowMultiScanner();
       }
     } else {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowMultiScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowMultiScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
@@ -2930,7 +2930,7 @@ int ObSSTable::build_exist_iterator(const ObTableIterParam& iter_param, ObTableA
   } else {
     void* buf = NULL;
     ObSSTableRowExister* exister = NULL;
-    if (NULL == (buf = access_context.allocator_->alloc(sizeof(ObSSTableRowExister)))) {
+    if (NULL == (buf = access_context.stmt_allocator_->alloc(sizeof(ObSSTableRowExister)))) {
       ret = OB_ALLOCATE_MEMORY_FAILED;
       STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
     } else {
diff --git a/src/storage/ob_sstable_row_iterator.cpp b/src/storage/ob_sstable_row_iterator.cpp
index 6c261bf..62749f6 100644
--- a/src/storage/ob_sstable_row_iterator.cpp
+++ b/src/storage/ob_sstable_row_iterator.cpp
@@ -672,6 +672,59 @@ void ObSSTableRowIterator::reuse()
   prefetch_handle_depth_ = DEFAULT_PREFETCH_HANDLE_DEPTH;
   prefetch_micro_depth_ = DEFAULT_PREFETCH_MICRO_DEPTH;
 }
+void ObSSTableRowIterator::reuse_try(){
+  ObISSTableRowIterator::reset();
+  read_handles_.reset();
+  micro_handles_.reset();
+  sstable_micro_infos_.reset();
+
+  if (NULL != micro_exister_) {
+    micro_exister_->~ObMicroBlockRowExister();
+    micro_exister_ = NULL;
+  }
+  if (NULL != micro_getter_) {
+    micro_getter_->~ObMicroBlockRowGetter();
+    micro_getter_ = NULL;
+  }
+  if (NULL != micro_scanner_) {
+    micro_scanner_->~ObIMicroBlockRowScanner();
+    micro_scanner_ = NULL;
+  }
+  if (NULL != micro_lock_checker_) {
+    micro_lock_checker_->~ObMicroBlockRowLockChecker();
+    micro_lock_checker_ = NULL;
+  }
+
+  macro_block_iter_.reset();
+  iter_param_ = NULL;
+  access_ctx_ = NULL;
+  sstable_ = NULL;
+  query_range_ = NULL;
+  scan_step_ = 0;
+  is_opened_ = false;
+  is_base_ = false;
+  block_cache_ = NULL;
+  table_type_ = ObITable::MAJOR_SSTABLE;
+  sstable_snapshot_version_ = 0;
+  prefetch_handle_end_ = false;
+  prefetch_block_end_ = false;
+  cur_prefetch_handle_pos_ = 0;
+  cur_fetch_handle_pos_ = 0;
+  cur_read_handle_pos_ = 0;
+  cur_prefetch_micro_pos_ = 0;
+  cur_read_micro_pos_ = 0;
+  cur_micro_idx_ = -1;
+  cur_range_idx_ = -1;
+  io_micro_infos_.reset();
+  micro_info_iter_.reset();
+  block_index_handle_mgr_.reset();
+  block_handle_mgr_.reset();
+  table_store_stat_.reset();
+  skip_ctx_.reset();
+  storage_file_ = nullptr;
+  prefetch_handle_depth_ = DEFAULT_PREFETCH_HANDLE_DEPTH;
+  prefetch_micro_depth_ = DEFAULT_PREFETCH_MICRO_DEPTH;
+}
 
 int ObSSTableRowIterator::get_read_handle(const ObExtStoreRowkey& ext_rowkey, ObSSTableReadHandle& read_handle)
 {
diff --git a/src/storage/ob_sstable_row_iterator.h b/src/storage/ob_sstable_row_iterator.h
index dadb33b..2b46bf9 100644
--- a/src/storage/ob_sstable_row_iterator.h
+++ b/src/storage/ob_sstable_row_iterator.h
@@ -176,6 +176,9 @@ public:
   virtual int get_next_row(const ObStoreRow*& store_row);
   virtual void reset();
   virtual void reuse();
+  virtual void reuse_try(){
+    reset();
+  }
   OB_INLINE bool has_lob_column() const
   {
     return OB_NOT_NULL(lob_reader_);
@@ -321,6 +324,7 @@ public:
   virtual ~ObSSTableRowIterator();
   virtual void reset() override;
   virtual void reuse() override;
+  virtual void reuse_try() override;
   virtual int get_skip_range_ctx(
       ObSSTableReadHandle& read_handle, const int64_t cur_micro_idx, ObSSTableSkipRangeCtx*& skip_ctx);
   int get_row_iter_flag_impl(uint8_t& flag);

patch3

diff --git a/src/storage/blocksstable/ob_block_sstable_struct.cpp b/src/storage/blocksstable/ob_block_sstable_struct.cpp
index b455124b..3399b69c 100644
--- a/src/storage/blocksstable/ob_block_sstable_struct.cpp
+++ b/src/storage/blocksstable/ob_block_sstable_struct.cpp
@@ -4569,5 +4569,11 @@ int64_t ObFullMacroBlockMetaEntry::get_serialize_size() const
   return meta_.get_serialize_size() + schema_.get_serialize_size();
 }
 
+void ObSSTableMacroBlockId::reuse()
+{
+  // macro_block_id_.reset();
+  macro_block_id_in_files_ = -1;
+}
+
 }  // namespace blocksstable
 }  // namespace oceanbase
diff --git a/src/storage/blocksstable/ob_block_sstable_struct.h b/src/storage/blocksstable/ob_block_sstable_struct.h
index 42237e0c..a583f15c 100644
--- a/src/storage/blocksstable/ob_block_sstable_struct.h
+++ b/src/storage/blocksstable/ob_block_sstable_struct.h
@@ -859,6 +859,7 @@ struct ObSSTableMacroBlockId {
 
   OB_INLINE bool is_valid() const;
   OB_INLINE void reset();
+  void reuse();
   TO_STRING_KV(K_(macro_block_id), K_(macro_block_id_in_files));
 };
 
@@ -871,6 +872,11 @@ struct ObMacroBlockCtx final {
 
   OB_INLINE bool is_valid() const;
   OB_INLINE void reset();
+  OB_INLINE void reuse() {
+    // file_ctx_ = NULL;
+    // sstable_block_id_.reuse();
+    // sstable_ = nullptr;
+  }
 
   OB_INLINE const blocksstable::MacroBlockId& get_macro_block_id() const;
   TO_STRING_KV(K_(sstable_block_id), KP_(file_ctx));
diff --git a/src/storage/blocksstable/ob_micro_block_row_scanner.cpp b/src/storage/blocksstable/ob_micro_block_row_scanner.cpp
index d6fd2648..6114829b 100644
--- a/src/storage/blocksstable/ob_micro_block_row_scanner.cpp
+++ b/src/storage/blocksstable/ob_micro_block_row_scanner.cpp
@@ -154,6 +154,29 @@ void ObIMicroBlockRowScanner::reset()
   is_inited_ = false;
 }
 
+void ObIMicroBlockRowScanner::reuse()
+{
+  lob_reader_.reuse();
+  has_lob_column_ = false;
+  param_ = NULL;
+  context_ = NULL;
+  range_ = NULL;
+  sstable_ = NULL;
+  column_map_.reuse();
+  // if (OB_NOT_NULL(reader_)) {
+  //   reader_->reset();
+  // }
+  current_ = ObIMicroBlockReader::INVALID_ROW_INDEX;
+  start_ = ObIMicroBlockReader::INVALID_ROW_INDEX;
+  last_ = ObIMicroBlockReader::INVALID_ROW_INDEX;
+  reverse_scan_ = false;
+  is_left_border_ = false;
+  is_right_border_ = false;
+  step_ = 1;
+  // macro_id_.reset();
+  is_inited_ = false;
+}
+
 void ObIMicroBlockRowScanner::rescan()
 {
   range_ = NULL;
@@ -445,7 +468,7 @@ int ObMicroBlockRowScanner::init(const ObTableIterParam& param, ObTableAccessCon
     STORAGE_LOG(WARN, "fail to get projector", K(ret));
   } else if (OB_FAIL(param_->get_column_map(false /*is get*/, column_id_map))) {
     STORAGE_LOG(WARN, "fail to get column id map", K(ret));
-  } else if (OB_FAIL(column_map_.init(*context_->allocator_,
+  } else if (OB_FAIL(column_map_.init(*context_->stmt_allocator_,
                  param_->schema_version_,
                  param_->rowkey_cnt_,
                  0, /*store count*/
@@ -557,6 +580,11 @@ void ObMicroBlockRowScanner::reset()
   ObIMicroBlockRowScanner::reset();
 }
 
+void ObMicroBlockRowScanner::reuse()
+{
+  ObIMicroBlockRowScanner::reuse();
+}
+
 int ObMultiVersionMicroBlockRowScanner::init(
     const ObTableIterParam& param, ObTableAccessContext& context, const ObSSTable* sstable)
 {
@@ -573,7 +601,7 @@ int ObMultiVersionMicroBlockRowScanner::init(
     STORAGE_LOG(WARN, "fail to get projector", K(ret));
   } else if (OB_FAIL(param_->get_column_map(context.use_fuse_row_cache_, column_id_map))) {
     STORAGE_LOG(WARN, "fail to get column id map", K(ret));
-  } else if (OB_FAIL(column_map_.init(*context_->allocator_,
+  } else if (OB_FAIL(column_map_.init(*context_->stmt_allocator_,
                  param_->schema_version_,
                  param_->rowkey_cnt_,
                  0, /*store count*/
@@ -1358,7 +1386,7 @@ int ObMultiVersionMicroBlockMinorMergeRowScanner::init(
     // minor merge should contain 2
     if (OB_FAIL(build_minor_merge_out_cols(*param_, out_cols, expect_multi_version_col_cnt))) {
       STORAGE_LOG(WARN, "fail to build minor merge out columns", K(ret));
-    } else if (OB_FAIL(column_map_.init(*context_->allocator_,
+    } else if (OB_FAIL(column_map_.init(*context_->stmt_allocator_,
                    param_->schema_version_,
                    param_->rowkey_cnt_,
                    0, /*store count*/
diff --git a/src/storage/blocksstable/ob_micro_block_row_scanner.h b/src/storage/blocksstable/ob_micro_block_row_scanner.h
index 89ec845a..76d7587c 100644
--- a/src/storage/blocksstable/ob_micro_block_row_scanner.h
+++ b/src/storage/blocksstable/ob_micro_block_row_scanner.h
@@ -64,6 +64,7 @@ public:
   virtual int get_next_rows(const storage::ObStoreRow*& rows, int64_t& count);
   virtual void reset();
   virtual void rescan();
+  virtual void reuse();
   int alloc_row(ObIAllocator& allocator, const int64_t cell_cnt, storage::ObStoreRow& row);
   virtual int get_cur_micro_row_count(int64_t& row_count) const;
 
@@ -127,6 +128,7 @@ public:
   virtual int open(const MacroBlockId& macro_id, const ObFullMacroBlockMeta& macro_meta,
       const ObMicroBlockData& block_data, const bool is_left_border, const bool is_right_border) override;
   void reset() override;
+  void reuse() override;
 
 protected:
   virtual int inner_get_next_row(const storage::ObStoreRow*& row) override;
diff --git a/src/storage/blocksstable/ob_row_reader.cpp b/src/storage/blocksstable/ob_row_reader.cpp
index 32880944..b02f48d9 100644
--- a/src/storage/blocksstable/ob_row_reader.cpp
+++ b/src/storage/blocksstable/ob_row_reader.cpp
@@ -650,7 +650,9 @@ int ObFlatRowReader::compare_meta_rowkey(const common::ObStoreRowkey& rhs, const
         if (OB_FAIL(read_obj_no_meta(obj_meta, allocator_, obj))) {
           STORAGE_LOG(WARN, "row reader fail to read column.", K(ret), K(i));
         } else {
-          cmp_result = obj.compare(rhs.get_obj_ptr()[i], common::CS_TYPE_INVALID);
+          if (OB_FAIL(obj.compare(rhs.get_obj_ptr()[i], common::CS_TYPE_INVALID, cmp_result))) {
+            return OB_ERR_UNEXPECTED;
+          }
         }
       }  // end for
     }
diff --git a/src/storage/memtable/ob_memtable.cpp b/src/storage/memtable/ob_memtable.cpp
index 3a4d28f7..eec61ec2 100644
--- a/src/storage/memtable/ob_memtable.cpp
+++ b/src/storage/memtable/ob_memtable.cpp
@@ -927,7 +927,7 @@ int ObMemtable::get(const storage::ObTableIterParam& param, storage::ObTableAcce
     const ColumnMap* param_column_map = nullptr;
     if (nullptr == row.row_val_.cells_) {
       if (nullptr ==
-          (row.row_val_.cells_ = static_cast<ObObj*>(context.allocator_->alloc(sizeof(ObObj) * out_cols->count())))) {
+          (row.row_val_.cells_ = static_cast<ObObj*>(context.stmt_allocator_->alloc(sizeof(ObObj) * out_cols->count())))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         TRANS_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
@@ -940,11 +940,11 @@ int ObMemtable::get(const storage::ObTableIterParam& param, storage::ObTableAcce
       TRANS_LOG(WARN, "fail to get column map", K(ret));
     } else if (NULL == param_column_map) {
       void* buf = NULL;
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ColumnMap)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ColumnMap)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         TRANS_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
-        local_map = new (buf) ColumnMap(*context.allocator_);
+        local_map = new (buf) ColumnMap(*context.stmt_allocator_);
         if (OB_FAIL(local_map->init(*out_cols))) {
           TRANS_LOG(WARN, "Fail to build column map, ", K(ret));
         }
@@ -1033,7 +1033,7 @@ int ObMemtable::get(const storage::ObTableIterParam& param, storage::ObTableAcce
     TRANS_LOG(WARN, "invalid argument, ", K(ret), K(param), K(context));
   } else if (OB_FAIL(context.store_ctx_->mem_ctx_->get_trans_status())) {
     TRANS_LOG(WARN, "trans already end", K(ret));
-  } else if (NULL == (get_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableGetIterator))) ||
+  } else if (NULL == (get_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableGetIterator))) ||
              NULL == (get_iter_ptr = new (get_iter_buffer) ObMemtableGetIterator())) {
     TRANS_LOG(WARN, "construct ObMemtableGetIterator fail");
     ret = OB_ALLOCATE_MEMORY_FAILED;
@@ -1082,7 +1082,7 @@ int ObMemtable::scan(const storage::ObTableIterParam& param, storage::ObTableAcc
   } else {
     if (param.is_multi_version_minor_merge_) {
       if (GCONF._enable_sparse_row) {
-        if (NULL == (scan_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableMultiVersionScanSparseIterator))) ||
+        if (NULL == (scan_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableMultiVersionScanSparseIterator))) ||
             NULL == (scan_iter_ptr = new (scan_iter_buffer) ObMemtableMultiVersionScanSparseIterator())) {
           TRANS_LOG(WARN,
               "construct ObMemtableMultiVersionScanSparseIterator fail",
@@ -1099,7 +1099,7 @@ int ObMemtable::scan(const storage::ObTableIterParam& param, storage::ObTableAcc
           TRANS_LOG(WARN, "scan iter init fail", "ret", ret, K(real_range), K(param), K(context));
         }
       } else {
-        if (NULL == (scan_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableMultiVersionScanIterator))) ||
+        if (NULL == (scan_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableMultiVersionScanIterator))) ||
             NULL == (scan_iter_ptr = new (scan_iter_buffer) ObMemtableMultiVersionScanIterator())) {
           TRANS_LOG(WARN,
               "construct ObMemtableScanIterator fail",
@@ -1117,7 +1117,7 @@ int ObMemtable::scan(const storage::ObTableIterParam& param, storage::ObTableAcc
         }
       }
     } else {
-      if (NULL == (scan_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableScanIterator))) ||
+      if (NULL == (scan_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableScanIterator))) ||
           NULL == (scan_iter_ptr = new (scan_iter_buffer) ObMemtableScanIterator())) {
         TRANS_LOG(WARN,
             "construct ObMemtableScanIterator fail",
@@ -1162,7 +1162,7 @@ int ObMemtable::multi_get(const storage::ObTableIterParam& param, storage::ObTab
     TRANS_LOG(WARN, "invalid argument, ", K(ret), K(param), K(context), K(rowkeys));
   } else if (OB_FAIL(context.store_ctx_->mem_ctx_->get_trans_status())) {
     TRANS_LOG(WARN, "trans already end", K(ret));
-  } else if (NULL == (mget_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableMGetIterator))) ||
+  } else if (NULL == (mget_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableMGetIterator))) ||
              NULL == (mget_iter_ptr = new (mget_iter_buffer) ObMemtableMGetIterator())) {
     TRANS_LOG(WARN,
         "construct ObMemtableMGetIterator fail",
@@ -1212,7 +1212,7 @@ int ObMemtable::multi_scan(const storage::ObTableIterParam& param, storage::ObTa
     TRANS_LOG(WARN, "invalid argument, ", K(ret), K(param), K(context), K(ranges));
   } else if (OB_FAIL(context.store_ctx_->mem_ctx_->get_trans_status())) {
     TRANS_LOG(WARN, "trans already end", K(ret));
-  } else if (NULL == (mscan_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableMScanIterator))) ||
+  } else if (NULL == (mscan_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableMScanIterator))) ||
              NULL == (mscan_iter_ptr = new (mscan_iter_buffer) ObMemtableMScanIterator())) {
     TRANS_LOG(WARN,
         "construct ObMemtableMScanIterator fail",
diff --git a/src/storage/ob_micro_block_handle_mgr.cpp b/src/storage/ob_micro_block_handle_mgr.cpp
index 028a2018..39ea7cca 100644
--- a/src/storage/ob_micro_block_handle_mgr.cpp
+++ b/src/storage/ob_micro_block_handle_mgr.cpp
@@ -45,6 +45,17 @@ void ObMicroBlockDataHandle::reset()
   io_handle_.reset();
 }
 
+void ObMicroBlockDataHandle::reuse()
+{
+  block_state_ = ObSSTableMicroBlockState::UNKNOWN_STATE;
+  table_id_ = 0;
+  block_ctx_.reset();
+  block_index_ = -1;
+  micro_info_.reset();
+  cache_handle_.reset();
+  io_handle_.reset();
+}
+
 int ObMicroBlockDataHandle::get_block_data(
     ObMacroBlockReader& block_reader, ObStorageFile* storage_file, ObMicroBlockData& block_data)
 {
diff --git a/src/storage/ob_micro_block_handle_mgr.h b/src/storage/ob_micro_block_handle_mgr.h
index 37f6d005..1ff90688 100644
--- a/src/storage/ob_micro_block_handle_mgr.h
+++ b/src/storage/ob_micro_block_handle_mgr.h
@@ -30,6 +30,7 @@ struct ObMicroBlockDataHandle {
   ObMicroBlockDataHandle();
   virtual ~ObMicroBlockDataHandle();
   void reset();
+  void reuse();
   int get_block_data(blocksstable::ObMacroBlockReader& block_reader, blocksstable::ObStorageFile* storage_file,
       blocksstable::ObMicroBlockData& block_data);
   TO_STRING_KV(
diff --git a/src/storage/ob_multiple_merge.cpp b/src/storage/ob_multiple_merge.cpp
index 8a01f9de..20893d3a 100644
--- a/src/storage/ob_multiple_merge.cpp
+++ b/src/storage/ob_multiple_merge.cpp
@@ -485,7 +485,7 @@ int ObMultipleMerge::get_next_row(ObStoreRow*& row)
         access_ctx_->table_scan_stat_->row_cache_hit_cnt_ += access_ctx_->access_stat_.row_cache_hit_cnt_;
         access_ctx_->table_scan_stat_->row_cache_miss_cnt_ += access_ctx_->access_stat_.row_cache_miss_cnt_;
       }
-      report_table_store_stat();
+      // report_table_store_stat();
     }
     if (OB_SUCC(ret)) {
       if (NULL != access_ctx_->table_scan_stat_) {
diff --git a/src/storage/ob_multiple_merge.h b/src/storage/ob_multiple_merge.h
index 12f8cdc2..a5a049ea 100644
--- a/src/storage/ob_multiple_merge.h
+++ b/src/storage/ob_multiple_merge.h
@@ -80,7 +80,7 @@ protected:
   const ObTableIterParam* get_actual_iter_param(const ObITable* table) const;
   int project_row(const ObStoreRow& unprojected_row, const common::ObIArray<int32_t>* projector,
       const int64_t range_idx_delta, ObStoreRow& projected_row);
-  void reuse_iter_array();
+  virtual void reuse_iter_array();
   virtual int skip_to_range(const int64_t range_idx);
 
 private:
diff --git a/src/storage/ob_multiple_scan_merge.cpp b/src/storage/ob_multiple_scan_merge.cpp
index 958c335e..750f80a3 100644
--- a/src/storage/ob_multiple_scan_merge.cpp
+++ b/src/storage/ob_multiple_scan_merge.cpp
@@ -184,6 +184,16 @@ void ObMultipleScanMerge::reuse()
   return ObMultipleScanMergeImpl::reuse();
 }
 
+void ObMultipleScanMerge::reuse_iter_array()
+{
+  ObStoreRowIterator* iter = NULL;
+  for (int64_t i = 0; i < iters_.count(); ++i) {
+    if (NULL != (iter = iters_.at(i))) {
+      iter->reuse();
+    }
+  }
+}
+
 int ObMultipleScanMerge::inner_get_next_row(ObStoreRow& row)
 {
   int ret = OB_SUCCESS;
diff --git a/src/storage/ob_multiple_scan_merge.h b/src/storage/ob_multiple_scan_merge.h
index 455860eb..a3bb83be 100644
--- a/src/storage/ob_multiple_scan_merge.h
+++ b/src/storage/ob_multiple_scan_merge.h
@@ -28,6 +28,7 @@ public:
   int open(const common::ObExtStoreRange& range);
   virtual void reset() override;
   virtual void reuse() override;
+  virtual void reuse_iter_array() override;
   inline void set_iter_del_row(const bool iter_del_row)
   {
     iter_del_row_ = iter_del_row;
diff --git a/src/storage/ob_old_sstable.cpp b/src/storage/ob_old_sstable.cpp
index d41f0c42..6d4e639a 100644
--- a/src/storage/ob_old_sstable.cpp
+++ b/src/storage/ob_old_sstable.cpp
@@ -861,21 +861,21 @@ int ObOldSSTable::scan(const ObTableIterParam& param, ObTableAccessContext& cont
     void* buf = NULL;
     ObISSTableRowIterator* row_scanner = NULL;
     if (context.query_flag_.is_whole_macro_scan()) {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowWholeScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowWholeScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
         row_scanner = new (buf) ObSSTableRowWholeScanner();
       }
     } else if (is_multi_version_minor_sstable()) {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableMultiVersionRowScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableMultiVersionRowScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
         row_scanner = new (buf) ObSSTableMultiVersionRowScanner();
       }
     } else {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
diff --git a/src/storage/ob_sstable.cpp b/src/storage/ob_sstable.cpp
index 13a3f0fa..138d06a0 100644
--- a/src/storage/ob_sstable.cpp
+++ b/src/storage/ob_sstable.cpp
@@ -1269,21 +1269,21 @@ int ObSSTable::scan(const ObTableIterParam& param, ObTableAccessContext& context
     void* buf = NULL;
     ObISSTableRowIterator* row_scanner = NULL;
     if (context.query_flag_.is_whole_macro_scan()) {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowWholeScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowWholeScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
         row_scanner = new (buf) ObSSTableRowWholeScanner();
       }
     } else if (is_multi_version_minor_sstable()) {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableMultiVersionRowScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableMultiVersionRowScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
         row_scanner = new (buf) ObSSTableMultiVersionRowScanner();
       }
     } else {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
diff --git a/src/storage/ob_sstable_row_iterator.cpp b/src/storage/ob_sstable_row_iterator.cpp
index 6c261bfd..9d9e822d 100644
--- a/src/storage/ob_sstable_row_iterator.cpp
+++ b/src/storage/ob_sstable_row_iterator.cpp
@@ -413,6 +413,7 @@ ObSSTableRowIterator::ObSSTableRowIterator()
       micro_getter_(NULL),
       micro_lock_checker_(NULL),
       micro_scanner_(NULL),
+      micro_scanner_inited_(false),
       read_handles_(),
       prefetch_handle_end_(false),
       prefetch_block_end_(false),
@@ -469,13 +470,13 @@ int ObSSTableRowIterator::inner_open(
     STORAGE_LOG(WARN, "Unexpected error, ", K(ret), K_(read_handle_cnt), K_(micro_handle_cnt));
   } else if (OB_FAIL(init_handle_mgr(iter_param, access_ctx, query_range))) {
     STORAGE_LOG(WARN, "fail to init handle mgr", K(ret), K(iter_param), K(access_ctx));
-  } else if (OB_FAIL(read_handles_.reserve(*access_ctx.allocator_, read_handle_cnt_))) {
+  } else if (OB_FAIL(read_handles_.reserve(*access_ctx.stmt_allocator_, read_handle_cnt_))) {
     STORAGE_LOG(WARN, "failed to reserve read handles", K(ret), K_(read_handle_cnt));
-  } else if (OB_FAIL(micro_handles_.reserve(*access_ctx.allocator_, micro_handle_cnt_))) {
+  } else if (OB_FAIL(micro_handles_.reserve(*access_ctx.stmt_allocator_, micro_handle_cnt_))) {
     STORAGE_LOG(WARN, "failed to reserve micro handles", K(ret), K_(micro_handle_cnt));
-  } else if (OB_FAIL(sstable_micro_infos_.reserve(*access_ctx.allocator_, micro_handle_cnt_))) {
+  } else if (OB_FAIL(sstable_micro_infos_.reserve(*access_ctx.stmt_allocator_, micro_handle_cnt_))) {
     STORAGE_LOG(WARN, "failed to reserve sstable micro infos", K(ret), K_(micro_handle_cnt));
-  } else if (OB_FAIL(sorted_sstable_micro_infos_.reserve(*access_ctx.allocator_, micro_handle_cnt_))) {
+  } else if (OB_FAIL(sorted_sstable_micro_infos_.reserve(*access_ctx.stmt_allocator_, micro_handle_cnt_))) {
     STORAGE_LOG(WARN, "failed to reserve sorted sstable micro infos", K(ret), K_(micro_handle_cnt));
   } else {
     sstable_ = static_cast<ObSSTable*>(table);
@@ -643,8 +644,31 @@ void ObSSTableRowIterator::reset()
 void ObSSTableRowIterator::reuse()
 {
   ObISSTableRowIterator::reuse();
+  // NOTE: Seems unnecessary
+  // read_handles_.reset();
+  // NOTE: Seems unnecessary
+  // micro_handles_.reuse();
+  sstable_micro_infos_.reuse();
+  // NOTE: Seems unnecessary
+  // sorted_sstable_micro_infos_.reuse();
+
+  if (NULL != micro_exister_) {
+    micro_exister_->~ObMicroBlockRowExister();
+    micro_exister_ = NULL;
+  }
+  if (NULL != micro_getter_) {
+    micro_getter_->~ObMicroBlockRowGetter();
+    micro_getter_ = NULL;
+  }
   if (NULL != micro_scanner_) {
-    micro_scanner_->rescan();
+    micro_scanner_->reuse();
+    micro_scanner_inited_ = false;
+    // micro_scanner_->~ObIMicroBlockRowScanner();
+    // micro_scanner_ = NULL;
+  }
+  if (NULL != micro_lock_checker_) {
+    micro_lock_checker_->~ObMicroBlockRowLockChecker();
+    micro_lock_checker_ = NULL;
   }
 
   macro_block_iter_.reset();
@@ -664,8 +688,8 @@ void ObSSTableRowIterator::reuse()
   cur_range_idx_ = -1;
   io_micro_infos_.reuse();
   micro_info_iter_.reuse();
-  block_index_handle_mgr_.reset();
-  block_handle_mgr_.reset();
+  // block_index_handle_mgr_.reset();
+  // block_handle_mgr_.reset();
   table_store_stat_.reuse();
   skip_ctx_.reset();
   storage_file_ = nullptr;
@@ -1572,23 +1596,27 @@ int ObSSTableRowIterator::open_cur_micro_block(ObSSTableReadHandle& read_handle,
   if (NULL == micro_scanner_) {
     // alloc scanner
     if (!sstable_->is_multi_version_minor_sstable()) {
-      if (NULL == (buf = access_ctx_->allocator_->alloc(sizeof(ObMicroBlockRowScanner)))) {
+      if (NULL == (buf = access_ctx_->stmt_allocator_->alloc(sizeof(ObMicroBlockRowScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory for micro block scanner, ", K(ret));
       } else {
         micro_scanner_ = new (buf) ObMicroBlockRowScanner();
       }
     } else {
-      if (NULL == (buf = access_ctx_->allocator_->alloc(sizeof(ObMultiVersionMicroBlockRowScanner)))) {
+      if (NULL == (buf = access_ctx_->stmt_allocator_->alloc(sizeof(ObMultiVersionMicroBlockRowScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory for micro block scanner, ", K(ret));
       } else {
         micro_scanner_ = new (buf) ObMultiVersionMicroBlockRowScanner();
       }
     }
+  }
+  if (!micro_scanner_inited_) {
     if (OB_SUCC(ret)) {
       if (OB_FAIL(micro_scanner_->init(*iter_param_, *access_ctx_, sstable_))) {
         STORAGE_LOG(WARN, "Fail to init micro scanner, ", K(ret), K(read_handle));
+      } else {
+        micro_scanner_inited_ = true;
       }
     }
   }
@@ -1690,7 +1718,7 @@ int ObSSTableRowIterator::init_handle_mgr(
     ret = OB_ERR_UNEXPECTED;
     STORAGE_LOG(WARN, "range count should be greater than 0", K(ret), K(range_count));
   } else if (1 == range_count) {
-    is_multi = false;
+    is_multi = true; // NOTE: force enable cache of *_handle_mgr_ for nlj benchmark
     is_ordered = false;
   } else {
     is_multi = true;
@@ -1701,9 +1729,9 @@ int ObSSTableRowIterator::init_handle_mgr(
             range_count >= USE_HANDLE_CACHE_RANGE_COUNT_THRESHOLD);
   }
   if (OB_SUCC(ret)) {
-    if (OB_FAIL(block_handle_mgr_.init(is_multi, true, *access_ctx.allocator_))) {
+    if (!block_handle_mgr_.is_inited() && OB_FAIL(block_handle_mgr_.init(is_multi, true, *access_ctx.stmt_allocator_))) {
       STORAGE_LOG(WARN, "failed to init block handle mgr", K(ret), K(is_multi), K(is_ordered));
-    } else if (OB_FAIL(block_index_handle_mgr_.init(is_multi, is_ordered, *access_ctx.allocator_))) {
+    } else if (!block_index_handle_mgr_.is_inited() && OB_FAIL(block_index_handle_mgr_.init(is_multi, is_ordered, *access_ctx.stmt_allocator_))) {
       STORAGE_LOG(WARN, "failed to init block index handle mgr", K(ret), K(is_multi), K(is_ordered));
     }
   }
diff --git a/src/storage/ob_sstable_row_iterator.h b/src/storage/ob_sstable_row_iterator.h
index dadb33b8..25f137ff 100644
--- a/src/storage/ob_sstable_row_iterator.h
+++ b/src/storage/ob_sstable_row_iterator.h
@@ -104,6 +104,12 @@ struct ObSSTableMicroBlockInfo {
   blocksstable::ObMicroBlockInfo micro_info_;
   int64_t micro_idx_;
   bool is_skip_;
+  void reuse() {
+    macro_ctx_.reuse();
+    // micro_info_.reset();
+    micro_idx_ = -1;
+    is_skip_ = false;
+  }
 };
 
 class ObSSTableMicroBlockInfoCmp {
@@ -250,6 +256,11 @@ public:
     array_ = nullptr;
     capacity_ = 0;
   }
+  inline void reuse() {
+    for (int64_t i = 0; i < capacity_; ++i) {
+      array_[i].reuse();
+    }
+  }
   int reserve(common::ObArenaAllocator& allocator, const int64_t count)
   {
     int ret = common::OB_SUCCESS;
@@ -406,6 +417,7 @@ private:
   blocksstable::ObMicroBlockRowGetter* micro_getter_;
   blocksstable::ObMicroBlockRowLockChecker* micro_lock_checker_;
   blocksstable::ObIMicroBlockRowScanner* micro_scanner_;
+  bool micro_scanner_inited_;
   ReadHandleArray read_handles_;
   bool prefetch_handle_end_;
   bool prefetch_block_end_;

patch4

diff --git a/diff b/diff
index 3a4d28f..ba7d295 100644
--- a/src/storage/memtable/ob_memtable.cpp
+++ b/src/storage/memtable/ob_memtable.cpp
@@ -1033,7 +1033,7 @@ int ObMemtable::get(const storage::ObTableIterParam& param, storage::ObTableAcce
     TRANS_LOG(WARN, "invalid argument, ", K(ret), K(param), K(context));
   } else if (OB_FAIL(context.store_ctx_->mem_ctx_->get_trans_status())) {
     TRANS_LOG(WARN, "trans already end", K(ret));
-  } else if (NULL == (get_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableGetIterator))) ||
+  } else if (NULL == (get_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableGetIterator))) ||
              NULL == (get_iter_ptr = new (get_iter_buffer) ObMemtableGetIterator())) {
     TRANS_LOG(WARN, "construct ObMemtableGetIterator fail");
     ret = OB_ALLOCATE_MEMORY_FAILED;
@@ -1082,7 +1082,7 @@ int ObMemtable::scan(const storage::ObTableIterParam& param, storage::ObTableAcc
   } else {
     if (param.is_multi_version_minor_merge_) {
       if (GCONF._enable_sparse_row) {
-        if (NULL == (scan_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableMultiVersionScanSparseIterator))) ||
+        if (NULL == (scan_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableMultiVersionScanSparseIterator))) ||
             NULL == (scan_iter_ptr = new (scan_iter_buffer) ObMemtableMultiVersionScanSparseIterator())) {
           TRANS_LOG(WARN,
               "construct ObMemtableMultiVersionScanSparseIterator fail",
@@ -1099,7 +1099,7 @@ int ObMemtable::scan(const storage::ObTableIterParam& param, storage::ObTableAcc
           TRANS_LOG(WARN, "scan iter init fail", "ret", ret, K(real_range), K(param), K(context));
         }
       } else {
-        if (NULL == (scan_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableMultiVersionScanIterator))) ||
+        if (NULL == (scan_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableMultiVersionScanIterator))) ||
             NULL == (scan_iter_ptr = new (scan_iter_buffer) ObMemtableMultiVersionScanIterator())) {
           TRANS_LOG(WARN,
               "construct ObMemtableScanIterator fail",
@@ -1117,7 +1117,7 @@ int ObMemtable::scan(const storage::ObTableIterParam& param, storage::ObTableAcc
         }
       }
     } else {
-      if (NULL == (scan_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableScanIterator))) ||
+      if (NULL == (scan_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableScanIterator))) ||
           NULL == (scan_iter_ptr = new (scan_iter_buffer) ObMemtableScanIterator())) {
         TRANS_LOG(WARN,
             "construct ObMemtableScanIterator fail",
@@ -1162,7 +1162,7 @@ int ObMemtable::multi_get(const storage::ObTableIterParam& param, storage::ObTab
     TRANS_LOG(WARN, "invalid argument, ", K(ret), K(param), K(context), K(rowkeys));
   } else if (OB_FAIL(context.store_ctx_->mem_ctx_->get_trans_status())) {
     TRANS_LOG(WARN, "trans already end", K(ret));
-  } else if (NULL == (mget_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableMGetIterator))) ||
+  } else if (NULL == (mget_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableMGetIterator))) ||
              NULL == (mget_iter_ptr = new (mget_iter_buffer) ObMemtableMGetIterator())) {
     TRANS_LOG(WARN,
         "construct ObMemtableMGetIterator fail",
@@ -1212,7 +1212,7 @@ int ObMemtable::multi_scan(const storage::ObTableIterParam& param, storage::ObTa
     TRANS_LOG(WARN, "invalid argument, ", K(ret), K(param), K(context), K(ranges));
   } else if (OB_FAIL(context.store_ctx_->mem_ctx_->get_trans_status())) {
     TRANS_LOG(WARN, "trans already end", K(ret));
-  } else if (NULL == (mscan_iter_buffer = context.allocator_->alloc(sizeof(ObMemtableMScanIterator))) ||
+  } else if (NULL == (mscan_iter_buffer = context.stmt_allocator_->alloc(sizeof(ObMemtableMScanIterator))) ||
              NULL == (mscan_iter_ptr = new (mscan_iter_buffer) ObMemtableMScanIterator())) {
     TRANS_LOG(WARN,
         "construct ObMemtableMScanIterator fail",
diff --git a/src/storage/memtable/ob_memtable_iterator.h b/src/storage/memtable/ob_memtable_iterator.h
index ef3a552..0601d87 100644
--- a/src/storage/memtable/ob_memtable_iterator.h
+++ b/src/storage/memtable/ob_memtable_iterator.h
@@ -100,6 +100,9 @@ public:
   {
     reset();
   }
+  virtual void reuse_try() override{
+    reset();
+  }
 
 private:
   // means GETITER
@@ -135,6 +138,9 @@ public:
   {
     reset();
   }
+  virtual void reuse_try() override{
+    reset();
+  }
   virtual int get_gap_end(int64_t& range_idx, const common::ObStoreRowkey*& gap_key, int64_t& gap_size) override
   {
     int ret = common::OB_SUCCESS;
@@ -216,6 +222,9 @@ public:
   {
     reset();
   }
+  virtual void reuse_try() override{
+    reset();
+  }
 
 public:
   static const int64_t ROW_ALLOCATOR_PAGE_SIZE = common::OB_MALLOC_NORMAL_BLOCK_SIZE;
@@ -254,6 +263,9 @@ public:
   {
     reset();
   }
+  virtual void reuse_try() override{
+    reset();
+  }
   virtual int skip_range(int64_t range_idx, const common::ObStoreRowkey* gap_key, const bool include_gap_key) override;
   virtual int get_gap_end(int64_t& range_idx, const common::ObStoreRowkey*& gap_key, int64_t& gap_size) override
   {
@@ -299,6 +311,9 @@ public:
   {
     reset();
   }
+  virtual void reuse_try() override{
+    reset();
+  }
   enum ScanState { SCAN_BEGIN, SCAN_UNCOMMITTED_ROW, SCAN_COMPACT_ROW, SCAN_MULTI_VERSION_ROW, SCAN_END };
   // the iteration process is divided into 2 phases:
   // iterating Complement SSTable / iterating Mini SSTable
diff --git a/src/storage/ob_handle_mgr.h b/src/storage/ob_handle_mgr.h
index 45fdfcb..4cf6617 100644
--- a/src/storage/ob_handle_mgr.h
+++ b/src/storage/ob_handle_mgr.h
@@ -21,7 +21,7 @@ namespace storage {
 template <typename Handle, typename Key, int64_t N>
 class ObHandleMgr {
 public:
-  ObHandleMgr() : is_inited_(false), is_multi_(false), is_ordered_(false), last_handle_(NULL), handle_cache_(NULL)
+  ObHandleMgr() : is_inited_(false), is_multi_(false), is_ordered_(false), last_handle_(NULL), handle_cache_(NULL), stmt_allocator_(NULL)
   {}
   virtual ~ObHandleMgr()
   {
@@ -33,21 +33,50 @@ public:
     is_ordered_ = false;
     if (NULL != last_handle_) {
       last_handle_->~Handle();
+      if (stmt_allocator_ != NULL){
+        stmt_allocator_->free(last_handle_); 
+      }
       last_handle_ = NULL;
     }
     if (NULL != handle_cache_) {
       handle_cache_->~ObHandleCache();
+      if (stmt_allocator_ != NULL){
+        stmt_allocator_->free(handle_cache_); 
+      } 
       handle_cache_ = NULL;
     }
     is_inited_ = false;
+    stmt_allocator_ = NULL;
   }
   int init(const bool is_multi, const bool is_ordered, common::ObArenaAllocator& allocator)
   {
     int ret = common::OB_SUCCESS;
     void* buf = NULL;
     if (OB_UNLIKELY(is_inited_)) {
-      ret = common::OB_INIT_TWICE;
-      STORAGE_LOG(WARN, "handle mgr is inited twice", K(ret));
+      if (is_multi){
+        if (is_ordered) {
+          if (last_handle_ == NULL){
+            if (OB_ISNULL(buf = allocator.alloc(sizeof(Handle)))) {
+              ret = common::OB_ALLOCATE_MEMORY_FAILED;
+              STORAGE_LOG(WARN, "failed to allocate last handle");
+            } else {
+              last_handle_ = new (buf) Handle();
+            }
+          }
+        } else{
+          if (handle_cache_ == NULL){
+            if (OB_ISNULL(buf = allocator.alloc(sizeof(HandleCache)))) {
+              ret = common::OB_ALLOCATE_MEMORY_FAILED;
+              STORAGE_LOG(WARN, "failed to allocate last handle");
+            } else {
+              handle_cache_ = new (buf) HandleCache();
+            }
+          }
+          
+        }
+      }     
+//      ret = common::OB_INIT_TWICE;
+//      STORAGE_LOG(WARN, "handle mgr is inited twice", K(ret));
     } else if (is_multi) {
       if (is_ordered) {
         if (OB_ISNULL(buf = allocator.alloc(sizeof(Handle)))) {
@@ -69,6 +98,7 @@ public:
       is_multi_ = is_multi;
       is_ordered_ = is_ordered;
       is_inited_ = true;
+      stmt_allocator_ = &allocator;
     }
     return ret;
   }
@@ -85,6 +115,7 @@ protected:
   bool is_ordered_;
   Handle* last_handle_;
   HandleCache* handle_cache_;
+  common::ObArenaAllocator* stmt_allocator_;
 };
 
 }  // namespace storage
diff --git a/src/storage/ob_i_store.h b/src/storage/ob_i_store.h
index e13283f..e2e5784 100644
--- a/src/storage/ob_i_store.h
+++ b/src/storage/ob_i_store.h
@@ -833,6 +833,10 @@ public:
   }
   virtual void reuse()
   {}
+  virtual void reset()
+  {}
+  virtual void reuse_try() 
+  {}
   virtual bool is_base_sstable_iter() const
   {
     return false;
diff --git a/src/storage/ob_multiple_get_merge.cpp b/src/storage/ob_multiple_get_merge.cpp
index ebfd26a..c568654 100644
--- a/src/storage/ob_multiple_get_merge.cpp
+++ b/src/storage/ob_multiple_get_merge.cpp
@@ -82,7 +82,7 @@ void ObMultipleGetMerge::reset_with_fuse_row_cache()
     handles_ = nullptr;
   }
   prefetch_cnt_ = 0;
-  reuse_iter_array();
+  reset_iter_array();
 }
 
 void ObMultipleGetMerge::reset()
diff --git a/src/storage/ob_multiple_merge.cpp b/src/storage/ob_multiple_merge.cpp
index 8a01f9d..6c59527 100644
--- a/src/storage/ob_multiple_merge.cpp
+++ b/src/storage/ob_multiple_merge.cpp
@@ -502,6 +502,10 @@ void ObMultipleMerge::reset()
   for (int64_t i = 0; i < iters_.count(); ++i) {
     if (NULL != (iter = iters_.at(i))) {
       iter->~ObStoreRowIterator();
+      if (OB_NOT_NULL(access_ctx_->stmt_allocator_)) {
+        access_ctx_->stmt_allocator_->free(iter);
+      }
+      iter = NULL;
     }
   }
   padding_allocator_.reset();
@@ -539,16 +543,29 @@ void ObMultipleMerge::reuse()
   read_memtable_only_ = false;
 }
 
-void ObMultipleMerge::reuse_iter_array()
+void ObMultipleMerge::reset_iter_array()
 {
   ObStoreRowIterator* iter = NULL;
   for (int64_t i = 0; i < iters_.count(); ++i) {
     if (NULL != (iter = iters_.at(i))) {
       iter->~ObStoreRowIterator();
+      if (OB_NOT_NULL(access_ctx_->stmt_allocator_)) {
+        access_ctx_->stmt_allocator_->free(iter);
+      }
+      iter = NULL;
     }
   }
   iters_.reuse();
 }
+void ObMultipleMerge::reuse_iter_array()
+{
+  ObStoreRowIterator *iter = NULL;
+  for (int64_t i = 0; i < iters_.count(); ++i) {
+    if (NULL != (iter = iters_.at(i))) {
+      iter->reuse_try(); 
+    }
+  }
+}
 
 int ObMultipleMerge::open()
 {
@@ -944,7 +961,7 @@ int ObMultipleMerge::refresh_table_on_demand()
   } else if (need_refresh) {
     if (OB_FAIL(save_curr_rowkey())) {
       STORAGE_LOG(WARN, "fail to save current rowkey", K(ret));
-    } else if (FALSE_IT(reuse_iter_array())) {
+    } else if (FALSE_IT(reset_iter_array())) {
     } else if (OB_FAIL(prepare_read_tables())) {
       STORAGE_LOG(WARN, "fail to prepare read tables", K(ret));
     } else if (OB_FAIL(reset_tables())) {
diff --git a/src/storage/ob_multiple_merge.h b/src/storage/ob_multiple_merge.h
index 12f8cdc..edc20f9 100644
--- a/src/storage/ob_multiple_merge.h
+++ b/src/storage/ob_multiple_merge.h
@@ -80,6 +80,7 @@ protected:
   const ObTableIterParam* get_actual_iter_param(const ObITable* table) const;
   int project_row(const ObStoreRow& unprojected_row, const common::ObIArray<int32_t>* projector,
       const int64_t range_idx_delta, ObStoreRow& projected_row);
+  void reset_iter_array();
   void reuse_iter_array();
   virtual int skip_to_range(const int64_t range_idx);
 
diff --git a/src/storage/ob_sstable.cpp b/src/storage/ob_sstable.cpp
index 13a3f0f..257c02d 100644
--- a/src/storage/ob_sstable.cpp
+++ b/src/storage/ob_sstable.cpp
@@ -1105,14 +1105,14 @@ int ObSSTable::get(const storage::ObTableIterParam& param, storage::ObTableAcces
     ObISSTableRowIterator* row_getter = NULL;
     if (is_multi_version_minor_sstable() && (context.is_multi_version_read(get_upper_trans_version()) ||
                                                 contain_uncommitted_row() || !meta_.has_compact_row_)) {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableMultiVersionRowGetter)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableMultiVersionRowGetter)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
         row_getter = new (buf) ObSSTableMultiVersionRowGetter();
       }
     } else {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowGetter)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowGetter)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
@@ -1163,14 +1163,14 @@ int ObSSTable::multi_get(const ObTableIterParam& param, ObTableAccessContext& co
       ObISSTableRowIterator* row_getter = NULL;
       if (is_multi_version_minor_sstable() && (context.is_multi_version_read(get_upper_trans_version()) ||
                                                   contain_uncommitted_row() || !meta_.has_compact_row_)) {
-        if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableMultiVersionRowMultiGetter)))) {
+        if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableMultiVersionRowMultiGetter)))) {
           ret = OB_ALLOCATE_MEMORY_FAILED;
           STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
         } else {
           row_getter = new (buf) ObSSTableMultiVersionRowMultiGetter();
         }
       } else {
-        if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowMultiGetter)))) {
+        if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowMultiGetter)))) {
           ret = OB_ALLOCATE_MEMORY_FAILED;
           STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
         } else {
@@ -1269,21 +1269,21 @@ int ObSSTable::scan(const ObTableIterParam& param, ObTableAccessContext& context
     void* buf = NULL;
     ObISSTableRowIterator* row_scanner = NULL;
     if (context.query_flag_.is_whole_macro_scan()) {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowWholeScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowWholeScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
         row_scanner = new (buf) ObSSTableRowWholeScanner();
       }
     } else if (is_multi_version_minor_sstable()) {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableMultiVersionRowScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableMultiVersionRowScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
         row_scanner = new (buf) ObSSTableMultiVersionRowScanner();
       }
     } else {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
@@ -1435,14 +1435,14 @@ int ObSSTable::multi_scan(const ObTableIterParam& param, ObTableAccessContext& c
     void* buf = NULL;
     ObISSTableRowIterator* row_scanner = NULL;
     if (is_multi_version_minor_sstable()) {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableMultiVersionRowMultiScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableMultiVersionRowMultiScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
         row_scanner = new (buf) ObSSTableMultiVersionRowMultiScanner();
       }
     } else {
-      if (NULL == (buf = context.allocator_->alloc(sizeof(ObSSTableRowMultiScanner)))) {
+      if (NULL == (buf = context.stmt_allocator_->alloc(sizeof(ObSSTableRowMultiScanner)))) {
         ret = OB_ALLOCATE_MEMORY_FAILED;
         STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
       } else {
@@ -2930,7 +2930,7 @@ int ObSSTable::build_exist_iterator(const ObTableIterParam& iter_param, ObTableA
   } else {
     void* buf = NULL;
     ObSSTableRowExister* exister = NULL;
-    if (NULL == (buf = access_context.allocator_->alloc(sizeof(ObSSTableRowExister)))) {
+    if (NULL == (buf = access_context.stmt_allocator_->alloc(sizeof(ObSSTableRowExister)))) {
       ret = OB_ALLOCATE_MEMORY_FAILED;
       STORAGE_LOG(WARN, "Fail to allocate memory, ", K(ret));
     } else {
diff --git a/src/storage/ob_sstable_row_iterator.cpp b/src/storage/ob_sstable_row_iterator.cpp
index 6c261bf..6170781 100644
--- a/src/storage/ob_sstable_row_iterator.cpp
+++ b/src/storage/ob_sstable_row_iterator.cpp
@@ -405,6 +405,7 @@ ObSSTableRowIterator::ObSSTableRowIterator()
       storage_file_(nullptr),
       is_opened_(false),
       is_base_(false),
+      is_rescan_(false),
       block_cache_(NULL),
       table_type_(ObITable::MAJOR_SSTABLE),
       sstable_snapshot_version_(0),
@@ -469,9 +470,9 @@ int ObSSTableRowIterator::inner_open(
     STORAGE_LOG(WARN, "Unexpected error, ", K(ret), K_(read_handle_cnt), K_(micro_handle_cnt));
   } else if (OB_FAIL(init_handle_mgr(iter_param, access_ctx, query_range))) {
     STORAGE_LOG(WARN, "fail to init handle mgr", K(ret), K(iter_param), K(access_ctx));
-  } else if (OB_FAIL(read_handles_.reserve(*access_ctx.allocator_, read_handle_cnt_))) {
+  } else if (OB_FAIL(read_handles_.reserve(*access_ctx.stmt_allocator_, read_handle_cnt_))) {
     STORAGE_LOG(WARN, "failed to reserve read handles", K(ret), K_(read_handle_cnt));
-  } else if (OB_FAIL(micro_handles_.reserve(*access_ctx.allocator_, micro_handle_cnt_))) {
+  } else if (OB_FAIL(micro_handles_.reserve(*access_ctx.stmt_allocator_, micro_handle_cnt_))) {
     STORAGE_LOG(WARN, "failed to reserve micro handles", K(ret), K_(micro_handle_cnt));
   } else if (OB_FAIL(sstable_micro_infos_.reserve(*access_ctx.allocator_, micro_handle_cnt_))) {
     STORAGE_LOG(WARN, "failed to reserve sstable micro infos", K(ret), K_(micro_handle_cnt));
@@ -608,7 +609,7 @@ void ObSSTableRowIterator::reset()
     micro_lock_checker_->~ObMicroBlockRowLockChecker();
     micro_lock_checker_ = NULL;
   }
-
+  
   macro_block_iter_.reset();
   iter_param_ = NULL;
   access_ctx_ = NULL;
@@ -636,6 +637,7 @@ void ObSSTableRowIterator::reset()
   table_store_stat_.reset();
   skip_ctx_.reset();
   storage_file_ = nullptr;
+  is_rescan_ = false;
   prefetch_handle_depth_ = DEFAULT_PREFETCH_HANDLE_DEPTH;
   prefetch_micro_depth_ = DEFAULT_PREFETCH_MICRO_DEPTH;
 }
@@ -669,6 +671,59 @@ void ObSSTableRowIterator::reuse()
   table_store_stat_.reuse();
   skip_ctx_.reset();
   storage_file_ = nullptr;
+  is_rescan_ = false;
+  prefetch_handle_depth_ = DEFAULT_PREFETCH_HANDLE_DEPTH;
+  prefetch_micro_depth_ = DEFAULT_PREFETCH_MICRO_DEPTH;
+}
+void ObSSTableRowIterator::reuse_try(){
+  ObISSTableRowIterator::reset();
+//  read_handles_.reset();
+//  micro_handles_.reset();
+  sstable_micro_infos_.reset();
+
+  if (NULL != micro_exister_) {
+    micro_exister_->~ObMicroBlockRowExister();
+    micro_exister_ = NULL;
+  }
+  if (NULL != micro_getter_) {
+    micro_getter_->~ObMicroBlockRowGetter();
+    micro_getter_ = NULL;
+  }
+  if (NULL != micro_scanner_) {
+    micro_scanner_->~ObIMicroBlockRowScanner();
+    micro_scanner_ = NULL;
+  }
+  if (NULL != micro_lock_checker_) {
+    micro_lock_checker_->~ObMicroBlockRowLockChecker();
+    micro_lock_checker_ = NULL;
+  }
+  
+  is_rescan_ = true;
+  macro_block_iter_.reset();
+  iter_param_ = NULL;
+  access_ctx_ = NULL;
+  sstable_ = NULL;
+  query_range_ = NULL;
+  scan_step_ = 0;
+  is_opened_ = false;
+  is_base_ = false;
+  block_cache_ = NULL;
+  table_type_ = ObITable::MAJOR_SSTABLE;
+  sstable_snapshot_version_ = 0;
+  prefetch_handle_end_ = false;
+  prefetch_block_end_ = false;
+  cur_prefetch_handle_pos_ = 0;
+  cur_fetch_handle_pos_ = 0;
+  cur_read_handle_pos_ = 0;
+  cur_prefetch_micro_pos_ = 0;
+  cur_read_micro_pos_ = 0;
+  cur_micro_idx_ = -1;
+  cur_range_idx_ = -1;
+  io_micro_infos_.reset();
+  micro_info_iter_.reset();
+  table_store_stat_.reset();
+  skip_ctx_.reset();
+  storage_file_ = nullptr;
   prefetch_handle_depth_ = DEFAULT_PREFETCH_HANDLE_DEPTH;
   prefetch_micro_depth_ = DEFAULT_PREFETCH_MICRO_DEPTH;
 }
@@ -1701,9 +1756,12 @@ int ObSSTableRowIterator::init_handle_mgr(
             range_count >= USE_HANDLE_CACHE_RANGE_COUNT_THRESHOLD);
   }
   if (OB_SUCC(ret)) {
-    if (OB_FAIL(block_handle_mgr_.init(is_multi, true, *access_ctx.allocator_))) {
+    if(is_rescan_ == true){
+      is_multi = true;
+    }
+    if (OB_FAIL(block_handle_mgr_.init(is_multi, true, *access_ctx.stmt_allocator_))) {
       STORAGE_LOG(WARN, "failed to init block handle mgr", K(ret), K(is_multi), K(is_ordered));
-    } else if (OB_FAIL(block_index_handle_mgr_.init(is_multi, is_ordered, *access_ctx.allocator_))) {
+    } else if (OB_FAIL(block_index_handle_mgr_.init(is_multi, is_ordered, *access_ctx.stmt_allocator_))) {
       STORAGE_LOG(WARN, "failed to init block index handle mgr", K(ret), K(is_multi), K(is_ordered));
     }
   }
diff --git a/src/storage/ob_sstable_row_iterator.h b/src/storage/ob_sstable_row_iterator.h
index dadb33b..bcb0f16 100644
--- a/src/storage/ob_sstable_row_iterator.h
+++ b/src/storage/ob_sstable_row_iterator.h
@@ -176,6 +176,9 @@ public:
   virtual int get_next_row(const ObStoreRow*& store_row);
   virtual void reset();
   virtual void reuse();
+  virtual void reuse_try(){
+    reset();
+  }
   OB_INLINE bool has_lob_column() const
   {
     return OB_NOT_NULL(lob_reader_);
@@ -236,7 +239,7 @@ public:
 template <typename T>
 class ObSimpleArray {
 public:
-  ObSimpleArray() : array_(nullptr), capacity_(0)
+  ObSimpleArray() : array_(nullptr), capacity_(0), stmt_allocator_(NULL) 
   {}
   ~ObSimpleArray()
   {
@@ -247,6 +250,9 @@ public:
     for (int64_t i = 0; i < capacity_; ++i) {
       array_[i].~T();
     }
+    if( NULL != stmt_allocator_ ){
+      stmt_allocator_->free(array_);
+    }
     array_ = nullptr;
     capacity_ = 0;
   }
@@ -262,6 +268,7 @@ public:
       } else {
         array_ = new (buf) T[count];
         capacity_ = count;
+        stmt_allocator_ = &allocator;
       }
     }
     return ret;
@@ -274,6 +281,7 @@ public:
 private:
   T* array_;
   int64_t capacity_;
+  common::ObArenaAllocator* stmt_allocator_;  
 };
 
 struct ObFastSkipChecker {
@@ -321,6 +329,7 @@ public:
   virtual ~ObSSTableRowIterator();
   virtual void reset() override;
   virtual void reuse() override;
+  virtual void reuse_try() override;
   virtual int get_skip_range_ctx(
       ObSSTableReadHandle& read_handle, const int64_t cur_micro_idx, ObSSTableSkipRangeCtx*& skip_ctx);
   int get_row_iter_flag_impl(uint8_t& flag);
@@ -398,6 +407,7 @@ protected:
 private:
   bool is_opened_;
   bool is_base_;
+  bool is_rescan_;
   blocksstable::ObIMicroBlockCache* block_cache_;
   ObITable::TableType table_type_;
   int64_t sstable_snapshot_version_;

patch4的主要思想就是缓存了微块(rescan场景下缓存了微块，然后在ObSSTableRowIterator的reuse()中标记这是rescan场景)；然后将read_hanldes_和micro_handles_重用了。

ObSSTableRowIterator等memtable或者sstable的iterator的reuse_try()函数是如何被调用的呢？因为每次rescan都会调用ObMultipleMerge的reuse()函数（其实它的子类的reuse()函数基本上也要调用这个函数），然后这个reuse()函数里面调用了ObMultipleMerge::reuse_iter_array()函数，然后我们修改了ObMultiMerge::reuse_iter_array()函数，使得它不是简单地reset所有iterator，而是reuse_try它们。

在ObSSTableRowIterator::reuse_try()函数中，我们取消reset read_handles_和reset micro_handles_ ，这样就可以再次用这两种handle了，并且在ObSimpleArray类中reset()函数中加入对所有元素的释放操作，这个操作比较重要。因为我们是通过stmt_allocator来给read_handles_和mico_handles_来申请空间，应该要找个合适的时机来释放这个空间。

我想我已经搞清楚了为什么使用HandleCache，也即在rescan场景下缓存微块，并且将read_handles_和micro_handles重用，可以提升性能。HandleCache是将这次scan右表需要的所有微块都缓存下来，micro_handles是将这些缓存的微块按照扫描顺序排好，micro_handles最里面有buf指针，指向微块数据，HandleCache也是的，这两个buf指针是相同的，也即HandleCache和micro_handles是对同一个微块数据的引用。当我们重用HandleCache和micro_handles时，下一次rescan就省了两部分工作：

不用重新从磁盘读取微块了，因为可能在上一个rescan微块已经缓存在内存中了。
不用重新给micro_handles和read_handles分配空间了。以前每次rescan都要给micro_handles和read_handles分配空间，但是这两个的分配器换成了stmt_allocator后，这两个的空间就不会在新的rescan处被回收，我们就可以复用这个空间。但是有一点需要注意

int reserve(common::ObArenaAllocator& allocator, const int64_t count)
{
  int ret = common::OB_SUCCESS;
  if (capacity_ < count) {
    void* buf = nullptr;
    reset();
    if (OB_ISNULL(buf = allocator.alloc(sizeof(T) * count))) {
      ret = common::OB_ALLOCATE_MEMORY_FAILED;
      STORAGE_LOG(WARN, "failed to allocate array", K(ret));
    } else {
      array_ = new (buf) T[count];
      capacity_ = count;
    }
  }
  return ret;
}

这是micro_handles和read_handles的空间分配函数，当我们不reset它们时，capacity_属性就不用清0，因此就不会走if路线了。

源码阅读

架构

OB的block cache是微块级的，不建议缓存宏块。但是不懂是怎么通过range或者rowkey定位到宏块，又从宏块中找到对应的微块，反正最后是能够在一个rescan内，把所有微块给拿到内存中的。

OB的ObMicroBlockIndexHandle存储的是ObMicroBlockIndexCache，它是宏块中对微块的索引，通过它可以方便地在宏块中找到微块。

在ObHandleMgr::init()函数中，如果是is multi并且is ordered，说明这个不断传下来的rowkey或者range都是有序的，例如rowkey是1, 2, 4, 8, 20这样的，这样的话，我们就只需要对一个ObMicroBlockHandle进行缓存就好了，不需要HandleCache来缓存那么多的ObMicroBlockHandle。

从ObExecuteResult::get_next_row()函数读NLJ的流程

1	int ObExecuteResult::get_next_row(ObExecContext& ctx, const common::ObNewRow*& row)

ret = get_next_row();
// ...
const ObOpSpec& spec = static_engine_root_->get_spec();
// ...
if (OB_ISNULL(
                row_.cells_ = static_cast<ObObj*>(ctx.get_allocator().alloc(sizeof(ObObj) * spec.output_.count())))) {
// ...

我们可以看到get_next_row()函数应该是实际得到一行数据，结果保存在static_engine_root_.get_spec().output_中。

其实这里的static_engine_root_是ObOperator*类型的，是一个算子。

1	int ObExecuteResult::get_next_row()

while (OB_SUCC(ret) && !got_row) {
  if (OB_FAIL(static_engine_root_->get_next_row())) {
    if (OB_ITER_END == ret) {
      plan_ctx->inc_bind_array_idx();
      if (OB_FAIL(static_engine_root_->switch_iterator())) {
      }
    }
  }else {
    got_row = true;
  }
}

调用ObOperator::get_next_row()得到下一行数据，如果是OB_ITER_END，那就switch_iterator()，但是不明白这个函数是做什么的？

1	int ObOperator::get_next_row()

if (OB_FAIL(startup_filter(filtered)))
// ...
while (OB_SUCC(ret)) {
  if (OB_FAIL(inner_get_next_row())) {
  } else {
    if (!spec_.filters_.empty()) {
      bool filtered = false;
      if (OB_FAIL(filter_row(filtered))) {
        LOG_WARN("filter row failed", K(ret), "type", spec_.type_, "op", op_name());
      } else {
        if (filtered) {
          continue;
        }
      }
    }
  }
  break;
}
// ...
if (!got_first_row_) {
  op_monitor_info_.first_row_time_ = oceanbase::common::ObClockGenerator::getClock();
  ;
  got_first_row_ = true;
}

不懂startup_filter()函数是做什么的。当filtered为true时，表示过滤不成功。当还没有得到第一行记录的时候，记录一下当前的时间到op_monitor_info_中。

1	int ObNestedLoopJoinOp::inner_get_next_row()

if (OB_UNLIKELY(LEFT_SEMI_JOIN == MY_SPEC.join_type_ || LEFT_ANTI_JOIN == MY_SPEC.join_type_)) {
    if (OB_FAIL(join_row_with_semi_join())) {
    }
} else {
  output_row_produced_ = false;
  while (OB_SUCC(ret) && !output_row_produced_) {
    state_operation = this->ObNestedLoopJoinOp::state_operation_func_[state_];
    if (OB_ITER_END == (ret = (this->*state_operation)())) {
      func = FT_ITER_END;
      ret = OB_SUCCESS;
    } else if (OB_FAIL(ret)) {
      LOG_WARN("failed state operation", K(ret), K(state_));
    } else {
      func = FT_ITER_GOING;
    }
    if (OB_SUCC(ret)) {
      state_function = this->ObNestedLoopJoinOp::state_function_func_[state_][func];
      if (OB_FAIL((this->*state_function)()) && OB_ITER_END != ret) {
        LOG_WARN("failed state function", K(ret), K(state_), K(func));
      }
    }
  }  // while end
}
// ...

在构造ObNestLoopJoinOp的时候

ObNestedLoopJoinOp::ObNestedLoopJoinOp(ObExecContext& exec_ctx, const ObOpSpec& spec, ObOpInput* input)
    : ObBasicNestedLoopJoinOp(exec_ctx, spec, input),
      state_(JS_READ_LEFT),
      mem_context_(nullptr),
      is_left_end_(false),
      last_store_row_(),
      save_last_row_(false)
{
  state_operation_func_[JS_JOIN_END] = &ObNestedLoopJoinOp::join_end_operate;
  state_function_func_[JS_JOIN_END][FT_ITER_GOING] = NULL;
  state_function_func_[JS_JOIN_END][FT_ITER_END] = &ObNestedLoopJoinOp::join_end_func_end;

  state_operation_func_[JS_READ_LEFT] = &ObNestedLoopJoinOp::read_left_operate;
  state_function_func_[JS_READ_LEFT][FT_ITER_GOING] = &ObNestedLoopJoinOp::read_left_func_going;
  state_function_func_[JS_READ_LEFT][FT_ITER_END] = &ObNestedLoopJoinOp::read_left_func_end;

  state_operation_func_[JS_READ_RIGHT] = &ObNestedLoopJoinOp::read_right_operate;
  state_function_func_[JS_READ_RIGHT][FT_ITER_GOING] = &ObNestedLoopJoinOp::read_right_func_going;
  state_function_func_[JS_READ_RIGHT][FT_ITER_END] = &ObNestedLoopJoinOp::read_right_func_end;
}

join_row_with_semi_join()不知道干什么的？

在构造函数，state_初始化为JS_READ_LEFT。我们看到首先有一个while循环，所以肯定是在while循环中先read_left_operate()，然后再执行read_left_func_going()，在read_left_func_going()的最后，设置state_为JS_READ_RIGHT，然后就继续循环，读取右表中的一行read_right_operate()，然后就继续循环执行read_right_func_going()，这里会调用ObJoinOp::calc_other_conds()函数进行判断是否满足filter条件，如果满足，就设置output_row_produced_为true，这样就可以结束循环。所以这个循环的目的就是为了得到一行满足filter的数据，当然在上层，我们也进行了一个filter判断，我觉得这是多余的了。

这里state_是个很重要的属性，控制着NLJ的执行方向，是读左表，还是读右表，还是read_left_func_going()，具体这些执行怎么做的，后面具体说。

内存管理

ObArenaAllocator

我们先看page_arena.h这个文件中是怎么定义和实现ObArenaAllocator这个类的。

alloc

// class ObArenaAllocator final : public ObIAllocator
public:
  virtual void* alloc(const int64_t sz) override
  {
    return arena_.alloc_aligned(sz);
  }

// template <typename CharT = char, class PageAllocatorT = DefaultPageAllocator>
// class PageArena
/** allocate sz bytes */
  CharT* alloc_aligned(const int64_t sz, const int64_t alignment = 16)
  {
    ensure_cur_page();

    // common case
    CharT* ret = NULL;
    if (NULL != cur_page_ && sz > 0) {
      int64_t align_offset = get_align_offset(cur_page_->alloc_end_, alignment);
      int64_t adjusted_sz = sz + align_offset;

      if (adjusted_sz <= cur_page_->remain()) {
        ret = cur_page_->alloc(adjusted_sz) + align_offset;
        if (NULL != ret) {
          used_ += align_offset;
        }
      } else if (is_normal_overflow(sz)) {
        Page* new_page = extend_page(page_size_);
        if (NULL != new_page) {
          cur_page_ = new_page;
        }
        if (NULL != cur_page_) {
          ret = cur_page_->alloc(sz);
        }
      } else if (lookup_next_page(sz)) {
        if (NULL != cur_page_) {
          ret = cur_page_->alloc(sz);
        }
      } else {
        ret = alloc_big(sz);
      }

      if (NULL != ret) {
        used_ += sz;
      }
    }
    return ret;
  }

ObArenaAllocator每次是在一个page内进行内存分配的，如果在当前的page内可以分配，就分配，如果不能，就用ModulePageAllocator分配一个新的页，在这个页内进行分配。所有用ModulePageAllocator分配的页是用链表连接起来的。

// template <typename CharT = char, class PageAllocatorT = DefaultPageAllocator>
// class PageArena
  Page* extend_page(const int64_t sz)
  {
    Page* page = cur_page_;
    if (NULL != page) {
      page = page->next_page_;
      if (NULL != page) {
        page->reuse();
      } else {
        page = alloc_new_page(sz);
      }
    }
  }

使用PageArena::extend_page()来扩展一个页。

// template <typename CharT = char, class PageAllocatorT = DefaultPageAllocator>
// class PageArena
  Page* alloc_new_page(const int64_t sz)
  {
    void* ptr = page_allocator_.alloc(sz);
      page = new (ptr) Page((char*)ptr + sz);
      total_ += sz;
      ++pages_;
  }

PageArena::alloc_new_page()调用了ModulePageAllocator::alloc()函数来分配一个内存页。这个alloc最终会调用到

1	void* ObMallocAllocator::alloc(const int64_t size, const oceanbase::lib::ObMemAttr& attr)

} else if (OB_UNLIKELY(inner_attr.tenant_id_ >= PRESERVED_TENANT_COUNT)) {
  const int64_t slot = inner_attr.tenant_id_ % PRESERVED_TENANT_COUNT;
  obsys::CRLockGuard guard(locks_[slot]);
  allocator = get_tenant_ctx_allocator(inner_attr.tenant_id_, inner_attr.ctx_id_);
  if (!OB_ISNULL(allocator)) {
    ptr = allocator->alloc(size, inner_attr);
  }
} else {
  allocator = allocators_[inner_attr.tenant_id_][inner_attr.ctx_id_];
  if (!OB_ISNULL(allocator)) {
    ptr = allocator->alloc(size, inner_attr);
  }
}

这里可以发现，我们要分配内存，就要使用allocator池中的一个allocator来分配，这里超过了我的知识范围，但是使用哪个allocator可能是和租户什么的有关。我看过一篇文章，解释过OB的租户的内存：OB 内存分配概述 (qq.com)

读左表的流程

ObNestLoopJoinOp::read_left_operate() 函数调用 ObJoinOp::get_next_left_row()函数。

int ObJoinOp::get_next_left_row()
{
  int ret = common::OB_SUCCESS;
  left_row_joined_ = false;
  if (OB_FAIL(left_->get_next_row()) && OB_ITER_END != ret) {
    LOG_WARN("fail to get next left row", K(ret));
  }
  return ret;
}

设置左表读出的row还没有join，然后调用left_的get_next_row()，它是ObTableScanOp动态类型。

int ObTableScanOp::inner_get_next_row() {
  else if (OB_FAIL(get_next_row_with_mode())) {
  } else {
    output_row_count_++;
    NG_TRACE_TIMES_WITH_TRACE_ID(1, cur_trace_id_, get_row);
    if (MY_SPEC.is_vt_mapping_ &&
        OB_FAIL(vt_result_converter_->convert_output_row(eval_ctx_, MY_SPEC.mapping_exprs_, MY_SPEC.storage_output_))) {
      LOG_WARN("failed to convert output row", K(ret));
    }
  }
}

get_next_row_with_mode()成功后，就让output_row_count_增加。

int ObTableScanIterator::get_next_row(common::ObNewRow*& row) {
  if (OB_FAIL(row_iter_->get_next_row(row))) {
    if (OB_ITER_END != ret) {
      STORAGE_LOG(WARN, "fail to get next row", K(ret));
    } else {
      ret = OB_SUCCESS;
      while (OB_SUCC(ret)) {
        if (OB_FAIL(iter_.get_next_iter(row_iter_))) {
          if (OB_ITER_END != ret) {
            STORAGE_LOG(WARN, "fail to get next iter", K(ret));
          } else {
            STORAGE_LOG(DEBUG, "table scan iterator reaches end");
          }
        } else if (OB_FAIL(row_iter_->get_next_row(row))) {
          if (OB_ITER_END != ret) {
            STORAGE_LOG(WARN, "fail to get next row", K(ret));
          } else {
            STORAGE_LOG(DEBUG, "table scan row iterator reaches end");
            ret = OB_SUCCESS;
          }
        } else {
          break;
        }
      }
    }
    }
}

这里有个iter_和row_iter_属性，iter_是ObTableScanIterIterator对象，row_iter_是ObTableScanRangeArrayRowIterator对象。iter_可以产生多个row_iter_，当一个row_iter_遍历到结束后，就让iter_下一个row_iter_来继续遍历。结合多range和多rowkey的前提，可能是在ObTableScanIterator中多个ObTableScanRangeArrayRowIterator，它们由ObTableScanIterIterator来管理，并且和多range一一对应，比方说，根据主键来查表，where中主键的范围是key<7 或者 key > 100，那么这两个range就会对应两个ObTableScanRangeArrayRowIterator。（由于OB中的表是索引组织表，因此只要知道要查找的主键的范围，那么就可以根据索引轻松找到想要的元组记录在哪，这也是为什么OB有range的概念的原因，想一下，要是记录随便放在表文件中，有range有什么用呢，还是要一个页一个页地扫描）。

1
2
3

int ObTableScanStoreRowIterator::get_next_row(ObStoreRow*& cur_row) {
  OB_FAIL(main_iter_->get_next_row(cur_row));
}

这个函数就调用了get_next_row()。main_iter_是ObQueryRowIterator类型，动态类型估计是ObMultipleScanMerge。

1	int ObMultipleMerge::get_next_row(ObStoreRow*& row);

这个get_next_row()函数很长，里面大致有这几个过程：

access_ctx_相关的操作
refresh_table_on_demand()，在必要的时候刷新table，可能会执行一些reuse_iter_array, reset_tables等的操作
inner_get_next_row()，得到一个没有投影的行
check_row_in_current_range()，不知道check什么
row_filter_->check() 判断这个行是否被过滤
project_row() 将行进行投影
其他关于行的操作
最后是更新access_ctx_中的一些统计量。

NLJ的rescan流程

sql层的ObTableScanOp.cpp的inner_close()函数可以看到有table_allocator_的reuse()调用，这个时候就是把以前用allocator申请的空间给释放掉。

rescan的实现是在ObTableScanStoreRowIterator.cpp中的rescan()函数中可以看到，其实就是调用ObMultipleMerge::reuse_iter_array()把很多iterator给重用了，然后再调用open_iter()函数重新打开这些iterator，并没有涉及到数据的操作。

在ob_table_scan_op.cpp中的rt_rescan()函数中写了单机rescan的大致流程。

NLJ的右表读取流程

左表肯定是通过scan方式来得到数据，因为是要遍历左表的所有满足条件的行。

由于在我们的case中，右表是通过索引回表的方式得到数据的，因此是这样的流程：ObIndexMerge中先访问索引，得到每个rowkey，然后通过rowkey通过get的方式访问主表，来得到具体的行数据。这个流程实现在ObIndexMerge::get_next_row()函数中。

考虑src/storage/ob_handle_mgr.h/oceanbase::storage::ObHandleMgr::init()函数，它的部分调用链是：ObSSTableRowIterator::inner_open() -> ObSSTableRowIterator::init_handle_mgr() -> ObHandleMgr::init()。

我们可以发现这里面有一个HandleCache对象：

} else if (is_multi) {
  if (is_ordered) {
    if (OB_ISNULL(buf = allocator.alloc(sizeof(Handle)))) {
      ret = common::OB_ALLOCATE_MEMORY_FAILED;
      STORAGE_LOG(WARN, "failed to allocate last handle");
    } else {
      last_handle_ = new (buf) Handle();
    }
  } else {
    if (OB_ISNULL(buf = allocator.alloc(sizeof(HandleCache)))) {
      ret = common::OB_ALLOCATE_MEMORY_FAILED;
      STORAGE_LOG(WARN, "failed to allocate last handle");
    } else {
      handle_cache_ = new (buf) HandleCache();
    }
  }
}

“当is_multi为true时，说明有多个range或者rowkey传下来，这样就可以走这个分支，然后就可以使用这个HandleCache。我们考虑rescan的场景，每次rescan都会有新的range下来，而且这个range和上次rescan的range是连续的，因此就相当于is_multi为true的情况，如果这个时候我们不使用HandleCache，就是不够优化的。”

考虑src/ob_sstable_row_iterator.cpp/prefetch_block()函数。这个函数就是预取micro block的，但是为什么要预取呢，因为OB内部取磁盘数据是异步执行的，我们可以边读A微块边从磁盘取B微块，当A微块读取完成之后，说不定B微块就读好了。

ObHandleMgr和预取没有太大关系，只是我们在预取之后的数据之上，加了层Handle的cache。

我们是先从cache中找我们需要的微块，

for (int64_t i = 0; OB_SUCC(ret) && i < sstable_micro_cnt; ++i) {
  const ObSSTableMicroBlockInfo& sstable_micro = sstable_micro_infos[i];
  ObMicroBlockDataHandle& micro_handle = micro_handles_[sstable_micro.micro_idx_ % micro_handle_cnt_];
  bool need_submit_io = false;
  if (OB_FAIL(block_handle_mgr_.get_micro_block_handle(iter_param_->table_id_,
          sstable_micro.macro_ctx_,
          storage_file_->get_file_id(),
          sstable_micro.micro_info_.offset_,
          sstable_micro.micro_info_.size_,
          sstable_micro.micro_info_.index_,
          micro_handle))) {
    // cache miss

如果没有找到，再从磁盘IO来找。因为每次找新的range或者rowkey会定位到一个微块，可能这个微块和上个读取微块相同，如果这时发现这个相同的微块在HandleCache中找到了，就很好。

1 2	// ob_micro_block_handle_mgr.h class ObMicroBlockHandleMgr : public ObHandleMgr<ObMicroBlockDataHandle, blocksstable::ObMicroBlockCacheKey, 64>

1
2
3

// ob_sstable_row_iterator.h
typedef ObSimpleArray<ObMicroBlockDataHandle> BlockDataHandleArray;
BlockDataHandleArray micro_handles;

1 2	// ob_sstable_row_iterator.h typedef ObSimpleArray<ObSSTableReadHandle> ReadHandleArray;

问题

rescan中的prefetch是怎么回事？

stmt_allocator申请的空间在何时释放的？

迭代器打开时做了什么事情？

ObTableScanOp的ObNewRowIterator result_属性在哪初始化的？

在ObTableScanOp::do_table_scan()函数中调用了

1	if (OB_FAIL(das->table_scan(scan_param_, ab_iters_))) {

这个das最终又调用了

1	ObPartitionService::table_scan(ObVTableScanParam& vparam, common::ObNewRowIterator*& result)

这里的result就是我们要的输出参数；然后又调用了

1	ObPartitionStorage::table_scan(ObTableScanParam& param, const int64_t data_max_schema_version, common::ObNewRowIterator*& result)

在这个函数里面有一条调用

1	if (OB_UNLIKELY(NULL == (iter = rp_alloc(ObTableScanIterator, ObTableScanIterator::LABEL)))) {

在什么层级下是线程安全的？

钉钉群消息

线程检查工具：https://www.jianshu.com/p/1f29ae9fceee

https://github.com/oceanbase/oceanbase/issues/488 这里面的issue，认领的话，在这里回复就好了

@王运来来哥发的机器不能访问外网吗？
export http_proxy=’http://172.16.0.232:8259‘
export https_proxy=$http_proxy

测试主机密码：6vqSJOonTr52LhzFmnUm

首发！OceanBase社区版入门教程开课啦！https://mp.weixin.qq.com/s/04YjSUsNoKtIRsC0OC394A

@nauta ob支持执行请求时打开trace_log，打开方式有两种，一种是通过hint中的trace_log字段，这种方式只对携带hint的当前语句生效；另一种是通过session变量ob_enable_trace_log，这种方式对这个session的后续所有语句生效。打开trace_log后，通过show trace可以拿到上一次的trace_log，从中可以获取trace_id。同时show trace还可以看到这条请求大致的性能统计。使用示例如下， # 语句级 select /+ trace_log=on /c1 from t1 limit 2; show trace; # session级 set ob_enable_trace_log = ‘ON’; select count(*) from t1; show trace; last_trace_id 使用select last_trace_id();可以查看上一条语句执行的trace_id，然后在日志中grep查找相关信息。

引用

https://github.com/oceanbase/oceanbase/wiki/how_to_debug