Which PikiwiDB functionalities are relevant/related to the feature request?
No response
Description
rocksdb 加几行代码就能增加迭代器跳过删除key和老版本的计数。最后用这个计数放到UpdateSpecificKeyStatistics。感觉非常高效精准的优化kCompactKey
使用如下
`printf("本次迭代跳过 %llu 条\n", iter->skip_count());
UpdateSpecificKeyStatistics(key.ToString(), iter->skip_count());`
对了，建议在AddCompactKeyTaskIfNeeded 函数增加compact任务检测，不至于现在compact的是很久以前访问的key
if (bw_->bg_tasks_queue_.size() > 0){ return Status::OK(); }
我在3.0.16测试，需要增加5行代码在这
`vim include/rocksdb/iterator.h +29
29 class Iterator : public Cleanable {
30 public:
31 Iterator() {}
32 virtual ~Iterator() {}
33
34 // An iterator is either positioned at a key/value pair, or
35 // not valid. This method returns true iff the iterator is valid.
36 virtual bool Valid() const = 0;
37
38 virtual uint64_t skip_count() const { return 0; } //增加的第一行
vim db/db_iter.cc +51
51 class DBIter final: public Iterator {
52 public:
53 // The following is grossly complicated. TODO: clean it up
54 // Which direction is the iterator currently moving?
55 // (1) When moving forward, the internal iterator is positioned at
56 // the exact entry that yields this->key(), this->value()
57 // (2) When moving backwards, the internal iterator is positioned
58 // just before all entries whose user key == this->key().
59 enum Direction {
60 kForward,
61 kReverse
62 };
63
64 uint64_t skip_count_ = 0; //增加的第二行
65 uint64_t skip_count() const override; //增加的第三行
327 uint64_t DBIter::skip_count() const { return skip_count_; } //增加的第四行
1326 uint64_t ArenaWrappedDBIter::skip_count() const { return db_iter_->skip_count(); }。//增加的第五行
vim db/db_iter.h +45
45 class ArenaWrappedDBIter : public Iterator {
46 public:
47 virtual ~ArenaWrappedDBIter();
48
49 // db/db_iter.h (class ArenaWrappedDBIter 内)
50 uint64_t skip_count() const override; //增加的第六行
`
上面是给迭代器增加 skip_count_
收集跳过，还需要在DBIter::FindNextUserEntryInternal在跳过的地方（就是所有PERF_COUNTER_ADD后面）增加++skip_count_;
比如以下是我增加的一个地方
vim db/db_iter.cc
464 PERF_COUNTER_ADD(internal_delete_skipped_count, 1);
465 ++skip_count_;
Proposed solution
改善AddCompactKeyTaskIfNeeded
Alternatives considered
优化UpdateSpecificKeyStatistics
Which PikiwiDB functionalities are relevant/related to the feature request?
No response
Description
rocksdb 加几行代码就能增加迭代器跳过删除key和老版本的计数。最后用这个计数放到UpdateSpecificKeyStatistics。感觉非常高效精准的优化kCompactKey
使用如下
`printf("本次迭代跳过 %llu 条\n", iter->skip_count());
UpdateSpecificKeyStatistics(key.ToString(), iter->skip_count());`
对了，建议在AddCompactKeyTaskIfNeeded 函数增加compact任务检测，不至于现在compact的是很久以前访问的key
if (bw_->bg_tasks_queue_.size() > 0){ return Status::OK(); }我在3.0.16测试，需要增加5行代码在这
`vim include/rocksdb/iterator.h +29
29 class Iterator : public Cleanable {
30 public:
31 Iterator() {}
32 virtual ~Iterator() {}
33
34 // An iterator is either positioned at a key/value pair, or
35 // not valid. This method returns true iff the iterator is valid.
36 virtual bool Valid() const = 0;
37
38 virtual uint64_t skip_count() const { return 0; } //增加的第一行
vim db/db_iter.cc +51
51 class DBIter final: public Iterator {
52 public:
53 // The following is grossly complicated. TODO: clean it up
54 // Which direction is the iterator currently moving?
55 // (1) When moving forward, the internal iterator is positioned at
56 // the exact entry that yields this->key(), this->value()
57 // (2) When moving backwards, the internal iterator is positioned
58 // just before all entries whose user key == this->key().
59 enum Direction {
60 kForward,
61 kReverse
62 };
63
64 uint64_t skip_count_ = 0; //增加的第二行
65 uint64_t skip_count() const override; //增加的第三行
327 uint64_t DBIter::skip_count() const { return skip_count_; } //增加的第四行
1326 uint64_t ArenaWrappedDBIter::skip_count() const { return db_iter_->skip_count(); }。//增加的第五行
vim db/db_iter.h +45
45 class ArenaWrappedDBIter : public Iterator {
46 public:
47 virtual ~ArenaWrappedDBIter();
48
49 // db/db_iter.h (class ArenaWrappedDBIter 内)
50 uint64_t skip_count() const override; //增加的第六行
`
上面是给迭代器增加 skip_count_
收集跳过，还需要在DBIter::FindNextUserEntryInternal在跳过的地方（就是所有PERF_COUNTER_ADD后面）增加++skip_count_;
比如以下是我增加的一个地方
vim db/db_iter.cc
464 PERF_COUNTER_ADD(internal_delete_skipped_count, 1);
465 ++skip_count_;
Proposed solution
改善AddCompactKeyTaskIfNeeded
Alternatives considered
优化UpdateSpecificKeyStatistics