Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,36 @@ When adding code, strictly follow existing similar code in similar contexts, inc

After adding code, you must first conduct self-review and refactoring attempts to ensure good abstraction and reuse as much as possible.

## Logging Standards

All log messages in FE (Java) must follow these rules. Code review must check new/modified log statements against these standards.

### Log Level Guidelines

- **ERROR**: System/component-level failures requiring human intervention. Must include exception stack traces and context.
- **WARN**: Recoverable abnormal situations that may affect users. Must NOT be used for messages that repeat continuously under normal conditions.
- **INFO**: Key business events, state changes, important operation completions. Must NOT be used for per-second periodic reports or per-request tracing.
- **DEBUG**: Detailed operational/debugging info such as per-RPC calls, per-tablet scheduling, per-file-system operations.

### Grammar and Style Rules

1. **Use correct English grammar** — avoid Chinglish patterns:
- ✅ `"Finished checking tablets"` / ❌ `"finished to check tablets"`
- ✅ `"Begin to schedule tablets"` / ❌ `"beginning to tablet scheduler"`
- ✅ `"Start saving image"` / ❌ `"start to save image"`
2. **Capitalize the first letter** of log messages.
3. **Keep acronyms uppercase**: `GC`, `RPC`, `IO`, `SQL`, `JDBC`, `LDAP`, `HDFS`.
4. **Include sufficient context**: Log messages must contain the object identifier (table name, DB name, ID, etc.) to make the log actionable.
- ✅ `"Failed to create table: db={}, table={}, reason={}"` / ❌ `"Failed to create a table."`
5. **Use consistent key=value format**: Use `key=value` with camelCase keys and no spaces around `=`.

### Output Rules

1. **No logging on idle cycles**: Periodic daemons must NOT log at INFO when there is zero work. Use DEBUG or skip.
2. **Aggregate high-frequency logs**: For operations that fire every second, produce periodic summaries (e.g., every 5 minutes) instead of per-event logs.
3. **Do not serialize full Thrift/Protobuf objects**: Extract and log only key fields (ID, status, names). Single INFO log lines should not exceed 200 characters.
4. **Suppress third-party library noise**: Configure log4j2 to set third-party loggers (Kerberos, Hive MetaStore client, Airlift, Hadoop security) to WARN or OFF.

## Code Review

When conducting code review (including self-review and review tasks), it is necessary to complete the key checkpoints according to our `code-review` skill and provide conclusions for each key checkpoint (if applicable) as part of the final written description. Other content does not require individual responses; just check them during the review process.
Expand Down
8 changes: 4 additions & 4 deletions be/src/agent/task_worker_pool.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1045,7 +1045,7 @@ void update_tablet_meta_callback(StorageEngine& engine, const TAgentTaskRequest&
}
}

LOG(INFO) << "finish update tablet meta task. signature=" << req.signature;
LOG(INFO) << "Finished updating tablet meta task. signature=" << req.signature;
if (req.signature != -1) {
TFinishTaskRequest finish_task_request;
finish_task_request.__set_task_status(status.to_thrift());
Expand Down Expand Up @@ -1782,7 +1782,7 @@ void create_tablet_callback(StorageEngine& engine, const TAgentTaskRequest& req)
}
};
DorisMetrics::instance()->create_tablet_requests_total->increment(1);
VLOG_NOTICE << "start to create tablet " << create_tablet_req.tablet_id;
VLOG_NOTICE << "Start creating tablet " << create_tablet_req.tablet_id;

std::vector<TTabletInfo> finish_tablet_infos;
VLOG_NOTICE << "create tablet: " << create_tablet_req;
Expand Down Expand Up @@ -2410,7 +2410,7 @@ void calc_delete_bitmap_callback(CloudStorageEngine& engine, const TAgentTaskReq
SCOPED_ATTACH_TASK(engine_task.mem_tracker());
if (req.signature != calc_delete_bitmap_req.transaction_id) {
// transaction_id may not be the same as req.signature, so add a log here
LOG_INFO("begin to execute calc delete bitmap task")
LOG_INFO("Start executing calc delete bitmap task")
.tag("signature", req.signature)
.tag("transaction_id", calc_delete_bitmap_req.transaction_id);
}
Expand Down Expand Up @@ -2443,7 +2443,7 @@ void make_cloud_committed_rs_visible_callback(CloudStorageEngine& engine,
if (!config::enable_cloud_make_rs_visible_on_be) {
return;
}
LOG(INFO) << "begin to make cloud tmp rs visible, txn_id="
LOG(INFO) << "Start making cloud tmp rs visible, txn_id="
<< req.make_cloud_tmp_rs_visible_req.txn_id
<< ", tablet_count=" << req.make_cloud_tmp_rs_visible_req.tablet_ids.size();

Expand Down
10 changes: 5 additions & 5 deletions be/src/cloud/cloud_compaction_action.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@ Status CloudCompactionAction::_handle_show_compaction(HttpRequest* req, std::str
return Status::InternalError("check param failed: missing tablet_id");
}

LOG(INFO) << "begin to handle show compaction, tablet id: " << tablet_id;
LOG(INFO) << "Start handling show compaction, tablet id: " << tablet_id;

//TabletSharedPtr tablet = _engine.tablet_manager()->get_tablet(tablet_id);
CloudTabletSPtr tablet = DORIS_TRY(_engine.tablet_mgr().get_tablet(tablet_id));
Expand All @@ -129,7 +129,7 @@ Status CloudCompactionAction::_handle_show_compaction(HttpRequest* req, std::str
}

tablet->get_compaction_status(json_result);
LOG(INFO) << "finished to handle show compaction, tablet id: " << tablet_id;
LOG(INFO) << "Finished handling show compaction, tablet id: " << tablet_id;
return Status::OK();
}

Expand All @@ -139,7 +139,7 @@ Status CloudCompactionAction::_handle_run_compaction(HttpRequest* req, std::stri
uint64_t tablet_id = 0;
uint64_t table_id = 0;
RETURN_NOT_OK_STATUS_WITH_WARN(_check_param(req, &tablet_id, &table_id), "check param failed");
LOG(INFO) << "begin to handle run compaction, tablet id: " << tablet_id
LOG(INFO) << "Start handling run compaction, tablet id: " << tablet_id
<< " table id: " << table_id;

// check compaction_type equals 'base' or 'cumulative'
Expand Down Expand Up @@ -186,7 +186,7 @@ Status CloudCompactionAction::_handle_run_status_compaction(HttpRequest* req,
uint64_t tablet_id = 0;
RETURN_NOT_OK_STATUS_WITH_WARN(_check_param(req, &tablet_id, TABLET_ID_KEY),
"check param failed");
LOG(INFO) << "begin to handle run status compaction, tablet id: " << tablet_id;
LOG(INFO) << "Start handling run status compaction, tablet id: " << tablet_id;

if (tablet_id == 0) {
// overall compaction status
Expand Down Expand Up @@ -233,7 +233,7 @@ Status CloudCompactionAction::_handle_run_status_compaction(HttpRequest* req,
// not running any compaction
*json_result = absl::Substitute(json_template, run_status, msg, tablet_id, compaction_type);
}
LOG(INFO) << "finished to handle run status compaction, tablet id: " << tablet_id;
LOG(INFO) << "Finished handling run status compaction, tablet id: " << tablet_id;
return Status::OK();
}

Expand Down
2 changes: 1 addition & 1 deletion be/src/cloud/cloud_delete_task.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ namespace doris {
using namespace ErrorCode;

Status CloudDeleteTask::execute(CloudStorageEngine& engine, const TPushReq& request) {
VLOG_DEBUG << "begin to process delete data. request=" << ThriftDebugString(request);
VLOG_DEBUG << "Start processing delete data. request=" << ThriftDebugString(request);

if (!request.__isset.transaction_id) {
return Status::InvalidArgument("transaction_id is not set");
Expand Down
14 changes: 7 additions & 7 deletions be/src/cloud/cloud_engine_calc_delete_bitmap_task.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ void CloudEngineCalcDeleteBitmapTask::add_succ_tablet_id(int64_t tablet_id) {
Status CloudEngineCalcDeleteBitmapTask::execute() {
int64_t transaction_id = _cal_delete_bitmap_req.transaction_id;
OlapStopWatch watch;
VLOG_NOTICE << "begin to calculate delete bitmap. transaction_id=" << transaction_id;
VLOG_NOTICE << "Start calculating delete bitmap. transaction_id=" << transaction_id;
std::unique_ptr<ThreadPoolToken> token =
_engine.calc_tablet_delete_bitmap_task_thread_pool().new_token(
ThreadPool::ExecutionMode::CONCURRENT);
Expand Down Expand Up @@ -113,7 +113,7 @@ Status CloudEngineCalcDeleteBitmapTask::execute() {
// wait for all finished
token->wait();

LOG(INFO) << "finish to calculate delete bitmap on transaction."
LOG(INFO) << "Finished calculating delete bitmap on transaction."
<< "transaction_id=" << transaction_id << ", cost(us): " << watch.get_elapse_time_us()
<< ", error_tablet_size=" << _error_tablet_ids->size()
<< ", res=" << _res.to_string();
Expand Down Expand Up @@ -145,7 +145,7 @@ void CloudTabletCalcDeleteBitmapTask::set_tablet_state(int64_t tablet_state) {
}

Status CloudTabletCalcDeleteBitmapTask::handle() const {
VLOG_DEBUG << "start calculate delete bitmap on tablet " << _tablet_id
VLOG_DEBUG << "Start calculating delete bitmap on tablet " << _tablet_id
<< ", txn_id=" << _transaction_id;
SCOPED_ATTACH_TASK(_mem_tracker);
int64_t t1 = MonotonicMicros();
Expand Down Expand Up @@ -236,8 +236,8 @@ Status CloudTabletCalcDeleteBitmapTask::handle() const {
for (const auto& sub_txn_id : _sub_txn_ids) {
ss << sub_txn_id << ", ";
}
LOG(INFO) << "start calc delete bitmap for txn_id=" << _transaction_id << ", sub_txn_ids=["
<< ss.str() << "], table_id=" << tablet->table_id()
LOG(INFO) << "Start calculating delete bitmap for txn_id=" << _transaction_id
<< ", sub_txn_ids=[" << ss.str() << "], table_id=" << tablet->table_id()
<< ", partition_id=" << tablet->partition_id() << ", tablet_id=" << _tablet_id
<< ", start_version=" << _version;
std::vector<RowsetSharedPtr> invisible_rowsets;
Expand All @@ -255,7 +255,7 @@ Status CloudTabletCalcDeleteBitmapTask::handle() const {
empty_rowset_count++;
continue;
}
LOG(INFO) << "start calc delete bitmap for txn_id=" << _transaction_id
LOG(INFO) << "Start calculating delete bitmap for txn_id=" << _transaction_id
<< ", sub_txn_id=" << sub_txn_id << ", table_id=" << tablet->table_id()
<< ", partition_id=" << tablet->partition_id() << ", tablet_id=" << _tablet_id
<< ", start_version=" << _version << ", cur_version=" << version;
Expand Down Expand Up @@ -283,7 +283,7 @@ Status CloudTabletCalcDeleteBitmapTask::handle() const {
}
});
auto total_update_delete_bitmap_time_us = MonotonicMicros() - t3;
LOG(INFO) << "finish calculate delete bitmap on tablet"
LOG(INFO) << "Finished calculating delete bitmap on tablet"
<< ", table_id=" << tablet->table_id() << ", transaction_id=" << _transaction_id
<< ", tablet_id=" << tablet->tablet_id()
<< ", get_tablet_time_us=" << get_tablet_time_us
Expand Down
4 changes: 2 additions & 2 deletions be/src/cloud/cloud_rowset_writer.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ Status CloudRowsetWriter::init(const RowsetWriterContext& rowset_writer_context)
}

Status CloudRowsetWriter::_build_rowset_meta(RowsetMeta* rowset_meta, bool check_segment_num) {
VLOG_NOTICE << "start to build rowset meta. tablet_id=" << rowset_meta->tablet_id()
VLOG_NOTICE << "Start building rowset meta. tablet_id=" << rowset_meta->tablet_id()
<< ", rowset_id=" << rowset_meta->rowset_id()
<< ", check_segment_num=" << check_segment_num;
// Call base class implementation
Expand Down Expand Up @@ -162,7 +162,7 @@ Status CloudRowsetWriter::build(RowsetSharedPtr& rowset) {
}

Status CloudRowsetWriter::_collect_all_packed_slice_locations(RowsetMeta* rowset_meta) {
VLOG_NOTICE << "start to collect packed slice locations for rowset meta. tablet_id="
VLOG_NOTICE << "Start collecting packed slice locations for rowset meta. tablet_id="
<< rowset_meta->tablet_id() << ", rowset_id=" << rowset_meta->rowset_id();
if (!_context.packed_file_active) {
return Status::OK();
Expand Down
8 changes: 4 additions & 4 deletions be/src/cloud/cloud_snapshot_loader.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ Status CloudSnapshotLoader::download(const std::map<std::string, std::string>& s
return Status::InternalError("Storage backend not initialized.");
}

LOG(INFO) << "begin to transfer snapshot files. num: " << src_to_dest_path.size()
LOG(INFO) << "Start transferring snapshot files. num: " << src_to_dest_path.size()
<< ", broker addr: " << _broker_addr << ", job: " << _job_id
<< ", task id: " << _task_id;

Expand Down Expand Up @@ -187,7 +187,7 @@ Status CloudSnapshotLoader::download(const std::map<std::string, std::string>& s
RETURN_IF_ERROR(_engine.cloud_snapshot_mgr().make_snapshot(
target_tablet_id, *_storage_resource, file_mapping, true, &hdr_slice));

LOG(INFO) << "finish to make snapshot for tablet: " << target_tablet_id;
LOG(INFO) << "Finished making snapshot for tablet: " << target_tablet_id;

// 1.5. download files
for (auto& nested_iter : remote_files) {
Expand All @@ -202,7 +202,7 @@ Status CloudSnapshotLoader::download(const std::map<std::string, std::string>& s
std::string target_file = find->second;
std::string full_remote_file = remote_path + "/" + remote_file + "." + file_stat.md5;
std::string full_target_file = target_path + "/" + target_file;
LOG(INFO) << "begin to download from " << full_remote_file << " to "
LOG(INFO) << "Start downloading from " << full_remote_file << " to "
<< full_target_file;
io::FileReaderOptions nested_reader_options {
.cache_type = io::FileCachePolicy::NO_CACHE,
Expand Down Expand Up @@ -239,7 +239,7 @@ Status CloudSnapshotLoader::download(const std::map<std::string, std::string>& s
// (TODO) Add bvar metrics to track download time
} // end for src_to_dest_path

LOG(INFO) << "finished to download snapshots. job: " << _job_id << ", task id: " << _task_id;
LOG(INFO) << "Finished downloading snapshots. job: " << _job_id << ", task id: " << _task_id;
return status;
}

Expand Down
4 changes: 2 additions & 2 deletions be/src/cloud/cloud_tablet.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -493,7 +493,7 @@ void CloudTablet::delete_rowsets(const std::vector<RowsetSharedPtr>& to_delete,

uint64_t CloudTablet::delete_expired_stale_rowsets() {
if (config::enable_mow_verbose_log) {
LOG_INFO("begin delete_expired_stale_rowset for tablet={}", tablet_id());
LOG_INFO("Begin to delete_expired_stale_rowset for tablet={}", tablet_id());
}
std::vector<RowsetSharedPtr> expired_rowsets;
// ATTN: trick, Use stale_rowsets to temporarily increase the reference count of the rowset shared pointer in _stale_rs_version_map so that in the recycle_cached_data function, it checks if the reference count is 2.
Expand Down Expand Up @@ -554,7 +554,7 @@ uint64_t CloudTablet::delete_expired_stale_rowsets() {
manager.recycle_cache(tablet_id(), recycled_rowsets);
}
if (config::enable_mow_verbose_log) {
LOG_INFO("finish delete_expired_stale_rowset for tablet={}", tablet_id());
LOG_INFO("Finished deleting_expired_stale_rowset for tablet={}", tablet_id());
}

add_unused_rowsets(expired_rowsets);
Expand Down
16 changes: 8 additions & 8 deletions be/src/cloud/cloud_tablet_mgr.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -295,7 +295,7 @@ void CloudTabletMgr::erase_tablet(int64_t tablet_id) {
}

void CloudTabletMgr::vacuum_stale_rowsets(const CountDownLatch& stop_latch) {
LOG_INFO("begin to vacuum stale rowsets");
LOG_INFO("Start vacuuming stale rowsets");
std::vector<std::shared_ptr<CloudTablet>> tablets_to_vacuum;
tablets_to_vacuum.reserve(_tablet_map->size());
_tablet_map->traverse([&tablets_to_vacuum](auto&& t) {
Expand All @@ -311,12 +311,12 @@ void CloudTabletMgr::vacuum_stale_rowsets(const CountDownLatch& stop_latch) {

num_vacuumed += t->delete_expired_stale_rowsets();
}
LOG_INFO("finish vacuum stale rowsets")
LOG_INFO("Finished vacuuming stale rowsets")
.tag("num_vacuumed", num_vacuumed)
.tag("num_tablets", tablets_to_vacuum.size());

{
LOG_INFO("begin to remove unused rowsets");
LOG_INFO("Start removing unused rowsets");
std::vector<std::shared_ptr<CloudTablet>> tablets_to_remove_unused_rowsets;
tablets_to_remove_unused_rowsets.reserve(_tablet_map->size());
_tablet_map->traverse([&tablets_to_remove_unused_rowsets](auto&& t) {
Expand All @@ -327,7 +327,7 @@ void CloudTabletMgr::vacuum_stale_rowsets(const CountDownLatch& stop_latch) {
for (auto& t : tablets_to_remove_unused_rowsets) {
t->remove_unused_rowsets();
}
LOG_INFO("finish remove unused rowsets")
LOG_INFO("Finished removing unused rowsets")
.tag("num_tablets", tablets_to_remove_unused_rowsets.size());
if (config::enable_check_agg_and_remove_pre_rowsets_delete_bitmap) {
int64_t max_useless_rowset_count = 0;
Expand All @@ -352,7 +352,7 @@ void CloudTabletMgr::vacuum_stale_rowsets(const CountDownLatch& stop_latch) {
g_max_rowsets_with_useless_delete_bitmap.set_value(max_useless_rowset_count);
g_max_rowsets_with_useless_delete_bitmap_version.set_value(
max_useless_rowset_version_count);
LOG(INFO) << "finish check_agg_delete_bitmap_for_stale_rowsets, cost(us)="
LOG(INFO) << "Finished check_agg_delete_bitmap_for_stale_rowsetsing, cost(us)="
<< watch.get_elapse_time_us()
<< ". max useless rowset count=" << max_useless_rowset_count
<< ", tablet_id=" << tablet_id_with_max_useless_rowset_count
Expand All @@ -374,7 +374,7 @@ std::vector<std::weak_ptr<CloudTablet>> CloudTabletMgr::get_weak_tablets() {
}

void CloudTabletMgr::sync_tablets(const CountDownLatch& stop_latch) {
LOG_INFO("begin to sync tablets");
LOG_INFO("Start syncing tablets");
int64_t last_sync_time_bound = ::time(nullptr) - config::tablet_sync_interval_s;

auto weak_tablets = get_weak_tablets();
Expand Down Expand Up @@ -421,7 +421,7 @@ void CloudTabletMgr::sync_tablets(const CountDownLatch& stop_latch) {
}
}
}
LOG_INFO("finish sync tablets").tag("num_sync", num_sync);
LOG_INFO("Finished syncing tablets").tag("num_sync", num_sync);
}

Status CloudTabletMgr::get_topn_tablets_to_compact(
Expand Down Expand Up @@ -523,7 +523,7 @@ Status CloudTabletMgr::get_topn_tablets_to_compact(
void CloudTabletMgr::build_all_report_tablets_info(std::map<TTabletId, TTablet>* tablets_info,
uint64_t* tablet_num) {
DCHECK(tablets_info != nullptr);
VLOG_NOTICE << "begin to build all report cloud tablets info";
VLOG_NOTICE << "Start building all report cloud tablets info";

HistogramStat tablet_version_num_hist;

Expand Down
2 changes: 1 addition & 1 deletion be/src/exec/operator/olap_scan_operator.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -774,7 +774,7 @@ Status OlapScanLocalState::prepare(RuntimeState* state) {
}
if (config::enable_mow_verbose_log &&
_tablets[i].tablet->enable_unique_key_merge_on_write()) {
LOG_INFO("finish capture_rs_readers for tablet={}, query_id={}",
LOG_INFO("Finished capturing_rs_readers for tablet={}, query_id={}",
_tablets[i].tablet->tablet_id(),
print_id(PipelineXLocalState<>::_state->query_id()));
}
Expand Down
4 changes: 2 additions & 2 deletions be/src/exec/scan/scanner_scheduler.h
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@ class ThreadPoolSimplifiedScanScheduler MOCK_REMOVE(final) : public ScannerSched
#ifndef BE_TEST
stop();
#endif
LOG(INFO) << "Scanner sche " << _sched_name << " shutdown";
LOG(INFO) << "Scanner scheduler " << _sched_name << " shutdown";
}

void stop() override {
Expand Down Expand Up @@ -260,7 +260,7 @@ class TaskExecutorSimplifiedScanScheduler final : public ScannerScheduler {
#ifndef BE_TEST
stop();
#endif
LOG(INFO) << "Scanner sche " << _sched_name << " shutdown";
LOG(INFO) << "Scanner scheduler " << _sched_name << " shutdown";
}

void stop() override {
Expand Down
2 changes: 1 addition & 1 deletion be/src/exec/sink/writer/vtablet_writer.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1980,7 +1980,7 @@ Status VTabletWriter::close(Status exec_status) {

// print log of add batch time of all node, for tracing load performance easily
std::stringstream ss;
ss << "finished to close olap table sink. load_id=" << print_id(_load_id)
ss << "Finished closing olap table sink. load_id=" << print_id(_load_id)
<< ", txn_id=" << _txn_id
<< ", node add batch time(ms)/wait execution time(ms)/close time(ms)/num: ";
for (auto const& pair : node_add_batch_counter_map) {
Expand Down
Loading
Loading