-
-
Notifications
You must be signed in to change notification settings - Fork 1.7k
fix(drivers/teldrive): enhance file listing and upload functionality with pagination and random chunk naming #2034
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
…with pagination and random chunk naming
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR enhances the Teldrive driver’s listing and upload behavior to better match Teldrive’s API capabilities and large-folder workloads.
Changes:
- Adds paginated, concurrent listing in
List, fetching all pages from/api/fileswith a limit of 500 items per page usingerrgroup. - Introduces
RandomChunkNameconfiguration and wiring so single- and multi-part uploads can use random MD5-based part names instead of deterministic*.part.XXX. - Adjusts upload behavior, including progress reporting and retry configuration, and increases the default
chunk_sizeto 500 MiB in the driver’s metadata.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
drivers/teldrive/driver.go |
Implements paginated, concurrent listing in List and propagates maxRetried into the single-upload path in Put. |
drivers/teldrive/upload.go |
Adds random chunk naming, threads context.Context through low-level upload requests, tweaks single-upload chunking and progress semantics, and aligns single- and multi-part upload behavior. |
drivers/teldrive/meta.go |
Updates Teldrive driver metadata to increase the chunk_size default to 500 MiB and introduces the random_chunk_name boolean option. |
Comments suppressed due to low confidence (1)
drivers/teldrive/upload.go:371
- In
uploadSingleChunk, the sametask.readeris reused across retries without ever seeking back to the start of the chunk, so after a failed attempt subsequent retries will read from the end of the section and effectively send an empty body. This differs fromdoSingleUpload, which explicitly seeks its section reader to offset 0 before each retry. To ensure retries resend the full chunk data, resettask.reader(e.g.,Seek(0, io.SeekStart)) before constructing the request body for each attempt.
err := d.singleUploadRequest(ctx, fileId, func(req *resty.Request) {
uploadParams := map[string]string{
"partName": partName,
"partNo": strconv.Itoa(task.chunkIdx),
"fileName": task.fileName,
}
req.SetQueryParams(uploadParams)
req.SetBody(driver.NewLimitedUploadStream(ctx, task.reader))
req.SetHeader("Content-Length", strconv.Itoa(int(task.chunkSize)))
}, filePart)
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…mutex and restructuring data handling
|
What should I do next? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Chaloemchai <chaloemchai.yy@gmail.com>
Description / 描述
This PR introduces critical improvements to the Teldrive driver for better reliability and feature parity with Teldrive's capabilities:
Listfunction to fetch files in batches of 500 per page (following the [Teldrive API Documentation](https://teldrive-docs.pages.dev/docs/api#tag/files/get/files)). It useserrgroupto fetch multiple pages concurrently, ensuring that directories containing thousands of items are listed completely and efficiently.RandomChunkNameconfiguration. When enabled, each uploaded file chunk is assigned a random MD5 hash as its name instead of the default part naming scheme. This aligns with Teldrive's native support for randomized part names to prevent potential naming pattern conflicts.Motivation and Context / 背景
How Has This Been Tested? / 测试
random_chunk_nameenabled. Confirmed on the server-side that the uploaded parts are named using random hashes as expected.Checklist / 检查清单
我已阅读 CONTRIBUTING 文档。
go fmtor prettier.我已使用
go fmt或 prettier 格式化提交的代码。我已为此 PR 添加了适当的标签(如无权限或需要的标签不存在,请在描述中说明,管理员将后续处理)。
我已在适当情况下使用"Request review"功能请求相关代码作者进行审查。
我已相应更新了相关仓库(若适用)。