⚡ Bolt: Parallelize sitemap data fetching with Promise.all#74
⚡ Bolt: Parallelize sitemap data fetching with Promise.all#74
Conversation
Replaces sequential awaits in the sitemap generation loop with Promise.all for fetching data. This parallelizes fetching multiple resources and across multiple years, improving generation speed. Co-authored-by: anyulled <100741+anyulled@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important Review skippedReview was skipped due to path filters ⛔ Files ignored due to path filters (1)
CodeRabbit blocks several paths by default. You can override this behavior by explicitly including those paths in the path filters. For example, including ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughRefactors sitemap generation to aggregate per-year URLs via parallelized Promise.all calls (with defensive .catch fallbacks), adds a diary entry documenting the change, reformats the GitHub Actions workflow YAML, updates scorecard report formatting, adjusts a Cypress selector, and removes two package override entries. Changes
Sequence Diagram(s)sequenceDiagram
participant Sitemap as Sitemap.fn
participant Speakers as getSpeakers(year)
participant Talks as getTalks(year)
participant Aggregator as YearAggregator
Sitemap->>Aggregator: for each year -> create local yearUrls
par Parallel fetch for year
Sitemap->>Speakers: request speakers for year
Sitemap->>Talks: request talks for year
and
Speakers-->>Sitemap: speakers[] (or [] on .catch)
Talks-->>Sitemap: talks[] (or [] on .catch)
end
Sitemap->>Aggregator: push root, speaker and talk URLs into yearUrls
Aggregator->>Sitemap: return yearUrls
Sitemap->>Sitemap: Promise.all(yearPromises) -> flatten -> return full sitemap
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the performance of sitemap generation by refactoring data fetching logic to utilize parallel execution. By moving from sequential Highlights
Changelog
Ignored Files
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request effectively refactors the sitemap generation logic to parallelize data fetching using Promise.all, which should significantly improve build performance. The implementation is solid. I've added a couple of suggestions to further improve the code's clarity and conciseness by removing redundant error handling and using the .flat() method for array manipulation. Overall, this is a great performance enhancement.
| } | ||
|
|
||
| const speakers = await getSpeakers(year); | ||
| const [speakers, sessionGroups] = await Promise.all([getSpeakers(year).catch(() => []), getTalks(year).catch(() => [])]); |
There was a problem hiding this comment.
The .catch(() => []) calls are redundant here. The getSpeakers and getTalks functions already have internal try/catch blocks that handle errors by logging them and returning an empty array. As a result, these promises will always resolve and never reject. Removing the .catch() handlers will make the code cleaner and clearer without changing its behavior.
| const [speakers, sessionGroups] = await Promise.all([getSpeakers(year).catch(() => []), getTalks(year).catch(() => [])]); | |
| const [speakers, sessionGroups] = await Promise.all([getSpeakers(year), getTalks(year)]); |
| for (const yearUrls of yearUrlsArrays) { | ||
| urls.push(...yearUrls); | ||
| } |
There was a problem hiding this comment.
You can simplify this loop by using the .flat() method to flatten the array of arrays and then spreading the result into urls.push(). This is more concise and idiomatic.
urls.push(...yearUrlsArrays.flat());
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (2)
app/sitemap.ts (2)
31-31: Consider bounded concurrency for long-term stability.Line 31 fans out all years at once. As editions grow, adding a small concurrency cap can prevent burst load/rate-limit spikes during sitemap generation.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@app/sitemap.ts` at line 31, The current years.map(async (year) => ...) (creating yearPromises) fans out all years concurrently; change this to use bounded concurrency by wrapping the per-year async task with a concurrency limiter (e.g., p-limit or an internal semaphore/worker pool) and a small CONCURRENCY constant, then build yearPromises via years.map(year => limit(() => theExistingPerYearAsyncFunction(year))) or via chunked/for-loop dispatch to ensure only CONCURRENCY promises run at once; update references to yearPromises so the rest of the sitemap generation awaits the limited set as before.
51-52: Remove redundant.catch(() => [])wrappers aroundgetSpeakers/getTalks.At Line 51, these catches duplicate existing fallback behavior and reduce error visibility if those functions change later.
♻️ Suggested simplification
- const [speakers, sessionGroups] = await Promise.all([getSpeakers(year).catch(() => []), getTalks(year).catch(() => [])]); + const [speakers, sessionGroups] = await Promise.all([getSpeakers(year), getTalks(year)]);🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@app/sitemap.ts` around lines 51 - 52, Remove the redundant .catch(() => []) wrappers around getSpeakers and getTalks in the Promise.all call: replace Promise.all([getSpeakers(year).catch(() => []), getTalks(year).catch(() => [])]) with Promise.all([getSpeakers(year), getTalks(year)]), letting existing fallback/error handling in getSpeakers/getTalks or their callers handle failures; update references to the destructured variables speakers and sessionGroups accordingly.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@app/sitemap.ts`:
- Line 31: The current years.map(async (year) => ...) (creating yearPromises)
fans out all years concurrently; change this to use bounded concurrency by
wrapping the per-year async task with a concurrency limiter (e.g., p-limit or an
internal semaphore/worker pool) and a small CONCURRENCY constant, then build
yearPromises via years.map(year => limit(() =>
theExistingPerYearAsyncFunction(year))) or via chunked/for-loop dispatch to
ensure only CONCURRENCY promises run at once; update references to yearPromises
so the rest of the sitemap generation awaits the limited set as before.
- Around line 51-52: Remove the redundant .catch(() => []) wrappers around
getSpeakers and getTalks in the Promise.all call: replace
Promise.all([getSpeakers(year).catch(() => []), getTalks(year).catch(() => [])])
with Promise.all([getSpeakers(year), getTalks(year)]), letting existing
fallback/error handling in getSpeakers/getTalks or their callers handle
failures; update references to the destructured variables speakers and
sessionGroups accordingly.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 836134e3-6d0e-4d94-8b3a-d82fb9af8f88
📒 Files selected for processing (5)
.github/workflows/scorecard.yml.jules/bolt.mdapp/sitemap.tsscorecard-badge.jsonscorecard-report.md
- Fixed `cypress/e2e/home/home-editions.cy.ts` to query `.hero8-header__event-line` instead of `h5` to match the actual DOM. - Removed conflicting `ajv` and `minimatch` overrides in `package.json` that broke `npm ci` for testing and linting tools. Co-authored-by: anyulled <100741+anyulled@users.noreply.github.com>
- Fixed `cypress/e2e/home/home-editions.cy.ts` to query `.hero8-header__event-line` instead of `h5` to match the actual DOM. - Removed conflicting `ajv` and `minimatch` overrides in `package.json` that broke `npm ci` for testing and linting tools. - Re-ran `npm install` to update `package-lock.json` properly so `npm ci` doesn't fail. Co-authored-by: anyulled <100741+anyulled@users.noreply.github.com>
- Fixed `cypress/e2e/home/home-editions.cy.ts` to query `.hero8-header__event-line` instead of `h5` to match the actual DOM. - Removed conflicting `ajv` and `minimatch` overrides in `package.json` that broke `npm ci` for testing and linting tools. - Re-ran `npm install` to update `package-lock.json` properly so `npm ci` doesn't fail. - Fixed scorecard format failures. Co-authored-by: anyulled <100741+anyulled@users.noreply.github.com>
1 participant
💡 What: Replaced sequential
awaitloops inside theapp/sitemap.tsgeneration function with parallelPromise.allfetching.🎯 Why: The original implementation iterated over
yearsusing afor...ofloop and then sequentially awaitedgetSpeakersandgetTalksinternally. This created a performance bottleneck during the build time since the fetching of resources across different years and even within a single year was blocked until the previous fetch completed.📊 Impact: Reduces total wait time for data fetching significantly during static sitemap generation by parallelizing operations.
🔬 Measurement: Inspect the build time locally or check the static rendering time of
sitemap.xml.PR created automatically by Jules for task 14581543466514914908 started by @anyulled
Summary by CodeRabbit
Refactor
Documentation
Style
Chores
Tests