Add optional stats files to heartbeat export - #1604
Conversation
Greptile SummaryThis PR adds optional dashboard-statistics files to heartbeat export archives and exposes the option on the imports and exports settings page.
Confidence Score: 0/5This PR is not safe to merge because stats-enabled exports remain unreachable from the UI, fail during weekly snapshot generation and leave one generated CSV formula-active. The checkbox does not maintain reactive submission state, processed_export_snapshot calls weekly_project_stats with unsupported keywords, weekly ranges still ignore the export dates, weekly project cells bypass CSV sanitisation and the new controller regression test cannot execute its intended path. Files Needing Attention: app/javascript/pages/Users/Settings/ImportsExports.svelte, app/services/dashboard_data/snapshots.rb, app/jobs/heartbeat_export_job.rb and test/controllers/my/heartbeats_controller_test.rb
|
| Filename | Overview |
|---|---|
| app/javascript/pages/Users/Settings/ImportsExports.svelte | Adds the stats checkbox and hidden fields, but the checkbox state is not reactive so the selection is not reliably submitted. |
| app/controllers/my/heartbeats_controller.rb | Correctly forwards include_stats when the parameter is the literal string "true". |
| app/jobs/heartbeat_export_job.rb | Adds stats files to export archives, but weekly project values bypass the new CSV formula sanitiser. |
| app/services/dashboard_data/snapshots.rb | Adds the export snapshot, but calls weekly_project_stats with unsupported keywords while the underlying range calculation remains fixed to twelve recent weeks. |
| test/controllers/my/heartbeats_controller_test.rb | Adds controller coverage that cannot execute because its user is uninitialised and its parameter values do not enable either boolean. |
| test/jobs/heartbeat_export_job_test.rb | Adds archive-entry coverage for enabled and disabled stats exports, though the enabled path is blocked by the snapshot keyword mismatch. |
Sequence Diagram
sequenceDiagram
participant U as User
participant S as Settings page
participant C as Heartbeats controller
participant J as Heartbeat export job
participant D as Dashboard snapshots
participant Z as ZIP archive
U->>S: Select Include stats and export
S->>C: POST export parameters
C->>J: Enqueue export with include_stats
J->>D: Build export snapshot
D-->>J: Grouped statistics
J->>Z: Add heartbeat JSON and stats files
J-->>U: Email temporary download link
Prompt To Fix All With AI
### Issue 1
app/javascript/pages/Users/Settings/ImportsExports.svelte:50
**Stats selection stays false**
When a user checks Include stats and submits either export form, `includeStats` is a plain variable rather than Svelte state, so the hidden fields continue submitting `include_stats=false` and the stats option remains unreachable.
```suggestion
let includeStats = $state(false);
```
### Issue 2
app/services/dashboard_data/snapshots.rb:257
**Unsupported keywords abort stats exports**
When an export requests stats, this call passes `start_date` and `end_date` to `weekly_project_stats`, whose signature accepts only `user` and `scope`. Ruby raises an unknown-keyword `ArgumentError`, so stats generation stops before the files are added to the archive.
### Issue 3
app/jobs/heartbeat_export_job.rb:154
**Weekly projects bypass CSV sanitisation**
When a stored heartbeat project begins with a spreadsheet formula prefix, this writer emits the project directly instead of applying the sanitisation used by `stats_to_csv`, leaving a formula-active cell in `weekly_project_stats.csv` when the export is opened in spreadsheet software.
**How this was verified:** Stored project values retain formula prefixes, flow into weekly project keys and are written here without the adjacent CSV sanitiser.
```suggestion
safe_project = project.to_s.gsub(/\A([=+\-@\t\r])/, "'\1")
csv << [week, safe_project, duration]
```
### Issue 4
test/controllers/my/heartbeats_controller_test.rb:76-83
**Stats controller test cannot run**
When this test runs, it dereferences an uninitialised `@user` before posting the request. Even after initialising the user, the submitted values are `"1"` while the controller enables both options only for `"true"`, so the test cannot reach or verify the stats-enabled enqueue path.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Reviews (2): Last reviewed commit: "Merge branch 'main' into add-stats" | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
|
Current checklist of things until ready for review:
#1606 Needs to be merged for this PR to work. |
| let remoteApiKey = $state(""); | ||
| let importOverlay = $state<Partial<HeartbeatImportStatusProps> | null>(null); | ||
| let overlayStartTime = $state<number | null>(null); | ||
| let includeStats = false; |
There was a problem hiding this comment.
When a user checks Include stats and submits either export form, includeStats is a plain variable rather than Svelte state, so the hidden fields continue submitting include_stats=false and the stats option remains unreachable.
| let includeStats = false; | |
| let includeStats = $state(false); |
Context Used: AGENTS.md (source)
Knowledge Base Used: Heartbeat imports and exports
Prompt To Fix With AI
This is a comment left during a code review.
Path: app/javascript/pages/Users/Settings/ImportsExports.svelte
Line: 50
Comment:
**Stats selection stays false**
When a user checks Include stats and submits either export form, `includeStats` is a plain variable rather than Svelte state, so the hidden fields continue submitting `include_stats=false` and the stats option remains unreachable.
```suggestion
let includeStats = $state(false);
```
**Context Used:** AGENTS.md ([source](https://github.com/hackclub/hackatime/blob/main/AGENTS.md))
**Knowledge Base Used:** [Heartbeat imports and exports](https://app.greptile.com/mahadk/-/custom-context/knowledge-base/hackclub/hackatime/-/docs/heartbeat-imports-and-exports.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| operating_system_stats: Heartbeat.attributed_durations_by(scope, :operating_system), | ||
| category_stats: Heartbeat.attributed_durations_by(scope, :category), | ||
|
|
||
| weekly_project_stats: weekly_project_stats(user: user, scope: scope, start_date: start_date, end_date: end_date), |
There was a problem hiding this comment.
When an export requests stats, this call passes start_date and end_date to weekly_project_stats, whose signature accepts only user and scope. Ruby raises an unknown-keyword ArgumentError, so stats generation stops before the files are added to the archive.
Knowledge Base Used:
Prompt To Fix With AI
This is a comment left during a code review.
Path: app/services/dashboard_data/snapshots.rb
Line: 257
Comment:
**Unsupported keywords abort stats exports**
When an export requests stats, this call passes `start_date` and `end_date` to `weekly_project_stats`, whose signature accepts only `user` and `scope`. Ruby raises an unknown-keyword `ArgumentError`, so stats generation stops before the files are added to the archive.
**Knowledge Base Used:**
- [Dashboards and user insights](https://app.greptile.com/mahadk/-/custom-context/knowledge-base/hackclub/hackatime/-/docs/dashboards-and-user-insights.md)
- [Heartbeat imports and exports](https://app.greptile.com/mahadk/-/custom-context/knowledge-base/hackclub/hackatime/-/docs/heartbeat-imports-and-exports.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.|
|
||
| stats.each do |week, projects| | ||
| projects.each do |project, duration| | ||
| csv << [week, project, duration] |
There was a problem hiding this comment.
When a stored heartbeat project begins with a spreadsheet formula prefix, this writer emits the project directly instead of applying the sanitisation used by stats_to_csv, leaving a formula-active cell in weekly_project_stats.csv when the export is opened in spreadsheet software.
How this was verified: Stored project values retain formula prefixes, flow into weekly project keys and are written here without the adjacent CSV sanitiser.
| csv << [week, project, duration] | |
| safe_project = project.to_s.gsub(/\A([=+\-@\t\r])/, "'\1") | |
| csv << [week, safe_project, duration] |
Knowledge Base Used: Heartbeat tracking and ingestion
Prompt To Fix With AI
This is a comment left during a code review.
Path: app/jobs/heartbeat_export_job.rb
Line: 154
Comment:
**Weekly projects bypass CSV sanitisation**
When a stored heartbeat project begins with a spreadsheet formula prefix, this writer emits the project directly instead of applying the sanitisation used by `stats_to_csv`, leaving a formula-active cell in `weekly_project_stats.csv` when the export is opened in spreadsheet software.
**How this was verified:** Stored project values retain formula prefixes, flow into weekly project keys and are written here without the adjacent CSV sanitiser.
```suggestion
safe_project = project.to_s.gsub(/\A([=+\-@\t\r])/, "'\1")
csv << [week, safe_project, duration]
```
**Knowledge Base Used:** [Heartbeat tracking and ingestion](https://app.greptile.com/mahadk/-/custom-context/knowledge-base/hackclub/hackatime/-/docs/heartbeat-tracking.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| test "POST export enqueues job with include_stats when checked" do | ||
| assert_enqueued_with( | ||
| job: HeartbeatExportJob, | ||
| args: [@user.id, hash_including(include_stats: true, all_data: true)] | ||
| ) do | ||
| post export_my_heartbeats_url, params: { all_data: "1", include_stats: "1" } | ||
| end | ||
| end |
There was a problem hiding this comment.
When this test runs, it dereferences an uninitialised @user before posting the request. Even after initialising the user, the submitted values are "1" while the controller enables both options only for "true", so the test cannot reach or verify the stats-enabled enqueue path.
Context Used: In the Hackatime repo, please ensure that the PR i... (source)
Knowledge Base Used: Heartbeat imports and exports
Prompt To Fix With AI
This is a comment left during a code review.
Path: test/controllers/my/heartbeats_controller_test.rb
Line: 76-83
Comment:
**Stats controller test cannot run**
When this test runs, it dereferences an uninitialised `@user` before posting the request. Even after initialising the user, the submitted values are `"1"` while the controller enables both options only for `"true"`, so the test cannot reach or verify the stats-enabled enqueue path.
**Context Used:** In the Hackatime repo, please ensure that the PR i... ([source](https://app.greptile.com/review/custom-context?memory=instruction-0))
**Knowledge Base Used:** [Heartbeat imports and exports](https://app.greptile.com/mahadk/-/custom-context/knowledge-base/hackclub/hackatime/-/docs/heartbeat-imports-and-exports.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.1 participant
Summary of the problem
These changes are made to provide the user an easier way to get the data shown in graphs.
Describe your changes
This pull request adds the ability to include detailed statistics in the heartbeat data export, providing users with additional CSV and JSON files containing breakdowns of their coding activity. The main changes involve updating the export flow to accept a new
include_statsparameter, generating various stats files, and enhancing the export job to bundle these files into the exported ZIP archive.Export Flow Enhancements:
my/heartbeats_controller.rbnow accept aninclude_statsparameter, which is passed to the export job to trigger stats file generation.Heartbeat Export Job Improvements:
HeartbeatExportJobnow accepts and handles theinclude_statsparameter, generating additional CSV and JSON statistics files (such as project durations, language stats, editor stats, weekly project stats, coding rhythm, etc.) and includes them in the ZIP archive if requested.Statistics Generation:
processed_export_snapshotwas added toDashboardData::Snapshotsto aggregate all relevant statistics for the export, including project, language, editor, OS, category breakdowns, weekly project stats, and coding rhythm.Other Minor Changes:
csvlibrary inheartbeat_export_job.rbto support CSV generation.These changes collectively allow users to optionally receive detailed, structured statistics about their coding activity alongside the standard heartbeat export.
Screenshots / Media