improvement(cleanup): cleanup refs along in logs cleanup job#4661
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
PR SummaryMedium Risk Overview Reworks retention scoping in Adds Reviewed by Cursor Bugbot for commit b9ffe66. Configure here. |
Greptile SummaryThis PR extends the log cleanup job to also delete orphaned large-value storage refs (S3 keys embedded in
Confidence Score: 5/5Safe to merge — the core cleanup correctness is sound and the dispatcher bug fix is well-reasoned. The two comments are background-job performance observations that do not affect data integrity. The cross-batch reference guard in filterLargeValueKeysWithoutRetainedReferences is logically correct: onBatch runs before the DELETE, so excluding current-batch IDs via notInArray accurately represents which logs will survive. The dispatcher correctly isolates personal-workspace plan resolution from org-level subscriptions, fixing the previously identified enterprise-member bug. The per-key sequential full-table scan and redundant workspace queries in dispatchCleanupJobs are worth addressing but do not produce wrong results. apps/sim/background/cleanup-logs.ts — the per-key position() scan loop warrants a closer look before this runs at enterprise scale. Important Files Changed
Sequence DiagramsequenceDiagram
participant CJ as CleanupLogsJob
participant CBD as chunkedBatchDelete
participant DB as Database
participant Filter as filterLargeValueKeysWithoutRetainedRefs
participant S3 as StorageService
CJ->>CBD: chunkedBatchDelete(workflowExecutionLogs)
loop per workspace chunk x batch
CBD->>DB: "SELECT rows WHERE workspaceId IN AND startedAt < retentionDate"
DB-->>CBD: rows[]
CBD->>CJ: onBatch(rows)
CJ->>Filter: filterLargeValueKeysWithoutRetainedRefs(keys, deletedLogIds)
loop per unique key
Filter->>DB: SELECT id WHERE id NOT IN deletedLogIds AND position(key in executionData) LIMIT 1
DB-->>Filter: referencingLog?
end
Filter-->>CJ: unreferencedKeys[]
CJ->>S3: deleteFile for each unreferencedKey
CBD->>DB: DELETE WHERE id IN batch ids
end
Reviews (3): Last reviewed commit: "cleanup code" | Re-trigger Greptile |
|
bugbot run |
|
@greptile |
|
bugbot run |
|
@greptile |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit b9ffe66. Configure here.

Summary
Update cleanup logs job to cleanup associated refs.
Also updated cleanup job to use billing helpers from rest of platform.
Type of Change
Testing
Tested manually
Checklist