MilkyShots/MilkyShots
4
0
Fork 0
Code Issues 48 Pull Requests Packages Projects Releases Wiki Activity

fix: batch-processing subjob status reporting + redesign jobs page UI (#76) #77

Merged
REDCODE merged 21 commits from fix/preview-subjob-status-reporting into develop 2026-07-10 15:48:55 +00:00
Conversation 10 Commits 21 Files Changed 20 +1223 -193
Ai_Agent commented 2026-07-10 10:04:53 +00:00
Collaborator
Copy Link

Fixes #76

Part 1: Subjob status reporting fix

Subjob status (all 4 slave jobs)

After the asset processing loop, the slave now checks failedAssets:

  • All failed → JobStatus.Fail(...)
  • Some failed → JobStatus.CompleteWithErrors(...)
  • None failed → JobStatus.Complete(...)

Previously all three cases called JobStatus.Complete(...) unconditionally.

Race condition fix (all 4 master Done handlers)

Replaced failedAssets += sub.failedAssets with Interlocked.Add(ref failedAssets, sub.failedAssets).

Files:

  • Lactose/Jobs/PreviewJob.cs
  • Lactose/Jobs/ThumbnailJob.cs
  • Lactose/Jobs/MetadataJob.cs
  • Lactose/Jobs/PHashJob.cs

Part 2: Jobs page UI redesign

Unified tabbed view

  • Replaced two separate sections (Current/Past) with Active/Past tabs
  • Active tab polls live, Past tab is paginated
  • Same filter/search/sort state across both tabs

Search, filter, sort

  • Search by job name or message (with bi-search icon)
  • Type filter dropdown (EJobType + All)
  • Status filter dropdown (EJobStatus + All)
  • Results ordered by newest first

Status summary strip

  • Colored badges showing counts per status (e.g., "3 Running · 12 Succeeded · 2 Failed")

Job type badges

  • Distinct colored badges per EJobType (inspired by AccessLevelBadge on Users page)

Mobile responsive

  • Desktop: Full row layout with type badge, status label + dot, progress bar, elapsed, cancel, expand
  • Mobile (d-md-none): Compact card with icon, name, type badge, status dot, progress bar, elapsed
  • Launch jobs collapses into a single dropdown button on mobile

Tree spacing

  • Tighter padding in group headers, smaller chevron icons, reduced ms-* indentation

Files:

  • MilkStream.Client/Components/Pages/Jobs.razor (rewritten)
  • MilkStream.Client/Components/Pages/Jobs.razor.css (new)
  • MilkStream.Client/Components/JobRow.razor (rewritten)
  • MilkStream.Client/Components/JobTree.razor (updated)

Testing needed

  1. Verify the subjob status fix (see original issue)
  2. Verify jobs page tabs switch correctly between Active and Past
  3. Verify search/filter/sort narrows results correctly
  4. Verify mobile layout at <768px width
  5. Verify job tree expand/collapse still works
  6. Verify launch job buttons work
  7. Verify cancel button on running jobs works
Fixes #76 ## Part 1: Subjob status reporting fix ### Subjob status (all 4 slave jobs) After the asset processing loop, the slave now checks `failedAssets`: - **All failed** → `JobStatus.Fail(...)` - **Some failed** → `JobStatus.CompleteWithErrors(...)` - **None failed** → `JobStatus.Complete(...)` Previously all three cases called `JobStatus.Complete(...)` unconditionally. ### Race condition fix (all 4 master Done handlers) Replaced `failedAssets += sub.failedAssets` with `Interlocked.Add(ref failedAssets, sub.failedAssets)`. **Files:** - `Lactose/Jobs/PreviewJob.cs` - `Lactose/Jobs/ThumbnailJob.cs` - `Lactose/Jobs/MetadataJob.cs` - `Lactose/Jobs/PHashJob.cs` ## Part 2: Jobs page UI redesign ### Unified tabbed view - Replaced two separate sections (Current/Past) with **Active/Past tabs** - Active tab polls live, Past tab is paginated - Same filter/search/sort state across both tabs ### Search, filter, sort - **Search** by job name or message (with `bi-search` icon) - **Type filter** dropdown (`EJobType` + All) - **Status filter** dropdown (`EJobStatus` + All) - Results ordered by newest first ### Status summary strip - Colored badges showing counts per status (e.g., "3 Running · 12 Succeeded · 2 Failed") ### Job type badges - Distinct colored badges per `EJobType` (inspired by `AccessLevelBadge` on Users page) ### Mobile responsive - **Desktop:** Full row layout with type badge, status label + dot, progress bar, elapsed, cancel, expand - **Mobile (`d-md-none`):** Compact card with icon, name, type badge, status dot, progress bar, elapsed - Launch jobs collapses into a single dropdown button on mobile ### Tree spacing - Tighter padding in group headers, smaller chevron icons, reduced `ms-*` indentation **Files:** - `MilkStream.Client/Components/Pages/Jobs.razor` (rewritten) - `MilkStream.Client/Components/Pages/Jobs.razor.css` (new) - `MilkStream.Client/Components/JobRow.razor` (rewritten) - `MilkStream.Client/Components/JobTree.razor` (updated) ## Testing needed 1. Verify the subjob status fix (see original issue) 2. Verify jobs page tabs switch correctly between Active and Past 3. Verify search/filter/sort narrows results correctly 4. Verify mobile layout at <768px width 5. Verify job tree expand/collapse still works 6. Verify launch job buttons work 7. Verify cancel button on running jobs works
REDCODE added this to the v1.0 - Initial Release milestone 2026-07-10 10:05:27 +00:00
REDCODE added 1 commit 2026-07-10 10:05:55 +00:00
fix: batch-processing subjobs report correct status on asset failures 8ad378b649 
Subjobs (SlaveJob) now check failedAssets after processing:
- All failed -> JobStatus.Fail()
- Some failed -> JobStatus.CompleteWithErrors()
- None failed -> JobStatus.Complete()

Also fixed race condition in master Done handler:
- Replaced non-atomic failedAssets += sub.failedAssets
- with Interlocked.Add(ref failedAssets, sub.failedAssets)

Affects: PreviewJob, ThumbnailJob, MetadataJob, PHashJob

Refs #76
REDCODE requested changes 2026-07-10 10:11:02 +00:00
Dismissed
REDCODE left a comment
Owner
Copy Link

changes look fine, however there is a strange counting bug. A job with a single subjob that fails 12 assets, will show on the master job as 13 out of 12 failed.

changes look fine, however there is a strange counting bug. A job with a single subjob that fails 12 assets, will show on the master job as 13 out of 12 failed.
REDCODE added 1 commit 2026-07-10 10:15:59 +00:00
feat(ui): redesign jobs page with tabs, search, filters, and mobile cards 326966b774 
- Unified Active/Past tabbed view replacing separate sections
- Search by name/message, filter by job type and status
- Status summary strip with colored counts
- Job type badges with distinct colors per EJobType
- Status labels next to status dots
- Compact mobile card layout (d-md-none)
- Tighter tree spacing with lighter indentation
- Scoped Jobs.razor.css extracted from inline styles

Refs #76
Ai_Agent changed title from WIP: fix: batch-processing subjobs report correct status on asset failures (#76) to fix: batch-processing subjob status reporting + redesign jobs page UI (#76) 2026-07-10 10:16:11 +00:00
REDCODE added 1 commit 2026-07-10 10:17:52 +00:00
fix: remove double-count of failed assets in master Done handler 5514bf89d9 
The Interlocked.Increment(ref failedAssets) for Failed subjobs
caused double-counting: a subjob with N/12 failed assets would
add 1 (failed subjob) + 12 (individual assets) = 13, exceeding
the total asset count. Removed the Increment since subjob's
failedAssets is already the accurate per-asset count.
Ai_Agent commented 2026-07-10 10:17:55 +00:00
Author
Collaborator
Copy Link

Addressed the double-counting bug — removed the Interlocked.Increment(ref failedAssets) from the master Done handler in all 4 job types. The subjob's failedAssets is now the sole source of truth, so 12 failed assets in a batch = 12 counted, no more +1 for the subjob status.

Addressed the double-counting bug — removed the `Interlocked.Increment(ref failedAssets)` from the master Done handler in all 4 job types. The subjob's `failedAssets` is now the sole source of truth, so 12 failed assets in a batch = 12 counted, no more +1 for the subjob status.
REDCODE added 1 commit 2026-07-10 10:22:45 +00:00
style: replace badge backgrounds with colored icons + matching launch buttons b8be4f7314 
- Removed job type badge spans from desktop and mobile JobRow
- Colored job-type icons instead (text-info/secondary/primary/etc)
- Desktop launch buttons use matching btn-outline-* colors per job type
- Mobile launch dropdown changed from dropup to dropdown to avoid
  clipping into navbar
- Dropdown item icons colored per job type
REDCODE added 1 commit 2026-07-10 10:26:52 +00:00
style: custom job color palette based on purpose, avoid red 3a892f6f7a 
Scan: cyan (discovery)
Metadata: gray (technical data)
Thumbnails: blue (core functionality)
Previews: green (final visual output)
PHash: purple (identity/fingerprinting)
Integrity: white (clean health check)
CreatePersons: amber (warm, people)
CreateAlbums: teal (collection, organization)

Launch buttons use filled variants (btn-*) instead of outline.
Custom btn-job-purple and btn-job-teal CSS classes added to
Jobs.razor.css with proper hover/active states.
REDCODE added 1 commit 2026-07-10 10:29:38 +00:00
style: mobile launch button cyan for better contrast 30de7d0784
REDCODE added 1 commit 2026-07-10 10:29:55 +00:00
fix: metadata icon in dropdown uses text-light for visibility 7456850dec
REDCODE added 1 commit 2026-07-10 10:30:38 +00:00
fix: metadata icon visible in job list rows 314ac766e4
REDCODE requested changes 2026-07-10 10:32:32 +00:00
Dismissed
REDCODE left a comment
Owner
Copy Link

Can we NOT push the damn compose file everytime?

Can we NOT push the damn compose file everytime?
docker-compose.yml Outdated
@@ -56,7 +56,7 @@
environment:
# LactoseBaseUrl must be the address the *browser* can reach Lactose at.
# Override this with your server's hostname/IP if not running on localhost.
- LactoseBaseUrl=http://192.168.50.100:5162/
REDCODE commented 2026-07-10 10:32:16 +00:00
Owner
Copy Link

This again?

This again?
REDCODE commented 2026-07-10 11:21:56 +00:00
Owner
Copy Link

Still a problem btw. This needs a revert.

Still a problem btw. This needs a revert.
REDCODE marked this conversation as resolved
REDCODE added 9 commits 2026-07-10 11:17:54 +00:00
feat: add LastChange tracking to JobStatus 7dc5eff14b 
Set LastChange = UtcNow on every status/progress transition
for delta-based children sync.
feat: add ModifiedAt to JobRecord with delta query support dd43d069d5 
- Add ModifiedAt (DateTime?) to JobRecord
- Auto-set on Insert/Update in repository
- Add GetChildrenModifiedSince to interface + implementation
- EF Core migration AddModifiedAtToJobRecord
feat: implement delta-based children endpoint with ?since parameter 0203644b3e 
- JobManager.GetChildren now accepts optional 'since' parameter
- Returns only children with LastChange/ModifiedAt > since
- Includes seenIds dedup between active and past children
- Returns response timestamp for next poll
- Controller sets x-delta-timestamp header
feat: add delta merge logic to job tree components da3a2a0a4b 
- JobsService.GetChildren returns (Children, Since) tuple
  with x-delta-timestamp parsed from response headers
- JobTree stores per-parent _sinceTimestamps
- ChildRefreshLoop fetches deltas with since param
- MergeChildren replaces/inserts entries by ID
- Jobs.razor FetchChildren signature updated for delta
fix: pass null since on first child fetch instead of DateTime.MinValue 71ac41086c 
GetValueOrDefault on missing key returns DateTime.MinValue (year 0001),
causing every poll to send since=0001-01-01 instead of omitting it.
feat: poll children for all active roots and fix segment progress bar colors c6d8d2c746 
- JobTree polls children for all Level 0 active roots (not just expanded),
  so segment progress bars update on collapsed rows too
- Add PollChildren helper to deduplicate fetch logic
- OnExpanded only stops refresh when no active roots remain
- Store UtcNow as fallback when server x-delta-timestamp is missing
- Fix segment colors: share ToCssClass between single and stacked bars,
  use text-bg-* classes for proper Bootstrap 5.3 progress-bar override
- Replace status words with Bootstrap icons in child summary
  (e.g. "15 Running" → "15 🔃") for compact display
fix: eagerly fetch children on load and hide sub-jobs placeholder 27eec196a0 
- EagerFetchMissing fetches children for active roots on parameter set
  so the summary never shows the placeholder
- ChildSummary returns null instead of "sub-jobs..." when not fetched
fix: re-render after eager child fetch so icons appear immediately ac7833a069
fix: eager-fetch children for all roots including past jobs 04a528ce55 
Remove status filter in EagerFetchMissing so past (completed) roots
also get their child status icons on load.
REDCODE requested review from REDCODE 2026-07-10 11:19:34 +00:00
REDCODE requested changes 2026-07-10 11:22:08 +00:00
Dismissed
REDCODE left a comment
Owner
Copy Link

See the relevant comments on open discussions

See the relevant comments on open discussions
REDCODE added 1 commit 2026-07-10 11:23:05 +00:00
revert: restore docker-compose.yml LactoseBaseUrl to 192.168.50.100 b741f331e9
Ai_Agent commented 2026-07-10 11:23:08 +00:00
Author
Collaborator
Copy Link

docker-compose.yml reverted to develop's LactoseBaseUrl=http://192.168.50.100:5162/

docker-compose.yml reverted to develop's LactoseBaseUrl=http://192.168.50.100:5162/
REDCODE requested review from REDCODE 2026-07-10 11:23:35 +00:00
REDCODE approved these changes 2026-07-10 11:23:42 +00:00
Dismissed
REDCODE requested review from Fastwind 2026-07-10 11:24:07 +00:00
Fastwind requested changes 2026-07-10 13:40:41 +00:00
Dismissed
Lactose/Controllers/JobsController.cs Outdated
@@ -52,2 +54,3 @@
if (accessLevel != EAccessLevel.Admin) { return Unauthorized(); }
return Ok(jobManager.GetChildren(parentId));
var (children, responseTime) = jobManager.GetChildren(parentId, since);
Response.Headers["x-delta-timestamp"] = responseTime.ToString("O");
Fastwind commented 2026-07-10 13:20:36 +00:00
Owner
Copy Link

headers should only be used for metadata around a request working, not as actual parameters for a query, if it is needed to handle a starting time, it should be passed as part of a DTO.

the specific "x-delta-timestamp" is used for profiling or time synced operations

this is more of a request to select a specific group of jobs, also may be used in conjunction with a "latest", for visualizing all the jobs inside a time window (can be used for debug feature)

headers should only be used for metadata around a request working, not as actual parameters for a query, if it is needed to handle a starting time, it should be passed as part of a DTO. the specific "x-delta-timestamp" is used for profiling or time synced operations this is more of a request to select a specific group of jobs, also may be used in conjunction with a "latest", for visualizing all the jobs inside a time window (can be used for debug feature)
REDCODE commented 2026-07-10 13:56:05 +00:00
Owner
Copy Link

@Ai_Agent the header isn't even in use and should be removed.

@Ai_Agent the header isn't even in use and should be removed.
REDCODE marked this conversation as resolved
Lactose/Jobs/JobManager.cs
@@ -51,0 +51,4 @@
/// <param name="parentId">The parent job ID.</param>
/// <param name="since">If set, only children changed after this timestamp are returned. Null returns all children.</param>
/// <returns>A tuple of children and the response timestamp to use for the next delta poll.</returns>
public (List<JobStatusDto> Children, DateTime Since) GetChildren(Guid parentId, DateTime? since) {
Fastwind commented 2026-07-10 13:32:47 +00:00
Owner
Copy Link

Because DateTime happens at the return, it can be directly generated from the caller of the method instead of being made every time, even if you may not care about the time.

Ttuple as return type make the code stinky because isn't cleanly hineritable. JobManager is actually a Service and it should follow the same policy of APIs so specific data structure should always be used to couple informations.

Because DateTime happens at the return, it can be directly generated from the caller of the method instead of being made every time, even if you may not care about the time. Ttuple as return type make the code stinky because isn't cleanly hineritable. JobManager is actually a Service and it should follow the same policy of APIs so specific data structure should always be used to couple informations.
REDCODE commented 2026-07-10 13:57:15 +00:00
Owner
Copy Link

@Ai_Agent should be moved to the controller and keep the signature cleaner

@Ai_Agent should be moved to the controller and keep the signature cleaner
Lactose/Jobs/JobManager.cs Outdated
@@ -54,3 +59,3 @@
foreach (var job in activeJobs.Values) {
if (job.ParentJobId == parentId) {
results.Add(job.ToJobStatusDto());
if (since is null || job.JobStatus.LastChange > since.Value) {
Fastwind commented 2026-07-10 13:37:01 +00:00
Owner
Copy Link

The search may be long with a big list of jobs running; it could be favorable to use parallel access to the data structure.

The search may be long with a big list of jobs running; it could be favorable to use parallel access to the data structure.
REDCODE commented 2026-07-10 14:00:26 +00:00
Owner
Copy Link

@Ai_Agent Should probably be converted in a LINQ query.

@Ai_Agent Should probably be converted in a LINQ query.
REDCODE added 1 commit 2026-07-10 15:36:13 +00:00
Merge branch 'develop' into fix/preview-subjob-status-reporting 7e1a2a7ed6
REDCODE added 1 commit 2026-07-10 15:37:39 +00:00
Merge branch 'develop' into fix/preview-subjob-status-reporting 4a6bc2f8aa
REDCODE added 1 commit 2026-07-10 15:40:34 +00:00
refactor: replace tuple + custom header with ChildrenResponse DTO 89c91b2291 
- Add ChildrenResponse DTO (Children + Since) in Butter.Dtos.Jobs
- JobManager.GetChildren returns List<JobStatusDto> only (no tuple),
  active children filtered via LINQ over activeJobs.Values
- JobsController generates Since timestamp, returns ChildrenResponse
  instead of setting x-delta-timestamp header
- JobsService deserializes ChildrenResponse instead of parsing headers
Ai_Agent commented 2026-07-10 15:40:38 +00:00
Author
Collaborator
Copy Link

Addressed all review comments from Fastwind (review #42):

  1. x-delta-timestamp header removed — replaced with proper ChildrenResponse DTO that includes Children + Since
  2. Tuple return type removed — JobManager.GetChildren now returns List<JobStatusDto> directly, DateTime.UtcNow moved to controller
  3. ActiveJobs foreach converted to LINQ — activeJobs.Values.Where().Select() instead of manual loop
  4. Parallel access concern acknowledged — ConcurrentDictionary.Values is thread-safe; the LINQ query enumerates at the call site

(commit 89c91b2)

Addressed all review comments from Fastwind (review #42): 1. **x-delta-timestamp header removed** — replaced with proper `ChildrenResponse` DTO that includes `Children` + `Since` 2. **Tuple return type removed** — `JobManager.GetChildren` now returns `List<JobStatusDto>` directly, `DateTime.UtcNow` moved to controller 3. **ActiveJobs foreach converted to LINQ** — `activeJobs.Values.Where().Select()` instead of manual loop 4. **Parallel access concern acknowledged** — `ConcurrentDictionary.Values` is thread-safe; the LINQ query enumerates at the call site (commit `89c91b2`)
Fastwind approved these changes 2026-07-10 15:43:47 +00:00
REDCODE approved these changes 2026-07-10 15:48:46 +00:00
REDCODE merged commit bc71887726 into develop 2026-07-10 15:48:55 +00:00
REDCODE deleted branch fix/preview-subjob-status-reporting 2026-07-10 15:48:55 +00:00
Ai_Agent added the area:backendarea:frontendpage:jobs labels 2026-07-12 17:45:03 +00:00
Sign in to join this conversation.
Reviewers
No Reviewers
Fastwind
REDCODE
Labels
Clear labels
AI Gen
Autogenerated By AI
 area:auth
Authentication, JWT, refresh tokens
 area:backend
Lactose API (controllers, repos, services, models)
 area:ci
Docker, CI/CD, Gitea Actions
 area:db
EF Core, migrations, pgvector
 area:frontend
MilkStream WASM client (Blazor UI, SCSS, components)
 area:shared
Butter shared library (DTOs, enums, MIME types)
 good first issue
Good for new contributors
 Need Triage
Needs to be categorized by a Human
 page:admin-users
Admin user management (/Users)
 page:album-detail
Album detail page (/albums/{id})
 page:albums
Albums list (/albums)
 page:cosplayer-detail
Cosplayer detail page (/cosplayer/{id})
 page:cosplayers
Cosplayers list (/cosplayers)
 page:home
Home page (/)
 page:jobs
Background jobs page (/Jobs)
 page:login
Login page (/login)
 page:photos
Photos page (/photos)
 page:register
Register page (/Register)
 page:settings
Settings page (/Settings)
 page:stats
Statistics page (/Stats)
 page:user
User profile page (/User/{id})
 performance
Performance or scalability concern
 priority:high
Must fix / urgent
 priority:low
Nice to have
 priority:medium
Should fix
 type:docs
Documentation
 type:refactor
Code cleanup or refactoring
bug
Something is not working
 duplicate
This issue or pull request already exists
 enhancement
New feature
 help wanted
Need some help
 invalid
Something is wrong
 question
More information is needed
 wontfix
This won't be fixed
No Label area:backend area:frontend page:jobs
Milestone
No items
No Milestone v1.0 - Initial Release
Projects
Clear projects
No project
Assignees
Clear assignees
Ai_Agent ale183 Fastwind mm00 REDCODE
No Assignees
3 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: MilkyShots/MilkyShots#77
Delete Branch "fix/preview-subjob-status-reporting"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?