Implements the scan-infrastructure work tracked in #183, #184, #185, #186 and #187 (all in milestone v1.1).
Summary
Reworks the file-system crawl and person/album discovery jobs so unchanged data is never re-scanned, exposes a "deep" scan mode and folder selection, and merges Create Persons into a single discovery job.
New DirectoryScanState table stores each directory's last-known mtime. The crawl always descends (directory mtime only reflects direct changes), but only enumerates files and reconciles the DB for directories whose mtime changed.
Batched inserts/updates/soft-deletes per directory (no more one Save() per file). Uses a dedicated asset context so the change tracker can be cleared per directory without detaching scan-state rows.
Deletion/rename sweep runs for changed directories.
New partial index IX_Assets_FolderId_Id on (FolderId, Id) filtered on DeletedAt IS NULL powers keyset pagination.
CreateAlbumsJob is now the single "Create Persons & Albums" job and also covers person-only folders. CreatePersonsJob is deleted (scheduler method, controller case, Jobs page button removed; EJobType.CreatePersons kept so past JobRecords still deserialize).
Per-folder cursor Folder.LastDiscoveryScanAt; repeat runs scan only new assets. Auto-deep rescan when the folder's regex pattern changes (LastDiscoveryScannedRegex).
Keyset pagination over the new index, parallel regex matching per batch.
Single-pass assignment via a temporary AssetAlbumStaging table and bulk INSERT INTO "AlbumAsset" ... ON CONFLICT DO NOTHING (removes the second full scan and per-batch EXISTS queries).
Cover backfill via one grouped query instead of N+1.
JobRequestDto.Parameters conventions: literal deep and repeatable folder=<guid>. Applied to FileSystemScan and CreateAlbums; controller validates and scheduler/controller wired accordingly.
Frontend
Jobs page launch modal for Scan Files / Create Persons & Albums with a folder checkbox list and an Incremental/Deep toggle.
Settings → Folders: per-folder Scan (incremental) and Deep scan buttons.
JobsService.StartJob accepts parameters.
Migration
AddIncrementalScanState adds the Folder cursor columns, DirectoryScanState + AssetAlbumStaging tables, and the IX_Assets_FolderId_Id partial index (replacing the redundant IX_Assets_FolderId FK index).
Testing
Full solution builds clean (dotnet build MilkyShots.sln).
Manual verification pending: run incremental + deep scans and discovery on a live folder, confirm unchanged directories are skipped and repeat runs process only new assets.
Implements the scan-infrastructure work tracked in #183, #184, #185, #186 and #187 (all in milestone v1.1).
## Summary
Reworks the file-system crawl and person/album discovery jobs so unchanged data is never re-scanned, exposes a "deep" scan mode and folder selection, and merges Create Persons into a single discovery job.
### Backend
- **Incremental file-system scans** (#183, #184)
- New `DirectoryScanState` table stores each directory's last-known mtime. The crawl always descends (directory mtime only reflects direct changes), but only enumerates files and reconciles the DB for directories whose mtime changed.
- Batched inserts/updates/soft-deletes per directory (no more one `Save()` per file). Uses a dedicated asset context so the change tracker can be cleared per directory without detaching scan-state rows.
- Deletion/rename sweep runs for changed directories.
- New partial index `IX_Assets_FolderId_Id` on `(FolderId, Id)` filtered on `DeletedAt IS NULL` powers keyset pagination.
- **Merged, incremental discovery** (#187)
- `CreateAlbumsJob` is now the single "Create Persons & Albums" job and also covers person-only folders. `CreatePersonsJob` is deleted (scheduler method, controller case, Jobs page button removed; `EJobType.CreatePersons` kept so past `JobRecords` still deserialize).
- Per-folder cursor `Folder.LastDiscoveryScanAt`; repeat runs scan only new assets. Auto-deep rescan when the folder's regex pattern changes (`LastDiscoveryScannedRegex`).
- Keyset pagination over the new index, parallel regex matching per batch.
- Single-pass assignment via a temporary `AssetAlbumStaging` table and bulk `INSERT INTO "AlbumAsset" ... ON CONFLICT DO NOTHING` (removes the second full scan and per-batch EXISTS queries).
- Cover backfill via one grouped query instead of N+1.
- **Deep scan + folder selection** (#185, #186)
- `JobRequestDto.Parameters` conventions: literal `deep` and repeatable `folder=<guid>`. Applied to `FileSystemScan` and `CreateAlbums`; controller validates and scheduler/controller wired accordingly.
### Frontend
- Jobs page launch modal for Scan Files / Create Persons & Albums with a folder checkbox list and an Incremental/Deep toggle.
- Settings → Folders: per-folder Scan (incremental) and Deep scan buttons.
- `JobsService.StartJob` accepts parameters.
## Migration
`AddIncrementalScanState` adds the `Folder` cursor columns, `DirectoryScanState` + `AssetAlbumStaging` tables, and the `IX_Assets_FolderId_Id` partial index (replacing the redundant `IX_Assets_FolderId` FK index).
## Testing
- Full solution builds clean (`dotnet build MilkyShots.sln`).
- Manual verification pending: run incremental + deep scans and discovery on a live folder, confirm unchanged directories are skipped and repeat runs process only new assets.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Implements the scan-infrastructure work tracked in #183, #184, #185, #186 and #187 (all in milestone v1.1).
Summary
Reworks the file-system crawl and person/album discovery jobs so unchanged data is never re-scanned, exposes a "deep" scan mode and folder selection, and merges Create Persons into a single discovery job.
Backend
DirectoryScanStatetable stores each directory's last-known mtime. The crawl always descends (directory mtime only reflects direct changes), but only enumerates files and reconciles the DB for directories whose mtime changed.Save()per file). Uses a dedicated asset context so the change tracker can be cleared per directory without detaching scan-state rows.IX_Assets_FolderId_Idon(FolderId, Id)filtered onDeletedAt IS NULLpowers keyset pagination.CreateAlbumsJobis now the single "Create Persons & Albums" job and also covers person-only folders.CreatePersonsJobis deleted (scheduler method, controller case, Jobs page button removed;EJobType.CreatePersonskept so pastJobRecordsstill deserialize).Folder.LastDiscoveryScanAt; repeat runs scan only new assets. Auto-deep rescan when the folder's regex pattern changes (LastDiscoveryScannedRegex).AssetAlbumStagingtable and bulkINSERT INTO "AlbumAsset" ... ON CONFLICT DO NOTHING(removes the second full scan and per-batch EXISTS queries).JobRequestDto.Parametersconventions: literaldeepand repeatablefolder=<guid>. Applied toFileSystemScanandCreateAlbums; controller validates and scheduler/controller wired accordingly.Frontend
JobsService.StartJobaccepts parameters.Migration
AddIncrementalScanStateadds theFoldercursor columns,DirectoryScanState+AssetAlbumStagingtables, and theIX_Assets_FolderId_Idpartial index (replacing the redundantIX_Assets_FolderIdFK index).Testing
dotnet build MilkyShots.sln).