quintodrome/ui/src/layout/ActivityPanel.jsx
Deluan Quintão 28d5299ffc
feat(scanner): implement selective folder scanning and file system watcher improvements (#4674)
* feat: Add selective folder scanning capability

Implement targeted scanning of specific library/folder pairs without
full recursion. This enables efficient rescanning of individual folders
when changes are detected, significantly reducing scan time for large
libraries.

Key changes:
- Add ScanTarget struct and ScanFolders API to Scanner interface
- Implement CLI flag --targets for specifying libraryID:folderPath pairs
- Add FolderRepository.GetByPaths() for batch folder info retrieval
- Create loadSpecificFolders() for non-recursive directory loading
- Scope GC operations to affected libraries only (with TODO for full impl)
- Add comprehensive tests for selective scanning behavior

The selective scan:
- Only processes specified folders (no subdirectory recursion)
- Maintains library isolation
- Runs full maintenance pipeline scoped to affected libraries
- Supports both full and quick scan modes

Examples:
  navidrome scan --targets "1:Music/Rock,1:Music/Jazz"
  navidrome scan --full --targets "2:Classical"

* feat(folder): replace GetByPaths with GetFolderUpdateInfo for improved folder updates retrieval

Signed-off-by: Deluan <deluan@navidrome.org>

* test: update parseTargets test to handle folder names with spaces

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(folder): remove unused LibraryPath struct and update GC logging message

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(folder): enhance external scanner to support target-specific scanning

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(scanner): simplify scanner methods

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(watcher): implement folder scanning notifications with deduplication

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(watcher): add resolveFolderPath function for testability

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(watcher): implement path ignoring based on .ndignore patterns

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(scanner): implement IgnoreChecker for managing .ndignore patterns

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(ignore_checker): rename scanner to lineScanner for clarity

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(scanner): enhance ScanTarget struct with String method for better target representation

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scanner): validate library ID to prevent negative values

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(scanner): simplify GC method by removing library ID parameter

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(scanner): update folder scanning to include all descendants of specified folders

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): allow selective scan in the /startScan endpoint

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(scanner): update CallScan to handle specific library/folder pairs

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(scanner): streamline scanning logic by removing scanAll method

Signed-off-by: Deluan <deluan@navidrome.org>

* test: enhance mockScanner for thread safety and improve test reliability

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(scanner): move scanner.ScanTarget to model.ScanTarget

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: move scanner types to model,implement MockScanner

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(scanner): update scanner interface and implementations to use model.Scanner

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(folder_repository): normalize target path handling by using filepath.Clean

Signed-off-by: Deluan <deluan@navidrome.org>

* test(folder_repository): add comprehensive tests for folder retrieval and child exclusion

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(scanner): simplify selective scan logic using slice.Filter

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(scanner): streamline phase folder and album creation by removing unnecessary library parameter

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(scanner): move initialization logic from phase_1 to the scanner itself

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(tests): rename selective scan test file to scanner_selective_test.go

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(configuration): add DevSelectiveWatcher configuration option

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(watcher): enhance .ndignore handling for folder deletions and file changes

Signed-off-by: Deluan <deluan@navidrome.org>

* docs(scanner): comments

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(scanner): enhance walkDirTree to support target folder scanning

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scanner, watcher): handle errors when pushing ignore patterns for folders

Signed-off-by: Deluan <deluan@navidrome.org>

* Update scanner/phase_1_folders.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* refactor(scanner): replace parseTargets function with direct call to scanner.ParseTargets

Signed-off-by: Deluan <deluan@navidrome.org>

* test(scanner): add tests for ScanBegin and ScanEnd functionality

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(library): update PRAGMA optimize to check table sizes without ANALYZE

Signed-off-by: Deluan <deluan@navidrome.org>

* test(scanner): refactor tests

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(ui): add selective scan options and update translations

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(ui): add quick and full scan options for individual libraries

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(ui): add Scan buttonsto the LibraryList

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(scan): update scanning parameters from 'path' to 'target' for selective scans.

* refactor(scan): move ParseTargets function to model package

* test(scan): suppress unused return value from SetUserLibraries in tests

* feat(gc): enhance garbage collection to support selective library purging

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scanner): prevent race condition when scanning deleted folders

When the watcher detects changes in a folder that gets deleted before
the scanner runs (due to the 10-second delay), the scanner was
prematurely removing these folders from the tracking map, preventing
them from being marked as missing.

The issue occurred because `newFolderEntry` was calling `popLastUpdate`
before verifying the folder actually exists on the filesystem.

Changes:
- Move fs.Stat check before newFolderEntry creation in loadDir to
  ensure deleted folders remain in lastUpdates for finalize() to handle
- Add early existence check in walkDirTree to skip non-existent target
  folders with a warning log
- Add unit test verifying non-existent folders aren't removed from
  lastUpdates prematurely
- Add integration test for deleted folder scenario with ScanFolders

Fixes the issue where deleting entire folders (e.g., /music/AC_DC)
wouldn't mark tracks as missing when using selective folder scanning.

* refactor(scan): streamline folder entry creation and update handling

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(scan): add '@Recycle' (QNAP) to ignored directories list

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(log): improve thread safety in logging level management

* test(scan): move unit tests for ParseTargets function

Signed-off-by: Deluan <deluan@navidrome.org>

* review

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: deluan <deluan.quintao@mechanical-orchard.com>
2025-11-14 22:15:43 -05:00

231 lines
6.7 KiB
JavaScript

import React, { useState, useEffect } from 'react'
import { useSelector } from 'react-redux'
import { useNotify, useTranslate } from 'react-admin'
import {
Popover,
CircularProgress,
IconButton,
makeStyles,
Tooltip,
Card,
CardContent,
CardActions,
Divider,
Box,
Typography,
} from '@material-ui/core'
import { FiActivity } from 'react-icons/fi'
import { BiError } from 'react-icons/bi'
import { VscSync } from 'react-icons/vsc'
import { GiMagnifyingGlass } from 'react-icons/gi'
import subsonic from '../subsonic'
import { useInitialScanStatus } from './useInitialScanStatus'
import { useInterval } from '../common'
import { useScanElapsedTime } from './useScanElapsedTime'
import { formatDuration, formatShortDuration } from '../utils'
import config from '../config'
const useStyles = makeStyles((theme) => ({
wrapper: {
position: 'relative',
color: (props) => (props.up ? null : 'orange'),
},
progress: {
color: theme.palette.primary.light,
position: 'absolute',
top: 10,
left: 10,
zIndex: 1,
},
button: {
color: 'inherit',
zIndex: 2,
},
counterStatus: {
minWidth: '20em',
},
error: {
color: theme.palette.error.main,
},
card: {
maxWidth: 'none',
},
cardContent: {
padding: theme.spacing(2, 3),
},
}))
const getUptime = (serverStart) =>
formatDuration((Date.now() - serverStart.startTime) / 1000)
const Uptime = () => {
const serverStart = useSelector((state) => state.activity.serverStart)
const [uptime, setUptime] = useState(getUptime(serverStart))
useInterval(() => {
setUptime(getUptime(serverStart))
}, 1000)
return <span>{uptime}</span>
}
const ActivityPanel = () => {
const serverStart = useSelector((state) => state.activity.serverStart)
const up = serverStart.startTime
const scanStatus = useSelector((state) => state.activity.scanStatus)
const elapsed = useScanElapsedTime(
scanStatus.scanning,
scanStatus.elapsedTime,
)
const [acknowledgedError, setAcknowledgedError] = useState(null)
const isErrorVisible =
scanStatus.error && scanStatus.error !== acknowledgedError
const classes = useStyles({
up: up && (!scanStatus.error || !isErrorVisible),
})
const translate = useTranslate()
const notify = useNotify()
const [anchorEl, setAnchorEl] = useState(null)
const open = Boolean(anchorEl)
useInitialScanStatus()
const handleMenuOpen = (event) => {
if (scanStatus.error) {
setAcknowledgedError(scanStatus.error)
}
setAnchorEl(event.currentTarget)
}
const handleMenuClose = () => setAnchorEl(null)
const triggerScan = (full) => () => subsonic.startScan({ fullScan: full })
useEffect(() => {
if (serverStart.version && serverStart.version !== config.version) {
notify('ra.notification.new_version', 'info', {}, false, 604800000 * 50)
}
}, [serverStart, notify])
const tooltipTitle = scanStatus.error
? `${translate('activity.status')}: ${scanStatus.error}`
: translate('activity.title')
const lastScanType = (() => {
switch (scanStatus.scanType) {
case 'full':
return translate('activity.fullScan')
case 'quick':
return translate('activity.quickScan')
case 'full-selective':
case 'quick-selective':
return translate('activity.selectiveScan')
default:
return ''
}
})()
return (
<div className={classes.wrapper}>
<Tooltip title={tooltipTitle}>
<IconButton className={classes.button} onClick={handleMenuOpen}>
{!up || isErrorVisible ? (
<BiError data-testid="activity-error-icon" size={'20'} />
) : (
<FiActivity data-testid="activity-ok-icon" size={'20'} />
)}
</IconButton>
</Tooltip>
{scanStatus.scanning && (
<CircularProgress size={24} className={classes.progress} />
)}
<Popover
id="panel-activity"
anchorEl={anchorEl}
anchorOrigin={{
vertical: 'bottom',
horizontal: 'right',
}}
transformOrigin={{
vertical: 'top',
horizontal: 'right',
}}
open={open}
onClose={handleMenuClose}
>
<Card className={classes.card}>
<CardContent className={classes.cardContent}>
<Box display="flex" className={classes.counterStatus}>
<Box component="span" flex={2}>
{translate('activity.serverUptime')}:
</Box>
<Box component="span" flex={1}>
{up ? <Uptime /> : translate('activity.serverDown')}
</Box>
</Box>
</CardContent>
<Divider />
<CardContent className={classes.cardContent}>
<Box display="flex" className={classes.counterStatus}>
<Box component="span" flex={2}>
{translate('activity.totalScanned')}:
</Box>
<Box component="span" flex={1}>
{scanStatus.folderCount || '-'}
</Box>
</Box>
<Box display="flex" className={classes.counterStatus} mt={2}>
<Box component="span" flex={2}>
{translate('activity.scanType')}:
</Box>
<Box component="span" flex={1}>
{lastScanType}
</Box>
</Box>
<Box display="flex" className={classes.counterStatus} mt={2}>
<Box component="span" flex={2}>
{translate('activity.elapsedTime')}:
</Box>
<Box component="span" flex={1}>
{formatShortDuration(elapsed)}
</Box>
</Box>
{scanStatus.error && (
<Box
display="flex"
flexDirection="column"
mt={2}
className={classes.error}
>
<Typography variant="subtitle2">
{translate('activity.status')}:
</Typography>
<Typography variant="body2">{scanStatus.error}</Typography>
</Box>
)}
</CardContent>
<Divider />
<CardActions>
<Tooltip title={translate('activity.quickScan')}>
<IconButton
onClick={triggerScan(false)}
disabled={scanStatus.scanning}
>
<VscSync />
</IconButton>
</Tooltip>
<Tooltip title={translate('activity.fullScan')}>
<IconButton
onClick={triggerScan(true)}
disabled={scanStatus.scanning}
>
<GiMagnifyingGlass />
</IconButton>
</Tooltip>
</CardActions>
</Card>
</Popover>
</div>
)
}
export default ActivityPanel