oceans2alaska
610a3bb447
- Add privacy projection (core/privacy.py), private report storage, bounded
process capture; omit identities, free text and unapproved paths
- Mask remote peer IPs: only loopback/host/wildcard stay numeric, others are
reduced to remote_scope private/public (schema + scoring updated)
- Map ss TCP states to canonical names; ASCII-only numeric fields; fix
"Microsoft Visual C++" product matching (Python and PowerShell)
- Merge 7fed3b9: -Diagnostic switch, serialization exit code 3, PS7 module
path, UI error detail; keep hardened task/certificate collectors
- Add audit docs, privacy docs and regression tests
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
129 lines
6.8 KiB
Markdown
129 lines
6.8 KiB
Markdown
> **Superseded in part:** the privacy hardening (see `PRIVACY_HARDENING.md`) disables certificate
|
|
> enumeration and narrows task collection to name/path/state. The task and certificate
|
|
> collector behavior described below no longer applies; the diagnostic switch, exit-code 3
|
|
> serialization handling and PowerShell 7 module path remain in effect.
|
|
|
|
# Windows collector reliability validation — 2026-10-30
|
|
|
|
## Root causes
|
|
|
|
The supplied Server 2025 assessment reported tasks failed and 16 certificate
|
|
errors, but its generic errors did not retain exception types. Read-only native
|
|
reproduction established two defects:
|
|
|
|
* Tasks: one action executable made .NET Framework `Path.GetFileName` throw a
|
|
`MethodInvocationException` wrapping `ArgumentException`. The projection was
|
|
inside a single assignment/query catch, so that one value discarded all tasks.
|
|
* Certificates: the Windows certificate provider returned store `PSPath` values
|
|
such as `Microsoft.PowerShell.Security\Certificate::My`, without LocalMachine.
|
|
Using those paths produced `ItemNotFoundException` / `ObjectNotFound` for each
|
|
store. Store-list success alone incorrectly gave partial credit when every
|
|
store query failed.
|
|
|
|
The original code already enumerated stores separately; recursion was not the
|
|
cause. Missing modules and JSON conversion were checked as failure paths, but
|
|
were not the reproduced causes on this host.
|
|
|
|
## Changes
|
|
|
|
Executable basenames now use string operations and the existing strict allowlist,
|
|
avoiding filesystem path validation. Malformed values become an omission marker;
|
|
task records remain visible. Each task is projected and JSON-checked separately.
|
|
Principal group accounts are used when UserId is absent. Arguments, XML, inline
|
|
scripts, event queries and credentials are never projected.
|
|
|
|
Certificates use explicit `Cert:\LocalMachine\<store>` paths. Each store query
|
|
and certificate projection is isolated; empty successful stores receive credit,
|
|
inaccessible stores produce errors, and listing names alone does not give credit
|
|
when all listed stores fail. Only subject, issuer, thumbprint, expiration and
|
|
store are collected. No private key or certificate export APIs are used.
|
|
|
|
Read helpers promote nonterminating errors to terminating errors. Per-record JSON
|
|
checks reject unrepresentable records. Final JSON failures exit with code 3 and
|
|
Python records `serialization_failed`, never a successful empty collection.
|
|
Existing subprocess timeout, UTF-8, output limits and schema validation remain.
|
|
|
|
Optional `-Diagnostic` / Python `--diagnostic` adds exception class and category to
|
|
existing sanitized error messages. Fixed command stages appear in error scope.
|
|
No raw messages, task names, arguments, store names, targets, stack traces or
|
|
error identifiers are included in diagnostics. Inventory remains sensitive and
|
|
should be stored privately. Task/certificate UI detail pages display these error
|
|
scopes, codes and messages; Collection checks and reports retain their contract.
|
|
|
|
The production Python backend deliberately continues to use trusted Windows
|
|
PowerShell 5.1. Direct PowerShell 7 invocation also includes the inbox Windows
|
|
module directory, without opening module lookup to user directories. Commands
|
|
autoload from those inbox module paths; missing commands are explicit failures.
|
|
No scoring policy, Linux collector or other inventory module was changed.
|
|
|
|
## Results
|
|
|
|
On the local Windows Server 2025 host, native 5.1 collection returned:
|
|
|
|
| Collector | Status | Records | Errors |
|
|
| --- | --- | ---: | ---: |
|
|
| Tasks | complete | 192 (46 disabled) | 0 |
|
|
| Certificates | complete | 49 across 7 populated stores | 0 |
|
|
|
|
Full Python subprocess collection passed schema validation, returned the same
|
|
record counts, and completed all nine checks. Evidence coverage rose from the
|
|
supplied assessment's 81% to 100%. Confidence remained moderate under the existing
|
|
classification policy; complete collection does not establish retirement safety.
|
|
|
|
`python -m unittest discover -s tests -v`: **64 tests, 62 passed, 2 skipped**.
|
|
Skips: the Linux-only subprocess test on Windows and PowerShell 7 (not installed).
|
|
UI navigation/history/export tests, Linux parsing tests, scoring regressions,
|
|
Windows subprocess limits and schema tests passed. Native PowerShell AST and
|
|
synthetic tests passed under 5.1. Added cases cover normal/disabled/empty tasks,
|
|
malformed actions, per-task failures, nonterminating query denial, multiple and
|
|
empty stores, partial/all-store denial, redacted diagnostics and serialization
|
|
failures. A dedicated regression checks lower coverage/confidence without risk
|
|
inflation for failed tasks and partial certificates.
|
|
|
|
## Exact validation commands
|
|
|
|
Run from the repository root in an authorized elevated Windows PowerShell console.
|
|
These commands only read inventory; they do not change Server configuration or
|
|
execution policy. Choose a new output path for each saved inventory.
|
|
|
|
```powershell
|
|
$PSVersionTable.PSVersion
|
|
powershell.exe -NoLogo -NoProfile -NonInteractive -File .\tests\powershell\Test-Collectors.ps1
|
|
python -m unittest discover -s tests -v
|
|
|
|
foreach ($section in @('tasks', 'certificates')) {
|
|
$raw = & powershell.exe -NoLogo -NoProfile -NonInteractive -File .\powershell\Get-ServerInventory.ps1 -Section $section -Diagnostic
|
|
if ($LASTEXITCODE -ne 0) { throw 'Collector process failed' }
|
|
$r = $raw | ConvertFrom-Json
|
|
[pscustomobject]@{
|
|
Section = $section
|
|
Status = $r.status
|
|
Records = @($r.data).Count
|
|
Errors = $r.errors
|
|
} | ConvertTo-Json -Depth 5
|
|
if ($r.status -ne 'complete') { throw 'Review incomplete collection' }
|
|
if (@($r.data).Count -eq 0) { throw 'No records: verify actual host contents independently' }
|
|
}
|
|
|
|
python -m samsara.collectors.windows --diagnostic --output .\windows-validation-new.json
|
|
python .\tools\validate_windows_host.py --output-dir .\windows-validation-new
|
|
|
|
# When PowerShell 7 is already installed:
|
|
pwsh.exe -NoLogo -NoProfile -NonInteractive -File .\tests\powershell\Test-Collectors.ps1
|
|
pwsh.exe -NoLogo -NoProfile -NonInteractive -File .\powershell\Get-ServerInventory.ps1 -Section tasks -Diagnostic
|
|
pwsh.exe -NoLogo -NoProfile -NonInteractive -File .\powershell\Get-ServerInventory.ps1 -Section certificates -Diagnostic
|
|
```
|
|
|
|
The nonempty assertion above is a validation expectation for this host, not a
|
|
collector rule: legitimately empty successful queries remain complete.
|
|
|
|
## Remaining limitations
|
|
|
|
PowerShell 7 runtime execution and other Server releases were not available for
|
|
live validation. Tests run under each runtime when installed; 5.1 syntax and
|
|
Server 2016/2019/2022/2025 identity fixtures pass. Access restrictions may hide
|
|
objects even when an API reports success. A terminating failure of the initial
|
|
task enumeration cannot recover task registrations the API never returned, and
|
|
is reported failed. Inventory is a current observation, not historical activity.
|
|
No Windows Server configuration was modified.
|