oceans2alaska/Samsara
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Privacy hardening, remote-peer IP masking and review fixes

Browse Source 
- Add privacy projection (core/privacy.py), private report storage, bounded
  process capture; omit identities, free text and unapproved paths
- Mask remote peer IPs: only loopback/host/wildcard stay numeric, others are
  reduced to remote_scope private/public (schema + scoring updated)
- Map ss TCP states to canonical names; ASCII-only numeric fields; fix
  "Microsoft Visual C++" product matching (Python and PowerShell)
- Merge 7fed3b9: -Diagnostic switch, serialization exit code 3, PS7 module
  path, UI error detail; keep hardened task/certificate collectors
- Add audit docs, privacy docs and regression tests

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Not Dave
2026-11-04 18:50:24 -08:00
parent 7fed3b9ab9
commit 610a3bb447
51 changed files with 2719 additions and 604 deletions
Download Patch File Download Diff File Expand all files Collapse all files

253
README.md
View File

@@ -1,10 +1,18 @@
# Samsara
Source repository: [oceans2alaska/Samsara](https://git.doorstodurin.com/oceans2alaska/Samsara).
[Source repository](https://git.doorstodurin.com/oceans2alaska/Samsara).
Windows Server and Red Hat Enterprise Linux retirement assessment using Python 3.11+
and PySide6. Inventory, dependency evidence and heuristic retirement risk remain
separate from historical usage validation and administrator retirement approval.
Windows Server and Red Hat Enterprise Linux retirement assessment. Collect inventory, review dependencies and risks, and save evidence-based reports. Python 3.11+ and PySide6.
**Privacy hardening is implemented, but production approval is pending.** This is
not a HIPAA compliance or PHI-free attestation. Read
[privacy controls, configuration and acceptance gates](docs/PRIVACY_HARDENING.md)
before collecting real inventory. Historical audit documents remain in `audit/`.
## Get started
## Desktop setup
From the project root on a Linux desktop:
```bash
python3 -m venv .venv
@@ -13,135 +21,184 @@ python -m pip install -r requirements.txt
python -m samsara.main
```
On Windows, create the environment with `py -3 -m venv .venv` and activate `.venv\Scripts\activate`. Run from the project root in a desktop session.
On Windows, use `py -3 -m venv .venv`, then
`.venv\Scripts\python.exe -m pip install -r requirements.txt` and
`.venv\Scripts\python.exe -m samsara.main`.
Linux desktop compatibility includes Mint. Debian-based desktops may need `python3-venv libegl1 libopengl0 libxcb-cursor0 libxkbcommon-x11-0` installed through their package manager. If Qt reports a missing xcb cursor library, install the package providing `libxcb-cursor.so.0`.
Desktop dependencies are pinned to `PySide6_Essentials` (Qt Core/Gui/Widgets) and
`jsonschema`; WebEngine, PDF and Qt Addons are not required. The existing development
environment still includes the former umbrella packages; requirements changes do
not uninstall them. `constraints-tested-linux.txt` records observed versions, not
a complete platform/hash lock or vulnerability clearance. Obtain packages using
approved provisioning; installation normally requires external downloads.
## Analyze
Linux desktop compatibility includes Mint. Debian-based desktops may need
`python3-venv libegl1 libopengl0 libxcb-cursor0 libxkbcommon-x11-0` through their
package manager. Run the GUI in a graphical desktop session.
1. Open **Analyze**. Leave **Assessment source → This computer** and **Operating system → Auto-detect** selected.
2. Click **Analyze**. The app identifies the local OS, runs its available read-only collector and prepares the assessment report.
3. The **Summary** page opens automatically with a retirement risk estimate, OS type/version, important software and enabled scheduled jobs/scripts. Higher scores mean more retirement concerns; the score is not retirement approval.
4. Open detailed tabs for **Networking**, **Software**, **Tasks**, **Dependencies**, **Services**, **Storage**, **Roles**, **Certificates**, **Risk findings**, **Collection checks** and **System**. Tables are searchable and sortable.
5. Click **Save HTML report**, or use **Reports** to export HTML/JSON. **History** reopens assessments from this session.
## Assessment workflow
Explicit profiles include Windows Server 2016, 2019, 2022 and 2025; RHEL 7 (legacy), 8, 9 and 10; and RHEL-compatible Linux. A local profile must match the detected OS. Auto-detect also allows local testing on other Linux distributions. Profile choices do not imply that every live collector is implemented or every release is validated.
1. Choose **Analyze → This computer → Auto-detect → Analyze** to assess the execution
host. Explicit profiles include Server 2016/2019/2022/2025 and RHEL 7 (legacy),
8/9/10 or compatible distributions. A selected profile must match the host.
2. Review **Summary**: OS/version, known software families, scheduled-work state,
retirement risk, evidence coverage and confidence. A low score is not clearance.
3. Use searchable/sortable detailed tabs: Networking, Software, Tasks, Dependencies,
Services, Storage & shares, Roles, Certificates, Risk findings, Collection checks
and System. Omitted fields and missing scopes require manual verification.
4. Save minimized HTML or JSON directly inside the configured private local report
directory. New filenames only. HTML creates no raw inventory sidecar.
5. **History** retains up to ten minimized assessments in the session by default.
**Clear session assessments** releases application references; it does not securely
erase memory or delete exports.
Choose **Saved inventory file** to analyze existing JSON; the recorded OS is displayed without probing another computer. **Windows Server sample** and **RHEL sample** provide clearly labeled synthetic examples. No synthetic results are loaded into history automatically.
**Saved inventory file** imports bounded local JSON and recomputes the assessment.
Source claims and old findings are not trusted. Legacy input enters memory before
filtering: do not intentionally import known patient-bearing files to redact them.
**Windows Server sample** and **RHEL sample** are clearly synthetic; they never scan
another host or populate history automatically.
## Windows Server assessments: local collection
## Privacy defaults and local output
Local Windows collection is implemented for the Server 2016/2019/2022/2025 release builds. **Acceptance is pending a successful real inventory on an authorized Windows Server.** Development was performed on Linux, without a PowerShell runtime. Synthetic tests are not Windows compatibility certification.
`privacy_policy.json` under `samsara/assets/` is an administrator-reviewed deployment
configuration. Defaults omit host/domain/account identities, custom free-text names,
certificate identities and all paths. A finite catalogue retains recognizable
application families. States, PIDs, ports, numeric IPs, OS versions, capacities and
counts remain useful, but are still confidential infrastructure metadata.
Use an existing authorized account on the Windows Server being assessed. The collector runs locally; a Windows 10/11 desktop cannot scan another server in this phase. No WinRM, SSH, agent installation, domain administrator account or saved credentials are required. Local elevation is never automatic. Start with least privilege; denied/hidden scopes become failed or partial checks. If your organization authorizes a local elevated token for broader inventory, open that session yourself. AppLocker/WDAC, constrained language and script signing/execution policies may block collection; follow your organization's approved signing/distribution process. The app does not bypass policy.
Only **exact, explicitly approved local share roots** can be retained. Approval of
one root never authorizes child paths or filenames. No file contents are inspected.
There is no automatic patient-name detector or guarantee that infrastructure
metadata cannot contain PHI before projection.
### GUI on a compatible Windows desktop environment
Set `report_directory` to an approved local directory. Linux creates a missing root
with mode 0700 and files with 0600; existing broadly readable roots are rejected.
Windows requires a provisioned protected directory ACL and checks child ACLs before
writing. Common remote paths, symlinks and reparse points are rejected. Cloud sync,
backups, unusual storage backends and OS incidental writes require separate review.
See the [configuration and Windows directory setup instructions](docs/PRIVACY_HARDENING.md).
Use 64-bit Python 3.11+ already available on the authorized execution host. Keep `powershell/`, `schemas/` and `samsara/` together in the checkout. From its root in Command Prompt:
## Local Windows Server collection
The code targets the Server 2016, 2019, 2022 and 2025 release builds.
**Actual Windows inventory, revised PowerShell queries and Windows output ACLs have
not been validated on a Windows execution host in this environment.** Synthetic
Python tests are not Windows compatibility certification. Do not deploy to production
until authorized lab validation and CHS review are complete.
Run locally using an existing authorized account and 64-bit Python 3.11+. A Windows
10/11 reporting desktop does not remotely scan servers in this phase. No WinRM/SSH
configuration, service restart, agent installation, saved credentials or automatic
elevation is performed. Do not use Domain Administrator by default. Permission
failures and incomplete scopes remain visible; provider ACL filtering may be silent.
Follow normal application-control/script-signing policy; no execution-policy bypass.
The GUI runs collection in a background worker. Each PowerShell section has a
45-second limit, an overall 300-second budget and bounded output. Stdout remains in
bounded memory instead of an inventory temporary file; stderr is discarded.
Cancellation/timeouts terminate and reap the owned child process.
For headless Server/Core collection, install only the headless requirements through
approved provisioning. Keep `samsara/`, `powershell/` and `schemas/` together. After
configuring and provisioning `C:\SamsaraReports` as described above:
```bat
py -3 -m venv .venv
.venv\Scripts\python.exe -m pip install -r requirements.txt
.venv\Scripts\python.exe -m samsara.main
.venv\Scripts\python.exe -m pip install -r requirements-headless.txt
.venv\Scripts\python.exe -m samsara.collectors.windows --output C:\SamsaraReports\windows-inventory.json
```
Choose **Analyze → This computer → Auto-detect → Analyze**, or select the matching Windows Server release. Windows PowerShell 5.1 at the Windows system path performs the local inventory in the background. Review the Summary and each tab, especially **Collection checks**, then save HTML/JSON. The history and report source say **Local Windows read-only collection**. Cancel terminates and reaps the active PowerShell process and keeps the previous assessment. Each section has a 45-second limit, overall execution a 300-second budget, and output a 10 MiB limit; timed-out sections remain visible as incomplete.
Use a new filename. Copy minimized output only through an organization-approved
transfer process; Samsara provides no remote transport. GUI compatibility is separate
from collector compatibility: use headless collection if the Qt version cannot run
on the server OS or no graphical desktop is present.
Qt desktop compatibility is separate from collector compatibility. Current Qt 6.12 lists Windows 10 1809+ and Windows 11 as supported desktops; do **not** assume a Qt GUI will run on Server 2016 or Server Core. Use headless collection on those hosts and analyze the resulting file on a compatible desktop. See [Qt's Windows platform requirements](https://doc.qt.io/qt-6/windows.html).
### Headless Windows collection (including Server Core / Server 2016)
With existing 64-bit Python 3.11+, no PySide6 installation or graphical desktop is required. From the repository root in Command Prompt:
```bat
py -3 -m venv .venv
.venv\Scripts\python.exe -m pip install "jsonschema>=4.21,<5"
mkdir reports
.venv\Scripts\python.exe -m samsara.collectors.windows --output reports\windows-inventory.json
```
Use a new filename; the CLI will not overwrite an existing file. The CLI uses the same supervised collector, server identity gate and schema validation as the desktop. Copy the JSON manually to the reporting desktop and choose **Saved inventory file**, then Analyze. Imports retain timestamps, safe error codes and the presence of permission limitations, remove raw diagnostics/source claims, and recalculate risk. No remote transport is implemented.
If Python is unavailable, the PowerShell-only driver can produce an inventory with no Python/Qt dependency, subject to your existing script policy. From a local Windows PowerShell 5.1 prompt in the checkout root:
```powershell
$json = & .\powershell\Get-ServerInventory.ps1
if ($LASTEXITCODE -ne 0) { throw 'Local inventory collection failed.' }
[IO.File]::WriteAllText((Join-Path $PWD 'windows-inventory.json'), $json, (New-Object System.Text.UTF8Encoding($false)))
```
Choose a new destination in the final line to retain earlier evidence. This direct path is **not supervised by Python's time/output/cancellation bounds**; prefer the Python CLI where available. Import validates the returned schema before desktop analysis.
### Coverage and missing evidence
| Section | Collected | Known limitations |
| Section | Default scope | Omitted or unverified |
| --- | --- | --- |
| System | OS caption, Server release/build, boot, IPs/adapters, domain membership | Requires readable CIM identity; AD enrichment not performed |
| Roles | Installed roles/features, state and feature type | Local ServerManager/module access needed; installation does not prove use |
| Services | All visible states, startup type, account, PID | Includes running services; command lines and secrets deliberately excluded |
| Networking | TCP listeners/connections, endpoints, PID/name | Snapshot only; direction, historical use, UDP and protected processes may be unknown |
| Tasks | Visible task names/paths, state, enabled setting, principal, safe executable basename and selected trigger fields | No arguments, XML, inline scripts, event query details or execution history; visibility depends on account |
| Software | Machine uninstall registrations in both registry views | No Win32_Product; per-user, portable, container and unregistered applications not covered |
| Storage / shares | Local volumes, capacity/free space, configured SMB share paths including administrative shares | No contents/ACLs/backups; optional read-only open-file aggregate counts identify business-share activity only when the share root can be correlated without an administrative alias |
| Certificates | Subject, issuer, expiry, thumbprint and machine store | No private keys, user stores, file-based certificates or application bindings |
| System | Fixed OS family/release/build, boot, literal IPs, adapter labels, domain-joined flag | Domain/host identity unless exact hostname approved; no AD enrichment |
| Roles | Installed identifiers/state | Free-text names; installation does not prove use |
| Services | Canonical families/opaque labels, states, startup, PID | Accounts, descriptions, executable paths/arguments |
| Network | TCP endpoints, state, PID, canonical process family | Snapshot direction/history may be unknown; no UDP/application content |
| Tasks | Opaque identity, maintenance category, state and inferred enabled state | Actions, commands, principals, trigger details and history; partial scope |
| Software | Canonical or approved names and numeric versions from machine registry | Publisher, dates, uninstall commands, per-user/portable software |
| Storage/shares | Fixed local drive metadata, capacities, configured shares and exact approved roots | Labels, child paths, filenames, contents, open-file/session activity; partial scope |
| Certificates | Not collected by default | Certificate identities/extensions/objects and private keys are not queried |
Each check includes status, UTC start/end times, sanitized errors, permission limitations and scope limitations. `complete` means the stated query scope succeeded; it does not prove complete retirement evidence. Sensitive checks run non-elevated are conservatively partial. Some Windows providers can silently filter objects by ACL; even elevated results need administrator review. Partial accessible data are retained. Invalid/unsupported Server identity stops the assessment; failed other checks never receive demo replacements.
Check timestamps/status/errors are recorded. `complete` describes the documented
query scope, not complete retirement evidence. Privacy omissions lower confidence
and trigger review; they do not add workload risk points.
### Required real-host validation (still outstanding)
On a supported, authorized Server, use the headless environment above and a new output directory:
Required lab validation, using the configured private directory and new output names:
```bat
.venv\Scripts\python.exe -m unittest discover -s tests -v
.venv\Scripts\python.exe tools\validate_windows_host.py --output-dir reports\windows-validation-01
.venv\Scripts\python.exe tools\validate_windows_host.py --output-dir C:\SamsaraReports
```
The second command performs **real local collection**, validates the inventory schema and writes `windows-assessment.json`, `windows-assessment.html` and `validation-summary.json`. Review the full report privately. The summary omits hostnames, service accounts, IPs, paths and certificate identifiers and is suitable for sharing validation status after review. It reports per-section status/error codes and row counts; it does not claim retirement approval. Run under the intended least-privilege account and, only if authorized, repeat in an elevated session with a different directory to assess visibility differences.
The second command runs **real local Windows collection**, writes minimized
`windows-assessment.json`, `windows-assessment.html` and `validation-summary.json`,
and prints categorical validation status. It refuses existing files and arbitrary
output directories. Review results privately. Repeat under intended least privilege
and separately authorized elevation, using separate provisioned directories/policy.
All four Server releases still require real-host acceptance evidence.
| Release | Python synthetic fixtures / orchestration / UI | Native PowerShell runtime | Actual host inventory |
| --- | --- | --- | --- |
| Server 2016 | Passed | Not run here | Pending |
| Server 2019 | Passed | Not run here | Pending |
| Server 2022 | Passed | Not run here | Pending |
| Server 2025 | Passed | Not run here | Pending |
The direct PowerShell driver remains available, but is not supervised by Python's
output/time limits or private-file writer. Do not use shell redirection or arbitrary
file writes as a substitute for the protected CLI workflow.
The implementation is **not considered complete** until real-host evidence is obtained. Do not describe the pending rows as tested Windows support. Packaged Windows installers are also unvalidated.
## Linux collection
## Linux collection and common safety
Local Linux collection observes OS/version/boot, literal addresses, loaded systemd
service/timer state, TCP metadata, RPM/dpkg packages and mount metadata. It does not
access mounted remote filesystems. Missing commands/access produce failed checks.
Default cron files and detailed timer properties are no longer read: schedule timing,
commands, user schedulers and execution history need an approved separate review.
Certificates, share exports/consumers and directory integration are uncollected.
The Linux collector observes OS/kernel/boot information, addresses, loaded systemd services and timers (including available calendar/next-run metadata), readable system cron entries, TCP snapshots, RPM or dpkg software and mounted filesystems. Missing commands or restricted permissions produce visible failed checks. Service accounts, user crontabs/timers, anacron/periodic-script configuration, certificate locations, share exports, disk capacity/free space and directory integration remain incomplete or uncollected. Results reflect the execution process's namespace and permissions. Remote mounts are not accessed.
Collection is read-only, requires no sudo and does not restart services, configure remoting, install software or read private keys, secret-bearing command lines or environment variables. Cancel stops the active collector and retains the previous assessment. Commands have time and output limits.
The summary highlights name-matched server software and running application services; this is not a complete application classifier. Scheduled entries show configured work, not confirmed execution, and cron timezones are unverified. Full inventory and collection limitations remain available in the tabs and exported report.
## Retirement risk and confidence
Scoring model v2 scores grouped workload evidence, not inventory counts. The final 0–100 score is the **strongest workload score**; services, ports and clients supporting the same workload are not added. Standard Windows services/features, default administrative shares, browser/Steam client activity, unknown traffic and collection failures do not automatically receive risk points. Installed/stopped software is not active-use evidence.
Coverage percentage and evidence confidence are displayed separately. Failed task/certificate checks lower confidence and require review without implying an active application. Low/zero risk never grants retirement clearance: historical usage validation, business ownership and administrator approval are always required.
Configure scoring through **Settings → Load scoring policy JSON**, starting with `samsara/assets/scoring_policy.json`. The applied policy and exact per-workload score explanation are recorded in JSON and HTML; history keeps its original policy. See [scoring rules, limitations and regression scenarios](docs/SCORING.md).
To re-score the test server without contacting it again, restart Samsara and open its previous JSON inventory or assessment using **Saved inventory file**. Existing v1 raw inventory is compatible; old scores are ignored and recomputed. The saved timestamp/source remain explicit. The original report JSON was not provided for development, so the 1,565-point regression uses synthetic data matching the reported counts.
HTML now summarizes actionable findings and confidence rather than dumping hundreds of services. Each HTML export creates a linked, content-addressed inventory JSON file beside it. Keep both files together; the JSON assessment export also retains all inventory. Every routine service, feature, administrative share and connection remains available in the detailed UI tabs and full JSON.
## Headless Linux inventory
The standard-library-only collector can export local inventory without Qt:
For the default application-local private `reports` root:
```bash
python3 -m samsara.collectors.linux --output /tmp/server-inventory.json
python3 -m samsara.collectors.linux --output reports/linux-inventory.json
```
Use a new filename; existing files are not overwritten. Open the result using **Saved inventory file** on either desktop. Imported check assertions are not independently verified. Import validates the v1 schema, limits input to 10 MiB, removes external diagnostic details and service/task command fields, and recalculates risk.
The root's parent must exist; output is minimized and files are exclusive/private.
Use a configured approved absolute directory for deployment. Collection never uses
sudo or changes remote settings. Native utilities and OS logging/caching behavior
still require operational review; read-only queries do not mean data cannot disclose
sensitive information.
## Development
## Retirement scoring
Scoring model v2 uses the maximum grouped workload score (0–100), never unrestricted
inventory totals. Routine Windows services/features, administrative shares,
Steam/browser client traffic, unknown connections and collection failures do not
receive automatic workload points. Known application PID/listener/client evidence
can support a workload finding; no activity does not prove a host unused.
Coverage/confidence are independent from risk. Historical usage validation, business
ownership and administrator approval are always required. Settings loads a bounded
data-only scoring JSON; numeric policy values and exact rationale are preserved.
Policy display names are canonicalized to avoid free-text disclosure.
See [scoring details](docs/SCORING.md).
## Verification and remaining work
```bash
python -m unittest discover -s tests -v
QT_QPA_PLATFORM=offscreen .venv/bin/python -m unittest discover -s tests -v
```
Qt tests run offscreen. See [platform support](docs/PLATFORMS.md), [architecture and plan](docs/IMPLEMENTATION_PLAN.md) and [developer guide](docs/DEVELOPMENT.md). Run from source; Windows packaging and real-host collector acceptance remain pending. PowerShell AST/synthetic tests run when a native runtime is available and are explicitly skipped otherwise. On this Linux development host, 61 tests were discovered: 60 passed and the native PowerShell test was skipped; no real Windows inventory has been collected.
Synthetic tests cover identifying canaries in every major free-text path, stale
findings/marker bypasses, exact-root approval, private/exclusive writes, invalid
schemas/timestamps, process output limits/cancellation, UI/history, report escaping
and the seven retirement scenarios. Native PowerShell tests are explicitly skipped
if no runtime is available. No production collector, package installation/update or
external advisory lookup was performed during hardening.
Outstanding: native Windows lab validation and ACL/race testing, least-privilege
validation, platform/hash locks, native-component SBOM, authorized vulnerability
scan, signed distribution, approved storage/egress/retention controls and CHS
security/privacy approval. These prevent a claim of production readiness or HIPAA
compliance. The old bundle/reports are not automatically updated or scrubbed.
[Developer guide](docs/DEVELOPMENT.md) · [Platform notes](docs/PLATFORMS.md) ·
[Architecture](docs/IMPLEMENTATION_PLAN.md) · [Security hardening](docs/PRIVACY_HARDENING.md)