Samsara/docs/SCORING.md

# Retirement scoring model v2

## Purpose and migration

The previous engine added points for every running service, installed role, listener, connection and administrative share. A routine Windows snapshot could accumulate thousands of raw points and saturate at Critical. Model v2 removes that inventory-counting behavior. Collection schema remains v1; assessment metadata is versioned separately as `model_version: 2.0`.

Re-analyze a previous JSON report through **Analyze → Saved inventory file**. Its raw inventory and collection time are retained; the old assessment is ignored and a v2 assessment is calculated with the selected policy. No new Windows access is needed to re-score existing evidence. Imported data remain explicitly unverified. Old HTML scores do not change; generate a new report. Scores from v1 and v2 are not directly comparable.

## Evidence classifications

- **Confirmed workload dependency:** observed activity attributed to a recognized application listener, a verified domain-controller identity, or read-only SMB open-file evidence correlated to a business share root. For TCP this confirms observed transport activity, not connection initiation, business ownership, application health, or permission to retire.
- **Potential workload dependency:** a meaningful installed workload role, running recognized application service, attributed listener, non-administrative share or nonroutine configured schedule. Configuration alone does not prove current use.
- **Standard operating system activity:** baseline services/components, administrative shares, management networking and recognized maintenance schedules. Zero workload points; complete data remain in inventory.
- **Unknown / insufficient evidence:** unmatched services, direction/ownership gaps, client activity, installed/stopped historical application hints and failed checks. Unknown observations never imply an active workload and never imply non-use.

Classification is conservative and name-based in places. Unknown applications, custom service names and interpreters such as Java cannot be reliably mapped from a port alone. SQL client libraries, stopped database/game services, browsers and Steam client traffic do not establish hosted workloads. Review unmatched services with owners; a low score can reflect rule/collection gaps.

## Bounded scoring

1. Group correlated observations by workload family (SQL, IIS, web, file sharing, etc.). Multiple instances or clients in a family do not add points.
2. Select the strongest applicable rule within each workload group. A service, its listener and 100 connections remain one workload score.
3. Final score is **the maximum workload score**, or zero if none are scored. Scores are never added, even across independent workload families. Independent workloads still appear separately for administrator review.

| Strongest evidence | Default workload score |
| --- | --- |
| Meaningful installed workload role only | 15 |
| Running recognized application service | 35 |
| Listener attributed to a recognized application | 45 |
| Application transport activity on an attributed listener | 70 |
| Configured non-administrative share | 20 |
| File-sharing transport with share access still unverified | 40 |
| Business-share-root-correlated open files | 80 |
| Enabled custom/nonroutine schedule | 20 |
| CIM ProductType=2 domain controller identity | 95 |

Thresholds: 0 informational, 1–24 low, 25–59 medium, 60–89 high, 90–100 critical. These are triage categories, not probabilities or a validated retirement-readiness percentage. A running IIS service without identifiable worker traffic is potential evidence (35), not automatically an actively consumed application. HTTP.sys `System`/PID 4 traffic alone cannot identify an IIS site; those connections remain unknown unless more precise attribution is available. A role install without a domain-controller identity is not a verified DC.

The bundled policy is `samsara/assets/scoring_policy.json`. Copy it, adjust documented integer weights/ordered thresholds and coverage weights, then use **Settings → Load scoring policy JSON**. Data-only policies accept no executable expressions. Changes apply to future assessments; history retains earlier results and policy snapshots. Every assessment includes the full policy, SHA-256 identity, selected rules, evidence references and which groups determine the final maximum. Co-leading groups can share that maximum; their scores still are not added. Raw evidence references such as `services[4]` and `network[12]` point into the separately preserved inventory arrays.

## Direction and correlation

- Listener address/port plus the same nonzero owning PID can support **Inbound (inferred)**. This does not capture the initiating TCP handshake. A known application attribution is also required to score observed application clients.
- Explicit source direction is labeled **reported**, not independently verified, and still requires listener/process correlation to upgrade to observed application-client evidence.
- Ephemeral local ports to non-ephemeral remote ports without an attributed listener can be **Outbound (inferred)**. The port-range assumption is heuristic and may differ on configured hosts.
- Loopback, IPv4-mapped loopback and same-host addresses are identified first. These and outbound/unknown directions do not become external client evidence.
- A bare 443, 1433 or game-related port never identifies an application. Steam and browser process activity never create a dedicated game-server workload.
- Listener matching uses local address compatibility and a positive PID, not just a common port. HTTP.sys, shared processes, missing PIDs, process exits, dual-stack behavior and non-TCP activity can leave attribution unknown.

File-server confirmation is stricter than TCP 445: a connection could be administrative access. `Get-SmbOpenFile` reads open-file metadata; the collector requires `Path = configured share root + ShareRelativePath` to distinguish access through an administrative drive root from a nested business share. It exports only aggregate counts and observation time. Same-root administrative aliases are explicitly ambiguous and cannot confirm business-share use. Same-root business aliases cannot be individually distinguished and remain one workload family. File contents, file names, session user names and client identities are not exported by this query. Denied access reduces confidence. Zero open files never proves inactivity. See [Microsoft's Get-SmbOpenFile metadata](https://learn.microsoft.com/en-us/powershell/module/smbshare/get-smbopenfile).

## Coverage, confidence and review

Coverage is calculated separately from weighted check statuses: complete=1, partial=0.5 by default, failed/not_collected/missing=0. Permission limitations or errors on a nominally complete check conservatively receive partial credit. Required scopes and weights are explicit in policy. Scope limitations continue to apply even with 100% coverage.

Evidence confidence is low below 60% coverage, moderate below 90% or with incomplete checks/unclassified services/unknown directions, and high otherwise. Confidence describes the collected and classified snapshot, not historical inactivity. It is not a statistical confidence interval. Checks with access errors trigger manual review; they never add workload score. Positive evidence is not erased when unrelated collection is incomplete.

Historical monitoring over a representative business cycle, infrequent workloads, ownership, service accounts, external dependencies, backups/retention, rollback and administrator approval must be verified before retirement. `review_required` is always true. The application does not declare a server safe to retire.

## Reporting and validation

HTML leads with actionable workload groups, scoring rationale, confidence and missing checks. Routine inventory is summarized in zero-point context groups rather than repeated as medium-risk dependencies. A content-addressed `.inventory-<hash>.json` sidecar preserves every collected inventory row; its SHA-256 and link are in the report. Keep it with the HTML. JSON assessment export also retains full inventory, model/policy and detailed observations. Existing different sidecars are never overwritten.

Regression fixtures cover a clean Windows Server, an inactive former game server, active SQL, active IIS, a domain controller, business file-share use and incomplete collection. An additional regression reconstructs the reported 1,565-point pattern (86 services, 19 connections, 14 features, 23 listeners, three admin shares and two failed checks): v2 gives zero workload risk and reduced confidence, while retaining all observations. This is a synthetic reproduction of the supplied counts, not a replay of an unavailable original inventory file.

These tests do not certify business dependency discovery. Re-score the real test server's saved JSON, review results with its owner, and validate any new SMB activity fields on Windows. Native PowerShell tests remain skipped in the Linux development environment.