Lookout
A Windows app, installed from an MSI, whose service keeps the monitoring agents on a Windows desktop at the versions git declares and reports its own health to folly. tallboy runs it.
Lookout is the lab's monitoring agent for a Windows desktop such as tallboy. Its service installs windows_exporter, OhmGraphite and Vector, the three agents Install Windows monitoring describes, and keeps them at the versions and configuration that apps/lookout/policy/policy.json declares. A tray icon shows the state. tallboy runs it, on top of the agents the runbook's script installed.
Use it#
| Surface | What it does |
|---|---|
| Tray icon | A ring in the taskbar's colour with a badge: green when every agent is current and answering, amber on drift, a failed sync or a pending reboot, red when an agent is down, grey when the service does not answer |
| Flyout, on a left click | A card per agent with its version, state and problems, the time of the last sync, Sync now, pause convergence for an hour, and links to this desktop's Grafana board, the Event Viewer and status.json |
| Menu, on a right click | The summary, Open, Sync now, pause and Exit |
| Notifications | One toast each time the state changes |
C:\ProgramData\Lookout\status.json | The last report, for a script |
The service converges on a schedule from the policy, and at once on Sync now. A pause holds the schedule only. Sync now converges through a pause.
How it works#
The policy is git. apps/lookout/policy/policy.json names each agent's version, download URL, SHA256 and, for OhmGraphite and Vector, the URL of its config file in apps/lookout/windows/monitoring/. The service fetches the policy from main on every pass and keeps the last copy in C:\ProgramData\Lookout\, so a desktop that is offline converges to the policy it last saw.
Each pass observes the three agents, converges the ones that drifted, waits for a restarted agent's port to answer, then observes again. A version is compared without a v prefix or build metadata, so windows_exporter's v0.31.8 and OhmGraphite's 0.38.0+<commit> match the declared 0.31.8 and 0.38.0.
| Agent | Drift means | Convergence |
|---|---|---|
windows_exporter | Another version, or other collectors on the service command line | Remove the installed product, then install the MSI with the declared collectors; a flag on the command line beats config.yaml, and Windows Installer refuses to reconfigure an installed version |
| OhmGraphite | Another version, a different config, or no firewall rule | Unpack the zip, rewrite the config, register and restart the service, open the port |
| Vector | Another version, a different config, or no scheduled task | Remove the old product, install the MSI, write the config, register and start the SYSTEM task at boot |
The service writes lookout.prom into windows_exporter's textfile_inputs directory, so folly scrapes the agent's own state with the host's metrics, and logs to the Application event log under source Lookout, which Vector ships to VictoriaLogs. The tray and the service talk over the named pipe lookout. The tray is WPF in the Fluent theme, so it follows the light or dark setting and the accent colour; the flyout asks the Desktop Window Manager for an acrylic backdrop and rounded corners, which Windows 10 ignores.
The MSI, built by WiX from apps/lookout/packaging/Lookout.wxs, holds two programs. The tray is the app in Settings and the Start menu, and starts at logon through the Run key the installer writes. The service runs as LocalSystem. The Start menu shortcut carries the AppUserModelID lolwtf.Lookout, which is what lets the tray show notifications without package identity; when that fails, the tray icon's balloon tip is the notification.
The agent updates itself. When the policy's agent entry names a version other than the running one, the service downloads that MSI, checks its SHA256 and starts msiexec without waiting. The installer stops the service, replaces the files and starts it again.
Limits#
- The MSI is unsigned, so SmartScreen asks once on a downloaded copy. Choose More info, then Run anyway.
- Clicking a notification opens nothing. An unpackaged app needs a COM activator for that, and the tray has none.
- Memory Integrity still blocks the sensor driver. See Check Memory Integrity.
- The tray reaches the pipe on tallboy, and
windows_exporterreads the textfile there. Whether notifications appear is unconfirmed.
Operate#
folly alerts in clusters/folly/monitoring/windows-exporters.yaml:
| Alert | Fires when |
|---|---|
LookoutStale | The service has not finished a pass in 2 days |
LookoutComponentDrifted | An agent has not matched the policy for 6 hours |
LookoutComponentUnhealthy | An agent is not running for 30 minutes; the only alert that sees Vector stop |
To change an agent version, edit policy.json and merge. Every desktop converges on its next pass. To test a policy before merging, put its raw URL in policyUrl in C:\ProgramData\Lookout\settings.json and restart the Lookout service.
To roll out a new Lookout, raise Version in apps/lookout/Directory.Build.props and merge. The lookout workflow publishes the GitHub release lookout-v<version> with the MSI and its SHA256 in the notes. Then put that version, the asset URL and the hash in the agent entry of policy.json and merge again.
Reference#
- Source:
apps/lookout/ - Policy:
apps/lookout/policy/policy.json - Tests and build:
mise run lookout:test; the MSI builds and releases in.github/workflows/lookout.yml - Releases:
lookout-v<version>on the repository's releases page - Grafana: the Windows Desktops dashboard,
clusters/folly/monitoring/grafana-dashboards/windows.json
Linked from 1
- Install Windows monitoringAdd a Windows desktop to folly's Prometheus, install or update its monitoring agents, and find why it stopped reporting.