Dedicated eye-tracker study (EYE-TRACKER-001, #412)¶

Status: research note. No implementation yet. No vendor SDK, dependency, config key or code is added by this note. Date: 2026-10-15 (every web source below was accessed on this date). Answers: the checklist in ROADMAP.md Phase 7 and AGENT_TASKS.md EYE-TRACKER-001. Risk closed or narrowed: R-22 in RISK_REGISTER.md. Related: ADR-v2-145, ADR-v2-149, ADR-019.

How to read this note¶

Every claim carries one of three labels.

• FACT — stated by the cited page. Wording in quotation marks is the page's own.
• INFERENCE — my reasoning from facts. It may be wrong; it is here so a maintainer can check it.
• UNVERIFIED — I looked and could not confirm it, or the page is gated. Nothing marked UNVERIFIED may be used as a licence or API claim. A wrong licence statement is worse than none.

Retrieval caveat. The pages were read through a retrieval tool that returns a model-written summary of each page, not the raw HTML. Quotations are the summary's quotations of the page. Before any licence or redistribution decision, a maintainer must re-read the primary licence text in a browser. This note is a map of where to look, not legal advice.

Project licence. YazSes is Apache-2.0 (pyproject.toml, LICENSE). The issue text and this study's brief say "MIT"; the repository says Apache-2.0, and that is what this note assumes.

Summary of the recommendation¶

1. Do not add any vendor SDK now. None of the proprietary SDKs below has a redistribution grant that I could verify, and the Tobii Stream Engine documentation is not publicly retrievable at all.
2. The existing GazeBackend seam is the right shape for a prototype, and the wrong shape for a product. Its sample is a camera-derived feature (yaw, pitch), not a point; it has no timestamp; and the daemon refuses to start gaze without a webcam calibration file. Details in What the current seam gets wrong. No bug exists today, because no tracker exists, so no code or test is proposed.
3. When a tracker is wanted, define GazeSampleProvider as a small, vendor-type-free Protocol that emits a normalized screen point with a monotonic timestamp, and keep every vendor type inside one adapter module. Proposed contract below.
4. A tracker provider is bring-your-own-SDK. It imports the vendor library lazily, ships none of its binaries, and is an optional extra only where the vendor library is verifiably redistributable and permissively licensed.
5. Neither the OS-level APIs nor the webcam path is a stand-in for a dedicated tracker. Keep the accuracy class separate, as README.md already requires.

Per-device and per-platform findings¶

Tobii Pro SDK (research trackers: Spark, Nano, Fusion, Spectrum)¶

Tobii Eye Tracker 5 and Stream Engine (consumer trackers)¶

Gazepoint (GP3 family)¶

EyeTech (TM5 Mini and relatives)¶

Pupil Labs (Core, Neon)¶

OS-level options¶

Cross-vendor matrix (what the evidence supports)¶

What the current seam gets wrong for a tracker¶

I read src/yazses/gaze/ and src/yazses/perception/. There is no GazeSampleProvider in code; the name appears only in design/ (ADR-v2-145, README.md, ROADMAP.md). The live seam is:

• gaze/base.py: GazeBackend — name, estimate() -> (yaw, pitch) | None, close().
• estimate_sample() -> GazeSample(x, y, confidence) is not in the Protocol. GazeTargeter._sample finds it with getattr, and eyeeval/gaze_field.py calls it directly.
• gaze/targeter.py always resolves a window through calibration.predict(yaw, pitch) (gaze/zones.py: resolve_window); CalibrationMap is an affine fit from camera features to desktop pixels (gaze/calibrate.py).
• core/daemon.py returns no targeter at all when no calibration file exists ("not calibrated; run yazses gaze calibrate"), and the calibration CLI collects samples with backend.estimate().
• gaze/topology.py: CalibrationContext records camera_id, camera_width, camera_height.
• gaze/factory.py: _BACKENDS is a closed table of mediapipe and l2cs.

Against what a tracker delivers, five gaps follow. They are design constraints, not bugs: with no tracker backend, nothing misbehaves, so I am proposing no test and no code.

1. Feature versus point. The seam's sample is an eye-direction feature that needs a learned map. A dedicated tracker already reports a point on a display (Tobii Pro: normalized display area; Windows: DIP). Routing it through CalibrationMap would mean a fake identity "calibration" and storing a screen position in a field called yaw. It would work in a prototype and mislead readers.
2. No timestamp on the sample. Freshness exists only inside SharedGazeBackend (MAX_SAMPLE_AGE_S). Trackers stream continuously on their own clock (two clocks, for Tobii), so the staleness rule has to be part of the provider contract, not a private detail of one adapter. This is the same stale-signal hazard as R-06.
3. Calibration gating is webcam-shaped. The daemon will not start a targeter without the affine file, and CalibrationContext has no place for "tracker X bound to display Y".
4. No device discovery or health. The webcam path has a camera index and a failure reason; there is nothing for "tracker unplugged" or "vendor service not running".
5. Duck-typed optional method. estimate_sample being found by getattr means a provider can silently lose its confidence and be treated as 1.0 (the code says so for l2cs).

Proposed minimal GazeSampleProvider (design only)¶

Vendor types never cross this boundary. Names are proposals for a future ADR, not accepted API.

@dataclass(frozen=True)
class TrackerSample:
    x: float            # normalized, 0.0..1.0, origin top-left, y grows downward
    y: float
    timestamp_s: float  # time.monotonic() clock, taken/converted by the adapter
    confidence: float   # 0.0..1.0; 1.0/0.0 from a validity flag if that is all the vendor gives
    display_id: str     # which display the 0..1 square refers to

@runtime_checkable
class GazeSampleProvider(Protocol):
    @property
    def name(self) -> str: ...
    def start(self) -> None: ...                 # idempotent; the device is opened here, not at import
    def latest(self) -> TrackerSample | None: ... # newest fresh sample; never blocks beyond a stated bound
    def status(self) -> ProviderStatus: ...       # connected / reason; no coordinates in it
    def close(self) -> None: ...

Coordinate contract. A provider emits a point normalized over one named display, origin top-left. Conversion to the canonical desktop-logical-px space is done by YazSes core from DisplayTopology, not by the adapter, so ADR-v2-149's "one named canonical space, centralised conversion" stays true. Adapters whose source is not display-normalized (Windows DIP, Pupil screen mapping) convert inside the adapter and say so. A provider that cannot name its display emits nothing.

Timestamp contract. timestamp_s is on time.monotonic(), exactly as perception/signals.py already specifies, so one freshness rule serves both sources. The adapter, not the core, owns clock mapping (Tobii's two clocks, Pupil time). Device clock values are not exposed.

Confidence. In [0, 1] and defined per adapter. Binary validity maps to 1.0 or "no sample", because the existing [gaze] confidence_min gate would otherwise compare a measured number with a constant. A tracker's confidence is not comparable with the webcam's eye-agreement figure and must not share a threshold unseen (INFERENCE).

Calibration ownership. Two honest cases:

• Vendor-owned (Tobii Eye Tracker Manager, Windows Eye Control, Pupil's own tools): YazSes stores no affine map. It stores only a context record, a generalised CalibrationContext whose camera_id-style field becomes a sensor id plus the display binding, so a display or sensor change invalidates it under ADR-v2-149's rules. This is an additive schema change needing its own ADR.
• SDK-driven calibration (Pro SDK and Gazepoint expose calibration APIs, UNVERIFIED in detail): out of scope for v1. Do not re-implement vendor calibration.

Coexistence with the webcam provider.

• A tracker is not a camera consumer (already stated in ADR-v2-145). It takes no lease on the shared perception source, so head pose and face switch keep running on the webcam while gaze comes from the tracker.
• Exactly one gaze source is active, chosen by explicit user configuration. No fusion in v1.
• If the tracker drops out, gaze answers None and routing keeps the focused window, which is what the targeter already does for a failed webcam. It must not silently fall back to webcam gaze: that is a different accuracy class, and README.md forbids blending the claims. Status must name the reason.
• GazeBackend stays valid for the webcam and l2cs. The migration path is an adapter that presents a provider as a GazeBackend for the targeter, which is how ADR-v2-145 already describes it.
• eyeeval needs a fake provider with scripted samples (pure, injected), and any tracker-specific evaluation must report derived metrics only.

What a tracker provider must not do¶

• No cloud. No vendor account sign-in, licence activation, telemetry or "cloud" feature as a condition of use. Whether any SDK does this is UNVERIFIED, so each adapter needs a check before it merges. Anything that opens a socket needs an ADR-019 row, and the loopback-only guard pattern used for llm_cleanup applies; a LAN stream (Pupil Neon) is an escalation, not a freebie.
• No raw stream storage. No gaze coordinates, eye positions, pupil data, sample rates or device logs written to disk, to log output above debug, to crash reports or to issue attachments. Persisted data is limited to the calibration context (no samples). Windows' own guidance calls this data "potentially sensitive personal data" (FACT).
• No data the core does not use. Drop eye position in millimetres, pupil diameter, per-eye data and head pose at the adapter. Minimisation is cheaper than a later privacy review.
• No vendor types in the core. No import of a vendor module at top level (AGENTS.md rule 3), no vendor class in a signature, no vendor enum in config.
• No bundled proprietary binaries and no wheel from a proprietary licence in a default extra, until a maintainer has read the primary licence and obtained written confirmation.
• No change to webcam accuracy claims, no shared threshold, no shared calibration file.
• Ships off by default with a dormant enabled = False (rule 2); a new config key is L3 work.

Open questions for a maintainer¶

1. Read the primary Tobii Pro SDK agreement and Stream Engine agreement in a browser; is a user-installed SDK called from an Apache-2.0 tool inside the field of use? (The research licence page suggests not for general public distribution, but that is a summary.)
2. Is Windows' view-relative gaze usable from a windowless daemon? Needs a Windows machine with a supported tracker.
3. Which tracker class does the project actually want to support first? The matrix above says Gazepoint (loopback socket) and Pupil Core (loopback ZeroMQ) are the cheapest to reason about, but neither's field names and timestamps are verified here.
4. Does the project want the CalibrationContext generalisation (sensor id) as its own ADR?

Sources and access dates¶

All accessed 2026-10-15 unless stated. "Summary" means the retrieval tool's summary of the page.

Nothing about this repository or its user was sent to any of these sites.