## Source Summary

This April 2025 arXiv paper presents an LLM-powered NPC dialogue system that operates across games and social platforms, with emotional modeling and persistent memory. The system takes player voice input (transcribed), maintains a memory of past interactions per NPC, models emotional state transitions, and generates contextually appropriate responses that are then text-to-speech synthesised. The cross-platform design means the same NPC can exist in different games or virtual spaces with consistent personality and memory. This represents the state of the art in voice-driven NPC interaction and defines the API surface that a voice daemon serving gamers should target.

---

## Key Claims

- [EVIDENCE src-009] LLM-driven NPCs with emotional modeling and persistent memory are demonstrated across multiple games and social platforms.
- [EVIDENCE src-009] The system takes natural voice input (via ASR transcription) — no command syntax required.
- [EVIDENCE src-009] Emotional state modeling allows NPCs to respond differently based on the player's prior relationship history, not just the current utterance.
- [EVIDENCE src-009] The system targets future development of emotional modeling and persistent memory as key differentiators over keyword-based NPC dialogue.
- [HYPOTHESIS] The bottleneck for offline deployment of LLM NPCs is the inference latency of the LLM response generation, not the ASR front-end — a fast offline ASR (like faster-whisper) with a cloud LLM is the current practical compromise.
- [HYPOTHESIS] A voice daemon that provides emotion-annotated transcripts (valence + arousal alongside text) would be a value-add integration point for LLM NPC systems — reducing duplicated audio processing between the voice daemon and the game engine.

---

## Technical Mechanisms

### Architecture / System Design

[EVIDENCE src-009] Pipeline: Player microphone → ASR → transcript → LLM (with emotional state context + memory) → text response → TTS → NPC voice. The LLM receives a prompt that includes: current NPC emotional state, conversation history (persistent memory), and the player's transcribed utterance. The LLM outputs both a text response and an updated emotional state.

### Core Algorithms or Methods

[EVIDENCE src-009] Emotional modeling uses a state machine or prompt-engineered emotional trajectory — the NPC's emotional state evolves based on player utterances and NPC memory. Persistent memory is implemented via retrieval-augmented generation (conversation history indexed and retrieved by semantic similarity).

### Key Data Structures or APIs

[EVIDENCE src-009] NPC state representation: `{emotional_state: {...}, memory: [...], relationship_level: float}`. The player input interface expects a text transcript — compatible with YazSes output. [HYPOTHESIS] YazSes could expose a structured transcript event (text + emotion annotation + speaker turn metadata) that NPC systems consume directly, reducing the voice processing each game engine must implement independently.

---

## Identified Gaps

### Stated Limitations

- [EVIDENCE src-009] Cloud LLM dependency is implied; fully offline LLM NPC at interactive latency is not demonstrated.
- [EVIDENCE src-009] Cross-platform persistent memory raises privacy concerns that are not addressed.

### Inferred Gaps

- [HYPOTHESIS] The system does not handle the case where the player's voice input is command-like rather than conversational (e.g., "attack" vs. "you betrayed me") — the routing between command dispatch and NPC dialogue is not specified.
- [HYPOTHESIS] No evaluation of the system with users who have dysarthric or accented speech is reported.

### Evaluation / Coverage Gaps

- [EVIDENCE src-009] Player engagement metrics are not reported in the preprint abstract; validation methodology is unclear.

---

## Applicability to Project

[EVIDENCE src-009] The cross-platform LLM NPC design defines a clear integration target for YazSes' gaming mode: YazSes provides the ASR + emotion annotation layer; the game engine or NPC platform provides the LLM reasoning and TTS output. [HYPOTHESIS] A YazSes "gaming protocol" — a structured JSON event stream containing transcript, emotion, confidence, and timing metadata — would let any game engine implementing this protocol consume YazSes output without custom audio processing.

[EVIDENCE src-009] The command-vs-dialogue routing gap (identified above as an inferred gap) is something YazSes' existing grammar classifier can solve: commands matching the grammar go to dispatch.py; unmatched utterances are tagged as DICTATE and forwarded to the NPC dialogue system. [HYPOTHESIS] This would make YazSes a natural "voice middleware" layer in LLM-game architectures, translating audio into structured events without the game needing to do any speech processing.

---

## Notes for SoA Matrix

**Covers:**
- `emotional_voice_gaming` — primary: LLM NPC with emotional state + memory from voice input
- `llm_intent_routing_offline` — LLM-mediated command/dialogue routing (though cloud-dependent currently)
- `ambient_context_awareness` — NPC memory provides persistent context for voice interaction

**Partially covers (~):**
- `offline_edge_deployment` — gap: cloud LLM required currently; offline SLM for NPC is future work

**Does not cover (✗):**
- `xr_spatial_voice` — standard games, not XR
- `silent_speech_biosignal` — not in scope
- `multimodal_fusion` — voice only
- `accessibility_aac` — not in scope
- `code_voice_coding` — not in scope