feat(scarfmon): track empty-assistant turns + document Nous quirk

User reports chats "dying" on Nous models — screenshot shows the
assistant bubble stuck with `(°□°) deliberating...` and a
1.7s turn-duration pill (turn DID complete; the content is the
problem). The literal placeholder string isn't in Scarf's source;
it's coming from Hermes or Nous itself when the model emits a
brief thought stream and then fails to produce any visible
output.

ScarfMon trace confirms the failure mode:
  mac.sendViaACP    →  firstThoughtByte (25 bytes)
  mac.handleACPEvent  ✓
  mac.sendPrompt     ✓ (1.7s, normal)
  finalizeStreamingMessage  ✓ (turn cleanly closed)

So Scarf sees no transport error — the turn finalized normally
with empty assistant text plus a small thought stream. The
visible "deliberating" text is content Hermes/Nous chose to
substitute for the missing response.

Adds `mac.emptyAssistantTurn` event (category .chatStream) that
fires whenever a turn finalizes with empty `streamingAssistantText`
and empty `streamingToolCalls`. Bytes carry the thinking-text
length so we can distinguish:
  - bytes=0: total empty turn (model produced nothing)
  - bytes>0: thoughts-only turn (model thought but didn't answer)

Both are user-visible failures. The fix is upstream — Hermes
should refuse to finalize a turn with no response and surface
an error, OR Nous should not return empty responses with the
placeholder string. Document this finding so a future capture
that shows multiple `mac.emptyAssistantTurn` events confirms
the rate / model-correlation.

For now Scarf surfaces the same UX as before (no UI change in
this commit). A follow-on commit could intercept this case and
replace the bubble with a clearer "Model returned no response"
banner, but that requires a confident heuristic for which
empty-finalize cases are real failures vs. legitimate
no-response turns.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Alan Wizemann
2026-05-05 12:40:21 +02:00
parent f2ddcbbd60
commit f6dc45b397
@@ -899,6 +899,24 @@ public final class RichChatViewModel {
|| !streamingThinkingText.isEmpty || !streamingThinkingText.isEmpty
|| !streamingToolCalls.isEmpty || !streamingToolCalls.isEmpty
// ScarfMon surface turns that finalize with NO visible
// assistant text. Common Nous-model failure mode: model
// emits a few thought-stream bytes then falls silent;
// Hermes finalizes with empty content; the user sees a
// stuck "(°°) deliberating..." placeholder bubble. The
// event fires for both the all-empty case (which gets
// removed below) and the thoughts-only case (which is
// kept as a permanent message with empty body) both
// are user-visible failures worth tracking.
if streamingAssistantText.isEmpty && streamingToolCalls.isEmpty {
ScarfMon.event(
.chatStream,
"emptyAssistantTurn",
count: 1,
bytes: streamingThinkingText.utf8.count
)
}
if hasContent { if hasContent {
let id = nextLocalId let id = nextLocalId
nextLocalId -= 1 nextLocalId -= 1