mirror of
https://github.com/awizemann/scarf.git
synced 2026-05-08 02:14:37 +00:00
feat(scarfmon): track empty-assistant turns + document Nous quirk
User reports chats "dying" on Nous models — screenshot shows the assistant bubble stuck with `(°□°) deliberating...` and a 1.7s turn-duration pill (turn DID complete; the content is the problem). The literal placeholder string isn't in Scarf's source; it's coming from Hermes or Nous itself when the model emits a brief thought stream and then fails to produce any visible output. ScarfMon trace confirms the failure mode: mac.sendViaACP → firstThoughtByte (25 bytes) mac.handleACPEvent ✓ mac.sendPrompt ✓ (1.7s, normal) finalizeStreamingMessage ✓ (turn cleanly closed) So Scarf sees no transport error — the turn finalized normally with empty assistant text plus a small thought stream. The visible "deliberating" text is content Hermes/Nous chose to substitute for the missing response. Adds `mac.emptyAssistantTurn` event (category .chatStream) that fires whenever a turn finalizes with empty `streamingAssistantText` and empty `streamingToolCalls`. Bytes carry the thinking-text length so we can distinguish: - bytes=0: total empty turn (model produced nothing) - bytes>0: thoughts-only turn (model thought but didn't answer) Both are user-visible failures. The fix is upstream — Hermes should refuse to finalize a turn with no response and surface an error, OR Nous should not return empty responses with the placeholder string. Document this finding so a future capture that shows multiple `mac.emptyAssistantTurn` events confirms the rate / model-correlation. For now Scarf surfaces the same UX as before (no UI change in this commit). A follow-on commit could intercept this case and replace the bubble with a clearer "Model returned no response" banner, but that requires a confident heuristic for which empty-finalize cases are real failures vs. legitimate no-response turns. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -899,6 +899,24 @@ public final class RichChatViewModel {
|
|||||||
|| !streamingThinkingText.isEmpty
|
|| !streamingThinkingText.isEmpty
|
||||||
|| !streamingToolCalls.isEmpty
|
|| !streamingToolCalls.isEmpty
|
||||||
|
|
||||||
|
// ScarfMon — surface turns that finalize with NO visible
|
||||||
|
// assistant text. Common Nous-model failure mode: model
|
||||||
|
// emits a few thought-stream bytes then falls silent;
|
||||||
|
// Hermes finalizes with empty content; the user sees a
|
||||||
|
// stuck "(°□°) deliberating..." placeholder bubble. The
|
||||||
|
// event fires for both the all-empty case (which gets
|
||||||
|
// removed below) and the thoughts-only case (which is
|
||||||
|
// kept as a permanent message with empty body) — both
|
||||||
|
// are user-visible failures worth tracking.
|
||||||
|
if streamingAssistantText.isEmpty && streamingToolCalls.isEmpty {
|
||||||
|
ScarfMon.event(
|
||||||
|
.chatStream,
|
||||||
|
"emptyAssistantTurn",
|
||||||
|
count: 1,
|
||||||
|
bytes: streamingThinkingText.utf8.count
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
if hasContent {
|
if hasContent {
|
||||||
let id = nextLocalId
|
let id = nextLocalId
|
||||||
nextLocalId -= 1
|
nextLocalId -= 1
|
||||||
|
|||||||
Reference in New Issue
Block a user