mirror of
https://github.com/awizemann/scarf.git
synced 2026-05-10 18:44:45 +00:00
feat(templates): hackernews-digest template + dogfooding test harness
First pass of the dogfooding-templates initiative. Each pre-release cycle ships one new official `.scarftemplate` and uses installing/exercising that template as the regression test. v1 lands the harness scaffolding plus the first template under it. - HackerNews Daily Digest template (`templates/awizemann/hackernews-digest/`): config-driven (min_score / max_items / topics) cron-only template. No secrets — keeps the harness minimal until the fake-Keychain shim lands. Bundle validates against `tools/build-catalog.py`; entry added to `templates/catalog.json`. - `SCARF_HERMES_HOME` env-var override at `HermesProfileResolver` — the seam every Layer-B test relies on to drive Scarf against an isolated Hermes home. Bypasses cache + active_profile lookup; rejects relative paths. 5 unit tests + 3 ServerContext integration tests. - `TestModeFlags.shared.isTestMode` — reads `--scarf-test-mode` once from `CommandLine.arguments`. Wiring only; gating sites (Sparkle, capability probe, first-run walkthrough) land as Layer-B exercises them. - Layer A (`scarf/scarfTests/TemplateE2ETests.swift`): parses + plans the shipped HN bundle the way the app does at install time; asserts manifest, config schema, dashboard widgets, and cron prompt contract. Mirrors the existing site-status-checker coverage. - Layer B scaffold (`scarf/scarfUITests/TemplateInstallUITests.swift`): proves the launch-arg + env-var plumbing reaches Scarf. Full install click-through deferred until fixture-Hermes-home and accessibility IDs land. Wiki pages added separately on the `.wiki-worktree` branch: - `Template-Ideas.md` — backlog of 9 v1-feasible templates + full-spec v3 epic for Project-Site-as-Living-Surface (eBay listings use case). - `Test-Harness.md` — contributor guide for extending the harness. Verification: scarfTests 124/124, ScarfCore 220/220, new Layer A 3/3, Layer B scaffold 1/1, build-catalog.py + its 28 unit tests all green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Alan Wizemann
Binary file not shown.
@@ -0,0 +1,81 @@
|
||||
# HackerNews Daily Digest — Agent Instructions
|
||||
|
||||
This project keeps a daily digest of HackerNews top stories filtered to the score threshold and (optional) topic keywords the user configured. The same instructions apply whether you're Hermes, Claude Code, Cursor, Codex, Aider, or any other agent that reads `AGENTS.md`.
|
||||
|
||||
## Project layout
|
||||
|
||||
- `.scarf/config.json` — **the source of truth for filter settings.** Written by Scarf's install/configure UI. Holds:
|
||||
- `values.min_score` (number, default 100) — minimum HN score to include.
|
||||
- `values.max_items` (number, default 15) — cap on items per digest run.
|
||||
- `values.topics` (array of strings, default `[]`) — keywords to mark in the digest. Empty array means "no topic highlighting; include every story above the score threshold."
|
||||
- `.scarf/manifest.json` — cached copy of `template.json`. Don't modify.
|
||||
- `digest.md` — append-only markdown log. Newest run at the top. Each run is a section with the ISO-8601 timestamp as the heading. Created on the first run if it doesn't exist.
|
||||
- `.scarf/dashboard.json` — Scarf dashboard. **Only the `value` fields of the three stat widgets and the `items` array of the "Top Stories" list widget should be updated.** The section titles, widget types, and structure must stay intact.
|
||||
|
||||
## How configuration works
|
||||
|
||||
The user configures this project through Scarf's UI — not by editing files directly. On install, a form asked them for the score threshold, item cap, and any topic keywords; those values landed in `.scarf/config.json`. They can edit those values any time via the **Configuration** button on the project dashboard header.
|
||||
|
||||
Read configuration like this (JSON, via whatever file-read tool you have):
|
||||
|
||||
```
|
||||
cat .scarf/config.json
|
||||
# → { "values": { "min_score": 100, "max_items": 15, "topics": ["rust", "ai"] }, ... }
|
||||
```
|
||||
|
||||
**Never** edit `.scarf/config.json` yourself. If the user asks "raise the score threshold" or "add a topic" in chat, tell them to open the Configuration button on the dashboard.
|
||||
|
||||
## First-run bootstrap
|
||||
|
||||
If `digest.md` doesn't exist, create it with a one-line header:
|
||||
|
||||
```
|
||||
# HackerNews Daily Digest
|
||||
|
||||
Newest run at the top. Each section is a single digest.
|
||||
```
|
||||
|
||||
## What to do when the cron job fires
|
||||
|
||||
The cron prompt Scarf registers for this project carries **absolute paths** (the installer substitutes `{{PROJECT_DIR}}` at install time) — you don't need to figure out the project's location yourself. Use whatever absolute paths appear in the prompt you received; if you're working in the project's interactive chat instead, the paths below are relative to the project root.
|
||||
|
||||
1. Read `.scarf/config.json`. Extract `values.min_score` (number), `values.max_items` (number), and `values.topics` (array). Apply defaults (100 / 15 / `[]`) for any missing field.
|
||||
2. Fetch `https://hacker-news.firebaseio.com/v0/topstories.json`. Take the first `max_items * 3` IDs — that gives headroom for the score filter to drop low-scorers without re-fetching.
|
||||
3. For each ID, fetch `https://hacker-news.firebaseio.com/v0/item/<id>.json`. Keep only items where:
|
||||
- `type == "story"`,
|
||||
- `score >= min_score`,
|
||||
- either `url` or `text` is non-null.
|
||||
4. Truncate the surviving list to `max_items`.
|
||||
5. If `topics` is non-empty, walk each surviving item and find the first keyword whose lowercase form is a substring of the lowercase title. Tag the item with that keyword in `[brackets]`. If no keyword matches, leave the item un-tagged.
|
||||
6. Build a digest section:
|
||||
```
|
||||
## <ISO-8601 timestamp>
|
||||
|
||||
- [<score>] <title> [<topic>]? — <url or https://news.ycombinator.com/item?id=<id>>
|
||||
- …
|
||||
```
|
||||
Use the HN comments URL when the item has no external `url`.
|
||||
7. Prepend the section to `digest.md` (newest at top).
|
||||
8. Update `.scarf/dashboard.json`:
|
||||
- `Top Story Score` stat widget: `value` = the highest score in your filtered list (or `0` if the list is empty).
|
||||
- `Items Tracked` stat widget: `value` = number of items in the filtered list.
|
||||
- `Last Run` stat widget: `value` = the ISO-8601 timestamp.
|
||||
- `Top Stories` list widget `items`: one entry per filtered story:
|
||||
- `text`: `"[<score>] <title>"`
|
||||
- `status`: `"ok"` if the story matched a topic, otherwise `"pending"`.
|
||||
9. If the cron job has a `deliver` target set, emit a one-line summary (`12 items, top score 487 — "<title>"`) as the agent's final response so the delivery mechanism picks it up.
|
||||
|
||||
## What not to do
|
||||
|
||||
- Don't modify the structure of `dashboard.json` (section titles, widget types, widget titles, `columns`). Only the values listed above are writable.
|
||||
- Don't edit `.scarf/config.json` — that's the user's responsibility via the Configuration UI.
|
||||
- Don't truncate `digest.md` — it's the historical record. If it grows past 1 MB, add a one-line note at the top of the file asking the user to archive it.
|
||||
- Don't fetch any URL other than `hacker-news.firebaseio.com` (the digest source) or the items the user explicitly asks about. No scraping, no other news sources.
|
||||
- Don't paginate past the first `max_items * 3` IDs. If the score filter eats all of them, write an empty digest section noting "no stories above threshold today" and update widgets to zero.
|
||||
|
||||
## When the user asks you things
|
||||
|
||||
- "What's in today's digest?" — read the top section of `digest.md` and summarize.
|
||||
- "Run the digest now" — do everything in the cron flow above, then summarize the results in chat.
|
||||
- "Why is [story] not in the digest?" — read the last 3–5 sections of `digest.md` and check whether the story appeared. If not, suggest the most likely cause (score below threshold, item type wasn't `story`, item appeared after the most recent run).
|
||||
- "Change the threshold" / "add a topic" — tell them: *"Click the Configuration button on the dashboard header (the slider icon, next to the folder). Adjust the values there and save. The next cron run will pick it up."* Don't try to edit config.json yourself.
|
||||
@@ -0,0 +1,40 @@
|
||||
# HackerNews Daily Digest
|
||||
|
||||
A minimal news-aggregation project that fetches HackerNews top stories once a day, filters them by score and (optional) topic keywords, and keeps a rolling markdown log + a live Scarf dashboard.
|
||||
|
||||
**Requires Scarf 2.3+** — uses the Configuration form during install and on-demand re-edit.
|
||||
|
||||
## What you get
|
||||
|
||||
- **Configurable score threshold** — only stories at or above this score show up. HN front page averages ~150; lower it to widen the net, raise it to focus on the truly viral.
|
||||
- **Configurable item cap** — keeps each digest from sprawling. Default 15.
|
||||
- **Optional topic keywords** — a list of keywords (case-insensitive substring match against titles). Items that match a keyword get a `[topic]` tag in the digest and `"ok"` status in the dashboard list. Empty list = include every story above threshold, no highlighting.
|
||||
- **No API keys** — HackerNews' Firebase API is fully public. Nothing in this project's `.scarf/config.json` is secret; no Keychain entries are created.
|
||||
- **`digest.md`** — agent's append-only log. New runs prepend at the top. Created automatically on first run.
|
||||
- **`.scarf/dashboard.json`** — live dashboard with stat widgets (top score, items tracked, last run) and a Top Stories list.
|
||||
- **Cron job `Daily HN digest`** — registered (paused) by the installer; tag `[tmpl:awizemann/hackernews-digest]`. Runs daily at 8:00 AM when enabled.
|
||||
|
||||
## First steps
|
||||
|
||||
1. During install, fill in the Configuration form — set `min_score`, `max_items`, and any topic keywords you care about. (All have sensible defaults if you just want to skip it.) Hit Continue, then Install.
|
||||
2. After install, open the **Cron** sidebar and enable the `[tmpl:awizemann/hackernews-digest] Daily HN digest` job. It's paused on install so nothing runs without your explicit say-so.
|
||||
3. From the project's dashboard, ask your agent to run the job now: *"Run the HN digest and update the dashboard."*
|
||||
4. Future runs happen automatically at 8 AM daily.
|
||||
|
||||
## Changing filters later
|
||||
|
||||
Click the **Configuration** button (slider icon, dashboard toolbar) to re-open the form pre-filled with your current values. Adjust score, max items, or topics. Save. The next cron run picks up the changes.
|
||||
|
||||
## Customizing
|
||||
|
||||
- **Change the schedule.** Edit the cron job in the Cron sidebar — accepts `30m`, `every 2h`, or standard cron expressions like `0 8 * * *`.
|
||||
- **Switch sources.** This template is HN-only by design. To pull from Lobsters, Reddit, or RSS, fork it (export from a Scarf project, edit `cron/jobs.json`'s prompt, re-import) — most of the agent contract is generic.
|
||||
- **Add alerting.** Set a `deliver` target on the cron job (Discord, Slack, Telegram) — the agent will post the run summary there instead of just writing to `digest.md`.
|
||||
|
||||
## Recommended model
|
||||
|
||||
`claude-haiku-4` works well — this is a simple HTTP-fetch + filter + markdown task. Haiku keeps costs low when the cron runs daily. The recommendation appears in the Configuration form; Scarf doesn't auto-switch your active model, so adjust via Settings if you'd like.
|
||||
|
||||
## Uninstalling
|
||||
|
||||
Right-click the project in the sidebar → **Uninstall Template…** (or click the shippingbox icon on the dashboard header). Scarf walks you through exactly what's about to be removed: template-installed files in the project dir, the `[tmpl:…]` cron job, and the configuration values you entered (`config.json`; this template stores no secrets so there's nothing in Keychain to clean up). User-created files (like `digest.md`) are preserved.
|
||||
@@ -0,0 +1,7 @@
|
||||
[
|
||||
{
|
||||
"name": "Daily HN digest",
|
||||
"schedule": "0 8 * * *",
|
||||
"prompt": "Generate the HackerNews daily digest for the Scarf project at {{PROJECT_DIR}}. Read {{PROJECT_DIR}}/.scarf/config.json to get `values.min_score` (number, default 100), `values.max_items` (number, default 15), and `values.topics` (array of strings, default []). Fetch the top story IDs from https://hacker-news.firebaseio.com/v0/topstories.json and take the first `max_items * 3` IDs (gives headroom for the score filter to drop low-scorers). For each ID, fetch https://hacker-news.firebaseio.com/v0/item/<id>.json and keep only `type==\"story\"` items with `score >= min_score` and a non-null `url` or `text`. Cap the surviving list at `max_items`. If `topics` is non-empty, mark each surviving item with a `[topic]` tag for the first matching keyword (case-insensitive substring match against the title). Build a markdown digest section with the ISO-8601 timestamp as the heading and one bullet per item (`- [<score>] <title> [<topic>]? — <url or HN comments link>`). Prepend that section to {{PROJECT_DIR}}/digest.md (create the file with a one-line header if it doesn't exist). Update {{PROJECT_DIR}}/.scarf/dashboard.json: set the `Top Story Score` stat widget's `value` to the highest score, the `Items Tracked` stat widget's `value` to the count of items, and the `Last Run` stat widget's `value` to the ISO-8601 timestamp. Replace the `Top Stories` list widget's `items` array with one entry per item (text = `[<score>] <title>`, status = `\"ok\"` if the item has a topic match else `\"pending\"`). Preserve every other field in dashboard.json as-is. Reply with a one-line summary like '12 items, top score 487 — \"<title>\"'."
|
||||
}
|
||||
]
|
||||
@@ -0,0 +1,63 @@
|
||||
{
|
||||
"version": 1,
|
||||
"title": "HackerNews Digest",
|
||||
"description": "A daily roll-up of HackerNews top stories above your configured score threshold. The stat widgets and Top Stories list update each time the cron job runs; the digest itself is prepended to `digest.md` in the project root.",
|
||||
"theme": { "accent": "orange" },
|
||||
"sections": [
|
||||
{
|
||||
"title": "Today's Digest",
|
||||
"columns": 3,
|
||||
"widgets": [
|
||||
{
|
||||
"type": "stat",
|
||||
"title": "Top Story Score",
|
||||
"value": 0,
|
||||
"icon": "flame.fill",
|
||||
"color": "orange",
|
||||
"subtitle": "highest-scoring item"
|
||||
},
|
||||
{
|
||||
"type": "stat",
|
||||
"title": "Items Tracked",
|
||||
"value": 0,
|
||||
"icon": "list.bullet.rectangle",
|
||||
"color": "blue",
|
||||
"subtitle": "above your score threshold"
|
||||
},
|
||||
{
|
||||
"type": "stat",
|
||||
"title": "Last Run",
|
||||
"value": "never",
|
||||
"icon": "clock",
|
||||
"color": "gray",
|
||||
"subtitle": "ISO-8601 timestamp"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"title": "Top Stories",
|
||||
"columns": 1,
|
||||
"widgets": [
|
||||
{
|
||||
"type": "list",
|
||||
"title": "Top Stories (populated after first run)",
|
||||
"items": [
|
||||
{ "text": "Run the digest once to populate — the agent reads your Configuration, fetches HackerNews' top stories, and fills this list.", "status": "pending" }
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"title": "How to Use",
|
||||
"columns": 1,
|
||||
"widgets": [
|
||||
{
|
||||
"type": "text",
|
||||
"title": "Quick Start",
|
||||
"format": "markdown",
|
||||
"content": "**1.** Review your configuration — click the **slider icon** (top-right of this dashboard) to open Configuration. Set `min_score`, `max_items`, and any `topics` keywords you want highlighted.\n\n**2.** Enable the `[tmpl:awizemann/hackernews-digest] Daily HN digest` cron job in the Cron sidebar. It ships paused — nothing runs until you say so.\n\n**3.** Ask your agent: *\"Run the HN digest now.\"* The Top Stories list populates, the stat widgets update, and a fresh entry lands at the top of `digest.md`.\n\n**4.** Daily at 8 AM the cron job fires automatically. Change the schedule in the Cron sidebar if you want a different cadence.\n\nSee `README.md` and `AGENTS.md` in the project root for the full spec."
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,60 @@
|
||||
{
|
||||
"schemaVersion": 2,
|
||||
"id": "awizemann/hackernews-digest",
|
||||
"name": "HackerNews Daily Digest",
|
||||
"version": "1.0.0",
|
||||
"minScarfVersion": "2.3.0",
|
||||
"minHermesVersion": "0.9.0",
|
||||
"author": {
|
||||
"name": "Alan Wizemann",
|
||||
"url": "https://github.com/awizemann"
|
||||
},
|
||||
"description": "A daily digest of HackerNews top stories. Pulls Hacker News' Firebase API, filters by minimum score and optional topics, prepends a markdown digest to digest.md, and keeps the dashboard's top stories list current. No API keys required.",
|
||||
"category": "news",
|
||||
"tags": ["news", "digest", "hackernews", "cron", "starter", "configurable"],
|
||||
"contents": {
|
||||
"dashboard": true,
|
||||
"agentsMd": true,
|
||||
"cron": 1,
|
||||
"config": 3
|
||||
},
|
||||
"config": {
|
||||
"schema": [
|
||||
{
|
||||
"key": "topics",
|
||||
"type": "list",
|
||||
"itemType": "string",
|
||||
"label": "Highlight Topics (optional)",
|
||||
"description": "Keywords or phrases to highlight in the digest (case-insensitive substring match against story titles). Leave empty to include every top story above the score threshold.",
|
||||
"required": false,
|
||||
"minItems": 0,
|
||||
"maxItems": 20,
|
||||
"default": []
|
||||
},
|
||||
{
|
||||
"key": "min_score",
|
||||
"type": "number",
|
||||
"label": "Minimum Score",
|
||||
"description": "Only include stories at or above this point score. HN's front page averages ~150; lower this to widen the net, raise it to focus on viral-only items.",
|
||||
"required": false,
|
||||
"min": 1,
|
||||
"max": 1000,
|
||||
"default": 100
|
||||
},
|
||||
{
|
||||
"key": "max_items",
|
||||
"type": "number",
|
||||
"label": "Maximum Items",
|
||||
"description": "Cap on how many stories appear in each digest. Avoids blowing up the dashboard list when HN has a busy day.",
|
||||
"required": false,
|
||||
"min": 5,
|
||||
"max": 50,
|
||||
"default": 15
|
||||
}
|
||||
],
|
||||
"modelRecommendation": {
|
||||
"preferred": "claude-haiku-4",
|
||||
"rationale": "Simple HTTP fetch + filter + markdown render. Haiku is plenty fast and the cheapest option for a daily run."
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user