Skip to content
Easy AI How-To

AdvancedDGX SparkBuilding

Hybrid log watching on a DGX Spark: regex counters plus Nemotron 3 Nano Omni on a real 1M-lines/hour SIP proxy log

AI engineering is just process engineering. — Greg Alton

My last recipe ended with a promise: try a smaller model, and try a real busy log. The synthetic test said the hybrid pattern works: cheap counters read every line, and a model explains what tripped. But synthetic logs are polite. Real ones aren't.

So I switched the Spark from Nemotron 3.5 Super VL to the smaller Nemotron 3 Nano Omni, pointed the watcher at a live SIP proxy log moving at more than a million lines an hour, and let it run for 15 minutes. I bought my DGX Spark to learn by doing. Here's what I learned, including the parts that didn't work.

Time
About 25 minutes
Checked
Tools
vLLM, vLLM structured outputs, NVIDIA DGX Spark playbooks

If a button name on the screen does not match this guide, follow the action. These products rename buttons often. This is an advanced guide: it uses code, and each technical term is explained the first time it appears. Model names and prices change, so check the linked docs before you ship.

Steps

  1. The setup

    In the first recipe, a vision model couldn't keep up with a fast-scrolling terminal, but counters plus a model could. That was on synthetic logs. This time I pointed the same pattern at a live SIP proxy log, read-only.

    • The stream: ssh ... tail -F on the log, piped to the Spark. Nothing was written on the server, and the raw lines stayed on the Spark.
    • The counters: the regex watcher from the first recipe, running on every line.
    • The model: Nemotron 3 Nano Omni (NVFP4), served by vLLM on the Spark. I swapped it in for Nemotron 3.5 Super VL because the first test showed the model was the bottleneck.
    • The trigger: when a counter trips (and every N seconds for context), send the model a recent sample and ask for a JSON verdict.
    • The run: 15 minutes.

  2. The results

    Measure Result
    Lines in 15 minutes about 292k
    Average rate 324 lines/s (about 1.17M lines/hour)
    Watcher CPU about 0.6% (1.2% for the whole system)
    Counter trips 12
    Time from trip to model verdict 0.75–2.35 s
    Image call, Nano Omni 1.12 s average (median 0.72 s, p90 1.61 s)
    Image call, Super VL (earlier) about 2.4–4.1 s
    Text call, Nano Omni vs Super VL 0.72 s vs 1.16 s
    GPU use 8% average, 96% peak during calls
    Invalid JSON replies 12%

    The counters read every line of a real 1M-plus-lines-an-hour log for less than 1% CPU. The model answered every trip within about 2.4 seconds. The smaller model was roughly 2–3 times faster on images, and the GPU sat mostly idle between calls.

  3. The weak spots

    It wasn't all good news, and this is the useful part.

    • The error counter was mostly noise. It matched 1,345 lines. About 65% were harmless debug lines that just contain the word "error." Only about 35 were true error-level lines, plus 6 warnings. A word match isn't a severity.
    • The model mostly echoed the counters. Ask it "these counters tripped, what's going on?" and it tends to say the counters tripped. That adds little.
    • 12% of replies were invalid JSON. Each one needs a retry or a fallback, or an alert gets dropped.
    • The log is big. It rotates daily at about 3–4 GB uncompressed per day, with no size cap. Anything that re-reads whole files will hurt, so stream new lines only.

  4. Fix 1: level-aware regex

    Match on the log level field, not on a word anywhere in the line. Adjust the pattern to your log's format:

    import re

# Match the level token near the start of the line, not "error" anywhere.
LEVEL = re.compile(r"^\S+\s+\S+\s+\[?(DEBUG|INFO|NOTICE|WARN(?:ING)?|ERROR|CRIT(?:ICAL)?|FATAL)\]?\b", re.I)

def level_of(line):
    m = LEVEL.search(line)
    return m.group(1).upper() if m else None

# Count ERROR and above for alerts. Ignore DEBUG entirely, even if it says "error".
ALERT_LEVELS = {"ERROR", "CRIT", "CRITICAL", "FATAL"}

    In this run, that change alone would have turned 1,345 "errors" into about 35 real ones.

  5. Fix 2: guided JSON in vLLM

    Don't hope for JSON; constrain it. vLLM's OpenAI-compatible server supports structured outputs, so you can pass a JSON schema and the decoder can only produce valid output:

    import requests

SCHEMA = {
    "type": "object",
    "properties": {
        "verdict": {"type": "string", "enum": ["ignore", "watch", "page"]},
        "likely_cause": {"type": "string"},
        "new_pattern": {"type": "string"},
        "next_check": {"type": "string"},
    },
    "required": ["verdict", "likely_cause", "new_pattern", "next_check"],
}

r = requests.post("http://localhost:8000/v1/chat/completions", json={
    "model": "your-model",
    "messages": [{"role": "user", "content": prompt}],
    "response_format": {"type": "json_schema",
                        "json_schema": {"name": "verdict", "schema": SCHEMA}},
    "temperature": 0, "max_tokens": 300,
}, timeout=60)

    Check the vLLM structured outputs docs for the options your version supports. Keep a fallback anyway: the counter's own alert should go out even if the model fails.

  6. Fix 3: ask for what counters can't tell you

    The counters already know how many. Don't ask the model to recount. Ask for what needs judgment:

    • Is this new? Does the sample look different from normal traffic?
    • What's the likely cause? For example, one client retrying, or many failing at once.
    • What should a human check next?
    • Is there a pattern worth a new counter? The best output is a regex you can add, so the cheap layer gets smarter over time.

    Send the counter totals as context and say plainly: "Do not repeat these numbers."

  7. The takeaway

    The counters do the heavy lifting, and the model adds context. The counters read every line, cost almost nothing, and never miss. The model is fast enough now (about a second a call on Nano Omni) to explain every trip, but it only earns its place when you ask it questions the counters can't answer.

    Next I'll rerun this with level-aware counters, guided JSON, and the new prompt, and see if the model starts adding real insight. Follow the DGX Spark section for the next recipe.

Tips

  • Stream with tail -F, not tail -f, so the watcher survives daily log rotation.

  • Keep it read-only. The server only runs tail; all parsing and the model run on the Spark.

  • Keep logs local. Raw lines, addresses, and numbers never leave your network.

  • Pick the smallest model that answers fast. The model was idle most of the run, so speed per call matters more than size.

Common questions

Why not let the model read every line?

At about 324 lines a second, that's far more than a model can read in real time. Regex counters handled it for under 1% CPU.

Text or screenshot for the sample?

Text when you have it. Text calls were faster (0.72 s on Nano Omni) and can't be misread. Use screenshots only when a screen is all you have.

Was the model useful at all?

It was fast and reliable about responding, but it mostly repeated the counters. That's a prompt problem, not a speed problem, and it's what Fix 3 is for.

More guides

Occasional notes

Hear when a new guide is up

A short note when a guide is added. No account, and no emails until this form is connected to a real list.

Placeholder. This form is not connected to an email service. Nothing you type is saved or sent.