Developers

Collectors & Schedules

Every number in this app was written by a scheduled job in yuno-nms-cron. This page says which job, when it runs, what it writes, who reads it, and whether it ran.

Live status

The latest run of each tracked job, straight from the jobLogs table the jobs write to. If a card is amber or red, the numbers that job feeds are stale — check here before chasing a “wrong” uptime figure.

Sign in to see the live status of each job here — last run, how long it took, whether it is overdue.

What the collectors are

This app never calls Ruijie, Huawei or Omada when you open a page. A separate service, yuno-nms-cron, does that on a schedule and writes what it learns into the shared database; the app only reads. So every figure on screen has an age: it is as fresh as the last run of the job that wrote it.

There are two families. The hourly poll keeps “right now” current all day. The overnight batch (weather, traffic, users, device logs) fills in yesterday between midnight and dawn. And there is one job that runs hourly but produces a daily number — the report card — which is the one people get confused by, so it has its own section below.

What runs when

One day, Philippine time. Each bar is a run; its length is how long the job takes. The red line is now.

PHT
0003060912151821

Hourly poll

hourly at :30

Daily report card (current)

hourly at :00

Daily report card (shadow)

hourly at :55

Weather

daily at 00:10 PHT

Traffic

daily at 01:00 PHT

Users

daily at 02:00 PHT

Device logs

daily at 03:00 PHT

Sites sync

daily · time not read off the Scheduler

Devices sync

daily · time not read off the Scheduler

Route health

daily · time not read off the Scheduler

Project behaviours

weekly · time not read off the Scheduler

UTC
1619220104071013
scheduled runproposed, not scheduled yetnow (PHT)Blank row: time not yet read off the Scheduler.

Observed, not authoritative

Times marked observed were read off database row timestamps, not off the Scheduler. The Scheduler dashboard is the only source of truth for when a job fires. When the cron moves to Cloud Run, paste the Cloud Scheduler list into lib/collectors.ts and these bars become exact.
Blank rows are jobs nobody has read the time for yet: sites sync, devices sync, route health, project behaviours.

The two uptime jobs

current-uptime and uptime are the two names everyone mixes up. One is status now; the other is a score per day. Both run every hour.

Hourly poll · /cron/current-uptime

The headcount

“What is the status of every device right now?”

RunsEvery hour at :30, ~17 min
AsksEach site: which devices are on, and since when — one question per site
WritesOne row per device per hour (the timeline) and one 'right now' row per device
FeedsDevice lists, online/offline badges, helpdesk context, the AI daily digest

If someone says 'polling', this is the job. Rebuilt in PR #34; being renamed to poll.

Daily report card · /cron/uptime

The score per day

“How many hours was each device up on each day?”

RunsEvery hour at :00 — 'daily' is what it produces, not how often it runs
AsksToday: every device individually, ~3,400 calls, ~3 h. Soon: nothing — it will read the poll's own rows in seconds
WritesOne row per device per day, one average per site per day
FeedsDashboard uptime %, NOC uptime, observer, reports and billing, AI insights

Billing is per AP: every device keeps its own daily hours. That does not change when the job is rebuilt.

What is changing

The report card is being rebuilt to read the poll's own rows instead of asking the vendor about every device again: seconds instead of hours, and no lost hours when a run is killed. It is writing to shadow tables and being diffed against the current job for a few days before anything switches.
Per-device data does not change. AP1, AP2, the switch, the router — each keeps its own row and its own hours. Billing is per AP; that is the grain and it stays.
The hourly poll now also refreshes the online/offline badge on the devices table, so a badge is at most an hour old instead of up to three.

Where each number comes from

Read left to right: the cloud a job reads from, the job, the tables it writes, the screens that read those tables. To answer “why does the dashboard say 92 %”, find Dashboard uptime % on the right and follow the row back.

Read from

RuijieHuaweiOmada

Job

Hourly poll

/cron/current-uptime

Writes

deviceUptimeHourlyLogsdeviceCurrentUptimesdevices.onlineStatus
Read bySite device listDevice uptime pagesHelpdesk ticket contextAI daily digestOnline/offline badges

Read from

RuijieHuaweiOmada

Job

Daily report card (current)

/cron/uptime

Writes

deviceUptimeSnapshotsgroupUptimeAveragesdevices.onlineStatus
Read byDashboard uptime %NOC uptimeObserverReports & billingGroup uptime pageAI insights

Read from

nothing — reads the poll's own rows

Job

Daily report card (shadow)

/cron/daily-uptime

Writes

deviceUptimeDailyShadowgroupUptimeDailyShadow
Read byNothing yet — diffed against the current report card

Read from

Google Weather

Job

Weather

/cron/weather

Writes

localityWeatherDays
Read bySite behaviour analysisAI insights

Read from

RuijieHuaweiOmada

Job

Traffic

/cron/traffic

Writes

deviceTrafficSnapshotsgroupTrafficSummaries
Read byTraffic chartsReports

Read from

RuijieHuaweiOmada

Job

Users

/cron/users

Writes

usersuserActivitydeviceUserSnapshotsgroupUserSummaries
Read byUser countsReports

Read from

Omada

Job

Device logs

/cron/device-logs

Writes

deviceAlertLogs
Read byDevice alert history

Read from

RuijieHuaweiOmada

Job

Sites sync

/cron/groups

Writes

groupsgroupSiteCodeHistory
Read byEverything — sites are the spine

Read from

RuijieHuaweiOmada

Job

Devices sync

/cron/devices

Writes

devicesnotifications
Read byDevice listsNotifications

Read from

This app

Job

Route health

/cron/route-health

Writes

notifications
Read byNotificationsRoute planning

Read from

This app

Job

Project behaviours

/cron/project-behaviors

Writes

projectBehaviors (via dict-api)
Read byProject behaviour insights

Every job

JobRouteCadenceTakesTrackedNote
Hourly poll/cron/current-uptimehourly~17 minjobLogsBeing renamed to `poll`; `/cron/poll` already answers. ~17 min from Heroku US, ~3 min once the cron runs in Seoul next to the database.
Daily report card (current)/cron/uptimehourly~3 hjobLogsOne copy at a time since PR #34; a copy takes ~3 h, so the day's score lags. Being replaced by `daily-uptime` once the shadow diff is accepted.
Daily report card (shadow)/cron/daily-uptimehourly~1 minjobLogsScheduler entry pending (hourly at :55). Rolls up yesterday and today each run.
Weather/cron/weatherdaily~2 minlogs only00:10 PHT is a requirement, not a slot: Google only serves the last 24 h, so a later run permanently caps the day.
Traffic/cron/trafficdaily~3 hlogs only
Users/cron/usersdaily~11 hlogs onlyThe longest job — runs into the working day. Not yet tracked in jobLogs.
Device logs/cron/device-logsdaily~30 minlogs only
Sites sync/cron/groupsdaily—logs only
Devices sync/cron/devicesdaily—logs only
Route health/cron/route-healthdaily—logs only
Project behaviours/cron/project-behaviorsweekly—logs only

Tracked means the job writes one row per run to jobLogs with a heartbeat, so a killed or skipped run is visible and the live strip above can show it. The overnight batch is not tracked yet; its only trace is the server log and the row timestamps it leaves behind.

When something looks wrong

  1. Check the live strip first. A stale or failed job explains most “the number is wrong” reports. Note the job's name and the time.
  2. Find which job feeds the screen in the flow diagram. A wrong uptime % on the dashboard is the report card; a wrong online/offline badge is the hourly poll.
  3. Look at the run in jobLogs: status, duration, error counts by type, the slowest calls, and where a still-running job is. The cron README has the queries.
  4. Re-run one project before re-running everything: /cron/projects/<id>/current-uptime is tracked separately and never blocks the schedule.

Do not

Trigger a job by hand while the same job is running — it will be refused, and if it were not, two copies would double the vendor calls.
Change when a job runs without reading its note first. The weather job's time is a requirement; the poll and the report card are spaced so they do not collide.
Add a vendor call inside a per-device loop. The report card takes three hours today for exactly that reason.

Where the truth lives

  • The cron repo README (yuno-nms-cron/README.md) — the engineering description of every job, its tables, its budget and its environment variables. Start with the section “What each job is (plain English)”.
  • The Scheduler dashboard (Heroku today, Cloud Scheduler after the move) — the only authority on when a job fires.
  • lib/collectors.ts in this repo — the data behind this page: names, times, tables, readers. Edit it when a job changes, and the clock, the flow and the table follow.
  • jobLogs — what actually happened, run by run, for the tracked jobs.