Coordinator / NOC Operator
Coordinator and NOC Operator are the same role in this app — one person, one login, both jobs. This is your daily routine end to end: the shift schedule, the triage flow, the ticket pipeline, and the rules that make a day count as successful. Source: NOC Daily Playbook v2.
Your shift, in six steps
Open the NOC Dashboard (sidebar → Network Operation Center → your project → NOC Dashboard) and keep it on screen all shift. These six steps repeat every day, in this order:
Shift start · first 30 min
The Sweep
Check the NMS is polling — broken monitoring makes everything below fiction.
Count: N down out of N total. Real numbers.
Cross-check every down site against its provider cloud. This is the job.
Cluster by area, provider, or utility before opening anything.
Check weather and utility advisories.
Clear the "needs a ticket" counter — every site past 24h gets one today.
Cause-code everything, even if provisional.
Nobody watched the network overnight. This is the most important half hour of your day.
+0:30 · every day
Morning Report
Sites up / down out of total.
NMS health, and any NMS-vs-provider discrepancies.
Down sites grouped by cause code, not listed one by one.
Sites past 24h and whether each has a ticket.
Any cluster: area named, external reference, map attached.
Carried over from yesterday, plus next action.
Send it on quiet days too. "All up, nothing carried over" is a complete report.
All shift
The Loop
Always: NMS on screen. New offline site → run the triage flow.
Hourly: rescan the down list, advance the 24h clocks, chase what has not moved.
Every 2h: status line on any open ticket for a 24/7 site or a cluster.
Chase any ticket sitting in one pipeline stage without activity.
Acking alarms then going quiet until 5pm is watching, not monitoring.
Midday · 15 minutes
Reconcile
Past 24h with no ticket? Raise it now.
Ticket open but site recovered in both sources? Close it with a cause.
Still tagged UNK? Investigate or escalate.
Any ticket untouched 48h? Chase or escalate.
Spreadsheet disagrees with the ticket? The ticket wins.
Fifteen minutes here saves an hour at close.
1 hour before close
Pre-Close
Resolve anything an hour of work would close.
Escalate every 24/7 site still down, and every ticket that will breach, now.
Tell affected sites what's down overnight.
Chase carriers and coordinators one last time; log their commitment.
Write tomorrow's first action on every open ticket.
Nothing enters the unattended overnight window without a superadmin knowing.
Last 15 minutes
Handover
Lead with what stays down overnight and whether the site was told.
Sites whose 24h clock expires tomorrow — flag them for the next operator.
Open tickets, their stage, and the first action tomorrow.
Say "nothing outstanding" explicitly if true.
Post it and get it acknowledged before logging off (Helpdesk → Handover tab).
No handover = incomplete shift, however good the network was.
The triage flow: verify before you act
We are a network monitoring center, not an end-of-day monitoring center. A site showing down in the NMS is not yet an outage — it only becomes one after it survives verification and its hold window. Every offline site goes through this, in order:
- Site shows OFFLINE in the NMS. Note the site code, group, provider, and offline timer. Don't act on the NMS reading alone.
- Cross-check the provider cloud. Open the matching portal — Ruijie, Huawei, Omada, or Starlink — and check the same device. This is the verification step and it is never skipped.
Down on both
Real outage → go to step 3
Down on NMS, up on provider
Discrepancy → not a site outage
NMS-DISC, report it in the morning report, and escalate to a superadmin if more than two appear in a day.Provider portal unreachable
Treat as down on both
- Check the site's schedule tag. The hold rule depends entirely on this — check it before starting any clock.
Runs 24/7
Ticket immediately. No hold.
Scheduled
Inside its window? No ticket.
Standard site
Start the 24-hour hold.
- Still down at 24 hours → raise the ticket. The NMS already flags these for you: "Offline > 24h, needs a ticket." That counter should be zero every time you finish a sweep — it's the single clearest measure of whether the NOC is keeping up.
Why we still ticket intentional shutdowns
SCHED-OFF with the site's confirmation is a successful outcome, not a wasted ticket — it turns an unexplained gap into a documented one, and protects us when uptime is questioned.The color bar: which down sites need you
In the Sites tab of the NOC Dashboard, each site row can carry a colored bar down its left edge — the triage flow above, made visible at a glance. Hover the site code to see the exact reason.
- Amber — needs a ticket. The site has been offline more than 24 hours and has no open ticket. This is your cue to open one.
- Blue — tracked. There's an open ticket and it has had activity in the last 48 hours. It's covered and moving; no action needed.
- Red — neglected. There's an open ticket but nobody has touched it in 48 hours (napapabayaan). Chase it — add an update, move it forward, or escalate.
A down site with no bar is either offline under 24 hours (too early to require a ticket) or its ticket is already in Review or Done. Don't open a second ticket for a site that already shows blue or red.
Where the ticket goes: the helpdesk pipeline
Every ticket moves through five stages. In this app, the New and Coordinator stages are both worked by you — Coordinator and NOC Operator are the same role, not a handoff between two people. There's also no separate "NOC Lead" title: Review and escalation authority sit with a coordinator or superadmin.
| Stage | Owner | Moves on when |
|---|---|---|
| New | You (NOC Operator) | Ticket created with site code, offline duration, both-source verification result, and cause code (or UNK). |
| Coordinator | You (Coordinator) | Site or school contacted. Outcome recorded: intentional shutdown, power issue, or fault requiring intervention. |
| IT | IT Support | Remote diagnosis and remediation attempted and logged. Escalates to Installer if hands are needed. |
| Installer | Field | Onsite visit completed, findings recorded, service restored or blocker documented. |
| Review | Coordinator / Superadmin | Real cause code, usable resolution note, site confirmed back online in both sources. |
Watch this
Cause codes: every ticket gets one
Provisional is fine while a ticket is open — but it needs a real code, not UNK, before it closes.
SCHED-OFFConfirmed intentional shutdown, acknowledged by the siteNMS-DISCDown on NMS, up on provider cloudPWR-COMCommercial power out at sitePWR-SITEUnplugged, UPS, battery, breakerWX-RAINRain or floodingWX-STORMTyphoon or severe weatherSAT-OBSStarlink obstruction or rain fadeFIB-CUTFiber or cable cutCAR-FAULTCarrier fault, needs their ticket refEQP-FAILHardware failureCFG-CHGCaused by a change, ours or theirsSITE-ACCSite access refused or unavailableCLI-SIDEClient equipment or client actionMNT-PLNPlanned maintenanceUNKOpen only — never a closing codeThe rules, in one place
| Rule | Meaning |
|---|---|
| Verify twice | NMS reading alone is never enough. Provider cloud confirms or contradicts it. |
| 24-hour hold | Standard sites only. Recovered inside 24h → log it, no ticket. |
| 24/7 sites | No hold. Ticket the same day it drops. |
| Scheduled sites | Inside window → no ticket. Outside window → treat as standard. |
| Ticket anyway | Even confirmed intentional shutdowns get a ticket, closed with acknowledgement. |
| 48-hour rule | No ticket sits in one stage without activity for 48h. |
| Ticket is truth | If it's not in the ticket, it did not happen. The spreadsheet is output, not a record. |
Never
UNKWhat every ticket must contain
| Field | What |
|---|---|
| Site code + name | e.g. PP-BATS-2026-011 |
| Offline since | Timestamp, not "yesterday" |
| Verified | NMS + which provider portal, and what each showed |
| Schedule tag | 24/7, scheduled, or standard |
| Cause code | Provisional is fine on open |
| Contact attempts | Who, when, outcome |
| Next action + owner | Always populated |
One line to remember
Diagnose, then escalate
For each down site, work the cheapest action first:
Check the provider cloud / remote reset
Coordinator → IT Support
Look at the provider cloud/controller for the device's status and logs. If it is reachable or flapping, attempt a remote reset/reboot. Most outages clear here.
Call the site contact
Coordinator
If the remote reset does not bring it back, call the beneficiary/site contact to ask what is happening on the ground (power brownout, weather, unplugged router) and have them power-cycle the router.
Recommend a site visit
Dispatch Installer
ONLY if the remote reset failed AND an on-site restart also did not work, or the contact reports physical/hardware damage. Only then send a technician.
Escalation here means the response ladder above — remote reset, then a call, then (only as a last resort) dispatch. Separately, the table below is about visibility: when a coordinator or superadmin needs to know something is happening, regardless of whether it's resolved yet. When you escalate, hand off the full picture — project, site/group, device, how long it's been down, what you already tried, and anything the beneficiary reported.
| Escalate when | Timing |
|---|---|
| 24/7 site down | Same day |
| 3+ sites down in one area | Immediately |
| More than 2 NMS discrepancies in a day | Same day |
| NMS itself is down or not polling | Phone, now |
| Ticket neglected 48h | Same shift |
| Site unreachable / won't acknowledge | After 2 attempts |
| Site told us before we told them | Same shift, logged as a miss |
| Anything open at close of business | 1 hour before close |
Area-wide outages behave differently
When three or more sites within about 10 km go down together, the Outage Triage flags it as an area-wide cluster. These almost always come from a shared cause — a power utility outage or a backhaul provider issue — not the sites themselves.
For clusters, hold dispatch and coordinate with the power/backhaul provider. Sending an installer to one site won't fix a regional outage. Escalate immediately — see the table above.
Before you log off
You had a successful day if
UNKWhat success is not