SCADA historian, disturbance records and power-quality data serve different time scales and evidence needs. A one-second trend cannot replace a COMTRADE waveform, and a relay disturbance file is not a fleet historian. Integration should link these records by asset, event, source time, quality and configuration without resampling away the evidence.
This guide designs the data pipeline for MV IEDs, gateways, station servers and enterprise analytics using IEC 61850 reporting/files, IEC 60255-24 COMTRADE and IEC 61000-4-30 power-quality methods.
1. Separate the data products
| Product | Typical content | Main use |
|---|---|---|
| SCADA historian | Periodic/change analogs, statuses, alarms and quality | Operations, trends, KPI and asset context |
| SOE/event log | Source-time discrete events and command/audit sequence | Fault/switching reconstruction |
| Disturbance record | High-rate waveforms and digital channels | Protection/fault/breaker analysis |
| Power-quality record | Frequency, magnitude, dips/swells, interruption, unbalance, harmonics etc. | Compliance, supply quality and root cause |
| Condition/diagnostic data | Operations counts, coil/travel signatures, temperatures and health | Maintenance and reliability |
2. Common identity and metadata
- Stable station/voltage/bay/equipment/function/channel identity.
- IED/device serial, firmware, settings group/configuration revision and file origin.
- CT/VT/sensor ratios, phase, units, scale, sampling/report interval and data type.
- UTC source time, time quality/accuracy and receipt/ingest time separately.
- Process quality, test/substituted/blocked state and communication status.
- Trigger/cause, pre/post-fault duration and related protection/command event ID.
- File/hash/retention/access classification.
3. Historian acquisition
- Use IEC 61850 reports or gateway telecontrol data with source quality/time preserved.
- Apply deadband/change reporting for operations while retaining periodic integrity/freshness.
- Store quality with every sample; do not turn communication gaps into flat healthy lines.
- Record alarm acknowledgments/shelving/config changes in an audit/event store.
- Define resampling/aggregation functions and keep raw values for the required period.
- Stagger backfill/integrity loads and alarm queue overflow.
- Time-series compression must not remove breaker transitions or short PQ events from their proper record type.
4. COMTRADE disturbance records
- IEC 60255-24 defines COMTRADE exchange for transient waveform/event data; retain complete file set or CFF as applicable.
- Verify channel names, units, ratios, primary/secondary flag, sampling rates and time stamps.
- Capture pre-fault, fault, trip and post-fault duration sufficient for the study.
- Use trigger classes for trip/start/manual/threshold and avoid overwriting during event storms.
- Retrieve files without starving IED protection/reporting or station bandwidth.
- Hash/archive originals read-only; analysis conversions remain traceable.
- Correlate waveform fault inception with SOE protection start/trip and breaker contacts.
5. Power-quality integration
- Specify measurement class/method and parameters required from IEC 61000-4-30:2025.
- Use IEC 62586-2 functional/uncertainty evidence for instruments claiming class A/S functions where applicable.
- Keep event magnitude/duration, aggregation interval, uncertainty and transducer influence.
- Store waveform snapshots where needed for dips/swells/transients; a scalar event table alone may be insufficient.
- Coordinate thresholds with nominal/declaration values and site purpose.
- Distinguish monitor time/source failure from a real PQ interruption.
- Do not label relay-derived trends “compliant PQ” without declared tested method/class.
6. Event correlation model
- Create a station/event ID linking SOE, alarms, commands, COMTRADE and PQ records.
- Use UTC source times with measured uncertainty; do not sort solely by arrival time.
- Correlate topology snapshot and settings/configuration active at the event.
- Preserve late/backfilled data and its ingest time.
- When time uncertainty overlaps, report indeterminate order.
- Keep analyst annotations separate from immutable original evidence.
7. Storage, retention and performance
| Data | Sizing driver | Retention decision |
|---|---|---|
| Historian analog/status | Points × update/change rate × quality/time | Raw then aggregates by operations/regulatory need |
| SOE/audit | Normal plus bus/DC/cyber event burst | Long enough for investigation/compliance |
| COMTRADE | Channels × sample rate × duration × fault frequency | Original files plus selected long-term cases |
| PQ | Parameters/aggregates/events/waveforms | Contract/regulatory/engineering purpose |
| Condition data | Operations/signatures/fleet frequency | Asset life and model-training traceability |
8. Buffering and backfill
- Define IED/gateway/station buffer responsibilities and capacity for the longest outage.
- Use sequence/entry/event identity to detect gaps and duplicates.
- Throttle file/backfill transfer so live alarms/controls retain priority.
- Alarm buffer overflow and mark the affected interval incomplete.
- Do not overwrite a newer live state with an older backfilled sample.
- Test WAN/server outage while multiple IEDs generate disturbance files.
9. Data quality and validation
- Validate range, units, phase, scale, sign and monotonic counter behavior.
- Persist IEC 61850/target protocol quality and time quality.
- Detect frozen/stale source independently of unchanged process value.
- Track data transformations, resampling, compression and corrections.
- Reconcile historian point list with SCD/gateway/control-center database.
- Use calibration/test evidence for measurements supporting compliance or maintenance decisions.
10. Cybersecurity and evidence integrity
- Separate operational acquisition from enterprise analytics through controlled conduits/DMZ.
- Use read-only/export paths where practical; analytics must not gain breaker-control privilege.
- Role-protect deletion, correction, retention and configuration.
- Hash/sign originals and maintain chain-of-custody for investigations.
- Protect credentials/API/file shares and scan transferred files without disrupting operations.
- Back up/restore metadata plus data; test ransomware/incident isolation.
- Log access/export/change and synchronize evidence systems with known time quality.
11. FAT/SAT
- Freeze point/channel metadata, time architecture, triggers and retention.
- Inject known analog/status/quality/time and verify historian values/flags.
- Create a controlled fault/trigger and retrieve/parse COMTRADE channel/time/scale.
- Generate PQ steady/event cases within test capability and verify method metadata.
- Correlate SOE, command, waveform and PQ by source time/event ID.
- Lose IED link, station server and WAN; test buffers, backfill, gaps and duplicates.
- Apply simultaneous disturbance-file/event storm and measure live SCADA performance.
- Test unauthorized read/write/delete and evidence audit.
- Restore the database/files/metadata on spare infrastructure.
- At SAT, repeat with installed CT/VT, IEDs, clocks, networks and enterprise conduit.
12. Acceptance deliverables
- Data architecture/flow and asset/channel dictionary.
- SCD/point list/COMTRADE/PQ metadata mapping.
- Time/quality/error and event-correlation method.
- Capacity/retention/buffer/backfill calculation.
- Cyber roles, integrity, backup/restore and evidence procedure.
- FAT/SAT sample records, queries, parses and recovery evidence.
13. Analytics and export guardrails
- Analytics must retain source identity, quality, missing intervals, unit/ratio and configuration version.
- Do not train condition models on substituted/test data unless explicitly labeled and intended.
- Validate calculated KPI/feature equations against engineering cases and known faults.
- Version algorithms and preserve the raw inputs needed to reproduce every result.
- Keep predictions/advisories separate from authoritative protection/control unless a separately validated closed-loop design exists.
- Use controlled APIs/exports with pagination, rate limits and access audit; a large enterprise query must not degrade station operations.
- Document uncertainty and false-positive/negative review responsibility before maintenance decisions depend on a model.
References and further reading
- IEC 60255-24:2013 — COMTRADE transient-data exchange
- IEC 61000-4-30:2025 corrected 2026 — Power-quality measurement methods
- IEC 62586-2 consolidated edition — PQ functional tests/uncertainty
- IEC 61850-8-1 consolidated edition — Reports and file transfer
- IEC 61850-7-3 consolidated edition — Data/quality classes
- IEC 61850-10 edition 2.1 — Conformance/performance testing
Engineering note: Keep original waveforms and source-time quality; trends and dashboards are derived views, not replacements for evidence.