Stream processing costs: Azure Stream Analytics vs Managed Flink vs Dataflow vs Oracle Stream Analytics
Stream processing is what turns raw telemetry into something useful while it is still fresh: a vibration alert within seconds, a 5-minute average per machine, a count of rejected parts per shift. Unlike a database query, a streaming job never stops, so you pay for it every hour of the month.
Azure Stream Analytics: streaming units
Stream Analytics jobs are written in a SQL-like language and sized in streaming units (SUs). You pay per SU per hour, and on the Standard V2 plan the hourly price steps down as your monthly SU-hours grow, so larger or longer-running deployments pay less per unit.
- Simple to size: one number, the SU count, which you raise when a job falls behind.
- SQL only for most jobs (with JavaScript or C# functions for extras). That’s quick for filters and windowed aggregates, but limiting for complex logic.
Amazon Managed Service for Apache Flink: KPUs plus extras
AWS runs Apache Flink for you, sized in Kinesis Processing Units (KPUs). One KPU is 1 vCPU and 4 GB of memory, charged per hour. Two extras are easy to miss:
- One additional KPU per application for orchestration, charged whether your job is busy or not. A small job using one KPU actually pays for two.
- Running application storage, 50 GB per KPU, charged per GB-month.
Flink is very capable (Java, Python or SQL, exactly-once processing, large state), which makes it the strongest choice for complex jobs, but the per-application overhead makes many small jobs comparatively expensive. Combining related logic into fewer applications helps.
Google Cloud Dataflow (streaming): vCPU, memory and data processed
Dataflow runs Apache Beam pipelines. For streaming jobs you pay:
- Per vCPU-hour and per GB-hour of memory for the worker machines;
- Streaming Engine data processed, per GB, for the shuffle and state work Google runs outside your workers.
Dataflow autoscales workers with load, which helps spiky traffic, but a streaming job always keeps at least one worker running.
Oracle Stream Analytics: OCPUs on OCI GoldenGate
On Oracle Cloud, stream processing runs as Oracle Stream Analytics, part of OCI GoldenGate. Pipelines are built in a visual designer, and you pay per OCPU-hour for the deployment that runs them (one OCPU is two vCPUs). There are no separate charges for data processed.
- One number to size, like Azure’s streaming units: the OCPU count.
- The smallest deployment is one OCPU, all month, which is the floor for a small job. Check the per-OCPU rate against the other services’ minimums before choosing it for a small fleet.
The minimum you always pay
Each of these charges for the job running, not for the data arriving, so a quiet factory at night costs the same per hour as a busy one at noon. For small workloads, the minimum job size usually decides the bill:
| Service | Smallest always-on job |
|---|---|
| Azure Stream Analytics | The smallest SU size, all month |
| AWS Managed Flink | 1 KPU plus 1 orchestration KPU, plus storage |
| Google Dataflow | One worker’s vCPUs and memory, plus data processed |
| Oracle Stream Analytics | 1 OCPU, all month |
Do you need it at all?
Not always. If you only need dashboards and alerts that can be a minute or two behind, the time-series database can often compute rolling averages itself, and the stream processing line disappears. That’s why our estimator lets you switch stream processing off and see the difference.
Edge processing is another option: filtering and aggregating on the factory floor before data reaches the cloud. See edge runtimes compared.