English 中文(简体)
How to Process Events from now() to -10 minutes in Flink
原标题:

I want to create a Stream Flink app that will process the events that falls within the latest event timestamp to different time windows, e.g. 10 minutes, 1 day and 1 month.

So lets say I got an event at 12:01, this is my latest timestamp, so the process will grab all events produced from 12:01 to 12:11. Same for other window sizes.

I am not sure I understand how windows work, but I get that I have a stream and that events will be processed in a window, but there are multiple outputs for a window, and windows have a defined start and end, not a moving start-end timestamp. I guess this would look like a sliding window, where the stream is a fixed size by timestamps, so new events pop out old events as they come if they are out of the 10 minute window.

Is this something I set up in flink or in my datastream (maybe kafka)?

I am relatively new to Flink, and I checked the sliding window documentation, but it seems it serves a different function.

问题回答

暂无回答




相关问题
How to Process Events from now() to -10 minutes in Flink

I want to create a Stream Flink app that will process the events that falls within the latest event timestamp to different time windows, e.g. 10 minutes, 1 day and 1 month. So lets say I got an event ...

Flink Batch vs Stream how they process real time data

I have read the documentation of streaming mode and batch mode. I assume that if I have an unbounded stream and I apply windows (like tumbling) on it it becomes a bounded stream? Please correct and ...

Flink failed initials checkpoints

I have a flink job deployed on a local kind cluster, it saves checkpoints to AWS S3. The following error kept occurring in job manager log at the initial stage: 2023-07-07 19:33:48,657 INFO org....

RocksDB state backend - WhenTableAPI used

We are using TableAPI to execute few SQL queries to achieve the desired results. Whehn creating Windows out of the streams, we observed OutOfMenmory issues. We enabled RocksDB as the state backend in ...

flink on k8s How to set the minimum and maximum CPUs

I am using flink on k8s, and now there is a problem, the dynamic resource feature of k8s cannot be utilized, one of my flink tasks only occupies a small amount of CPU resources when it runs normally, ...

热门标签