Batch Processing

[It involves:]

1. Collecting and storing data over a period of time (hours, days, or even weeks).
2. Processing this data in bulk at scheduled intervals.
3. Producing some output data.

[Full Process]

1. Data Collection (File, Buffer, Warehouse, DB)
2. Pre-Process
3. Execution -> Usually in parts
4. Post Process (mark ACK)

Can Use: Hadoop (Map Reduce) | AWS Batch (AWS) | CRON (Custom)

Stream Processing

[Full Process]

1. Ingestion (Kafka Topics, Buffer)
2. Processing: Fillter -> Aggregate -> Windowing (stream - batching)

Can Use: Apache Kafka / Flink | AWS Kinesis