How to avoid losing events kept in memory on application shutdown?

by antonro   Last Updated August 23, 2019 21:05 PM

An application App-a I maintain publishes some events to an event bus for other team's application App-z. So far, the events were published right after being generated.

Now, I need to delay events ~30 minutes before publishing them. This is fairly easy to implement, but my concern is, that when App-a shuts down (restart, new version, etc), which typically happens a couple of times a week, these events will be lost.

What are strategies to deal with this situation? I have a couple of ideas, none of which I like too much:

  1. Persist the events somewhere, then have a scheduled job check the event dates and publish the ones older than 30 minutes. Cons: so far the application App-a doesn't persist anything so I would need to create & manage a new storage, schema, driver... Plus I would also need a new application to run the scheduled job.
  2. Have an intermediate service App-k doing the delay. The original application App-a would publish them as currently, then App-k can consume the events, wait for 30 minutes, publish them for App-a, and only at this point acknowledge event consumption to the event bus, to avoid having the same problem (losing in-memory events on shutdown) in App-k. Our broker is not Kafka so acknowledging an event is not as simple as committing and offset, so I'm not even sure waiting 30 for the ack is possible.

Before I get into any complicated and potentially bad mechanism I would like to hear suggestions for this problem.



Related Questions



Advantages of Event Sourcing over history tables

Updated July 21, 2015 13:02 PM