Users and customers are sending their messages or reviews from their devices. There are several such messages streaming from different users into the system. We must first be able to ingest these messages in real time manner. Further we should be able to processes every single message and take corrective action as needed. The processes would include following:

  1. Set the streams and sliding window and ingest the data in these streams in continuous manner.
  2. Find out the sentiment of the message [ positve, negative ] using IE (information extraction).
  3. Filter messages with negative sentiment and put them in separate stream for further action / processing.
  4. Find out a definitive pattern and send such events matching the patter to another stream for further review / action. The patter is as follows:

    5. Any particular product that gets minimum 3 consequtive negative sentiment messages from different users in span of 1000 sec, find this pattern in continuous sliding manner.

  5. Store few tripples in graph store like (user, POSTS_REVIEW, prod) and (prod, HAS_REVIEWS, revid), revid id review id and prod is product.
  6. Set running stats for different attributes in the event such as uniquw count for users or min/max/avg/sdtdev/sum/kurt for amount spend etc.
  7. Set up reverse index for messages such that it can used for text search by the user.
  8. Set up secondary indexes for several attributes that could be helpful in query and also internal stream joins/ filter etc.