Implementing Error notifications in NiFi

Technical Discussion
#1

Implement error notification in NiFi

Problem

There is no inbuilt error notification system in NiFi. We need to notify with an email when any processor in the process group fails. It should also contain the cause of error and location of process group

Solution

Failure relationships (such as not found, comms failure, etc) from all processors are to be connected to one Error Handling Process Group. The process group will read the nifi bulletin to fetch error records and send them in a mail.

Error Error Handling Process Group

  1. Replace Text
    Replace entire content of flowfile with username and password from variables.

  2. Invoke Http
    Gets access token using nifi rest api and put it in a flow file attribute.

  3. Invoke Http
    Fetches data from nifi bulletin using the access token

  4. JOLT Transform
    - Bulletin contains warning, debugs, errors etc and holds the elements for 5 minutes
    - JOLT Filters ERROR elements in the array and fetches the latest element.

  5. Evaluate Json Path
    - Reads Processor and Process group Id from the bulletin content and adds them as flow file attribute

  6. Put Email

    • Attaches the flow file containing cause of error with the email
    • Creates a message body containing “Source of Error - url” using Processor ID and Process Group attributes.
    • Send the notification Email

Process Group xml - Error Handling Process Group

1 Like