The NiFi instance was down, there were no recent changes in any processor group. upon examining the logs it was found that List SFTP failed. But it is not clear why would it cause the instance to go down and crash it again as soon as it restarts ?
Upon further examination it was found that instance was crashing due to insufficient space on the disk. The error on ListSFTP Processor was recurring which caused continuous additions in the nifi-app.log file. The log file kept growing in size until the system crashed due to low disk space.
This can be resolved by using proper configuration settings for nifi logs in logback.xml file, located in “conf” directory.
By default the totalSizeCap property is not specified. It defines the size at which log roll back is started.
The following properties can be configured to handle roll back of log file -
Total size of all log files
No of files after which rollover starts
max file size per log file
Creates a new log file every hour
These settings ensure regular rollback of log files and prevents out of space issues on the disk.