An output plugin to write messages to Hadoop HDFS over WebHDFS.
Tested with Hadoop 2.7.0.
The WebHDFS implementation uses modified code from webhdfs-java-client.
To start using this plugin place this jar in the plugins directory and restart graylog server.
Following parameters can be configured while launching the plugin
- Host - IP Address or hostname of the HDFS name node
- Port - WebHDFS port configured in HDFS.
- Username - Username of pseudo authentication (currently kerberos is not supported)
- File path - Path of file to store the messages. File name can be formatted with message fields or date formats. E.g ${source}%Y%m_%d.log for storing the messages based source and day.
- Message Format - Format of message to be written. Can be formatted with message fields like ${timestamp} | ${source} | ${short_message}
- Flush interval - Interval in seconds to flush the data to HDFS. Value of 0 means immediate.
