GrayLog setup

-Set the server for elasticsearch. (System parameters same as new cluster)

Steps to install elastic:

 wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.6.1.tar.gz
 tar xvf elasticsearch-1.6.1.tar.gz
 elasticsearch-1.6.1/bin/plugin -install mobz/elasticsearch-head
 elasticsearch-1.6.1/bin/plugin -install lukas-vlcek/bigdesk
 elasticsearch-1.6.1/bin/plugin install elasticsearch/elasticsearch-lang-mvel/1.6.0

• Steps to install mongo *

 wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.0.5.tgz
 tar -zxvf mongodb-linux-x86_64-3.0.5.tgz
 mkdir -p mongodb_graylog
 cp -R -n mongodb-linux-x86_64-3.0.5/ mongodb_graylog
 export PATH=/opt/mongodb_graylog/bin:$PATH

Steps to install Graylog server

 wget https://packages.graylog2.org/releases/graylog2-server/graylog-1.1.6.tgz

tar xvfz graylog-1.1.6.tgz

 cd graylog-1.1.6
 mkdir config
 cp graylog.conf.example config/server.conf

Steps to install GrayLog WebInterface

 wget https://packages.graylog2.org/releases/graylog2-web-interface/graylog-web-interface-1.1.6.tgz
 tar xvfz graylog-web-interface-1.1.6.tgz

=================================

setup Forwarder as follows:

Pre Requisite : Java 1.7 or above

1. GrayLog Forwarer installation steps

2. cd /opt

3. mkdir graylog_collector

4. cd graylog_collector

5. wget https://packages.graylog2.org/releases/graylog-collector/graylog-collector-0.4.0.tgz

6. tar xvfz graylog-collector-0.4.0.tgz

1. GrayLog Forwarder Run Command

2. bin/graylog-collector run -f /opt/graylog_collector/graylog-collector-0.4.0/config/collector.conf &

Attached in email is a sample config file , copy to the config directory in graylog collector setup and rename to collector.conf , update the file with logs to be forwarded.

---

 server-url = "http://172.16.1.113:12900/%22
 message-buffer-size = 128
 inputs {
 opt_ecom_emailnotif_log { ## This is the log input name , must be unique
 type = "file" ## type of log
 path = "/home/logs/emailnotif.log" ## absolute path to log file
 message-fields = {
 "logId" = "emailnotif"
 }

1. 1. Add below paramters if log messages are to be parsed using a pattern, else newline will be used.

1. 1. content-splitter = "PATTERN"

1. 1. content-splitter-pattern = "^\\[\\d{2} (Jan|Aug|Feb|Mar|Arp|May|Jun|Jul|Sep|Oct|Nov|Dec) \\d{2} \\d{2}:\\d{2}:\\d{2}\\]"

}

}

 outputs {
 quikrplatform_tcp_gelf { ## Output id
 type = "gelf"
 host = "172.16.1.113" ## graylog server ip
 port = 12201 ## input port ip , Make sure this is correct, graylog can run multiple inputs on different port.
 client-queue-size = 512
 client-connect-timeout = 5000
 client-reconnect-delay = 1000
 client-tcp-no-delay = true
 client-send-buffer-size = 32768
 inputs = "emailnotif_log" ## input to process
 }
 }

or

we can do like this also

 inputs {
 NM-PRD-WEB-24_curlCallTimeout_log { ## This is the log input name , must be unique
 type = "file" ## type of log
 path-glob-root = "/home/logs/"
 path-glob-pattern = "curlCallTimeout-*.log"
 message-fields = {
 "logId" = "curl-log"
 }

1. 1. Add below paramters if log messages are to be parsed using a pattern, else newline will be used.

1. content-splitter = "PATTERN"

1. content-splitter-pattern = "^\\[\\d{2} (Jan|Aug|Feb|Mar|Arp|May|Jun|Jul|Sep|Oct|Nov|Dec) \\d{2} \\d{2}:\\d{2}:\\d{2}\\]"

 outputs = "CurltimeOut-FE-GELF"

 }

 outputs {
 CurltimeOut-FE-GELF { ## Output id
 type = "gelf"
 host = "172.16.1.186" ## graylog server ip
 port = 12206 ## input port ip , Make sure this is correct, graylog can run multiple inputs on different port.
 client-queue-size = 512
 client-connect-timeout = 5000
 client-reconnect-delay = 1000
 client-tcp-no-delay = true
 client-send-buffer-size = 32768
 }

 }

---

For the first time graylog will not reflect if the log is not updating. For that you can just make a test entry into the end of the log file and once it is reflected in graylog you can remove the test entry from the log file

graylog server startup

There are 2 elastic search instance need to start

 1.cd /opt/elasticsearch-1.6.1
 bin/elasticsearch -Xmx30g -Xms30g -d
 cd /opt/elasticsearch-1.6.1_node_2
 bin/elasticsearch -Xmx30g -Xms30g -d
 2.cd /opt/mongodb_graylog/mongodb-linux-x86_64-3.0.5/
 nohup bin/mongod --master --dbpath /opt/graylog-els-datastore/data &
 3.cd /opt/graylog/
 bin/graylogctl start

 4.cd /opt/graylog/graylog-web-interface-1.1.6/
 nohup bin/graylog-web-interface &
 check the log file whether it is properly running

Remove already pid exists rm -fv /opt/graylog/graylog-web-interface-1.1.6/RUNNING_PID