Critical Server Alert Monitoring Through Hipchat App

What is HipChat:

HipChat is a Web service for internal/private chat and instant messaging. As well as one-on-one and group/topic chat, it also features cloud-based file storage, video calling, searchable message history and inline image viewing.

How we implemented HipChat in our Environment:

1.     Signup in HipChat.com using common email id [italerts@gmail.com]

2.     Created one chat Room [gai-italerts.hipchat.com]. Security=private

3.     Downloaded HipChat API to one of our monitoring server [alps]

Install HipChat API:

            [alps]# mkdir /usr/local/hipchat

            [alps]# cd /usr/local/hipchat

            [alps]# wget https://codeload.github.com/hipchat/hipchat-cli/zip/master

[alps]# unzip master

[alps]# vim /etc/hipchat

HIPCHAT_TOKEN=Shxfn45jWM4x70KoVWfDVjAcEEgQdk8DonZe7GaL

HIPCHAT_ROOM_ID=2566040

:wq

Here: HIPCHAT_TOKEN = Created from HipChat.comàLoginàasàAdminàGroup AdminàAPIàEnter the Admin PasswordàUnder Create New Tokenà

            Type: Notification

            Label: italertstoken

         HIPCHAT_ROOM_ID = is Rooms->API ID

      

4.     Create script for monitoring in Server [alps]

root@alps /]# cat /root/scripts/hipchat_alert.sh

#!/bin/bash

for server in server1.domain.com server2.domain.com server3.domain.com server4.domain.com server5.domain.com server6.domain.com

do

packet=`ping -c 3 -f $server | grep "packet" | awk -F% '{print $1}' | awk '{print $NF}'`

if [ $packet -eq 100 ]

then

/usr/local/hipchat/hipchat-cli-master/hipchat_room_message -i "$server NOT REACHABLE" -f zabbix.gai.net -n @all

fi

done

[root@alps /]#

5.     Schedule the Script in Cron:

###HIPCHAT alert created by Mansoor##

*/3 * * * * /bin/bash /root/scripts/hipchat_alert.sh > /tmp/hipchat_alert.txt

6.     Send Invitation to Users to Join into Group: Group AdminàUsersàAdd

7.     Install Mobile App from PlayStore & Desktop App [HipChat] from HipChat.com website

8.     Enable the Stay connected, Room Messages, One-to-One messages, Room mentions in Mobile App

9.     Add this HipChat Mobile App into “Protected Apps”, so that it won’t go offline

Extra:

 To send alert to All Room members

#/usr/local/hipchat/hipchat-cli-master/hipchat_room_message -i "$server NOT REACHABLE" -f zabbix.gai.net -n @all

OR

#echo “Server is Not Reachable” | /usr/local/hipchat/hipchat-cli-master/hipchat_room_message -f zabbix.gai.net -n @all

To send alert to All Specific Room members

#echo “Server is Not Reachable” | /usr/local/hipchat/hipchat-cli-master/hipchat_room_message -f zabbix.gai.net -n @mdmansoor