Learn Linux: September 2014

Here the threshold is 40% . If any process using swap memory more than 40% we will print the process id, TotalSwap, SwapUsed, SwapUsed in percentage.

Note: Below script only works where the VmSwap line prints in /proc/processid/status file (Works in RHEL/Centos 6 2.6.32-279.2.1.el6.centos.plus.x86_64)

#!/bin/bash
##Author == Md. Mansoor
for id in `ps aux | sort -k 4 -k 3 -nr | head -n5 | awk '{print $2}'`
do
swapkb=`cat /proc/$id/status | grep -i swap | awk '{print $2}'`
swapusedMB=`echo $((swapkb/1024))`
total=`free -m | awk 'FNR == 4 {print $2}'`
foutyper=`bc <<< $total*40/100`
swapusedper=`bc -l <<< $swapusedMB/$total*100 | awk -F. '{print $1}'`
if [ $swapusedMB -gt $foutyper ]; then
user=`ps aux | grep $id | grep -v grep | awk '{print $1}'`
cmd=`cat /proc/$id/cmdline`
echo "Above 40% swap used ProcessID = $id TotalSwap = $total SwapUsed = $swapusedMB SwapUsedin% = $swapusedper% MB User = $user Command = $cmd"
fi
done

##Another Script:
##This works for all linux version including old kernel
##Server running lower than procpu-3.2.7-26.rpm package should instead of above use below script. Reference: https://rhn.redhat.com/errata/RHBA-2013-1338.html

#! /bin/bash
# Author: Md. Mansoor
for i in /proc/[0-9]*; do
pid=$(echo $i | sed -e 's/\/proc\///g')
#echo $pid
swap_pid=$(cat /proc/$pid/smaps | awk 'BEGIN{total=0}/^Swap:/{total+=$2}END{print total}')
total=`free -m | awk 'FNR == 4 {print $2}'`
swapusedMB=`echo $((swap_pid/1024))`
#echo "swap used in MB $swapusedMB MB"
swapusedper=`bc -l <<< $swapusedMB/$total*100 | awk -F. '{print $1}'`
#echo "$swapusedper in percentage"
foutyper=`bc <<< $total*10/100`
user=`ps aux | grep $pid | grep -v grep | awk '{print $1}'`
if [ $swapusedMB -gt $foutyper ]; then
cmd=`cat /proc/$pid/cmdline`
echo "Above 40% swap used ProcessID = $pid TotalSwap = $total SwapUsed = $swapusedMB SwapUsedin% = $swapusedper% MB User = $user Command = $cmd"
fi
done

Reference: http://www.quora.com/How-can-I-determine-which-process-is-contributing-to-paging-on-Linux

Master: alps.gai.net

Agent:

gai-1234.gai.net
(192.168.10.22)

Puppet
dashboard : http://alps.gai.net:3000
Username: puppet
Password:
puppet123

Pre-requisites: Configure NTP in Master & Agents

Step 1# Master Install & Configure:

##Install
Puppet Repo

#rpm
-ivh
https://yum.puppetlabs.com/el/6/products/x86_64/puppetlabs-release-6-7.noarch.rpm

#yum
install puppet-server

##Configure
Master

#vim

/etc/puppet/puppet.conf

[master]

certname=alps.gai.net

dns_alt_names=gateway.gai.net

Note:

dns_alt_names:


A list of valid hostnames for the master, which will be embedded in
its certificate. Defaults to the puppet master’s certname and
puppet, which is usually fine. If you are using a non-default
setting, set it

before


starting the puppet master for the first time.

Certname:


The name to use when handling certificates. Defaults to the fully
qualified domain name.

Default

:
(the system’s fully qualified domain name)

#/etc/init.d/puppetmaster
start

Step
2# Agent Install & Configure

##Install
Puppet Repo

#rpm
-ivh
https://yum.puppetlabs.com/el/6/products/x86_64/puppetlabs-release-6-7.noarch.rpm

#yum
install puppet

##Configure
Agent

#vim

/etc/puppet/puppet.conf

[agent]

    server=alps.gai.net

    report=true

    pluginsync=true

    certname=gai-1234.gai.net

    runinterval=60m

Here:

runinterval
= for applying configuration with master (This will update
configuration every 1hr with master, change based on your
requirement)

#/etc/init.d/puppet
start

Step
3##Creating & Sigining SSL Certificate for Agent in Master##

Note:
The Master and Agent connections running in SSL hence we have to sign
the agent certificate request in Master. Do the below steps for it.

Agent#
puppet agent --test

Master#puppet
cert list (#Once enter this command, you will see the agent FQDN)

##Sign
the Agent certificate:##

Master#puppet
cert sign xxxxx (Here append xxxxx as above command result client
FQDN)

##Checking
Agent###

Agent#
puppet agent --test (This will provide as Applying configuration with
some version number)

Step
4# Testing Master Agent:

In
Master:

vim
/etc/puppet/manifests/site.pp

##Add
the below lines

class
test_class {

   file
{ "/tmp/testfile":

      ensure
=> present,

      mode
  => 644,

      owner
 => root,

      group
 => root

#
tell puppet on which client to run the class

node
'

gai-1234.gai.net

'
{

    include
test_class

}

In
Agent:

#puppet
agent --test

Now
check /tmp/testfile will be created in Agent node.

Step
5# Installing Puppet Dashboard in Master server

Master#yum
install mysql mysql-server puppet-dashboard

#cp
/usr/share/puppet-dashboard/config/database.yml.example
/usr/share/puppet-dashboard/config/database.yml

#vim
/usr/share/puppet-dashboard/config/database.yml

#Modify
the lines as below

production:

  database:
dashboard_production

  username:
dashboard

  password:

secret_password

(Note:
this password should be same as below mysql user password)

  encoding:
utf8

  adapter:
mysql

#cp
/usr/share/puppet-dashboard/config/settings.yml.example
/usr/share/puppet-dashboard/config/settings.yml

#vim
/usr/share/puppet-dashboard/config/settings.yml

##Modify
lines as below

ca_server:
'alps.gai.net'

ca_port:
8140

enable_inventory_service:
true

inventory_server:
'alps.gai.net'

inventory_port:
8140

use_file_bucket_diffs:
true

file_bucket_server:
'alps.gai.net'

file_bucket_port:
8140

##Creating
Database & User & Tables

CREATE
DATABASE dashboard CHARACTER SET utf8;

CREATE
USER 'dashboard'@'localhost' IDENTIFIED BY '

secret_password';

GRANT
ALL PRIVILEGES ON dashboard.* TO 'dashboard'@'localhost';

flush
privileges;

#vim
/etc/mysql/my.cnf

#Add
below line

max_allowed_packet
= 32M

##Once
Dashboard has its database, it can create its tables, but this has to
be done manually with the db:migrate rake task. For typical use with
the production environment:

##Run
the below commands in terminal

#rake
RAILS_ENV=production db:migrate

##Testing
That Dashboard is Working

#sudo
-u puppet-dashboard ./script/server -e production (use the below
service instead of this)

or

/etc/init.d/puppet-dashboard
start

/etc/init.d/puppet-dashboard-workers
start

browse
and connect as http://alps.gai.net:3000

##Configuring
Puppet

Using
Dashboard for Reports

For Dashboard to receive reports, there are two requirements:

All agent nodes have to be configured to submit reports to the master.
The master has to be configured to send reports to Dashboard.

Make
sure that all agents have reporting turned on:

#
puppet.conf (on each agent)

[agent]

    report
= true

#
puppet.conf (on puppet master)

[master]

    reports
= store, http

    reporturl
= http://dashboard.example.com:3000/reports/upload

#Restart
services (Master)

#/etc/indi.d/puppetmaster
restart

#/etc/indi.d/puppet-dashboard
restart

#/etc/indi.d/puppet-dashboard-workers
restart

#Restart
services (Agent)

#/etc/indi.d/puppet
restart

##Enabling
Inventory for puppet aent

Master#
vim /etc/puppet/auth.conf

#Go
to end of file#

##Disable
below lines

path
/

auth
any

#Add
below lines

path
/facts

auth
any

method
find, search

allow
*

#Restart
services

#/etc/init.d/puppetmaster
stop

#/etc/init.d/puppetmaster
start

#/etc/init.d/puppetmaster-dashboard
stop

#/etc/init.d/puppetmaster-dashboard
start

#/etc/init.d/puppetmaster-dashboard-workers
stop

#/etc/init.d/puppetmaster-dashboard-workers
start

Now
browse: http://puppetmasterip:3000
check the inventory tab

Step
6# ##Running Puppet Master using apache instead of services
(puppetmaster)

Refer:

http://docs.puppetlabs.com/guides/passenger.html

#yum
install httpd httpd-devel mod_ssl ruby-devel rubygems gcc

#Install
Rack/Passenger

#sudo
gem install rack passenger

#sudo
passenger-install-apache2-module

#sudo
mkdir -p /usr/share/puppet/rack/puppetmasterd

#sudo
mkdir /usr/share/puppet/rack/puppetmasterd/public
/usr/share/puppet/rack/puppetmasterd/tmp

#sudo
cp /usr/share/puppet/ext/rack/config.ru
/usr/share/puppet/rack/puppetmasterd/

#sudo
chown puppet:puppet /usr/share/puppet/rack/puppetmasterd/config.ru

#chown
puppet:puppet /usr/share/puppet/rack/puppetmasterd/config.ru

#chown
apache:apache /usr/share/puppet/rack/puppetmasterd/public -R

#chown
apache:apache /usr/share/puppet/rack/puppetmasterd/tmp -R

#vim
/etc/httpd/conf.d/puppetmaster.conf

#Add
below lines

#
This Apache 2 virtual host config shows how to use Puppet as a Rack

#
application via Passenger. See

#
http://docs.puppetlabs.com/guides/passenger.html for more
information.

#
You can also use the included config.ru file to run Puppet with other
Rack

#
servers instead of Passenger.

#
#Below modules lines you will get while running above gem install
rack passenger

LoadModule
passenger_module
/usr/lib/ruby/gems/1.8/gems/passenger-4.0.37/buildout/apache2/mod_passenger.so


  <IfModule
mod_passenger.c>


    PassengerRoot
/usr/lib/ruby/gems/1.8/gems/passenger-4.0.37


    PassengerDefaultRuby
/usr/bin/ruby


  </IfModule>

#
you probably want to tune these settings

PassengerHighPerformance
on

PassengerMaxPoolSize
12

PassengerPoolIdleTime
1500

#
PassengerMaxRequests 1000

PassengerStatThrottleRate
120

#RackAutoDetect
Off

#RailsAutoDetect
Off

Listen
8140

<VirtualHost
*:8140>


   SSLEngine
On


   #
Only allow high security cryptography. Alter if needed for
compatibility.


   SSLProtocol
            All -SSLv2


   SSLCipherSuite
         HIGH:!ADH:RC4+RSA:-MEDIUM:-LOW:-EXP


   SSLCertificateFile
     /var/lib/puppet/ssl/certs/alps.gai.net.pem


   SSLCertificateKeyFile
  /var/lib/puppet/ssl/private_keys/alps.gai.net.pem


   SSLCertificateChainFile
/var/lib/puppet/ssl/ca/ca_crt.pem


   SSLCACertificateFile
   /var/lib/puppet/ssl/ca/ca_crt.pem


   SSLCARevocationFile
    /var/lib/puppet/ssl/ca/ca_crl.pem


   SSLVerifyClient
        optional


   SSLVerifyDepth
         1


   SSLOptions
             +StdEnvVars +ExportCertData


   #
These request headers are used to pass the client certificate


   #
authentication information on to the puppet master process


   RequestHeader
set X-SSL-Subject %{SSL_CLIENT_S_DN}e


   RequestHeader
set X-Client-DN %{SSL_CLIENT_S_DN}e


   RequestHeader
set X-Client-Verify %{SSL_CLIENT_VERIFY}e


   DocumentRoot
/usr/share/puppet/rack/puppetmasterd/public


   <Directory
/usr/share/puppet/rack/puppetmasterd/>


     Options
None


     AllowOverride
None


     #
Apply the right behavior depending on Apache version.


     <IfVersion
< 2.4>


       Order
allow,deny


       Allow
from all


     </IfVersion>


     <IfVersion
>= 2.4>


       Require
all granted


     </IfVersion>


   </Directory>


   ErrorLog
/var/log/httpd/alps_puppet_master_ssl_error.log


   CustomLog
/var/log/httpd/alps_puppet_master_ssl_access.log combined

</VirtualHost>

#
Stop puppetmaster & restart http

#/etc/init.d/puppetmaster
stop

#/etc/init.d/httpd
restart

#Check
listen


#netst
-tupln | grep LISTEN | grep 8140

Step
7# ##Running Puppet Dashboard using apache instead of services
(puppet-dashboard)

Refer:
http://docs.puppetlabs.com/dashboard/passenger.html

#cp
/usr/share/puppet-dashboard/ext/passenger/dashboard-vhost.conf
/etc/init.d/conf.d/

#vim
/etc/httpd/conf.d/dashboard-vhost.conf

#Add
below lines or modify

LoadModule
passenger_module
/usr/lib/ruby/gems/1.8/gems/passenger-4.0.37/buildout/apache2/mod_passenger.so

   <IfModule
mod_passenger.c>

     PassengerRoot
/usr/lib/ruby/gems/1.8/gems/passenger-4.0.37

     PassengerDefaultRuby
/usr/bin/ruby

</IfModule>

PassengerHighPerformance
on

PassengerMaxPoolSize
12

PassengerPoolIdleTime
1500

PassengerStatThrottleRate
120

Listen
3000

<VirtualHost
*:3000>

        ServerName
puppet.gai.net

        DocumentRoot
/usr/share/puppet-dashboard/public/

        <Directory
/usr/share/puppet-dashboard/public/>

                Options
None

                Order
allow,deny

                allow
from all

        </Directory>

  ErrorLog
/var/log/httpd/puppet.gai.net_error.log

  LogLevel
warn

  CustomLog
/var/log/httpd/puppet.gai.net_access.log combined

  ServerSignature
On

<Location
"/">

    Order
allow,deny

    Allow
from 192.168.0.125 # your puppet master's IP

    Satisfy
any

    AuthName
"Puppet Dashboard"

    AuthType
Basic

    AuthUserFile
/etc/httpd/htpasswd

    Require
valid-user

</Location>

</VirtualHost>

##Stop
puppet-dashboard services & restart http

#/etc/init.d/puppet-dashboard
stop

#/etc/init.d/httpd
restart

##Classes Examples:

#To Copy configuration file:

vim /etc/puppet/manifests/site.pp

class centos_6_repo_class {

file { "/etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6":

owner => root,

group => root,

mode => 644,

source => "puppet:///modules/centos_6_repo/RPM-GPG-KEY-CentOS-6"

}

file { "/etc/yum.repos.d/CentOS-Base.repo":

owner => root,

group => root,

mode => 644,

source => "puppet:///modules/centos_6_repo/CentOS-Base.repo"

}

# tell puppet on which client to run the class

node 'gai-1234.gai.net' {

include centos_6_repo_class

}

To Install a package:

#vim /etc/puppet/manifests/site.pp

class ftp-0-17-54_install_class {

package { "ftp":

ensure => "0.17-54.el6"

}

node 'genistra.gai.net' {

include ftp-0-17-54_install_class

}

Pupper Agent for Windows:

Download from https://downloads.puppetlabs.com/windows/puppet-3.4.3.msi

create file as C:\Program Files\Puppet Labs\Puppet Enterprise\puppet\conf\puppet.conf

add below lines

[agent]
    server = alps.gai.net
    certname = thinkpad4
        report=true
        pluginsync=true
        runinterval=2m

In server site.pp file add below lines

if $osfamily == 'windows' {
    File { source_permissions => ignore }
}

class win-check {
file { "C:/temp/CentOS-Base.repo":
    ensure => present,
    group => Administrators,
    source => "puppet:///modules/centos_6_repo/CentOS-Base.repo"
}

}

node 'thinkpad4' {
    include win-check
}

URLS Referred:

http://docs.puppetlabs.com/dashboard/manual/1.2/bootstrapping.html#testing-that-dashboard-is-working

http://docs.puppetlabs.com/dashboard/passenger.html

http://www.stuartmacfarlane.co.uk/installing-puppet-3-master-and-agent-on-centos-6-3

http://www.6tech.org/2013/01/how-to-install-puppet-open-source-on-centos-6-3/

http://www.ende-der-vernunft.org/2012/10/19/howto-install-puppet-dashboard-passenger-activemq-and-mcollective-entirely-from-repos-on-centos-6/

http://usrlocal.com/2012/11/puppet-dashboard/

http://www.puppetcookbook.com/posts/select-a-file-based-on-a-fact.html

http://rene.bz/yum-repo-and-package-dependencies-puppet/