Lock Down EC2 usage by Region

Lock down the usage of EC2 based on region:

{
“Statement”: [
{
"Effect": "Allow",
"Action": [
"ec2:*"
],
“Condition”: {
“StringLike”: {
“ec2:Region”: “us-east-1″
}
},
“Resource”: [
"*"
]
}
]
}

Note in order for this policy to work with the mobile AWS console application you must also include access to CloudWatch.

AWS ELB : Log xforward header presented by an elastic loadbalancer

To log xforward headers provided by an AWS ELB you can include the following in you Apache configuration. For an added bonus I also shifted all of the ELB entries into a separate log file:

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent

SetEnvif User-Agent "^ELB-HealthChecker" ELB
CustomLog logs/elb_log combined env=ELB

SetEnvIf REMOTE_ADDR "(.+)" CLIENTIP=$1
SetEnvIf X-Forwarded-For "^([0-9.]+)" CLIENTIP=$1
LogFormat "%{CLIENTIP}e %D %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" xforward_combined

CustomLog logs/access_log xforward_combined env=!ELB

For IIS similar results can be achieved by installing the advanced logging plugin. After installing this plug in, you need to restart the IIS Manager if it was open during the installation. To actually log the xforward headers create a log file with the X-Forwarded-For request header included:

• Go to the "Advanced Logging" snap in under the IIS section of either individual website home pages in IIS Manager, or under the server home page in IIS Manager. If you would like Advanced Logging configured for all sites hosted on this particular instance, select the latter. 
• Under Advanced Logging, click on "Edit Logging Fields" under the Actions pane on the right side of the screen.
• In this window, click on "Add Field".
• Within this dialog, populate the fields with the following values:

-- Field ID: X-Forwarded-For
-- Category: Default
-- Source type: Request Header
-- Source name: X-Forwarded-For

• Click OK on both the Add Field Dialog and the Edit Logging Fields dialog.
• Click on the Logging Defintion named %COMPUTERNAME%-Server and then select "Edit Log Definition" on the Actions pane on the right side of the screen (you can also create your own logging definition by clicking on the "Add Log Definition" link).
• Within the Edit/Add Log Definition dialog, click on the "Select Fields" button (you will also need to fill in additional information if you are adding a new log definition).
• Within the Select Fields dialog, check "X-Forwarded-For" and click OK.
• Click on "Apply" under the Actions pane.
• Restart IIS either through the IIS Manager or by issuing an iisreset command from the command line.

Solaris 10: Pools and Zones

Here is a simple script I created to create a top "like" output of process information based on a selected pool. I find it helpful when attempting to troubleshoot a performance issue on our large systems running several zones inside the same resource pool.

Google Doc Link

Zones: pkgchk fails inside a zone

Recently I ran into an issue where pkgchk failed for a particular packages inside of a zone. No worries, I figured that I shutdown the zone and reattach to get the global to copy in the required binaries; however this did not seem to work. Even performing an "attach -U" was unsuccessful; the package still failed pkgchk when tested.

Since the global had the same version of the package that was missing from the zone (when checked with pkginfo -l ) I found the following procedure worked to lay down the binaries again inside the zone:

# zlogin ZONE

ZONE# cd /var/sadm/pkg

ZONE# rm -rf PACKAGE_DIR

ZONE# exit

# zoneadm -z ZONE halt

# zoneadm -z ZONE detach

# zoneadm -z ZONE attach -U

# zoneadm -z boot

After completing the above steps the Global, not seeing the meta information for the package added back the package. When the zone came back up I was able to log back into the zone and pkgchk completed without any errors!

In conclusion it seems that the zone attach process does not look at the contents of /var/sadm/install/contents; it only goes by the content of  /var/sadm/pkg directory. Good to know.

Solaris: removing a disk label

We recently had an issue where an existing unused LUN was shrunk on the backend array, after it had been labeled by Solaris. Esentially the disk when from 1TB to 990GB so that rather than creating an sliced/EFI disk group the LUN could be included in a CDS diskgroup. Attempts to change the disk size via format and fmthard were unsuccessful. Having nothing to lose, data wise or via trying, I dd the header off the disk which did the trick.

Please not that using this command on a LUN will completely wipe out the disk label and should not be used on any disk that is currently in use or has data you would like to read. First run dd to wipe out the label information from the disk


# dd if=/dev/zero  of=/dev/rdsk/c5t500A09838779AF95d36s2 count=24
24+0 records in
24+0 records out

After the dd completed , the LUN looked new when check with format and I could relabel and stick it in a CDS diskgroup with all of his disk friends:

-bash-3.00# format c5t500A09838779AF95d36s2

c5t500A09838779AF95d36s2: configured with capacity of 989.74GB

selecting c5t500A09838779AF95d36s2

Solaris 10: What are your HBAs used for?

Here is a shell script I wrote to help quickly identify what types of devices are hanging off of each of a particular system's HBAs. Typically we have dedicated tape and storage fabrics so it speeds up figuring out which one is which based on the output of cfgadm and fcinfo; check it out.

I'm not sure if this would work with older version of Solaris, but it was written for Solaris 10.

Google Doc Link

VXFS: enclosure based naming issues

Some times you will see disks appear to have "enclosure based naming" under Veritas when that is not the actual naming scheme in effect. To validate the current naming scheme runng the following:

# vxddladm get namingscheme
NAMING_SCHEME       PERSISTENCE    LOWERCASE      USE_AVID
============================================================
OS Native           No             Yes            Yes

If you are set to OS native this can happen for a couple of reason , the most common I see are the system is attached to a NetApp arrary with ALUA enabled or MPXIO is running in Solaris.

To disable MPXIO in Solaris 10, edit the file /kernel/drv/fp.conf file; so the MPXIO line reads:

mpxio-disable="yes";

After modifying fp.conf reboot the system (I prefer reconfiguration "reboot -- -r") the names of these disks should be back to the ugly long cname variety when check with vxdisk list.

EMC: Unlicensed paths in PowerPath

 To fix the licensing issue use one of the following procedures:

Symmetrix
powermt check_registration  (Make sure the license shows with full capabilities.)
powermt set policy=so dev=all  (Set the Policy to SymmOpt.)
powermt display dev=all  (Make sure all device paths are now licensed.)
powermt save  (Save configuration.)

CLARiiON
powermt check_registration  (Make sure the license shows with full capabilities.)
powermt set policy=co dev=all (Set the Policy to ClarOpt.)
powermt display dev=all (Make sure all device paths are now licensed.)
powermt save (Save configuration.)

Other Storage Systems
powermt check_registration  (Make sure the license shows with full capabilities.)
powermt set policy=ad dev=all  (Set the correct policy.)
powermt display dev=all  (Make sure all device paths are now licensed.)
powermt save  (Save configuration.)


Note for UNIX Servers: If the license key is not installed, use the emcpreg ·install command.

HP: Smart Array Utilities

Starting the Smart Array drive utility web page:

[root]~# which cpqacuxe
/usr/sbin/cpqacuxe

[root]~# cpqacuxe -R
Array Configuration Utility version 7.70.12.0
Make sure that you have gone through the following checklist:
   1. Change the administrator password to something other than
      the default.
   2. Only run ACU on servers that are on a local intranet
      or a secure network.
   3. Secure the management port (port 2301 or 2381)
      on your network.

Remote connection enabled!

*** If you don’t know the Administrator password to the web interface,
    cd /opt/compaq/cpqacuxe/bld, and run hphmmopasswd

*** The webpage can be accesses at https://SERVER-IP:2381


Disable the Smart Array Drive utility web page:

[root]~# cpqacuxe -stop
[root]~# ps -ef |grep cpq
root     31910 14121  0 13:41 pts/0    00:00:00 grep cpq


Locating Controller slot:

[root]~/#: hpacucli controller all show

MSA1000 at SGM04450VL         (sn: #######L, wwn: ##########)
Smart Array 5300 in Slot 3      ()
Verify Controller status:


Checking Controller status:

[root]~# hpacucli controller slot=1 show status

Smart Array P400 in Slot 1
   Controller Status: OK
   Cache Status: OK
   Battery Status: OK


Verifying rebuild status:

[root]~#: hpacucli controller slot=3 logicaldrive all show

Smart Array 5300 in Slot 3

array A
  logicaldrive 1 (341 GB, 5, OK)
  logicaldrive 2 (341 GB, 5, OK)
  logicaldrive 3 (341 GB, 5, Rebuilding)
  logicaldrive 4 (341 GB, 5, Ready for Rebuild)
Check Physical Disks:

devapp1:/home # hpacucli controller slot=3  pd all show


Show HP related logs:

 # hplog -v

 *** shows ILM related log messages

# hpaducli -f /tmp/hpadu.out

VCS: Adding Oracle databases located in Solaris zones

Here is a script which I wrote to assist in the addition of Oracle databases running inside of a Solaris zone to a VCS configuration. The script does not actual modify the main.cf, rather it creates a shell script for each zone it finds new databases that need to be added to the VCS configuration. The output scripts are titled ZONENAME.dbvcs. The hares statements are included to create the needed resources are scripted if three criteria are met:

1. The database is not already included in the main.cf configuration
2. The database is currently running
3. The oratab file indicates the database in question should be automatically started on boot.

Additionally Listener resources are added as well, provided that they are running at the time of script execution. Again if the listener is already included in the main.cf then the hares commands for the NetLsnr are not included. 

One final note the naming convention in our VCS configurations are quite rigid; therefore all zone resources are name zn_ZONENAME; IP addresses are ip_ZONENAME; Listener lsnr_ZONENAME; database pmon_ZONENAME.

Google Doc Link

Solaris: send mail with attachment from Solaris command line

I like to use this function bellow in my profile to send files to my mail account as attachements; to use in your profile simply replace EMAIL@ADDRESS with your email address  on a mail enabled host:

function  mailme(){


        /usr/bin/uuencode $(echo $1) $(echo $1)|/usr/bin/mailx -s "mailme from $(/usr/bin/hostname)" EMAIL@ADDRESS
}

Redhat: Swapfile

Steps needed to create a 1K swapfile:


[root@prod_app]~# df -h /local
Filesystem            Size  Used Avail Use% Mounted on
/dev/cciss/c1d0p7      26G   21G  4.1G  84% /local

[root@prod_app]~# dd if=/dev/zero of=/local/swapfile.1k bs=1024 count=1

[root@prod_app]~# mkswap /local/swapfile.1k

[root@prod_app]~# ls -l /local/swapfile.1k

[root@prod_app]~# swapon /local/swapfile.1k

Redhat: RPM commands

Installing a RPM
# rpm -ivh package-1.0-2.i386.rpm

Note: this installs the rpm package, version 1.0-2, i386 architecture 

Removing a RPM

# rpm -ev package

Display information about a package

# rpm -qi setup-2.5.37-1.3
Name : setup Relocations: (not relocatable) Version : 2.5.37 Vendor: Red Hat, Inc. Release : 1.3 Build Date: Mon 20 Jun 2005 04:40:37 PM EDT Install Date: Thu 15 Feb 2007 09:28:36 AM EST Build Host: decompose.build.redhat.com Group : System Environment/Base Source RPM: setup-2.5.37-1.3.src.rpm Size : 37056 License: public domain Signature : DSA/SHA1, Thu 30 Jun 2005 01:27:57 PM EDT, Key ID 219180cddb42a60e Packager : Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla> Summary : A set of system configuration and setup files. Description : The setup package contains a set of important system configuration and setup files, such as passwd, group, and profile.


Display file associated with a package

# rpm -ql setup-2.5.37-1.3 

/etc/aliases 

/etc/bashrc 

/etc/csh.cshrc 

/etc/csh.login 

/etc/environment 

/etc/exports 

/etc/filesystems 

/etc/group 

/etc/host.conf 

/etc/hosts.allow 

/etc/hosts.deny 

/etc/inputrc 

/etc/motd 

/etc/passwd 

/etc/printcap 

/etc/profile 

/etc/profile.d 

/etc/protocols 

/etc/securetty 

/etc/services 

/etc/shells 

/usr/share/doc/setup-2.5.37 

/usr/share/doc/setup-2.5.37/uidgid


Locate which rpm a file comes from

# rpm -qf /etc/passwd 

setup-2.5.37-1.3


Display all installed packages

# rpm -qa|more 

basesystem-8.0-4 

beecrypt-3.1.0-6 

expat-1.95.7-4 

....

Article: Raspberry Pi

Can't wait for the release of RaspberryPi A new ARM GNU Linux computer, check it out!

Article: Stuxnet

Very cool article on Stuxnet a must read

VCS: PanicSystemOnDGLoss

This is a configurable option that is on by default in the 5.0 version of VCS. If a disk group becomes completely invisible to a VCS node PanicSystemOnDGLoss option will panic the system immediately; the idea being it will improve recovery time as the service groups will start failing over to the remote side faster.

The down side - on extremely busy cluster nodes the DG agent can fail to complete in the allotted time and triger this option; which is how I learned about it. So, In the event that a cluster node appears to crash for seemingly no reason check the following URL for details.

To check if the VCS option is turned on, run the following command (1 is enabled, 0 is disabled):

# grep "boolean Panic" /etc/VRTSvcs/conf/config/types.cf

boolean PanicSystemOnDGLoss = 0

How to change the default value for PanicSystemOnDGLoss:

# haconf -makerw
# haattr -default DiskGroup PanicSystemOnDGLoss 0
# haconf -dump -makero

Confirm that both the temporary and persistent definitions have been reset to 0:

# haattr -display DiskGroup |grep Panic
PanicSystemOnDGLoss [boolean/scalar] = 0

# grep "boolean Panic" /etc/VRTSvcs/conf/config/types.cf
boolean PanicSystemOnDGLoss = 0


How to change the value for this attribute for individual DiskGroup resources:
In certain cases, it may be necessary to change this attribute to 1 for few selected DiskGroup resources.

# haconf -makerw
# hares -display DG_Res1 | grep PanicSystemOnDGLoss
DG_Res1 PanicSystemOnDGLoss global 0
# hares -modify DG_Res1 PanicSystemOnDGLoss 1
# hares -display DG_Res1 | grep PanicSystemOnDGLoss
DG_Res1 PanicSystemOnDGLoss global 1
# haconf -dump -makero

VXFS: Idle LUN Probing and NetApp

If you are seeing a LUN go online-offline over and over in your DMP log and you are using NetApp for your back end storage it might be due to Veritas' idle LUN probing feature.

The idle LUN probing sub-module of VXFS feature is designed to allow volume manager to establish state information for paths that are not current receiving any IO activity. However it does not work with NetApp arrays; to turn this option off on a system with NetApp LUNs issue the following command that is not system impacting. See the following URL for details:

           

# vxdmpadm settune dmp_probe_idle_lun=off

VCS: and the mount locked issue V-3-26360

V-3-26360 This error code is observed when attempting to umount a filesystem mounted under VCS control and is accompanied with the error text “file system /opt/oracle has been mount locked”. To umount the file system complete the following tasks:

            # /opt/VRTS/bin/umount -o mntunlock=VCS /opt/oracle

V-3-21705 This error code is observed when attempting to umount a VCS mount locked filesystem; which fails even with the previous mentioned “mntunlock” option. The workaround is to lock the file system again using fsadm with the same lock name and then attempt to umount the filesystem again.

            # fsadm -o mntunlock=VCS /nfsvol2 

# fsadm -o mntlock=VCS /nfsvol2 

# /opt/VRTS/bin/umount -o mntunlock=VCS /opt/oracle

Solaris 10: Zone config save to zone path

Here is a script I wrote to save the configuration of a running zone to it's zone path. As all of our zones are running on sparse root on SAN devices, we can moves zones around via SRDF/RecoverPoint between datacenters or DG export/import locally). This script, which we run from cron ensures that we have a good zone configuration if we need to modify an old configuration on the new destination. Simple, but useful.

Google Doc Link

Solaris 10: Zones gathering network configuration

Here is a script I wrote to gather all the network configuration information for a system and all of it's supported zones. check it out.  You get a nice little print out of the Zone, the NIC used, IP address, broadcast address, netmask and the range of the subnet where the IP exists. We don't do a lot of class C networks so I don't really know how well this will work on a C.....


A word of caution: when I was playing around creating this script I whacked the entire network configuration of the server I was testing on. ifconfig is your friend but can also be a deadly enemy.....


Google Doc Link