basic deployment script using rsync

Here is a simple rsync script that you can use to deploy/provision/baseline any webapp with


#!/usr/bin/env sh

user_name=devuser
export user_name

private_key=config/id_dsa
export private_key

remote_host=192.168.3.111
export remote_host

local_path=./
export local_path

remote_path=/var/www/vhosts/usmagazine
export remote_path

exclude_file=config/rsync_exclude.txt
export exclude_file

# -e in the rsync command forces rysnc to use ssh as the transport protocol and then
# it passes -ax to the ssh command to disable interactive shell and x11 on the server

# -C in the rsync command causes rsync to ignore the subversion and cvs folders in the
# directory tree

# -a in the rysnc command puts rsync in archive mode recursing all folders and
# preserving users symlinks permissions and timestamps

# -z turns on rsync compression

# --delete will delete any files from the remote file system that don't exist on the
# local file system

# --force forcibly answers yes to any prompts for confirmation from rsync

# --exclude-from passes in a file that contains patterns (1 per line) that match files
# using rsync's pattern matching syntax (* for wildcard etc.) a line preceded with a +
# tells rsync to include any files matching the pattern and a line preceded by - tells
# rsync to ignore files matching the pattern. By default all files are included so
# most times you only have to make patterns to black list certain files (for instance
# configuration files that are specific to your sandbox and that should not be
# transferred to a production server)

rsync --progress -azC --force --delete --exclude-from=$exclude_file -e "ssh -ax -i $private_key" $local_path $user_name

don't forget to register your rhel!

Damn, I haven't used the branded version of redhat enterprise linux in a while. I completely forgot that they make you register the box before you can access the package repos AT ALL. I thought they were just asking me to register for the support options... I kept trying all the usual packages with both yum and apt and though apt failed fairly silently:

apt-get install tomcat5
Reading Package Lists... Done
Building Dependency Tree... Done
E: Couldn't find package tomcat5


YUM kept saying:

yum install tomcat5 tomcat5-admin-webapps tomcat5-webapps
Loaded plugins: rhnplugin, security
This system is not registered with RHN.
RHN support will be disabled.
Setting up Install Process
Parsing package install arguments
No package tomcat5 available.
No package tomcat5-admin-webapps available.
No package tomcat5-webapps available.
Nothing to do

RHN support will be disabled... not all repos will be disabled. It even disabled third party repos like DAG!

Force me to register! Damn YOU!!! anyway this is all I had to do to register... surprisingly quick and painless...

/usr/sbin/rhnreg_ks --username=**************** --password=**********

I hate registering.

$%@!#& mysql root user has no permissions error 1044

Sometimes you just can't seem to win...well at least not for 3 hours or so...

Somehow my mysql root user lost all of it's permissions. I think it might be because I moved the database files over from another harddrive that I upgraded to a larger one at somepoint and then did a re-install of mysql and then just over wrote the data directory and started mysql backup without actually setting up the new mysql instance. All the version numbers and paths were the same so I figured it would be a go. Apparently not. Here is the error I kept getting no matter what I tried:

ERROR 1044 (42000): Access denied for user 'root'@'localhost' to database 'mysql'

I tried every database and kept getting

ERROR 1044 (42000): Access denied for user 'root'@'localhost' to database 'blah'

DAMN!

anyway i found this stupid trick in the mysql user forums (note the date of the post!):

Posted by [name withheld] on July 30 2003 11:58pm [Delete] [Edit]
when you are simply trying to:
C:\mysql\bin>mysql -uroot -p mysql

and you get:

ERROR 1044: Access denied for user: '@127.0.0.1' to database 'mysql'

Here is what I do. The key is to supply your real ip address for the -h (host) parameter. On windows, from the command prompt type 'ipconfig' to see your ip address. Once you have that, do the following:

C:\mysql\bin>mysql -h 192.168.0.1 -u root -p mysql
Enter password: ****************

// then I explicitly add root@127.0.0.1 to the user table, so after this I can log in as you would expect

GRANT ALL PRIVILEGES ON *.* TO root@localhost IDENTIFIED BY 'root-password' WITH GRANT OPTION;

GRANT ALL PRIVILEGES ON *.* TO root@127.0.0.1 IDENTIFIED BY 'root-password' WITH GRANT OPTION;

Why would mysql care if you are connecting from an externaly available ip address on your own computer I don't know. All i do know is that it wouldn't let me grant the root user permissions via the loopback address or via sockets.

This one was a doozy!

Creating Tap interface on Windows to allow for communication with Virtualbox guest os

OpenVPN *a project that I have always loved since its early days (I created a very reliable vpn for 10 users on different os's with an old pentium 90 as the host machine back in 2003 on it) Happens to come with a tun/tap driver for windows 2000 and XP as far as I know... possibly even Vista. Using this tun/tap driver you can actually create tap devices for your own purposes (not just for OpenVPN use) and if your thinking what I am typing you can create a setup much like in my last post where your virtual box guest os uses the tap interface for it's host only network adapter and thus you don't need any host to guest (and vice versa) network communication to leave the host machine. IE you don't need the host machine to be connected to any actual network for the two os's to talk to each other.

I have torn the following step by step instructions from here (http://milksnot.com/joomla/index.php?option=com_content&view=article&id=29&Itemid=25)

Download and install OpenVPN for Windows.
After installation of OpenVPN, a so-called TAP interface should already be installed.
Now rename the existing TAP interface to 'OpenVPN'. We won't otherwise touch this adapter, because we might want to use it with OpenVPN. If you won't be using OpenVPN, then you can skip the part below were you install a second TAP adapter. You do however need to do the configuration bit.
You can add a TAP interface in two ways: Either use the installed script at "start/program files/openvpn/Add a new TAP-Win32 virtual ethernet adapter" or do it manually. If you prefer the latter, then you do not even have to install OpenVPN. You can extract the driver from the installation package and use only that. If you prefer a manual install, this is how you would go about it:

Open Control Panel and select Add Hardware
Select 'Yes, I have already connected the hardware'
Select 'Add a new hardware device'
Select 'Manually select'
Select 'Network adapters'
Select 'Have disk'
Browse to 'C:\Program Files\OpenVPN\driver' and select 'OemWin2k.inf'
Select 'TAP-Win32 Adapter'
Some messages may appear about driver signing. Ignore them.
You do not need to reboot in order to use the new interfaces. Removing a device can be done in Computer Management/Device Manager/Right-click-on-device/uninstall. Now configure the new TAP interface.

Open the Network Connections window and look for the new adapter. It will be called something like 'Local Area Connection'.
Rename the adapter to 'TAP'
Open TAP's properties and browse to General/Adapter/Advanced
Set the adapter's Media Status to 'Always Connected'. If we skip this, then the host machine won't be active on the TAP's network.
Now configure the IP address and mask of the TAP adapter. NOTE: Use a range not in use by any of your other adapters. I spent bloody two hours trying to discover why my networks were not networking only to discover I forgot to disable two VMware network interfaces which were using the same range as I was using with the TAP interfaces.
Using the adapter in VirtualBox: When configuring an interface on a virtual machine, select 'Attached to: Host Interface' and the select the adapter called 'TAP-Win32 adapter V8 #2' from the list of adapters. And from here on it's business as usual.

Creating Host Only network adapter with virtual box on OS X

Here is a step by step on how to create a Host Only Virtual network adapater for Virtualbox in the same sense that host only network adapters work with vmware fusion or parallels. This method uses a tap virtual network adapter on the host os to allow the host os AND the virtual machine to communicate without ever having to send an ethernet frame out onto a wire (or into the air) everything happens within the host os.

Start by downloading the Tun/Tap software from the sourceforge page here: http://tuntaposx.sourceforge.net/

Next add the following to a bash script and give it execute permissions (Taken from this post on the vbox forrums: http://forums.virtualbox.org/viewtopic.php?f=8&t=14871&p=66322#p66322)

echo "starting"
exec 4<>/dev/tap0
ifconfig tap0 10.10.10.1 10.10.10.255
ifconfig tap0 up
ping -c1 10.10.10.1
echo "ending"
export PS1="tap interface>"
dd of=/dev/null <&4 & # continuously reads from buffer and dumps to null

sudo run the script.

Next modify your vbox guest os via the command line like so:

VBoxManage modifyvm "MyVM" -nic2 hostif #make the second network adapter host-networking
VBoxManage modifyvm "MyVM" -hostifdev2 tap0: # connect that adapter to tap0: (make sure to include the : colon after the tap0)

Start the guest machine.

Configure the guest machine (depending on your distro) to activate the eth1 device with a static ip address in the 10.10.10.255 range with the netmask of 255.255.255.0

Now from the hostos try and ping the guest os on the ip address you gave eth1 above.

If all goes well it should respond to the ping (firewall permitting of course)

If you feel like it go ahead and modify the /etc/hosts file on your host os to point a human readable name like "virtualbox" at the ip address you configured for eth1 on the guest os.

how to sha1 checksum a file in os x using openssl

openssl dgst -sha1 filenamegoeshere

thoughts on an agile-waterfall mixed process

I don't often talk about the business behind software engineering but recently I have had to talk process a lot more with other stakeholders in projects and below is a summary of what I have started to formalize as a agile-waterfall combination process. Though I agree that a fully agile software devleopment process like scrum, XP etc is probably better and more efficient, it is a hard sell for some clients who are so used to the traditional waterfall process. It is also hard to tell some execs in a client services company to let go of their need to have client "sign-off" on everything. Anyways here is a first stab:

requirements gathering / exploration
requirements document
client priority analysis
functional specifications
effort analysis of line items in functional spec
prototype risky functionality
mood boards for design
content and functionality document (with summarized effort based on knowledge from effort analysis of functional spec and client priority analysis)
(at this point we should know what the client wants for the initial launch of the product based on effort and priority. Things that are too costly and low priority should be put off until post launch)
wireframes
user testing of wireframes (not required)
annotated wireframes with func spec / inform wireframes from func spec and vice versa
design round based on wireframes
user testing of design (not required)
scaffold functionality based on wireframes (this task might be broken into a dozen or more scrum sprints depending on the size of the project)
content entry
user testing of scaffolding (not required)
design round
further user testing of design (not required)
sign off of design
build out full functionality to work with design (again this task might be broken into a dozen or more scrum sprints depending on the size of the project)
content entry
QA
Bug fixes
product launch
post launch feature requirements review and exploration (begin process over again)