Monitoring disk space in linux

Recently I ran into a situation where I need to monitor disk space on a couple servers while a code fix was being made. I needed to stop and start services when the disks got too full.

To do this I used a bash one liner loop to output information from the df command:

while :; do df; sleep 10; done

Sleeping for 10 seconds as to not thrash the disk horribly bad.

The disk free command gives you output similar to the following:

Filesystem 1K-blocks Used Available Use% Mounted on
/dev/xvda1 8256952 4582112 3255412 59% /
udev 3806520 8 3806512 1% /dev
tmpfs 762928 180 762748 1% /run
none 5120 0 5120 0% /run/lock
none 3814624 0 3814624 0% /run/shm
/dev/xvdc 41276736 180236 38999760 1% /mnt

And thats it, you get to loop over the command you want until you hit ctrl-c to exit the loop.

Fixing mass amounts of merge conflicts with Git

At work I use VCR to capture output to make cucumber test runs faster.  One problem is that these cassettes expire and cause merge conflicts.  I recently found a solution to clean those up with grep and git.  Let’s look at the command below:

$ grep -lr '<<<<<<<' . | xargs git checkout --ours

Here we see using grep to search in files for  ‘<<<<<<<‘ and then use git checkout –ours to use our version of the conflict. We could also use –theirs if we wanted to use their version of the commit.

Find and replace in directory with grep and sed

Recently I needed to find an replace a method name in a project as part of a re-organization. So rather than just using an editor to do it for me I figured I would spend some time with a small science project on the command line.

What I was able to come up with is a combination of using grep to find all the files that had the method name in them and pipe that list to set to do the global substitution.

grep -lr -e 'bad_method_name' * | xargs -n1 sed -i '' 's/bad_method_name/new_method_name/g'

As you we see in the code above we have our command starting with grep, looking for ‘bad_method_name’ in * and are piping that to sed.  With sed the only issue I had was with the -i flag.  At first I didn’t specify anything for that flag and had errors in OS X about a malformed sed command.  Passing a blank string to it fixed that.

Remote Pairing: SOCKS Proxy

I’m a big fan of remote pair programming. The problem I often run into is the need to share a web browser. In the past I have always jumped strait to a screen sharing tool like Join.me or TeamViewer. The problem here is these technologies tend to use quite a bit of bandwidth.

The solution recently I have found is to use a SOCKS proxy to the hosting machine.

The proxy allows us to tunnel our traffic through that person’s machine and therefore allowing you to access things local to them like a VM which may not be publicly accessible.

This solution also has the added benefit of the remote pair not needing to do anything but have an ssh server running.

To start a tunnel we’ll need to use the following command but with some substitutions:

$ ssh -p pairing_server_ssh_port -N -D 9999 ssh_user@pairing_server

Let’s explore this ssh connection string!

First the -p flag followed by the port your pair is running their SSH server on. This flag is optional if your pair is using the standard port 22.

Next we include the -N flag which which tells ssh client to not execute remotely which keeps the process running in our shell allowing us to close the tunnel with a ‘Control+c’.

After that another flag is the -D which actually is the SOCKS proxy flag. This flag takes a port number as an argument. This port number needs to be open on your machine. It is the port that your browser will connect to to tunnel traffic to your pair’s machine.

Then we’ll need to finish filling out the username and server information.

Once you have the command filled out hit enter to start the tunnel running.

After our tunnel is running we just need to configure our browser to use the SOCKS proxy.

The easiest browser to setup for a SOCKS proxy is Firefox so we’ll need that installed on our machine.

Next we’ll need to open Firefox preferences and navigate to the ‘Advanced’ section (1). Once there we’ll want to go to the ‘Network’ preferences (2) and choose the ‘Connection’ settings (3).

Now we’ll specify the Manual proxy and host for the SOCKS server as seen below:

After clicking OK and closing all the preference windows you’ll be able to use Firefox to browse the web through your pair’s internet connection. Like always when remote pairing, etiquette is key and make sure you are using this browser for activities that you need to and not for internet searching, email, etc. All of your traffic for this browser will be going over this tunnel and through your pair’s machine.

Using this tool in-conjunction with Joe Kutner’s Remote Pairing: Collaborative Tools for Distributed Development tmux pairing setup will give you a very low bandwidth solution to web development remote pairing.

Using Virtualbox for development VMs

If you’re like me sometimes I just find it easier to use a Virtual Machine for doing development work, especially when it is a complex system with many moving parts.  Recently I started to work on a project that I had inherited from another developer.  The project was partially setup on a remote development server.  I wanted to under stand how the pieces went together and be able to pass it along to other co-workers, so I decided to build a VM.

I have been a heavy VMware Fusion user for many years yet not everyone in my office has the luxury of a license for it.  I decided to give Virtualbox a go for this project.

At first I was really happy with it, I got my machine up and running in no time and was working away until I came to setting up the server and realized for the sake of sane hostfile management I wanted it to have a static ip.  I decided to switch the network adapter over to NAT which is where the pain began.

I enjoy just shelling into my machines and working that way, once I switched to NAT I could not just shell to the ip address of the vm.  This seemed odd to me, VMware creates a virtual network for you which apparently Virtualbox does not.  So I went diving and here is what I found:

You’ll need to enable port forwarding to the vm’s NIC. To start open up the network configuration section of your vm.

The Virtualbox network interface menu

After opening the network tab, click on the “Port Forwarding” button under the “Advanced” section.

Then fill out the sections with the relative information.  I found that if you try and put in a Host IP the setup doesn’t work quite right.  Here I’m forwarding my localhost port 2222 to port 22 on the guest vm along with port 8080 to port 80.

Now from my host machine I can use the following shell command to ssh in $ ssh aip@localhost -p 2222 which will use the port-forwarding and let me access my machine.

The same logic applies to trying to view the site running on the vm, in my browser I just have to access http://localhost:8080.

Now this is a bit of work but it is the trade off for a free tool for virtualizing your environments.  It’s working for me right now but your mileage may very.