I have been using Capistrano for deployment for a few years. In general when it is properly set up, one usually have no issues. Just run a simple one line command and watch it does its magic. I use it for Rails deployment, but you may use it for other deployments as well.

On this day I had an out of usual situation where I had to restart my AWS EC2 instance. I was running out of disk space and hence needed to create a larger volume for disk space. And this requires that I stop my instance before I could detach the old volume and attach a new one. This is a pretty straight forward procedure. Except for one little inconvenience. Whenever you stop an instance and restart it, you will be given a new public IP address. I’m sure there’s way to have a fixed IP address but I’m not on that plan that provide that option. Or maybe I just did not dig deep enough to find out how to do that. You know how it is with busy programmer.

So, with a new IP address for my server certainly throw things off with my Capistrano deployment script. That should be a simple fix. Just update the IP address in that case. As a side note, I should have used a domain name in the place of an IP address. That way I just need to update my DNS record and not have to touch the script. But for a number of different reasons, which I will not dwell upon here, I used the IP address to specify my server to deploy to.

Upon running the deployment script, I encountered a deployment failure with error message as follows:

SSHKit::Runner::ExecuteError: Exception while executing on host 54.173.xx.xxx: git exit status: 1
git stdout: Nothing written
git stderr: ssh: connect to host 54.210.yy.yyy port 22: Connection timed out
error: Could not fetch origin

54.173.xx.xxx is the new IP address of my server while 54.210.yy.yyy was my old IP address. I could not figure out for hours where the old IP address came from, since I have gone through over and over again all my cap scripts to find that without any result. I googled and I hunted to see if perhaps this IP might have been cached somewhere.

After hours of searching I finally found the culprit. It was located in the repo config file on the server itself. In the application root directory you will find a repo folder which has a file called config. This config file is part of the setting for the Git. And in this file it was still using the old IP.

I don’t know exactly how the deployment flow in execution, apparently during the deployment, it was going to do a git clone from the origin. And it uses the setting in this config file on my server for the server address to do a SSH. And of course, the connection timed out.

Anyhow just to note it down here so that in case anyone encountered the same problem, you would not have to spend hours trying to track it down.