divan's blog
As Go community slowly moving towards established and well understood patterns and practices of dependency management, there are still some confusing moments. One of them is automating repeatable build process using containers along with using dependencies in private repositories.
Private repositories on Github are often is a source of confusion when using , but it has easy workaround by adding two lines to your :
or as a oneliner:
But the most confusing part is trying to make the whole build process work inside the container. I will use Docker as an example, as it’s most popular container at the moment.
The problem
Imagine, you have two packages and , where imports :
In normal workflow, you setup GOPATH, you SSH keys, gitconfig and you’re done - simple will work and download both packages:
But now, you want to make the build process reproducible on any machine, even on CI instance, so you pack everything in Docker container. You will probably use simple Dockerfile based on official :
Dockerfile:
Build script
This setup will not work because Docker container used for building (foo-build) doesn’t container dependency, SSH keys and proper gitconfig. And, apparently, it’s not trivial simply to add the keys - you have to deal with a bunch of obstacles, mainly on the SSH side. So, let’s go through quickly and setup working solution.
The solution
ssh vs https
First of all, on the building stage () you will encounter the following error message:
What it does mean, is that your access to github is granted using SSH keys, but command, which is invoked by , is trying to clone repository using HTTPS form and you don’t have credentials set up.
Workaround for this is easy, and described in the beginning of this post, so we just have to add this to our Dockerfile right before calling :
Keys
The next error you’ll see is host key verification error:
Well, it’s simply because our Docker container doesn’t have SSH keys yet. And the right approach is not trivial, so let’s go through it.
First of all, we want every developer or CI to use it’s own keys for accessing private repo. If person has access to , she’s definitely has an access to and the keys are usually in file.
You can’t copy keys into container, though. Dockerfile’s ADD and COPY commands can copy files from the current directory only, so you can’t just add to your Dockerfile. One of the solution is to write wrapper script that will copy private key to local directory and then to the container, but it’s still not very safe and elegant solution.
What we can do, is to mount volume using docker’s command line flag. The first approach will be probably to mount the whole directory, but it’s tricky
This command will work as expected on MacOS X (using latest Docker Beta, at least), but not in Linux box. The reason is the files ownership for file. The (which is invoked by , which is invoked by ) expects this file to have the same user ownership as a running user. Inside the container the user is , but the mounted directory most probably has ovnership of your normal Linux user, say, and inside the container it looks like:
making SSH to complain and abort:
The solution is to mount only the key and workaround host checking later.
Error will be the same, though, but rerunning it with option, we’ll see the reason:
Of course, we don’t want to interact manually with ssh prompt, so we have to find a way to force it. There is an SSH client option for that, called StrictHostChecking.
StrictHostChecking
Typically, you have file named , which holds information about, well, known hosts. But in our container, there is no such file, so we have to use the client option to supress those checks. The easiest way to do this is to put this option into the file - yes, the one we had ownership problems with.
But, we only need one option, so it’s ok to create this file on the fly inside the container. Add to Dockerfile:
Rerun the step and you’ll finally have success!
Conclusion
The final Dockerfile:
and the build steps:
You may put those steps to the Makefile or custom build script, and can safely use it locally or in CI or whatever.
Private SSH key is copied once into the temporary container, used for building, which is removed immediately. Nice and safe solution.
-