This post highlights a few problems with the way the Golang - specifically the go get command - works when using private repositories and SSH authentication, alongside the Windows Subsystem for Linux (WSL). This post goes into details of workarounds, as well as some security tips.
TL;DR
Here is a quick overview of the key points to make it all work nicely:
- Set an environment variable
export GIT_SSH_COMMAND="ssh.exe -i github.com"before using thego getcommand, making sure to also have a SSH configuration file in place. - Choose a SSH agent to deal with SSH keys with passphrases.
- Create a git rewrite rule to force SSH authentication instead of HTTP.
- Create an alias to SSH (
ssh.exe) inside WSL for general git activity.
The Problem(s)
Well, there are actually a few I wanted to solve. The first is that the go get command doesn’t respect your SSH git configuration, even if you set a ~/.gitconfig file for that repository, or globally. Both are ignored when trying to download a private repo from GitHub, which is a problem for internal projects that rely on some private repos.
Secondly, since I’m using Git with SSH keys that are passphrase protected, I need a SSH agent that decrypts the key when used. For me, convenience is really important here, I don’t want to be asked hundreds of times a day to confirm my SSH key passphrases.
I know I’m using a somewhat unconventional setup. A Windows system as my day-to-day driver, and VSCode connected to a WSL (Ubuntu) VM for all development projects. Ideally, WSL will need to share SSH auth keys with the local Windows system.
Examples
Here are a few common errors I would see with the go get command to download a module from a private repository in GitHub. In the first example, the go binary is not able to locate the SSH private key passphrase, or it thinks it doesn’t exist:
In the second example, the go binary couldn’t find the correct module (it thinks it’s public).
I also made the mistake of assuming go module names are case insensitive. I was wrong. For example, the following private repositories are different in regards to the go get command; only the first will actually work.
go get github.com/RedMapleTech/scan-target # Our actual GitHub Org name
go get github.com/redMapletech/scan-target # Not found (lowercase r)
In the pure git world, doing a git clone would work fine with both.
Go Config
Eventually I figured it out, with big thanks to a user on Stackoverflow. The trick is to set a specific environment variable that go get interprets - as explained below.
For reference, I’m using Go version go1.21.5 linux/amd64 on WSL.
Environment variable
Set the following environment variables, making sure a SSH config install on the Windows system, and is in the path.
export GOPRIVATE=github.com/org_name
export GIT_SSH_COMMAND="ssh.exe -i github.com"
What about .netrc?
I only found out recently that you can use .netrc to authenticate go/git, through the use of Personal Access Tokens. But I quickly learned these PATs do not work for private repos within an organisation, which is a bummer.
The following was placed in ~/.netrc:
machine github.com
login xxxxxxxx
password github_pat_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
machine gitlab.com
login xxxxxxxx
password glpat-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Git Config
If you’re like me and want to use SSH throughout your development process, keep reading. Some colleagues prefer the Git Credential Manager, which does deal with GitHub login on Windows and the WSL.
Rewrite HTTPS to SSH
The first thing to do is to create a git rule that rewrites default behaviour, using SSH instead of HTTPS. This can be either done per repository, or in your global config. The following example sets up rewrites for three different git services.
# Rewrite rules
[url "ssh://[email protected]/"]
insteadOf = https://github.com/
[url "ssh://[email protected]/"]
insteadOf = https://gitlab.com/
[url "ssh://[email protected]/"]
insteadOf = https://bitbucket.org/
So whenever I do a git clone https://... it automatically uses SSH for authentication. Great!
SSH key management
Using SSH keys with a strong and random passphrase is good security practice.
Windows OpenSSH agent
The Windows OpenSSH agent worked really, really well for me. Whenever I logged-in, a Windows service called OpenSSH Authentication Agent is started and essentially unlocked all my private SSH keys, ready for use. I don’t have to worry about supplying passphrases, as it’s only done once, when you first add it to the agent using ssh-add.exe.
Here’s a few list of keys it currently manages:
You’ll also notice I tend to use ED25519 type keys.
1Password SSH agent
Later on, I realised 1Password (our password management solution) has a neat feature that allows users to store SSH keys that can be loaded through its own SSH agent (note that the default Windows one mentioned above must be disabled for this extension to work).
To enable the feature, go to 1Password settings, and then developer. You’ll see the following menu where you can tick the Use SSH agent feature.
Once that’s done, each time you try use a SSH key, a notification will be displayed indicating you need to authorise the request. I have it set up to use my fingerprint, via Windows Hello. Notice it says "1Password is trying to allow..." in the following screenshot.
Another benefit to using 1Password is that I can also automatically sign git commits. Read more here.
To enable signing in WSL you can update your .gitconfig file and add the following:
[user]
name = Naz Markuta
email = [email protected]
signingkey = ssh-ed25519 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
[commit]
gpgsign = true
[gpg]
format = ssh
[gpg "ssh"]
# Use local or system install
# program = "/mnt/c/Users/<USERNAME>/AppData/1Password/app/8/op-ssh-sign.exe"
program = "/mnt/c/Program Files/1Password/app/8/op-ssh-sign.exe"
[core]
sshCommand = ssh.exe
WSL and SSH config
The following applies to users who use Windows systems alongside a WSL VM.
Command alias
Note: This is just for
git cloneactivities. You still need to specifyssh.exein theGIT_SSH_COMMANDenvironment variable forgo getcommands to work with private repos.
To clone private git repositories on my WSL system, I have an alias to the Windows version of SSH (ssh.exe), rather than using the default Linux SSH client. You can add the following lines to your ~/.bashrc or ~/.zshrc files to create one:
alias ssh="ssh.exe"
alias ssh-add="ssh-add.exe"
And then refresh your shell with source ~/.bashrc.
SSH config
Note: Ensure your private SSH key e.g.
~/.ssh/id_ed25519_github_wslis stored on the Windows file system, and NOT on the Linux file system. We are using the Windows OpenSSH agent, or in my case the Windows 1Password agent, to manage SSH passphrases.
This is what I have set to access GitHub a SSH config, so whenever I run git clone [email protected]:example/repo.git the command knows where to find the required authentication details.
# Github
Host github.com
User git
HostName github.com
PreferredAuthentications publickey
IdentityFile ~/.ssh/id_ed25519_github_wsl
There are also advanced use-cases for running with 1Password available here. For example, where you use the IdentityFile field to point to the SSH public key generated by 1Password, or the IdentityAgent field that points to a network socket.
Summary
The default behaviour of go get for downloading modules that are in private repositories could be improved when using SSH for authentication. This post shows how to configure and apply workarounds to help developers that want to use SSH keys for Git in non-traditional set ups, such as Windows Subsystem for Linux (WSL).
Release notes
Naz wrote this blog last year, and for various reasons (like building a new website and rebranding the company and two products) we never got around to publishing it. But there’s no point just leaving it in draft for ever, so voila! Sorry Naz, and all the best…
Related Posts
