While working on Thoughtworks Go as our Continuous Delivery tool, we needed to setup SSH keys so the agents and the server can communicate with Github and Stash.
The location of the keys was a bit of a mystery, and how/where to store the keys an interesting problem.
On Linux for both the agent and server, Go will create a user and group called "go", and sets the home directory to "/var/go". You can find this by looking at the /etc/passwd file (cat /etc/passwd | grep go:). By default, this directory does not exist, so you'll need to create it with read permissions restricted to the go user.
For windows, it is pretty easy - create a user that the service should run under, and use that user's home directory.
For securing the keys, we ended up using ChefVault. It took a bit of time to learn it, but it is a really handy utility. We are storing not only our SSH keys in this, but all our service account passwords too!
Below is a quick example of a recipe you can use to setup your SSH keys on a Linux server. Only some minor edits needed for Windows