Turns out that this was due to the socket connection from Nginx timing out as GitLab has to compile all its assets when you're first attempting to connect, and because the VM I'm using is very small I was hitting this problem head on.
Thankfully, someone else had run into the same problem a few months ago and a solution was tucked away in a support thread. You can give ruby a kick by manually requesting that the resources be compiled as a final step in the install process.
sudo -u gitlab -H bundle exec rake assets:precompile RAILS_ENV=production
This churned away for a few minutes, and once complete I was able to access without any further issues, and with the CPU load remaining nice and low (2-3%).