I have 2 build agents available, however I have multiple items in my queue waiting to build. The jobs don't seem to be getting assigned to any of the agents. Each of the jobs in the queue lists both build agents as being compatible with the build. I don't see any errors in my build agent logs and I don't see any errors in server logs. I have rebooted all machines involved multiple times. All are running the latest 8.0.1 version of TeamCity. All jobs show the Time To Start as "Delayed". Even if I clear the queue and manually run the jobs, they still sit in the queue.
I have since completely rebuilt both build agents from scratch (new OS install, clean agent installation and configuration). Other symptoms I have noticed: If I restart the service, it takes a very long time to stop (I'm up over 20 minutes waiting). When I reboot the machine, I will get a few builds out of the system before it stops building again.
Could you attach screenshots of the build queue with the content of all the popups for one of the builds? (especially that over icon in "Time to start" column)
What agents are listed as compatible if you click on the agents link of the queued build?
Do you have any non-bundled plugins installed on the server?
If there are no obvious reasons, please attach server logs and a couple of server thread dumps. If you do not want to share those publicly, please create an issue in the tracker and attach there with limited visibility.
This morning, I changed the vm from 768MB to 1.75GB of RAM. This seems to have solved the issue for now. It is odd that this fixed the issue, as on 768MB of RAM, the system did not appear to be out of memory. I am going to run it this way for a few days and I'll let you know if the issue reappears. In the mean time, I will collect logs for you to see either way.