Is there any way to set up admin notifications when there are no build agents available? Particularly on upgrades, my local agent always seems to need to be reinstalled to become available and I don't get any notifications that there are no agents.
They should upgrade, but sometimes they don't even when the log says they're upgrading.
There could be other situations like you have an outside agent that goes offline and you won't know until you see pending builds with no agents available. To me, it would seem that the teamcity server process could easily monitor if there was a pending build that hasn't been served up or that there were no connected agents.
Teamcity indeed doesn't have enough built-in capabilities to help with administration. There's a summary ticket for issues like this: http://youtrack.jetbrains.com/issue/TW-2795
However, you can use REST API to retrieve list of agents and queued builds (http://confluence.jetbrains.com/display/TCD8/REST+API) and setup an external notification (zabbix, etc.) to notify you when situtation like this happens.