After upgrading to 4.0 (and now 4.0.1), my gems are not appearing under External Libraries in the left sidebar. I am now running RubyMine 4.0.1 on OSX 10.6.8 (Snow Leopard).
If I open the same project with RubyMine 3.2.4 I see the gems. The gems are all defined in a Gemfile at the top level of my Rails project. I'm using rvm, and the gems all appear as expected in the RubyMine settings dialog under "Ruby SDK and gems." The External Libraries node in the sidebar just has directories: 1.9.1, 1.9.1, 1.9.1 (library home), rubystubs19, site_ruby, vendor_ruby, x86_64-darwin10.8.0, x86_64-darwin10.8.0, x86_64-darwin10.8.0.