Try updating the driver (try the proprietary one if it exists and you are not using it). Consider also googling for problems associated with your graphics card and/or the VMware with a graphics card and solutions people have found for them.
My personal guess is that it is a problem with VMware or your configuration of VMware. Consider changing the Accelerate 3d graphics setting under Virtual Machine > Settings > Display.
If you are able, I would actually go with a dual-boot setup for Ubuntu and Windows (which is what I use). I’ve never used Ubuntu through VMware, so I can only speculate as to what may be the cause of your problem…