Don't do anything that Kojic guy tells you to, I'm not sure he's even read the correct thread.
For starters, what is your actual GPU? A 2GB Ati card doesn't tell us penis.
Now is the whining most certainly coming from your case? Definitely not from your speakers? Basically if you can run games on high settings & your PC does not crash/artifact, it's most probably not your PSU. If it can handle that load, it can do almost anything.
There are 2 plausible explanations that straight away pop into my head & they both involve the GPU :
1. - When you run higher graphics settings your GPU is spinning the fans faster for more cooling since it runs hotter, if the fan bearings are dodgy or badly lubricated it could simply be the fans "squeaking" but of course at 2000RPM it all sounds like a whine. Higher you push the settings, hotter the GPU gets, faster the fans spin, louder that whine gets.
This explains why it whines at high settings. To test if this is the cause, get software for your GPU & manually change the fan speeds. Set it to the lowest then set it to the highest and see if there is a difference. If it's possible, physically open your case - that should help you identify where the noise is coming from.
2. - On the other hand when you are running a less GPU taxing game such as Minecraft & your GPU is able to render it at something ridiculous like 200FPS, you can get whining from the capacitors or voltage regulators of your GPU. It's not exactly a problem but it can be annoying. It's literally just your GPU whining cos it's rendering this game so easily & fast that it's becoming bored. It is too fast & has too much potential which is not being used - just like how a genius kid would get bored in a class full of normal kids.
This would explain why it whines when you look at the floor in Minecraft - ie. When there is nothing much for your GPU to actually render your framerate shoots up and that makes the whine worse. To test if this is the cause, load up any other low-end game and drop all of the settings to the lowest. With low graphics settings & resolution to render, your GPU will barely be doing any work and will probably start whining again from being bored.
Give that a shot & let us know what's up.