You might try using the keyboard shortcuts instead of the mouse.

You'll find all the Windows shortcuts with a web search.

I've never used the icon to go to my Desktop. That shortcut toggle is WinKey+D.

There are also ways to get around this new feature in Windows 7 if you use Instant Text 7 Pro.