Are you clicking on the X in the upper right corner when

you have only one document open? That does exit the program in 2010. Clicking on the X when more than one document is open closes that specific document only.

You can use Ctrl + W instead when one document is open. You can also add the File/Close command to the Quick Access Toolbar if you must use the mouse.