Unlike Outlook 2007, Outlook 2010 will not hide itself by default when you minimize it.
Right-click on the Outlook icon in the taskbar next to the clock.
Tip: if you can't see it, click on the arrow button.
Select "Hide when minimized" from the context menu: the feature is enabled when a checkmark shows next to that option. If you still see Outlook in the taskbar, click on it to restore it, and once more to minimize. The taskbar button will be hidden.
Double-click on the notification area icon to show Outlook, or press your keyboard shortcut.
Note: only Outlook's main window is affected; messages, reminders, etc. minimize normally. To show minimized Outlook, just uncheck that option from that same dropdown menu.
To start Outlook minimized, right-click on the shortcut you launch it with: choose "Properties" and confirm that the "Shortcut" tab is selected. Choose "Minimized" the "Run", and click "OK".