Note: this tutorial applies to Outlook 2010, 2007, and 2003 - on Windows 7, Vista, and XP.
Outlook displays a new mail icon in the taskbar whenever new messages are downloaded from the server, in addition to showing a "desktop alert":
If you don't hide Outlook when minimized, its taskbar button will also change to show a yellow envelope icon in overlay to indicate new, unread emails:
These are redundant, so you can hide the new mail icon: in Outlook 2010, click on File (top left), and choose "Options". In Outlook 2007 / Outlook 2003, go to "Tools > Options".
In Outlook 2010 / 2007, select "Mail" on the left: under Message arrival (third section), uncheck the "Show an envelope icon in the taskbar" and click "OK" to apply.
Outlook 2003: click "E-mail Options" in the Preferences tab, and "Advanced E-mail Options". Uncheck "Show an envelope icon in the notification area" and click "OK" three times.
You can also hide Outlook icon from the taskbar altogether:
In Windows 7, click on the arrow button in the notification area, and select "Customize". Select "Hide icon and notifications" from the dropdown next to Outlook and click OK.
In Windows Vista, right-click on the start button and choose "Properties"; select the "Notification Area" tab and click "Customize" next to Hide inactive icons. Pick "Hide" from the dropdown next to Outlook and click OK twice.
In Windows XP, right click on the taskbar and choose "Properties"; click "Customize" next to Hide inactive icons. Choose "Always hide" next to Outlook and click OK twice.