Tip: clicking the "+" sign will also make a new folder; if you accidentally clicked on a new folder command, simply hit the escape key (Esc) to abort the operation.
Note: you cannot rename the standard folders, or the "Saved Mail" default storage folder. Clicking on the gear will also give you the total message count and the size of the folder.