Ah, now i see what you mean
No idea why that arrow is persistent on your machine.
Over here, it behaves as expected, the arrow is only visible when the mous hoovers over the button.
Try adding another button to kicker that opens a menu (like the print applet or a application menu) and check if that button also has a permanent arrow.
If you want to checkout if there is a difference between the used icons: you can find the corresponding .desktop file at
$KDEDIRshare/apps/kicker/builtins/kmenu.desktop
Regards, Rinse