Erik Bangemann at Ars Technica tries out Shower Shock, a soap laced with caffeine and peppermint oil. Erik's a coffee drinker and didn't get much except the peppermint pick-me-up, but his wife possibly got something. The article runs toward my first guess, which is that caffeine shouldn't penetrate skin very well. If you want to absorb the caffeine quickly you should wash your mouth out with the soap, or just make a bunch of small cuts. However, caffeine soap might help with psoriasis .
I googled around and see that Shower shock has been offered by thinkgeek for a while now. Dave Barry has weighed in with a serving suggestion.