For those people, and you know who you are, here's a slightly convoluted solution that certainly seems to work on OSX 10.8 (Mountain Lion), even after a reboot.
Third-party software required
Key Remap 4 MacBook and PC Keyboard Hack
When you've installed them both you'll need to reboot your Mac.
1) Go into System Preferences -> Keyboard, click on Modifier Keys… and change Caps Lock Key to No Action.
2) Repeat for each keyboard, then click on OK and Show All
3) Click on PC Keyboard Hack in the Other section at the bottom of System Preferences
4) Make sure the Setting pane looks like this, with Change Caps Lock checked and the keycode set to 110
5) Click on "Show All" at the top, then go into KeyRemap4MacBook, again located in the Other section
6) Scroll down till you find the entry that reads For PC Users
7) Click on the right arrow to show the contents and, inside this, open up Change PC Application Key and make sure the option that reads Application Key to CapsLock is checked, like this :-
8) Close preferences. That was easy, right ?