Since moving to OS X 10.6.5 and switching to 64bit, I have noticed from time to time the Apple Finder Dock can get confused. This usually happens on a right-click or control-click of an icon in the Dock to hide an application, force quit a running application or to use a contextual menu from the app icon in the Dock. This is kind of annoying. If it never happens to you, great and consider yourself lucky. If you also get hit with this every once and while, you know what I mean. You can logout to restart the Dock app or even reboot the computer to get it to restart the Dock, but here is a quick way, even if you never even use Terminal.app to quickly rectify the issue.
Here is what a piece of the OS X Dock looks like on my computer in normal mode
Here is what the same Dock looks like when it goes into what I call ‘Dark Dock’
They don’t look all that different. Apple did this to mimic the functionality of Expose but for the Dock to try and give you some eye-candy to provide user focus on the icons you are working with which is great for the brief time you might be messing with something, but if you do have magnification on and it gets stuck you lose that as well as it feels like the Dock is frozen and waiting for you to hit an option or select something.
A quick way to restart the Dock and shake it loose is to drop into a Terminal / Console and issue a kill -HUP to the Dock process. This will stop and start the process for the Dock again and kick it back to normal. A quick way to do that is start the Terminal.app (Applications/Utilities) and then issue two commands. Pretty easy to do even if you are console-phobic.
You first need to find what the Dock process is and you can do that via this command.
ps -ef | grep Dock
It will give you the process number of the Dock process. You will need that for the second command to restart it. The process id is the second number. A command to restart the Dock process once you know the ID is
kill -HUP 187
In this example 187 is the number I got back from the first command. Here is a screenshot of my console in Terminal.app running this.
In the screenshot, I got 197 back as the ID of the Dock process. Then, I issued the kill -HUP 197 to toggle the Dock to get it unstuck.