http://arxiv.org/abs/cond-mat/0502336

I'll save you most of the reading. Some eggheads have come up with a way to use plasma fields to block the transmission of light in a way that an object covered in these fields becomes invisable to visable light. By changing the composition of the plasma field they can block infrared or ultra violet instead.

Lots of the things we have to deal with seem to be able to disappear in plain sight - is this how they do it?

It's possible that some kind of being somewhere might be able to do this, but the beings you and I deal with don't exactly conform to to the laws of physics as they are currently understood. My understanding is that different beings use different methods to get the same result.

"Clouding of men's minds," to borrow a phrase from The Shadow, is one way that I have run across. I've seen security footage where a person/creature has slipped past armed security guards or other persons unnoticed. I've also read reports of the opposite, where eyewitnesses describe seeing/fighting something, but security cameras can't pick the person/thing up (at least in the normal visible spectrum).

There are also documented cases of martial arts masters who have trained the ability to stay just out of peripheral vision by moving constantly, dropping prone, moving to the side or behind the person, etc. The techniqe involves no magic other than the physical demands of constant erratic movement, silence, and anticipating where the person will look next. With this being possible, it opens the possibility of other stories, like ninjas, or the infamous "smoke Thuggees" of India.