Amateur astronomers use that trick to see barely visible objects such as comets with the unaided eye; faint-object perception is better with slightly averted vision than it is straight on, because there are fewer rods right in the fovea.

(Also, I think it's not so much color vs. magnitude perception as it is bright vs. faint light perception. The rods are monochrome sensors, but it's the cones that actually provide the brightness information in bright light; the rods are completely oversaturated under daylight conditions and don't do anything useful. There are people with a rare condition that makes their cones completely nonfunctional; they see in complete monochrome, unlike most color-blind people, and they also have to wear dark sunglasses during the day or they can't see at all.)