Reflections. Signals don't only go directly from one place to another. At 2.4GHz, they bounce around quite a bit. When the various paths converge at one point, they sometimes cancel and sometimes add. These reflections are not stable as things do tend to move around. Done properly, you can actually move a metal reflector around until you have a deep null (no signal) or a spectacular peak.
What else is moving around during the day? If the two radios can see each other directly, you'll have a fairly solid and stable signal. However, if you're shooting through a wall or two, or have highly reflective (metal) environment, it's going to vary. How much variation is difficult to predict.
Ideal size for a 2.4GHz reflector is 1/2 wave which is about 6.25cm. Therefore, it doesn't take much metal to make a fairly good reflector.
You can improve things by NOT using an omndirectional antenna, which has the worst time with reflections to/from all directions. A reflector or directional antenna will be a big help instabilizing the signal. Since the client end is stationary, that's fairly easy to do depending on what you're using for hardware.
You might also want to fire up Netstumbler and watch the signal strength change as you walk around the room. I can see signal strength changes as cars drive by outside my office picture window perhaps 10m away.