Casey: I Did Not Have Sex With That Woman

Our sources connected with Casey and Co. say he's ballistic at reports that he had an affair while Jamie Lynn was six months pregnant.

As for the picture of Casey with Kelli Dawson (the Sienna Miller of the South), he claims it was taken before he even hooked up with J.L.

Casey has called the Spears clan to vent and they believe his story.

And Casey has a point ... where he lives, there are just a bunch of one horse towns, so everybody knows everyone else's business.

Casey says it's all about Southern Sienna trying to make a buck.