Randy Jackson -- Nearly an 'American Idol' Casualty

Sources close to production tell TMZ there was "serious talk" of a clean sweep of the judge's panel ... but we're told "Idol" brass wanted to keep a connection to the original judge's table, so Randy was spared.

As we previously reported, Simon Cowell left for "The X Factor," Ellen DeGeneres stepped down after one season and Kara DioGuardi was let go -- leaving Randy as the only remaining judge.