Joe Manganiello Not Dating Caitlin O'Connor Just For Chance to Have Kids

Joe Manganiello and Caitlin O'Connor are talking about starting a family, but that's not the only reason they're together ... despite how Sofia Vergara feels about her ex-husband.

Sources close to the couple tell TMZ … Sofia's claims about Joe divorcing her because she wouldn't have children, negatively followed him to his new relationship with Caitlin.

We're told the Joe/Sofia narrative that's out there is fueling internet trolls to attack Caitlin on social media ... blasting Joe's actress girlfriend as "only a babymaker."

Our sources say the online hate is getting so bad, Joe's removing some negative comments from his social media posts featuring Caitlin and his family.

Joe pushed back on Sofia's narrative this week in an interview with Men's Journal ... saying he and his now ex-wife had deep talks about having kids from the start, but he made it clear to her if she was done having children, he'd understand.

He also said the divorce was more about growing apart than starting a family.

Now, our sources are doubling down ... telling us Joe isn't just with Caitlin because she will give him children. We're told they both have an undeniable bond, an incredible connection and are deeply in love.

In fact, we're told Joe and Caitlin didn't even discuss children until a couple months into their relationship ... and they both agree they want kids.

Our sources characterized these conversations as part of the normal progression for many relationships, stressing kids weren't the be-all, end-all for JM and CO.