Pam Oliver -- I Didn't Mean to Insult New Yorkers ... But I Won't Apologize

"NFL on Fox" sideline reporter Pam Oliver says she was only KIDDING when she ripped New Yorkers at a Super Bowl event in NY last week ... but Pam tells TMZ she refuses to apologize.

FYI -- Oliver's been taking some serious heat ever since she asked the Super Bowl Host Committee how they planned to "instruct people of this area, particularly New Yorkers, to be nice" when the SB comes to the Big Apple in a couple of months.

Yesterday in NY, Pam addressed the issue head on -- saying she understands why people are pissed ... but chalks the whole thing up to a bad joke.

Pam says she loves New York ... her husband's from NY ... and she even owns an "I Love NY" shirt.

But don't expect an apology -- because even though Pam says she learned a lot from the controversy, she doesn't feel like she did anything wrong.