Frat to Johnny Manziel -- You're TOO YOUNG to Party & Booze Here

Johnny Manziel was asked to leave a University of Texas frat party this weekend because he tried to drink a beer ... and frat officials became worried he would be a liability because he's only 20 years old, TMZ has learned.

The Heisman Trophy winner has been at the center of a media firestorm ever since a video emerged showing the Texas A&M quarterback being escorted out of a UT frat party Saturday night ... while one guy screamed, "Get the f**k out!!"

But our frat sources tell us ... the decision to boot Manziel wasn't based on school rivalry -- it was about staying out of trouble with school officials.

One source tells us ... some of the brothers became concerned when they saw Johnny taking pictures at the party while holding a beer in his hand ... and worried that a media backlash would result in the school punishing the fraternity for underage drinking.

So, we're told, the fraternity brothers quickly made the decision to ask Johnny to leave the party ... and during his exit, one of the frat guys decided to act like a frat guy ... and started with the heckling.

As we previously reported, Manziel is legally ALLOWED to drink in the state of Texas ... as long as he's accompanied by a parent.