Michael Oher, the retired NFL star whose life inspired the Oscar-nominated filmThe Blind Side, filed a legal petition to terminate a conservatorship on Monday allegingLeigh Anne Tuohy and Sean Tuohy lied about adopting himwhile making millions off his name.

Matthew Sharpe/Getty

Michael Oher #74 of the Ole Miss Rebels stands with his family during senior ceremonies prior to a game against the Mississippi State Bulldogs at Vaught-Hemingway Stadium

Michael Oher at the 2009 NFL Draft.Jeff Zelevansky/Getty

Baltimore Ravens #23 draft pick Michael Oher poses for a photograph with his family at Radio City Music Hall for the 2009 NFL Draft

Jeff Zelevansky/Getty

The movie went on to make more than $330 million at the box office, plus more as the film gained even more notoriety when it was nominated for Best Picture at the 82nd Academy Awards and Sandra Bullock won the Oscar Award for Best Actress for her portrayal of Leigh Anne.

The Tuohys and Oher did not immediately respond to PEOPLE’s requests for comment.

For years, Oher has expressed remorse over how the film depicts him — especially the way it frames him as a student who struggled in the classroom.

The Tuohys, however, alleged in their 2010 book,In a Heartbeat: Sharing the Power of Cheerful Giving, that they split the profits “five ways,” according toESPN, which first reported the news.

Oher first met the Tuohy family when he was a student at Briarcrest Christian School in Eads, Tenn., where he often stayed with classmates’ families because he was “on his own” and “nearly penniless,” according to Monday’s legal filing.

Oher had bounced around a number of foster homes since he was 11 years old, having been raised in a family of 12. His mother Denise had struggled with drug addiction and his father died in 2004.

Monday’s filing alleges the Tuohys were one of the families he would “occasionally” stay with during that time. “Where other parents of Michael’s classmates saw Michael simply as a nice kid in need, Conservators Sean Tuohy and Leigh Anne Tuohy saw something else: a gullible young man whose athletic talent could be exploited for their own benefit.”

Never miss a story — sign up forPEOPLE’s free daily newsletterto stay up-to-date on the best of what PEOPLE has to offer, from celebrity news to compelling human interest stories.The Tuohys took Oher on shopping trips and invited him to officially move in during the summer of 2004, according to the filing. “Almost immediately after Michael moved in, the Tuohys presented him with what he understood to be legal papers that were a necessary step in the adoption process,” the filing claims.

The conservatorship papers he wound up signing instead gave the Tuohys “total control” over Oher’s ability to sign contracts, the petition alleges. His filing claims the Tuohy family “falsely and publicly represented themselves” as Oher’s adoptive parents.

Oher’s relationship with the Tuohy family began to fall apart after the release of the 2009 film, which bothered him because it depicted him as unintelligent. He then learned he was the only member of the family not receiving royalty checks from the film, his attorney J. Gerard Stranch IV told ESPN.

source: people.com