Lonnie Brantley Jr. avoided prison by striking a plea deal for lying to federal housing regulators during an investigation into his troubled mortgage lending business.

But almost three years after his 2015 conviction, Brantley is facing the prospect of being locked up. That’s because he has repeatedly dodged his $3.3 million restitution obligation, prosecutors say, while living in a 5,300-square-foot Southlake home, driving luxury vehicles and taking lavish vacations to Horseshoe Bay outside of Austin and other places.

The government is asking a judge to find Brantley, 55, in default and force him either to sell his home and other assets and increase his monthly payments to $2,775 or to send him to prison...