Let’s take a look at Lev 21:12.
I could not find exact verse because I forgot, but perhaps Lev 16:4?
As far as I understand as long as a priest is dressed in a Priestly Garments, he can't go beyond Tabernacle courtyard. I think this is it.
During the transportation of the Tabernacle a priest would be dressed in his own, normal garments.
I was thinking of that as a possibility (remove garments to leave the court) but I have not found adequate textual support yet.
There is Lev 6:11, referring to going outside the camp.