If the wall under the desk isn't particularly visible, you could always like, tape some paper up to the wall to catch the dirt.