For upshifting it won't hurt anything. But I won't do it regularly for downshifting. They have a pushbutton lockout on 2nd and 1st gear when downshifting for a reason.

I regularly switch the selector into D3 when going down steep hills for more engine braking in 3rd gear to increase brake life.