it does not hurt anything if you dont grind the gears while doing it. It is all a matter of engine speed and timming.

driving a truck, you usually dont use the clutch, only for take-off and downshifting... but once you get really good, you can downshift without the clutch too.