That stuff you are talking about is carbon.

Replacing it is up to you, but remember that the exhaust gas coming out of that crack is usually between 1300 and 1500 degrees F. If that doesn't bother you, keep using it. I would replace it now if it was my car.