Are you replacing frame and all or just the glass?

If it is just the glass then the caulking bead is usually on the inside of the window. In that case just scrape out the old caulking bead, set window and re-caulk it from inside.

If you are replacing the frame and all then I am also pretty sure from looking at the photos, that you will also have to do that from inside..

So I guess either way you are working from inside the basement.