Does it only happen when you have the AC on by any chance? If so, the AC will draw enough electrical load that the brake booster vacuum is affected. Because the brake booster affects the pedal height, a change in vacuum will directly lower the pedal height, that's why you might feel the loss in brake pressure when the AC is drawing load on the system. Next time test and see if it only happens with the AC on, if it does then it's most likely just the AC load that's affecting the brake pressure. The other thing to check might be uneven pads or rotors, they tend to produce that spongy effect on brake pressure as well.