Yes it is the overflow container and no it should not be empty. There is a full mark on the container. Like I said if you are repeatingly having to add antifreeze then get it checked out before you do any further damage.