Definitely tell them it's too expensive for you ($4,000 for the two of you? That's a car, son!). Is there no way you could find a cheaper place nearby to stay? It sounds like you still want to go, just not at this price. There certainly shouldn't be a requirement that you stay at the same place.