Cost of living is not the sole indicator of how rich a city, town, county or state is. In California and NY you are taxed heavily...moreso than in NY. The taxes my wife pays to the city, county and state because she only works in NY is absolutely crazy...God forbid we actually lived there.
Take a state like NH for example...they only tax your property and prepared food. So your house, car, and the meal you ate at the Outback was taxed...but not your salary that was used to pay for those things. Cost of living there is lower because of this...however many of the costs are the same. My parents house for example is only slightly lower in value in NH than it would be in Fairfield, CT.
The wealth that is in Fairfield county is astronomical but the cost of living is not as high as other areas such as California or Westchester.