I had the same issue. What we did was tried to decide if our house was going to be an investment or a dwelling. Once we decided that was where we wanted to raise our kids until they are 18, then the decision was easy.

I now don't care if the values go up or down. We just made sure we could pay the mortgage even in very hard times. i

I think if we decided to go the other way, as an investment, we would have held on for just a little longer until we saw something positive in the economy.