The contractor came to look at our house. The good news is that he doesn't think that he will need to take any of the drywall out. He seems to think that he will be able to get the plywood out without too much damage to the siding, and we will the replacing the wood molding around the jutting out part of the house, because it is all rotting too.
He says it will probably be between $800 and $1000. It's looking like it will be a lot bigger job than I thought it would be.
Ps. I just realized that I may not have written about this before. If you want/need the backstory or a better explanation of what is going on, say so in the comments.