Should be an option to downgrade existing buildings to allow conquered castles to be altered to the players preferred set up.
Click on say the market and the building can be downgraded a level. It would take half the time the building took to upgrade and you get a portion of your resources back.
Help maximise troop numbers and good be speeded up by gold.