Most likely your water heater needs to be flushed.

CAUTION: Most manufacturers recommend hiring a professional to flush your water heater. If you plan on doing this yourself, read the owner's manual to keep from being hurt and or damaging the water heater.