Rather than trying to soldier on with a partially allocated cache just
throw the cache away and pretend we don't have one in case we can get a
full cache next time around.

Signed-off-by: Mark Brown <broonie@opensource.wolfsonmicro.com>