Ministers say PM had 'up to date' data before delaying Freedom Day