How can a pre-existing marriage between a convert and an unbeliever be classified as adultery?

From my last question in Ramadan. I can sort of, kind of see the marriage being annulled. I can try to not have an issue with that meaning the converted party is up for grabs for any believer. But this part just doesn't make any sense.