� is used to represent an invalid character, so it makes sense that it'd appear often when bad data is being rendered (or good data is being rendered improperly).

Everything is 0s and 1s to a computer. What a pattern of 0s and 1s encodes is decided by people--often arbitrarily. Over the years there have been attempts to standardize encodings but, for legacy reasons, older encodings are still valid.

The 0s and 1s that encode ' in UTF-8 (a standardized encoding) are the same 0s and 1s that encode ’ in CP-1252 (a legacy encoding).

The � symbol is shown when the 0s and 1s don't encode anything of meaning.

Right single quote (’) in UTF-8 (https://www.compart.com/en/unicode/U+2019) has the same bytes as ’ in cp1252 (which is more or less "ASCII" if we're doing ELI5). https://en.wikipedia.org/wiki/Windows-1252

Sometimes your keyboard or program's settings will use right single quote for apostrophes instead of a normal apostrophe. ' you might notice how this one is straight and not bent a certain way. This setting is often called smart quotes.