How to fix unicode REPLACEMENT CHARACTER � in postgres database

All we need is an easy explanation of the problem, so here it is.

Is there a way fix the � problem in postgres database:

How to fix unicode REPLACEMENT CHARACTER � in postgres database

As you can see there are names with accents and are not displayed corretly in the database.
I would like to know if there is a way to fix this without dropping the database and restoring it… ?

When I execute

SELECT CAST (userfirstname AS bytea) FROM tab WHERE id = 42;`

this is the result i get:

               bytea
--------------------------------------
 \x5374efbfbd7068616e696520284d6d6529
(1 ligne)

How to solve :

I know you bored from this bug, So we are here to help you! Take a deep breath and look at the explanation of your problem. We have many solutions to this problem, But we recommend you to use the first method because it is tested & true method that will 100% work for you.

Method 1

That character (0xEFBFBD in UTF-8) really is Unicode 0xFFFD, the “replacement character” �.

So if that’s not what you want, the problem must have occurred when you inserted the data into the database. PostgreSQL does not surreptitiously modify strings, so some software most have done that before the data were put into the database.

Since the characters you would like to see are not all equal (I spot é and ç, for example), there is no automatic way to repair the damage.

Note: Use and implement method 1 because this method fully tested our system.
Thank you 🙂

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0

Leave a Reply