How to Fix json.loads Unexpected UTF-8 BOM Error in Python

In Python, You will get an error while retrieving the data from any 3rd party API request. In fact, when response content converts to JSON format using json.loads method, it throws an json.decoder.JSONDecodeError: Unexpected UTF-8 BOM error. In this article we are going to see how to fix json.loads() Unexpected UTF-8 BOM error in Python.

How to Fix json.loads Unexpected UTF-8 BOM error in Python. We have seen solutions to fix Unexpected UTF-8 BOM errors when using json.loads in Python.

The error was occur by the json.loads(r.text), so when text content convert to JSON format we have getting following error:

json.decoder.JSONDecodeError: Unexpected UTF-8 BOM (decode using utf-8-sig): line 1 column 1 (char 0)

The response content coming from the API, but \ufeff Unicode character coming at the beginning. This Unicode character value \ufeff (or \xef\xbb\xbf in binary) is a byte order mark (BOM) character.

Python: Fix json.loads Unexpected json.decoder UTF-8 BOM Error

Following are 4 different solutions. Basically all following solutions we have to decode the data using utf-8-sign encoding. This way we can fix the error.

Solution 1 Decode content using utf-8-sig

In this solution, we can use decode() method on the return value of the string.encode() method. This is the most efficient solution to fix this error.

decoded_data = r.text.encode().decode('utf-8-sig')
data = json.loads(decoded_data)

Solution 2 Decode response content

This solution is a straightforward method to fix this issue. We have used the decode() method on r.content.

decoded_data = r.content.decode('utf-8-sig')
data = json.loads(decoded_data)

Solution 3 Encode requests.response object

In this solution, you can use the encoding property on the response object. This way we can skip the previous examples showing the calling of encode() and decode() methods.

r.encoding = 'utf-8-sig'
data = json.loads(r.text)

Solution 4 Use Python codecs module

You can use the Python codecs module. We can use the codecs.decode() method to decode the data using utf-8-sig encoding. The codecs.decode() method accepts a bytes object. Thus, you have to convert the string into a bytes object using encode() method.

decoded_data = codecs.decode(r.text.encode(), 'utf-8-sig')
data = json.loads(decoded_data)

Bottom Line

All in all, if the json.loads() method throws an unexpected UTF-8 BOM error. It means BOM values are existing in the response data. In this article we have seen 4 different solutions to rid out this json.loads Unexpected UTF-8 BOM error in Python.

Another most common error: object arrays cannot be loaded when allow_pickle=false. You can check fixexception to find an appropriate solution.

We hope you have found this article helpful. Let us know your questions or feedback if any through the comment section in below. You can subscribe our newsletter and get notified when we publish new articles. Moreover, you can explore here other interesting articles.