HomePage of Laszlo SZATHMARY | Python / unicode to ascii

#############################

Migrated to https://pythonadventures.wordpress.com/2010/12/17/unicode-to-ascii/.

Problem

I had the following unicode string: "Kellemes Ünnepeket!" that I wanted to simplify to this: "Kellemes Unnepeket!", that is strip "Ü" to "U". Furthermore, most of the strings were normal ascii, only some of them were in unicode.

Solution

import unicodedata

title = ...   # get the string somehow
try:
    # if the title is a unicode string, normalize it
    title = unicodedata.normalize('NFKD', title).encode('ascii','ignore')
except TypeError:
    # if it was not a unicode string => OK, do nothing
    pass

I used the following resources:

http://www.peterbe.com/plog/unicode-to-ascii
http://stackoverflow.com/questions/196345/how-to-check-if-a-string-in-python-is-in-ascii