Handle Turkish uppercase and lowercase correctly, need to modify/override built-in functions? : <p>I am working with multilingual text data, among others with Russian using the Cyrillic alphabet and Turkish. I basically have to compare the words in two files <code>my_file</code> and <code>check_file</code> and if the words in <code>my_file</code> can be found in <code>check_file</code>, write them in an output file keeping the meta-information about these words from both input files.</p>

<p>Some words are lowercased while other words are capitalised so I have to lowercase all the words to compare them. As I use Python 3.6.5 and Python 3 uses unicode as default, it handles lowercasing and later on capitalising the words correctly for Cyrillic. For Turkish however, some letters are not handled correctly. Uppercase <code>'İ'</code> should correspond to lowercase <code>'i'</code>, uppercase <code>'I'</code> should correspond to lowercase <code>'ı'</code> and lowercase <code>'i'</code> should correspond to uppercase <code>'İ'</code> which is not the case if I type the following in the console:</p>

<pre><code>&gt;&gt;&gt; print('İ'.lower())
i̇  # somewhat not rendered correctly, corresponds to unicode 'i\u0307'
&gt;&gt;&gt; print('I'.lower())
i
&gt;&gt;&gt; print('i'.upper())
I
</code></pre>

<p>I am doing as follows (simplified sample code):</p>

<pre><code># python my_file check_file language

import sys

language = sys.argv[3]

# code to get the files as lists

my_file_list = [['ıspanak', 'N'], ['ısır', 'N'], ['acık', 'V']]
check_file_list = [['109', 'Ispanak', 'food_drink'], ['470', 'Isır', 'action_words'], [409, 'Acık', 'action_words']]

# get the lists as dict
my_dict = {}
check_dict = {}

for l in my_file_list:
    word = l[0].lower()
    pos = l[1]
    my_dict[word] = pos

for l in check_file_list:
    word_id = l[0]
    word = l[1].lower()
    word_cat = l[2]
    check_dict[word] = [word_id, word_cat]

# compare the two dicts
for word, pos in my_dict.items():
    if word in check_dict:
        word_id = check_dict[word][0]
        word_cat = check_dict[word][1]
        print(word, pos, word_id, word_cat)
</code></pre>

<p>This gives me only one result but it should give me the three words as result:</p>

<pre><code>acık V 409 action_words
</code></pre>

<p>What I've done so far based on <a href="https://stackoverflow.com/questions/19703106/python-and-turkish-capitalization?utm_medium=organic&amp;utm_source=google_rich_qa&amp;utm_campaign=google_rich_qa">this question</a>:</p>

<ol>
<li>Read the accepted answer which proposes to use <a href="https://pypi.org/project/PyICU/" rel="noreferrer">PyICU</a> but I want my code to be useable without people having to install stuff so I didn't implement it.</li>
<li>Tried to <code>import locale</code> and <code>locale.setlocale(locale.LC_ALL, 'tr_TR.UTF-8')</code> as mentionned in the question but it didn't change anything.</li>
<li><p>Implement two functions <code>turkish_lower(self)</code> and <code>turkish_upper(self)</code> for the three problematic letters as described in the second answer which seems to be the only solution: </p>

<pre><code>def turkish_lower(self):
    self = re.sub(r'İ', 'i', self)
    self = re.sub(r'I', 'ı', self)
    self = self.lower()
    return self

def turkish_upper(self):
    self = re.sub(r'i', 'İ', self)
    self = self.upper()
    return self
</code></pre></li>
</ol>

<p>But how can I use these two functions without having to check <code>if language == 'Turkish'</code> every time? Should I override the built-in functions <code>lower()</code> and <code>upper()</code>? If yes, what is the pythonic way of doing it? Should I implement classes for the various languages I'm working with and override the built-in functions inside the class for Turkish?</p>
