Miss character when using add_redact_annot to replace English text to Arabic text

[Trung-eta]

Hi everyone,
I am creating a PDF translation tool based on OpenAI, then I will replace the text before translation with the text after translation. However, when I translate into Arabic and use the add_redact_annot and apply_redactions functions, some lines lose their content. I am using the NotoSansArabic-Regular.ttf font. I truly do not understand the cause of this issue, as in other languages like Japanese and Vietnamese, I do not encounter this problem.
I send pdf files before and after my translation for everyone to see. My tool only translates English words and keeps Japanese words intact
Could someone provide me with some suggestions to continue investigating? I would greatly appreciate it.

Here is my processing code:

def exportMultiplePagePdf(input_path, word_positions, translated_words, lang):
    output_path = 'pdfs/output.pdf'

    print('lang', lang)
    print('lang', font[lang]['file'])

    pdfmetrics.registerFont(TTFont(font[lang]['name'], font[lang]['file']))
    packet = io.BytesIO()

    doc = fitz.open(input_path)

    # Create a new PDF with Reportlab
    output = PdfWriter()

    # Create a new packet and canvas for each page
    for page_num in range(doc.page_count):
        packet = io.BytesIO()

        page = doc[page_num]
        height = page.rect.height

        page.insert_text(
            (20,20),  # just somewhere on the page
            " ", # just anything - it will be invisible
            fontname=font[lang]['name'],
            fontfile=font[lang]['file'],
            render_mode=3,  # 3 makes text invisible!
            )

        if page_num < len(word_positions) and word_positions[page_num]:
            for i, rect_str in enumerate(word_positions[page_num]):
                position = parse_rect(rect_str)

                block_width = position[2] - position[0]

                if block_width < 50:
                    position[2] = position[0] + 50

                x, y, _, _ = position
                y = height - y - 2

                if lang == 'Arabic':
                    translated_words[page_num][i] = translated_words[page_num][i][::-1]
                # translated_words[page_num][i] = 'ك'
                page.add_redact_annot(position, text=translated_words[page_num][i], fontsize=10, align=1, fill=(0.5, 0.5, 0.5), text_color=(0, 0, 0), cross_out=True, fontname=font[lang]['name'],)
        else:
            words_logger.info(f"There is a page that cannot be translated:\n")

        page.apply_redactions()

        doc.save(packet)

        # Move to the top of the bundle and add canvas content
        packet.seek(0)
        new_pdf = PdfReader(packet)
        output.add_page(new_pdf.pages[page_num])

    # Write the output to a real file
    with open(output_path, "wb") as output_stream:
        output.write(output_stream)

    doc.close()

before.pdf
after.pdf

[JorjMcKie]

Many scripting systems, especially in Asia are complex and require post-processing before writing, called "text shaping". Examples are Devanagari-based languages and also right-to-left languages like Arabic, Persian and Hebrew.
For these languages, character-based text writing methods like page.insert_textbox() should not be used: the output will very likely be wrong.
Use page.insert_htmlbox instead. Apart from text shaping, this method supports a plethora of other cool features.