Konsole/kwrite/other freeze when trying to display certain international character (e.g. \u0acb) #58

Basic information

TDE version: R14.0.11
Distribution: Debian GNU/Linux 11 (bullseye)
Hardware: amd64 (x86_64)

Description

At least konsole and kwrite freeze when trying to display characters like \u0acb (in UTF-8 \xe0\xab\x8b). It does not apply to all indic characters, for example \u0ab5 works well. Try (in bash in trinity konsole):

echo -e "\u0acb"

Change of the font did not change the situation.

It is not that unprobable to get to that situation even for people not using that language - for example I was investigating, why TDE panel launcher for "pavucontrol" stopped displaying icon after an upgrade from buster to bullseye. During that, I opened the file pavucontrol.desktop, and konsole freezed. The problem was with the line 15 of that file containg Name[gu]

(The same freeze did not happen in KDE konsole 20.12.3 using Qt 5.15.2).

To me it seems suspocious around the line
kernel/qfontengine_x11.cpp:2626 in bool TQOpenType::positionAndAdd

2626:    if (item->num_glyphs < (int)otl_buffer->in_length) {
2627:        item->num_glyphs = otl_buffer->in_length;
2628:        return FALSE;
2629:    }

otl_buffer_in_length is always 3 in this case
item->num_glyphs is rewritten to 2 in TQFontEngineXft::stringToCMap
qfontengine_x11.cpp:1702

1700:#endif // QT_XFT2
1701:
1702:    *nglyphs = len;
1703:    return NoError;
1704:}

in static bool indic_shape in qscriptengine_x11.cpp:1848

1846:        if (!indic_shape_syllable(openType, &syllable, invalid)) {
1847:            IDEBUG("syllable shaping failed, syllable requests %d glyphs", syllable.num_glyphs);
1848:            item->num_glyphs += syllable.num_glyphs;
1849:            return FALSE;
1850:        }

is infinitely increasing item.num_glyphs by 2 (without any effect on the condition in bool TQOpenType::positionAndAdd, because of bool TQOpenType::positionAndAdd reassignment in bool TQOpenType::positionAndAdd)

Steps to reproduce

Open trinity konsole with bash
Check encoding is UTF-8 and you have any UTF-8 locale
Do

echo -e "\u0acb"

It freezes.

Backtrace

of suspicious line

#0  TQOpenType::positionAndAdd (this=this@entry=0x555ffcd2d240, item=item@entry=0x7ffd295443b0, 
    doLogClusters=doLogClusters@entry=false) at kernel/qfontengine_x11.cpp:2626
#1  0x00007f43a38d8a43 in indic_shape_syllable (openType=openType@entry=0x555ffcd2d240, item=item@entry=0x7ffd295443b0, 
    invalid=<optimized out>) at kernel/qscriptengine_x11.cpp:1697
#2  0x00007f43a38d9b0c in indic_shape (item=0x7ffd29544460) at kernel/qscriptengine_x11.cpp:1846
#3  0x00007f43a38dc121 in TQTextEngine::shape (this=0x7ffd29544590, item=0) at kernel/qtextengine_unix.cpp:108
#4  0x00007f43a38daa1d in TQTextLayout::currentItem (this=this@entry=0x7ffd29544558) at kernel/qtextlayout.cpp:324
#5  0x00007f43a37e9ac0 in TQPainter::drawText (this=0x7ffd29544820, x=<optimized out>, y=<optimized out>, str=..., 
    pos=<optimized out>, pos@entry=0, len=<optimized out>, dir=<optimized out>) at kernel/qpainter_x11.cpp:3104
#6  0x00007f43a37e9df1 in TQPainter::drawText (this=<optimized out>, x=<optimized out>, y=<optimized out>, str=..., 
    len=<optimized out>, dir=<optimized out>) at kernel/qpainter_x11.cpp:3025
#7  0x00007f43a1225333 in TEWidget::drawAttrStr (this=0x555ffca3acd0, paint=..., rect=..., str=..., attr=0x555ffcd3fde4, 
    pm=false, clear=true) at ../konsole/konsole/TEWidget.cpp:750
#8  0x00007f43a12261a3 in TEWidget::setImage (this=0x555ffca3acd0, newimg=0x555ffcd3f9a0, lines=38, columns=91)
    at ../konsole/konsole/TEWidget.cpp:924
#9  0x00007f43a123e561 in TEmulation::showBulk (this=0x555ffcc97e50) at ../konsole/konsole/TEmulation.cpp:540
#10 0x00007f43a123c8fc in TEmulation::tqt_invoke (this=0x555ffcc97e50, _id=19, _o=0x7ffd29544c40)
    at konsole/konsole/TEmulation.moc:347
#11 0x00007f43a12314a2 in TEmuVt102::tqt_invoke (this=0x555ffcc97e50, _id=19, _o=0x7ffd29544c40)
    at konsole/konsole/TEmuVt102.moc:131
#12 0x00007f43a386a0a0 in TQObject::activate_signal (this=0x555ffcc97ee0, clist=<optimized out>, o=o@entry=0x7ffd29544c40)
    at kernel/qobject.cpp:2813
#13 0x00007f43a386a18e in TQObject::activate_signal (this=this@entry=0x555ffcc97ee0, signal=<optimized out>)
    at kernel/qobject.cpp:2747
#14 0x00007f43a3b10c05 in TQTimer::timeout (this=this@entry=0x555ffcc97ee0) at .moc/release-shared-mt/moc_ntqtimer.cpp:91
#15 0x00007f43a3888e5c in TQTimer::event (this=0x555ffcc97ee0, e=<optimized out>) at kernel/qtimer.cpp:222
#16 0x00007f43a381a558 in TQApplication::internalNotify (this=<optimized out>, receiver=0x555ffcc97ee0, e=0x7ffd29544cf0)
    at kernel/qapplication.cpp:2883
#17 0x00007f43a381151c in TQEventLoop::activateTimers (this=this@entry=0x555ffca9e220) at kernel/qeventloop_unix_glib.cpp:694
#18 0x00007f43a37fe61d in TQEventLoop::gsourceDispatch (this=this@entry=0x555ffca9e220, gs=gs@entry=0x555ffcaa1d60)
    at kernel/qeventloop_x11_glib.cpp:595
#19 0x00007f43a37fe713 in qt_gsource_dispatch (source=0x555ffcaa1d60, callback=<optimized out>, user_data=<optimized out>)
 #20 0x00007f43a2498e6b in g_main_context_dispatch () from /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
#21 0x00007f43a2499118 in ?? () from /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
#22 0x00007f43a24991cf in g_main_context_iteration () from /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
#23 0x00007f43a37fd9cb in TQEventLoop::processEvents (this=0x555ffca9e220, flags=<optimized out>)
    at kernel/qeventloop_x11_glib.cpp:279
#24 0x00007f43a382c6f1 in TQEventLoop::enterLoop (this=0x555ffca9e220) at kernel/qeventloop.cpp:227
#25 0x00007f43a382c689 in TQEventLoop::exec (this=0x555ffca9e220) at kernel/qeventloop.cpp:174
#26 0x00007f43a11edfbd in kdemain (argc=1, argv=0x555ffc9f3a00) at ../konsole/konsole/main.cpp:590
#27 0x00007f43a4847125 in tdeinitmain (argc=1, argv=0x555ffc9f3a00) at konsole/konsole/konsole_tdeinit_module.cpp:3
#28 0x0000555ffaf16fec in launch (argc=1, _name=<optimized out>, args=<optimized out>, cwd=<optimized out>, 
    envc=<optimized out>, envs=<optimized out>, reset_env=false, tty=0x0, avoid_loops=false, 
    startup_id_str=0x555ffc9f974b "sysel;1639693050;368187;1906_TIME18055034") at ../tdeinit/tdeinit.cpp:674
#29 0x0000555ffaf18154 in handle_launcher_request (sock=<optimized out>) at ../tdeinit/tdeinit.cpp:1234
#30 0x0000555ffaf188fe in handle_requests (waitForPid=waitForPid@entry=0) at ../tdeinit/tdeinit.cpp:1440
#31 0x0000555ffaf14ead in main (argc=argc@entry=5, argv=argv@entry=0x7ffd29545fb8, envp=0x7ffd29545d50)
    at ../tdeinit/tdeinit.cpp:1906
#32 0x00007f43a30ebd0a in __libc_start_main (main=0x555ffaf14080 <main(int, char**, char**)>, argc=5, argv=0x7ffd29545fb8, 
    init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffd29545fa8) at ../csu/libc-start.c:308
#33 0x0000555ffaf158aa in _start ()

## Basic information - TDE version: R14.0.11 - Distribution: Debian GNU/Linux 11 (bullseye) - Hardware: amd64 (x86_64) ## Description At least konsole and kwrite freeze when trying to display characters like \u0acb (in UTF-8 \xe0\xab\x8b). It does not apply to all indic characters, for example \u0ab5 works well. Try (in bash in trinity konsole): ``` echo -e "\u0acb" ``` Change of the font did not change the situation. It is not that unprobable to get to that situation even for people not using that language - for example I was investigating, why TDE panel launcher for "pavucontrol" stopped displaying icon after an upgrade from buster to bullseye. During that, I opened the file pavucontrol.desktop, and konsole freezed. The problem was with the line 15 of that file containg Name\[gu\] (The same freeze did not happen in KDE konsole 20.12.3 using Qt 5.15.2). To me it seems suspocious around the line kernel/qfontengine_x11.cpp:2626 in bool TQOpenType::positionAndAdd ``` 2626: if (item->num_glyphs < (int)otl_buffer->in_length) { 2627: item->num_glyphs = otl_buffer->in_length; 2628: return FALSE; 2629: } ``` otl_buffer_in_length is always 3 in this case item->num_glyphs is rewritten to 2 in TQFontEngineXft::stringToCMap qfontengine_x11.cpp:1702 ``` 1700:#endif // QT_XFT2 1701: 1702: *nglyphs = len; 1703: return NoError; 1704:} ``` in static bool indic_shape in qscriptengine_x11.cpp:1848 ``` 1846: if (!indic_shape_syllable(openType, &syllable, invalid)) { 1847: IDEBUG("syllable shaping failed, syllable requests %d glyphs", syllable.num_glyphs); 1848: item->num_glyphs += syllable.num_glyphs; 1849: return FALSE; 1850: } ``` is infinitely increasing item.num_glyphs by 2 (without any effect on the condition in bool TQOpenType::positionAndAdd, because of bool TQOpenType::positionAndAdd reassignment in bool TQOpenType::positionAndAdd) ## Steps to reproduce 1. Open trinity konsole with bash 2. Check encoding is UTF-8 and you have any UTF-8 locale 3. Do ``` echo -e "\u0acb" ``` It freezes. ## Backtrace of suspicious line ``` #0 TQOpenType::positionAndAdd (this=this@entry=0x555ffcd2d240, item=item@entry=0x7ffd295443b0, doLogClusters=doLogClusters@entry=false) at kernel/qfontengine_x11.cpp:2626 #1 0x00007f43a38d8a43 in indic_shape_syllable (openType=openType@entry=0x555ffcd2d240, item=item@entry=0x7ffd295443b0, invalid=<optimized out>) at kernel/qscriptengine_x11.cpp:1697 #2 0x00007f43a38d9b0c in indic_shape (item=0x7ffd29544460) at kernel/qscriptengine_x11.cpp:1846 #3 0x00007f43a38dc121 in TQTextEngine::shape (this=0x7ffd29544590, item=0) at kernel/qtextengine_unix.cpp:108 #4 0x00007f43a38daa1d in TQTextLayout::currentItem (this=this@entry=0x7ffd29544558) at kernel/qtextlayout.cpp:324 #5 0x00007f43a37e9ac0 in TQPainter::drawText (this=0x7ffd29544820, x=<optimized out>, y=<optimized out>, str=..., pos=<optimized out>, pos@entry=0, len=<optimized out>, dir=<optimized out>) at kernel/qpainter_x11.cpp:3104 #6 0x00007f43a37e9df1 in TQPainter::drawText (this=<optimized out>, x=<optimized out>, y=<optimized out>, str=..., len=<optimized out>, dir=<optimized out>) at kernel/qpainter_x11.cpp:3025 #7 0x00007f43a1225333 in TEWidget::drawAttrStr (this=0x555ffca3acd0, paint=..., rect=..., str=..., attr=0x555ffcd3fde4, pm=false, clear=true) at ../konsole/konsole/TEWidget.cpp:750 #8 0x00007f43a12261a3 in TEWidget::setImage (this=0x555ffca3acd0, newimg=0x555ffcd3f9a0, lines=38, columns=91) at ../konsole/konsole/TEWidget.cpp:924 #9 0x00007f43a123e561 in TEmulation::showBulk (this=0x555ffcc97e50) at ../konsole/konsole/TEmulation.cpp:540 #10 0x00007f43a123c8fc in TEmulation::tqt_invoke (this=0x555ffcc97e50, _id=19, _o=0x7ffd29544c40) at konsole/konsole/TEmulation.moc:347 #11 0x00007f43a12314a2 in TEmuVt102::tqt_invoke (this=0x555ffcc97e50, _id=19, _o=0x7ffd29544c40) at konsole/konsole/TEmuVt102.moc:131 #12 0x00007f43a386a0a0 in TQObject::activate_signal (this=0x555ffcc97ee0, clist=<optimized out>, o=o@entry=0x7ffd29544c40) at kernel/qobject.cpp:2813 #13 0x00007f43a386a18e in TQObject::activate_signal (this=this@entry=0x555ffcc97ee0, signal=<optimized out>) at kernel/qobject.cpp:2747 #14 0x00007f43a3b10c05 in TQTimer::timeout (this=this@entry=0x555ffcc97ee0) at .moc/release-shared-mt/moc_ntqtimer.cpp:91 #15 0x00007f43a3888e5c in TQTimer::event (this=0x555ffcc97ee0, e=<optimized out>) at kernel/qtimer.cpp:222 #16 0x00007f43a381a558 in TQApplication::internalNotify (this=<optimized out>, receiver=0x555ffcc97ee0, e=0x7ffd29544cf0) at kernel/qapplication.cpp:2883 #17 0x00007f43a381151c in TQEventLoop::activateTimers (this=this@entry=0x555ffca9e220) at kernel/qeventloop_unix_glib.cpp:694 #18 0x00007f43a37fe61d in TQEventLoop::gsourceDispatch (this=this@entry=0x555ffca9e220, gs=gs@entry=0x555ffcaa1d60) at kernel/qeventloop_x11_glib.cpp:595 #19 0x00007f43a37fe713 in qt_gsource_dispatch (source=0x555ffcaa1d60, callback=<optimized out>, user_data=<optimized out>) #20 0x00007f43a2498e6b in g_main_context_dispatch () from /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0 #21 0x00007f43a2499118 in ?? () from /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0 #22 0x00007f43a24991cf in g_main_context_iteration () from /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0 #23 0x00007f43a37fd9cb in TQEventLoop::processEvents (this=0x555ffca9e220, flags=<optimized out>) at kernel/qeventloop_x11_glib.cpp:279 #24 0x00007f43a382c6f1 in TQEventLoop::enterLoop (this=0x555ffca9e220) at kernel/qeventloop.cpp:227 #25 0x00007f43a382c689 in TQEventLoop::exec (this=0x555ffca9e220) at kernel/qeventloop.cpp:174 #26 0x00007f43a11edfbd in kdemain (argc=1, argv=0x555ffc9f3a00) at ../konsole/konsole/main.cpp:590 #27 0x00007f43a4847125 in tdeinitmain (argc=1, argv=0x555ffc9f3a00) at konsole/konsole/konsole_tdeinit_module.cpp:3 #28 0x0000555ffaf16fec in launch (argc=1, _name=<optimized out>, args=<optimized out>, cwd=<optimized out>, envc=<optimized out>, envs=<optimized out>, reset_env=false, tty=0x0, avoid_loops=false, startup_id_str=0x555ffc9f974b "sysel;1639693050;368187;1906_TIME18055034") at ../tdeinit/tdeinit.cpp:674 #29 0x0000555ffaf18154 in handle_launcher_request (sock=<optimized out>) at ../tdeinit/tdeinit.cpp:1234 #30 0x0000555ffaf188fe in handle_requests (waitForPid=waitForPid@entry=0) at ../tdeinit/tdeinit.cpp:1440 #31 0x0000555ffaf14ead in main (argc=argc@entry=5, argv=argv@entry=0x7ffd29545fb8, envp=0x7ffd29545d50) at ../tdeinit/tdeinit.cpp:1906 #32 0x00007f43a30ebd0a in __libc_start_main (main=0x555ffaf14080 <main(int, char**, char**)>, argc=5, argv=0x7ffd29545fb8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffd29545fa8) at ../csu/libc-start.c:308 #33 0x0000555ffaf158aa in _start () ```

Input:

I use TDE 14.1 on buster
With Dejavu Sans Mono font konsole CPU usage went up to 100%
I changed the font to Noto Mono and it printed fine

For me (on bullseye) it freezes even after switching to Noto Mono. However according to the code, it is probably font dependent. Maybe there is some relevant change to the font between debian releases.

Uhm... Debian bookworm, TDE 14.1-dev, locale utf-8, encoding utf-8. I can't reproduce the problem, I tried 5 or 6 different fonts. All works fine.

Quick try:
Upgraded konsole-trinity and its dependencies to 14.1 (testing repository):

Install: libtqt3-mt-data:amd64 (4:14.1.0~s158-0debian11.0.0+13, automatic)
Upgrade: libtqt3-mt:amd64 (4:14.0.11-0debian11.0.0+0, 4:14.1.0~s158-0debian11.0.0+13), libtqtinterface:amd64 (4:14.0.11-0debian11.0.0+0, 4:14.1.0~s86-0debian11.0.0+12), libr0:amd64 (4:14.0.11-0debian11.0.0+0, 4:14.1.0~s74-0debian11.0.0+8), konsole-trinity:amd64 (4:14.0.11-0debian11.0.0+0, 4:14.1.0~s875-0debian11.0.0+15), tdelibs14-trinity:amd64 (4:14.0.11-0debian11.0.0+0, 4:14.1.0~s588-0debian11.0.0+15), libdbus-1-tqt:amd64 (4:14.0.11-0debian11.0.0+0, 4:14.1.0~s104-0debian11.0.0+19)

restarted TDE,
still freezes

I forgot to write, that during the freeze, it was looping in
TQTextEngine::shape in kernel/qtextengine_unix.cpp:98-110

 98:        while (1) {
 99://         tqDebug("    . num_glyphs=%d, used=%d, item.num_glyphs=%d", num_glyphs, used, shaper_item.num_glyphs);
100:            ensureSpace(shaper_item.num_glyphs);
101:            shaper_item.num_glyphs = num_glyphs - used;
102://          tqDebug("    .. num_glyphs=%d, used=%d, item.num_glyphs=%d", num_glyphs, used, shaper_item.num_glyphs);
103:            shaper_item.glyphs = glyphs(&si);
104:            shaper_item.advances = advances(&si);
105:            shaper_item.offsets = offsets(&si);
106:            shaper_item.attributes = glyphAttributes(&si);
107:            shaper_item.log_clusters = logClusters(&si);
108:            if (scriptEngines[shaper_item.script].shape(&shaper_item))
109:                break;
110:        }

where shape was returning false because of the line in the backtrace above where item->num_glyphs was always 2 and (int)otl_buffer->in_length was always 3 as described in the original post. However (int)otl_buffer->in_length in qtextengine was increasing in each loop.

@vasekj thanks for the detailed explanation, I had a look through the backtrace and the code.
We should try to understand why item->num_glyphs is less than (int)otl_buffer->in_length in order to come up with a proper fix. Are you able to dig deeper into this?
If we can't find anything, we may need to bail out from the while loop after too many passes, but we need to understand the effect of it correctly.
I occasionally have konsole freeze when I am in mc and I press some "wrong" key, although I can't yet identify why and what triggers it. It may be the same root cause.