Bug 159000 - WRITER: Insert Special Characters menu window only shows a part of available characters in certain fonts (But accessible through Unicode)
Summary: WRITER: Insert Special Characters menu window only shows a part of available ...
Status: UNCONFIRMED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
7.6.4.1 release
Hardware: All Windows (All)
: medium minor
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-01-03 16:47 UTC by Kati L.
Modified: 2024-01-03 21:24 UTC (History)
0 users

See Also:
Crash report or crash signature:


Attachments
Screenshot of some example characters (155.98 KB, image/png)
2024-01-03 21:00 UTC, Kati L.
Details
The window showing the missing combining diacritics' place (73.24 KB, image/png)
2024-01-03 21:18 UTC, Kati L.
Details
Screenshot showing a section of the special characters that are shown (71.38 KB, image/png)
2024-01-03 21:24 UTC, Kati L.
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Kati L. 2024-01-03 16:47:39 UTC
Description:
Hello.

This problem is partly there because of my font of choice / preference, which is Cavolini in all of its four supported forms (regular / italic / bold / italic bold). At least in Arial and Times New Roman, the needed characters are available in the Insert Special Characters menu window. However, even if some sources claim that Cavolini wouldn't support these characters, Unicode does, and provenly, they do exist within the font data. For some reason, OL Writer doesn't recognize or show them, even if they are accessible through the Unicode.

I would need to insert in my otherwise English text some foreign words, for instance transliterated Mandarin words with the corresponding characters denoting tones. The same goes for Turkish and the Turkic (family of) languages, for instance.

It doesn't work in this font by inserting the (combining) diacritic through the same window and typing the alphabetic character normally - or inserting the alphabetic character also through the window. This would be the standard(keyboard) typing order, if those "dead keys" would exist in the configuration of your own keyboard. I tried the reverse order as well, inserting the alphabetic character first and then the (combining) diacritic. The same end result: an orphaned diacritic either preceding or superceding a Latin character.

As it turns out, the diacritics given in the Insert Special Characters menu aren't actually _combining_ diacritics. 

For the best convenience, I'll demonstrate using a character that doesn't appear in the most common language configurations or selections: the Latin Small Letter O with a double acute accent. This is to avoid any system settings that force a pre-formed combination character automatically after the specific typing sequence occurs.


[This actually used to be an issue in the older versions of LO Writer (the one I had in use, about three years ago), as well as the reverted character appearing in the default application font, instead of the document default font or the selected font inside that text block. (As an anecdote, some of these strange reversions still appear in certain older drafts of my texts, retained in the PDF conversions made for the sake of easier proofreading, as a funny throwback.)]

Example: the double acute accent found in the menu is [U+02dd]. The correct, combining acute accent ought to be [U+030b]. The actual combining diacritics can't be found anywhere in the available selection in the Insert Special Characters menu window - but, when typing them in the text in their Unicode form followed by the combination key [alt+x], does bring out the correct combined characters (the vowel with the diacritic correctly above it) - in the used font, as a cherry on top. Therefore, they have to exist in the data of the font files, so who don't they show in the menu window?

AND the alternative pre-formed combination characters can also be accessed through typing in its Unicode "tag" (pardon me for the makeshift term, I couldn't find the actual one anywhere, even on Wikipedia, and "Unicode code" was too tautological for my own comfort) with [alt+x], even if they either aren't anywhere visible or listed in the access menu. 

I double-checked the actual identification through the [alt+x] command, and the characters typed in through using the Latin character superceded by the Unicode tag that form into the combination character aren't forced into reverting to their special preformed combination characters, even if those exist in the data and are accessible through typing the commands. 

For users who often need combination characters or unusual combined characters, not finding them in the Insert Special Character menu window (even if they evidently exist in the currently used font!), or not being able to use the combining diacritics through the same menu, is inconvenient. So is the constant Unicode typing in the middle of a text.

There neither are adequate instructions for this method offered anywhere that would be easily found or accessed. Only an hour of googling finally provided me the solution, which is to write the Latin character _before_ inserting the Unicode to the text, _without_ a [space], which adds in the inconvenience since in the default settings, any [ctrl+v] automatically inserts a [space] after the pasted piece. It requires an additional [backspace] to be used, before the [alt+x] can be employed. This also is a bit counter-intuitive, already because the usual *typing order on a keyboard* is the total *reverse* of this, the diacritic preceding the alphabetic character. 

I've fumbled with this countless times, until I now finally found the correct answer through endless googling. And, using the Unicode for both of the parts in the combination doesn't work, at least without something used to connect the two Unicode tags together, and I couldn't find the specific command/code for this, not being a coder. The Latin character needs to be in the typed form (use [alt+x] if necessary, when applying a specific base character through copy/pasting from a Unicode table, as is sometimes convenient), and the combining diacritic in the Unicode tag form. I feel like a f:ng private investigator, now. :D

Creating keyboard shortcuts is said to be possible, so I'm likely going to go through that trouble next, but I'd be grateful if you could consider checking the functionality of this feature, as it's a pretty essential component of word processing. 

Thanks!

Steps to Reproduce:
First, to produce the incorrect forms:

* Make certain of that the font is set as Cavolini. It doesn't matter if it's the default document font, the default LibreOffice font, or just the font you've selected either for the text, or only from the Insert Special Character Menu, once you open it. I've tested all of these options, along the last few months, if not today.

Test #1
1. Open the Insert Special Character menu window
2. Search for the needed diacritic 
3. Browse for it, because search produces no results
4. Select the diacritic Double Acute Accent (it appears to be [U+02dd]) and hope it's the correct one
5. Insert the diacritic to the text by using the respective button in the window, and close the window if it doesn't do that by itself
6. Type in the Latin small character (o); it's [U+006f] if you need a specific character for your convenience
7. Nothing happens. You should see ˝ and o separately in the text. 

(Test #2)
8. Test the reverse order (phase 6 first, then 1 through 5)
9. Nothing still happens. You should see o and ˝ separately in the text, this time in this order.

Test #3
Now, test the other technique.
1. Insert the Latin Small Letter O by simply typing it from the keyboard. It's [U+006f], in case you're using Unicode definitions and need to be certain of the right configuration / Unicode section.
2. Without leaving a [space], type down the Unicode tag [U+030b], which is the correct Unicode for a Combining Double Acute Accent.
3. Use the combination key [alt+x]
4. You get a combined character in the text. 

Test #4
You can also use the pre-formed combination key:
1. Type into the text the Unicode tag [U+0151], which is the correct Unicode for the character ő (Latin Small Letter O with double acute accent)
2. Press combination key [alt+x]
3. You have the pre-formed combination character in the text.

This can all be repeated while using the Times New Roman or Arial fonts, and the Insert Special Character menu window will show you a much larger variation of choices for the characters. The tested characters will still be functional also in Cavolini, despite not showing in the menu.


Actual Results:
The options Test #3 and Test #4 should produce the necessary combination character in the used font (Cavolini). 

Expected Results:
The Insert Special Character menu window ought to also offer the choices for the _combining_ diacritics, and the rest of the combination characters - especially since not all of the characters exist in the rather limited default menu. I assume that the software somehow thinks that these characters aren't supported, even if they clearly are. The possible solution in that case would be to allow the software to recognize which options are available, instead of limiting the choices to very few possibilities which don't cover the actual needs.

Cavolini is a neat and cute, yet very legible handwriting style font that is both airy and easy to read. It doesn't contain easily confused character forms aside from the Latin Capital Letter I and the Latin Small Letter L, or the numeral 1 in some cases, which makes it ideal for writing more accessible texts, or longer pieces with a good spacing. It's a versatile font which is only made more usable by its ability to appear legible and tidy even in its variant forms and while using stylistic typographical tricks such as shadowing, widening, spacing and altering the character height. Therefore, it's an important font and deserves fully supported basic features in the software.


Reproducible: Always


User Profile Reset: No

Additional Info:
Version: 7.6.4.1 (X86_64) / LibreOffice Community
Build ID: e19e193f88cd6c0525a17fb7a176ed8e6a3e2aa1
CPU threads: 4; OS: Windows 10.0 Build 22621; UI render: Skia/Raster; VCL: win
Locale: fi-FI (fi_FI); UI: fi-FI
Calc: threaded
Comment 1 Mike Kaganski 2024-01-03 17:28:28 UTC
(In reply to Kati L. from comment #0)
> Example: the double acute accent found in the menu is [U+02dd]. The correct,
> combining acute accent ought to be [U+030b]. The actual combining diacritics
> can't be found anywhere in the available selection in the Insert Special
> Characters menu window - but, when typing them in the text in their Unicode
> form followed by the combination key [alt+x], does bring out the correct
> combined characters (the vowel with the diacritic correctly above it) - in
> the used font, as a cherry on top. Therefore, they have to exist in the data
> of the font files, so who don't they show in the menu window?

This is misconception. Your text has the font set to your choice. The controls display it. But specific glyphs that you request might come from a different - substitution - font.
Comment 2 Kati L. 2024-01-03 21:00:53 UTC
Created attachment 191742 [details]
Screenshot of some example characters

Here's an example of the characters that are found, or aren't found in the font.
Comment 3 Kati L. 2024-01-03 21:11:27 UTC
(In reply to Mike Kaganski from comment #1)
> (In reply to Kati L. from comment #0)
> > Example: the double acute accent found in the menu is [U+02dd]. The correct,
> > combining acute accent ought to be [U+030b]. The actual combining diacritics
> > can't be found anywhere in the available selection in the Insert Special
> > Characters menu window - but, when typing them in the text in their Unicode
> > form followed by the combination key [alt+x], does bring out the correct
> > combined characters (the vowel with the diacritic correctly above it) - in
> > the used font, as a cherry on top. Therefore, they have to exist in the data
> > of the font files, so who don't they show in the menu window?
> 
> This is misconception. Your text has the font set to your choice. The
> controls display it. But specific glyphs that you request might come from a
> different - substitution - font.

It appears that I wasn't still being meticulous enough, even if I thought that I'd been way too nitpicky. 

I'm aware of this fact. I just forgot to mention that, too, in my list of several other options.

I attached a screencapture to demonstrate that certain combinations indeed do exist within the font, whereas other characters are substituted with the equivalents from another font. 

I believe I already described this feature here:
"
[This actually used to be an issue in the older versions of LO Writer (the one I had in use, about three years ago), as well as the reverted character appearing in the default application font, instead of the document default font or the selected font inside that text block. (As an anecdote, some of these strange reversions still appear in certain older drafts of my texts, retained in the PDF conversions made for the sake of easier proofreading, as a funny throwback.)]"

The forced reversion / substitution no longer is an issue where the said characters actually exist within the font. Even in these cases (in the older versions of this software), they did, and the problem seemed to arise from having converted a document to .odt from .doc, or opening a document originally written in MSWord in LOWrite and saving it as .odt. Somehow, it didn't recognize the characters, even if the font remained the same in both formats and original softwares where the documents were written. But, like said, this no longer is an issue as long as the characters actually do exist in the used font.

So, in the case of my complaints, the fonts do appear in the intended font, once I use the Unicode entering technique. They don't seem (falsely) to exist, within the entire set of characters in the Insert Special Character menu window. Will attach another screenshot of the said window. Sorry it doesn't appear as a picture in this thread, I reached the extent of my data skills at this time of the night, but was so frustrated by the misconception of my assumed misconception that I decided to return here and clarify the matter.

Sorry. I'm an incurable geek, and I get needlessly annoyed when something that should be relatively effortless suddenly requires extensive work just in order to type in a single character.
Comment 4 Kati L. 2024-01-03 21:18:28 UTC
Created attachment 191743 [details]
The window showing the missing combining diacritics' place

The full range of diacritics appear here, but they all are of the non-combining kind that stands alone, separately
Comment 5 Kati L. 2024-01-03 21:24:53 UTC
Created attachment 191744 [details]
Screenshot showing a section of the special characters that are shown

Some of the special characters do appear within the menu window, and did already at the point when the forced substituting to another font was still in effect. Most of the characters I need are actually here (including the Latin Small Letter A with a breve, which was the biggest issue for a long while), but I would still be happier if the full range of the available combining diacritics were available.

As another point, the same combinations found here as pre-formed combined characters do also work in the intended way as separate Unicode typing applications, for instance in the case of the double acute accent over small Latin o. I've tested the both methods (inserting from the menu window and inserting as typed Unicode[alt+x]), and they work. The separate combining diacritic and the letter aren't replaced by the pre-formed combination character here, at least presumedly, since they can be turned back and forth to Unicode and characters through the same [alt+x] combo key?