Discussion:
[dev] noice and NetBSD curses
Cág
2018-07-31 21:35:30 UTC
Permalink
Hey,

The thread is here[0], and I guess this is more of curses problem, but
since I first noticed it with noice, here it is.

The question: what do rover and noice do differently, that the former
displays Unicode filenames right, and the latter only under the LC_ALL=C
local.

P.S. 2f30 guise please make your site bright as it was. My eyes broke,
am deaf now thenks

[0]: http://mail-index.netbsd.org/netbsd-users/2018/07/31/msg021180.html

--
caóc
Hiltjo Posthuma
2018-08-01 07:55:16 UTC
Permalink
Post by Cág
Hey,
The thread is here[0], and I guess this is more of curses problem, but
since I first noticed it with noice, here it is.
The question: what do rover and noice do differently, that the former
displays Unicode filenames right, and the latter only under the LC_ALL=C
local.
Maybe setlocale(LC_CTYPE, ""), not sure if that is correct. You could
technically mix different locales in filenames or use (almost) any byte
sequence though.
Post by Cág
P.S. 2f30 guise please make your site bright as it was. My eyes broke,
am deaf now thenks
No... no don't do it.
Post by Cág
[0]: http://mail-index.netbsd.org/netbsd-users/2018/07/31/msg021180.html
--
caóc
--
Kind regards,
Hiltjo
Silvan Jegen
2018-08-01 09:57:38 UTC
Permalink
Post by Hiltjo Posthuma
Post by Cág
Hey,
The thread is here[0], and I guess this is more of curses problem, but
since I first noticed it with noice, here it is.
The question: what do rover and noice do differently, that the former
displays Unicode filenames right, and the latter only under the LC_ALL=C
local.
Maybe setlocale(LC_CTYPE, ""), not sure if that is correct. You could
technically mix different locales in filenames or use (almost) any byte
sequence though.
According to the manpage, setlocale(LC_CTYPE, "") will read the locale
from the environment variables. I wonder if LC_ALL=C actually implies
UTF-8 encoding or not...

What does running 'locale' print on both machines?


Cheers,

Silvan
Hiltjo Posthuma
2018-08-01 15:29:14 UTC
Permalink
Post by Silvan Jegen
Post by Hiltjo Posthuma
Post by Cág
Hey,
The thread is here[0], and I guess this is more of curses problem, but
since I first noticed it with noice, here it is.
The question: what do rover and noice do differently, that the former
displays Unicode filenames right, and the latter only under the LC_ALL=C
local.
Maybe setlocale(LC_CTYPE, ""), not sure if that is correct. You could
technically mix different locales in filenames or use (almost) any byte
sequence though.
According to the manpage, setlocale(LC_CTYPE, "") will read the locale
from the environment variables. I wonder if LC_ALL=C actually implies
UTF-8 encoding or not...
It does not.
Post by Silvan Jegen
What does running 'locale' print on both machines?
Cheers,
Silvan
--
Kind regards,
Hiltjo
Silvan Jegen
2018-08-01 17:26:35 UTC
Permalink
Post by Hiltjo Posthuma
Post by Silvan Jegen
Post by Hiltjo Posthuma
Post by Cág
Hey,
The thread is here[0], and I guess this is more of curses problem, but
since I first noticed it with noice, here it is.
The question: what do rover and noice do differently, that the former
displays Unicode filenames right, and the latter only under the LC_ALL=C
local.
Maybe setlocale(LC_CTYPE, ""), not sure if that is correct. You could
technically mix different locales in filenames or use (almost) any byte
sequence though.
According to the manpage, setlocale(LC_CTYPE, "") will read the locale
from the environment variables. I wonder if LC_ALL=C actually implies
UTF-8 encoding or not...
It does not.
Yeah, that was my guess too since I have seen something like "C.UTF-8"
before.

Then I am wondering why noice shows Unicode names with LC_ALL=C though.


Cheers,

Silvan
Post by Hiltjo Posthuma
Post by Silvan Jegen
What does running 'locale' print on both machines?
Cheers,
Silvan
--
Kind regards,
Hiltjo
Cág
2018-08-01 21:55:41 UTC
Permalink
Post by Silvan Jegen
What does running 'locale' print on both machines?
Mine's are all en_US.UTF-8, except for LC_COLLATE. I don't set anything
other than LANG in profile.

--
caóc
Silvan Jegen
2018-08-02 08:41:49 UTC
Permalink
Post by Cág
Post by Silvan Jegen
What does running 'locale' print on both machines?
Mine's are all en_US.UTF-8, except for LC_COLLATE. I don't set anything
other than LANG in profile.
I have the same locale set up (LANG=en_US.UTF-8) and on my (Linux)
machine, both rover and noice (compiled from tip) show the "früh.mp3"
filename correctly without me having to set LC_ALL=C.

What you could try is to check what setlocale() returns in both cases.
There may be an error when setting up the locale in the noice case.


Cheers,

Silvan
Cág
2018-08-05 14:56:23 UTC
Permalink
Post by Silvan Jegen
I have the same locale set up (LANG=en_US.UTF-8) and on my (Linux)
machine, both rover and noice (compiled from tip) show the "früh.mp3"
filename correctly without me having to set LC_ALL=C.
Keep in mind, this is NetBSD with its own curses library. It was
suggested that it's a bug somehow covered by ncurses.
Post by Silvan Jegen
What you could try is to check what setlocale() returns in both cases.
There may be an error when setting up the locale in the noice case.
If I simply printf setlocale() nothing strange appears: C for COLLATE,
and en_US.UTF-8 for everything else. The same is inside noice when
spawning a shell.

--
caóc
Silvan Jegen
2018-08-07 13:58:57 UTC
Permalink
Post by Cág
Post by Silvan Jegen
I have the same locale set up (LANG=en_US.UTF-8) and on my (Linux)
machine, both rover and noice (compiled from tip) show the "früh.mp3"
filename correctly without me having to set LC_ALL=C.
Keep in mind, this is NetBSD with its own curses library. It was
suggested that it's a bug somehow covered by ncurses.
I thought that maybe one of the programs is linked against the
non-"...w" variant of the curses library (i. e. "libncurses" instead
of "libncursesw") but it turns out that NetBSD's curses library does
not have this separation at all. Interesting stuff.
Post by Cág
Post by Silvan Jegen
What you could try is to check what setlocale() returns in both cases.
There may be an error when setting up the locale in the noice case.
If I simply printf setlocale() nothing strange appears: C for COLLATE,
and en_US.UTF-8 for everything else. The same is inside noice when
spawning a shell.
So the locale doesn't seem to be an issue.

Not sure how to debug this further. Especially not without a working
NetBSD installation (and I am lazy...).


Cheers,

Silvan
Cág
2018-08-07 18:18:14 UTC
Permalink
Post by Silvan Jegen
I thought that maybe one of the programs is linked against the
non-"...w" variant of the curses library (i. e. "libncurses" instead
of "libncursesw") but it turns out that NetBSD's curses library does
not have this separation at all. Interesting stuff.
It can handle wide characters --
https://github.com/NetBSD/src/blob/trunk/lib/libcurses/cchar.c
Post by Silvan Jegen
Not sure how to debug this further. Especially not without a working
NetBSD installation (and I am lazy...).
You can grab the code here https://github.com/sabotage-linux/netbsd-curses
and try to link noice against it.

--
caóc
Cág
2018-08-07 18:34:03 UTC
Permalink
Post by Silvan Jegen
I thought that maybe one of the programs is linked against the
non-"...w" variant of the curses library (i. e. "libncurses" instead
of "libncursesw") but it turns out that NetBSD's curses library does
not have this separation at all. Interesting stuff.
Some programs, like already mentioned vifm, have ugly installers that
look specifically for ncurses and ncursesw, and pkgsrc has wrappers
USE_CURSES and FAKE_NCURSES for them to use the system's curses. So,
yeah, you don't need additional switches for wide characters.

--
caóc
Cág
2018-08-01 21:49:21 UTC
Permalink
Post by Hiltjo Posthuma
Maybe setlocale(LC_CTYPE, ""), not sure if that is correct. You could
technically mix different locales in filenames or use (almost) any byte
sequence though.
rover, which I mentioned in the OP, has it too:
https://github.com/lecram/rover/blob/master/rover.c#L405


--
caóc
Cág
2018-08-10 21:16:04 UTC
Permalink
Hi,

The problem was solved here:
http://mail-index.netbsd.org/netbsd-users/2018/08/10/msg021219.html

I'm attaching Leonardo's patch, please review.

--
caóc

Loading...