Hi André,
we are seeing a build error on your machine when locales in uClibc-ng are enabled: http://autobuild.buildroot.net/results/d5b/d5ba81eea9223569ba5b363551c4a2f70...
Can you show us locale -a from your machine?
For a working tolower(), the uClibc-ng code needs to use setlocale on the host.
But it seems only these are tried to use: en_US en_US en_GB
best regards Waldemar
Hello,
On Thu, 3 Aug 2017 07:48:28 +0200, Waldemar Brodkorb wrote:
we are seeing a build error on your machine when locales in uClibc-ng are enabled: http://autobuild.buildroot.net/results/d5b/d5ba81eea9223569ba5b363551c4a2f70...
Can you show us locale -a from your machine?
For a working tolower(), the uClibc-ng code needs to use setlocale on the host.
But it seems only these are tried to use: en_US en_US en_GB
It is worth mentioning that Buildroot is already checking that at least one UTF-8 locale is available on the host machine, for the needs of uClibc locale support. From support/dependencies/dependencies.sh:
if grep ^BR2_NEEDS_HOST_UTF8_LOCALE=y $BR2_CONFIG > /dev/null; then if ! which locale > /dev/null ; then echo echo "You need locale support on your build machine to build a toolchain supporting locales" exit 1 ; fi if ! locale -a | grep -q -i utf8$ ; then echo echo "You need at least one UTF8 locale to build a toolchain supporting locales" exit 1 ; fi fi
And the only package selecting BR2_NEEDS_HOST_UTF8_LOCALE is:
package/uclibc/Config.in: select BR2_NEEDS_HOST_UTF8_LOCALE
Is this check insufficient now ?
Best regards,
Thomas
Hi Thomas,
Is this check insufficient now ?
I think it's sufficient.
Please post output of the following commands: $ locale $ locale -a $ cat /etc/os-release
It would be very helpful to reproduce build failure.
On Thu, Aug 3, 2017 at 10:23 AM, Thomas Petazzoni < thomas.petazzoni@free-electrons.com> wrote:
Hello,
On Thu, 3 Aug 2017 07:48:28 +0200, Waldemar Brodkorb wrote:
we are seeing a build error on your machine when locales in uClibc-ng are enabled: http://autobuild.buildroot.net/results/d5b/
d5ba81eea9223569ba5b363551c4a2f7044ff8ec/
Can you show us locale -a from your machine?
For a working tolower(), the uClibc-ng code needs to use setlocale on the host.
But it seems only these are tried to use: en_US en_US en_GB
It is worth mentioning that Buildroot is already checking that at least one UTF-8 locale is available on the host machine, for the needs of uClibc locale support. From support/dependencies/dependencies.sh:
if grep ^BR2_NEEDS_HOST_UTF8_LOCALE=y $BR2_CONFIG > /dev/null; then if ! which locale > /dev/null ; then echo echo "You need locale support on your build machine to build a toolchain supporting locales" exit 1 ; fi if ! locale -a | grep -q -i utf8$ ; then echo echo "You need at least one UTF8 locale to build a toolchain supporting locales" exit 1 ; fi fi
And the only package selecting BR2_NEEDS_HOST_UTF8_LOCALE is:
package/uclibc/Config.in: select BR2_NEEDS_HOST_UTF8_LOCALE
Is this check insufficient now ?
Best regards,
Thomas
Thomas Petazzoni, CTO, Free Electrons Embedded Linux and Kernel engineering http://free-electrons.com
I resent the mail. Previous was rejected by mailing list. Sorry for duplication.
---------- Forwarded message ---------- From: Eugene Yudin e.yudin@ndmsystems.com Date: Thu, Aug 3, 2017 at 11:11 AM Subject: Re: [Buildroot] buildroot uClibc-ng build failure To: Thomas Petazzoni thomas.petazzoni@free-electrons.com Cc: Waldemar Brodkorb wbx@openadk.org, André Hentschel nerv@dawncrow.de, devel@uclibc-ng.org, buildroot@buildroot.org
Hi Thomas,
Is this check insufficient now ?
I think it's sufficient.
Please post output of the following commands: $ locale $ locale -a $ cat /etc/os-release
It would be very helpful to reproduce build failure.
On Thu, Aug 3, 2017 at 10:23 AM, Thomas Petazzoni <thomas.petazzoni@free- electrons.com> wrote:
Hello,
On Thu, 3 Aug 2017 07:48:28 +0200, Waldemar Brodkorb wrote:
we are seeing a build error on your machine when locales in uClibc-ng are enabled: http://autobuild.buildroot.net/results/d5b/d5ba81eea9223569b
a5b363551c4a2f7044ff8ec/
Can you show us locale -a from your machine?
For a working tolower(), the uClibc-ng code needs to use setlocale on the host.
But it seems only these are tried to use: en_US en_US en_GB
It is worth mentioning that Buildroot is already checking that at least one UTF-8 locale is available on the host machine, for the needs of uClibc locale support. From support/dependencies/dependencies.sh:
if grep ^BR2_NEEDS_HOST_UTF8_LOCALE=y $BR2_CONFIG > /dev/null; then if ! which locale > /dev/null ; then echo echo "You need locale support on your build machine to build a toolchain supporting locales" exit 1 ; fi if ! locale -a | grep -q -i utf8$ ; then echo echo "You need at least one UTF8 locale to build a toolchain supporting locales" exit 1 ; fi fi
And the only package selecting BR2_NEEDS_HOST_UTF8_LOCALE is:
package/uclibc/Config.in: select BR2_NEEDS_HOST_UTF8_LOCALE
Is this check insufficient now ?
Best regards,
Thomas
Thomas Petazzoni, CTO, Free Electrons Embedded Linux and Kernel engineering http://free-electrons.com
Am 03.08.2017 um 10:11 schrieb Eugene Yudin:
Hi Thomas,
Is this check insufficient now ?
I think it's sufficient.
Please post output of the following commands: $ locale $ locale -a $ cat /etc/os-release
$ locale LANG=de_DE.utf8 LANGUAGE=de_DE.utf8 LC_CTYPE="de_DE.utf8" LC_NUMERIC="de_DE.utf8" LC_TIME="de_DE.utf8" LC_COLLATE="de_DE.utf8" LC_MONETARY="de_DE.utf8" LC_MESSAGES="de_DE.utf8" LC_PAPER="de_DE.utf8" LC_NAME="de_DE.utf8" LC_ADDRESS="de_DE.utf8" LC_TELEPHONE="de_DE.utf8" LC_MEASUREMENT="de_DE.utf8" LC_IDENTIFICATION="de_DE.utf8" LC_ALL=de_DE.utf8
$ locale -a C C.UTF-8 de_DE.utf8 POSIX
OS is Ubuntu 14.04.5 LTS Should I install the english locale?
Yes. Please install the locale "en_US.UTF-8". It's used in buildroot by default: package/uclibc/uclibc.mk:46:UCLIBC_LOCALES = en_US
On Tue, Aug 8, 2017 at 7:52 PM, André Hentschel nerv@dawncrow.de wrote:
Am 03.08.2017 um 10:11 schrieb Eugene Yudin:
Hi Thomas,
Is this check insufficient now ?
I think it's sufficient.
Please post output of the following commands: $ locale $ locale -a $ cat /etc/os-release
$ locale LANG=de_DE.utf8 LANGUAGE=de_DE.utf8 LC_CTYPE="de_DE.utf8" LC_NUMERIC="de_DE.utf8" LC_TIME="de_DE.utf8" LC_COLLATE="de_DE.utf8" LC_MONETARY="de_DE.utf8" LC_MESSAGES="de_DE.utf8" LC_PAPER="de_DE.utf8" LC_NAME="de_DE.utf8" LC_ADDRESS="de_DE.utf8" LC_TELEPHONE="de_DE.utf8" LC_MEASUREMENT="de_DE.utf8" LC_IDENTIFICATION="de_DE.utf8" LC_ALL=de_DE.utf8
$ locale -a C C.UTF-8 de_DE.utf8 POSIX
OS is Ubuntu 14.04.5 LTS Should I install the english locale?
done:
$ locale -a C C.UTF-8 de_DE.utf8 en_US en_US.iso88591 en_US.utf8 POSIX
Am 08.08.2017 um 21:07 schrieb Eugene Yudin:
Yes. Please install the locale "en_US.UTF-8". It's used in buildroot by default: package/uclibc/uclibc.mk:46:UCLIBC_LOCALES = en_US
On Tue, Aug 8, 2017 at 7:52 PM, André Hentschel <nerv@dawncrow.de mailto:nerv@dawncrow.de> wrote:
Am 03.08.2017 um 10:11 schrieb Eugene Yudin: > Hi Thomas, > >> Is this check insufficient now ? > I think it's sufficient. > > Please post output of the following commands: > $ locale > $ locale -a > $ cat /etc/os-release $ locale LANG=de_DE.utf8 LANGUAGE=de_DE.utf8 LC_CTYPE="de_DE.utf8" LC_NUMERIC="de_DE.utf8" LC_TIME="de_DE.utf8" LC_COLLATE="de_DE.utf8" LC_MONETARY="de_DE.utf8" LC_MESSAGES="de_DE.utf8" LC_PAPER="de_DE.utf8" LC_NAME="de_DE.utf8" LC_ADDRESS="de_DE.utf8" LC_TELEPHONE="de_DE.utf8" LC_MEASUREMENT="de_DE.utf8" LC_IDENTIFICATION="de_DE.utf8" LC_ALL=de_DE.utf8 $ locale -a C C.UTF-8 de_DE.utf8 POSIX OS is Ubuntu 14.04.5 LTS Should I install the english locale?-- Best regards, Eugene
On Tue, Aug 8, 2017 at 12:07 PM, Eugene Yudin e.yudin@ndmsystems.com wrote:
Yes. Please install the locale "en_US.UTF-8". It's used in buildroot by default: package/uclibc/uclibc.mk:46:UCLIBC_LOCALES = en_US
Just curious: why not defaulting to C (or C.UTF-8) locale?
Hello,
On Tue, 8 Aug 2017 18:52:14 +0200, André Hentschel wrote:
Please post output of the following commands: $ locale $ locale -a $ cat /etc/os-release
$ locale LANG=de_DE.utf8 LANGUAGE=de_DE.utf8 LC_CTYPE="de_DE.utf8" LC_NUMERIC="de_DE.utf8" LC_TIME="de_DE.utf8" LC_COLLATE="de_DE.utf8" LC_MONETARY="de_DE.utf8" LC_MESSAGES="de_DE.utf8" LC_PAPER="de_DE.utf8" LC_NAME="de_DE.utf8" LC_ADDRESS="de_DE.utf8" LC_TELEPHONE="de_DE.utf8" LC_MEASUREMENT="de_DE.utf8" LC_IDENTIFICATION="de_DE.utf8" LC_ALL=de_DE.utf8
$ locale -a C C.UTF-8 de_DE.utf8 POSIX
OS is Ubuntu 14.04.5 LTS Should I install the english locale?
As per the current Buildroot requirements, it shouldn't be needed. We require one UTF-8 locale to be installed for uClibc to build its locale data, but not specifically the en_US.UTF-8 locale or any other english locale. If that has changed in uClibc, then we should adjust the dependency in Buildroot accordingly, but it would be a bit annoying.
Best regards,
Thomas
I describe the current situation: 1) user enables locale support (BR2_TOOLCHAIN_BUILDROOT_LOCALE); 2) user optionally can set custom list of locales (BR2_GENERATE_LOCALE).
The option "BR2_GENERATE_LOCALE" is handled in "package/uclibc/uclibc.mk": UCLIBC_GENERATE_LOCALES = $(call qstrip,$(BR2_GENERATE_LOCALE))
ifeq ($(UCLIBC_GENERATE_LOCALES),) # We need at least one locale UCLIBC_LOCALES = en_US else # Strip out the encoding part of locale names, if any UCLIBC_LOCALES = \ $(foreach locale,$(UCLIBC_GENERATE_LOCALES),\ $(firstword $(subst .,$(space),$(locale)))) endif ... define UCLIBC_LOCALE_CONFIG ... $(call KCONFIG_ENABLE_OPT,UCLIBC_BUILD_MINIMAL_LOCALE,$(@D)/.config) $(call KCONFIG_SET_OPT,UCLIBC_BUILD_MINIMAL_LOCALES,"$(UCLIBC_LOCALES)",$(@D)/.config) ... endef
We can see that with empty list buildroot passes "en_US" to uClibc-ng.
The system library makefile handles option the following way (extra/locale/Makefile.in): $(locale_OUT)/wctables.h: $(locale_OUT)/gen_wctype @$(disp_gen) $(Q)for locale in $(call qstrip,$(UCLIBC_BUILD_MINIMAL_LOCALES)) en_US en_GB; do \ $< $(FLAG-locale-verbose) $$locale > $@ || \ $< $(FLAG-locale-verbose) $$locale.UTF-8 > $@ || \ $< $(FLAG-locale-verbose) $$locale.iso8859-1 > $@ && break; \ done
The program "gen_wctype" receives locale name as command line parameter. That application call the function setlocale() inside it. Some time ago that call was disabled to workaround bug with locales. That's the reason for the absence of problems in the past. Changes in the last release return this call.
On Wed, Aug 9, 2017 at 12:00 AM, Thomas Petazzoni < thomas.petazzoni@free-electrons.com> wrote:
Hello,
On Tue, 8 Aug 2017 18:52:14 +0200, André Hentschel wrote:
Please post output of the following commands: $ locale $ locale -a $ cat /etc/os-release
$ locale LANG=de_DE.utf8 LANGUAGE=de_DE.utf8 LC_CTYPE="de_DE.utf8" LC_NUMERIC="de_DE.utf8" LC_TIME="de_DE.utf8" LC_COLLATE="de_DE.utf8" LC_MONETARY="de_DE.utf8" LC_MESSAGES="de_DE.utf8" LC_PAPER="de_DE.utf8" LC_NAME="de_DE.utf8" LC_ADDRESS="de_DE.utf8" LC_TELEPHONE="de_DE.utf8" LC_MEASUREMENT="de_DE.utf8" LC_IDENTIFICATION="de_DE.utf8" LC_ALL=de_DE.utf8
$ locale -a C C.UTF-8 de_DE.utf8 POSIX
OS is Ubuntu 14.04.5 LTS Should I install the english locale?
As per the current Buildroot requirements, it shouldn't be needed. We require one UTF-8 locale to be installed for uClibc to build its locale data, but not specifically the en_US.UTF-8 locale or any other english locale. If that has changed in uClibc, then we should adjust the dependency in Buildroot accordingly, but it would be a bit annoying.
Best regards,
Thomas
Thomas Petazzoni, CTO, Free Electrons Embedded Linux and Kernel engineering http://free-electrons.com
On 10-08-17 11:09, Eugene Yudin wrote:
The system library makefile handles option the following way (extra/locale/Makefile.in): $(locale_OUT)/wctables.h: $(locale_OUT)/gen_wctype @$(disp_gen) $(Q)for locale in $(call qstrip,$(UCLIBC_BUILD_MINIMAL_LOCALES)) en_US en_GB; do \ $< $(FLAG-locale-verbose) $$locale > $@ || \ $< $(FLAG-locale-verbose) $$locale.UTF-8 > $@ || \ $< $(FLAG-locale-verbose) $$locale.iso8859-1 > $@ && break; \ done
The program "gen_wctype" receives locale name as command line parameter. That application call the function setlocale() inside it. Some time ago that call was disabled to workaround bug with locales. That's the reason for the absence of problems in the past. Changes in the last release return this call.
So the problem really is that this command tries a number of locales, but not any of the locales actually present on the system?
I would propose to: - replace the "en_US en_GB" with "C C.UTF-8" since the latter is much more likely to be present on the host; - use '$(sort $(patsubst %.utf8,%,$(shell locale -a)))' as an ultimate fallback.
Does that sound reasonable? I didn't look at the history if this particular piece of code, I guess it must have gone through several detours already...
Regards, Arnout
Yes. In that particular case loop iterates through en_US (from buildroot), en_US and en_GB. Of cause, that list can be patched but I don't know if that's correct. It's look a bit strange if user passes locale which actually absent.
Waldemar, what you think about it?
I made several tests with different locales (en_US, en_GB, ru_RU, C.UTF-8 and C). The generated header is the same for all the above locale except pure C.
Comment in the same makefile tells about problem with some locales. I think it would be better to use C locale for fallback.
I attached the tested patch to modify fallback behavior based on Arnout draft. I also changed the order of calls to raise priority of C.UTF-8 locale.
On Thu, Aug 10, 2017 at 12:36 PM, Arnout Vandecappelle arnout@mind.be wrote:
On 10-08-17 11:09, Eugene Yudin wrote:
The system library makefile handles option the following way (extra/locale/Makefile.in): $(locale_OUT)/wctables.h: $(locale_OUT)/gen_wctype @$(disp_gen) $(Q)for locale in $(call qstrip,$(UCLIBC_BUILD_MINIMAL_LOCALES))
en_US
en_GB; do \ $< $(FLAG-locale-verbose) $$locale > $@ || \ $< $(FLAG-locale-verbose) $$locale.UTF-8 > $@ || \ $< $(FLAG-locale-verbose) $$locale.iso8859-1 > $@ &&
break; \
doneThe program "gen_wctype" receives locale name as command line parameter. That application call the function setlocale() inside it. Some time ago that call was disabled to workaround bug with locales. That's the reason for the absence of problems in the past. Changes in the last release return this call.
So the problem really is that this command tries a number of locales, but not any of the locales actually present on the system?
I would propose to:
- replace the "en_US en_GB" with "C C.UTF-8" since the latter is much more
likely to be present on the host;
- use '$(sort $(patsubst %.utf8,%,$(shell locale -a)))' as an ultimate
fallback.
Does that sound reasonable? I didn't look at the history if this particular piece of code, I guess it must have gone through several detours already...
Regards, Arnout
-- Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
Hi Eugene, Eugene Yudin wrote,
Yes. In that particular case loop iterates through en_US (from buildroot), en_US and en_GB. Of cause, that list can be patched but I don't know if that's correct. It's look a bit strange if user passes locale which actually absent.
Waldemar, what you think about it?
I made several tests with different locales (en_US, en_GB, ru_RU, C.UTF-8 and C). The generated header is the same for all the above locale except pure C.
Comment in the same makefile tells about problem with some locales. I think it would be better to use C locale for fallback.
I attached the tested patch to modify fallback behavior based on Arnout draft. I also changed the order of calls to raise priority of C.UTF-8 locale.
Looks good to me. First trying the user configured locale (.UTF-8) and if it does not exist fallback to C.UTF-8.
Not sure if we should do configured locale (en_US.UTF-8) then C.UTF_8 and then even C, when C doesn't build workable output?
best regards Waldemar
-
André Hentschel -
Arnout Vandecappelle -
Eugene Yudin -
Max Filippov -
Thomas Petazzoni -
Waldemar Brodkorb