8 Jun
2016
8 Jun
'16
3:32 p.m.
Hello,
I found an issue when running uClibc test "regex/testregex".
I launched tests using "uclibcng-testrunner.sh" script. And the script
printed that "testregex" had been passed successfully. But when I
opened "testregex.out" file I found, that many "testregex" sub-tests
failed.
This behavior is caused because test returns 0 exit code even when sub-
tests fail.
As a result we don't know was the "testregex" execution successful or
not though it is marked as "PASSED".
The problem was found when running tests on ARC HS38 with Linux 4.6.1.
It also was reproduced on ARMv5 with Linux 4.5.3.
--
Best regards,
Vlad Zakharov <Vladislav.Zakharov(a)synopsys.com>
9 Jun
9 Jun
9:22 p.m.
Hi Vlad,
Vlad Zakharov wrote,
Hello,
I found an issue when running uClibc test "regex/testregex".
I launched tests using "uclibcng-testrunner.sh" script. And the script
printed that "testregex" had been passed successfully. But when I
opened "testregex.out" file I found, that many "testregex" sub-tests
failed.
This behavior is caused because test returns 0 exit code even when sub-
tests fail.
As a result we don't know was the "testregex" execution successful or
not though it is marked as "PASSED".
The problem was found when running tests on ARC HS38 with Linux 4.6.1.
It also was reproduced on ARMv5 with Linux 4.5.3.
Thanks for the report.
Do you have any preferred way how we should fix the issue?
Should we fail if any sub-test fail?
best regards
Waldemar
10 Jun
10 Jun
6:58 a.m.
Hi Waldemar,
On Thu, 2016-06-09 at 23:22 +0200, Waldemar Brodkorb wrote:
Hi Vlad,
Vlad Zakharov wrote,
Let me answer on behalf of Vlad while he's off today.
Indeed I think it worth returning <> 0 value on any subtest failure.
Otherwise IMHO results will be a bit confusing.
-Alexey
Hello,
I found an issue when running uClibc test "regex/testregex".
I launched tests using "uclibcng-testrunner.sh" script. And the script
printed that "testregex" had been passed successfully. But when I
opened "testregex.out" file I found, that many "testregex" sub-tests
failed.
This behavior is caused because test returns 0 exit code even when sub-
tests fail.
As a result we don't know was the "testregex" execution successful or
not though it is marked as "PASSED".
The problem was found when running tests on ARC HS38 with Linux 4.6.1.
It also was reproduced on ARMv5 with Linux 4.5.3.
Thanks for the report.
Do you have any preferred way how we should fix the issue?
Should we fail if any sub-test fail?
15 Jun
15 Jun
6:37 p.m.
Hi Alexey,
Alexey Brodkin wrote,
Hi Waldemar,
On Thu, 2016-06-09 at 23:22 +0200, Waldemar Brodkorb wrote:
Do you have a patch suggestion for this?
best regards
Waldemar
Hi Vlad,
Vlad Zakharov wrote,
Let me answer on behalf of Vlad while he's off today.
Indeed I think it worth returning <> 0 value on any subtest failure.
Otherwise IMHO results will be a bit confusing.
Hello,
I found an issue when running uClibc test "regex/testregex".
I launched tests using "uclibcng-testrunner.sh" script. And the script
printed that "testregex" had been passed successfully. But when I
opened "testregex.out" file I found, that many "testregex" sub-tests
failed.
This behavior is caused because test returns 0 exit code even when sub-
tests fail.
As a result we don't know was the "testregex" execution successful or
not though it is marked as "PASSED".
The problem was found when running tests on ARC HS38 with Linux 4.6.1.
It also was reproduced on ARMv5 with Linux 4.5.3.
Thanks for the report.
Do you have any preferred way how we should fix the issue?
Should we fail if any sub-test fail?
16 Jun
16 Jun
1:33 p.m.
Hi Waldemar,
On Wed, 2016-06-15 at 20:37 +0200, Waldemar Brodkorb wrote:
Hi Alexey,
Alexey Brodkin wrote,
Vlad is already looking at this.
Stay tuned and patches will arrive soon I hope :)
-Alexey
Hi Waldemar,
On Thu, 2016-06-09 at 23:22 +0200, Waldemar Brodkorb wrote:
Do you have a patch suggestion for this?
Hi Vlad,
Vlad Zakharov wrote,
Let me answer on behalf of Vlad while
he's off today.
Indeed I think it worth returning <> 0 value on any subtest failure.
Otherwise IMHO results will be a bit confusing.
Hello,
I found an issue when running uClibc test "regex/testregex".
I launched tests using "uclibcng-testrunner.sh" script. And the script
printed that "testregex" had been passed successfully. But when I
opened "testregex.out" file I found, that many "testregex" sub-tests
failed.
This behavior is caused because test returns 0 exit code even when sub-
tests fail.
As a result we don't know was the "testregex" execution successful or
not though it is marked as "PASSED".
The problem was found when running tests on ARC HS38 with Linux 4.6.1.
It also was reproduced on ARMv5 with Linux 4.5.3.
Thanks for the report.
Do you have any preferred way how we should fix the issue?
Should we fail if any sub-test fail?
3081
days inactive
3089
days old
4 comments
3 participants
participants (3)
-
Alexey Brodkin -
Vlad Zakharov -
Waldemar Brodkorb