SuperH conflict of arch/sh/__set_thread_area vs thread/__set_thread_area

SuperH conflict of arch/sh/__set_thread_area vs thread/__set_thread_area

[Christian Lamparter]

Hello,

I'm trying to add a port for a SH4-like ARCH to OpenWRT, which uses the latest
musl-1.1.10 as the default libc. I'm having the following problem when building
the toolchain:

During the final linker-step, the symbol "__set_thread_area"  declared twice.
This is because the SH architecture provides a separate __set_thread_area [0],
(other archs use the standard syscall wrapper from [1]).

Obviously, I want this issue fixed. However I'm new to SuperH and musl, that's
why I need advise :-D. For now, I defined the src/thread/__set_thread_area as
a weak symbol. Now, that's just a crude hack, what would be better solution?
(I can make and post the patch if necessary - But sadly, I can't test it on the
hardware yet)?

Regards,
Christian  

[0] <http://git.musl-libc.org/cgit/musl/tree/arch/sh/src/__set_thread_area.c>
[1] <http://git.musl-libc.org/cgit/musl/tree/src/thread/__set_thread_area.c>

curious file access problem

[Sebastian Gottschall]

Hello Guys

can someone explain why a file read operation on /proc/net/route will 
return everything but the default gateway?
this happens only with musl and does for sure affect the command "route" 
in busybox

here a short example of this curiosity
strings does not print the default gateway.
cat does print it with default gateway

root@DD-WRT:~# strings /proc/net/route
Iface   Destination     Gateway         Flags   RefCnt  Use Metric  
Mask            MTU     Window  IRTT

ath1    00C1580A        00000000        0001    0       0 0       
00FFFFFF        0       0       0

lo      0000007F        00000000        0001    0       0 0       
000000FF        0       0       0

br0     0000FEA9        00000000        0001    0       0 0       
0000FFFF        0       0       0

br0     0001A8C0        00000000        0001    0       0 0       
00FFFFFF        0       0       0

root@DD-WRT:~# cat /proc/net/route
Iface   Destination     Gateway         Flags   RefCnt  Use Metric  
Mask            MTU     Window  IRTT

ath1    00000000        01C1580A        0003    0       0 0       
00000000        0       0       0

ath1    00C1580A        00000000        0001    0       0 0       
00FFFFFF        0       0       0

lo      0000007F        00000000        0001    0       0 0       
000000FF        0       0       0

br0     0000FEA9        00000000        0001    0       0 0       
0000FFFF        0       0       0

br0     0001A8C0        00000000        0001    0       0 0       
00FFFFFF        0       0       0


Sebastian

Re: curious file access problem

[Rich Felker]

On Thu, Aug 20, 2015 at 03:03:09AM +0200, Sebastian Gottschall wrote:
> Hello Guys
> 
> can someone explain why a file read operation on /proc/net/route
> will return everything but the default gateway?
> this happens only with musl and does for sure affect the command
> "route" in busybox
> 
> here a short example of this curiosity
> strings does not print the default gateway.
> cat does print it with default gateway

Which package is the strings command you're using coming from? Have
you looked at strace output for both commands? That might help
determine if it's something weird with how the reading is taking
place, or if strings is just deciding via its own logic that the one
line is non-text.

BTW in the future please send a new email rather than replying to an
existing thread to start a new topic. The way you did it messes up
threading. Just changing the subject does not start a new thread.

Rich

Re: curious file access problem

[Bastian Bittorf]

* Sebastian Gottschall <s.gottschall@dd-wrt.com> [20.08.2015 08:13]:
> root@DD-WRT:~# strings /proc/net/route
[...]

> root@DD-WRT:~# cat /proc/net/route
[...]

i cannot see this prob on openwrt.
what is your arch and your exact musl version?

can you also provide this:
hexdump -C /proc/net/route | head -n20

bye, bastian

Re: curious file access problem

[Sebastian Gottschall]

Am 20.08.2015 um 08:48 schrieb Bastian Bittorf:
> * Sebastian Gottschall <s.gottschall@dd-wrt.com> [20.08.2015 08:13]:
>> root@DD-WRT:~# strings /proc/net/route
> [...]
>
>> root@DD-WRT:~# cat /proc/net/route
> [...]
>
> i cannot see this prob on openwrt.
> what is your arch and your exact musl version?
same es openwrt. but it seems to be related only to kernel 4.1 (or newer)
in 3.18 it does not happen in openwrt

>
> can you also provide this:
> hexdump -C /proc/net/route | head -n20
when i'm back home. i will provide this info. but should be the same as 
strings. i wrote also a small test programm which used fopen and outputs 
the content
and its also missing the default gateway
>
> bye, bastian
>

Re: curious file access problem

[Bastian Bittorf]

* Sebastian Gottschall <s.gottschall@dd-wrt.com> [20.08.2015 13:21]:
> >i cannot see this prob on openwrt.
> >what is your arch and your exact musl version?
> same es openwrt. but it seems to be related only to kernel 4.1 (or newer)
> in 3.18 it does not happen in openwrt

i can see it too - not on 3.18 but with kernel 4.1.5
you are right, busybox 'route -n' is affected and does not
see a default route (the same for hexdump, strings...)

e.g. this does *not* work on the affected router (ar71xx/mips):
hexdump -C /proc/net/route | grep wlan0.00000000
(empty output - but works with kernel 3.18 on the same box)

ofcourse 'ip route' is working (does not parse '/proc/net/route')

bye, bastian

Re: curious file access problem

[Rich Felker]

On Thu, Aug 20, 2015 at 03:25:39PM +0200, Bastian Bittorf wrote:
> * Sebastian Gottschall <s.gottschall@dd-wrt.com> [20.08.2015 13:21]:
> > >i cannot see this prob on openwrt.
> > >what is your arch and your exact musl version?
> > same es openwrt. but it seems to be related only to kernel 4.1 (or newer)
> > in 3.18 it does not happen in openwrt
> 
> i can see it too - not on 3.18 but with kernel 4.1.5
> you are right, busybox 'route -n' is affected and does not
> see a default route (the same for hexdump, strings...)

In that case, it sounds like somebody broke the kernel. This is rather
unfortunate, and it would be great if someone could look into the bug
and get it fixed before these broken kernels are too widespread...

Rich

Re: curious file access problem

[Bastian Bittorf]

* Rich Felker <dalias@libc.org> [20.08.2015 16:14]:
> > i can see it too - not on 3.18 but with kernel 4.1.5
> > you are right, busybox 'route -n' is affected and does not
> > see a default route (the same for hexdump, strings...)
> 
> In that case, it sounds like somebody broke the kernel. This is rather
> unfortunate, and it would be great if someone could look into the bug
> and get it fixed before these broken kernels are too widespread...

to be more exact:
'route -n' is does not see a default route (while parsing
/proc/net/route) but 'cat /proc/net/route' see's it (so the kernel is
fine). busybox is the same version (so busybox is fine).
so whats left? 8-))) at least this is something fishy.
(and i want somebody who can reproduce this e.g. on x86)

bye, bastan

Re: curious file access problem

[Rich Felker]

On Thu, Aug 20, 2015 at 04:26:14PM +0200, Bastian Bittorf wrote:
> * Rich Felker <dalias@libc.org> [20.08.2015 16:14]:
> > > i can see it too - not on 3.18 but with kernel 4.1.5
> > > you are right, busybox 'route -n' is affected and does not
> > > see a default route (the same for hexdump, strings...)
> > 
> > In that case, it sounds like somebody broke the kernel. This is rather
> > unfortunate, and it would be great if someone could look into the bug
> > and get it fixed before these broken kernels are too widespread...
> 
> to be more exact:
> 'route -n' is does not see a default route (while parsing
> /proc/net/route) but 'cat /proc/net/route' see's it (so the kernel is
> fine).

That doesn't follow. This is almost certainly a kernel bug where
different sequences of read syscalls see different contents in the
/proc-bsed pseudo-file due to some sort of state corruption or other
bug in the 'file position' state the kernel keeps for it. Stuff like
this has happened before in /proc and it wouldn't surprise me at all
if someone broke it again.

Rich

Re: curious file access problem

[Sebastian Gottschall]

Am 20.08.2015 um 15:48 schrieb Rich Felker:
> On Thu, Aug 20, 2015 at 03:25:39PM +0200, Bastian Bittorf wrote:
>> * Sebastian Gottschall <s.gottschall@dd-wrt.com> [20.08.2015 13:21]:
>>>> i cannot see this prob on openwrt.
>>>> what is your arch and your exact musl version?
>>> same es openwrt. but it seems to be related only to kernel 4.1 (or newer)
>>> in 3.18 it does not happen in openwrt
>> i can see it too - not on 3.18 but with kernel 4.1.5
>> you are right, busybox 'route -n' is affected and does not
>> see a default route (the same for hexdump, strings...)
> In that case, it sounds like somebody broke the kernel. This is rather
> unfortunate, and it would be great if someone could look into the bug
> and get it fixed before these broken kernels are too widespread...
there are some fib tree enhancements. i nailed it down to a patchset 
which i can apply and revert to let the problem show up
the interesting thing is that uclibc doesnt show that behaviour. but i 
dont know why. maybe its some sort of application based routing table?
>
> Rich
>

Re: curious file access problem

[Rich Felker]

On Thu, Aug 20, 2015 at 04:51:56PM +0200, Sebastian Gottschall wrote:
> Am 20.08.2015 um 15:48 schrieb Rich Felker:
> >On Thu, Aug 20, 2015 at 03:25:39PM +0200, Bastian Bittorf wrote:
> >>* Sebastian Gottschall <s.gottschall@dd-wrt.com> [20.08.2015 13:21]:
> >>>>i cannot see this prob on openwrt.
> >>>>what is your arch and your exact musl version?
> >>>same es openwrt. but it seems to be related only to kernel 4.1 (or newer)
> >>>in 3.18 it does not happen in openwrt
> >>i can see it too - not on 3.18 but with kernel 4.1.5
> >>you are right, busybox 'route -n' is affected and does not
> >>see a default route (the same for hexdump, strings...)
> >In that case, it sounds like somebody broke the kernel. This is rather
> >unfortunate, and it would be great if someone could look into the bug
> >and get it fixed before these broken kernels are too widespread...
> there are some fib tree enhancements. i nailed it down to a patchset
> which i can apply and revert to let the problem show up
> the interesting thing is that uclibc doesnt show that behaviour. but
> i dont know why. maybe its some sort of application based routing
> table?

Again, if the bug is in the kernel's tracking of the pseudo-file
position state (which seems clear now), then whether it manifests or
not is almost certainly a matter of _how_ the reading takes place
(small chunks, big chunks, read vs readv vs pread, whether or not
seeking is involved, etc.). And this is going to vary between
applications, and if stdio is used by the application to perform the
reading, then it's also going to vary between stdio implementations
(and thus libcs).

Rich

Re: curious file access problem

[Timo Teras]

On Thu, 20 Aug 2015 11:00:20 -0400
Rich Felker <dalias@libc.org> wrote:

> On Thu, Aug 20, 2015 at 04:51:56PM +0200, Sebastian Gottschall wrote:
> > Am 20.08.2015 um 15:48 schrieb Rich Felker:
> > >On Thu, Aug 20, 2015 at 03:25:39PM +0200, Bastian Bittorf wrote:
> > >>* Sebastian Gottschall <s.gottschall@dd-wrt.com> [20.08.2015
> > >>13:21]:
> > >>>>i cannot see this prob on openwrt.
> > >>>>what is your arch and your exact musl version?
> > >>>same es openwrt. but it seems to be related only to kernel 4.1
> > >>>(or newer) in 3.18 it does not happen in openwrt
> > >>i can see it too - not on 3.18 but with kernel 4.1.5
> > >>you are right, busybox 'route -n' is affected and does not
> > >>see a default route (the same for hexdump, strings...)
> > >In that case, it sounds like somebody broke the kernel. This is
> > >rather unfortunate, and it would be great if someone could look
> > >into the bug and get it fixed before these broken kernels are too
> > >widespread...
> > there are some fib tree enhancements. i nailed it down to a patchset
> > which i can apply and revert to let the problem show up
> > the interesting thing is that uclibc doesnt show that behaviour. but
> > i dont know why. maybe its some sort of application based routing
> > table?
> 
> Again, if the bug is in the kernel's tracking of the pseudo-file
> position state (which seems clear now), then whether it manifests or
> not is almost certainly a matter of _how_ the reading takes place
> (small chunks, big chunks, read vs readv vs pread, whether or not
> seeking is involved, etc.). And this is going to vary between
> applications, and if stdio is used by the application to perform the
> reading, then it's also going to vary between stdio implementations
> (and thus libcs).

DaveM's net -stable queue has:
http://patchwork.ozlabs.org/patch/507167/

Sounds like this is the fix for a kernel regression.

/Timo

Re: curious file access problem

[Sebastian Gottschall]

Am 20.08.2015 um 20:30 schrieb Timo Teras:
> On Thu, 20 Aug 2015 11:00:20 -0400
> Rich Felker <dalias@libc.org> wrote:
>
>> On Thu, Aug 20, 2015 at 04:51:56PM +0200, Sebastian Gottschall wrote:
>>> Am 20.08.2015 um 15:48 schrieb Rich Felker:
>>>> On Thu, Aug 20, 2015 at 03:25:39PM +0200, Bastian Bittorf wrote:
>>>>> * Sebastian Gottschall <s.gottschall@dd-wrt.com> [20.08.2015
>>>>> 13:21]:
>>>>>>> i cannot see this prob on openwrt.
>>>>>>> what is your arch and your exact musl version?
>>>>>> same es openwrt. but it seems to be related only to kernel 4.1
>>>>>> (or newer) in 3.18 it does not happen in openwrt
>>>>> i can see it too - not on 3.18 but with kernel 4.1.5
>>>>> you are right, busybox 'route -n' is affected and does not
>>>>> see a default route (the same for hexdump, strings...)
>>>> In that case, it sounds like somebody broke the kernel. This is
>>>> rather unfortunate, and it would be great if someone could look
>>>> into the bug and get it fixed before these broken kernels are too
>>>> widespread...
>>> there are some fib tree enhancements. i nailed it down to a patchset
>>> which i can apply and revert to let the problem show up
>>> the interesting thing is that uclibc doesnt show that behaviour. but
>>> i dont know why. maybe its some sort of application based routing
>>> table?
>> Again, if the bug is in the kernel's tracking of the pseudo-file
>> position state (which seems clear now), then whether it manifests or
>> not is almost certainly a matter of _how_ the reading takes place
>> (small chunks, big chunks, read vs readv vs pread, whether or not
>> seeking is involved, etc.). And this is going to vary between
>> applications, and if stdio is used by the application to perform the
>> reading, then it's also going to vary between stdio implementations
>> (and thus libcs).
> DaveM's net -stable queue has:
> http://patchwork.ozlabs.org/patch/507167/
>
> Sounds like this is the fix for a kernel regression.
>
> /Timo
>
absolutelly. i will try it and give feedback

Re: curious file access problem

[Sebastian Gottschall]

the refered patch fixes the issue. all back to normal now.
thank you for investigation


Sebastian

Am 20.08.2015 um 20:30 schrieb Timo Teras:
> On Thu, 20 Aug 2015 11:00:20 -0400
> Rich Felker <dalias@libc.org> wrote:
>
>> On Thu, Aug 20, 2015 at 04:51:56PM +0200, Sebastian Gottschall wrote:
>>> Am 20.08.2015 um 15:48 schrieb Rich Felker:
>>>> On Thu, Aug 20, 2015 at 03:25:39PM +0200, Bastian Bittorf wrote:
>>>>> * Sebastian Gottschall <s.gottschall@dd-wrt.com> [20.08.2015
>>>>> 13:21]:
>>>>>>> i cannot see this prob on openwrt.
>>>>>>> what is your arch and your exact musl version?
>>>>>> same es openwrt. but it seems to be related only to kernel 4.1
>>>>>> (or newer) in 3.18 it does not happen in openwrt
>>>>> i can see it too - not on 3.18 but with kernel 4.1.5
>>>>> you are right, busybox 'route -n' is affected and does not
>>>>> see a default route (the same for hexdump, strings...)
>>>> In that case, it sounds like somebody broke the kernel. This is
>>>> rather unfortunate, and it would be great if someone could look
>>>> into the bug and get it fixed before these broken kernels are too
>>>> widespread...
>>> there are some fib tree enhancements. i nailed it down to a patchset
>>> which i can apply and revert to let the problem show up
>>> the interesting thing is that uclibc doesnt show that behaviour. but
>>> i dont know why. maybe its some sort of application based routing
>>> table?
>> Again, if the bug is in the kernel's tracking of the pseudo-file
>> position state (which seems clear now), then whether it manifests or
>> not is almost certainly a matter of _how_ the reading takes place
>> (small chunks, big chunks, read vs readv vs pread, whether or not
>> seeking is involved, etc.). And this is going to vary between
>> applications, and if stdio is used by the application to perform the
>> reading, then it's also going to vary between stdio implementations
>> (and thus libcs).
> DaveM's net -stable queue has:
> http://patchwork.ozlabs.org/patch/507167/
>
> Sounds like this is the fix for a kernel regression.
>
> /Timo
>

Re: curious file access problem

[Bastian Bittorf]

* Sebastian Gottschall <s.gottschall@dd-wrt.com> [20.08.2015 08:13]:
> this happens only with musl and does for sure affect the command
> "route" in busybox

some more checks:
busybox's version is the same here. works on 3.18 with
musl https://dev.openwrt.org/changeset/46134/trunk (26-june)
and does not work on kernel 4.1.5 with musl
https://dev.openwrt.org/changeset/46481/trunk (24-july)
but there are a lot of patches. it would help if somebody
can test this e.g. on gentoo.

bye, bastian

Re: SuperH conflict of arch/sh/__set_thread_area vs thread/__set_thread_area

[Bobby Bingham]

On Thu, Aug 20, 2015 at 02:44:11AM +0200, Christian Lamparter wrote:
> Hello,
> 
> I'm trying to add a port for a SH4-like ARCH to OpenWRT, which uses the latest
> musl-1.1.10 as the default libc. I'm having the following problem when building
> the toolchain:
> 
> During the final linker-step, the symbol "__set_thread_area"  declared twice.
> This is because the SH architecture provides a separate __set_thread_area [0],
> (other archs use the standard syscall wrapper from [1]).

There is an empty file src/thread/sh/__set_thread_area.s which should be
overriding src/thread/__set_thread_area.c.  Therefore, only the definition
from arch/sh/src/__set_thread_area.c should be getting linked in.

Can you tell why src/thread/__set_thread_area.c is still getting used?

> 
> Obviously, I want this issue fixed. However I'm new to SuperH and musl, that's
> why I need advise :-D. For now, I defined the src/thread/__set_thread_area as
> a weak symbol. Now, that's just a crude hack, what would be better solution?
> (I can make and post the patch if necessary - But sadly, I can't test it on the
> hardware yet)?
> 
> Regards,
> Christian  
> 
> [0] <http://git.musl-libc.org/cgit/musl/tree/arch/sh/src/__set_thread_area.c>
> [1] <http://git.musl-libc.org/cgit/musl/tree/src/thread/__set_thread_area.c>

Re: SuperH conflict of arch/sh/__set_thread_area vs thread/__set_thread_area

[Rich Felker]

On Thu, Aug 20, 2015 at 02:44:11AM +0200, Christian Lamparter wrote:
> Hello,
> 
> I'm trying to add a port for a SH4-like ARCH to OpenWRT, which uses the latest
> musl-1.1.10 as the default libc. I'm having the following problem when building
> the toolchain:
> 
> During the final linker-step, the symbol "__set_thread_area"  declared twice.
> This is because the SH architecture provides a separate __set_thread_area [0],
> (other archs use the standard syscall wrapper from [1]).
> 
> Obviously, I want this issue fixed. However I'm new to SuperH and musl, that's
> why I need advise :-D. For now, I defined the src/thread/__set_thread_area as
> a weak symbol. Now, that's just a crude hack, what would be better solution?
> (I can make and post the patch if necessary - But sadly, I can't test it on the
> hardware yet)?

Bobby Bingham's reply explains what the issue is. Did you make a new
arch name rather than using the existing sh arch for your port? If
it's ABI-compatible, it would probably make more sense to use the
existing one and extend it to support your hardware. I'm working on an
sh-related project right now (see http://0pf.org/j-core.html) and one
of my goals is to get sh more unified as a platform where it's
possible to have binaries for the baseline ISA that work on lots of
different hardware models (including nommu ones).

Rich

Re: SuperH conflict of arch/sh/__set_thread_area vs thread/__set_thread_area

[Christian Lamparter]

On Wednesday, August 19, 2015 11:04:02 PM Rich Felker wrote:
> On Thu, Aug 20, 2015 at 02:44:11AM +0200, Christian Lamparter wrote:
> > Hello,
> > 
> > I'm trying to add a port for a SH4-like ARCH to OpenWRT, which uses the latest
> > musl-1.1.10 as the default libc. I'm having the following problem when building
> > the toolchain:
> > 
> > During the final linker-step, the symbol "__set_thread_area"  declared twice.
> > This is because the SH architecture provides a separate __set_thread_area [0],
> > (other archs use the standard syscall wrapper from [1]).
> > 
> > Obviously, I want this issue fixed. However I'm new to SuperH and musl, that's
> > why I need advise :-D. For now, I defined the src/thread/__set_thread_area as
> > a weak symbol. Now, that's just a crude hack, what would be better solution?
> > (I can make and post the patch if necessary - But sadly, I can't test it on the
> > hardware yet)?
> 
> Bobby Bingham's reply explains what the issue is. Did you make a new
> arch name rather than using the existing sh arch for your port? 
Initially, yes I did. I had the ARCH at "sh4". This was because OpenWRT 
already had infrastructure for some sh-(sub)targets (sh3, sh3eb, sh4, sh4eb)
in place. But they seem to be unused and untested. The only target which has
support for SuperH is UserModeLinux. However, it will probably run into the
same issue.

Now, I've changed ARCH to "sh" and set the CPU_TYPE to sh4 [toolchain
dir changed to toolchain-sh_sh4_gcc-5.2.0_musl-1.1.10]. But still no 
luck, the original error code remains the same.

src/thread/__set_thread_area.lo: In function `__set_thread_area':
__set_thread_area.c:(.text+0x0): multiple definition of `__set_thread_area'
arch/sh/src/__set_thread_area.lo:__set_thread_area.c:(.text+0x0): first defined here
collect2: error: ld returned 1 exit status
Makefile:142: recipe for target 'lib/libc.so' failed

> If it's ABI-compatible, it would probably make more sense to use the
> existing one and extend it to support your hardware.
In case you want to know: Hardware is a 2013 Sat>IP set-top-box
(idl4k platform) with a STx7108 (that's sh4, little endian, 
with mmu and with fpu).

> I'm working on an sh-related project right now 
> (see http://0pf.org/j-core.html) and one of my goals is to get sh
> more unified as a platform where it's possible to have binaries
> for the baseline ISA that work on lots of different hardware models
> (including nommu ones).
The J4 arch is sadly 2 years ahead of time ;-) [EU4]. But I think
I'll definitely need some help to figure out - Another issue seems
to be that the arch string changed from sh*-linux to sh-*-linux
between gcc-4.8 and gcc-5.2... So, what to do?

Regards,
Christian

Re: SuperH conflict of arch/sh/__set_thread_area vs thread/__set_thread_area

[Christian Lamparter]

(Added Openwrt-dev - since this isn't a musl issue)

On Thursday, August 20, 2015 12:21:55 PM Christian Lamparter wrote:
> On Wednesday, August 19, 2015 11:04:02 PM Rich Felker wrote:
> > On Thu, Aug 20, 2015 at 02:44:11AM +0200, Christian Lamparter wrote:
> > > I'm trying to add a port for a SH4-like ARCH to OpenWRT, which uses the latest
> > > musl-1.1.10 as the default libc. I'm having the following problem when building
> > > the toolchain:
> > > 
> > > During the final linker-step, the symbol "__set_thread_area"  declared twice.
> > > This is because the SH architecture provides a separate __set_thread_area [0],
> > > (other archs use the standard syscall wrapper from [1]).
> > > 
> > > Obviously, I want this issue fixed. However I'm new to SuperH and musl, that's
> > > why I need advise :-D. For now, I defined the src/thread/__set_thread_area as
> > > a weak symbol. Now, that's just a crude hack, what would be better solution?
> > > (I can make and post the patch if necessary - But sadly, I can't test it on the
> > > hardware yet)?
> > 
> > Bobby Bingham's reply explains what the issue is. Did you make a new
> > arch name rather than using the existing sh arch for your port? 
> Initially, yes I did. I had the ARCH at "sh4". This was because OpenWRT 
> already had infrastructure for some sh-(sub)targets (sh3, sh3eb, sh4, sh4eb)
> in place. But they seem to be unused and untested. The only target which has
> support for SuperH is UserModeLinux. However, it will probably run into the
> same issue.
> 
> Now, I've changed ARCH to "sh" and set the CPU_TYPE to sh4 [toolchain
> dir changed to toolchain-sh_sh4_gcc-5.2.0_musl-1.1.10]. But still no 
> luck, the original error code remains the same.
> 
> src/thread/__set_thread_area.lo: In function `__set_thread_area':
> __set_thread_area.c:(.text+0x0): multiple definition of `__set_thread_area'
> arch/sh/src/__set_thread_area.lo:__set_thread_area.c:(.text+0x0): first defined here
> collect2: error: ld returned 1 exit status
> Makefile:142: recipe for target 'lib/libc.so' failed

Ok, I have a update and I fixed this issue with __set_thread_area.s.
The reason why this didn't work and was harder to debug than usual is
because of the involvement of the "patch" utility with the -E flag
(--remove-empty-files).

Openwrt adds a number of patches to the musl-1.1.10.tar.gz [0] (actually
they seem to be cherry-picked from musl git). Now, one of the patches 
"001-git-2015-07-22.patch" contains:

commit f9d84554bae0fa17c9a1d724549c4408022228a5 [1]
Author: Rich Felker <dalias@aerifal.cx>
Date:   Tue Jun 16 14:28:30 2015 +0000

    add support for sh2 interrupt-masking-based atomics to sh port
 
This commit reduces the "src/thread/sh/__set_thread_area.s" to an empty
file (but it doesn't remove it). But due to bad luck, the script that
patches the file [2] (it's called patch-kernel.sh, but it's used to patch
the toolchain as well) uses the pesky "-E" flag (line 40):
 [...] ${PATCH:-patch} -f -p1 -E -d ${targetdir} 
                              ^^

==> And that's why the __set_thread_area.s file disappears.

I came up with two possible fixes for OpenWRT, unless you want to
fix this in musl (via non-empty __set_thread_area) instead: 

1. have some content in __set_thread_area.s to prevent it from being
   removed.

---

--- a/src/thread/sh/__set_thread_area.s
+++ b/src/thread/sh/__set_thread_area.s
@@ -0,0 +1,1 @@
+/* the patch utility might remove empty files */

---


2. patch patch-kernel.sh

---
--- a/scripts/patch-kernel.sh
+++ b/scripts/patch-kernel.sh
@@ -37,7 +37,7 @@ for i in ${patchdir}/${patchpattern} ; do
     [ -d "${i}" ] && echo "Ignoring subdirectory ${i}" && continue	
     echo ""
     echo "Applying ${i} using ${type}: " 
-    ${uncomp} ${i} | ${PATCH:-patch} -f -p1 -E -d ${targetdir} 
+    ${uncomp} ${i} | ${PATCH:-patch} -f -p1 -d ${targetdir}
     if [ $? != 0 ] ; then
         echo "Patch failed!  Please fix $i!"
 	exit 1

---

Does anyone have any preferences? I'm opting for "patch patch-kernel"
(and I will send a proper patch next week to OpenWRT even if no one
cares now :-D ) 

Regards,
Christian

[0] <https://dev.openwrt.org/browser/trunk/toolchain/musl/patches>
[1] <http://git.musl-libc.org/cgit/musl/commit/?id=f9d84554bae0fa17c9a1d724549c4408022228a5>
[2] <https://dev.openwrt.org/browser/trunk/scripts/patch-kernel.sh>

Re: SuperH conflict of arch/sh/__set_thread_area vs thread/__set_thread_area

[Rich Felker]

On Fri, Aug 21, 2015 at 04:07:39PM +0200, Christian Lamparter wrote:
> (Added Openwrt-dev - since this isn't a musl issue)
> 
> On Thursday, August 20, 2015 12:21:55 PM Christian Lamparter wrote:
> > On Wednesday, August 19, 2015 11:04:02 PM Rich Felker wrote:
> > > On Thu, Aug 20, 2015 at 02:44:11AM +0200, Christian Lamparter wrote:
> > > > I'm trying to add a port for a SH4-like ARCH to OpenWRT, which uses the latest
> > > > musl-1.1.10 as the default libc. I'm having the following problem when building
> > > > the toolchain:
> > > > 
> > > > During the final linker-step, the symbol "__set_thread_area"  declared twice.
> > > > This is because the SH architecture provides a separate __set_thread_area [0],
> > > > (other archs use the standard syscall wrapper from [1]).
> > > > 
> > > > Obviously, I want this issue fixed. However I'm new to SuperH and musl, that's
> > > > why I need advise :-D. For now, I defined the src/thread/__set_thread_area as
> > > > a weak symbol. Now, that's just a crude hack, what would be better solution?
> > > > (I can make and post the patch if necessary - But sadly, I can't test it on the
> > > > hardware yet)?
> > > 
> > > Bobby Bingham's reply explains what the issue is. Did you make a new
> > > arch name rather than using the existing sh arch for your port? 
> > Initially, yes I did. I had the ARCH at "sh4". This was because OpenWRT 
> > already had infrastructure for some sh-(sub)targets (sh3, sh3eb, sh4, sh4eb)
> > in place. But they seem to be unused and untested. The only target which has
> > support for SuperH is UserModeLinux. However, it will probably run into the
> > same issue.
> > 
> > Now, I've changed ARCH to "sh" and set the CPU_TYPE to sh4 [toolchain
> > dir changed to toolchain-sh_sh4_gcc-5.2.0_musl-1.1.10]. But still no 
> > luck, the original error code remains the same.
> > 
> > src/thread/__set_thread_area.lo: In function `__set_thread_area':
> > __set_thread_area.c:(.text+0x0): multiple definition of `__set_thread_area'
> > arch/sh/src/__set_thread_area.lo:__set_thread_area.c:(.text+0x0): first defined here
> > collect2: error: ld returned 1 exit status
> > Makefile:142: recipe for target 'lib/libc.so' failed
> 
> Ok, I have a update and I fixed this issue with __set_thread_area.s.
> The reason why this didn't work and was harder to debug than usual is
> because of the involvement of the "patch" utility with the -E flag
> (--remove-empty-files).
> 
> Openwrt adds a number of patches to the musl-1.1.10.tar.gz [0] (actually
> they seem to be cherry-picked from musl git). Now, one of the patches 
> "001-git-2015-07-22.patch" contains:
> 
> commit f9d84554bae0fa17c9a1d724549c4408022228a5 [1]
> Author: Rich Felker <dalias@aerifal.cx>
> Date:   Tue Jun 16 14:28:30 2015 +0000
> 
>     add support for sh2 interrupt-masking-based atomics to sh port
>  
> This commit reduces the "src/thread/sh/__set_thread_area.s" to an empty
> file (but it doesn't remove it). But due to bad luck, the script that
> patches the file [2] (it's called patch-kernel.sh, but it's used to patch
> the toolchain as well) uses the pesky "-E" flag (line 40):
>  [...] ${PATCH:-patch} -f -p1 -E -d ${targetdir} 
>                               ^^
> 
> ==> And that's why the __set_thread_area.s file disappears.
> 
> I came up with two possible fixes for OpenWRT, unless you want to
> fix this in musl (via non-empty __set_thread_area) instead: 

There's a third possible fix: Just waiting for that patch to be
dropped once they're using a base release of musl that already has it
upstream. However I suspect this is an issue that could come up again,
since IIRC musl has a number of empty files in the source tree and we
sometimes add more, so my leaning would be to get OpenWRT to fix it.

> 1. have some content in __set_thread_area.s to prevent it from being
>    removed.
> 
> ---
> 
> --- a/src/thread/sh/__set_thread_area.s
> +++ b/src/thread/sh/__set_thread_area.s
> @@ -0,0 +1,1 @@
> +/* the patch utility might remove empty files */
> 
> ---

This seems like a non-solution since the problem is not the code
that's in musl (that would be fine) but a patch OpenWRT is shipping
based on commit to musl. The latter patch is what would need to be
fixed, and it could be fixed manually if this approach is desired, but
I think that's a bad idea because the issue might come up again in the
future with other files.

> 2. patch patch-kernel.sh
> 
> ---
> --- a/scripts/patch-kernel.sh
> +++ b/scripts/patch-kernel.sh
> @@ -37,7 +37,7 @@ for i in ${patchdir}/${patchpattern} ; do
>      [ -d "${i}" ] && echo "Ignoring subdirectory ${i}" && continue	
>      echo ""
>      echo "Applying ${i} using ${type}: " 
> -    ${uncomp} ${i} | ${PATCH:-patch} -f -p1 -E -d ${targetdir} 
> +    ${uncomp} ${i} | ${PATCH:-patch} -f -p1 -d ${targetdir}
>      if [ $? != 0 ] ; then
>          echo "Patch failed!  Please fix $i!"
>  	exit 1
> 
> ---
> 
> Does anyone have any preferences? I'm opting for "patch patch-kernel"
> (and I will send a proper patch next week to OpenWRT even if no one
> cares now :-D ) 

This seems like a better approach to me, but I don't know if it would
adversely affect the kernel or other components.

Rich

Re: SuperH conflict of arch/sh/__set_thread_area vs thread/__set_thread_area

[Felix Fietkau]

On 2015-08-21 16:23, Rich Felker wrote:
>> 2. patch patch-kernel.sh
>> 
>> ---
>> --- a/scripts/patch-kernel.sh
>> +++ b/scripts/patch-kernel.sh
>> @@ -37,7 +37,7 @@ for i in ${patchdir}/${patchpattern} ; do
>>      [ -d "${i}" ] && echo "Ignoring subdirectory ${i}" && continue	
>>      echo ""
>>      echo "Applying ${i} using ${type}: " 
>> -    ${uncomp} ${i} | ${PATCH:-patch} -f -p1 -E -d ${targetdir} 
>> +    ${uncomp} ${i} | ${PATCH:-patch} -f -p1 -d ${targetdir}
>>      if [ $? != 0 ] ; then
>>          echo "Patch failed!  Please fix $i!"
>>  	exit 1
>> 
>> ---
>> 
>> Does anyone have any preferences? I'm opting for "patch patch-kernel"
>> (and I will send a proper patch next week to OpenWRT even if no one
>> cares now :-D ) 
> 
> This seems like a better approach to me, but I don't know if it would
> adversely affect the kernel or other components.
Applied this change in r46723 - I don't expect any side effects in the
kernel or other components, since this flag seems to be relevant for
non-standard patch formats only, and we usually keep our patches consistent.

- Felix