[9fans] coherence function in kernel (especially in raspberry pi port)

[9fans] coherence function in kernel (especially in raspberry pi port)

[yoann padioleau]

Hi,

What is exactly the logic of the calls to coherence?
When do we need to call coherence in the kernel?
For example I see this code in bcm/taslock.c

	coherence();
	l->key = 0;
	coherence();

bcm/taslock.c seems actually mostly a copy paste of port/taslock.c
with an extra call to coherence before the assignment above.

Also, in the code of coherence for ARM,
what is the meaning of the instructions ISB and DSB?
The I and D seems to corrrespond to instruction vs data cache,
but I fail to understand the meaning of S and B.

Re: [9fans] coherence function in kernel (especially in raspberry pi

[Richard Miller]

Have a look at the ARM document "Barrier Litmus Tests and Cookbook",
and especially section 7.2 "Acquiring and Releasing a Lock".

After reading this document, I came to the conclusion that the
coherence() call in the unlock() function in port/taslock.c
belongs before zeroing the l->key instead of after it.  I made
the change only for the bcm kernel because I haven't researched
the memory semantics for all the other cpu architectures.  I
think it would probably be safe to make the change in ../port,
but someone else can make that decision.

I think it would also be safe to remove the coherence() call
after zeroing the l->key, but I kept it in for paranoia's sake.

Re: [9fans] coherence function in kernel (especially in raspberry pi port)

[Charles Forsyth]

[-- Attachment #1: Type: text/plain, Size: 159 bytes --]

On 28 March 2017 at 19:21, yoann padioleau <aryx.padator@gmail.com> wrote:

> but I fail to understand the meaning of S and B.


Synchronisation Barrier

[-- Attachment #2: Type: text/html, Size: 446 bytes --]

Re: [9fans] coherence function in kernel (especially in raspberry pi port)

[Charles Forsyth]

[-- Attachment #1: Type: text/plain, Size: 1555 bytes --]

On 28 March 2017 at 19:21, yoann padioleau <aryx.padator@gmail.com> wrote:

> For example I see this code in bcm/taslock.c
>
>         coherence();
>         l->key = 0;
>         coherence();
>
> bcm/taslock.c seems actually mostly a copy paste of port/taslock.c
> with an extra call to coherence before the assignment above.
>

I've got that extra coherence call in port/taslock.c now (although my more
recent kernels use MCS locks exclusively)

You need to look at the full code fragment:

l->p = nil;
l->pc = 0;
l->m = nil;
coherence();
l->key = 0;
coherence();

assume coherence is a suitable barrier instruction on a given machine.
the calls are there to control the ordering (of stores in this case) as
seen by an observer,
which depends on the memory ordering model of a given processor or system
implementation.

the sequence is clearing references in the lock, to the Proc and the Mach
on which it's running.
we don't care about the order in which those stores are observed.
the first barrier ensures that all are observable before the critical store
to key that releases the Lock.

the second barrier ensures that once the lock is free, no further store can
be observed before
the lock-clearing store has been observed.

there is a general assumption in the system that any multiprocessor running
these kernels will have cache-coherency enabled across processors, although
that need not extend to devices (where the device drivers can do the little
dances needed to ensure coherent views of memory).

[-- Attachment #2: Type: text/html, Size: 2826 bytes --]