std/channels is slower than system/channels_builtin or hangs (EDIT: gives EXC_BAD_ACCESS) #17392
Comments
when true: # D20210316T000718
import std/times
when defined(newChan):
import std/channels
import std/[isolation]
var
sender: array[15, Thread[void]]
receiver: array[15, Thread[void]]
let maxItems = 1000
const sentinel = @["end"] # xxx maybe also show a solution with `Atomic` or mutex
proc getPayload(): string =
for i in 0..<10:
result.add $i
const payload = getPayload()
when defined(newChan):
var chan = newChannel[seq[string]](maxItems)
proc recvHandler() =
var x: Isolated[seq[string]]
while true:
x = chan.recvIso()
if x.extract == sentinel: break
elif defined(oldChan):
import std/[isolation]
var chan: Channel[seq[string]]
chan.open(maxItems = maxItems)
proc recvHandler() =
var x: Isolated[seq[string]]
while true:
x = isolate(chan.recv())
if x.extract == sentinel: break
else: static: doAssert false
let n = 1000
proc sendHandler() =
for i in 0..<n:
chan.send(@[payload])
chan.send(sentinel)
var tot = 0.0
proc benchmark() =
for t in mitems(sender):
t.createThread(sendHandler)
for t in mitems(receiver):
t.createThread(recvHandler)
let t = epochTime()
joinThreads(sender)
joinThreads(receiver)
let t2 = epochTime()
tot += t2 - t
for i in 0..100:
benchmark()
echo totOld channels also leak. |
This comment has been minimized.
This comment has been minimized.
Closed
|
Edited: [First example] when defined case10a2: # D20210317T164951:here gitissue
import std/times
when defined(newChan):
import std/[channels, isolation]
var
sender: array[15, Thread[void]]
receiver: array[15, Thread[void]]
let maxItems = 100
const sentinel = @["end"]
const payload = newString(5000)
when defined(newChan):
var chan = newChannel[seq[string]](maxItems)
proc recvHandler() =
while true:
let x = chan.recv()
if x == sentinel: break
elif defined(oldChan):
var chan: Channel[seq[string]]
chan.open(maxItems = maxItems)
proc recvHandler() =
var x: seq[string]
while true:
x = chan.recv()
if x == sentinel: break
else: static: doAssert false
let n = 30
proc sendHandler() =
for i in 0..<n:
chan.send(@[payload])
chan.send(sentinel)
var tot = 0.0
proc benchmark() =
for t in mitems(sender):
t.createThread(sendHandler)
for t in mitems(receiver):
t.createThread(recvHandler)
let t = epochTime()
joinThreads(sender)
joinThreads(receiver)
let t2 = epochTime()
tot += t2 - t
for i in 0..100:
benchmark()
echo totnim c --threads -d:newChan --gc:arc -d:danger -d:case10a2 -o:/tmp/z010a2 -g $timn_D/tests/nim/all/t12015.nim for i in {1..200}; do /tmp/z010a2 ; done bt for all threads: (previously i was only showing main thread, but now using |
|
Edited: [Second example] and in this example, contrary to previous one, the hang happens with both oldChan and newChan; when defined case10c: # D20210317T170408
#[
nim r --threads -d:newChan --gc:arc -d:danger -d:case10c -o:/tmp/z010c2 -g $timn_D/tests/nim/all/t12015.nim
]#
import std/times
when defined(newChan):
import std/[channels, isolation]
var
sender: array[15, Thread[void]]
receiver: array[15, Thread[void]]
let maxItems = 3
const sentinel = @["end"] # xxx maybe also show a solution with `Atomic` or mutex
const payload = newString(5000)
when defined(newChan):
var chan = newChannel[seq[string]](maxItems)
proc recvHandler() =
var x: seq[string]
while true:
x = chan.recv()
if x == sentinel: break
elif defined(oldChan):
var chan: Channel[seq[string]]
chan.open(maxItems = maxItems)
proc recvHandler() =
var x: seq[string]
while true:
x = chan.recv()
if x == sentinel: break
else: static: doAssert false
let n = 10
proc sendHandler() =
for i in 0..<n:
chan.send(@[payload])
chan.send(sentinel)
var tot = 0.0
proc benchmark() =
for t in mitems(sender):
t.createThread(sendHandler)
for t in mitems(receiver):
t.createThread(recvHandler)
let t = epochTime()
joinThreads(sender)
joinThreads(receiver)
let t2 = epochTime()
tot += t2 - t
for i in 0..5000:
echo i
benchmark()
echo totit's simpler to reproduce the bug in this case: nim r --threads -d:oldChan --gc:arc -d:danger -d:case10c -g $timn_D/tests/nim/all/t12015.nim nim r --threads -d:newChan --gc:arc -d:danger -d:case10c -g $timn_D/tests/nim/all/t12015.nim (EDIT) without -d:dangerthis is more informative as to explaining the deadlock: it shows 2 interesting things:
|
|
here's more info (see above for code #17392 (comment)) nim -v: 4bfc5a9 nim c --threads -d:newChan --stacktrace:off --gc:arc -d:case10c -g $timn_D/tests/nim/all/t12015.nim lldb -o run /Users/timothee/git_clone/nim/timn/tests/nim/all/t12015 |
timotheecour
changed the title
|
First example doesn't hang on my Linux machine So does the second example Maybe it is arch specific? |
that's possible; I'm using osx 10.15.7, just retested "first example" now as of nim 2365b52 and it's the same as before, it hangs within < 10s. |
|
Yeah, I tested it on ubuntu 20.04(it is the host system) (single core) |
|
after #17717:
(note: i've in the meantime upgraded osx to big sur (11.2.3) from 10.15 but that shouldn't be related) @xflywind can you reproduce the timing difference i'm noting for |
|
Well ... can you find out why it is slower? :-) |
|
update: after trying #17884, nothing improved compared to what i wrote in #17392 (comment), and also there was 1 case where |
For me, it is about 3x slower with
Neither did for me the first time I ran it. But on repeated runs, sometimes it does run fine and sometimes it hangs at some random point (not the same one every time). See if you can reproduce it with multiple runs. |
what frequency of the runs exhibits the hang, and for what nim commit? as of 8ce69d5 i can't reproduce the hang after running but i still observe the ~4x slowdown with -d:newChan (and the other points from #17392 (comment) still apply) |
I was on ~30 commits old compiler. Now I updated to the latest commit (
Out of 10 times I have run it, the hang happened 4 times. (And it is still ~3x slower) |
|
this is potentially related to #17946 |
timotheecour
changed the title
|
Ref #18801 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Example
adapted from benchmark from nim-lang/website#274 (found while attempting to improve the example in nim-lang/website#274)
Current Output
nim c --threads --gc:arc --lib:lib -d:oldChan -d:danger -o:/tmp/z01 main.nim
nim c --threads --gc:arc --lib:lib -d:newChan -d:danger -o:/tmp/z02 main.nim
for i in {1..2000}; do /tmp/z01 ; done
0.5778560638427734
0.4162607192993164
0.4190213680267334
0.6117517948150635
0.4808053970336914
0.4466562271118164
0.4234135150909424
etc... (doesn't hang)
for i in {1..2000}; do /tmp/z02 ; done
1.274321317672729
1.335514068603516
1.379576206207275
1.460080862045288
1.465322256088257
1.588825702667236
1.528374195098877
1.564008951187134
1.734921455383301
1.611903429031372
1.667627096176147
(hangs here...)
maxItemsnote 1
some debug info for the hang:
note 2
probably not related to the performance difference or the hang of std/channels, but where are resources deallocated for
createThread?all I see is a disabled proc with an suspicious comment:
Additional Information
1.5.1 d5eb658
The text was updated successfully, but these errors were encountered: