Windows: ipfs -r <path> behaves inconsistent and can crash

[jurgen-kluft]

The current binary on Windows does not recursively add directories, also when executing the same operation multiple times the number of files added is not consistent and ipfs can crash:

d:\Programs>ipfs add -r BC3
159.00 KB / 11.45 MB [===>---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 1.36 % 0←
added QmVHxkjE99HKARHmuiBhJNd8dkoBhshrGN7Z8eL7sUsukP BC3\7zxa.dll
159.37 KB / 11.45 MB [===>---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 1.36 % 0←
added QmYLCwWNTYPR2BXhjQ3Wdb9W28ey4QiWVSPiYQEnttmr5Z BC3\BC3Key.txt
160.31 KB / 11.45 MB [===>---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 1.37 % 0←
added QmTjcHKiyDP6omPtU6mvwHx4Xkzg9iSwMsYDpNSLQHSvod BC3\BCCommands.xml
160.99 KB / 11.45 MB [===>---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 1.37 % 0←
added QmYyAix11qNWBiy5QVwVorB3us8udSts7AFKWc8HpKWoXj BC3\BCCommands.xml.bak
164.35 KB / 11.45 MB [===>---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 1.40 % 0←
added QmR7iLdvHbGUoAXk69TV47GYHNxXu4EnXAeVxDtZXBiLBM BC3\BCSessions.xml
167.82 KB / 11.45 MB [===>---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 1.43 % 0←
added QmXVU28oPa3zL78jtK2h2dZ9XErTMjdmLs93fu6pMaHR3o BC3\BCSessions.xml.bak
296.32 KB / 11.45 MB [=====>-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 2.53 % 0←
added QmW8jZohtFiZSqE2rNbE1ju6XcYBGoDnZfm5iV1QUc9s5z BC3\BCShellEx.dll
676.80 KB / 11.45 MB [============>------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 5.77 % 0←
added QmbQzpby7cJt1oQHBSDcAydPz32HePFLYBL1YjWn5VYC3P BC3\BCShellEx64.dll
683.32 KB / 11.45 MB [============>------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 5.83 % 0←
added QmPjFVSKg7dK51RHqQj8dbPvfPKsnAiCxUMDJZN2tExoUg BC3\BCState.xml
689.84 KB / 11.45 MB [=============>-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 5.88 % 0←
added QmXsfKfFyN7whzArGFcZyMewPNnnLEWHXezowdZe6RkbJo BC3\BCState.xml.bak
780.84 KB / 11.45 MB [==============>----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 6.66 % 0←
added QmeNTpHYSoGBuK2Mh74URFA8fwhzhkiCRUvY4tnisTAV9V BC3\BComp.com
867.84 KB / 11.45 MB [================>--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 7.40 % 0←
added QmcYaQ6ezMbFRYz7wLqc9dzBHWZ6B1CUAbrY9ZkDpibWSS BC3\BComp.exe
Exception 0xe06d7363 0x19930520 0x6f9c0 0x7fefd70b3dd
PC=0x7fefd70b3dd

syscall.Syscall9(0x7fefeed13b0, 0x7, 0x224, 0xc0822368b0, 0x1, 0xc082236898, 0x0, 0xc082236868, 0x0, 0x0, ...)
/usr/local/go/src/runtime/syscall_windows.go:150 +0x5c
syscall.WSASend(0x224, 0xc0822368b0, 0x80000000001, 0xc082236898, 0x0, 0xc082236868, 0x0, 0x0, 0x0)
/usr/local/go/src/syscall/zsyscall_windows.go:1529 +0xc4
net.(netFD).Write.func1(0xc082236868, 0x0, 0x0)
/usr/local/go/src/net/fd_windows.go:513 +0x8c
net.(ioSrv).ExecIO(0xc08202a230, 0xc082236868, 0xe12ce8, 0x7, 0xfaf7a8, 0x1000, 0x0, 0x0)
/usr/local/go/src/net/fd_windows.go:160 +0xd5
net.(netFD).Write(0xc082236780, 0xc08224effa, 0x7006, 0x7006, 0x0, 0x0, 0x0)
/usr/local/go/src/net/fd_windows.go:514 +0x181
net.(conn).Write(0xc08202a268, 0xc08224effa, 0x7006, 0x7006, 0x0, 0x0, 0x0)
/usr/local/go/src/net/net.go:184 +0xeb
bufio.(Writer).Write(0xc0822231c0, 0xc08224effa, 0x7006, 0x7006, 0xffa, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:594 +0xd8
net/http/internal.(chunkedWriter).Write(0xc08217df00, 0xc08224e000, 0x8000, 0x8000, 0x8000, 0x0, 0x0)
/usr/local/go/src/net/http/internal/chunked.go:169 +0x1e5
io.copyBuffer(0x1b7370, 0xc08217df00, 0x1b72f0, 0xc082242be0, 0xc08224e000, 0x8000, 0x8000, 0xf98b6, 0x0, 0x0)
/usr/local/go/src/io/io.go:383 +0x2d0
io.Copy(0x1b7370, 0xc08217df00, 0x1b72f0, 0xc082242be0, 0xc08217df00, 0x0, 0x0)
/usr/local/go/src/io/io.go:351 +0x6b
net/http.(transferWriter).WriteBody(0xc0822444d0, 0x1b7318, 0xc08202a270, 0x0, 0x0)
/usr/local/go/src/net/http/transfer.go:218 +0x2b9
net/http.(Request).write(0xc0820caee0, 0x1b6e50, 0xc0822231c0, 0x0, 0xc082246750, 0x0, 0x0)
/usr/local/go/src/net/http/request.go:462 +0xbc0
net/http.(persistConn).writeLoop(0xc0820a9e40)
/usr/local/go/src/net/http/transport.go:1015 +0x283
created by net/http.(Transport).dialConn
/usr/local/go/src/net/http/transport.go:686 +0xca4

goroutine 1 [chan receive]:
github.com/ipfs/go-ipfs/core/commands.glob.func3(0x1b5a30, 0xc0820f4300, 0x1b7000, 0xc0822449a0)
/go/src/github.com/ipfs/go-ipfs/core/commands/add.go:255 +0x4b0
main.callCommand(0x1b5c30, 0xc082222940, 0x1b5a30, 0xc0820f4300, 0x1309b80, 0x12f9e60, 0x0, 0x0, 0x0, 0x0)
/go/src/github.com/ipfs/go-ipfs/cmd/ipfs/main.go:348 +0x85e
main.(*cmdInvocation).Run(0xc0822226c0, 0x1b5c30, 0xc082222940, 0x0, 0x0, 0x0, 0x0)
/go/src/github.com/ipfs/go-ipfs/cmd/ipfs/main.go:186 +0x1ec
main.main()
/go/src/github.com/ipfs/go-ipfs/cmd/ipfs/main.go:151 +0x662

goroutine 6 [chan receive]:
github.com/ipfs/go-ipfs/Godeps/_workspace/src/github.com/codahale/metrics.init.1.func2()
/go/src/github.com/ipfs/go-ipfs/Godeps/_workspace/src/github.com/codahale/metrics/metrics.go:321 +0x7a
created by github.com/ipfs/go-ipfs/Godeps/_workspace/src/github.com/codahale/metrics.init.1
/go/src/github.com/ipfs/go-ipfs/Godeps/_workspace/src/github.com/codahale/metrics/metrics.go:328 +0x82

goroutine 7 [select]:
github.com/ipfs/go-ipfs/vendor/QmQg1J6vikuXF9oDvm4wpdeAUvvkVEKW1EYDw9HhTMnP2b/go-log.(*MirrorWriter).logRoutine(0xc082006ac0)
/go/src/github.com/ipfs/go-ipfs/vendor/QmQg1J6vikuXF9oDvm4wpdeAUvvkVEKW1EYDw9HhTMnP2b/go-log/writer.go:71 +0x343
created by github.com/ipfs/go-ipfs/vendor/QmQg1J6vikuXF9oDvm4wpdeAUvvkVEKW1EYDw9HhTMnP2b/go-log.NewMirrorWriter
/go/src/github.com/ipfs/go-ipfs/vendor/QmQg1J6vikuXF9oDvm4wpdeAUvvkVEKW1EYDw9HhTMnP2b/go-log/writer.go:38 +0xe9

goroutine 8 [syscall]:
os/signal.loop()
/usr/local/go/src/os/signal/signal_unix.go:22 +0x1f
created by os/signal.init.1
/usr/local/go/src/os/signal/signal_unix.go:28 +0x3e

goroutine 15 [chan receive]:
main.(IntrHandler).Handle.func1(0xc082191980, 0xc08217cff0)
/go/src/github.com/ipfs/go-ipfs/cmd/ipfs/main.go:542 +0xa1
created by main.(IntrHandler).Handle
/go/src/github.com/ipfs/go-ipfs/cmd/ipfs/main.go:547 +0x9c

goroutine 40 [IO wait]:
net.runtime_pollWait(0x1b6c58, 0x72, 0xc082257610)
/usr/local/go/src/runtime/netpoll.go:157 +0x67
net.(pollDesc).Wait(0xc0822368f0, 0x72, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:73 +0x41
net.(ioSrv).ExecIO(0xc08202a228, 0xc0822367e0, 0xe12ce0, 0x7, 0xfaf7a0, 0x86, 0x0, 0x0)
/usr/local/go/src/net/fd_windows.go:182 +0x177
net.(netFD).Read(0xc082236780, 0xc08224a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/net/fd_windows.go:463 +0x17e
net.(conn).Read(0xc08202a268, 0xc08224a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/net/net.go:172 +0xeb
net/http.noteEOFReader.Read(0x1b6dd8, 0xc08202a268, 0xc0820a9e98, 0xc08224a000, 0x1000, 0x1000, 0xb67280, 0x0, 0x0)
/usr/local/go/src/net/http/transport.go:1370 +0x6e
net/http.(noteEOFReader).Read(0xc082242b00, 0xc08224a000, 0x1000, 0x1000, 0x5, 0x0, 0x0)
:126 +0xd7
bufio.(Reader).fill(0xc082248120)
/usr/local/go/src/bufio/bufio.go:97 +0x1f0
bufio.(Reader).ReadSlice(0xc082248120, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:328 +0x221
net/http/internal.readLine(0xc082248120, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/http/internal/chunked.go:110 +0x52
net/http/internal.(chunkedReader).beginChunk(0xc082246ab0)
/usr/local/go/src/net/http/internal/chunked.go:47 +0x40
net/http/internal.(chunkedReader).Read(0xc082246ab0, 0xc08226c001, 0x5ff, 0x5ff, 0x0, 0x0, 0x0)
/usr/local/go/src/net/http/internal/chunked.go:77 +0xbe
net/http.(body).readLocked(0xc082223440, 0xc08226c001, 0x5ff, 0x5ff, 0x0, 0x0, 0x0)
/usr/local/go/src/net/http/transfer.go:632 +0x9f
net/http.(body).Read(0xc082223440, 0xc08226c001, 0x5ff, 0x5ff, 0x0, 0x0, 0x0)
/usr/local/go/src/net/http/transfer.go:624 +0x117
net/http.(bodyEOFSignal).Read(0xc082223480, 0xc08226c001, 0x5ff, 0x5ff, 0x0, 0x0, 0x0)
/usr/local/go/src/net/http/transport.go:1296 +0x271
github.com/ipfs/go-ipfs/commands/http.(httpResponseReader).Read(0xc08202a2b0, 0xc08226c001, 0x5ff, 0x5ff, 0xfaa544, 0x0, 0x0)
/go/src/github.com/ipfs/go-ipfs/commands/http/client.go:307 +0x8b
encoding/json.(Decoder).refill(0xc082258000, 0x0, 0x0)
/usr/local/go/src/encoding/json/stream.go:152 +0x28e
encoding/json.(Decoder).readValue(0xc082258000, 0x1, 0x0, 0x0)
/usr/local/go/src/encoding/json/stream.go:128 +0x422
encoding/json.(Decoder).Decode(0xc082258000, 0xb2ba40, 0xc082246ea0, 0x0, 0x0)
/usr/local/go/src/encoding/json/stream.go:57 +0x160
github.com/ipfs/go-ipfs/commands/http.decodeTypedVal(0x1b40a8, 0xd0e720, 0xc082258000, 0x0, 0x0, 0x0, 0x0)
/go/src/github.com/ipfs/go-ipfs/commands/http/client.go:291 +0x10d
github.com/ipfs/go-ipfs/commands/http.readStreamedJson(0x1b5a30, 0xc0820f4300, 0x1b70b0, 0xc08202a2b0, 0xc082248540)
/go/src/github.com/ipfs/go-ipfs/commands/http/client.go:268 +0x1f8
created by github.com/ipfs/go-ipfs/commands/http.getResponse
/go/src/github.com/ipfs/go-ipfs/commands/http/client.go:207 +0x510

goroutine 34 [select]:
net/http.(persistConn).readLoop(0xc0820a9d90)
/usr/local/go/src/net/http/transport.go:976 +0xace
created by net/http.(Transport).dialConn
/usr/local/go/src/net/http/transport.go:685 +0xc7f

goroutine 35 [select]:
net/http.(persistConn).writeLoop(0xc0820a9d90)
/usr/local/go/src/net/http/transport.go:1009 +0x413
created by net/http.(Transport).dialConn
/usr/local/go/src/net/http/transport.go:686 +0xca4

goroutine 38 [select]:
net/http.(persistConn).readLoop(0xc0820a9e40)
/usr/local/go/src/net/http/transport.go:976 +0xace
created by net/http.(Transport).dialConn
/usr/local/go/src/net/http/transport.go:685 +0xc7f
rax 0x5eecd46f
rbx 0x6f9c0
rcx 0x6f250
rdi 0x180069528
rsi 0x3f0c8c0
rbp 0x6f990
rsp 0x6f860
r8 0x0
r9 0x0
r10 0x180000000
r11 0x6f8a0
r12 0x3f0c8c0
r13 0x1
r14 0xc0822368b0
r15 0x224
rip 0x7fefd70b3dd
rflags 0x206
cs 0x33
fs 0x53
gs 0x2b

[djdv]

I submitted a pull request that should fix Windows path issues but it has yet to be merged, could you rerun with this and let me know if you still have issues?
#1933

[whyrusleeping]

yeah, i'd like to see how that PR affects this issue too

[jurgen-kluft]

i am not setup to get a build from source, so someone else has to test this for you.

[djdv]

@jurgen-kluft
You can obtain the latest builds from the buildbot if you'd like:
https://gobuilder.me/github.com/ipfs/go-ipfs/cmd/ipfs?branch=master
Otherwise I could run the tests myself but I'd need the data that's causing this issue, the PR was tested on several cases prior to the merge with success in all cases so it should be fine on the current master.

[jurgen-kluft]

Ok, will test it tomorrow (24 Dec) on the same set of files.

[ottuzzi]

Hi,

I tried ipfs built from today sources using go-1.5.2-win32 and everything seems working as expected.
Let's wait for Jurgen test.

Bye
Piero

[jurgen-kluft]

After downloading the master build (0.3.11-dev) and testing it on the same directory the same issue is still there:

1. Repeatedly calling 'ipfs add -r BC3' gives inconsistent results
2. Repeatedly calling 'ipfs add -r BC3' can crash ipfs

Here is a list of the content of the BC3 directory:
d:\Programs\BC3\Files\

12/22/2015 11:13 ----
d:\Programs\BC3\Helpers\ 12/22/2015 11:07 ----
d:\Programs\BC3\7zxa.dll 162,816 06/17/2008 12:04 -a--
d:\Programs\BC3\BC3Key.txt 377 04/27/2015 09:50 -a--
d:\Programs\BC3\BCCommands.xml 969 08/10/2015 09:48 -a--
d:\Programs\BC3\BCCommands.xml.bak 694 08/10/2015 09:48 -a--
d:\Programs\BC3\BComp.com 93,184 11/21/2008 13:17 -a--
d:\Programs\BC3\BComp.exe 89,088 11/21/2008 13:17 -a--
d:\Programs\BC3\BCompare.chm 1,194,418 04/26/2010 10:38 -a--
d:\Programs\BC3\BCompare.exe 9,064,536 04/26/2010 10:38 -a--
d:\Programs\BC3\BCSessions.xml 3,438 11/06/2015 18:09 -a--
d:\Programs\BC3\BCSessions.xml.bak 3,553 11/06/2015 18:09 -a--
d:\Programs\BC3\BCShellEx.dll 131,584 10/15/2009 10:31 -a--
d:\Programs\BC3\BCShellEx64.dll 389,612 10/15/2009 10:31 -a--
d:\Programs\BC3\BCState.xml 6,676 12/16/2015 09:37 -a--
d:\Programs\BC3\BCState.xml.bak 6,676 12/16/2015 09:37 -a--
d:\Programs\BC3\License.html 8,937 04/12/2010 18:23 -a--
d:\Programs\BC3\Readme.txt 1,559 02/16/2010 16:10 -a--
d:\Programs\BC3\unrar.dll 178,176 07/06/2009 15:33 -a--

The 'Files' directory is empty and the 'Helpers' directory content is this:
d:\Programs\BC3\Helpers\PdfToText.exe 663,552 07/09/2009 11:01 -a--
d:\Programs\BC3\Helpers\XLS_to_CSV_Single.vbs 791 07/15/2009 10:08 -a--

[djdv]

I've mirrored the structure and ran over it several times without issue. My files are just dummies but I don't think that would matter if it was a path related issue. Are there any kind of special files like symlinks, etc.?
I also have to ask that you confirm the old version is not being used when called, you should notice the lack of \'s in the add output, they should be replaced with /'s like in the attached screenshot, if that's not happening you're still calling an old version.

Resulting hash:
Qma7ozABEoa8xAtujW36jor5qQgVdqqF3eE2xJPTiD8MnN

[jurgen-kluft]

Ok, i tested it again and verified that it is version 0.3.11-dev.
It works and i get the same output as you (except for the hashes), i cannot explain however what i did different this morning. This morning i did verified version 0.3.11-dev, added the directory and it failed. The only thing i did different is that i first used the 32-bits version and then the 64-bits version, they both work fine now which is great.

After some fiddling i figured out the difference, i was running without a daemon which i had running this morning. So, to reproduce this; first run 'ipfs daemon' in another command window, then open another command window and run 'ipfs add -r BC3'.

[jurgen-kluft]

This is the output of the daemon (local to our LAN network) (this log is the result of calling several 'ipfs add -r BC3')

d:\Programs\ipfs>ipfs daemon
Initializing daemon...
Swarm listening on /ip4/10.0.14.39/tcp/4001
Swarm listening on /ip4/127.0.0.1/tcp/4001
Swarm listening on /ip4/192.168.56.1/tcp/4001
Swarm listening on /ip4/192.168.99.1/tcp/4001
Swarm listening on /ip6/::1/tcp/4001
API server listening on /ip4/127.0.0.1/tcp/5001
Gateway (readonly) server listening on /ip4/127.0.0.1/tcp/8080
Daemon is ready
←[0;37m12:59:23.920 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: NextPart: multipart: Part Read: read tcp4 127.0.0.1:5001->127.0.0.1:60910: wsarecv: An existing connection was forcibly closed by the remote host. ←[0;37mhandler.go:270←[0m
←[0;37m12:59:25.087 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: unexpected line in Next(): "\x01 u\n" ←[0;37mhandler.go:270←[0m
←[0;37m13:04:30.721 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: NextPart: multipart: Part Read: read tcp4 127.0.0.1:5001->127.0.0.1:61192: wsarecv: An existing connection was forcibly closed by the remote host. ←[0;37mhandler.go:270←[0m
←[0;37m13:04:33.391 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: unexpected line in Next(): "\x00t\x04\xdf?\xeb\x02\xdb?\x9b\xdf\xe0f\xa9\t\x00u\x06\xb0\x01\xeb\x04\xdd\xd81\xc0\x9b\xdb\xe2\xd9m\xf8\x9b\xebV\xac\b\xc0t\x04< t\xf7N\xc31\x
←[0;37mhandler.go:270←[0m
←[0;37m13:04:35.663 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: NextPart: multipart: Part Read: read tcp4 127.0.0.1:5001->127.0.0.1:61200: wsarecv: An existing connection was forcibly closed by the remote host. ←[0;37mhandler.go:270←[0m
←[0;37m13:04:36.545 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: NextPart: multipart: Part Read: read tcp4 127.0.0.1:5001->127.0.0.1:61205: wsarecv: An existing connection was forcibly closed by the remote host. ←[0;37mhandler.go:270←[0m
←[0;37m13:04:37.199 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: NextPart: multipart: Part Read: read tcp4 127.0.0.1:5001->127.0.0.1:61209: wsarecv: An existing connection was forcibly closed by the remote host. ←[0;37mhandler.go:270←[0m
←[0;37m13:04:37.834 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: NextPart: multipart: Part Read: read tcp4 127.0.0.1:5001->127.0.0.1:61214: wsarecv: An existing connection was forcibly closed by the remote host. ←[0;37mhandler.go:270←[0m
←[0;37m13:04:38.436 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: NextPart: multipart: Part Read: read tcp4 127.0.0.1:5001->127.0.0.1:61218: wsarecv: An existing connection was forcibly closed by the remote host. ←[0;37mhandler.go:270←[0m
←[0;37m13:04:39.050 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: NextPart: multipart: Part Read: read tcp4 127.0.0.1:5001->127.0.0.1:61223: wsarecv: An existing connection was forcibly closed by the remote host. ←[0;37mhandler.go:270←[0m
←[0;37m13:04:39.597 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: NextPart: multipart: Part Read: read tcp4 127.0.0.1:5001->127.0.0.1:61227: wsarecv: An existing connection was forcibly closed by the remote host. ←[0;37mhandler.go:270←[0m
←[0;37m13:04:40.250 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: NextPart: multipart: Part Read: read tcp4 127.0.0.1:5001->127.0.0.1:61231: wsarecv: An existing connection was forcibly closed by the remote host. ←[0;37mhandler.go:270←[0m
←[0;37m13:04:40.794 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: NextPart: multipart: Part Read: read tcp4 127.0.0.1:5001->127.0.0.1:61235: wsarecv: An existing connection was forcibly closed by the remote host. ←[0;37mhandler.go:270←[0m
←[0;37m13:04:41.395 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: NextPart: multipart: Part Read: read tcp4 127.0.0.1:5001->127.0.0.1:61239: wsarecv: An existing connection was forcibly closed by the remote host. ←[0;37mhandler.go:270←[0m
←[0;37m13:04:42.684 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: NextPart: multipart: Part Read: read tcp4 127.0.0.1:5001->127.0.0.1:61243: wsarecv: An existing connection was forcibly closed by the remote host. ←[0;37mhandler.go:270←[0m
←[0;37m13:04:43.345 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: NextPart: multipart: Part Read: read tcp4 127.0.0.1:5001->127.0.0.1:61247: wsarecv: An existing connection was forcibly closed by the remote host. ←[0;37mhandler.go:270←[0m
←[0;37m13:04:43.863 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: NextPart: multipart: Part Read: read tcp4 127.0.0.1:5001->127.0.0.1:61254: wsarecv: An existing connection was forcibly closed by the remote host. ←[0;37mhandler.go:270←[0m
←[0;37m13:04:44.411 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: NextPart: multipart: Part Read: read tcp4 127.0.0.1:5001->127.0.0.1:61258: wsarecv: An existing connection was forcibly closed by the remote host. ←[0;37mhandler.go:270←[0m
←[0;37m13:04:45.556 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: NextPart: multipart: Part Read: read tcp4 127.0.0.1:5001->127.0.0.1:61267: wsarecv: An existing connection was forcibly closed by the remote host. ←[0;37mhandler.go:270←[0m
←[0;37m13:04:51.179 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: NextPart: multipart: Part Read: read tcp4 127.0.0.1:5001->127.0.0.1:61278: wsarecv: An existing connection was forcibly closed by the remote host. ←[0;37mhandler.go:270←[0m
←[0;37m13:04:54.720 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: NextPart: multipart: Part Read: read tcp4 127.0.0.1:5001->127.0.0.1:61291: wsarecv: An existing connection was forcibly closed by the remote host. ←[0;37mhandler.go:270←[0m
←[0;37m13:04:56.167 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: NextPart: multipart: Part Read: read tcp4 127.0.0.1:5001->127.0.0.1:61297: wsarecv: An existing connection was forcibly closed by the remote host. ←[0;37mhandler.go:270←[0m
←[0;37m13:05:03.903 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: unexpected line in Next(): "Y\x18\xc7tu`OȽ\x04"\xfc\x9d\x9e\x120GѭpX\xae\xa1\xb8&\xa3YKFᏏY\xec\xf9h\x0f\xf66\xd1\xcd\x034\xb8\x11\xa9\xf4\x01VЂ)?a\n" ←[0;37mhandler.go:270
←[0;37m13:05:06.001 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: NextPart: multipart: Part Read: read tcp4 127.0.0.1:5001->127.0.0.1:61307: wsarecv: An existing connection was forcibly closed by the remote host. ←[0;37mhandler.go:270←[0m
←[0;37m13:05:08.353 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: NextPart: multipart: Part Read: read tcp4 127.0.0.1:5001->127.0.0.1:61312: wsarecv: An existing connection was forcibly closed by the remote host. ←[0;37mhandler.go:270←[0m
←[0;37m13:05:09.312 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: NextPart: multipart: Part Read: read tcp4 127.0.0.1:5001->127.0.0.1:61316: wsarecv: An existing connection was forcibly closed by the remote host. ←[0;37mhandler.go:270←[0m
←[0;37m13:05:10.337 ←[31mERROR ←[0;34mcommands/h: ←[0merr: multipart: unexpected line in Next(): "\x00\f\x05\x03\x00\x02\x06\x00\x00\x00\x00\x04\a\x00\x00\x00\x00\b\t\x00\x00\x00\x00\x01\n" ←[0;37mhandler.go:270←[0m

[djdv]

That's odd, when I ran those tests the daemon was running for me, running again without the daemon results in success again. Perhaps what happened before was that you were running an old instance of the daemon but running a newer version of the client, I'm not sure though.

Either way are the files being added at all and are they intact or does ipfs still crash?
Those errors from the daemon seem to be multipart related not path related. Possible relevant issues: #1688, #2112.

[jurgen-kluft]

The daemon is the same executable as the one that is used with 'ipfs add'.
In the network there are only 2 daemons, one is at 10.0.14.14 the other is at 10.0.14.39.
The 10.0.14.14 is writeable the other one is readonly, they both only have each other in the bootstrap list.
As far as i can see the files are under the .ipfs folder.

Let me know what kind of more tests you want me to do.

[ottuzzi]

Hi,

yesterday I tried without daemon. Today I tried with daemon started and still it works as expected. In your stack trace I can see problems with multipart message and I think there is a problem with go 1.5.1 with them.
I built ipfs with go-1.5.2-win32 and everything seems ok even after adding 15GB worth of data.
If you do not fear to run my own built executable you can download it from [1].

Bye
Piero

[1]https://drive.google.com/file/d/0B7Wwt7isWomtT1BoeV9CVWdaMXM/view?usp=sharing

[djdv]

@ottuzzi
Just for reference the master branch binaries from the buildbot are currently being built with Go 1.5-2, only the ipfs 3.10 releases are being built with Go 1.5.1.

@jurgen-kluft
I'm not actually sure myself, I'd wait to see what @whyrusleeping says in #2112 or here.
(Paging for guidance/awareness)

[jurgen-kluft]

@ottuzzi
I have done the same tests with your executable and i get the same errors (multipart: NextPart)

[djdv]

Closing this due to age, things are better now.