Filenames with extended characters are not handled well #406
Comments
|
Hello @vaniwaar, thanks for reporting this issue and welcome to drive! |
|
Ping! |
|
Sorry for the delayed response. vaniwaar@storage ~ $ drive version |
|
Hmm, you can properly bundle the build info by running $ go get github.com/odeke-em/drive/drive-gen && drive-genFor sure, please mail the reproducing cases to me and I'll check them out at the end of this week. |
|
Just sent a tar file reproducing this situation. --Ron |
|
Hey @vaniwaar, I got sometime and I've tried to replicate your issue but in vain as you'll see in this YouTube video Is there a way you could reliably reproduce this issue? Thanks. |
|
I will give it a go when I get home. FWIW, the machine on which I am running this is Linux Mint 17. --Ron |
|
Video of the error occurring for me is at https://drive.google.com/file/d/0B4ia4frohAd6WEUyTnJ3cG53LUU/view?usp=sharing This is on a Linux box: Linux version 3.13.0-37-generic (buildd@kapok) (gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1) ) #64-Ubuntu SMP Mon Sep 22 21:28:38 UTC 2014 |
|
Didn't mean to close this... |
|
The other problem is that you've never run $ go get github.com/odeke-em/drive/drive-gen && drive-genlike I had asked in comment #406 (comment). Also you are running code that is more than a month old yet I don't have the commit tag that I had requested for. $ go get -u -v github.com/odeke-em/drive/drive-gen && drive-genthen ping me and retry that example. |
|
Updated and now have 0.3.1. Tried your command and it crashes: vaniwaar@storage ~/gdrive/testing $ drive version goroutine 1 [select]: goroutine 3 [syscall]: goroutine 8 [finalizer wait]: |
|
That's a gcc-go issue that you could file against the Go project if you'd like. However, try re-running it as it could be a bunch of factors. |
|
Fixed the crash problem. Now it won't complete due to: vaniwaar@storage ~ $ go get -u -v github.com/odeke-em/drive/drive-gen && drive-gengit.luolix.top/odeke-em/drive (download) I c++ all day long but I don't know the first thing about go... --Ron |
|
I once encountered that and after dealing with gcc-go on Ubuntu plus other issues with it, I decided to install gc go from the official Go downloads page https://golang.org/doc/install and not gccgo. Worked for me, feel free to try it. |
|
Getting closer. Now the last few lines are: github.com/odeke-em/ripper (download) If you would like, I could give you an account on my machine and you could just work from there if it is easier... --Ron |
|
No thanks, that's alright. $ go get github.com/odeke-em/drive/drive-genShould run if your gopath is properly set. |
|
Ok, I got things going and the problem is still exactly as it was before: vaniwaar@storage ~/gdrive/testing $ drive version
|
|
Hello again @vaniwaar, my mid term exams are finally done and I've gotten a little free time. To test out if this issue is a problem with your setup, golang version, OS packages that aid in string comparison etc, would you mind running this program at https://github.com/odeke-em/go-utils/blob/master/strcmp-check/main.go $ go get github.com/odeke-em/go-utils/strcmp-check
$ echo <trippy_filename> | strcmp-check
$ # or
$ cat filenames.txt | strcmp-checkif you see a message like "panic" then it is a problem with your Go version, but please give me information on what the versions are before re-installing etc. Thanks. |
|
Hope your midterms went well! I don't get any output: vaniwaar@storage ~ $ echo gdrive/testing/foo/98\ Boléro\ (from\ 10).mp3 | strcmp-check |
|
vaniwaar@storage ~ $ go version |
|
How about? $ echo "98 Boléro (from 10).mp3" | strcmp-check |
|
vaniwaar@storage ~ $ echo "98 Boléro (from 10).mp3" | strcmp-check |
|
Edit: Hello @vaniwaar I think @colinkeenan might have diagnosed your problem in issue #472 and he made a wiki entry here https://github.com/odeke-em/drive/wiki |
|
I have updated the wiki (https://github.com/odeke-em/drive/wiki#linux-terminal-character-encoding) with more information about setting |
|
Thanks again @colinkeenan, in that case I'll remove the screenshot I attached to avoid any confusion so that users can then go directly to the link you've shared. |
|
@vaniwaar ping! |
|
Doesn't fix the problem... I have LANG set to en_US.UTF-8 but it still exhibits the same problem. --Ron |
|
@colinkeenan, I missed that instruction. Thanks! --Ron |
|
@vaniwaar good point. @colinkeenan good point, we could add a sanity check at the beginning. |
|
Hey folks, I was wondering if y'all could try out the output of this https://gist.github.com/odeke-em/3f8eabb5547168fd402b and see if it catches your invalid locale. If so, we can adapt this code into drive to help catch this cases. |
|
It was in an inconvenient format because I assume I was not to include the "$" and ">" at the beginning of each line. I copy and pasted it line by line leaving that first character off. I got an error though: |
|
Ooops my bad, let me it in the gist, please see https://gist.github.com/odeke-em/3f8eabb5547168fd402b#file-main-go. :%s/^> / /gand it will strip them out |
|
Yeah please properly setup your $GOPATH as shown here https://github.com/odeke-em/drive#requirements |
|
I had already done that the first time I reported an issue. Those paths are defined already: |
|
And you aren't able to build that example? To just get the example running in your case https://gist.github.com/odeke-em/3f8eabb5547168fd402b#file-build-alt-md |
|
|
|
Please make sure you have the content of the gist https://gist.github.com/odeke-em/3f8eabb5547168fd402b#file-main-go pasted into main.go. |
|
I started over and went slowly. And, of course, on the |
|
I tried |
|
No worries. I'll restep you through it |
|
I made a little progress. Not knowing anything about As you can see, now the error message changed to |
|
Please check out the comments I made in #406 (comment). |
|
I have been following those instructions correctly from the beginning. That was what generated the original error. Please look at what I have now done and see if you can understand what is wrong or different with my |
|
I have had the correct |
|
There isn't a single step in #406 (comment) that has |
|
What did you intend me to do with the last step of that comment? The last step is: c) perform the step in https://gist.github.com/odeke-em/3f8eabb5547168fd402b#file-build-alt-md And, when I click that link, it just takes me to the following code:
Which never worked because nothing was got from the get command and also makes no sense since the previous steps already obtained Now, since I've had |
|
Well, although I don't understand it, I went ahead and ran
after already having |
|
Alright, if you go get asciinema/asciinema/util then you can run the content of main.go simply by
|
|
Yes so finally that run it right, your result in #406 (comment) is it running alright, so your locale is set, all good then. |
|
Ok. I just ran it and it worked (i.e. no output). What does it mean. I thought the point was to generate some sort of error about the locale or something. |
|
Not really, the point of that code was to crash and tell the user that their locale was not properly setup which is what will happen in the future if this code is incorporated. This way drive can't be run until they go through the setup that you provided in the wiki as well as the conversation in this issue. This will address the concerns raised in #406 (comment) and #406 (comment) as this issue is detectable. |
|
Well, not quite though. I believe he had his environment set to utf8 from the beginning. The problem was that the filenames were not encoded the same way. I thought he could set his locale to match the filename encoding and it would work, but he could not get that to work. However, simply converting the filenames to utf8 does work. I think this utility will help, but you still need to detect whether or not the uploaded filename matches the local filename of the file just uploaded. It won't match if the filename encoding is not utf8 when the locale is utf8. |
|
Ah I see, thanks. Do you think there is a way we could reliably detect this at startup time ie the discrepancy between the encoding? Would it be a good approach to take? Or do we just have to incur a comparison after every upload? Also there is a step before upload in trying to match files up, this will fail in any case if the locale and encoding discrepancies exist. |
|
Somehow it was uploading the file with the ? in the filename. If you can stop the upload from happening just by seeing that the filename has changed, then that would be enough. That's what happens when done in the browser. The error message isn't helpful at all though. It was something like "server error". |
|
vaniwaar@storage ~ $ cd locale-test/ |
|
@vaniwaar you should probably put your locale back to utf-8 since the attempted switch to iso-8859-15 didn't solve your problem. At least you confirmed this test works. I think he is planning to add it to drive and stop drive from working in non utf-8 environments, so you will need to put yours back to utf-8. You can run this script to verify you were successful. |
|
This seems to be happening with the mere presence of spaces in the file name. On some of my files with spaces (not all of them) the "drive push/pull" always tries to add/remove the files. By going to Drive's web console and removing the spaces (e.g. replacing with underscores) it works well going forward. Just rename back to have spaces and problem returns. I tried running with -v but no useful data showed. How to enable debug to send to you? |
I have a bunch of music files with extended ascii characters in their names. When I try to push these, drive does not retain the name properly:
vaniwaar@storage ~/gdrive $ drive push -r foo
Resolving...
Addition count 2
Proceed with the changes? [Y/n]:
vaniwaar@storage ~/gdrive $ drive push -r foo
Resolving...
Deletion count 1
Addition count 1
Proceed with the changes? [Y/n]:n
vaniwaar@storage ~/gdrive $ ls foo
98 Bol?ro (from 10).mp3
vaniwaar@storage ~/gdrive $
Unfortunately, I can't attach the tar file with this example but I can send it to someone if requested.
--Ron
The text was updated successfully, but these errors were encountered: