Skip to content
This repository has been archived by the owner on Nov 9, 2017. It is now read-only.

git status always show random files are modified on Win7-x64/SSD driver. #229

Closed
lygstate opened this issue Aug 8, 2014 · 45 comments
Closed
Labels

Comments

@lygstate
Copy link

lygstate commented Aug 8, 2014

I reset the whole source tree everytime, and even delete the whole folder, and clone the new one
It's always should some file are modified, but the fact, it doesn't,
That's very annoy,

  (使用 "git add <file>..." 更新要提交的内容)
  (使用 "git checkout -- <file>..." 丢弃工作区的改动)

        修改:     xpcom/sample/nsSample.manifest
        修改:     xpcom/tests/unit/bug725015.manifest
        修改:     xpcom/tests/unit/data/SmallApp.app/Contents/Resources/English.lproj/MainMenu.nib/keyedobjects.nib
        修改:     xpcom/tests/unit/data/bug121341.properties
        修改:     xpcom/tests/unit/data/iniparser02.ini
        修改:     xpcom/tests/unit/data/iniparser03.ini
        修改:     xpcom/tests/unit/data/iniparser04.ini
        修改:     xpcom/tests/unit/data/iniparser05.ini
        修改:     xpcom/tests/unit/data/iniparser06.ini
        修改:     xpcom/tests/unit/data/iniparser07.ini
        修改:     xpcom/tests/unit/data/iniparser08.ini
        修改:     xpcom/tests/unit/data/iniparser09.ini
        修改:     xpcom/tests/unit/data/iniparser10.ini
        修改:     xpcom/tests/unit/data/iniparser11.ini
        修改:     xpcom/tests/unit/data/iniparser12.ini
        修改:     xpcom/tests/unit/data/iniparser13.ini
        修改:     xpcom/tests/unit/data/iniparser14.ini
        修改:     xpcom/tests/unit/data/iniparser15.ini
        修改:     xpcom/tests/unit/data/presentation.key/thumbs/st0.tiff
        修改:     xpcom/tests/unit/test_bug725015.js
        修改:     xpcom/threads/HangMonitor.h
        修改:     xulrunner/app/install_app.py
        修改:     xulrunner/installer/windows/Header.bmp


@t-b
Copy link

t-b commented Aug 14, 2014

Well I can't reproduce that here.
Does it make a difference if you have the repository on the ssd or not?

@t-b t-b added the WorksForMe label Sep 2, 2014
@DarkNami
Copy link

Maybe it is 'line endings' issue ?

Reference:

@haimgel
Copy link

haimgel commented Nov 4, 2014

I have a very similar problem: after "git clone", different set of files is reported as "modified". This is related to CRLF conversion for sure -- all files in question are defined in .gitattributes, and some of them may have been comitted with CRLF endings.

The real problem, as I see it, is that the results are inconsistent. After, say, 10 runs of "git clone" and "git status", about 25% reports are "clean", and the rest report some files as changed. But the number of files changes from run to run. Really confusing.

This is not related to SSDs, at least for me. Happens on SSDs and on normal hard disks. I do have Windows 7 x64, though.

@linquize
Copy link

linquize commented Nov 5, 2014

Does it happen in v1.9.4.msysgit.2?

@haimgel
Copy link

haimgel commented Nov 5, 2014

Yes, I'm using the latest version.

@YueLinHo
Copy link

YueLinHo commented Nov 5, 2014

@hageltech
What happen if using git clone with --no-checkout (-n) option and then perform the checkout after the clone finished?

@haimgel
Copy link

haimgel commented Nov 5, 2014

I tried with -n and then did checkout after the clone finished. Still the same problem. Some checkouts were dirty, some were not. Moreover, I did one more test: did one "checkout -n", then copied the bare repository several times using simple filesystem copy, and did checkout in each of them. Some were dirty, some were not, again. So the problem seems to be in the checkout, not in clone.

@YueLinHo
Copy link

YueLinHo commented Nov 5, 2014

Perhaps, try to locate one modified file and use git check-attr -a <that_file_pathname>

@haimgel
Copy link

haimgel commented Nov 5, 2014

Did in both trees (where the file is ok and where it is reported as modified). Both return the same:
EmailTestDlg.dfm: text: set
EmailTestDlg.dfm: eol: crlf

This is what I have in .gitattributes for this extension:
*.dfm text eol=crlf

This is what I have in gitconfig (in core section, related to this issue, possibly):
autocrlf = true
eol = crlf

@YueLinHo
Copy link

YueLinHo commented Nov 5, 2014

Then, what is the EOL of that file actually? LF?

@haimgel
Copy link

haimgel commented Nov 5, 2014

Nope, that's the weird thing: both cases, file is correctly CRLF in the working directory. However, it is shown as "modified" sometimes, and when it does, I cannot make that status go away by any means I know (.e.g git reset --hard, etc.)

@YueLinHo
Copy link

YueLinHo commented Nov 5, 2014

Could you please try to create a dummy problematic repository?

@kusma
Copy link
Member

kusma commented Nov 5, 2014

@hageltech: Do you have a virus scanner running in the background?

@haimgel
Copy link

haimgel commented Nov 5, 2014

@kusma: I have Microsoft Security Essentials running... I have hard time believing it is to blame, though.
@YueLinHo: I'll try doing that later today.

@haimgel
Copy link

haimgel commented Nov 5, 2014

@YueLinHo: OK, I couldn't cause this issue to appear with very small dummy testing repositories. However, I did this:

I took one existing repository I have no problem sharing (it's open source), added my .gitattributes file there, and the issue started to appear. So:

Clone this repo:
git clone https://github.com/hageltech/msgit_test.git

Then clone the repo locally several times (at least 10 times to be sure):
git clone msgit_test 1
git clone msgit_test 2
git clone msgit_test 3
...

See how "git status | wc -l" reports different number of files as changed in "1", "2", ...

I just run these tests in a clean Windows 7 in a VM, and the issue is there, so it's not something else on my machine that is interfering.

@YueLinHo
Copy link

YueLinHo commented Nov 6, 2014

@hageltech

I tested it, it happened at 4th clone.
Then, I review the last commit 371f51f of testing repo.
Seems forget to do the normalization, so I re-do it by these steps:

  1. Add .gitattributes
  2. Delete all files execpte the .gitattributes
  3. Revert all deleted files
  4. commit .gitattributes and all normalized files

See https://github.com/YueLinHo/msgit_test
Then, re-test it for 20 times. no problem anymore.

As @DarkNami mention:
see Dealing with line endings
Especially the Refreshing a repository after changing line endings part.

@lygstate
Copy link
Author

lygstate commented Nov 6, 2014

Finally have this issue to be watched, I have a small group of people using msysgit 1.9.4 that always facing this problem, that's really annoying in daily life. There is nothing problem with the MSE.

@haimgel
Copy link

haimgel commented Nov 6, 2014

@YueLinHo: Thank you very much for pointing me to the workaround. My main concern is:

  • msgit behaves unpredictably. When "git clone" give different results from run to run, it's not ok, normalization or not normalization. This is the reason it gets confusing, because everything appears to work ok for a while, and then after say 5 commits and clones you get "modified files" problem. it's not trivial to attribute the issue to .gitattributes commit, especially when everything worked ok for a while.
  • msgit behaves differently from the original git. Git always lists all affected files as "modified", and msgit produces such result only occasionally.

So, in my opinion, it's still a regression.

@YueLinHo
Copy link

YueLinHo commented Nov 6, 2014

@hageltech

msgit behaves unpredictably. When "git clone" give different results from run to run, it's not ok, normalization or not normalization

I suppose that it will be not happened if you commit .gitattributes with the normalization.
Otherwise, you will get into that situation as you said, but you still can do the normalization after 5 commits is commit into repository. Then, I also suppose it will be not happened again.

In the other hand, indeed, it is a issue that committing .gitattributes without the normalization leads to that git does not list all files which should be normalized for each new clone.

I guess the root cause is on the timing of refresh index.

BTW, the normalization steps I tested and listed in this comment are via TortoiseGit.
(Also try CLI once and it works too.)

@haimgel
Copy link

haimgel commented Nov 7, 2014

@YueLinHo: Additional info:

Normalization does not work as intended

Steps to reproduce:

Clone my test repo:
git clone https://github.com/hageltech/msgit_test.git

Try to normalize as you suggested above:
git rm --cached -r . & git reset --hard & git status

Each time you run it, you get a different result. Sometimes files are listed as changed, sometimes they are not. End result is, this is not reliable with msgit.

I suggest you remove the "worksforme" tag... This seems to be a reproducible, serious issue with msgit. No way to normalize line ends with msgit, I had to use git on Linux to be sure all files are processed.

@YueLinHo
Copy link

YueLinHo commented Nov 7, 2014

Have you deleted all files first?

@haimgel
Copy link

haimgel commented Nov 7, 2014

Yep, I just changed the command to:
rm -Rf * & git rm --cached -r . & git reset --hard & git status

Same result. Sometimes files are listed as changed, sometimes "working directory clean".

Just run the above over and over...

@YueLinHo
Copy link

YueLinHo commented Nov 7, 2014

make sure only the .gitattributes is not deleted?

@haimgel
Copy link

haimgel commented Nov 7, 2014

Yes, @YueLinHo, I verified that manually also. Only .git directory and .gitattributes remain after "rm -Rf *".

@YueLinHo
Copy link

YueLinHo commented Nov 7, 2014

:-/
I test 14 times, 6 of them do not list all modified file. but all(6) of them show 63 modified files after normalization.
(The rest 8 shows 63 modified files directly.)

(using 1.9.2.0)

@haimgel
Copy link

haimgel commented Nov 7, 2014

I don't quite follow what you're saying... I'm implying that the normalization process itself (e.g. delete all files, do git reset") is flaky and does not always normalize all files. That the clone was doing weird things is just a symptom of this root problem.

Can you reproduce this or not? Do you agree normalization does not work as intended?

@YueLinHo
Copy link

YueLinHo commented Nov 8, 2014

Upgrade to 1.9.4.2 and re-test it

  1. cloned your testing repo to 20 repo
  2. only 1 local repo, named it A repo, shows 51 modified files, the others show 63 modified files. (expect 63 modified files)
  3. do normalize in A. Then, it shows 63 modified files as I expect.

I can reproduce the problem that it shows different number of modified for each new clone.
I can not reproduce the situation that normalization is failed.
So, based on my tests, I must to say that the normalization works for me.

I can/will do more test. ^_^

(BTW, I ever do a dangerous test that delete .git\index directly, then perform reset mixed.
Still works. Did this test, because I don't like to delete all file in working tree. not make sense for large repo.)

@YueLinHo
Copy link

YueLinHo commented Nov 8, 2014

@hageltech
SRROY. I agree normalization does not work always.
At least, the following steps does not work always:

  1. delete all files (except .gitattributes and .git folder)
  2. "git rm --cached -r ."
  3. "git reset --hard"

The following is another 75 testing result:
I cloned another 75 repo.
49 of 75: shows as I expect
26 of 75: do not.
then
3 of 26: normalization does not works at first time. (but second try.)

I feeling it might be some racing condition.
does the normalization always work in Linux?

(BTW, I don't have the right to remove "WorksForMe" tag. me, a user too.)

@haimgel
Copy link

haimgel commented Nov 8, 2014

I wrote a little script to do "rm -Rf * && git rm --cached -r . && git reset --hard & git status" repeatedly on Linux, with git 2.0.0, and the results are inconsistent too!

It behaves a little different, though... After around 30-40 "good" iterations I get one bad one, where working directory is listed as "clean".

So the problem is much less pronounced on Linux, but I must say, at this point, that the issue seems to be with Git in general, and not msysgit specific. Although this is more visible on Windows, the proper course of action is to check whether this still occurs with the latest version of git, and if yes, then to raise this issue at git mailing list.

@YueLinHo
Copy link

YueLinHo commented Nov 8, 2014

agree that

@lygstate
Copy link
Author

lygstate commented Nov 8, 2014

@t-b , Hello, can you remove WorksForMe for this issue.

@haimgel
Copy link

haimgel commented Nov 8, 2014

I took this issue to Git mailing list, since it is not restricted to msysgit (although it is more pronounced on msysgit):

http://thread.gmane.org/gmane.comp.version-control.git/259202

@dscho
Copy link
Member

dscho commented Nov 10, 2014

Thanks. Closing here.

@dscho dscho closed this as completed Nov 10, 2014
@YueLinHo
Copy link

Could you please also change the label from WorksForMe to Upstream? ^_^ Thank you.

@dscho dscho added Upstream and removed WorksForMe labels Nov 10, 2014
@lygstate
Copy link
Author

Which is the upstream bug? This is not enough.

@YueLinHo
Copy link

The conclusion
@DarkNami provides some references, such as Dealing with line endings, and might solve the problem.
@hageltech provides more information and a dummy repo which try to figure out what happened, also did a lot of tests for Windows and Linux.

These lead to:
Based on Dealing with line endings,
the normalization does not work ALWAYS(but usually), even on Linux OS.

So, @hageltech reports it to upstream.
I agree that, and I suppose @dscho also agree that.


@lygstate
If these are not enough, I suppose you need to do something enough to reopen this issue.
(Have you tried the normalization? Perhaps you experience different problem.)
Thank you. :)

@linquize
Copy link

But random is strange.

@YueLinHo
Copy link

@linquize
yeah, that is the problem.

@lygstate
Copy link
Author

lygstate commented Dec 3, 2014

Please don't close this issue by now, the git community doesn't response yet.

@dscho
Copy link
Member

dscho commented Dec 3, 2014

@lygstate do I understand correctly that you are offering to put in the effort it takes to debug the problem properly?

@lygstate
Copy link
Author

lygstate commented Dec 3, 2014

@dscho I means the git community doesn't response for this issue, so should we re-ping for this issue?

@dscho
Copy link
Member

dscho commented Dec 3, 2014

@lygstate I think that the only way you can get what you want is to actually invest your own time, effort and/or money into it. If you expect others to do your bidding, you might get lucky, but there is no guarantee. This is my personal opinion, of course.

@lygstate
Copy link
Author

lygstate commented Dec 3, 2014

@dscho You are totally talking about another thing, I means this issue should be re-open to tracking it.

@YueLinHo
Copy link

YueLinHo commented Dec 3, 2014

@lygstate
It's the same thing. It means you need to do something, bring up enough information to re-open this issue.

@lygstate
Copy link
Author

lygstate commented Dec 3, 2014

I've done the conversation, bye

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

8 participants