Skip to content
This repository has been archived by the owner on Jun 1, 2023. It is now read-only.

Package install runs parallel #29

Merged
merged 1 commit into from
Aug 28, 2013
Merged

Package install runs parallel #29

merged 1 commit into from
Aug 28, 2013

Conversation

elcuervo
Copy link
Contributor

Since we have bash on our side we can speed up the package installation.

And yes... it's because of this rubygems/bundler#2481 :bowtie:

@pote
Copy link
Contributor

pote commented Aug 28, 2013

This is the best pull request ever, you're awesome <3

pote added a commit that referenced this pull request Aug 28, 2013
Package install runs in parallel
@pote pote merged commit 0aeda96 into VividCortex:master Aug 28, 2013
@gravis
Copy link

gravis commented Aug 29, 2013

What if some packages have the same dependencies?
Don't you expect some issues there (2 go processes could write the same files at the same moment, even if probability is low)?
What do you think?

@elcuervo
Copy link
Contributor Author

@gravis you mean two johnny-deps running at the same time?

@gravis
Copy link

gravis commented Aug 29, 2013

I mean 2 go get -v -u -d, since they're running in parallel now.
The blocking I see would be:

A => B => C
D => B

If A and D gets installed at the same time, there's a tiny chance to install B concurrently (which could lead to broken files). I know, it's really unlikely, but still. Maybe a simple note in the README about that troubleshooting (in which case, the packages should be reinstalled from scratch)?
My 2c ;)

@elcuervo
Copy link
Contributor Author

Since go get uses git under the hood I don't really think that it will lead to broken files: http://code.google.com/p/go/source/browse/src/cmd/go/vcs.go#94
Even if it's a fresh clone I don't see this scenario happening. But I'll try to force it and see how it goes.

@gravis
Copy link

gravis commented Aug 29, 2013

yes, the best test I see is to run 2 git command at the same time, cloning the same repo, and see how it goes.
I would bet on a successful result :)
Probably git is putting a lock file inside the folder, but then the second go get will probably fail, and won't install the remaining packages ;)
Sorry, I really like your approach, and don't want to criticize anything, I just had bad experiences like that in the past :)

@elcuervo
Copy link
Contributor Author

Criticism is the smartest way of communication if you do it right. Your comments are prefect on that way so keep them coming!. We all want this tool to get better and better.
Two pull at the same moment will make one of them fail but that won't affect the consistency of the files but it'll make johnny-deps crash.
I agree with adding some sort of troubleshooting part in the README.

Thank you for seeing it!
👉

@gravis
Copy link

gravis commented Aug 29, 2013

Welcome! 👍

@pote
Copy link
Contributor

pote commented Aug 29, 2013

Great call @gravis , I hadn't thought of this but it certainly merits some documentation and possibly a test.

Dependency resolution inside johnny-deps is something Im thinking about, I'll keep this in mind and think of a way to avoid the edge case.

@elcuervo elcuervo deleted the parallel_install branch August 29, 2013 18:00
@elcuervo
Copy link
Contributor Author

elcuervo commented Sep 1, 2013

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants