Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

spinning up branches: phase 1 #4990

Closed
pdurbin opened this issue Aug 21, 2018 · 18 comments
Closed

spinning up branches: phase 1 #4990

pdurbin opened this issue Aug 21, 2018 · 18 comments
Assignees

Comments

@pdurbin
Copy link
Member

pdurbin commented Aug 21, 2018

At tech hours we just talked about the idea of spinning up arbitrary branches on AWS. I was asked to make an issue we can estimate and I'm going to try to keep this small but the whiteboard notes below reflect some future thinking as well.

Definition of done:

img_20180821_162009884

@djbrooke
Copy link
Contributor

Thanks @pdurbin, we'll estimate this today.

@pdurbin
Copy link
Member Author

pdurbin commented Aug 23, 2018

@donsizemore just fixed up dataverse-ansible so that it will deploy arbitrary branches (thanks!!). You just add branch name to tests/group_vars/vagrant.yml like this:

murphy:dataverse-ansible pdurbin$ git diff
diff --git a/tests/group_vars/vagrant.yml b/tests/group_vars/vagrant.yml
index 35fa041..f276df7 100644
--- a/tests/group_vars/vagrant.yml
+++ b/tests/group_vars/vagrant.yml
@@ -8,7 +8,7 @@ dataverse:
     blocked_endpoints: "admin,test"
     blocked_policy: "localhost-only"
     location: "http://localhost:8080/api"
-  branch:
+  branch: "4926-unlock-dataset-api"
   copyright: "© Your Institution"
   db:
     name: dvndb
murphy:dataverse-ansible pdurbin$ 

And then you run vagrant up. Heads up to @dlmurphy that this might be immediately useful to you when writing messaging. It works today.

@matthew-a-dunlap
Copy link
Contributor

My first goal in picking this up is to explore scripting the creation of an EC2 instance and dropping in relevant information (e.g. branch name) for ansible to pick up and use for bringing up Dataverse

@matthew-a-dunlap
Copy link
Contributor

matthew-a-dunlap commented Sep 14, 2018

Current state: the shell script is now creating an EC2 instance and passing the branch as expected to ansible. There are one or two outstanding issues related to having ansible work in our current usecase. One is that the current ansible process has an upgrade step which causes it to upgrade itself when running it locally, causing an error. The other is that for some reason ansible seems to use the latest release (its default functionality) even after building whatever arbitrary branch that it was pointed to. These are things to look into on Monday.

@donsizemore
Copy link
Contributor

want me to comment out the upgrade step permanently? it wasn’t causing problems in Vagrant

the release version is also an extra variable you can pass to ansible; you’ll want to coordinate that with the specified branch

@matthew-a-dunlap
Copy link
Contributor

@donsizemore that sounds great (removing the step). For the release version, not all branches have been released, what does ansible expect in that case? It seemed when I had the branch as develop and the release version as 4.9.2 (hardcoded default) none of the new develop changes were deployed

@donsizemore
Copy link
Contributor

there was/is no standard for naming custom files per version, so i punted and had it explode the dvinstall.zip for a given version (and i bump the default with each release)

if there’s a branch specified ansible does a git clone and a mvn package, then overwrites the warfile before deploying.

if you want we can pull necessary modified files from a given branch (usually schema.xml or some bit of json) but i’d be interested in seeing the output of your ansible run; phil and i each verified that it will deploy a given branch properly.

@matthew-a-dunlap
Copy link
Contributor

@donsizemore i may have something wrong in my setup, I'll pick up on it Monday!

@matthew-a-dunlap
Copy link
Contributor

Looks like my issue with the branch was due to the order of operations of where ansible looks for attributes. I've worked around it and things flow correctly!

Right now the script & ansible only support branches on the IQSS dataverse repo due to similar variable issues. I decided to save that for phase 2 of this story.

pdurbin added a commit that referenced this issue Sep 24, 2018
Also put Ansible before Puppet since dataverse-puppet hasn't been
updated since March 2016.
@kcondon kcondon self-assigned this Sep 24, 2018
@dlmurphy
Copy link
Contributor

dlmurphy commented Sep 24, 2018

I just walked through the instructions and had a bear of a time trying to get things working. With some help from @matthew-a-dunlap I managed to eventually successfully spin up a branch and then terminate it. Here are the points where I had difficulty, our guide could perhaps address these:

  • Adding the AWS CLI executable to my command line path was a very confusing process. Matthew linked me to Amazon's guide, but it doesn't explain the process very well, and we ended up having to find a stack overflow answer that finally got it to work. We should consider explaining this part of the AWS setup process in our guide.

  • I followed all the steps under "Configure AWS CLI", but when I ran the script it totally ignored the config files. I ended up having to use the AWS configure command and input the config parameters that way. Maybe we should just instruct users to run that command instead of creating those text files.

@kcondon
Copy link
Contributor

kcondon commented Sep 27, 2018

@pdurbin
I had aws cli already installed so I was able to spin up two separate branches using the command supplied, verified it worked, then destroyed the instance using the provided command.

Would like to discuss any doc changes that might help with Derek's reported issues. Otherwise seems ready to merge.

I did find one issue: lots of errors in logs about not being able to send email to smtp.yourinstitution.edu when doing basic tasks in app.

Also, it was pointed out that build number does not indicate which branch is running, wondering whether you can poke build.properties to hack that.

@pdurbin
Copy link
Member Author

pdurbin commented Sep 28, 2018

@kcondon thanks, that's good feedback. For the SMTP thing I opened IQSS/dataverse-ansible#28 but the branch should already be there. It was added in IQSS/dataverse-ansible@d4dc7cb . What's missing from the build number is the commit. Let's chat about it to make sure we're seeing the same thing. We can discuss doc changes too, like you said.

@donsizemore
Copy link
Contributor

@pdurbin dataverse-ansible should be munging build.properties when a branch is passed. i'll check on that.

@pdurbin
Copy link
Member Author

pdurbin commented Sep 28, 2018

I sat with @dlmurphy to discuss his two bullets above and made improvements to the docs in 562c9c7 and 746eec5

I also spoke with @kcondon and @djbrooke and it sounds like we're happy with how phase 1 has turned out so I'm moving this to QA.

@donsizemore yes, the branch is being added to the bottom right corner but not the commit. I'll ping you in IRC or in the dataverse-ansible repo about this. Thanks for adding the branch in IQSS/dataverse-ansible@d4dc7cb !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants