Create initial tower objects when we start the worker #14283

carbonin · 2017-03-10T21:38:22Z

These objects will be used for dynamically creating things like job templates during playbook runs from automate.

Requires #14377 for storing the ansible side ids with the provider.

https://www.pivotaltracker.com/story/show/140098929

carbonin · 2017-03-10T21:43:34Z

All, marked as WIP until you all sign off on where this kind of stuff belongs. I tested out the logic on an appliance so I really just need opinions on where this should live. (Of course if the logic is wrong, please tell me that too 😜 )

@blomquisg from a provider side view, is this where this kind of thing belongs? I don't know what the responsibilities of the provider/manager typically are so I added it here just because we already had a subclass for embedded ansible for the provider class.

@Fryguy I'm not sure I would want to add the other seeding stuff here. Maybe something in the worker for that as it's more system/filesystem related?

@gmcculloug Is this about what you guys need to create the job templates for the service runs? The provider instance will be accessible to you guys when you need this info I hope?

carbonin · 2017-03-10T21:44:28Z

Ping @jrafanie
This is the bit that will break if Apache isn't up and running when the worker starts. We may want to figure that out before merging this ...

Fryguy · 2017-03-10T22:04:04Z

app/models/manageiq/providers/ansible_tower/shared/provider.rb

+  end
+
+  def default_organization(connection = connect)
+    @@default_org ||= connection.api.organizations.all(


Not sure why you are using a class variable. In fact, given that those are inherited, I think that's generally bad right?

Well, I don't want it to be an instance variable because I want to cache more aggressively than that. I don't have much experience with class instance variables, but it feels like I would need to write explicit accessors to use them from an instance context, right? Plus it's all a bit more complicated because this is a mixin ...

I'm not crazy about any of the options, what do you think?

Fryguy · 2017-03-10T22:05:12Z

Not a fan of the term "Sandbox" in these objects. Sandbox implies things you can play with and wreck. This is more of a "ManageIQ default"...closer to seeding.

Fryguy · 2017-03-10T22:07:25Z

When we discussed yesterday I thought this would live as part of the embeddedansible worker during its do_before_work_loop phase. Did that not work as expected?

carbonin · 2017-03-13T13:09:10Z

@Fryguy From a timing perspective, yes, it will be called from around there, but I'm not sold on where to put the code.

I felt like it would be a bit strange for the automate methods to call out to a worker class or instance to get the default values so I put everything on the provider.

Like I said, I think the rest of the plugin import stuff should belong with the worker, but I'm a bit split on this part.

gmcculloug · 2017-03-15T15:06:12Z

@bzwei @bdunne Please review

carbonin · 2017-03-15T15:21:11Z

app/models/manageiq/providers/ansible_tower/shared/provider.rb

+
+  def default_organization(connection = connect)
+    @@default_org ||= connection.api.organizations.all(
+      :name => product_name


Is this a reliable way to identify these objects? If the product_name changes we will not be able to find our defaults.

Maybe this is a case for adding some kind of "system" identifier @Fryguy ? Then we can just look them up that way after creating them.

On second (third?) thought a flag won't really help, not for the organization at least, because we don't collect inventory for the organization.

carbonin · 2017-03-15T21:45:00Z

This is the bit that will break if Apache isn't up and running when the worker starts. We may want to figure that out before merging this ...

Turns out it wasn't that complicated #14353 😅

Fryguy · 2017-03-16T15:04:54Z

app/models/manageiq/providers/ansible_tower/shared/provider.rb

@@ -58,8 +58,85 @@ def url=(new_url)
    default_endpoint.url = uri.to_s
  end

+  def create_default_data


This should not be in shared I don't think

carbonin · 2017-03-16T15:42:48Z

Okay, spoke with @Fryguy and we're taking a bit of a new approach.

We're going to store these in custom attributes related to the provider. That way we don't have to rely on our modeling and potentially conflict with the refresher by adding fields (like a system flag or something) to the AR instances and we also don't have to constantly connect to the provider and match names to find things.

The custom attribute setter/getters will live in a concern and they will be used by the EmbeddedAnsibleWorker (for setting) and automate will use the getters when they need to create stuff referencing these objects.

We also won't need a migration, so that's good 😄

carbonin · 2017-03-17T12:53:33Z

I split out the provider side stuff into a separate PR here #14377

I'm going to make this one specific to the EmbeddedAnsibleWorker stuff.

bzwei · 2017-03-17T13:23:26Z

app/models/embedded_ansible_worker/object_management.rb

+  def ensure_host(provider, connection)
+    return if provider.default_host
+
+    provider.default_host = connection.api.hosts.create!(


@carbonin make sure the created localhost has variable ansible_connection: local

@bzwei Is that in the variables yaml string?

I'm guessing that would be :variables => {'ansible_connection' => "local"}.to_yaml for the API?

I think this should be addressed now.

miq-bot · 2017-03-17T19:18:37Z

This pull request is not mergeable. Please rebase and repush.

bdunne · 2017-03-17T20:47:01Z

app/models/embedded_ansible_worker/object_management.rb

+  def ensure_organization(provider, connection)
+    return if provider.default_organization
+
+    provider.default_organization = connection.api.organizations.create!(


Will this even work? default_orgainzation= creates a CustomAttribute on the Provider right? But you're passing in a AnsibleTowerClient::Organization instance.

I'm passing an id. See https://github.com/ManageIQ/manageiq/pull/14283/files/b67becaf26a4faffbc9d6b6c199837cdbe804a15#diff-45e99ec23caedc019723f05f7817652eR25

bdunne · 2017-03-17T20:49:43Z

app/models/embedded_ansible_worker/runner.rb

@@ -36,6 +38,17 @@ def setup_ansible
    _log.info("calling EmbeddedAnsible.start")
    EmbeddedAnsible.start
    _log.info("calling EmbeddedAnsible.start finished")
+
+    5.times do


Is 5 seconds long enough for the server to start up and respond?

EmbeddedAnsible.start is synchronous so this felt unnecessary to me, but this solved the 502 errors I was getting when testing on an appliance.

We can also make the WAIT_FOR_ANSIBLE_SLEEP a worker setting, but I like to lean towards fewer knobs where I can 😄

hmmm, it returned to you but it wasn't ready yet? Can we try to hit it using curl or something?

Can we try to hit it using curl or something?

That's essentially what EmbeddedAnsible.alive? does.

Does it make sense to have start do the sleep until alive? so start doesn't return when it's not yet alive?

Yeah, we could do that @jrafanie 👍

bdunne · 2017-03-17T20:52:28Z

app/models/embedded_ansible_worker/runner.rb

+
+    5.times do
+      if EmbeddedAnsible.alive?
+        break


Would it be simpler if this acted like guard clause?

5.times do break if EmbeddedAnsible.alive? _log.info("Waiting for EmbeddedAnsible to respond") sleep WAIT_FOR_ANSIBLE_SLEEP end

Yup thanks.

bdunne · 2017-03-17T21:33:02Z

spec/models/embedded_ansible_worker/runner_spec.rb

@@ -10,7 +10,9 @@
    let(:runner) {
      worker
      allow_any_instance_of(described_class).to receive(:worker_initialization)
-      described_class.new(:guid => worker_guid)
+      r = described_class.new(:guid => worker_guid)
+      allow(r).to receive(:worker).and_return(worker)


This is only necessary because of the stub above. Instead can we just tap the .new and call #find_worker_record?

#initialize calls: https://github.com/ManageIQ/manageiq/blob/master/app/models/miq_worker/runner.rb#L51
#worker_initialization calls: https://github.com/ManageIQ/manageiq/blob/master/app/models/miq_worker/runner.rb#L58
#starting_worker_record calls: https://github.com/ManageIQ/manageiq/blob/master/app/models/miq_worker/runner.rb#L170
#find_worker_record sets @worker: https://github.com/ManageIQ/manageiq/blob/master/app/models/miq_worker/runner.rb#L165

👍 I'll look into this.

😢 I have to finish fixing this. So much code now depends on all of this junk running when you call .new though.

So I tried to do a bit to get rid of this and it turns out we have to jump through hoops because sync_config changes the server's zone to the one called "default" which ends up being nil unless we Zone.seed ...

Even then there is no way to deal with how the runner looks up the worker, I would still need to stub the .worker method (attribute) to make sure we don't actually call #remove_demo_data and #ensure_initial_objects.

So then the last option would be to get rid of the allow_any_instance_of and also stub the #worker method ... That makes the test that asserts that we change the zone of the provider fail because somehow we are getting this "default" zone in there.

I think the answer here will be to just remove the zone/role stuff from settings. Then we don't have to deal with multiple sources of information for this type of thing.

Also, I don't really want to do that in this PR. I'll work on that as a follow up and fix this up there.

bdunne · 2017-03-17T21:53:22Z

spec/models/embedded_ansible_worker_spec.rb

+  context "ObjectManagement concern" do
+    let(:provider)       { FactoryGirl.create(:provider_embedded_ansible) }
+    let(:api_connection) do
+      conn = double("AnsibleAPIConnection")


It's kind of strange to stub #api to return the connection object itself. This could be:

let(:api_connection) { double("AnsibleAPIConnection", :api => tower_api) } let(:tower_api) { double("TowerAPI") }

Actually, now that I read further...

let(:tower_api) { double("TowerAPI", :organizations => org_collection, .....) } let(:org_collection) { double("AnsibleOrgCollection", :all => [org_resource]) }

Then you can get rid of the before block with all of the allows.

bdunne · 2017-03-17T22:01:23Z

spec/models/manageiq/providers/embedded_ansible/provider_spec.rb

+
+  context "DefaultAnsibleObjects concern" do
+    before do
+      subject.save


Why not FactoryGirl.create rather than adding the subject.save in the before block?

Ah, I think this was a rebase issue. I started this branch before #14383 was merged so instead of changing the subject for all the tests, I saved the record in just my context, during the rebase I think this line got left here.

I'll refactor in a new commit so that the individual specs define the subject rather than the shared example.

Thanks for catching this.

Also, I'll make this change in #14377

…bleWorker::Runner This creates a concern around creating and removing objects in the embedded ansible instance. In particular we remove all of the demo data that is initialized for us during setup and we create just the objects that we need and save the ids to the provider.

Without this we were we were getting "502 Bad Gateway" errors when trying to remove the demo data through API calls to the embedded ansible instance. This was because even though the setup playbook finished, the server would take just a bit longer to start to serve requests.

miq-bot · 2017-03-20T21:53:02Z

Checked commits carbonin/manageiq@3484703~...5d2d5df with ruby 2.2.6, rubocop 0.47.1, and haml-lint 0.20.0
7 files checked, 0 offenses detected
Everything looks good. ⭐

carbonin · 2017-03-21T13:07:00Z

@bdunne @jrafanie @Fryguy #14377 is merged so I think this one is ready for review.

gmcculloug · 2017-03-21T19:53:48Z

@bdunne @jrafanie @Fryguy Any comments? Would like to get this merged if it is ready as there are followup changes required to reference these new elements.

Specifically I am aware of this one: https://github.com/ManageIQ/manageiq/blob/master/app/models/service_template_ansible_playbook.rb#L94
cc @bzwei

jrafanie · 2017-03-21T19:59:04Z

lib/embedded_ansible.rb

+
+      _log.info("Waiting for EmbeddedAnsible to respond")
+      sleep WAIT_FOR_ANSIBLE_SLEEP
+    end


I like this better. Now the caller can be guaranteed to be alive or get an exception. 👍

bdunne

👍 LGTM

bdunne · 2017-03-21T20:12:15Z

spec/models/embedded_ansible_worker_spec.rb

+    let(:job_templ_resource)   { double("AnsibleJobTemplResource", :id => 16) }
+    let(:proj_resource)        { double("AnsibleProjResource", :id => 17) }
+
+    describe "#ensure_initial_objects" do


I'm not a fan of the describe with only one it inside, but don't change just because of that.

Interesting, I agree for contexts, but I always use a describe for the method under test regardless of how many tests I'm writing.

carbonin added core/workers enhancement providers/ansible_tower wip labels Mar 10, 2017

carbonin assigned blomquisg Mar 10, 2017

carbonin requested review from blomquisg, Fryguy and gmcculloug March 10, 2017 21:38

Fryguy reviewed Mar 10, 2017

View reviewed changes

carbonin force-pushed the add_embedded_tower_initial_objects branch from b1ec5f2 to 43e07a9 Compare March 15, 2017 15:20

carbonin commented Mar 15, 2017

View reviewed changes

Fryguy reviewed Mar 16, 2017

View reviewed changes

carbonin force-pushed the add_embedded_tower_initial_objects branch 2 times, most recently from 61d5029 to a03e434 Compare March 16, 2017 22:02

carbonin force-pushed the add_embedded_tower_initial_objects branch from a03e434 to bbd2c6b Compare March 17, 2017 12:54

carbonin requested review from jrafanie and removed request for blomquisg and gmcculloug March 17, 2017 12:55

carbonin assigned gtanzillo and unassigned blomquisg Mar 17, 2017

bzwei reviewed Mar 17, 2017

View reviewed changes

carbonin force-pushed the add_embedded_tower_initial_objects branch from bbd2c6b to 46b5018 Compare March 17, 2017 15:55

miq-bot added the unmergeable label Mar 17, 2017

carbonin force-pushed the add_embedded_tower_initial_objects branch from 1d646c1 to f9ec890 Compare March 17, 2017 19:46

miq-bot removed the unmergeable label Mar 17, 2017

carbonin force-pushed the add_embedded_tower_initial_objects branch from f9ec890 to b67beca Compare March 17, 2017 20:33

bdunne requested changes Mar 17, 2017

View reviewed changes

carbonin mentioned this pull request Mar 20, 2017

Add a concern for storing and accessing embedded ansible object ids #14377

Merged

carbonin force-pushed the add_embedded_tower_initial_objects branch 4 times, most recently from 30f0071 to f86348f Compare March 20, 2017 21:45

carbonin added 4 commits March 20, 2017 17:49

Make EmbeddedAnsible.api_connection public for the worker

3484703

Correct runner specs for calling new worker methods

f9ae577

carbonin force-pushed the add_embedded_tower_initial_objects branch from f86348f to 5d2d5df Compare March 20, 2017 21:49

carbonin changed the title ~~[WIP] Create initial tower objects when we start the worker~~ Create initial tower objects when we start the worker Mar 20, 2017

carbonin removed the wip label Mar 20, 2017

jrafanie reviewed Mar 21, 2017

View reviewed changes

bdunne approved these changes Mar 21, 2017

View reviewed changes

gmcculloug merged commit 5e631c1 into ManageIQ:master Mar 22, 2017

gmcculloug added this to the Sprint 57 Ending Mar 27, 2017 milestone Mar 22, 2017

carbonin deleted the add_embedded_tower_initial_objects branch May 18, 2017 17:14

Fryguy added the providers/automation label Jan 24, 2020

Create initial tower objects when we start the worker #14283

Create initial tower objects when we start the worker #14283

Conversation

carbonin commented Mar 10, 2017 • edited Loading

carbonin commented Mar 10, 2017

carbonin commented Mar 10, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Fryguy commented Mar 10, 2017

Fryguy commented Mar 10, 2017

carbonin commented Mar 13, 2017

gmcculloug commented Mar 15, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

carbonin commented Mar 15, 2017

Choose a reason for hiding this comment

carbonin commented Mar 16, 2017

carbonin commented Mar 17, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

miq-bot commented Mar 17, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

miq-bot commented Mar 20, 2017

carbonin commented Mar 21, 2017

gmcculloug commented Mar 21, 2017

Choose a reason for hiding this comment

bdunne left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

carbonin commented Mar 10, 2017 •

edited

Loading