This repository has been archived by the owner on Jul 6, 2022. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 99
Publish a roadmap #109
Comments
Here's a draft of the roadmap, as @seanmck, @krancour and I discussed today: We've defined stability by two different dimensions:
See below for stability plans for each. Services and PlansServices/Plans have three stability tiers:
Here are our timelines for service stabilities:
We aren't yet setting timelines on other service stabilities. Database SchemasThere are two tiers of database schemas to consider:
Our goal is to have a few services into We aren't yet setting timelines on declaring individual service schemas as stable. |
jeremyrickard
added a commit
to jeremyrickard/open-service-broker-azure
that referenced
this issue
Jan 24, 2018
jeremyrickard
pushed a commit
that referenced
this issue
Feb 6, 2018
* refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Few minor corrections to the Quickstart (#136) * Correcting the az ad sp delete command * Update quickstart-minikube.md Remove a leading $ from a few bash commands. copy paste on mac is messed up :-) * Update quickstart-minikube.md Clarification around logging in to WordPress * restoring prompts in quickstart * minor punctuation fix * removing distracting echo command * make prompts used consistently * kick of redis tests sooner (#155) * s/module lifecycle tests/service lifecycle tests/g (#156) * make storage know how to deal with service-specific types (#153) * don't use codec where we don't need it anymore (#161) * Updating CircleCI to define DOCKER_REPO env var (#167) * Updating CircleCI to define REGISTRY env var We previously set the REGISTRY environment variable in a deploy script. When that was removed to streamline the release process, we need to define it in Circle. Added two environment blocks to the publish-rc-images and publish-release-images jobs. Fixes: 145 * Qualified registry with docker.io * Changed REGISTRY to DOCKER_REPO * s/context/details/g (#163) * make fake bind/unbind functionality not rely on standard context * collapse standard provisioning context onto instance * do away with (sort of) redundant standard provisioning parameters * s/provisioning context/instance details/ * s/binding context/binding details/ * fix error handling when standard params aren't expected types * add comment that adds some clarity to instance comparisons * don't store credentials as their own field (#169) * quickstart fixes (#174) * adding missing docker pushes to release process * Adding build flags to inject the version and commit (#173) * Adding build flags to inject the version and commit This commit injects the version and commit to the OSBA binary. If there is no version set, then sets the version to “devel” * Using new variable for version * Removing redundant setting for main.commit * Printing version and commit on broker startup * Adding fields to the log message on startup * Removing build flags on the CLI builds * broker doesn't need to hang on to codec (#180) * remove quotes from devel pseudo-version string (#184) Merging without CI. * fix bug decoding tags (#183) * move version info to avoid import cycle later (#185) * Adding user-agent string to requests to ARM (#172) * Adding user-agent string to requests to ARM * get version for user agent string from version package * use separate redis clients for storage and async engine (#182) * Adding ability to filter lifecycle tests (#181) * Adding ability to filter lifecycle tests This adds the ability to filter lifecycle tests using an environment variable (TEST_MODULES). Via this variable, the developer can provide a comma delimited list of module names and only the test cases matching that module will be enabled. If the variable is empty, all tests will run. If an invalid module name is provided, no tests will run. * fix typo and lint error * remove unnecessary funtion * use redis namespaces (#188) * use redis namespaces for instances and bindings * use redis namespaces in async engine * start using lightweight-docker-go image for development tasks (#190) * simplify ci config (#194) * README work (#196) * README formatting + one cmdline fix * Link fixes * Revert heading to title case * append instead of replacing user-agent (#198) * append instead of replacing user-agent * fix lint errors * add deferred task execution to async package (#208) achieving this involved a significant refactor of the entire package. the good news is that the tests are improved beyond what we had before, so i think we can be more confident in the refactored package than the original. * reorganize async pakage/sub-packages more sanely (#210) * change provision/deprovision task names (#211) The are imperative and more accurate. * Add async support for "follow-up" tasks (#212) * make async job functions take task as argument * make async job functions return tasks * bonus: fix more task names * make async task executor submit follow-up tasks * make broker async processes leverage new async follow-up tasks feature * ignore .vscode/ (#221) * more async refactoring (#219) * break up worker.go and worker_test.go into multiple files * simplify heart * simplify cleaner * move heart from worker to engine * remove receive/execute/watch from worker to engine * let task receivers return right after fatal error * let task executor return right after fatal error * very minor comment cleanup * fix lint errors by removing unused error types * add svc and plan to instance during retrieval (#220) * retrieve svc when instance is retrieved * retrieve plan when instance is retrieved * improve provision step function signature * improve deprovision step function signature * improve update step function signature * lifecycles tests: set service and plan on instance * Changes to mysql bind. (#222) Change grant string in mysql bind to be bound to database Removed global privs Added Execute Fixes #197 * Initial roadmap submission (#226) * Initial Roadmap * Publish initial roadmap for OSBA Closes #109 * Spelling * Review comments and wordsmithing. * Refactor for generic filter capabilities (#218) * WIP: Proposal to refactor for generic filter capabilities 1.) Create new api/filter structure 2.) Refactored authenticator stuff there 3.) Created new interfaces: filters.Filter and filters.Chain 4.) Made basic/always authenticator implement filters.Filter interface * Fixed filename type-o. * Fixed filename type-o * Renamed Filter method to Execute. * Added test for filter chain * Refactor and style fixes * ignore .vscode/ (#221) * more async refactoring (#219) * break up worker.go and worker_test.go into multiple files * simplify heart * simplify cleaner * move heart from worker to engine * remove receive/execute/watch from worker to engine * let task receivers return right after fatal error * let task executor return right after fatal error * very minor comment cleanup * fix lint errors by removing unused error types * add svc and plan to instance during retrieval (#220) * retrieve svc when instance is retrieved * retrieve plan when instance is retrieved * improve provision step function signature * improve deprovision step function signature * improve update step function signature * lifecycles tests: set service and plan on instance * changed filters package to filter * Lint failure * Refactor filter package a little and re-add Chain interface * a few filter tweaks * Change how the database name for cosmos db is generated (#228) * Change the generation of the database name. A limitation in the current ARM based CosmosDB provisoning limits the length of the database name to 50 characters, including the name provided, the location name and a dash. In some situations, this means a UUID is invalid. If the length will be too much, this patch will randomly generate a string of the available length instead of using a UUID. * Refactored to use generate package * fix async cleaner (#233) * bug fix: fix wait again on children not deprovisioned * bug fix: it's ok for an instance's parent to not be found
jeremyrickard
added a commit
that referenced
this issue
Feb 6, 2018
* WIP Refactoring sqldb to POC (#142) * Refactoring sqldb to POC #124 Removing the "existing server" logic from the current sqldb module. This entails removing it from the provision/deprovision file and removing it from the types. Also removed the config.go file and updated the broker to not load that. We will need another way to provide/register existing servers. We will reimplement this behavior in a new service/service manager. * fix minor lint failure * Feature branch merge (#154) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Sqldb catalog poc merge master (#158) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Few minor corrections to the Quickstart (#136) * Correcting the az ad sp delete command * Update quickstart-minikube.md Remove a leading $ from a few bash commands. copy paste on mac is messed up :-) * Update quickstart-minikube.md Clarification around logging in to WordPress * restoring prompts in quickstart * minor punctuation fix * removing distracting echo command * make prompts used consistently * kick of redis tests sooner (#155) * s/module lifecycle tests/service lifecycle tests/g (#156) * make storage know how to deal with service-specific types (#153) * make provision and deprovision steps receive a reference instance (#160) * Merge master to feature (#175) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Few minor corrections to the Quickstart (#136) * Correcting the az ad sp delete command * Update quickstart-minikube.md Remove a leading $ from a few bash commands. copy paste on mac is messed up :-) * Update quickstart-minikube.md Clarification around logging in to WordPress * restoring prompts in quickstart * minor punctuation fix * removing distracting echo command * make prompts used consistently * kick of redis tests sooner (#155) * s/module lifecycle tests/service lifecycle tests/g (#156) * make storage know how to deal with service-specific types (#153) * don't use codec where we don't need it anymore (#161) * Updating CircleCI to define DOCKER_REPO env var (#167) * Updating CircleCI to define REGISTRY env var We previously set the REGISTRY environment variable in a deploy script. When that was removed to streamline the release process, we need to define it in Circle. Added two environment blocks to the publish-rc-images and publish-release-images jobs. Fixes: 145 * Qualified registry with docker.io * Changed REGISTRY to DOCKER_REPO * s/context/details/g (#163) * make fake bind/unbind functionality not rely on standard context * collapse standard provisioning context onto instance * do away with (sort of) redundant standard provisioning parameters * s/provisioning context/instance details/ * s/binding context/binding details/ * fix error handling when standard params aren't expected types * add comment that adds some clarity to instance comparisons * don't store credentials as their own field (#169) * quickstart fixes (#174) * adding missing docker pushes to release process * minor merge nit fixes * Decomposition of single service into multiple services (#176) * Decomposition of single service into multiple services 1.) This renames the existing service manager into a service to implement the all in one scenario (existing) 2.) Next, two new services are added (vm only and db only) 3.) Each lifecycle operation go file (bind, deprovision, provision, unbind) now declares a method for each service manager struct 4.) Added new services to the catalog 5.) DB only is largely TBD implemented, took a crack at starting the provision. 6.) Refactored catalog/service interface to add GetBindable() method 7.) Modified tests to only execute binding related tests if plan is bindable * Small spelling correction * Implemented database only lifecycle methods and lint fix. * DRY refactoring * Review comments and lint fix * add ref instance as instance attribute (#178) * create a relationship between two services (#179) * Refactor sqldb module to use parent instances in binding (#186) * Refactor sqldb module to use parent instances in binding Small refactor to decouple shared instance details and leverage parent instance for binding/unbinding. Further evolution for #124 * Updated error to be consistent. * merge latest from master (#187) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Few minor corrections to the Quickstart (#136) * Correcting the az ad sp delete command * Update quickstart-minikube.md Remove a leading $ from a few bash commands. copy paste on mac is messed up :-) * Update quickstart-minikube.md Clarification around logging in to WordPress * restoring prompts in quickstart * minor punctuation fix * removing distracting echo command * make prompts used consistently * kick of redis tests sooner (#155) * s/module lifecycle tests/service lifecycle tests/g (#156) * make storage know how to deal with service-specific types (#153) * don't use codec where we don't need it anymore (#161) * Updating CircleCI to define DOCKER_REPO env var (#167) * Updating CircleCI to define REGISTRY env var We previously set the REGISTRY environment variable in a deploy script. When that was removed to streamline the release process, we need to define it in Circle. Added two environment blocks to the publish-rc-images and publish-release-images jobs. Fixes: 145 * Qualified registry with docker.io * Changed REGISTRY to DOCKER_REPO * s/context/details/g (#163) * make fake bind/unbind functionality not rely on standard context * collapse standard provisioning context onto instance * do away with (sort of) redundant standard provisioning parameters * s/provisioning context/instance details/ * s/binding context/binding details/ * fix error handling when standard params aren't expected types * add comment that adds some clarity to instance comparisons * don't store credentials as their own field (#169) * quickstart fixes (#174) * adding missing docker pushes to release process * Adding build flags to inject the version and commit (#173) * Adding build flags to inject the version and commit This commit injects the version and commit to the OSBA binary. If there is no version set, then sets the version to “devel” * Using new variable for version * Removing redundant setting for main.commit * Printing version and commit on broker startup * Adding fields to the log message on startup * Removing build flags on the CLI builds * broker doesn't need to hang on to codec (#180) * remove quotes from devel pseudo-version string (#184) Merging without CI. * fix bug decoding tags (#183) * move version info to avoid import cycle later (#185) * Adding user-agent string to requests to ARM (#172) * Adding user-agent string to requests to ARM * get version for user agent string from version package * use separate redis clients for storage and async engine (#182) * Adding ability to filter lifecycle tests (#181) * Adding ability to filter lifecycle tests This adds the ability to filter lifecycle tests using an environment variable (TEST_MODULES). Via this variable, the developer can provide a comma delimited list of module names and only the test cases matching that module will be enabled. If the variable is empty, all tests will run. If an invalid module name is provided, no tests will run. * fix typo and lint error * remove unnecessary funtion * Merge master (#189) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Few minor corrections to the Quickstart (#136) * Correcting the az ad sp delete command * Update quickstart-minikube.md Remove a leading $ from a few bash commands. copy paste on mac is messed up :-) * Update quickstart-minikube.md Clarification around logging in to WordPress * restoring prompts in quickstart * minor punctuation fix * removing distracting echo command * make prompts used consistently * kick of redis tests sooner (#155) * s/module lifecycle tests/service lifecycle tests/g (#156) * make storage know how to deal with service-specific types (#153) * don't use codec where we don't need it anymore (#161) * Updating CircleCI to define DOCKER_REPO env var (#167) * Updating CircleCI to define REGISTRY env var We previously set the REGISTRY environment variable in a deploy script. When that was removed to streamline the release process, we need to define it in Circle. Added two environment blocks to the publish-rc-images and publish-release-images jobs. Fixes: 145 * Qualified registry with docker.io * Changed REGISTRY to DOCKER_REPO * s/context/details/g (#163) * make fake bind/unbind functionality not rely on standard context * collapse standard provisioning context onto instance * do away with (sort of) redundant standard provisioning parameters * s/provisioning context/instance details/ * s/binding context/binding details/ * fix error handling when standard params aren't expected types * add comment that adds some clarity to instance comparisons * don't store credentials as their own field (#169) * quickstart fixes (#174) * adding missing docker pushes to release process * Adding build flags to inject the version and commit (#173) * Adding build flags to inject the version and commit This commit injects the version and commit to the OSBA binary. If there is no version set, then sets the version to “devel” * Using new variable for version * Removing redundant setting for main.commit * Printing version and commit on broker startup * Adding fields to the log message on startup * Removing build flags on the CLI builds * broker doesn't need to hang on to codec (#180) * remove quotes from devel pseudo-version string (#184) Merging without CI. * fix bug decoding tags (#183) * move version info to avoid import cycle later (#185) * Adding user-agent string to requests to ARM (#172) * Adding user-agent string to requests to ARM * get version for user agent string from version package * use separate redis clients for storage and async engine (#182) * Adding ability to filter lifecycle tests (#181) * Adding ability to filter lifecycle tests This adds the ability to filter lifecycle tests using an environment variable (TEST_MODULES). Via this variable, the developer can provide a comma delimited list of module names and only the test cases matching that module will be enabled. If the variable is empty, all tests will run. If an invalid module name is provided, no tests will run. * fix typo and lint error * remove unnecessary funtion * use redis namespaces (#188) * use redis namespaces for instances and bindings * use redis namespaces in async engine * validate alias and parentAlias (#191) * Update lifecycle tests to run mssql db only (#192) * Update lifecycle tests to run mssql db only 1.) Small refactor to test_case_test.go to break provision into a reusable function 2.) Modified setup function signature to include a *service.Instance to use as a parent instance 3.) Added setup function in mssql tests 4.) Added lifecycle test for DB only 5.) Fixed db only privision bug Implemnts #124 * Lint fixes * test refactor to support "child tests" * Small rename * Test description update * handle storage aspects of instance aliases (#195) * merge latest from master (#199) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Few minor corrections to the Quickstart (#136) * Correcting the az ad sp delete command * Update quickstart-minikube.md Remove a leading $ from a few bash commands. copy paste on mac is messed up :-) * Update quickstart-minikube.md Clarification around logging in to WordPress * restoring prompts in quickstart * minor punctuation fix * removing distracting echo command * make prompts used consistently * kick of redis tests sooner (#155) * s/module lifecycle tests/service lifecycle tests/g (#156) * make storage know how to deal with service-specific types (#153) * don't use codec where we don't need it anymore (#161) * Updating CircleCI to define DOCKER_REPO env var (#167) * Updating CircleCI to define REGISTRY env var We previously set the REGISTRY environment variable in a deploy script. When that was removed to streamline the release process, we need to define it in Circle. Added two environment blocks to the publish-rc-images and publish-release-images jobs. Fixes: 145 * Qualified registry with docker.io * Changed REGISTRY to DOCKER_REPO * s/context/details/g (#163) * make fake bind/unbind functionality not rely on standard context * collapse standard provisioning context onto instance * do away with (sort of) redundant standard provisioning parameters * s/provisioning context/instance details/ * s/binding context/binding details/ * fix error handling when standard params aren't expected types * add comment that adds some clarity to instance comparisons * don't store credentials as their own field (#169) * quickstart fixes (#174) * adding missing docker pushes to release process * Adding build flags to inject the version and commit (#173) * Adding build flags to inject the version and commit This commit injects the version and commit to the OSBA binary. If there is no version set, then sets the version to “devel” * Using new variable for version * Removing redundant setting for main.commit * Printing version and commit on broker startup * Adding fields to the log message on startup * Removing build flags on the CLI builds * broker doesn't need to hang on to codec (#180) * remove quotes from devel pseudo-version string (#184) Merging without CI. * fix bug decoding tags (#183) * move version info to avoid import cycle later (#185) * Adding user-agent string to requests to ARM (#172) * Adding user-agent string to requests to ARM * get version for user agent string from version package * use separate redis clients for storage and async engine (#182) * Adding ability to filter lifecycle tests (#181) * Adding ability to filter lifecycle tests This adds the ability to filter lifecycle tests using an environment variable (TEST_MODULES). Via this variable, the developer can provide a comma delimited list of module names and only the test cases matching that module will be enabled. If the variable is empty, all tests will run. If an invalid module name is provided, no tests will run. * fix typo and lint error * remove unnecessary funtion * use redis namespaces (#188) * use redis namespaces for instances and bindings * use redis namespaces in async engine * start using lightweight-docker-go image for development tasks (#190) * simplify ci config (#194) * populate parent when retrieving instance (#200) * parent child count (#201) * fix arg name * break up storage test cases better * track number of children belonging to an instance * Merge master into feature branch (#209) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Few minor corrections to the Quickstart (#136) * Correcting the az ad sp delete command * Update quickstart-minikube.md Remove a leading $ from a few bash commands. copy paste on mac is messed up :-) * Update quickstart-minikube.md Clarification around logging in to WordPress * restoring prompts in quickstart * minor punctuation fix * removing distracting echo command * make prompts used consistently * kick of redis tests sooner (#155) * s/module lifecycle tests/service lifecycle tests/g (#156) * make storage know how to deal with service-specific types (#153) * don't use codec where we don't need it anymore (#161) * Updating CircleCI to define DOCKER_REPO env var (#167) * Updating CircleCI to define REGISTRY env var We previously set the REGISTRY environment variable in a deploy script. When that was removed to streamline the release process, we need to define it in Circle. Added two environment blocks to the publish-rc-images and publish-release-images jobs. Fixes: 145 * Qualified registry with docker.io * Changed REGISTRY to DOCKER_REPO * s/context/details/g (#163) * make fake bind/unbind functionality not rely on standard context * collapse standard provisioning context onto instance * do away with (sort of) redundant standard provisioning parameters * s/provisioning context/instance details/ * s/binding context/binding details/ * fix error handling when standard params aren't expected types * add comment that adds some clarity to instance comparisons * don't store credentials as their own field (#169) * quickstart fixes (#174) * adding missing docker pushes to release process * Adding build flags to inject the version and commit (#173) * Adding build flags to inject the version and commit This commit injects the version and commit to the OSBA binary. If there is no version set, then sets the version to “devel” * Using new variable for version * Removing redundant setting for main.commit * Printing version and commit on broker startup * Adding fields to the log message on startup * Removing build flags on the CLI builds * broker doesn't need to hang on to codec (#180) * remove quotes from devel pseudo-version string (#184) Merging without CI. * fix bug decoding tags (#183) * move version info to avoid import cycle later (#185) * Adding user-agent string to requests to ARM (#172) * Adding user-agent string to requests to ARM * get version for user agent string from version package * use separate redis clients for storage and async engine (#182) * Adding ability to filter lifecycle tests (#181) * Adding ability to filter lifecycle tests This adds the ability to filter lifecycle tests using an environment variable (TEST_MODULES). Via this variable, the developer can provide a comma delimited list of module names and only the test cases matching that module will be enabled. If the variable is empty, all tests will run. If an invalid module name is provided, no tests will run. * fix typo and lint error * remove unnecessary funtion * use redis namespaces (#188) * use redis namespaces for instances and bindings * use redis namespaces in async engine * start using lightweight-docker-go image for development tasks (#190) * simplify ci config (#194) * README work (#196) * README formatting + one cmdline fix * Link fixes * Revert heading to title case * append instead of replacing user-agent (#198) * append instead of replacing user-agent * fix lint errors * add deferred task execution to async package (#208) achieving this involved a significant refactor of the entire package. the good news is that the tests are improved beyond what we had before, so i think we can be more confident in the refactored package than the original. * Provision and Deprovision enhancements to use new async features (#204) * Enhancement of asynch engine This PR enhances the asynch engine to enable the concenpt of delayed or paused tasks. Tasks can be added to the engine in a queue separate from the main work queue. As delayed tasks are added, a resumer is notified and will periodically start the tasks up. Delayed tasks can also be started by engine clients. Using this, pkg/api/provision and deprovision were enhanced to determine if a provision or deprovision action needs to wait for another operation to finish. If so, they submit delayed tasks, either wait_for_parent or wait_for_children, whichever is appropriate. Those new jobs will make the same determination, and either readd themselves as delayed tasks, or initiate the task that would otherwise originally have been created. * Added bad parent response message * Merge master into feature branch (#209) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Few minor corrections to the Quickstart (#136) * Correcting the az ad sp delete command * Update quickstart-minikube.md Remove a leading $ from a few bash commands. copy paste on mac is messed up :-) * Update quickstart-minikube.md Clarification around logging in to WordPress * restoring prompts in quickstart * minor punctuation fix * removing distracting echo command * make prompts used consistently * kick of redis tests sooner (#155) * s/module lifecycle tests/service lifecycle tests/g (#156) * make storage know how to deal with service-specific types (#153) * don't use codec where we don't need it anymore (#161) * Updating CircleCI to define DOCKER_REPO env var (#167) * Updating CircleCI to define REGISTRY env var We previously set the REGISTRY environment variable in a deploy script. When that was removed to streamline the release process, we need to define it in Circle. Added two environment blocks to the publish-rc-images and publish-release-images jobs. Fixes: 145 * Qualified registry with docker.io * Changed REGISTRY to DOCKER_REPO * s/context/details/g (#163) * make fake bind/unbind functionality not rely on standard context * collapse standard provisioning context onto instance * do away with (sort of) redundant standard provisioning parameters * s/provisioning context/instance details/ * s/binding context/binding details/ * fix error handling when standard params aren't expected types * add comment that adds some clarity to instance comparisons * don't store credentials as their own field (#169) * quickstart fixes (#174) * adding missing docker pushes to release process * Adding build flags to inject the version and commit (#173) * Adding build flags to inject the version and commit This commit injects the version and commit to the OSBA binary. If there is no version set, then sets the version to “devel” * Using new variable for version * Removing redundant setting for main.commit * Printing version and commit on broker startup * Adding fields to the log message on startup * Removing build flags on the CLI builds * broker doesn't need to hang on to codec (#180) * remove quotes from devel pseudo-version string (#184) Merging without CI. * fix bug decoding tags (#183) * move version info to avoid import cycle later (#185) * Adding user-agent string to requests to ARM (#172) * Adding user-agent string to requests to ARM * get version for user agent string from version package * use separate redis clients for storage and async engine (#182) * Adding ability to filter lifecycle tests (#181) * Adding ability to filter lifecycle tests This adds the ability to filter lifecycle tests using an environment variable (TEST_MODULES). Via this variable, the developer can provide a comma delimited list of module names and only the test cases matching that module will be enabled. If the variable is empty, all tests will run. If an invalid module name is provided, no tests will run. * fix typo and lint error * remove unnecessary funtion * use redis namespaces (#188) * use redis namespaces for instances and bindings * use redis namespaces in async engine * start using lightweight-docker-go image for development tasks (#190) * simplify ci config (#194) * README work (#196) * README formatting + one cmdline fix * Link fixes * Revert heading to title case * append instead of replacing user-agent (#198) * append instead of replacing user-agent * fix lint errors * add deferred task execution to async package (#208) achieving this involved a significant refactor of the entire package. the good news is that the tests are improved beyond what we had before, so i think we can be more confident in the refactored package than the original. * Enhancement of asynch engine This PR enhances the asynch engine to enable the concenpt of delayed or paused tasks. Tasks can be added to the engine in a queue separate from the main work queue. As delayed tasks are added, a resumer is notified and will periodically start the tasks up. Delayed tasks can also be started by engine clients. Using this, pkg/api/provision and deprovision were enhanced to determine if a provision or deprovision action needs to wait for another operation to finish. If so, they submit delayed tasks, either wait_for_parent or wait_for_children, whichever is appropriate. Those new jobs will make the same determination, and either readd themselves as delayed tasks, or initiate the task that would otherwise originally have been created. * Remove resumer changes * removed references to submitdelayed task * Fixed debug statements that got clobbered in merge * Debug message * Fixed response code * Whitespace * Code review comments * Code review comments * Refactor based on code-review comments. * Code review comments * Lint fixes * Enhanced contrib/k8s/examples (#213) * Enhanced contrib/k8s/examples to illustrate the use of the new SQLDB catalog * Spelling errors * merge latest from master to poc branch (#232) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Few minor corrections to the Quickstart (#136) * Correcting the az ad sp delete command * Update quickstart-minikube.md Remove a leading $ from a few bash commands. copy paste on mac is messed up :-) * Update quickstart-minikube.md Clarification around logging in to WordPress * restoring prompts in quickstart * minor punctuation fix * removing distracting echo command * make prompts used consistently * kick of redis tests sooner (#155) * s/module lifecycle tests/service lifecycle tests/g (#156) * make storage know how to deal with service-specific types (#153) * don't use codec where we don't need it anymore (#161) * Updating CircleCI to define DOCKER_REPO env var (#167) * Updating CircleCI to define REGISTRY env var We previously set the REGISTRY environment variable in a deploy script. When that was removed to streamline the release process, we need to define it in Circle. Added two environment blocks to the publish-rc-images and publish-release-images jobs. Fixes: 145 * Qualified registry with docker.io * Changed REGISTRY to DOCKER_REPO * s/context/details/g (#163) * make fake bind/unbind functionality not rely on standard context * collapse standard provisioning context onto instance * do away with (sort of) redundant standard provisioning parameters * s/provisioning context/instance details/ * s/binding context/binding details/ * fix error handling when standard params aren't expected types * add comment that adds some clarity to instance comparisons * don't store credentials as their own field (#169) * quickstart fixes (#174) * adding missing docker pushes to release process * Adding build flags to inject the version and commit (#173) * Adding build flags to inject the version and commit This commit injects the version and commit to the OSBA binary. If there is no version set, then sets the version to “devel” * Using new variable for version * Removing redundant setting for main.commit * Printing version and commit on broker startup * Adding fields to the log message on startup * Removing build flags on the CLI builds * broker doesn't need to hang on to codec (#180) * remove quotes from devel pseudo-version string (#184) Merging without CI. * fix bug decoding tags (#183) * move version info to avoid import cycle later (#185) * Adding user-agent string to requests to ARM (#172) * Adding user-agent string to requests to ARM * get version for user agent string from version package * use separate redis clients for storage and async engine (#182) * Adding ability to filter lifecycle tests (#181) * Adding ability to filter lifecycle tests This adds the ability to filter lifecycle tests using an environment variable (TEST_MODULES). Via this variable, the developer can provide a comma delimited list of module names and only the test cases matching that module will be enabled. If the variable is empty, all tests will run. If an invalid module name is provided, no tests will run. * fix typo and lint error * remove unnecessary funtion * use redis namespaces (#188) * use redis namespaces for instances and bindings * use redis namespaces in async engine * start using lightweight-docker-go image for development tasks (#190) * simplify ci config (#194) * README work (#196) * README formatting + one cmdline fix * Link fixes * Revert heading to title case * append instead of replacing user-agent (#198) * append instead of replacing user-agent * fix lint errors * add deferred task execution to async package (#208) achieving this involved a significant refactor of the entire package. the good news is that the tests are improved beyond what we had before, so i think we can be more confident in the refactored package than the original. * reorganize async pakage/sub-packages more sanely (#210) * change provision/deprovision task names (#211) The are imperative and more accurate. * Add async support for "follow-up" tasks (#212) * make async job functions take task as argument * make async job functions return tasks * bonus: fix more task names * make async task executor submit follow-up tasks * make broker async processes leverage new async follow-up tasks feature * ignore .vscode/ (#221) * more async refactoring (#219) * break up worker.go and worker_test.go into multiple files * simplify heart * simplify cleaner * move heart from worker to engine * remove receive/execute/watch from worker to engine * let task receivers return right after fatal error * let task executor return right after fatal error * very minor comment cleanup * fix lint errors by removing unused error types * add svc and plan to instance during retrieval (#220) * retrieve svc when instance is retrieved * retrieve plan when instance is retrieved * improve provision step function signature * improve deprovision step function signature * improve update step function signature * lifecycles tests: set service and plan on instance * Changes to mysql bind. (#222) Change grant string in mysql bind to be bound to database Removed global privs Added Execute Fixes #197 * Initial roadmap submission (#226) * Initial Roadmap * Publish initial roadmap for OSBA Closes #109 * Spelling * Review comments and wordsmithing. * Refactor for generic filter capabilities (#218) * WIP: Proposal to refactor for generic filter capabilities 1.) Create new api/filter structure 2.) Refactored authenticator stuff there 3.) Created new interfaces: filters.Filter and filters.Chain 4.) Made basic/always authenticator implement filters.Filter interface * Fixed filename type-o. * Fixed filename type-o * Renamed Filter method to Execute. * Added test for filter chain * Refactor and style fixes * ignore .vscode/ (#221) * more async refactoring (#219) * break up worker.go and worker_test.go into multiple files * simplify heart * simplify cleaner * move heart from worker to engine * remove receive/execute/watch from worker to engine * let task receivers return right after fatal error * let task executor return right after fatal error * very minor comment cleanup * fix lint errors by removing unused error types * add svc and plan to instance during retrieval (#220) * retrieve svc when instance is retrieved * retrieve plan when instance is retrieved * improve provision step function signature * improve deprovision step function signature * improve update step function signature * lifecycles tests: set service and plan on instance * changed filters package to filter * Lint failure * Refactor filter package a little and re-add Chain interface * a few filter tweaks * Change how the database name for cosmos db is generated (#228) * Change the generation of the database name. A limitation in the current ARM based CosmosDB provisoning limits the length of the database name to 50 characters, including the name provided, the location name and a dash. In some situations, this means a UUID is invalid. If the length will be too much, this patch will randomly generate a string of the available length instead of using a UUID. * Refactored to use generate package * fix async cleaner (#233) * bug fix: fix wait again on children not deprovisioned * bug fix: it's ok for an instance's parent to not be found * Code comments
jeremyrickard
added a commit
to jeremyrickard/open-service-broker-azure
that referenced
this issue
Feb 6, 2018
* WIP Refactoring sqldb to POC (#142) * Refactoring sqldb to POC #124 Removing the "existing server" logic from the current sqldb module. This entails removing it from the provision/deprovision file and removing it from the types. Also removed the config.go file and updated the broker to not load that. We will need another way to provide/register existing servers. We will reimplement this behavior in a new service/service manager. * fix minor lint failure * Feature branch merge (#154) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Sqldb catalog poc merge master (#158) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Few minor corrections to the Quickstart (#136) * Correcting the az ad sp delete command * Update quickstart-minikube.md Remove a leading $ from a few bash commands. copy paste on mac is messed up :-) * Update quickstart-minikube.md Clarification around logging in to WordPress * restoring prompts in quickstart * minor punctuation fix * removing distracting echo command * make prompts used consistently * kick of redis tests sooner (#155) * s/module lifecycle tests/service lifecycle tests/g (#156) * make storage know how to deal with service-specific types (#153) * make provision and deprovision steps receive a reference instance (#160) * Merge master to feature (#175) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Few minor corrections to the Quickstart (#136) * Correcting the az ad sp delete command * Update quickstart-minikube.md Remove a leading $ from a few bash commands. copy paste on mac is messed up :-) * Update quickstart-minikube.md Clarification around logging in to WordPress * restoring prompts in quickstart * minor punctuation fix * removing distracting echo command * make prompts used consistently * kick of redis tests sooner (#155) * s/module lifecycle tests/service lifecycle tests/g (#156) * make storage know how to deal with service-specific types (#153) * don't use codec where we don't need it anymore (#161) * Updating CircleCI to define DOCKER_REPO env var (#167) * Updating CircleCI to define REGISTRY env var We previously set the REGISTRY environment variable in a deploy script. When that was removed to streamline the release process, we need to define it in Circle. Added two environment blocks to the publish-rc-images and publish-release-images jobs. Fixes: 145 * Qualified registry with docker.io * Changed REGISTRY to DOCKER_REPO * s/context/details/g (#163) * make fake bind/unbind functionality not rely on standard context * collapse standard provisioning context onto instance * do away with (sort of) redundant standard provisioning parameters * s/provisioning context/instance details/ * s/binding context/binding details/ * fix error handling when standard params aren't expected types * add comment that adds some clarity to instance comparisons * don't store credentials as their own field (#169) * quickstart fixes (#174) * adding missing docker pushes to release process * minor merge nit fixes * Decomposition of single service into multiple services (#176) * Decomposition of single service into multiple services 1.) This renames the existing service manager into a service to implement the all in one scenario (existing) 2.) Next, two new services are added (vm only and db only) 3.) Each lifecycle operation go file (bind, deprovision, provision, unbind) now declares a method for each service manager struct 4.) Added new services to the catalog 5.) DB only is largely TBD implemented, took a crack at starting the provision. 6.) Refactored catalog/service interface to add GetBindable() method 7.) Modified tests to only execute binding related tests if plan is bindable * Small spelling correction * Implemented database only lifecycle methods and lint fix. * DRY refactoring * Review comments and lint fix * add ref instance as instance attribute (#178) * create a relationship between two services (#179) * Refactor sqldb module to use parent instances in binding (#186) * Refactor sqldb module to use parent instances in binding Small refactor to decouple shared instance details and leverage parent instance for binding/unbinding. Further evolution for #124 * Updated error to be consistent. * merge latest from master (#187) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Few minor corrections to the Quickstart (#136) * Correcting the az ad sp delete command * Update quickstart-minikube.md Remove a leading $ from a few bash commands. copy paste on mac is messed up :-) * Update quickstart-minikube.md Clarification around logging in to WordPress * restoring prompts in quickstart * minor punctuation fix * removing distracting echo command * make prompts used consistently * kick of redis tests sooner (#155) * s/module lifecycle tests/service lifecycle tests/g (#156) * make storage know how to deal with service-specific types (#153) * don't use codec where we don't need it anymore (#161) * Updating CircleCI to define DOCKER_REPO env var (#167) * Updating CircleCI to define REGISTRY env var We previously set the REGISTRY environment variable in a deploy script. When that was removed to streamline the release process, we need to define it in Circle. Added two environment blocks to the publish-rc-images and publish-release-images jobs. Fixes: 145 * Qualified registry with docker.io * Changed REGISTRY to DOCKER_REPO * s/context/details/g (#163) * make fake bind/unbind functionality not rely on standard context * collapse standard provisioning context onto instance * do away with (sort of) redundant standard provisioning parameters * s/provisioning context/instance details/ * s/binding context/binding details/ * fix error handling when standard params aren't expected types * add comment that adds some clarity to instance comparisons * don't store credentials as their own field (#169) * quickstart fixes (#174) * adding missing docker pushes to release process * Adding build flags to inject the version and commit (#173) * Adding build flags to inject the version and commit This commit injects the version and commit to the OSBA binary. If there is no version set, then sets the version to “devel” * Using new variable for version * Removing redundant setting for main.commit * Printing version and commit on broker startup * Adding fields to the log message on startup * Removing build flags on the CLI builds * broker doesn't need to hang on to codec (#180) * remove quotes from devel pseudo-version string (#184) Merging without CI. * fix bug decoding tags (#183) * move version info to avoid import cycle later (#185) * Adding user-agent string to requests to ARM (#172) * Adding user-agent string to requests to ARM * get version for user agent string from version package * use separate redis clients for storage and async engine (#182) * Adding ability to filter lifecycle tests (#181) * Adding ability to filter lifecycle tests This adds the ability to filter lifecycle tests using an environment variable (TEST_MODULES). Via this variable, the developer can provide a comma delimited list of module names and only the test cases matching that module will be enabled. If the variable is empty, all tests will run. If an invalid module name is provided, no tests will run. * fix typo and lint error * remove unnecessary funtion * Merge master (#189) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Few minor corrections to the Quickstart (#136) * Correcting the az ad sp delete command * Update quickstart-minikube.md Remove a leading $ from a few bash commands. copy paste on mac is messed up :-) * Update quickstart-minikube.md Clarification around logging in to WordPress * restoring prompts in quickstart * minor punctuation fix * removing distracting echo command * make prompts used consistently * kick of redis tests sooner (#155) * s/module lifecycle tests/service lifecycle tests/g (#156) * make storage know how to deal with service-specific types (#153) * don't use codec where we don't need it anymore (#161) * Updating CircleCI to define DOCKER_REPO env var (#167) * Updating CircleCI to define REGISTRY env var We previously set the REGISTRY environment variable in a deploy script. When that was removed to streamline the release process, we need to define it in Circle. Added two environment blocks to the publish-rc-images and publish-release-images jobs. Fixes: 145 * Qualified registry with docker.io * Changed REGISTRY to DOCKER_REPO * s/context/details/g (#163) * make fake bind/unbind functionality not rely on standard context * collapse standard provisioning context onto instance * do away with (sort of) redundant standard provisioning parameters * s/provisioning context/instance details/ * s/binding context/binding details/ * fix error handling when standard params aren't expected types * add comment that adds some clarity to instance comparisons * don't store credentials as their own field (#169) * quickstart fixes (#174) * adding missing docker pushes to release process * Adding build flags to inject the version and commit (#173) * Adding build flags to inject the version and commit This commit injects the version and commit to the OSBA binary. If there is no version set, then sets the version to “devel” * Using new variable for version * Removing redundant setting for main.commit * Printing version and commit on broker startup * Adding fields to the log message on startup * Removing build flags on the CLI builds * broker doesn't need to hang on to codec (#180) * remove quotes from devel pseudo-version string (#184) Merging without CI. * fix bug decoding tags (#183) * move version info to avoid import cycle later (#185) * Adding user-agent string to requests to ARM (#172) * Adding user-agent string to requests to ARM * get version for user agent string from version package * use separate redis clients for storage and async engine (#182) * Adding ability to filter lifecycle tests (#181) * Adding ability to filter lifecycle tests This adds the ability to filter lifecycle tests using an environment variable (TEST_MODULES). Via this variable, the developer can provide a comma delimited list of module names and only the test cases matching that module will be enabled. If the variable is empty, all tests will run. If an invalid module name is provided, no tests will run. * fix typo and lint error * remove unnecessary funtion * use redis namespaces (#188) * use redis namespaces for instances and bindings * use redis namespaces in async engine * validate alias and parentAlias (#191) * Update lifecycle tests to run mssql db only (#192) * Update lifecycle tests to run mssql db only 1.) Small refactor to test_case_test.go to break provision into a reusable function 2.) Modified setup function signature to include a *service.Instance to use as a parent instance 3.) Added setup function in mssql tests 4.) Added lifecycle test for DB only 5.) Fixed db only privision bug Implemnts #124 * Lint fixes * test refactor to support "child tests" * Small rename * Test description update * handle storage aspects of instance aliases (#195) * merge latest from master (#199) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Few minor corrections to the Quickstart (#136) * Correcting the az ad sp delete command * Update quickstart-minikube.md Remove a leading $ from a few bash commands. copy paste on mac is messed up :-) * Update quickstart-minikube.md Clarification around logging in to WordPress * restoring prompts in quickstart * minor punctuation fix * removing distracting echo command * make prompts used consistently * kick of redis tests sooner (#155) * s/module lifecycle tests/service lifecycle tests/g (#156) * make storage know how to deal with service-specific types (#153) * don't use codec where we don't need it anymore (#161) * Updating CircleCI to define DOCKER_REPO env var (#167) * Updating CircleCI to define REGISTRY env var We previously set the REGISTRY environment variable in a deploy script. When that was removed to streamline the release process, we need to define it in Circle. Added two environment blocks to the publish-rc-images and publish-release-images jobs. Fixes: 145 * Qualified registry with docker.io * Changed REGISTRY to DOCKER_REPO * s/context/details/g (#163) * make fake bind/unbind functionality not rely on standard context * collapse standard provisioning context onto instance * do away with (sort of) redundant standard provisioning parameters * s/provisioning context/instance details/ * s/binding context/binding details/ * fix error handling when standard params aren't expected types * add comment that adds some clarity to instance comparisons * don't store credentials as their own field (#169) * quickstart fixes (#174) * adding missing docker pushes to release process * Adding build flags to inject the version and commit (#173) * Adding build flags to inject the version and commit This commit injects the version and commit to the OSBA binary. If there is no version set, then sets the version to “devel” * Using new variable for version * Removing redundant setting for main.commit * Printing version and commit on broker startup * Adding fields to the log message on startup * Removing build flags on the CLI builds * broker doesn't need to hang on to codec (#180) * remove quotes from devel pseudo-version string (#184) Merging without CI. * fix bug decoding tags (#183) * move version info to avoid import cycle later (#185) * Adding user-agent string to requests to ARM (#172) * Adding user-agent string to requests to ARM * get version for user agent string from version package * use separate redis clients for storage and async engine (#182) * Adding ability to filter lifecycle tests (#181) * Adding ability to filter lifecycle tests This adds the ability to filter lifecycle tests using an environment variable (TEST_MODULES). Via this variable, the developer can provide a comma delimited list of module names and only the test cases matching that module will be enabled. If the variable is empty, all tests will run. If an invalid module name is provided, no tests will run. * fix typo and lint error * remove unnecessary funtion * use redis namespaces (#188) * use redis namespaces for instances and bindings * use redis namespaces in async engine * start using lightweight-docker-go image for development tasks (#190) * simplify ci config (#194) * populate parent when retrieving instance (#200) * parent child count (#201) * fix arg name * break up storage test cases better * track number of children belonging to an instance * Merge master into feature branch (#209) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Few minor corrections to the Quickstart (#136) * Correcting the az ad sp delete command * Update quickstart-minikube.md Remove a leading $ from a few bash commands. copy paste on mac is messed up :-) * Update quickstart-minikube.md Clarification around logging in to WordPress * restoring prompts in quickstart * minor punctuation fix * removing distracting echo command * make prompts used consistently * kick of redis tests sooner (#155) * s/module lifecycle tests/service lifecycle tests/g (#156) * make storage know how to deal with service-specific types (#153) * don't use codec where we don't need it anymore (#161) * Updating CircleCI to define DOCKER_REPO env var (#167) * Updating CircleCI to define REGISTRY env var We previously set the REGISTRY environment variable in a deploy script. When that was removed to streamline the release process, we need to define it in Circle. Added two environment blocks to the publish-rc-images and publish-release-images jobs. Fixes: 145 * Qualified registry with docker.io * Changed REGISTRY to DOCKER_REPO * s/context/details/g (#163) * make fake bind/unbind functionality not rely on standard context * collapse standard provisioning context onto instance * do away with (sort of) redundant standard provisioning parameters * s/provisioning context/instance details/ * s/binding context/binding details/ * fix error handling when standard params aren't expected types * add comment that adds some clarity to instance comparisons * don't store credentials as their own field (#169) * quickstart fixes (#174) * adding missing docker pushes to release process * Adding build flags to inject the version and commit (#173) * Adding build flags to inject the version and commit This commit injects the version and commit to the OSBA binary. If there is no version set, then sets the version to “devel” * Using new variable for version * Removing redundant setting for main.commit * Printing version and commit on broker startup * Adding fields to the log message on startup * Removing build flags on the CLI builds * broker doesn't need to hang on to codec (#180) * remove quotes from devel pseudo-version string (#184) Merging without CI. * fix bug decoding tags (#183) * move version info to avoid import cycle later (#185) * Adding user-agent string to requests to ARM (#172) * Adding user-agent string to requests to ARM * get version for user agent string from version package * use separate redis clients for storage and async engine (#182) * Adding ability to filter lifecycle tests (#181) * Adding ability to filter lifecycle tests This adds the ability to filter lifecycle tests using an environment variable (TEST_MODULES). Via this variable, the developer can provide a comma delimited list of module names and only the test cases matching that module will be enabled. If the variable is empty, all tests will run. If an invalid module name is provided, no tests will run. * fix typo and lint error * remove unnecessary funtion * use redis namespaces (#188) * use redis namespaces for instances and bindings * use redis namespaces in async engine * start using lightweight-docker-go image for development tasks (#190) * simplify ci config (#194) * README work (#196) * README formatting + one cmdline fix * Link fixes * Revert heading to title case * append instead of replacing user-agent (#198) * append instead of replacing user-agent * fix lint errors * add deferred task execution to async package (#208) achieving this involved a significant refactor of the entire package. the good news is that the tests are improved beyond what we had before, so i think we can be more confident in the refactored package than the original. * Provision and Deprovision enhancements to use new async features (#204) * Enhancement of asynch engine This PR enhances the asynch engine to enable the concenpt of delayed or paused tasks. Tasks can be added to the engine in a queue separate from the main work queue. As delayed tasks are added, a resumer is notified and will periodically start the tasks up. Delayed tasks can also be started by engine clients. Using this, pkg/api/provision and deprovision were enhanced to determine if a provision or deprovision action needs to wait for another operation to finish. If so, they submit delayed tasks, either wait_for_parent or wait_for_children, whichever is appropriate. Those new jobs will make the same determination, and either readd themselves as delayed tasks, or initiate the task that would otherwise originally have been created. * Added bad parent response message * Merge master into feature branch (#209) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Few minor corrections to the Quickstart (#136) * Correcting the az ad sp delete command * Update quickstart-minikube.md Remove a leading $ from a few bash commands. copy paste on mac is messed up :-) * Update quickstart-minikube.md Clarification around logging in to WordPress * restoring prompts in quickstart * minor punctuation fix * removing distracting echo command * make prompts used consistently * kick of redis tests sooner (#155) * s/module lifecycle tests/service lifecycle tests/g (#156) * make storage know how to deal with service-specific types (#153) * don't use codec where we don't need it anymore (#161) * Updating CircleCI to define DOCKER_REPO env var (#167) * Updating CircleCI to define REGISTRY env var We previously set the REGISTRY environment variable in a deploy script. When that was removed to streamline the release process, we need to define it in Circle. Added two environment blocks to the publish-rc-images and publish-release-images jobs. Fixes: 145 * Qualified registry with docker.io * Changed REGISTRY to DOCKER_REPO * s/context/details/g (#163) * make fake bind/unbind functionality not rely on standard context * collapse standard provisioning context onto instance * do away with (sort of) redundant standard provisioning parameters * s/provisioning context/instance details/ * s/binding context/binding details/ * fix error handling when standard params aren't expected types * add comment that adds some clarity to instance comparisons * don't store credentials as their own field (#169) * quickstart fixes (#174) * adding missing docker pushes to release process * Adding build flags to inject the version and commit (#173) * Adding build flags to inject the version and commit This commit injects the version and commit to the OSBA binary. If there is no version set, then sets the version to “devel” * Using new variable for version * Removing redundant setting for main.commit * Printing version and commit on broker startup * Adding fields to the log message on startup * Removing build flags on the CLI builds * broker doesn't need to hang on to codec (#180) * remove quotes from devel pseudo-version string (#184) Merging without CI. * fix bug decoding tags (#183) * move version info to avoid import cycle later (#185) * Adding user-agent string to requests to ARM (#172) * Adding user-agent string to requests to ARM * get version for user agent string from version package * use separate redis clients for storage and async engine (#182) * Adding ability to filter lifecycle tests (#181) * Adding ability to filter lifecycle tests This adds the ability to filter lifecycle tests using an environment variable (TEST_MODULES). Via this variable, the developer can provide a comma delimited list of module names and only the test cases matching that module will be enabled. If the variable is empty, all tests will run. If an invalid module name is provided, no tests will run. * fix typo and lint error * remove unnecessary funtion * use redis namespaces (#188) * use redis namespaces for instances and bindings * use redis namespaces in async engine * start using lightweight-docker-go image for development tasks (#190) * simplify ci config (#194) * README work (#196) * README formatting + one cmdline fix * Link fixes * Revert heading to title case * append instead of replacing user-agent (#198) * append instead of replacing user-agent * fix lint errors * add deferred task execution to async package (#208) achieving this involved a significant refactor of the entire package. the good news is that the tests are improved beyond what we had before, so i think we can be more confident in the refactored package than the original. * Enhancement of asynch engine This PR enhances the asynch engine to enable the concenpt of delayed or paused tasks. Tasks can be added to the engine in a queue separate from the main work queue. As delayed tasks are added, a resumer is notified and will periodically start the tasks up. Delayed tasks can also be started by engine clients. Using this, pkg/api/provision and deprovision were enhanced to determine if a provision or deprovision action needs to wait for another operation to finish. If so, they submit delayed tasks, either wait_for_parent or wait_for_children, whichever is appropriate. Those new jobs will make the same determination, and either readd themselves as delayed tasks, or initiate the task that would otherwise originally have been created. * Remove resumer changes * removed references to submitdelayed task * Fixed debug statements that got clobbered in merge * Debug message * Fixed response code * Whitespace * Code review comments * Code review comments * Refactor based on code-review comments. * Code review comments * Lint fixes * Enhanced contrib/k8s/examples (#213) * Enhanced contrib/k8s/examples to illustrate the use of the new SQLDB catalog * Spelling errors * merge latest from master to poc branch (#232) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Few minor corrections to the Quickstart (#136) * Correcting the az ad sp delete command * Update quickstart-minikube.md Remove a leading $ from a few bash commands. copy paste on mac is messed up :-) * Update quickstart-minikube.md Clarification around logging in to WordPress * restoring prompts in quickstart * minor punctuation fix * removing distracting echo command * make prompts used consistently * kick of redis tests sooner (#155) * s/module lifecycle tests/service lifecycle tests/g (#156) * make storage know how to deal with service-specific types (#153) * don't use codec where we don't need it anymore (#161) * Updating CircleCI to define DOCKER_REPO env var (#167) * Updating CircleCI to define REGISTRY env var We previously set the REGISTRY environment variable in a deploy script. When that was removed to streamline the release process, we need to define it in Circle. Added two environment blocks to the publish-rc-images and publish-release-images jobs. Fixes: 145 * Qualified registry with docker.io * Changed REGISTRY to DOCKER_REPO * s/context/details/g (#163) * make fake bind/unbind functionality not rely on standard context * collapse standard provisioning context onto instance * do away with (sort of) redundant standard provisioning parameters * s/provisioning context/instance details/ * s/binding context/binding details/ * fix error handling when standard params aren't expected types * add comment that adds some clarity to instance comparisons * don't store credentials as their own field (#169) * quickstart fixes (#174) * adding missing docker pushes to release process * Adding build flags to inject the version and commit (#173) * Adding build flags to inject the version and commit This commit injects the version and commit to the OSBA binary. If there is no version set, then sets the version to “devel” * Using new variable for version * Removing redundant setting for main.commit * Printing version and commit on broker startup * Adding fields to the log message on startup * Removing build flags on the CLI builds * broker doesn't need to hang on to codec (#180) * remove quotes from devel pseudo-version string (#184) Merging without CI. * fix bug decoding tags (#183) * move version info to avoid import cycle later (#185) * Adding user-agent string to requests to ARM (#172) * Adding user-agent string to requests to ARM * get version for user agent string from version package * use separate redis clients for storage and async engine (#182) * Adding ability to filter lifecycle tests (#181) * Adding ability to filter lifecycle tests This adds the ability to filter lifecycle tests using an environment variable (TEST_MODULES). Via this variable, the developer can provide a comma delimited list of module names and only the test cases matching that module will be enabled. If the variable is empty, all tests will run. If an invalid module name is provided, no tests will run. * fix typo and lint error * remove unnecessary funtion * use redis namespaces (#188) * use redis namespaces for instances and bindings * use redis namespaces in async engine * start using lightweight-docker-go image for development tasks (#190) * simplify ci config (#194) * README work (#196) * README formatting + one cmdline fix * Link fixes * Revert heading to title case * append instead of replacing user-agent (#198) * append instead of replacing user-agent * fix lint errors * add deferred task execution to async package (#208) achieving this involved a significant refactor of the entire package. the good news is that the tests are improved beyond what we had before, so i think we can be more confident in the refactored package than the original. * reorganize async pakage/sub-packages more sanely (#210) * change provision/deprovision task names (#211) The are imperative and more accurate. * Add async support for "follow-up" tasks (#212) * make async job functions take task as argument * make async job functions return tasks * bonus: fix more task names * make async task executor submit follow-up tasks * make broker async processes leverage new async follow-up tasks feature * ignore .vscode/ (#221) * more async refactoring (#219) * break up worker.go and worker_test.go into multiple files * simplify heart * simplify cleaner * move heart from worker to engine * remove receive/execute/watch from worker to engine * let task receivers return right after fatal error * let task executor return right after fatal error * very minor comment cleanup * fix lint errors by removing unused error types * add svc and plan to instance during retrieval (#220) * retrieve svc when instance is retrieved * retrieve plan when instance is retrieved * improve provision step function signature * improve deprovision step function signature * improve update step function signature * lifecycles tests: set service and plan on instance * Changes to mysql bind. (#222) Change grant string in mysql bind to be bound to database Removed global privs Added Execute Fixes #197 * Initial roadmap submission (#226) * Initial Roadmap * Publish initial roadmap for OSBA Closes #109 * Spelling * Review comments and wordsmithing. * Refactor for generic filter capabilities (#218) * WIP: Proposal to refactor for generic filter capabilities 1.) Create new api/filter structure 2.) Refactored authenticator stuff there 3.) Created new interfaces: filters.Filter and filters.Chain 4.) Made basic/always authenticator implement filters.Filter interface * Fixed filename type-o. * Fixed filename type-o * Renamed Filter method to Execute. * Added test for filter chain * Refactor and style fixes * ignore .vscode/ (#221) * more async refactoring (#219) * break up worker.go and worker_test.go into multiple files * simplify heart * simplify cleaner * move heart from worker to engine * remove receive/execute/watch from worker to engine * let task receivers return right after fatal error * let task executor return right after fatal error * very minor comment cleanup * fix lint errors by removing unused error types * add svc and plan to instance during retrieval (#220) * retrieve svc when instance is retrieved * retrieve plan when instance is retrieved * improve provision step function signature * improve deprovision step function signature * improve update step function signature * lifecycles tests: set service and plan on instance * changed filters package to filter * Lint failure * Refactor filter package a little and re-add Chain interface * a few filter tweaks * Change how the database name for cosmos db is generated (#228) * Change the generation of the database name. A limitation in the current ARM based CosmosDB provisoning limits the length of the database name to 50 characters, including the name provided, the location name and a dash. In some situations, this means a UUID is invalid. If the length will be too much, this patch will randomly generate a string of the available length instead of using a UUID. * Refactored to use generate package * fix async cleaner (#233) * bug fix: fix wait again on children not deprovisioned * bug fix: it's ok for an instance's parent to not be found * Code comments
krancour
pushed a commit
that referenced
this issue
Feb 6, 2018
* Replacing empty json responses with error messages. Issue #66 points out that our error messages don't help determine the cause of operation failures. This PR adds an informative response to locations where we are returning a 400 or a 409 so the user has some extra infromation to debug the failure. These should be inline with the Broker Errors section of the OSB spec. * Adding error messages to operation failures Fixes #66 * Parent child services (#243) * WIP Refactoring sqldb to POC (#142) * Refactoring sqldb to POC #124 Removing the "existing server" logic from the current sqldb module. This entails removing it from the provision/deprovision file and removing it from the types. Also removed the config.go file and updated the broker to not load that. We will need another way to provide/register existing servers. We will reimplement this behavior in a new service/service manager. * fix minor lint failure * Feature branch merge (#154) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Sqldb catalog poc merge master (#158) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Few minor corrections to the Quickstart (#136) * Correcting the az ad sp delete command * Update quickstart-minikube.md Remove a leading $ from a few bash commands. copy paste on mac is messed up :-) * Update quickstart-minikube.md Clarification around logging in to WordPress * restoring prompts in quickstart * minor punctuation fix * removing distracting echo command * make prompts used consistently * kick of redis tests sooner (#155) * s/module lifecycle tests/service lifecycle tests/g (#156) * make storage know how to deal with service-specific types (#153) * make provision and deprovision steps receive a reference instance (#160) * Merge master to feature (#175) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Few minor corrections to the Quickstart (#136) * Correcting the az ad sp delete command * Update quickstart-minikube.md Remove a leading $ from a few bash commands. copy paste on mac is messed up :-) * Update quickstart-minikube.md Clarification around logging in to WordPress * restoring prompts in quickstart * minor punctuation fix * removing distracting echo command * make prompts used consistently * kick of redis tests sooner (#155) * s/module lifecycle tests/service lifecycle tests/g (#156) * make storage know how to deal with service-specific types (#153) * don't use codec where we don't need it anymore (#161) * Updating CircleCI to define DOCKER_REPO env var (#167) * Updating CircleCI to define REGISTRY env var We previously set the REGISTRY environment variable in a deploy script. When that was removed to streamline the release process, we need to define it in Circle. Added two environment blocks to the publish-rc-images and publish-release-images jobs. Fixes: 145 * Qualified registry with docker.io * Changed REGISTRY to DOCKER_REPO * s/context/details/g (#163) * make fake bind/unbind functionality not rely on standard context * collapse standard provisioning context onto instance * do away with (sort of) redundant standard provisioning parameters * s/provisioning context/instance details/ * s/binding context/binding details/ * fix error handling when standard params aren't expected types * add comment that adds some clarity to instance comparisons * don't store credentials as their own field (#169) * quickstart fixes (#174) * adding missing docker pushes to release process * minor merge nit fixes * Decomposition of single service into multiple services (#176) * Decomposition of single service into multiple services 1.) This renames the existing service manager into a service to implement the all in one scenario (existing) 2.) Next, two new services are added (vm only and db only) 3.) Each lifecycle operation go file (bind, deprovision, provision, unbind) now declares a method for each service manager struct 4.) Added new services to the catalog 5.) DB only is largely TBD implemented, took a crack at starting the provision. 6.) Refactored catalog/service interface to add GetBindable() method 7.) Modified tests to only execute binding related tests if plan is bindable * Small spelling correction * Implemented database only lifecycle methods and lint fix. * DRY refactoring * Review comments and lint fix * add ref instance as instance attribute (#178) * create a relationship between two services (#179) * Refactor sqldb module to use parent instances in binding (#186) * Refactor sqldb module to use parent instances in binding Small refactor to decouple shared instance details and leverage parent instance for binding/unbinding. Further evolution for #124 * Updated error to be consistent. * merge latest from master (#187) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Few minor corrections to the Quickstart (#136) * Correcting the az ad sp delete command * Update quickstart-minikube.md Remove a leading $ from a few bash commands. copy paste on mac is messed up :-) * Update quickstart-minikube.md Clarification around logging in to WordPress * restoring prompts in quickstart * minor punctuation fix * removing distracting echo command * make prompts used consistently * kick of redis tests sooner (#155) * s/module lifecycle tests/service lifecycle tests/g (#156) * make storage know how to deal with service-specific types (#153) * don't use codec where we don't need it anymore (#161) * Updating CircleCI to define DOCKER_REPO env var (#167) * Updating CircleCI to define REGISTRY env var We previously set the REGISTRY environment variable in a deploy script. When that was removed to streamline the release process, we need to define it in Circle. Added two environment blocks to the publish-rc-images and publish-release-images jobs. Fixes: 145 * Qualified registry with docker.io * Changed REGISTRY to DOCKER_REPO * s/context/details/g (#163) * make fake bind/unbind functionality not rely on standard context * collapse standard provisioning context onto instance * do away with (sort of) redundant standard provisioning parameters * s/provisioning context/instance details/ * s/binding context/binding details/ * fix error handling when standard params aren't expected types * add comment that adds some clarity to instance comparisons * don't store credentials as their own field (#169) * quickstart fixes (#174) * adding missing docker pushes to release process * Adding build flags to inject the version and commit (#173) * Adding build flags to inject the version and commit This commit injects the version and commit to the OSBA binary. If there is no version set, then sets the version to “devel” * Using new variable for version * Removing redundant setting for main.commit * Printing version and commit on broker startup * Adding fields to the log message on startup * Removing build flags on the CLI builds * broker doesn't need to hang on to codec (#180) * remove quotes from devel pseudo-version string (#184) Merging without CI. * fix bug decoding tags (#183) * move version info to avoid import cycle later (#185) * Adding user-agent string to requests to ARM (#172) * Adding user-agent string to requests to ARM * get version for user agent string from version package * use separate redis clients for storage and async engine (#182) * Adding ability to filter lifecycle tests (#181) * Adding ability to filter lifecycle tests This adds the ability to filter lifecycle tests using an environment variable (TEST_MODULES). Via this variable, the developer can provide a comma delimited list of module names and only the test cases matching that module will be enabled. If the variable is empty, all tests will run. If an invalid module name is provided, no tests will run. * fix typo and lint error * remove unnecessary funtion * Merge master (#189) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Few minor corrections to the Quickstart (#136) * Correcting the az ad sp delete command * Update quickstart-minikube.md Remove a leading $ from a few bash commands. copy paste on mac is messed up :-) * Update quickstart-minikube.md Clarification around logging in to WordPress * restoring prompts in quickstart * minor punctuation fix * removing distracting echo command * make prompts used consistently * kick of redis tests sooner (#155) * s/module lifecycle tests/service lifecycle tests/g (#156) * make storage know how to deal with service-specific types (#153) * don't use codec where we don't need it anymore (#161) * Updating CircleCI to define DOCKER_REPO env var (#167) * Updating CircleCI to define REGISTRY env var We previously set the REGISTRY environment variable in a deploy script. When that was removed to streamline the release process, we need to define it in Circle. Added two environment blocks to the publish-rc-images and publish-release-images jobs. Fixes: 145 * Qualified registry with docker.io * Changed REGISTRY to DOCKER_REPO * s/context/details/g (#163) * make fake bind/unbind functionality not rely on standard context * collapse standard provisioning context onto instance * do away with (sort of) redundant standard provisioning parameters * s/provisioning context/instance details/ * s/binding context/binding details/ * fix error handling when standard params aren't expected types * add comment that adds some clarity to instance comparisons * don't store credentials as their own field (#169) * quickstart fixes (#174) * adding missing docker pushes to release process * Adding build flags to inject the version and commit (#173) * Adding build flags to inject the version and commit This commit injects the version and commit to the OSBA binary. If there is no version set, then sets the version to “devel” * Using new variable for version * Removing redundant setting for main.commit * Printing version and commit on broker startup * Adding fields to the log message on startup * Removing build flags on the CLI builds * broker doesn't need to hang on to codec (#180) * remove quotes from devel pseudo-version string (#184) Merging without CI. * fix bug decoding tags (#183) * move version info to avoid import cycle later (#185) * Adding user-agent string to requests to ARM (#172) * Adding user-agent string to requests to ARM * get version for user agent string from version package * use separate redis clients for storage and async engine (#182) * Adding ability to filter lifecycle tests (#181) * Adding ability to filter lifecycle tests This adds the ability to filter lifecycle tests using an environment variable (TEST_MODULES). Via this variable, the developer can provide a comma delimited list of module names and only the test cases matching that module will be enabled. If the variable is empty, all tests will run. If an invalid module name is provided, no tests will run. * fix typo and lint error * remove unnecessary funtion * use redis namespaces (#188) * use redis namespaces for instances and bindings * use redis namespaces in async engine * validate alias and parentAlias (#191) * Update lifecycle tests to run mssql db only (#192) * Update lifecycle tests to run mssql db only 1.) Small refactor to test_case_test.go to break provision into a reusable function 2.) Modified setup function signature to include a *service.Instance to use as a parent instance 3.) Added setup function in mssql tests 4.) Added lifecycle test for DB only 5.) Fixed db only privision bug Implemnts #124 * Lint fixes * test refactor to support "child tests" * Small rename * Test description update * handle storage aspects of instance aliases (#195) * merge latest from master (#199) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Few minor corrections to the Quickstart (#136) * Correcting the az ad sp delete command * Update quickstart-minikube.md Remove a leading $ from a few bash commands. copy paste on mac is messed up :-) * Update quickstart-minikube.md Clarification around logging in to WordPress * restoring prompts in quickstart * minor punctuation fix * removing distracting echo command * make prompts used consistently * kick of redis tests sooner (#155) * s/module lifecycle tests/service lifecycle tests/g (#156) * make storage know how to deal with service-specific types (#153) * don't use codec where we don't need it anymore (#161) * Updating CircleCI to define DOCKER_REPO env var (#167) * Updating CircleCI to define REGISTRY env var We previously set the REGISTRY environment variable in a deploy script. When that was removed to streamline the release process, we need to define it in Circle. Added two environment blocks to the publish-rc-images and publish-release-images jobs. Fixes: 145 * Qualified registry with docker.io * Changed REGISTRY to DOCKER_REPO * s/context/details/g (#163) * make fake bind/unbind functionality not rely on standard context * collapse standard provisioning context onto instance * do away with (sort of) redundant standard provisioning parameters * s/provisioning context/instance details/ * s/binding context/binding details/ * fix error handling when standard params aren't expected types * add comment that adds some clarity to instance comparisons * don't store credentials as their own field (#169) * quickstart fixes (#174) * adding missing docker pushes to release process * Adding build flags to inject the version and commit (#173) * Adding build flags to inject the version and commit This commit injects the version and commit to the OSBA binary. If there is no version set, then sets the version to “devel” * Using new variable for version * Removing redundant setting for main.commit * Printing version and commit on broker startup * Adding fields to the log message on startup * Removing build flags on the CLI builds * broker doesn't need to hang on to codec (#180) * remove quotes from devel pseudo-version string (#184) Merging without CI. * fix bug decoding tags (#183) * move version info to avoid import cycle later (#185) * Adding user-agent string to requests to ARM (#172) * Adding user-agent string to requests to ARM * get version for user agent string from version package * use separate redis clients for storage and async engine (#182) * Adding ability to filter lifecycle tests (#181) * Adding ability to filter lifecycle tests This adds the ability to filter lifecycle tests using an environment variable (TEST_MODULES). Via this variable, the developer can provide a comma delimited list of module names and only the test cases matching that module will be enabled. If the variable is empty, all tests will run. If an invalid module name is provided, no tests will run. * fix typo and lint error * remove unnecessary funtion * use redis namespaces (#188) * use redis namespaces for instances and bindings * use redis namespaces in async engine * start using lightweight-docker-go image for development tasks (#190) * simplify ci config (#194) * populate parent when retrieving instance (#200) * parent child count (#201) * fix arg name * break up storage test cases better * track number of children belonging to an instance * Merge master into feature branch (#209) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Few minor corrections to the Quickstart (#136) * Correcting the az ad sp delete command * Update quickstart-minikube.md Remove a leading $ from a few bash commands. copy paste on mac is messed up :-) * Update quickstart-minikube.md Clarification around logging in to WordPress * restoring prompts in quickstart * minor punctuation fix * removing distracting echo command * make prompts used consistently * kick of redis tests sooner (#155) * s/module lifecycle tests/service lifecycle tests/g (#156) * make storage know how to deal with service-specific types (#153) * don't use codec where we don't need it anymore (#161) * Updating CircleCI to define DOCKER_REPO env var (#167) * Updating CircleCI to define REGISTRY env var We previously set the REGISTRY environment variable in a deploy script. When that was removed to streamline the release process, we need to define it in Circle. Added two environment blocks to the publish-rc-images and publish-release-images jobs. Fixes: 145 * Qualified registry with docker.io * Changed REGISTRY to DOCKER_REPO * s/context/details/g (#163) * make fake bind/unbind functionality not rely on standard context * collapse standard provisioning context onto instance * do away with (sort of) redundant standard provisioning parameters * s/provisioning context/instance details/ * s/binding context/binding details/ * fix error handling when standard params aren't expected types * add comment that adds some clarity to instance comparisons * don't store credentials as their own field (#169) * quickstart fixes (#174) * adding missing docker pushes to release process * Adding build flags to inject the version and commit (#173) * Adding build flags to inject the version and commit This commit injects the version and commit to the OSBA binary. If there is no version set, then sets the version to “devel” * Using new variable for version * Removing redundant setting for main.commit * Printing version and commit on broker startup * Adding fields to the log message on startup * Removing build flags on the CLI builds * broker doesn't need to hang on to codec (#180) * remove quotes from devel pseudo-version string (#184) Merging without CI. * fix bug decoding tags (#183) * move version info to avoid import cycle later (#185) * Adding user-agent string to requests to ARM (#172) * Adding user-agent string to requests to ARM * get version for user agent string from version package * use separate redis clients for storage and async engine (#182) * Adding ability to filter lifecycle tests (#181) * Adding ability to filter lifecycle tests This adds the ability to filter lifecycle tests using an environment variable (TEST_MODULES). Via this variable, the developer can provide a comma delimited list of module names and only the test cases matching that module will be enabled. If the variable is empty, all tests will run. If an invalid module name is provided, no tests will run. * fix typo and lint error * remove unnecessary funtion * use redis namespaces (#188) * use redis namespaces for instances and bindings * use redis namespaces in async engine * start using lightweight-docker-go image for development tasks (#190) * simplify ci config (#194) * README work (#196) * README formatting + one cmdline fix * Link fixes * Revert heading to title case * append instead of replacing user-agent (#198) * append instead of replacing user-agent * fix lint errors * add deferred task execution to async package (#208) achieving this involved a significant refactor of the entire package. the good news is that the tests are improved beyond what we had before, so i think we can be more confident in the refactored package than the original. * Provision and Deprovision enhancements to use new async features (#204) * Enhancement of asynch engine This PR enhances the asynch engine to enable the concenpt of delayed or paused tasks. Tasks can be added to the engine in a queue separate from the main work queue. As delayed tasks are added, a resumer is notified and will periodically start the tasks up. Delayed tasks can also be started by engine clients. Using this, pkg/api/provision and deprovision were enhanced to determine if a provision or deprovision action needs to wait for another operation to finish. If so, they submit delayed tasks, either wait_for_parent or wait_for_children, whichever is appropriate. Those new jobs will make the same determination, and either readd themselves as delayed tasks, or initiate the task that would otherwise originally have been created. * Added bad parent response message * Merge master into feature branch (#209) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Few minor corrections to the Quickstart (#136) * Correcting the az ad sp delete command * Update quickstart-minikube.md Remove a leading $ from a few bash commands. copy paste on mac is messed up :-) * Update quickstart-minikube.md Clarification around logging in to WordPress * restoring prompts in quickstart * minor punctuation fix * removing distracting echo command * make prompts used consistently * kick of redis tests sooner (#155) * s/module lifecycle tests/service lifecycle tests/g (#156) * make storage know how to deal with service-specific types (#153) * don't use codec where we don't need it anymore (#161) * Updating CircleCI to define DOCKER_REPO env var (#167) * Updating CircleCI to define REGISTRY env var We previously set the REGISTRY environment variable in a deploy script. When that was removed to streamline the release process, we need to define it in Circle. Added two environment blocks to the publish-rc-images and publish-release-images jobs. Fixes: 145 * Qualified registry with docker.io * Changed REGISTRY to DOCKER_REPO * s/context/details/g (#163) * make fake bind/unbind functionality not rely on standard context * collapse standard provisioning context onto instance * do away with (sort of) redundant standard provisioning parameters * s/provisioning context/instance details/ * s/binding context/binding details/ * fix error handling when standard params aren't expected types * add comment that adds some clarity to instance comparisons * don't store credentials as their own field (#169) * quickstart fixes (#174) * adding missing docker pushes to release process * Adding build flags to inject the version and commit (#173) * Adding build flags to inject the version and commit This commit injects the version and commit to the OSBA binary. If there is no version set, then sets the version to “devel” * Using new variable for version * Removing redundant setting for main.commit * Printing version and commit on broker startup * Adding fields to the log message on startup * Removing build flags on the CLI builds * broker doesn't need to hang on to codec (#180) * remove quotes from devel pseudo-version string (#184) Merging without CI. * fix bug decoding tags (#183) * move version info to avoid import cycle later (#185) * Adding user-agent string to requests to ARM (#172) * Adding user-agent string to requests to ARM * get version for user agent string from version package * use separate redis clients for storage and async engine (#182) * Adding ability to filter lifecycle tests (#181) * Adding ability to filter lifecycle tests This adds the ability to filter lifecycle tests using an environment variable (TEST_MODULES). Via this variable, the developer can provide a comma delimited list of module names and only the test cases matching that module will be enabled. If the variable is empty, all tests will run. If an invalid module name is provided, no tests will run. * fix typo and lint error * remove unnecessary funtion * use redis namespaces (#188) * use redis namespaces for instances and bindings * use redis namespaces in async engine * start using lightweight-docker-go image for development tasks (#190) * simplify ci config (#194) * README work (#196) * README formatting + one cmdline fix * Link fixes * Revert heading to title case * append instead of replacing user-agent (#198) * append instead of replacing user-agent * fix lint errors * add deferred task execution to async package (#208) achieving this involved a significant refactor of the entire package. the good news is that the tests are improved beyond what we had before, so i think we can be more confident in the refactored package than the original. * Enhancement of asynch engine This PR enhances the asynch engine to enable the concenpt of delayed or paused tasks. Tasks can be added to the engine in a queue separate from the main work queue. As delayed tasks are added, a resumer is notified and will periodically start the tasks up. Delayed tasks can also be started by engine clients. Using this, pkg/api/provision and deprovision were enhanced to determine if a provision or deprovision action needs to wait for another operation to finish. If so, they submit delayed tasks, either wait_for_parent or wait_for_children, whichever is appropriate. Those new jobs will make the same determination, and either readd themselves as delayed tasks, or initiate the task that would otherwise originally have been created. * Remove resumer changes * removed references to submitdelayed task * Fixed debug statements that got clobbered in merge * Debug message * Fixed response code * Whitespace * Code review comments * Code review comments * Refactor based on code-review comments. * Code review comments * Lint fixes * Enhanced contrib/k8s/examples (#213) * Enhanced contrib/k8s/examples to illustrate the use of the new SQLDB catalog * Spelling errors * merge latest from master to poc branch (#232) * refactor event function signatures (#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (#152) * Refactor to support allowing firewall rules. Fixes: #146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Few minor corrections to the Quickstart (#136) * Correcting the az ad sp delete command * Update quickstart-minikube.md Remove a leading $ from a few bash commands. copy paste on mac is messed up :-) * Update quickstart-minikube.md Clarification around logging in to WordPress * restoring prompts in quickstart * minor punctuation fix * removing distracting echo command * make prompts used consistently * kick of redis tests sooner (#155) * s/module lifecycle tests/service lifecycle tests/g (#156) * make storage know how to deal with service-specific types (#153) * don't use codec where we don't need it anymore (#161) * Updating CircleCI to define DOCKER_REPO env var (#167) * Updating CircleCI to define REGISTRY env var We previously set the REGISTRY environment variable in a deploy script. When that was removed to streamline the release process, we need to define it in Circle. Added two environment blocks to the publish-rc-images and publish-release-images jobs. Fixes: 145 * Qualified registry with docker.io * Changed REGISTRY to DOCKER_REPO * s/context/details/g (#163) * make fake bind/unbind functionality not rely on standard context * collapse standard provisioning context onto instance * do away with (sort of) redundant standard provisioning parameters * s/provisioning context/instance details/ * s/binding context/binding details/ * fix error handling when standard params aren't expected types * add comment that adds some clarity to instance comparisons * don't store credentials as their own field (#169) * quickstart fixes (#174) * adding missing docker pushes to release process * Adding build flags to inject the version and commit (#173) * Adding build flags to inject the version and commit This commit injects the version and commit to the OSBA binary. If there is no version set, then sets the version to “devel” * Using new variable for version * Removing redundant setting for main.commit * Printing version and commit on broker startup * Adding fields to the log message on startup * Removing build flags on the CLI builds * broker doesn't need to hang on to codec (#180) * remove quotes from devel pseudo-version string (#184) Merging without CI. * fix bug decoding tags (#183) * move version info to avoid import cycle later (#185) * Adding user-agent string to requests to ARM (#172) * Adding user-agent string to requests to ARM * get version for user agent string from version package * use separate redis clients for storage and async engine (#182) * Adding ability to filter lifecycle tests (#181) * Adding ability to filter lifecycle tests This adds the ability to filter lifecycle tests using an environment variable (TEST_MODULES). Via this variable, the developer can provide a comma delimited list of module names and only the test cases matching that module will be enabled. If the variable is empty, all tests will run. If an invalid module name is provided, no tests will run. * fix typo and lint error * remove unnecessary funtion * use redis namespaces (#188) * use redis namespaces for instances and bindings * use redis namespaces in async engine * start using lightweight-docker-go image for development tasks (#190) * simplify ci config (#194) * README work (#196) * README formatting + one cmdline fix * Link fixes * Revert heading to title case * append instead of replacing user-agent (#198) * append instead of replacing user-agent * fix lint errors * add deferred task execution to async package (#208) achieving this involved a significant refactor of the entire package. the good news is that the tests are improved beyond what we had before, so i think we can be more confident in the refactored package than the original. * reorganize async pakage/sub-packages more sanely (#210) * change provision/deprovision task names (#211) The are imperative and more accurate. * Add async support for "follow-up" tasks (#212) * make async job functions take task as argument * make async job functions return tasks * bonus: fix more task names * make async task executor submit follow-up tasks * make broker async processes leverage new async follow-up tasks feature * ignore .vscode/ (#221) * more async refactoring (#219) * break up worker.go and worker_test.go into multiple files * simplify heart * simplify cleaner * move heart from worker to engine * remove receive/execute/watch from worker to engine * let task receivers return right after fatal error * let task executor return right after fatal error * very minor comment cleanup * fix lint errors by removing unused error types * add svc and plan to instance during retrieval (#220) * retrieve svc when instance is retrieved * retrieve plan when instance is retrieved * improve provision step function signature * improve deprovision step function signature * improve update step function signature * lifecycles tests: set service and plan on instance * Changes to mysql bind. (#222) Change grant string in mysql bind to be bound to database Removed global privs Added Execute Fixes #197 * Initial roadmap submission (#226) * Initial Roadmap * Publish initial roadmap for OSBA Closes #109 * Spelling * Review comments and wordsmithing. * Refactor for generic filter capabilities (#218) * WIP: Proposal to refactor for generic filter capabilities 1.) Create new api/filter structure 2.) Refactored authenticator stuff there 3.) Created new interfaces: filters.Filter and filters.Chain 4.) Made basic/always authenticator implement filters.Filter interface * Fixed filename type-o. * Fixed filename type-o * Renamed Filter method to Execute. * Added test for filter chain * Refactor and style fixes * ignore .vscode/ (#221) * more async refactoring (#219) * break up worker.go and worker_test.go into multiple files * simplify heart * simplify cleaner * move heart from worker to engine * remove receive/execute/watch from worker to engine * let task receivers return right after fatal error * let task executor return right after fatal error * very minor comment cleanup * fix lint errors by removing unused error types * add svc and plan to instance during retrieval (#220) * retrieve svc when instance is retrieved * retrieve plan when instance is retrieved * improve provision step function signature * improve deprovision step function signature * improve update step function signature * lifecycles tests: set service and plan on instance * changed filters package to filter * Lint failure * Refactor filter package a little and re-add Chain interface * a few filter tweaks * Change how the database name for cosmos db is generated (#228) * Change the generation of the database name. A limitation in the current ARM based CosmosDB provisoning limits the length of the database name to 50 characters, including the name provided, the location name and a dash. In some situations, this means a UUID is invalid. If the length will be too much, this patch will randomly generate a string of the available length instead of using a UUID. * Refactored to use generate package * fix async cleaner (#233) * bug fix: fix wait again on children not deprovisioned * bug fix: it's ok for an instance's parent to not be found * Code comments * Replacing empty json responses with error messages. Issue #66 points out that our error messages don't help determine the cause of operation failures. This PR adds an informative response to locations where we are returning a 400 or a 409 so the user has some extra infromation to debug the failure. These should be inline with the Broker Errors section of the OSB spec. * Adding error messages to operation failures Fixes #66 * Fixed compile error after merge
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Primarily, the roadmap should outline the path to stability for the broker itself and all its constituent modules.
The text was updated successfully, but these errors were encountered: