From 15f6959a260d68d9cc2d9541d30738b788f35a5a Mon Sep 17 00:00:00 2001 From: Leonard Goodell Date: Wed, 18 Jan 2023 13:27:50 -0700 Subject: [PATCH 1/4] feat: Create UCR for using URIs to load files Signed-off-by: Leonard Goodell --- docs_src/design/TOC.md | 17 ++++---- docs_src/design/ucr/URIs-for-Files.md | 56 +++++++++++++++++++++++++++ 2 files changed, 65 insertions(+), 8 deletions(-) create mode 100644 docs_src/design/ucr/URIs-for-Files.md diff --git a/docs_src/design/TOC.md b/docs_src/design/TOC.md index 6c79f3ef0f..7ee65de49a 100644 --- a/docs_src/design/TOC.md +++ b/docs_src/design/TOC.md @@ -5,15 +5,16 @@ !!! note UCRs are listed in alphabetical order by title. -| Name/Link | Short Description | -| ------------------------------------------------------------ |---------------------------------------------------------------------| -| [Common Configuration](./ucr/Common Configuration.md) | Use Case for having Common configuration used by all EdgeX services | -| [Device Parent-Child Relationships](./ucr/Device-Parent-Child-Relationships.md) | Use Case for Device Parent-Child Relationships | -| [Extending Device Data](./ucr/Extending-Device-Data.md) | Use Case for Extending of Device Data by Application Services | +| Name/Link | Short Description | +|-------------------------------------------------------------------------------------|---------------------------------------------------------------------| +| [Common Configuration](./ucr/Common Configuration.md) | Use Case for having Common configuration used by all EdgeX services | +| [Device Parent-Child Relationships](./ucr/Device-Parent-Child-Relationships.md) | Use Case for Device Parent-Child Relationships | +| [Extending Device Data](./ucr/Extending-Device-Data.md) | Use Case for Extending of Device Data by Application Services | | [Provision Watch via Device Metadata](./ucr/Provision-Watch-via-Device-Metadata.md) | Use Case for Provision Watching via Additional Device Metadata | -| [Record and Replay](./ucr/Record-and-Replay.md) | Use Case for Recording and Replaying event/readings | -| [System Events for Devices](./ucr/System-Events-for-Devices.md) | Use Case for System Events for Device add/update/delete | -| [Microservice Authentication](./ucr/Microservice-Authentication.md) | Use Case for Microservice Authentication | +| [Record and Replay](./ucr/Record-and-Replay.md) | Use Case for Recording and Replaying event/readings | +| [System Events for Devices](./ucr/System-Events-for-Devices.md) | Use Case for System Events for Device add/update/delete | +| [Microservice Authentication](./ucr/Microservice-Authentication.md) | Use Case for Microservice Authentication | +| [URIs for files](.//ucr/URIs-for-Files.md) | Use Case for loading service files from URIs | ## Architectural Design Records (ADRs) diff --git a/docs_src/design/ucr/URIs-for-Files.md b/docs_src/design/ucr/URIs-for-Files.md new file mode 100644 index 0000000000..4dab13daf1 --- /dev/null +++ b/docs_src/design/ucr/URIs-for-Files.md @@ -0,0 +1,56 @@ +## URIs for files +### Submitters +- Lenny Goodell (Intel) + +## Change Log +- [pending](URL of PR) (2023-01-18) + +### Market Segments +- All + +### Motivation +Deployment at scale, i.e. identical or almost identical deployments across many locations, would benefit from the ability to loaded service files from a central location. This would allow the maintainer to make changes once to a shared file and have them apply to all or a subset of deployments. The following are some EdgeX service files that would benefit for this capability: + +- [Unit of Measure](https://github.com/edgexfoundry/edgex-go/blob/v2.3.0/cmd/core-metadata/res/uom.toml) file used by Core Metadata + + - Location of this file is specified in configuration [here](https://github.com/edgexfoundry/edgex-go/blob/v2.3.0/cmd/core-metadata/res/configuration.toml#L50) + +- Service Configuration files + - Location of these files are currently defaulted to be `./res/configuration.toml`, but can be overridden via -cf/--configFile command line flag. + - The Common Configuration ADR adds a new [common configuration file](https://docs.edgexfoundry.org/3.0/design/adr/0026-Common%20Configuration/#specifying-the-common-configuration-location) specified via the future -cc/--commonConfig command line flag. + +- Device Profiles, Device Definition and Provision Watchers + + - These files can reside in a device services local file system and are pushed to Core Metadata the first time the service starts. Example [here](https://github.com/edgexfoundry/device-onvif-camera/tree/v2.3.0/cmd/res) + + - These files are found by scanning the folders specified in configuration [here](https://github.com/edgexfoundry/device-sdk-go/blob/main/example/cmd/device-simple/res/configuration.toml#L94-L96) + + !!! note + These files are only pushed to Core Metadata the first time the device service is loaded. They are not currently re-pushed once they exist in Core Metadata even when the files have changed locally. Thus updating the files locally or in a shared location will not result in changing the contents of these files in Core Metadata. They still benefit from this capability during initial deployment and when new files are added. + + +Currently all files loaded by services are expected to be on the local file system, thus are duplicated many times when deploying at scale. + +### Target Users +- Software Deployer +- Software Integrator + +### Description +This UCR proposes to enhance loading of files in EdgeX by allowing the location of the file to be optionally specified as an URI. + +### Existing solutions +Loading shared files via a URI is not new in the software industry. Here is the Wiki page for [Uniform Resource Identifier](https://en.wikipedia.org/wiki/Uniform_Resource_Identifier) + +### Requirements +- Single EdgeX service files shall optionally be loaded via a specified URI. +- Sets of EdgeX service files (i.e. device service files) shall optionally be loaded via a URI. Details on how are left to the ADR. +- The URIs specified shall follow the [Uniform Resource Identifier](https://en.wikipedia.org/wiki/Uniform_Resource_Identifier) spec when authentication not required or using **basic-auth** in plain text in the URI, i.e. `username:password@` + - Only the `http`, `https`, and `file` schemes from the above spec shall be supported. + +- The URI spec shall be extended to allow the specifying of EdgeX service secrets from the service's Secret Store in order to avoid credentials in plain text. Details on how are left to the ADR. + +### Other Related Issues +- [Common Configuration ADR](https://docs.edgexfoundry.org/3.0/design/adr/0026-Common%20Configuration/): This ADR specifies that the common configuration file specified by the `-cc/--commonConfig` flag can be a URI to a remote files. The implementation of this portion of the ADR is dependent on the UCR and following ADR. + +### References +- None From 5aeea9a70021ba5daec2d9f875e1a812f4f8d148 Mon Sep 17 00:00:00 2001 From: Leonard Goodell Date: Wed, 18 Jan 2023 14:06:31 -0700 Subject: [PATCH 2/4] feat: added token config file and fixed typo Signed-off-by: Leonard Goodell --- docs_src/design/ucr/URIs-for-Files.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs_src/design/ucr/URIs-for-Files.md b/docs_src/design/ucr/URIs-for-Files.md index 4dab13daf1..8138f14308 100644 --- a/docs_src/design/ucr/URIs-for-Files.md +++ b/docs_src/design/ucr/URIs-for-Files.md @@ -9,7 +9,7 @@ - All ### Motivation -Deployment at scale, i.e. identical or almost identical deployments across many locations, would benefit from the ability to loaded service files from a central location. This would allow the maintainer to make changes once to a shared file and have them apply to all or a subset of deployments. The following are some EdgeX service files that would benefit for this capability: +Deployment at scale, i.e. identical or almost identical deployments across many locations, would benefit from the ability to load service files from a central location. This would allow the maintainer to make changes once to a shared file and have them apply to all or a subset of deployments. The following are some EdgeX service files that would benefit for this capability: - [Unit of Measure](https://github.com/edgexfoundry/edgex-go/blob/v2.3.0/cmd/core-metadata/res/uom.toml) file used by Core Metadata @@ -19,6 +19,10 @@ Deployment at scale, i.e. identical or almost identical deployments across many - Location of these files are currently defaulted to be `./res/configuration.toml`, but can be overridden via -cf/--configFile command line flag. - The Common Configuration ADR adds a new [common configuration file](https://docs.edgexfoundry.org/3.0/design/adr/0026-Common%20Configuration/#specifying-the-common-configuration-location) specified via the future -cc/--commonConfig command line flag. +- [Token Configuration](https://github.com/edgexfoundry/edgex-go/blob/v2.3.0/cmd/security-file-token-provider/res/token-config.json) file for Security File Token Provider + + - This file specifies the list of services to generate Vault tokens for. + - Device Profiles, Device Definition and Provision Watchers - These files can reside in a device services local file system and are pushed to Core Metadata the first time the service starts. Example [here](https://github.com/edgexfoundry/device-onvif-camera/tree/v2.3.0/cmd/res) From f91ae13df00ad374c89bafb5e54920b39c9a4601 Mon Sep 17 00:00:00 2001 From: Leonard Goodell Date: Thu, 19 Jan 2023 09:07:52 -0700 Subject: [PATCH 3/4] feat: Adjusted supported schemes to replase file with plain paths Signed-off-by: Leonard Goodell --- docs_src/design/ucr/URIs-for-Files.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs_src/design/ucr/URIs-for-Files.md b/docs_src/design/ucr/URIs-for-Files.md index 8138f14308..ee8779d25d 100644 --- a/docs_src/design/ucr/URIs-for-Files.md +++ b/docs_src/design/ucr/URIs-for-Files.md @@ -49,8 +49,9 @@ Loading shared files via a URI is not new in the software industry. Here is the - Single EdgeX service files shall optionally be loaded via a specified URI. - Sets of EdgeX service files (i.e. device service files) shall optionally be loaded via a URI. Details on how are left to the ADR. - The URIs specified shall follow the [Uniform Resource Identifier](https://en.wikipedia.org/wiki/Uniform_Resource_Identifier) spec when authentication not required or using **basic-auth** in plain text in the URI, i.e. `username:password@` - - Only the `http`, `https`, and `file` schemes from the above spec shall be supported. - + - Only the `http` and `https` schemes from the above spec shall be supported as well as `plain paths` as is today + - The `file` scheme shall not be supported as it doesn't allow for relative paths + - The URI spec shall be extended to allow the specifying of EdgeX service secrets from the service's Secret Store in order to avoid credentials in plain text. Details on how are left to the ADR. ### Other Related Issues From f0005c6ad6ef5035104837a96e08be2efeb09f00 Mon Sep 17 00:00:00 2001 From: Leonard Goodell Date: Wed, 25 Jan 2023 10:46:42 -0700 Subject: [PATCH 4/4] feat: Changed status to approved Signed-off-by: Leonard Goodell --- docs_src/design/ucr/URIs-for-Files.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs_src/design/ucr/URIs-for-Files.md b/docs_src/design/ucr/URIs-for-Files.md index ee8779d25d..1af68ebd66 100644 --- a/docs_src/design/ucr/URIs-for-Files.md +++ b/docs_src/design/ucr/URIs-for-Files.md @@ -3,7 +3,7 @@ - Lenny Goodell (Intel) ## Change Log -- [pending](URL of PR) (2023-01-18) +- [approved](https://github.com/edgexfoundry/edgex-docs/pull/938) (2023-01-25) ### Market Segments - All