Potential design for moving service information out of connection config

[cbruno10]

Is your feature request related to a problem? Please describe.
Currently, each connection requires a service ID, which makes it difficult to get all service info (which can be done with an aggregator, but feels a bit cumbersome). Also, for resources that don't require service information, a user shouldn't need to pass service info into the connection.

Describe the solution you'd like

• Not have service_id in the connection config
• Have a table, fastly_service, that lists services and gets info for services (possibly no change to the existing table)
• The listServices function in the fastly_service table could be used as a parent hydrate for any other table that requires a service ID, so it’d list all resources of that table type for each service returned from listServices
• Tables that don’t require service IDs would use their own API and not rely on the fastly_service table

However, some possible limitations:

• The Engineer role is required to list services
  • Role info: https://www.fastly.com/documentation/reference/api/services/service/
  • I’m not sure how common it’d be for users to have the Engineer role so they could list services
• I'm not sure how many APIs require a service ID vs. those that don't, and if there are any additional resource types

Describe alternatives you've considered
Creating an aggregator and a connection for each service

Additional context
Add any other context or screenshots about the feature request here.

[ParthaI]

Hi @cody / @timoguin ,

I have pushed some rough changes to the issue-29 branch based on the design discussed above.

It would be great if you could test it out in this branch and share your feedback with us. Once the table design is finalized, I will push the cleanup code changes and documentation updates before raising the PR.

Changes Overview:

• The service_id is no longer supported in the connection config.
• The fastly_service table has been updated to list all services accessible with the api_key.
• Most tables now require the service_id and service_version to populate the column values.
• Instead of using fastly_service as a parent hydrate, the fastly_service_version table is used because we need both the Service ID and Service Version to make API calls.
• The list config remains unchanged. The get config for the tables has been updated to include service_id and service_version as required qualifiers, since they were previously obtained from the connection config.
• For the fastly_acl table, we cannot use fastly_service_version as a parent hydrate because:
  • The fastly_acl_entry table uses fastly_acl as a parent hydrate.
  • Steampipe does not yet support parent hydrate chaining up to three levels down.
  • A wrapper hydrate function has been introduced to serve as a parent hydrate in the fastly_acl table.

The overall functionality will remain the same. Previously, results were obtained per connection per service ID. Now, results will be obtained per connection per api_key.

The Engineer role is required to list services.

I have invited a user with the "Access to billing, stats & analytics" role but am still able to list services with that user's API Key.

Steps to test the changes in the issue-29 branch:

1. Clone the repository: git clone https://github.com/turbot/steampipe-plugin-fastly.git
2. Navigate to the project directory: cd steampipe-plugin-fastly
3. Check out the issue-29 branch: git checkout issue-29
4. Build the plugin: make
5. Run the query to test the changes.

@cody, please review the table design and let me know if it looks good. Any suggestions would be highly appreciated.

Thanks!

[ParthaI]

Hi @cbruno10, @timoguin,

I wanted to provide an update: I've pushed additional changes to the same branch, issue-29, to support backward compatibility. Since we aren't sure how users are utilizing this plugin, removing support for the service_id from the connection config might cause issues in their environments. The changes are currently in this PR.

Changes overview:

• The existing functionality remains unchanged.
• The service_id and service_version are now OPTIONAL, meaning users can choose to specify them in the connection config or not.
• If only service_version is specified, an error will indicate that 'service_id' must be set in the connection configuration if 'service_version' is specified. Edit your connection configuration file and then restart steampipe.
• If only service_id is specified, we'll retrieve the row for the active version only (as per existing functionality).
• If there's no active version, rows will be returned based on the latest version (as per existing functionality).
• If neither is configured, rows will be returned for all services and all versions.

Please let me know if these changes look good or if any adjustments are needed. I'm open to making changes based on your suggestions.

Thanks!