Add support to new backend system

.changeset/brave-rocks-hide.md

@@ -0,0 +1,11 @@
+---
+'@spreadshirt/backstage-plugin-s3-viewer-backend': minor
+---
+
+Add support to the [new backend system](https://backstage.io/docs/backend-system/). 
+
+Follow the instructions in the [README.md](https://github.com/spreadshirt/backstage-plugin-s3/blob/main/plugins/s3-viewer-backend/README.md#new-backend-system)
+
+**DEPRECATION**: The method `setRefreshInterval` has been deprecated in favor of the usage of the configuration file to schedule the refresh.
+From now on, the schedule should be set using the `app-config.yaml` file. This method will be kept for some time as a fallback if the schedule
+has not been set via the configuration file.

plugins/s3-viewer-backend/README.md

@@ -66,6 +66,44 @@ To get started, follow these steps:
 
 3. Add the configuration in the `app-config.yaml` file. This is explained in detail in the next section.
 
+## New backend system
+
+The plugin also supports the [new backend system](https://backstage.io/docs/backend-system/). If you want to use the plugin add the following line in the `src/index.ts`:
+
+```typescript
+// In packages/backend/src/index.ts
+
+backend.add(import('@spreadshirt/backstage-plugin-s3-viewer-backend'));
+```
+
+This line will initialize the backend with all the default configurations. It is possible to customize it by adding a backend module to our app, by doing something like:
+
+```typescript
+import { s3ViewerExtensionPoint } from '@spreadshirt/plugin-s3-viewer-node';
+
+const s3ViewerExtensions = createBackendModule({
+  pluginId: 's3-viewer',
+  moduleId: 'extensions',
+  register(reg) {
+    reg.registerInit({
+      deps: {
+        extension: s3ViewerExtensionPoint,
+      },
+      async init({ extension }) {
+        /// ...
+      },
+    });
+  },
+});
+
+backend.add(s3ViewerExtensions());
+```
+
+The `extensions` type contains all the needed functions to override any of the elements that are defined in the next section.
+
+To enable the permissionMiddleware, which is needed when used together with the permissions setup, you can do it using the `app-config.yaml`
+by setting `s3.permissionMiddleware` to `true`.
+
 ## Configuration
 
 This plugin allows fetching the buckets from different endpoints and using different approaches. This is a full example entry in `app-config.yaml`:
@@ -108,6 +146,10 @@ s3:
     - platform: iam-endpoint-name
       buckets:
         - another-bucket-name
+  bucketRefreshSchedule:
+    frequency: { minutes: 30 }
+    timeout: { minutes: 1 }
+  permissionMiddleware: true
 ```
 
 ### bucketLocatorMethods
@@ -183,6 +225,14 @@ For security, when selecting the `radosgw-admin` mode, you need to specify a lis
 
 To achieve that you need to specify the __platform name__ and then an array of buckets to be allowed. You can add as much as you want. And if a platform is not defined here, by default all the buckets will be allowed.
 
+### bucketRefreshSchedule
+
+If set, the buckets provider will be executed with the defined schedule.
+
+### permissionMiddleware
+
+Used by the new backend system. This field is optional. If set to true, the permissionMiddleware will be enabled in the backend plugin.
+
 ## Customization
 
 Apart from the custom `CredentialsProvider`, it is also possible to make more changes to the plugin, so that it can match your internal requirements.
@@ -245,20 +295,13 @@ By default, the S3 API doesn't provide a straightforward way to fetch informatio
 
 ### Refresh Interval
 
-Finally, it might be useful to refresh the bucket information, so that this data is always up to date. By default, the refresh is disabled, and it has to be defined by the user. To use it, the user just needs to set the frequency of this update by using the `HumanDuration` type, as with the `scheduler` env.
-
-```typescript
-  const builder = S3Builder.createBuilder({
-    config: env.config,
-    logger: env.logger,
-    scheduler: env.scheduler,
-    discovery: env.discovery,
-    identity: env.identity,
-    permissions: env.permissions,
-    tokenManager: env.tokenManager,
-  }).setRefreshInternal({ minutes: 30 });
+Finally, it might be useful to refresh the bucket information, so that this data is always up to date. By default the refresh is disabled and it has to be defined by the user. To use it, the user just needs to set the frequency directly in the configuration file. For example, to refresh the list of buckets every half hour and using a timeout of 1 minute:
 
-  const { router } = await builder.build();
+```yaml
+s3:
+  bucketRefreshSchedule:
+    frequency: { minutes: 30 }
+    timeout: { minutes: 1 }
 ```
 
 ## Permissions Setup

plugins/s3-viewer-backend/config.d.ts

@@ -1,6 +1,14 @@
+import { TaskScheduleDefinitionConfig } from '@backstage/backend-tasks';
+
 /** Configuration for the S3 Viewer plugin */
 export interface Config {
   s3?: {
+    /**
+     * If defined, it sets the schedule used to refresh the list of buckets
+     * @visibility backend
+     * */
+    bucketRefreshSchedule: TaskScheduleDefinitionConfig | undefined;
+
     /** @visibility backend */
     bucketLocatorMethods: Array<
       | {
@@ -92,5 +100,11 @@ export interface Config {
        */
       buckets: Array<string>;
     }>;
+    /**
+     * If set to `true` the permissionMiddleware will be enabled. The default one will be used, but it can be customized.
+     * The permissionMiddleware is required if the permissions setup is used.
+     * @visibility backend
+     */
+    permissionMiddleware?: boolean;
   };
 }

plugins/s3-viewer-backend/src/index.ts

@@ -21,3 +21,4 @@ export {
   s3ViewerBucketConditions,
   matches,
 } from './permissions';
+export { s3ViewerPlugin as default } from './service';

plugins/s3-viewer-backend/src/service/S3Api.ts

@@ -158,7 +158,7 @@ export class S3Client implements S3Api {
     bucket: string,
     key: string,
   ): Promise<string> {
-    const s3Url = await this.discoveryApi.getExternalBaseUrl('s3');
+    const s3Url = await this.discoveryApi.getExternalBaseUrl('s3-viewer');
     const url = new URL(
       `${s3Url}/stream/${encodeURIComponent(bucket)}/${encodeURIComponent(
         key,

plugins/s3-viewer-backend/src/service/S3BucketsProvider.ts

@@ -10,8 +10,10 @@ import {
 } from '@spreadshirt/backstage-plugin-s3-viewer-common';
 import { LoggerService } from '@backstage/backend-plugin-api';
 import { S3 } from '@aws-sdk/client-s3';
-import { PluginTaskScheduler } from '@backstage/backend-tasks';
-import { HumanDuration } from '@backstage/types';
+import {
+  PluginTaskScheduler,
+  TaskScheduleDefinition,
+} from '@backstage/backend-tasks';
 import { matches } from '../permissions';
 
 export class S3BucketsProvider implements BucketsProvider {
@@ -23,7 +25,7 @@ export class S3BucketsProvider implements BucketsProvider {
     readonly scheduler: PluginTaskScheduler,
     readonly credentialsProvider: CredentialsProvider,
     readonly statsProvider: BucketStatsProvider | undefined,
-    readonly refreshInterval: HumanDuration | undefined,
+    readonly schedule: TaskScheduleDefinition | undefined,
   ) {
     this.buckets = [];
     this.bucketCreds = [];
@@ -34,14 +36,14 @@ export class S3BucketsProvider implements BucketsProvider {
     scheduler: PluginTaskScheduler,
     credentialsProvider: CredentialsProvider,
     statsProvider: BucketStatsProvider | undefined,
-    refreshInterval: HumanDuration | undefined,
+    schedule: TaskScheduleDefinition | undefined,
   ): S3BucketsProvider {
     const bucketsProvider = new S3BucketsProvider(
       logger,
       scheduler,
       credentialsProvider,
       statsProvider,
-      refreshInterval,
+      schedule,
     );
     // Don't wait for bucket fetch. This speeds up the backend startup process.
     bucketsProvider.start();
@@ -51,12 +53,14 @@ export class S3BucketsProvider implements BucketsProvider {
 
   async start(): Promise<void> {
     await this.fetchBuckets();
-    if (this.refreshInterval) {
+    if (this.schedule) {
       await this.scheduler.scheduleTask({
         id: 'refresh-s3-buckets',
         fn: async () => this.fetchBuckets(),
-        frequency: this.refreshInterval,
-        timeout: this.refreshInterval,
+        frequency: this.schedule.frequency,
+        timeout: this.schedule.timeout,
+        initialDelay: this.schedule.initialDelay,
+        scope: this.schedule.scope,
       });
     }
   }

plugins/s3-viewer-backend/src/service/S3Builder.ts

@@ -15,8 +15,11 @@ import {
   LoggerService,
   TokenManagerService,
 } from '@backstage/backend-plugin-api';
-import { PluginTaskScheduler } from '@backstage/backend-tasks';
-import { HumanDuration } from '@backstage/types';
+import {
+  PluginTaskScheduler,
+  TaskScheduleDefinition,
+  readTaskScheduleDefinitionFromConfig,
+} from '@backstage/backend-tasks';
 import {
   assertError,
   AuthenticationError,
@@ -37,6 +40,7 @@ import {
 import { getCombinedCredentialsProvider } from '../credentials-provider';
 import cookieParser from 'cookie-parser';
 import { noopMiddleware, s3Middleware } from '../middleware';
+import { HumanDuration } from '@backstage/types';
 import { matches, transformConditions } from '../permissions';
 
 export interface S3Environment {
@@ -54,7 +58,7 @@ export interface S3BuilderReturn {
 }
 
 export class S3Builder {
-  private refreshInterval: HumanDuration | undefined = undefined;
+  private refreshInterval: HumanDuration | undefined;
   private client?: S3Api;
   private credentialsProvider?: CredentialsProvider;
   private bucketsProvider?: BucketsProvider;
@@ -79,6 +83,20 @@ export class S3Builder {
       };
     }
 
+    // Temporarily maintain support for the `setRefreshInterval` method if the configuration
+    // is not used. Remove it in some of the next releases. Added a deprecation for now
+    const fallbackSchedule = this.refreshInterval
+      ? { frequency: this.refreshInterval, timeout: this.refreshInterval }
+      : undefined;
+
+    const schedule: TaskScheduleDefinition | undefined = config.has(
+      's3.bucketRefreshSchedule',
+    )
+      ? readTaskScheduleDefinitionFromConfig(
+          config.getConfig('s3.bucketRefreshSchedule'),
+        )
+      : fallbackSchedule;
+
     const credentialsProvider =
       this.credentialsProvider ?? this.buildCredentialsProvider();
 
@@ -89,7 +107,7 @@ export class S3Builder {
         scheduler,
         credentialsProvider,
         this.statsProvider,
-        this.refreshInterval,
+        schedule,
       );
 
     this.client =
@@ -165,8 +183,13 @@ export class S3Builder {
    *
    * @param refreshInterval - The refresh interval to reload buckets
    * @returns
+   * @deprecated Now the refresh interval is set via the app-config.yaml file.
+   * Define `s3.bucketRefreshSchedule` in your configuration file.
    */
   public setRefreshInterval(refreshInterval: HumanDuration) {
+    this.env.logger.warn(
+      "The method setRefreshInterval is deprecated. Please define the refresh interval via the config file in 's3.bucketRefreshSchedule' instead",
+    );
     this.refreshInterval = refreshInterval;
     return this;
   }

plugins/s3-viewer-backend/src/service/index.ts

@@ -1,2 +1,3 @@
 export * from './S3Api';
 export * from './S3Builder';
+export { s3ViewerPlugin } from './plugin';

plugins/s3-viewer-backend/src/service/plugin.ts

@@ -0,0 +1,84 @@
+import {
+  BucketStatsProvider,
+  BucketsProvider,
+  CredentialsProvider,
+  S3Api,
+  s3ViewerExtensionPoint,
+} from '@spreadshirt/backstage-plugin-s3-viewer-node';
+import {
+  coreServices,
+  createBackendPlugin,
+} from '@backstage/backend-plugin-api';
+import { S3Builder } from './S3Builder';
+
+export const s3ViewerPlugin = createBackendPlugin({
+  pluginId: 's3-viewer',
+  register(env) {
+    let s3Client: S3Api;
+    let s3CredentialsProvider: CredentialsProvider;
+    let s3BucketsProvider: BucketsProvider;
+    let s3BucketStatsProvider: BucketStatsProvider;
+
+    env.registerExtensionPoint(s3ViewerExtensionPoint, {
+      setClient(client) {
+        if (s3Client !== undefined) {
+          throw new Error('A S3 client has been already set');
+        }
+        s3Client = client;
+      },
+      setCredentialsProvider(credentialsProvider) {
+        if (s3CredentialsProvider !== undefined) {
+          throw new Error('A credentials provider has been already set');
+        }
+        s3CredentialsProvider = credentialsProvider;
+      },
+      setBucketsProvider(bucketsProvider) {
+        if (s3BucketsProvider !== undefined) {
+          throw new Error('A buckets provider has been already set');
+        }
+        s3BucketsProvider = bucketsProvider;
+      },
+      setBucketStatsProvider(bucketStatsProvider) {
+        if (s3BucketStatsProvider !== undefined) {
+          throw new Error('A bucket stats provider has been already set');
+        }
+        s3BucketStatsProvider = bucketStatsProvider;
+      },
+    });
+
+    env.registerInit({
+      deps: {
+        logger: coreServices.logger,
+        config: coreServices.rootConfig,
+        scheduler: coreServices.scheduler,
+        discovery: coreServices.discovery,
+        identity: coreServices.identity,
+        permissions: coreServices.permissions,
+        tokenManager: coreServices.tokenManager,
+        httpRouter: coreServices.httpRouter,
+      },
+      async init(deps) {
+        let builder = S3Builder.createBuilder(deps);
+
+        if (s3Client) {
+          builder = builder.setClient(s3Client);
+        }
+        if (s3CredentialsProvider) {
+          builder = builder.setCredentialsProvider(s3CredentialsProvider);
+        }
+        if (s3BucketsProvider) {
+          builder = builder.setBucketsProvider(s3BucketsProvider);
+        }
+        if (s3BucketStatsProvider) {
+          builder = builder.setBucketStatsProvider(s3BucketStatsProvider);
+        }
+        if (deps.config.getOptionalBoolean('s3.permissionMiddleware')) {
+          builder = await builder.useMiddleware();
+        }
+
+        const { router } = await builder.build();
+        deps.httpRouter.use(router);
+      },
+    });
+  },
+});