Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GridStoreAdapter and FSAdapter Config #778

Merged
merged 1 commit into from
Oct 23, 2020
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
183 changes: 182 additions & 1 deletion _includes/parse-server/file-adapters.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,96 @@ Parse Server allows developers to choose from several options when hosting files

* `GridStoreAdapter`, which is backed by MongoDB;
* `S3Adapter`, which is backed by [Amazon S3](https://aws.amazon.com/s3/); or
* `GCSAdapter`, which is backed by [Google Cloud Storage](https://cloud.google.com/storage/)
* `GCSAdapter`, which is backed by [Google Cloud Storage](https://cloud.google.com/storage/); or
* `FSAdapter`, local file storage

`GridStoreAdapter` is used by default and requires no setup, but if you're interested in using S3 or Google Cloud Storage, additional configuration information is available below.

When using files on Parse, you will need to use the `publicServerURL` option in your Parse Server config. This is the URL that files will be accessed from, so it should be a URL that resolves to your Parse Server. Make sure to include your mount point in this URL.

When using `Postgres`, you will need to configure `S3Adapter`, `GCSAdapter`, or `FSAdapter` for file support. The `GridStoreAdapter` does not work with `Postgres`.

## Configuring `GridStoreAdapter`

If you are using `Mongo` and don't need file encryption, there are no additional steps needed to use the `GridStoreAdapter`. If you'd like to enable file encryption follow these instructions to configure Parse Server to use `GridStoreAdapter` with file encryption.

***Notice: If you are coming from an older version of Parse Server versions <= 4.2.0, you should remove `fileKey` from your configurations as it is not being used anyways since legacy parse.com. `fileKey` has been repurposed for file encryption.***

### Set up file encryption (available in parse-server 4.3.0+)
The `GridStoreAdapter` can encrypt files at rest in `Mongo` using AES256-GCM, allowing the adapter to detect if files are tampered with.

To use, simply do any of the following:
- Use the environment variable `PARSE_SERVER_FILE_KEY`
- Pass in --fileKey in the command line when starting your server
- Initialize ParseServer with `fileKey="Your file encryptionKey"`.

An example starting your Parse Server in `index.js` is below:

```javascript
const api = new ParseServer({
databaseURI: databaseUri || 'mongodb://localhost:27017/dev',
cloud: process.env.PARSE_SERVER_CLOUD || __dirname + '/cloud/main.js',
appId: process.env.PARSE_SERVER_APPLICATION_ID || 'myAppId',
masterKey: process.env.PARSE_SERVER_MASTER_KEY || '',
fileKey: process.env.PARSE_SERVER_FILE_KEY, //Add your file key here. Keep it secret
...
});
```

Be sure not to lose your key or change it after encrypting files.

### Enabling encryption on a server that already has unencrypted files (available in parse-server 4.4.0+)
When this is the case, it is recommended to start up a development parse-server (or a separate process from your main process) that has the same configuration as your production server. On the development server, initialize the file adapter as above with the new key and do the following after initialization in your `index.js`:

```javascript
//You probably want to back up your unencrypted files before doing this.
//This can take awhile depending on how many files and how large they are. It will attempt to rotate the key of all files in your filesSubDirectory
const {rotated, notRotated} = await api.filesAdapter.rotateFileKey();
console.log('Files rotated to newKey: ' + rotated);
console.log('Files that couldn't be rotated to newKey: ' + notRotated);
```

### Rotating your encryption key (available in parse-server 4.4.0+)
Periodically you may want to rotate your fileKey for security reasons. When this is the case, it is recommended to start up a development parse-server (or a separate process from your main process) that has the same configuration as your production server. On the development server, initialize the file adapter with the new key and do the following in your `index.js` (you will need your `oldKey`):

```javascript
//This can take awhile depending on how many files and how large they are. It will attempt to rotate the key of all files in your filesSubDirectory
const {rotated, notRotated} = await api.filesAdapter.rotateFileKey({oldKey: oldKey});
console.log('Files rotated to newKey: ' + rotated);
console.log('Files that couldn't be rotated to newKey: ' + notRotated);
```

### Removing file encryption (available in parse-server 4.4.0+)
When this is the case, it is recommended to start up a development parse-server (or a separate process from your main process) that has the same configuration as your production server. Different from the previous examples, don't initialize your fileAdapter with a `fileKey`. Pass in your `oldKey` to `rotateFileKey()`.

```javascript
const api = new ParseServer({
databaseURI: databaseUri || 'mongodb://localhost:27017/dev',
cloud: process.env.PARSE_SERVER_CLOUD || __dirname + '/cloud/main.js',
appId: process.env.PARSE_SERVER_APPLICATION_ID || 'myAppId',
masterKey: process.env.PARSE_SERVER_MASTER_KEY || '',
//No fileKey here
...
});

//This can take awhile depending on how many files and how larger they are. It will attempt to rotate the key of all files in your filesSubDirectory
//It is not recommended to do this on the production server, deploy a development server to complete the process.
const {rotated, notRotated} = await api.filesAdapter.rotateFileKey({oldKey: oldKey});
console.log('Files rotated to unencrypted with noKey: ' + rotated);
console.log('Files that couldn't be rotated to unencrypted with noKey: ' + notRotated);

```

### Rotating the key for a subset of files
This is useful if for some reason there were errors and some of the files weren't rotated and returned in `notRotated`. The process is the same as the previous examples, but pass in your `oldKey` along with the array of `fileNames` to `rotateFileKey()`.

```javascript
//This can take awhile depending on how many files and how large they are. It will attempt to rotate the key of all files in your filesSubDirectory
const {rotated, notRotated} = await api.filesAdapter.rotateFileKey({oldKey: oldKey, fileNames: ["fileName1.png","fileName2.png"]});
console.log('Files rotated to newKey: ' + rotated);
console.log('Files that couldn't be rotated to newKey: ' + notRotated);
```

## Configuring `S3Adapter`

If you'd like to use Amazon S3, follow these instructions to configure Parse Server to use `S3Adapter`.
Expand Down Expand Up @@ -221,3 +305,100 @@ new GCSAdapter(projectId, keyfilePath, bucket, options)
| options | JavaScript object (map) that can contain: | |
| bucketPrefix | Key in `options`. Set to create all the files with the specified prefix added to the filename. Can be used to put all the files for an app in a folder with 'folder/'. | Optional. Default: '' |
| directAccess | Key in `options`. Controls whether reads are going directly to GCS or proxied through your Parse Server. | Optional. Default: false |

## Configuring `FSAdapter`

To use the `FSAdapter`, simply initialize your Parse Server in `index.js` by doing the following:

```javascript
var FSFilesAdapter = require('@parse/fs-files-adapter');

var fsAdapter = new FSFilesAdapter({
"filesSubDirectory": "my/files/folder" // optional
});

var api = new ParseServer({
appId: 'my_app',
masterKey: 'master_key',
filesAdapter: fsAdapter
})
```

***Notice: If used with Parse Server versions <= 4.2.0, DO NOT PASS in `PARSE_SERVER_FILE_KEY` or `fileKey` from parse-server. Instead pass your key directly to `FSFilesAdapter` using your own environment variable or hardcoding the string. Parse Server versions 4.3.0+ can pass in `PARSE_SERVER_FILE_KEY` or `fileKey`.***

### Using `FSAdapter` with multiple instances of Parse Server
When using parse-server-fs-adapter across multiple Parse Server instances it's important to establish "centralization" of your file storage (this is the same premise as the other file adapters, you are sending/recieving files through a dedicated link). You can accomplish this at the file storage level by Samba mounting (or any other type of mounting) your storage to each of your parse-server instances, e.g if you are using Parse Server via docker (volume mount your SMB drive to `- /Volumes/SMB-Drive/MyParseApp1/files:/parse-server/files`). All Parse Server instances need to be able to read/write to the same storage in order for parse-server-fs-adapter to work properly with parse-server. If the file storage isn't centralized, parse-server will have trouble locating files and you will get random behavior on client-side.

### Set up file encryption (available in parse-server-fs-adapter 1.1.0+)
The `FSAdapter` can encrypt files at rest for local storage using AES256-GCM, allowing the adapter to detect if files are tampered with.

To use, simply do the same as above, but add a `fileKey`:

```javascript
var FSFilesAdapter = require('@parse/fs-files-adapter');

var fsAdapter = new FSFilesAdapter({
"filesSubDirectory": "my/files/folder", // optional
"fileKey": "someKey" //mandatory if you want to encrypt files
});

var api = new ParseServer({
appId: 'my_app',
masterKey: 'master_key',
filesAdapter: fsAdapter
})
```

Be sure not to lose your key or change it after encrypting files.

### Enabling encryption on a server that already has unencrypted files (available in parse-server-fs-adapter 1.1.0+)
When this is the case, it is recommended to start up a development parse-server (or a separate process from your main process) that has the same configuration as your production server. On the development server, initialize the file adapter as above with the new key and do the following after initialization in your `index.js`:

```javascript
//You probably want to back up your unencrypted files before doing this.
//This can take awhile depending on how many files and how large they are. It will attempt to rotate the key of all files in your filesSubDirectory
const {rotated, notRotated} = await api.filesAdapter.rotateFileKey();
console.log('Files rotated to newKey: ' + rotated);
console.log('Files that couldn't be rotated to newKey: ' + notRotated);
```

### Rotating your encryption key (available in parse-server-fs-adapter 1.1.0+)
Periodically you may want to rotate your `fileKey` for security reasons. When this is the case, it is recommended to start up a development parse-server (or a separate process from your main process) that has the same configuration as your production server. On the development server, initialize the file adapter with the new key and do the following in your `index.js` (you will need your `oldKey`):

```javascript
//This can take awhile depending on how many files and how large they are. It will attempt to rotate the key of all files in your filesSubDirectory
const {rotated, notRotated} = await api.filesAdapter.rotateFileKey({oldKey: oldKey});
console.log('Files rotated to newKey: ' + rotated);
console.log('Files that couldn't be rotated to newKey: ' + notRotated);
```

### Removing file encryption (available in parse-server-fs-adapter 1.1.0+)
When this is the case, it is recommended to start up a development parse-server (or a separate process from your main process) that has the same configuration as your production server. Different from the previous examples, don't initialize your fileAdapter with a `fileKey`. Pass in your `oldKey` to `rotateFileKey()`.

```javascript
const api = new ParseServer({
databaseURI: databaseUri || 'mongodb://localhost:27017/dev',
cloud: process.env.PARSE_SERVER_CLOUD || __dirname + '/cloud/main.js',
appId: process.env.PARSE_SERVER_APPLICATION_ID || 'myAppId',
masterKey: process.env.PARSE_SERVER_MASTER_KEY || '',
filesAdapter: new FSFilesAdapter(), //No fileKey supplied
...
});

//This can take awhile depending on how many files and how larger they are. It will attempt to rotate the key of all files in your filesSubDirectory
//It is not recommended to do this on the production server, deploy a development server to complete the process.
const {rotated, notRotated} = await api.filesAdapter.rotateFileKey({oldKey: oldKey});
console.log('Files rotated to unencrypted with noKey: ' + rotated);
console.log('Files that couldn't be rotated to unencrypted with noKey: ' + notRotated);

```

### Rotating the key for a subset of files
This is useful if for some reason there were errors and some of the files weren't rotated and returned in `notRotated`. The process is the same as the previous examples, but pass in your `oldKey` along with the array of `fileNames` to `rotateFileKey()`.

```javascript
//This can take awhile depending on how many files and how large they are. It will attempt to rotate the key of all files in your filesSubDirectory
const {rotated, notRotated} = await api.filesAdapter.rotateFileKey({oldKey: oldKey, fileNames: ["fileName1.png","fileName2.png"]});
console.log('Files rotated to newKey: ' + rotated);
console.log('Files that couldn't be rotated to newKey: ' + notRotated);
```