Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update BulkExport.md documentation #3888

Merged
merged 3 commits into from
May 31, 2024
Merged

Update BulkExport.md documentation #3888

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ba5655d
Update BulkExport.md documentation
EXPEkesheth May 30, 2024
7039993
Update BulkExport.md
EXPEkesheth May 30, 2024
30a62a7
Update BulkExport.md
EXPEkesheth May 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions docs/BulkExport.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This feature allows data from the FHIR server to be exported. More details can b

There are two ways by which one can set the destination storage account to export data to. One way would be to use the connection string for the storage account and update the `FhirServer:Operations:Export:StorageAccountConnection` setting. The fhir-server will use the connection string to connect to the storage account and export data.

The other option would be to use the `FhirServer:Operations:Export:StorageAccountUri` setting with the uri of the storage account. For this option, we assume that the fhir-server has permissions to contribute data to the corresponding storage account. One way to achieve this (assuming you are running the fhir-server code in App Service with Managed Identity enabled) would be to give the App Service `Storage Blob Data Contributor` permissions for the storage account of your choice.
The other option would be to use the `FhirServer:Operations:Export:StorageAccountUri` setting with the uri of the storage account. For this option, we assume that the fhir-server has permissions to contribute data to the corresponding storage account. One way to achieve this (assuming you are running the fhir-server code in App Service with Managed Identity enabled) would be to give the App Service `Storage Blob Data Contributor` permissions for the storage account of your choice.

Currently, we only support Azure Blob storage as the destination.

Expand Down Expand Up @@ -42,16 +42,17 @@ For more details on Bulk Export, see the [Azure API for FHIR Export Data page](h

Below are set of additional query parameters users can specify in addition to ones defined in the Azure API For FHIR documentation
1. \_format in FHIR Server: \_format allows a user to select a format for the file structure that the export job creates. Different formats can be defined in the appSettings by combining constants, folder level breaks ('/'), and known tags. The tags will be replaced with data when the job is run. The three supported tags are:
* **resourcename**: replaces with the resource type being exported
* **timestamp**: replaces with a timestamp of the job's queried time
* **id**: replaces with the GUID of the export job
* **resourcename**: Replaced with the resource type being exported.
* **timestamp**: Replaced with a timestamp of the job's queried time.
* **id**: Replaced with the GUID of the export job.

1. \_max_count: \_max_count allows to reduce the number of resources exported by a single job. Users can use the _maxCount=xxxx query parameter or set MaximumNumberOfResourcesPerQuery in the export configuration section. The default is 10,000. Export operation needs memory to serialize the data when it is writing to the lake. To reduce out of memory exceptions due to additional memory, user can choose to reduce the _max_count value by decrements of 1000. It would be beneficial for user to increase the compute memory on FHIR server as well.

To use the format, you will need to set the following settings in the appSettings:

| appSetting | Description | Example Value|
|------------|-------------|--------------|
| FhirServer:Operations:Export:Formats:#:Name | Name of the format you plan to call. The # should be replaced as you can specify multiple formats. We provide default values for 0 and 1 (for when a container is specified and when a container is not specified) so recommend starting with 2 | TestFormat |
| FhirServer:Operations:Export:Formats:#:Name | Name the format you plan to call. The # should be replaced as you can specify multiple formats. We provide default values for 0 and 1 (for when a container is specified and when a container is not specified), so we recommend starting with 2 | TestFormat |
EXPEkesheth marked this conversation as resolved.
Show resolved Hide resolved
| FhirServer:Operations:Export:Formats:#:Format | Defines the format. The # should match the one used above. | test/\<resourcename>/\<id>/\<timestamp> |

In the table above, you would use format in the following way `GET https://<<FHIR service base URL>>/$export?_format=TestFormat`. The result would be an export saved in a folder structure **test/\<resourcename>/\<id>** and the file name would be **\<timestamp>.ndjson**.
Expand Down
Loading