Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Onyx File handling - preparation #4908

Merged
merged 38 commits into from
Jan 14, 2022
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
9c287f3
Package updates add localforage and updated Onyx
kidroca Aug 27, 2021
ee4f659
Omit `function` values from the persisted request
kidroca Aug 27, 2021
f37b502
Persist all `addAction` requests
kidroca Aug 27, 2021
5fd5dab
Add Expo File System to be used for mobile persistence
kidroca Aug 27, 2021
b677931
Use Onyx.prepareFile in addAction
kidroca Aug 28, 2021
4a484ec
Fix: AttachmentPicker Capture size information correctly
kidroca Aug 28, 2021
91d8f40
Update onyx hash, update lock file
kidroca Aug 28, 2021
a2ac108
Run pod install and add expo-file-system
kidroca Aug 30, 2021
e474acd
Remove storage event listener and logic
kidroca Aug 30, 2021
3ea51ec
Update onyx hash, update lock file
kidroca Aug 30, 2021
e9c5d50
Merge branch 'main' into kidroca/file-handling-poc
kidroca Oct 19, 2021
4073718
Update Onyx hash with the latest file storage POC
kidroca Nov 1, 2021
5971da5
Merge branch 'main' into kidroca/file-handling-poc
kidroca Nov 1, 2021
044278d
Remove expo-file-system
kidroca Nov 2, 2021
1da8e87
Report.js remove Onyx.prepareFile call
kidroca Nov 2, 2021
62de152
AttachmentPicker copy document attachments to cache
kidroca Nov 2, 2021
8d0f6cd
Bump react-native-flipper version
kidroca Nov 2, 2021
d33702c
Update Pods after package changes
kidroca Nov 2, 2021
7805eb0
Update Onyx hash after file handling changes
kidroca Nov 2, 2021
bbfdc5c
Update image picker and document picker libraries
kidroca Nov 2, 2021
2a02fb1
Revert reanimated to the previous version
kidroca Nov 2, 2021
06ef491
Merge branch 'main' into kidroca/file-handling-poc
kidroca Nov 2, 2021
29565c6
Run pod install
kidroca Nov 2, 2021
f862014
Update Onyx hash after file handling changes
kidroca Nov 3, 2021
f5de171
Remove SignoutManager.js - no longer needed
kidroca Nov 3, 2021
2ba8109
Merge branch 'main' into kidroca/file-handling-poc
kidroca Nov 15, 2021
4f322dd
Merge branch 'main' into kidroca/file-handling-poc
kidroca Dec 13, 2021
48c80d2
Revert "Persist all `addAction` requests"
kidroca Dec 13, 2021
1db1d72
Update onyx hash (file storage)
kidroca Dec 14, 2021
13083c8
Merge branch 'main' into kidroca/file-handling-poc
kidroca Dec 28, 2021
2539f47
Refactor: remove setShouldSignOut unused leftover
kidroca Dec 28, 2021
769141b
Add MoveToIndexedDB migration
kidroca Dec 28, 2021
856dd1e
Add MoveToIndexedDB tests
kidroca Dec 28, 2021
bd00a83
MigrateToIndexedDB: Don't remove non Onyx keys from localStorage
kidroca Dec 28, 2021
1e548cb
Package - add `inquirer` back
kidroca Dec 28, 2021
41004ea
MoveToIndexedDB - refactor to be more self-explanatory and add docs
kidroca Jan 6, 2022
555d9ae
Merge branch 'main' into kidroca/file-handling-poc
kidroca Jan 6, 2022
3266600
Add flipper 0.117.0 to match the pod speck
kidroca Jan 6, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 13 additions & 13 deletions ios/Podfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -410,11 +410,11 @@ PODS:
- react-native-config/App (= 1.4.1)
- react-native-config/App (1.4.1):
- React-Core
- react-native-document-picker (5.1.0):
- react-native-document-picker (7.1.1):
- React-Core
- react-native-flipper (0.103.0):
- react-native-flipper (0.117.0):
- React-Core
- react-native-image-picker (4.0.3):
- react-native-image-picker (4.1.2):
- React-Core
- react-native-netinfo (5.9.10):
- React-Core
Expand Down Expand Up @@ -530,7 +530,7 @@ PODS:
- React-Core
- RNReactNativeHapticFeedback (1.13.0):
- React-Core
- RNReanimated (2.3.0-alpha.1):
- RNReanimated (2.3.0-alpha.3):
- DoubleConversion
- FBLazyVector
- FBReactNativeSpec
Expand Down Expand Up @@ -825,7 +825,7 @@ SPEC CHECKSUMS:
CocoaAsyncSocket: 065fd1e645c7abab64f7a6a2007a48038fdc6a99
DoubleConversion: cf9b38bf0b2d048436d9a82ad2abe1404f11e7de
FBLazyVector: 7b423f9e248eae65987838148c36eec1dbfe0b53
FBReactNativeSpec: e6c5eb078317f6a15612c7e3b4d410c4d01bb703
FBReactNativeSpec: d2bbf7ed8374a5ef7e82afdce19c4e50731f1f0e
Firebase: 54cdc8bc9c9b3de54f43dab86e62f5a76b47034f
FirebaseABTesting: c3e48ebf5e7e5c674c5a131c68e941d7921d83dc
FirebaseAnalytics: 4751d6a49598a2b58da678cc07df696bcd809ab9
Expand Down Expand Up @@ -871,16 +871,16 @@ SPEC CHECKSUMS:
React-jsiexecutor: 124e8f99992490d0d13e0649d950d3e1aae06fe9
React-jsinspector: 500a59626037be5b3b3d89c5151bc3baa9abf1a9
react-native-config: d8b45133fd13d4f23bd2064b72f6e2c08b2763ed
react-native-document-picker: f2f73db94328c84e22144e369fb4a3ede47bc1f5
react-native-flipper: 169e8ba429b73ad637ce007337ce4b415e783799
react-native-image-picker: 474cf2c33c2b6671da53d293a16c97995f0aec15
react-native-document-picker: ebcce6982dd91aa6d2b9a38b666895c381015f1d
react-native-flipper: cd9eabd8917104c1bbdca2621717cdca3b2addef
react-native-image-picker: f45729c43d4f854508ab25c0d0f0f711a2a8a267
react-native-netinfo: 30fb89fa913c342be82a887b56e96be6d71201dd
react-native-pdf: 4b5a9e4465a6a3b399e91dc4838eb44ddf716d1f
react-native-performance: 6bd6cfac80594775fb782405fceaaf206becf53b
react-native-plaid-link-sdk: 0b2e16d091abfee30ba01cf37734182ad610f917
react-native-plaid-link-sdk: dcc247a441571b6e0b9029ab00a30f82e4cfa906
react-native-progress-bar-android: be43138ab7da30d51fc038bafa98e9ed594d0c40
react-native-progress-view: 21b1e29e70c7559c16c9e0a04c4adc19fce6ede2
react-native-safe-area-context: 79fea126c6830c85f65947c223a5e3058a666937
react-native-safe-area-context: 01158a92c300895d79dee447e980672dc3fb85a6
React-perflogger: aad6d4b4a267936b3667260d1f649b6f6069a675
React-RCTActionSheet: fc376be462c9c8d6ad82c0905442fd77f82a9d2a
React-RCTAnimation: ba0a1c3a2738be224a08092fa7f1b444ab77d309
Expand All @@ -899,18 +899,18 @@ SPEC CHECKSUMS:
RNCClipboard: 41d8d918092ae8e676f18adada19104fa3e68495
RNCMaskedView: fc29d354a40316a990e8fb46391f08aea829c3aa
RNCPicker: 6780c753e9e674065db90d9c965920516402579d
RNDateTimePicker: 7658208086d86d09e1627b5c34ba0cf237c60140
RNDateTimePicker: c9911be59b1f8670b9f244b85af3a7c295e175ed
RNFBAnalytics: 8ba84c2d31c64374d054c8621b998f25145ffddc
RNFBApp: 64c90ab78b6010ed5c3ade026dfe5ff6442c21fd
RNFBCrashlytics: 1de18b8cc36d9bcf86407c4a354399228cc84a61
RNFBPerf: e3a7269f573a4787810a32de51647cdcbe08dfb4
RNGestureHandler: 9b7e605a741412e20e13c512738a31bd1611759b
RNPermissions: 4c8a37b4dde50f1f152bf8cd08c4a43d2355829e
RNReactNativeHapticFeedback: b83bfb4b537bdd78eb4f6ffe63c6884f7b049ead
RNReanimated: 833ebd229b31e18a8933ebd0cd744a0f47d88c42
RNReanimated: 8df8a4f5fb1ae48b8378ad4bbc38e47ae0c41e9f
RNScreens: e8e8dd0588b5da0ab57dcca76ab9b2d8987757e0
RNSVG: ce9d996113475209013317e48b05c21ee988d42e
urbanairship-react-native: ee53526e1f81c5170863dd3e039df7f98730ef53
urbanairship-react-native: 60b4b4235838ff109a2639b639e2ef01d54ad455
Yoga: a7de31c64fe738607e7a3803e3f591a4b1df7393
YogaKit: f782866e155069a2cca2517aafea43200b01fd5a

Expand Down
70 changes: 47 additions & 23 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 6 additions & 5 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@
"expensify-common": "git://github.com/Expensify/expensify-common.git#b5d9167a26cfd40beebdfb79958b3bbed1af7b64",
"file-loader": "^6.0.0",
"html-entities": "^1.3.1",
"localforage": "^1.10.0",
"lodash": "4.17.21",
"metro-config": "^0.64.0",
"moment": "^2.27.0",
Expand All @@ -79,21 +80,21 @@
"react-native-bootsplash": "^3.2.0",
"react-native-collapsible": "^1.6.0",
"react-native-config": "^1.4.0",
"react-native-document-picker": "^5.1.0",
"react-native-document-picker": "^7.1.1",
"react-native-gesture-handler": "1.9.0",
"react-native-google-places-autocomplete": "^2.4.1",
"react-native-haptic-feedback": "^1.13.0",
"react-native-image-pan-zoom": "^2.1.12",
"react-native-image-picker": "^4.0.3",
"react-native-image-picker": "^4.1.2",
"react-native-keyboard-spacer": "^0.4.1",
"react-native-modal": "^11.10.0",
"react-native-onyx": "git+https://github.com/Expensify/react-native-onyx.git#ccb64c738b8bbe933b8997eb177f864e5139bd8d",
"react-native-onyx": "git+https://github.com/kidroca/react-native-onyx.git#45ddac448fb76ab2ae4d56cebef1d04e84158daa",
"react-native-pdf": "^6.2.2",
"react-native-performance": "^2.0.0",
"react-native-permissions": "^3.0.1",
"react-native-picker-select": "8.0.4",
"react-native-plaid-link-sdk": "^7.1.0",
"react-native-reanimated": "^2.3.0-beta.3",
"react-native-reanimated": "^2.3.0-alpha.1",
"react-native-render-html": "6.0.0-beta.8",
"react-native-safe-area-context": "^3.1.4",
"react-native-screens": "^3.0.0",
Expand Down Expand Up @@ -162,7 +163,7 @@
"portfinder": "^1.0.28",
"pusher-js-mock": "^0.3.3",
"react-hot-loader": "^4.12.21",
"react-native-flipper": "^0.103.0",
"react-native-flipper": "^0.117.0",
"react-native-performance-flipper-reporter": "^2.0.0",
"react-native-svg-transformer": "^0.14.3",
"react-test-renderer": "16.13.1",
Expand Down
30 changes: 18 additions & 12 deletions src/components/AttachmentPicker/index.native.js
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ function getImagePickerOptions(type) {
*/
const documentPickerOptions = {
type: [RNDocumentPicker.types.allFiles],
copyTo: 'cachesDirectory',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does android have a cachesDirectory? I only see it for IOS

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Documentation says:

[iOS and Android only] copyTo:"cachesDirectory" | "documentDirectory"
https://www.npmjs.com/package/react-native-document-picker#ios-and-android-only-copytocachesdirectory--documentdirectory

I take it the option copyTo and the value cachesDirectory is available in both

};

/**
Expand All @@ -65,7 +66,7 @@ function getDataForUpload(fileData) {
return {
name: fileData.fileName || fileData.name || 'chat_attachment',
type: fileData.type,
uri: fileData.uri,
uri: fileData.fileCopyUri || fileData.uri,
Copy link
Contributor Author

@kidroca kidroca Nov 2, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The document picker makes the attachment available until the application is terminated
To persist Document Attachments and be able to use them on the next app launch we need to move them to the cache dir
This way we can persist a reference for the attachment in Onyx and support offline files

The ImagePicker library is already making a copy in cache by default

size: fileData.fileSize || fileData.size,
};
}
Expand Down Expand Up @@ -117,17 +118,22 @@ class AttachmentPicker extends Component {
* Handles the image/document picker result and
* sends the selected attachment to the caller (parent component)
*
* @param {ImagePickerResponse|DocumentPickerResponse} attachment
* @param {Array<ImagePickerResponse|DocumentPickerResponse>} attachments
*/
pickAttachment(attachment) {
if (attachment) {
if (attachment.width === -1 || attachment.height === -1) {
this.showImageCorruptionAlert();
return;
}
const result = getDataForUpload(attachment);
this.completeAttachmentSelection(result);
pickAttachment(attachments = []) {
if (attachments.length === 0) {
return;
}

const fileData = _.first(attachments);

if (fileData.width === -1 || fileData.height === -1) {
this.showImageCorruptionAlert();
return;
}

const result = getDataForUpload(fileData);
this.completeAttachmentSelection(result);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The updated document picker responds with an array. The image picker would also respond with array that's why this was changed to accept an array value

}

/**
Expand Down Expand Up @@ -178,7 +184,7 @@ class AttachmentPicker extends Component {
}

// Resolve with the first (and only) selected file
return resolve(response.assets[0]);
return resolve(response.assets);
});
});
}
Expand Down Expand Up @@ -207,7 +213,7 @@ class AttachmentPicker extends Component {
/**
* Launch the DocumentPicker. Results are in the same format as ImagePicker
*
* @returns {Promise<DocumentPickerResponse>}
* @returns {Promise<DocumentPickerResponse[]>}
*/
showDocumentPicker() {
return RNDocumentPicker.pick(documentPickerOptions).catch((error) => {
Expand Down
5 changes: 4 additions & 1 deletion src/libs/Network.js
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,10 @@ function processNetworkRequestQueue() {
// We filter persisted requests from the normal Queue to remove duplicates
networkRequestQueue = _.reject(networkRequestQueue, (request) => {
if (!request.data.doNotRetry && request.data.persist) {
retryableRequests.push(request);
// exclude functions as they cannot be persisted
const requestToPersist = _.omit(request, val => _.isFunction(val));
retryableRequests.push(requestToPersist);

return true;
}
});
Expand Down
10 changes: 4 additions & 6 deletions src/libs/actions/Report.js
Original file line number Diff line number Diff line change
Expand Up @@ -1000,7 +1000,7 @@ function fetchAllReports(
*
* @param {Number} reportID
* @param {String} text
* @param {Object} [file]
* @param {File} [file]
*/
function addAction(reportID, text, file) {
// Convert the comment from MD into HTML because that's how it is stored in the database
Expand Down Expand Up @@ -1079,14 +1079,12 @@ function addAction(reportID, text, file) {

API.Report_AddComment({
reportID,
reportComment: commentText,
file,
reportComment: commentText,
clientID: optimisticReportActionID,

// The persist flag enables this request to be retried if we are offline and the app is completely killed. We do
// not retry attachments as we have no solution for storing them persistently and attachments can't be "lost" in
// the same way report actions can.
persist: !isAttachment,
// The persist flag enables this request to be retried if we are offline and the app is completely killed.
persist: true,
})
.then((response) => {
if (response.jsonCode === 408) {
Expand Down
23 changes: 0 additions & 23 deletions src/libs/listenToStorageEvents/index.js

This file was deleted.

Loading