Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DataStore Query Performance Very Slow #8405

Closed
3 tasks done
sacrampton opened this issue Jun 6, 2021 · 110 comments
Closed
3 tasks done

DataStore Query Performance Very Slow #8405

sacrampton opened this issue Jun 6, 2021 · 110 comments
Assignees
Labels
DataStore Related to DataStore category React Native React Native related issue

Comments

@sacrampton
Copy link

Before opening, please confirm:

JavaScript Framework

React Native

Amplify APIs

DataStore

Amplify Categories

api

Environment information

# Put output below this line


Describe the bug

We are porting our AppSync/GraphQL based app to DataStore and what we are finding is that queries on the cached database are very slow. We are also finding that pagination of the queries makes no difference to the performance of the queries.

There are many tables with the largest table having about 8,000 records and most other tables having less than 1000 records. There are only 5 tables/models that have data.

Whether we are connected to the network or working offline it makes no difference to the performance.

In issue #6994 it talks of a first time slowing in performance, but we see consistently slow performance.

I have recorded a video of the performance we are seeing - both online and offline - to give you an idea of the problem we are facing.

Google.Chrome.-.RPReplay_Final1622941095.MP4.-.Google.Chrome.mp4

Expected behavior

The database is local so we should be seeing almost instant responses to our queries. The performance of our existing AppSync GraphQL cache gives us almost instant responses so we expect the same or better performance than the existing AppSync GraphQL database.

Reproduction steps

Not sure how to replicate this as the database I have has 48 models and 200 GSI's and takes about 6 hours of incremental (manual) pushes to deploy into a new environment.

Code Snippet

// Put your code below this line.

Log output

// Put your logs below this line


aws-exports.js

No response

Manual configuration

No response

Additional configuration

No response

Mobile Device

iPhone 7

Mobile Operating System

14.5

Mobile Browser

Not relevant

Mobile Browser Version

No response

Additional information and screenshots

No response

@nubpro
Copy link
Contributor

nubpro commented Jun 6, 2021

I closed my original issue as I could no longer replicate the problem anymore

Do u happen to be using the Storage module in your app?

@sacrampton
Copy link
Author

Hi @nubpro - we are using the Storage module in the app for upload of photographs.

@sacrampton
Copy link
Author

sacrampton commented Jun 7, 2021

New Video - here is exactly the same application - with a few hundred records loaded versus over 7,000 in the previous video.

Seems loading more than a trivial amount of data is grinding it to a halt. What you see below is my expectation for what we should be getting with a larger data set - even 100K records.

MOV.File.viewer._.Microsoft.Teams.mp4

@iartemiev iartemiev added DataStore Related to DataStore category React Native React Native related issue labels Jun 7, 2021
@sacrampton
Copy link
Author

Hi @manueliglesias / @iartemiev

Is there anything I can provide you to move this along? Its on a critical time line for us to get this resolved.

Thanks

Stephen

@iartemiev
Copy link
Member

@sacrampton is the performance comparable between Simulator and physical devices on both iOS and Android? Or is one worse than the other? Does query performance degrade gradually as the number of records increases? Or does it tend to be acceptable up to a certain point and then rapidly drops?

Since you said only 5 of your models have data in them, I'll attempt to reproduce this with 5 models and 10k+ records in each. Will let you know what I find.

@iartemiev
Copy link
Member

Also, which version of aws-amplify are you currently using?

@sacrampton
Copy link
Author

Hi @iartemiev

I went through the list of models and data counts in more detail. The difference between the data in each model between the first and second videos is shown here.

Thanks...

Data By Model </style>
Name High Count Low Count
Asset 7,409 359
AssetVisit 1,127 100
Photo 4,667 352
ClientPickList 1,233 1,233
FailureMode 25 25
FieldList 2,540 2,540
FieldSet 147 147
GlobalPickList 2,970 2,970
OperatingUnit 1 1
Package 34 34
PhotoButton 13 13
Plant 1 1
Risk 37 37
Scale 53 53
Uom 1,099 1,099
User 1 1
  21,357 8,965

@sacrampton
Copy link
Author

Using aws-amplify version = 3.3.27

@sacrampton
Copy link
Author

Hi @iartemiev - I had another thought on this. On the Asset object I have potentially 400+ attributes, and potentially similar on the AssetVisit object. In DynamoDB there would rarely be more than 30 attributes on any individual record so the size is quite small. And in GraphQL we just download the attributes we need for a specific app so we are never trying to download 400+ attributes.

But I'm starting to think that DataStore might is creating 400+ attributes on every single record on the mobile. In my case they would be hundreds of empty attributes taking up space. Is that what might be going on? Can you test by creating a couple of models with hundreds of empty attributes and see if that is the issue?

@iartemiev
Copy link
Member

iartemiev commented Jun 8, 2021

Thanks for the additional info. Are you returning all of the records for those models when you're querying Asset, AssetVisit, etc.? Or are you using predicates to return a subset?

@sacrampton
Copy link
Author

Hi @iartemiev - sorry I'm probably not understanding the question. Our app is multi-tenanted. So we have a base query to return only the data that belongs to that client. But in our existing GraphQL app we would only return the specific attributes from say the Asset that the App needs (say 30 attributes and not 400+ attributes).

Also, I notice in the GraphQL when it is a BELONGS_TO association that the GraphQL query is returning all the associated data and I guess it is storing that against the record. For example, and Asset in most cases has a ParentAsset - so it seems that not only is the Asset getting all 400+ blank attributes, it is storing the 400+ blank attributes of the ParentAsset. The Asset has a lot of HAS_MANY connections, so each of those objects is also getting the 400+ blank attributes of the connected Asset in its GraphQL query.

@iartemiev
Copy link
Member

I meant when you're calling DataStore.query, are you passing it a predicate, e.g., DataStore.query(Asset, c => c.someField('eq', 'someValue') or perhaps using pagination or are you retrieving all of the records from the local store by calling DataStore.query(Asset) (no predicates or pagination)?

In response to your question about fields with empty values: all of the model fields in your schema will be part of the selection sets for the GraphQL operations that populate the local store. Any empty values will come back as null from AppSync and will be persisted as null in the local store.

For example, if the records have empty values in fields field1-field5.

type Post @model {
  id: ID!
  title: String!
  field1: String
  field2: String
  field3: String
  field4: String
  field5: String
}

They would be stored as:

{
  "id": "id1",
  "title": "someTitle",
  "field1": null,
  "field2": null,
  "field3": null,
  "field4": null,
  "field5": null
}

Do you know which fields your app will need ahead of time? Or is that determined programmatically? In the former case, you could modify your schema.js and remove the fields that you don't want to be synced down.

@sacrampton
Copy link
Author

Thanks @iartemiev. I have a further question related objects - whether the data from the related object is embedded in the original object inside DataStore. The queries tend to suggest that might be the case.

So for example, I might have an optional connection to a parent Post

type Post @model {
  id: ID!
  parentPost: Post @connection (fields: "postParentPostId")
  postParentPostId: ID
  title: String!
  field1: String
  field2: String
  field3: String
  field4: String
  field5: String
}

I am wondering if the Parent record is embedded in the current record like something like this.

{
  "id": "id1",
  "title": "someTitle",
  "postParentPostId" : "id2",
  "field1": null,
  "field2": null,
  "field3": null,
  "field4": null,
  "field5": null
  "parentPost" {
     "id": "id2",
     "title": "someTitle",
     "postParentPostId" : "id3",
     "field1": null,
     "field2": null,
     "field3": null,
     "field4": null,
     "field5": null
      }
}

Also, editing the schema.js is fine - I can do that. Do I need to edit any of the other files (ie. index.d.ts, etc.)

And I guess I can have a different schema.js for each application I have so each application can get to just the attributes it needs.

@sacrampton
Copy link
Author

Hi @iartemiev - to answer your question "...I meant when you're calling DataStore.query, are you passing it a predicate, e.g., DataStore.query(Asset, c => c.someField('eq', 'someValue') or perhaps using pagination or are you retrieving all of the records from the local store by calling DataStore.query(Asset) (no predicates or pagination)?..."

We are doing the base query with a predicate as you describe and we are using pagination (500 records per cycle) because 1000 causes a resource exceeded error.

@sacrampton
Copy link
Author

Hi @iartemiev - if we expand your type Post model above and have 100 x "field01": null fields and 2 fields as you show. In DynamoDB you would just have the id/title fields and this would be 32b in size. But if I look at the size of adding 100 null fields it swells to 1,532b. On a mobile where we are trying to save memory then it seems counterintuitive to be storing null fields. Is there any opportunity to store like DynamoDB? I guess not since I think you are using SQLite..

@sacrampton
Copy link
Author

sacrampton commented Jun 9, 2021

Hi @iartemiev - Some more results of testing with the original data set of 21,357 records detailed above. Querying the asset list before was taking 12 seconds. I have created a new schema.js that reduced the number of attributes on the records down significantly for the 3 models below.

  Original New
Asset 371 121
AssetVisit 352 43
Photo 112 60

The result was that by reducing the number of attributes on the records as shown above halves the response time (6 seconds to retrieve the assets list versus 12 seconds with the the previous one. To confirm, 100% of the attributes I removed were null. The null attributes are used by other (backend) parts of the solution - but not required in mobile app.

I think this shows that syncing null attributes to a record can dramatically impact performance. Given that there is a direct connection between storing null attributes and performance it follows that if we can get rid of all null attributes (like DynamoDB does) then we are going to be maximizing performance. The only way I can think of to get rid of all null values at all times is if you were able to use some sort of a NoSQL repository on the device that only stores non-null attributes, thus optimizing space and performance?

The beauty of GraphQL and DynamoDB is that I can create a comprehensive schema, but only store minimal data - so I have huge flexibility with economy and performance. But if we are storing a null value for every single possible attribute that takes away the flexibility.

@sacrampton
Copy link
Author

sacrampton commented Jun 9, 2021

Hi @iartemiev - another test result to share with you. Used another larger dataset for testing. This data set had these quantities in these 3 models. All the other models are approximately the same quantities - they don't vary by project as they are essentially reference data objects.

  Records
Asset 30,998
AssetVisit 9,315
Photo 25,776

I tried with the Original schema.js and the New schema.js defined above.

  • Original schema took approximately 10 minutes to download to the cache, then an attempt to query the Asset model results in the app crashing with memory issues.
  • New schema took approximately 8 minutes to download to the cache, but then the Asset model was successfully queried in 1 minute and 4 seconds.

Note that when we query a model the response is irrelevant to the amount of data we are retrieving. From the 30K assets I could do a query that retrieves 1 record and it will take 1 minute and 4 seconds. I could conduct another query that returns 113 records and it take 1 minute and 4 seconds.

@sacrampton
Copy link
Author

sacrampton commented Jun 9, 2021

Hi @iartemiev - one final update from my testing. On the desktop you ship a storage adapter for IndexedDB which is a NoSQL database. So we did a test on the desktop with the large 30K+ Asset data set described above using the full schema.

It took about 11 minutes for the full dataset to download - but after that a query of the Asset table took 2 seconds. This compares with 64 seconds on Mobile App (React Native).

@iartemiev
Copy link
Member

I'll try to respond to the questions/points in order

I have a further question related objects - whether the data from the related object is embedded in the original object inside DataStore.

For Has One and Belongs To relationships we resolve the connection at run time when you call DataStore.query on that model, but they are not saved that way to the local store.

We are doing the base query with a predicate as you describe and we are using pagination (500 records per cycle) because 1000 causes a resource exceeded error.

By "base query" are you referring to the initial base sync query between AppSync and the local store? Or calling DataStore.query to retrieve records from the local store?

On a mobile where we are trying to save memory then it seems counterintuitive to be storing null fields. Is there any opportunity to store like DynamoDB? I guess not since I think you are using SQLite..

That's not possible at the moment, but it's certainly something we will explore in the future.
On React Native, DataStore utilizes Async Storage for the local store implementation. Async Storage in turn uses different underlying persistence layers depending on the platform. On iOS it uses a serialized dictionary/key-value store, not SQLite.

Note that when we query a model the response is irrelevant to the amount of data we are retrieving. From the 30K assets I could do a query that retrieves 1 record and it will take 1 minute and 4 seconds. I could conduct another query that returns 113 records and it take 1 minute and 4 seconds.

Could you please share the query you're using here? Specifically, what predicate are you passing?

@sacrampton
Copy link
Author

Hi @iartemiev - thanks for answering my long chain of rambling "input" / observations.

type Asset @model
  @key(name: "byAssetPackageId", fields: ["assetPackageId"]) 
  @key(name: "byAssetPlantId", fields: ["assetPlantId"]) 
{
  id: ID!
  name: String!
  plant: Plant @connection (fields: ["assetPlantId"])
  assetPlantId: ID!
  package: Package @connection (fields: ["assetPackageId"])
  assetPackageId: ID
  auditCompleted: Boolean
}

type Plant @model
{
  id: ID!
  name: String!   
}

type Package @model
{
  id: ID!
  name: String!   
}

The base query would be to download all assets in a Plant

query baseQuery{
  syncAssets(filter:{
    assetPlantId:{eq:"xxxxxxxxxxxx"}
  }){items{id _version}}
}

Then in DataStore with the 30K assets we do a query to find Assets in a package that have not yet been audited. This query takes 1 minute 4 seconds to return the 2 records. If we remove the limit it returns 113 records and takes the same time.

const data = await DataStore.query(
 Asset,
 (asset) =>
 asset
 .assetPackageId('eq', userData.package.id)
 .auditCompleted('eq', false),
 {
 page: 0,
 limit: 2,
 },
 );

@iartemiev
Copy link
Member

Thank you, that is helpful. We'll be doing additional library benchmarking today in order to pinpoint the issue and will provide an update with our findings.

@iartemiev
Copy link
Member

iartemiev commented Jun 10, 2021

Quick update: I have been able to repro performance issues similar to those described above when using a model with 10 fields containing values + 100 fields with null values and > 30k records in the local store. Will be investigating causes and potential solutions.

@sacrampton
Copy link
Author

Hi @iartemiev - that is fantastic news that you were able to reproduce. Hopefully you can also reproduce it working perfectly with IndexedDB/Browser too. Await your feedback.

@iartemiev
Copy link
Member

iartemiev commented Jun 10, 2021

We've pinpointed the bottleneck to the multiGet method in Async Storage, it seems that we're brushing up against that library's performance limitations here.

We'll be brainstorming what improvements we could make in DataStore in order to mitigate this as much as possible. For your use case specifically, even with the reduced number of fields, are a considerable number of values in your records still null? If so, we have a couple of ideas on how we may be able to optimize for this.

As an aside, when you're seeing this poor performance do you have the React Native debugger enabled on your app? We're seeing significantly better performance (still suboptimal, but 2-3x faster) with the debugger disabled, so just want to point that out as well.

@sacrampton
Copy link
Author

@iartemiev - looks like good progress.

Ultimately, I'd like to retain access to all fields (like we do with IndexedDB) - but for now we can run with the reduced schema. And even in the reduced schema I would estimate that 80% of those fields are null in the dataset.

I don't believe we have debugger enabled - but I will check.

@sacrampton
Copy link
Author

@iartemiev - checked performance with and without debugger enabled. The 64 second response was with React Native Debugger enabled. With Debugger switched off the response is 22 seconds. As you said - its about a 3X improvement.

Worth remembering that 22 seconds was with the stripped down schema and is still 11X slower than the Browser/IndexedDB running with the full schema. Good to know we'll see that sort of improvement when we go to production, but 22 seconds with cut-down schema still isn't a solution.

Thanks for working this.

@iartemiev
Copy link
Member

@jeremy-white, thank you for calling this out. We'll need to make sure to add owner to the SQL schema in cases where it's implicitly defined (i.e., omitted from the model fields).

@sacrampton
Copy link
Author

Thanks @iartemiev - thanks for the feedback - can confirm your suggestion does indeed return data before it is synced to the server. Just need to work out how we get a list back now.

@jeremy-white
Copy link

Just a couple more things I've hit

Object {
  "code": 6,
  "message": "NOT NULL constraint failed: TestModel.items",
}

Hit this for HAS_MANY connections defined as

    items: [Item!] @connection(keyName: "byTestModel", fields: ["id"])

as well as

    items: [String!]

I've been able to get past this by changing isRequired to false in schema.js . . . this could also be my misunderstanding but I was under the impression that [String!] meant that the items in the array had to be defined but the attribute itself could be undefined, so I was only using ! to be extra cautious. I can certainly work around this one as well but just wanted to raise it as it works as I'd expect with AsyncStorage but fails against SQLite

I'm also seeing

Error: Field items should be of type string, object received.

For fields I define as AWSJSON and have values that are arbitrary JSON . . . I haven't been able to workaround this other than just removing those fields from schema.js so if you have a better workaround I would love to hear it.

I really appreciate your work on this @iartemiev as I think it will be a HUGE improvement!

@sacrampton
Copy link
Author

Hi @iartemiev - does anything in our predicate above jump out at you as a possible issue that is not getting un-synced data? We know we can retrieve a single un-synced record whose id we know. But the search for a list of records that included synced and un-synced records is something we are still struggling with. Any hints you might have on what we might be doing wrong in the above predicate would be much appreciated.

We are going live with our App this weekend - we wouldn't have got here without your amazing support in this effort. Much appreciated.

@iartemiev
Copy link
Member

iartemiev commented Jul 17, 2021

@sacrampton, I don't think there's anything inherently wrong with it. It's hard to know for sure without being familiar with your app code and knowing what values are being passed and what data is expected to be present in the local store. DataStore doesn't use a cache. Every time you execute DataStore.query, it performs a SQL query against your local database.

If I were debugging this, I would check that each of the operands has the expected value by logging them out to the console.

You can also try removing fields from your predicates one by one to determine which one is causing all the results to be filtered out.

Try something like the following in the same place as where you're normally attempting to query the local records (and before the sync):

const newAsset = await DataStore.save(new Asset({...assetData}));
const data = await DataStore.query(Asset, newAsset.id);

console.log(data.assetParentAssetId === mobileActiveLocation?.id, data.assetParentAssetId, mobileActiveLocation?.id);
console.log(data.assetPackageId === userPackageId, data.assetPackageId, userPackageId);
console.log(data.assetPlantId === userPlantId, data.assetPlantId, userPlantId);
console.log(data.isDeleted !== true, data.isDeleted);

If you have a false in any of those 4 console logs, that should give you the problematic field.

Lastly, if you have debug logs on, you should see the SQL SELECT statement that is generated by that predicate. Does it look correct?

@sacrampton
Copy link
Author

Hi @iartemiev - the count suggestion you had above works for smaller data sets in iOS (extremely slow with larger data sets - > 1 minute). But in Android we get the following error.

{​​​​​​​​"message":"no such column: false (code 1 SQLITE_ERROR): , while compiling: SELECT * FROM Asset WHERE auditCompleted = false","code":0}​​​​​​​​​​​​​​​

@iartemiev
Copy link
Member

@sacrampton SQLite represents boolean values as 0 and 1. So if you're specifying a literal false in your WHERE clause it attempts to look up a column with the name "false" as the error message suggests.

You can either specify WHERE auditCompleted = 0 or use a parameterized query. The latter will coerce a JS boolean to the SQLite representation.

re: performance, I'm surprised it's running that slowly. I'll try to reproduce on my end to confirm.

@sacrampton
Copy link
Author

sacrampton commented Jul 22, 2021

Hi @iartemiev - replacing "false" with 0 is not working for us... The error seems to be that the column we are calling is not there. But we know it is there as it works in other queries from DataStore - just not count in SQLite.

@iartemiev
Copy link
Member

What's the error you get when you query WHERE auditCompleted = 0?

By the way, are you calling SELECT COUNT(*) as count FROM Asset? or SELECT * FROM Asset? If you're doing the latter as your previous comment suggests, it will return all records from the DB, which is going to be slow. If you're just trying to get a count, there's no reason to do that.

@sacrampton
Copy link
Author

Hi @iartemiev - here are a few points....

On iOS if I do the following query it gives me the correct answer and does so very quickly.

SELECT COUNT(*) as count FROM Asset WHERE auditCompleted = true

If we execute exactly the same query on Android it gives this error

Object {
"code": 0,
"message": "Cannot bind argument at index 1 because the index is out of range. The statement has 0 parameters.",
}

We were using the other SELECT statement because we can't seem to get the COUNT to work with an AND statement - the following query in iOS returns an incorrect value of 1. In Android it returns the same error.

SELECT COUNT(*) as count FROM Asset WHERE auditCompleted = true AND packageName = JONESBORO

@iartemiev
Copy link
Member

Try parameterizing the query and enclose the WHERE clause expression in parentheses.

E.g.,

await db.executeSql('SELECT COUNT(*) as count FROM Asset WHERE (auditCompleted = ? AND packageName = ?)', [true, 'JONESBORO']);

@sacrampton
Copy link
Author

sacrampton commented Jul 22, 2021

Hi @iartemiev - thanks for that - did the trick now. So resolved all issues around the count. Thanks so much as always.

Still noticing the delta-sync on SQLite is slower than before. Not sure if you were seeing that.

@iartemiev
Copy link
Member

My pleasure! Very glad to hear that things are finally coming together!

Still noticing the delta-sync on SQLite is slower than before. Not sure if you were seeing that.

Hmm, I remember you mentioning that in your email, but I haven't observed it yet myself. I'll do some more benchmarking around this specifically when I get a chance.

  • How big is the performance difference?
  • Is the number of items returned by Delta sync similar when you were testing this with AsyncStorage vs. now with SQLite?

@mjaydeep01
Copy link

mjaydeep01 commented Aug 3, 2021

Hi @iartemiev - some more updates here.

We deleted node modules and pod files, and follow all the steps above, but when we check the code it is still getting data from AsyncStorage.

const model = 'Asset';
 
 const key = (await AsyncStorage.getAllKeys()).find((key) =>
 key.includes(`@AmplifyDatastore::user_${model}`),
 );
 
 console.log('Item from aync storage >>', await AsyncStorage.getItem(key));

So clearly we are not doing something right here. Can you see if there is anything obvious that we are omitting.

Suspect something is being cached in existing project so will create a new project from scratch, but that will take several hours, an see if that works for us.

@iartemiev, @sacrampton I am facing same issue on Android, no matter what i do, data is always being stored in the AsyncStorage. Steps, I followed are as below..

  1. Delete node_modules folder, any .lock file.
  2. Remove app from the Android device.
  3. npm install
  4. npm install aws-amplify@rn-sqlite amazon-cognito-identity-js@rn-sqlite aws-amplify-react-native@rn-sqlite react-native-sqlite-storage
  5. Install app on the device.
  6. On querying the data from datastore, data downloaded from the appsync, but stored in the AsyncStorage.

Am I missing any specific setting?

@sacrampton
Copy link
Author

Hi @iartemiev - want to push further into the slowness we are seeing for DeltaSync

DataStore creates a separate table in DynamoDB to manage the DeltaSync called "AmplifyDataStore-ENV".

There are no indexes in this table - just the partition key and sort key - where the partition key is table/date and sort key is time/id/version.

Our database is multi-tenanted - and we deal with assets in industrial plants. So I could have hundreds of other users in other plants making massive amounts of changes. But I might not have any users working my plant. The DeltaSync as I see it is going to have to sort through everyone else's changes just to work out there are zero changes that are going to be applicable to me.

When we initially hydrate the cache we do a base query which uses GSI's to get a quick response.

At the moment I'm seeing DeltaSync take about the same amount of time as the full sync (20 minutes). Today I know I was doing a lot of bulk updating of data in a few different plants through our web back end. Not an unusually large workload. But I am concerned from the slowness I'm seeing in our database and what I see in the DeltaSync table for DynamoDB has me worried that this is not scalable for a multi-tenanted environment.

You've been really good at coming up with solutions to get us moving - hopefully someone else has already come up with a solution to make the DeltaSync run in seconds rather than 20+ minutes.

@iartemiev
Copy link
Member

@mjaydeep01 try deleting your node_modules and lock file, then adding the following to your package.json and then running npm install:

  "resolutions": {
    "@aws-amplify/datastore": "rn-sqlite",
    "@aws-amplify/core": "rn-sqlite",
    "@aws-amplify/*": "rn-sqlite"
  },

@iartemiev
Copy link
Member

@sacrampton, I think this behavior likely warrants a separate GitHub issue, unless this is somehow related to the on-device database on React Native specifically (AsyncStorage or SQLite).

To better understand what's going on, I have some follow up questions:

  1. How many total records are in the delta sync table in DynamoDB at the time that you're seeing the 20 min delta sync time?
  2. How many of those records are being synced down to the app?
  3. Are you using DataStore.configure to change any of the sync-related settings (e.g., syncPageSize, fullSyncInterval, etc.)? If so, which settings are you using?
  4. Are you seeing roughly the same delta sync performance if you test this in a web app?

@sacrampton
Copy link
Author

Hi @iartemiev - have created a separate GitHub issue as requested - #8699

@mjaydeep01
Copy link

mjaydeep01 commented Aug 10, 2021

@iartemiev, thanks for the new suggestion...
npm does not seems to using this resolutions at all.
With Yarn install, I am getting following errors..

warning Resolution field "rn-sqlite" has an invalid version entry and may be ignored
warning Resolution field "rn-sqlite" has an invalid version entry and may be ignored
warning Resolution field "@aws-amplify/*" does not end with a valid package name and will be ignored

yarn --version --> 1.22.5
npm --version --> 7.20.5

Can you please suggest how to resolve this issues?

@iartemiev
Copy link
Member

@sacrampton, @jeremy-white, @mjaydeep01 - this feature has been officially released as part of aws-amplify@4.2.6! The documentation change will follow shortly, but in the meantime, here's how you can enable it in your app:

(Note: this new storage adapter is only compatible with React Native CLI-generated apps. Expo support will be added in the future)

Install:

$ npm install aws-amplify @aws-amplify/datastore-storage-adapter react-native-sqlite-storage aws-amplify-react-native amazon-cognito-identity-js @react-native-community/netinfo @react-native-async-storage/async-storage
$ npx pod-install

Enable in your app:

import { DataStore } from 'aws-amplify';
import { SQLiteAdapter } from '@aws-amplify/datastore-storage-adapter';

DataStore.configure({
  storageAdapter: SQLiteAdapter
});

I'm closing this issue, as the new storage adapter addresses the performance concerns expressed in the issue. Please create a new issue if you require assistance with this feature.

@sacrampton
Copy link
Author

Hi @iartemiev - thank you for going above and beyond to make this happen

@prasadk-foodhub
Copy link

prasadk-foodhub commented Jan 10, 2022

npm install aws-amplify @aws-amplify/datastore-storage-adapter react-native-sqlite-storage aws-amplify-react-native amazon-cognito-identity-js @react-native-community/netinfo @react-native-async-storage/async-storage

SqliteAdapter prints empty object.Please confirm sync is not running after doing this

@prasadk-foodhub
Copy link

prasadk-foodhub commented Jan 10, 2022

no such table: Setting
no such table: ModelMetadata

This are the errors logs.Please advise .I tried below code

import { DataStore } from 'aws-amplify';
import { SQLiteAdapter } from '@aws-amplify/datastore-storage-adapter';

DataStore.configure({
storageAdapter: SQLiteAdapter
});

@RemyNtshaykolo
Copy link

no such table: Setting no such table: ModelMetadata

This are the errors logs.Please advise .I tried below code

import { DataStore } from 'aws-amplify'; import { SQLiteAdapter } from '@aws-amplify/datastore-storage-adapter';

DataStore.configure({ storageAdapter: SQLiteAdapter });

Hey have you found a solution for this issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
DataStore Related to DataStore category React Native React Native related issue
Projects
None yet
Development

No branches or pull requests

8 participants