Expo next storage support config plugins

[Stevemoretz]

Proposal

Hello, this 6mb limit always has been a pain in the arm, so with the new EAS build system it's now possible to lift it. (According to Brent Vatne himself the creator of expo : https://expo.canny.io/feature-requests/p/ability-to-increase-async-storage-size).

It would be great to have a config plugin : https://docs.expo.dev/guides/config-plugins/
which can add support of next storage, so that 6mb limit would get lifted.

Alternatives

The alternative of course can be not using the next storage, and the old one but be able to configure the size, that is possible if you search this text in the link I provided in the next section Some plugins can be customized by passing an array, where the second argument is the options:

Implementation Details

I don't know much but I have seen other libraries do the same for example rn-firebase has done it, and many more.
But the full tutorial on that seems to be here : https://docs.expo.dev/guides/config-plugins/

Additional Context

No response

[wodin]

If it's just a matter of adding the following to gradle.properties a config plugin will be very simple:

AsyncStorage_db_size_in_MB=10

Or do you also need to update build.gradle?

[Stevemoretz]

If it's just a matter of adding the following to gradle.properties a config plugin will be very simple:

AsyncStorage_db_size_in_MB=10

Or do you also need to update build.gradle?

Hello thank you for the reply, for my needs if the next storage doesn't have this limitation : #640 (comment)

There's also an Android OS limitation on how big the data can be while reading from the DB (CursorWindow limitation) and that's around 2MB.

I would need that one, otherwise just having AsyncStorage_db_size_in_MB=10 option would be more than enough.

[wodin]

hmmm... if I understand you correctly, you need to be able to retrieve more than 2MB of data when reading a single key? It sounds like this "Android OS limitation" is not something that you could increase. Only the total database size.

If you don't mind, what kind of data is this?

Also, have you considered using something else to store the data? e.g. https://github.com/mrousavy/react-native-mmkv/
But I don't know if MMKV has size limitations. I don't think either AsyncStorage or MMKV are really intended for storing big values in a key.

[Stevemoretz]

hmmm... if I understand you correctly, you need to be able to retrieve more than 2MB of data when reading a single key? It sounds like this "Android OS limitation" is not something that you could increase. Only the total database size.

If you don't mind, what kind of data is this?

Also, have you considered using something else to store the data? e.g. https://github.com/mrousavy/react-native-mmkv/ But I don't know if MMKV has size limitations. I don't think either AsyncStorage or MMKV are really intended for storing big values in a key.

Hello Micheal, the exact use case of mine is with using redux-persistence it just stores all the data under one single key, I'm caching a few jsons coming back from the server, these jsons could grow over the time and that could cause the 2mb overflow problem.

I had considered the expo file system, haven't tried it yet though, never had heard of mmkv! Sounds amazing I'll open an issue there to ask see if they're also using the database or not!

Thank you again!

[Stevemoretz]

I got a response back from mmkv sounds like it's not useful for big objects as well, but that's okay, so forgetting about the 2mb limit.

How about a Plugin for changing the 6mb limit? That sounds like a good idea yet I couldn't make it myself since expo has support only for vscode for developing and validating plugins, but I have Intellij Idea, I couldn't figure out how to use the mod I know a mod exists exactly for changing gradle properties tried to use it but it's not really obvious what to do there!

The hook is : withGradleProperties
But couldn't find a single example

[github-actions]

This issue has been marked as stale due to inactivity. Please respond or otherwise resolve the issue within 7 days or it will be closed.

[wodin]

How about a Plugin for changing the 6mb limit? That sounds like a good idea yet I couldn't make it myself since expo has support only for vscode for developing and validating plugins, but I have Intellij Idea

You don't need VS Code. I don't use it. I've written a few simple plugins and I just test them by running expo prebuild and checking the results.

I couldn't figure out how to use the mod I know a mod exists exactly for changing gradle properties tried to use it but it's not really obvious what to do there!

The hook is : withGradleProperties But couldn't find a single example

Maybe these will help:

• https://github.com/expo/config-plugins/blob/9b22a71343d74d71353d8c6535e7d04da2fc4473/packages/react-native-webrtc/src/withDesugaring.ts#L9
• https://github.com/expo/expo/blob/a9c4f103b23f4ed907a362c31c12eae14382d95c/apps/native-component-list/app.config.js#L32

[ansh]

Here is a Stackoverflow issue that basically creates a config plugin for increasing the size

I also just wrote one with setting up Next Storage:

// withNextStorage.ts
// Next Storage Implementation for react-native-async-storage https://react-native-async-storage.github.io/async-storage/docs/advanced/next/
import {
  ConfigPlugin,
  createRunOncePlugin,
  withGradleProperties,
  withProjectBuildGradle,
} from "@expo/config-plugins";
import { MergeResults, mergeContents } from "@expo/config-plugins/build/utils/generateCode";

const pkg = require("@react-native-async-storage/async-storage/package.json");

const addKotlinGradlePlugin = (src: string): MergeResults => {
  return mergeContents({
    tag: "react-native-async-storage withNextStorage config plugin",
    src,
    newSrc: `classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlinVersion"`,
    anchor: /dependencies(?:\s+)?\{/,
    offset: 1,
    comment: "//",
  });
};

const withDangerousMod: ConfigPlugin = (config) => {
  return withProjectBuildGradle(config, (config) => {
    if (config.modResults.language === "groovy") {
      config.modResults.contents = addKotlinGradlePlugin(config.modResults.contents).contents;
    } else {
      throw new Error(
        "Cannot add Snapkit maven gradle because the project build.gradle is not groovy"
      );
    }
    return config;
  });
};

const withGradlePropertyMod: ConfigPlugin = (config) => {
  return withGradleProperties(config, (config) => {
    config.modResults.push({
      type: "property",
      key: "AsyncStorage_useNextStorage",
      value: "true",
    });
    return config;
  });
};

const withNextStorage: ConfigPlugin = (config) => {
  return withGradlePropertyMod(withDangerousMod(config));
};

export default createRunOncePlugin(withNextStorage, pkg.name, pkg.version);

I can publish this as a config plugin package. Let me know @tido64

[tido64]

I can publish this as a config plugin package. Let me know @tido64

@ansh: Can this be part of the AsyncStorage package? I think it would be nice if they came together. It would also make updating it a lot easier. What do you think, @krizzu?

[ansh]

@tido64 According to the Expo documentation, what I did is a “dangerous mod” and any interaction with gradle files should be handled by Autolinking. I wouldn’t know how to do that but @krizzu would need to do that rather than try to bundle the config plug-in I wrote with AsyncStorage.

[tido64]

@tido64 According to the Expo documentation, what I did is a “dangerous mod” and any interaction with gradle files should be handled by Autolinking

I see. I wonder if we should allow users to configure AsyncStorage from a more central place, like react-native.config.js. We would at least avoid recommending people to use a dangerous mod.

[wodin]

any interaction with gradle files should be handled by Autolinking.

I'm not sure that's right. The Expo docs say:

Android Gradle Files

Gradle files are written in either Groovy or Kotlin. They are used to manage dependencies, versioning, and other settings in the Android app. Instead of modifying them directly with the withProjectBuildGradle, withAppBuildGradle, or withSettingsGradle mods, utilize the static gradle.properties file.

The gradle.properties is a static key/value pair that groovy files can read from. For example, say you wanted to control some toggle in Groovy:

gradle.properties

expo.react.jsEngine=hermes

Then later in a Gradle file:

app/build.gradle

project.ext.react = [enableHermes: findProperty('expo.react.jsEngine') ?: 'jsc']

• For keys in the gradle.properties, use camel case separated by .s, and usually starting with the expo prefix to denote that the property is managed by prebuild.
• To access the property, use one of two global methods:
  • property: Get a property, throw an error if the property is not defined.
  • findProperty: Get a property without throwing an error if the property is missing. This can often be used with the ?: operator to provide a default value.

The above seems pretty clear on its own, but then they go on to say the stuff about using autolinking:

Generally, you should only interact with the Gradle file via Expo Autolinking, this provides a programmatic interface with the project files.

They also say:

Best practices for mods

Avoid regex: static modification is key. If you want to modify a value in an Android gradle file, consider using gradle.properties. If you want to modify some code in the Podfile, consider writing to JSON and having the Podfile read the static values.

So it's a bit ambiguous, but it still seems that it should be fine to modify AsyncStorage's build.gradle to look for a property and then use withGradleProperties (which is not dangerous) to set the property.

Here's an existing config plugin that makes use of withGradleProperties in case it's useful:

https://github.com/expo/config-plugins/blob/fee9e88c13e4a6b727f69d6868bc796e3d1d9aea/packages/react-native-webrtc/src/withDesugaring.ts

[ansh]

@wodin The config plugin I wrote above works great so I am aware of using withGradleProperties as well as the dangerous mod withProjectBuildGradle.

In any case, what I was answering was that @tido64 wants this to part of the AsyncStorage package. However, that is only possible by modifying the package and its Autolinking process. The config plugin I wrote would have nothing to do with that.

[wodin]

@ansh I don't really know much about Autolinking at all, despite having read the documentation on it. I also haven't looked too closely at what your config plugin actually changes in build.gradle. I am also a bit lost when it comes to most aspects of gradle. So I'm probably missing something.

But, would it be possible to get this to work by modifying AsyncStorage's build.gradle to check for a property and depending on the value of the property, enable or disable next storage support? Then provide a config plugin that calls withGradleProperties to set the property?

Or would the app's own build.gradle need to check the property? If that's the case then I'm confused as to how withGradleProperties is meant to be used if you still have to use something like withProjectBuildGradle to modify the build.gradle so that it can check the property.

[tido64]

After some digging, I just realized that we already provide an "easy" way for people to enable next storage: https://github.com/react-native-async-storage/async-storage/blob/774fb7828219823195ee704099bbbd902ffc5d07/website/docs/advanced/Next.md#enable

Is this not compatible with Expo?

[krizzu]

hey, Happy new year everyone!

From what I understand, the only way to add a custom property to gradle.properties for Expo to pick up is via config plugin's withGradleProperties correct?

If so, then the plugin would be as easy as calling withGradleProperties and adding AsyncStorage_useNextStorage=true. Expo already supports Kotlin (no need to add gradle plugin) and you can specify kotlin version via build properties.

[wodin]

@krizzu Happy New Year to you too :)

Yes, I think you're right!

[ansh]

@krizzu @wodin @tido64 I tried just adding the AsyncStorage_useNextStorage=true using withGradleProperties but it did not work. I had to write the addKotlinGradlePlugin as well which added classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlinVersion" to dependencies in build.gradle.

I forget the error but I can create a new repro if required.

[wodin]

If I create a new Expo app and then run npx expo prebuild -p android I find references to kotlin-gradle-plugin in the following dependencies' build.gradle files:

node_modules/expo-constants/android/build.gradle
node_modules/expo/android/build.gradle
node_modules/expo-keep-awake/android/build.gradle
node_modules/react-native/ReactAndroid/build.gradle
node_modules/expo-error-recovery/android/build.gradle
node_modules/expo-modules-core/android/build.gradle
node_modules/expo-font/android/build.gradle
node_modules/expo-application/android/build.gradle
node_modules/expo-file-system/android/build.gradle
node_modules/expo-splash-screen/android/build.gradle

but not in the app's build.gradle file.

So should this be in @react-native-async-storage/async-storage's build.gradle instead of the app's?
I see it already has something like that, but only if projectExampleDir == rootProjectDir.

async-storage/android/build.gradle

Lines 51 to 53 in cfa67c6

	if (projectExampleDir == rootProjectDir) {
	classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$asyncStorageKtVersion"
	}

If it's needed for "next storage" support then maybe this if statement could be amended to something like this?

        if ((projectExampleDir == rootProjectDir) || useNextStorage) {
            classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$asyncStorageKtVersion"
        }

[ansh]

@wodin That is a nice solution! Then the config plugin could be super simple and wouldn’t need any dangerous mods.

[krizzu]

   if ((projectExampleDir == rootProjectDir) || useNextStorage) {
       classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$asyncStorageKtVersion"
   }

Yes, that would work - original idea was that the root app (React Native app, using AsyncStorage) should apply the plugin and we'll just reuse it. With that change, simply setting AsyncStorage_useNextStorage=true will do the trick of enabling Next storage for project.

[wodin]

FWIW I've been told the following:

It is generally considered to be safe to include the kotlin-gradle-plugin in the build Gradle of a library. This plugin ensures that the app uses the correct Kotlin build tools, and it is a common practice for libraries to include it in their build Gradle as well, as seen in popular libraries like Coil (https://github.com/coil-kt/coil/blob/a3f7e9d8ce98783cf4ce1b1d84657104d6b123b7/coil-base/build.gradle.kts#L5).

EDIT: Corrected link: https://github.com/coil-kt/coil/blob/a3f7e9d8ce98783cf4ce1b1d84657104d6b123b7/buildSrc/build.gradle.kts#L12

[TreeOfLearning]

Was there ever any progress on this? I can't find any mention of a config plugin in the documentation. Our use case is that we need to store several strings, the total size of which will exceed 6MB, while each string will be between 1 and 2MB. We are running on an expo custom dev client in the managed workflow, so while we can use config plugins, we don't have the capability to modify build.gradle files to either increase the 6MB limit or use "Next storage". Any help would be appreciated!

[amitchaudhary140]

I am looking for plugin to exceed 6MB limit in expo managed app. Can someone post the solution pls.

[nympheastudio]

amit

on expo51 i got limited in expoGO but on production im not limited, with eas build in my app.json android storage is defined like this : "plugins": [ [ "expo-build-properties", { "android": { "AsyncStorage_db_size_in_MB": "500" }, "ios": { "deploymentTarget": "13.4" } } ],