Detect `ccache` and provide a default configuration #42051

kraenhansen · 2023-12-22T23:30:21Z

Summary:

Building native modules from source, may take a long time. Xcode already helps bring this down, by providing incremental builds, as long as the user doesn't delete their ios/build directory. But in some situations, i.e. when iterating the native code of an app or library or when the developer need to delete that ios/build directory, it's advantageous to use a compiler cache, such as ccache. This is already outlined in our "Speeding up your Build phase" guide.

But setting up an Xcode project to use Ccache with the correct configuration, isn't trivial in a way that doesn't require symlinking clang and clang++ or passing configuration via environment variables on every npm run ios invokation.

This PR takes its inspiration from the existing guide on setting up Ccache for Xcode, but applies the build settings only if an installation of ccache is detected and the feature is explicitly opted into via an argument to the react_native_post_install function or a USE_CCACHE environment variable. It uses two shell scripts to wrap the call to ccache, which both injects a default CCACHE_CONFIGPATH environment variable (i.e. it won't override this if already provided, to allow for customisations on CI), pointing to a ccache.config which works well with React Native projects (it has the same values as the guide mentions).

For context, I posted about this change in the ios channel of the contributors Discord server, where I discussed it with @cipolleschi and @Saadnajmi

Additional output printed when running `pod install`

When `ccache_available and ccache_enabled`

[Ccache]: Ccache found at /opt/homebrew/bin/ccache
[Ccache]: Setting CC, LD, CXX & LDPLUSPLUS build settings

When `ccache_available and !ccache_enabled`

[Ccache]: Ccache found at /opt/homebrew/bin/ccache
[Ccache]: Pass ':ccache_enabled => true' to 'react_native_post_install' in your Podfile or set environment variable 'USE_CCACHE=1' to increase the speed of subsequent builds

When `!ccache_available and ccache_enabled`

[!] [Ccache]: Install ccache or ensure your neither passing ':ccache_enabled => true' nor setting environment variable 'USE_CCACHE=1'

Otherwise

If the user doesn't have ccache installed and doesn't explicitly opt into this feature, nothing will be printed.

Changelog:

[IOS] [ADDED] - Added better support for ccache, to speed up subsequent builds of native code. After installing ccache and running pod install, the Xcode project is injected with compiler and linker build settings pointing scripts that loads a default Ccache configuration and invokes the ccache executable.

Test Plan:

I've tested this manually - would love some inspiration on how to automate this, if the reviewer deem it needed.
To test this locally:

Install Ccache and make sure the ccache executable is in your PATH (verify by running ccache --version)
Create a new template app instance and apply the changes of this PR to the node_modules/react-native package.
Set the USE_CCACHE environment variable using export USE_CCACHE=1.
Run pod install in the ios directory.
Check the stats of Ccache (running ccache -s).
Run npm run ios or build the project from Xcode.
Check the Ccache stats again to verify ccache is intercepting compilation ("Cacheable calls" should ideally be 100%).
To check the speed gain:
a. Delete the ios/builds directory
b. Zero out the ccache stats (by running ccache -z)
c. Run pod install again (only needed if you ran the initial pod install with new architecture enabled RCT_NEW_ARCH_ENABLED=1).
d. Run npm run ios or build the project from Xcode.
e. This last step should be significantly faster and you should see "Hits" under "Local storage" in the ccache stats approach 100%.

Saadnajmi · 2023-12-22T23:43:09Z

cc @tido64 but he might be out till after new years

packages/react-native/scripts/cocoapods/utils.rb

kraenhansen · 2023-12-22T23:49:11Z

packages/react-native/scripts/xcode/ccache.conf

+sloppiness = clang_index_store,file_stat_matches,include_file_ctime,include_file_mtime,ivfsoverlay,pch_defines,modules,system_headers,time_macros
+file_clone = true
+depend_mode = true
+inode_cache = true


I've been contemplating adding the compiler_check = content mentioned in the guide: https://reactnative.dev/docs/build-speed#using-this-approach-on-a-ci .. I believe it'll make compilations marginally slower, but it would make this even more useful on CI.

Have you checked how cacheable this config is? We've been using this on CI and have typically seen cache hits above 95% when bumping patch releases: https://github.com/microsoft/react-native-test-app/blob/449fe0254d9a8aebe89322fc182ddf7643e07c23/.ccache/ccache.conf

This config is the one suggested on our "Speeding up your Build phase" guide, just converted from env variables to the ccache conf format. I've seen 100% hit rates on a newly instantiated template while deleting the ios/build directory in between builds.

analysis-bot · 2023-12-22T23:58:41Z

Platform	Engine	Arch	Size (bytes)	Diff
android	hermes	arm64-v8a	16,577,949	-16
android	hermes	armeabi-v7a	n/a	--
android	hermes	x86	n/a	--
android	hermes	x86_64	n/a	--
android	jsc	arm64-v8a	19,955,040	+2
android	jsc	armeabi-v7a	n/a	--
android	jsc	x86	n/a	--
android	jsc	x86_64	n/a	--

Base commit: c75abef
Branch: main

kraenhansen · 2023-12-23T05:39:56Z

packages/react-native/scripts/cocoapods/utils.rb

+
+        ccache_path = `command -v ccache`.strip
+
+        if !ccache_path.empty?


We could add an env variable here to disable this behavior, in case the developer has ccache but wants the default build settings. It could be DISABLE_CCHACE=true or ENABLE_CCHACHE=false?

This should be opt-in. We cannot break existing setups. For example, it is possible to use remote caching. A hybrid app would also require a custom setup.

Note that these can be overwritten by the end-users Podfile, but I agree, it's probably safest to have users opt into this. I wonder if it would be polite enough to prompt (a simple log message) a developer that already has ccache installed to supply the opt in environment variable? Making this feature easier to discover.

I've updated the description with the additional output printed.

cipolleschi

A part of a couple of very nit styling thing with ruby, I like this very much.
It is completely optionals and has a great potential!

I'm looking forward to see if it can also improve our CircleCI buids.

cipolleschi · 2023-12-27T15:47:17Z

packages/react-native/scripts/cocoapods/utils.rb

+        ccache_path = `command -v ccache`.strip
+        ccache_available = !ccache_path.empty?
+
+        message_prefix = ccache_available ? "⚡️".yellow + "Ccache detected (found at %s): " % [ccache_path.italic] : "Unable to locate ccache: "


this prefix can become quite long, as it depends on the path found at line 92.
What about just have [CCACHE] as a general prefix and have a first line of log that is

Pod::UI.puts("#{message_prefix}: Ccache found at #{ccache_path}")

?

Also, I prefer using string interpolation rather than formatting. It feels more Swift/iOS and it is easier to read as the symbols are already replaced by their variable.

Makes sense 👍 I've updated the code to use string interpolation, a [Ccache] prefix, simplified the formatting used and updated the PR description with the output printed.

facebook-github-bot · 2023-12-27T16:13:56Z

@cipolleschi has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

…ilable

facebook-github-bot · 2023-12-27T20:46:07Z

@cipolleschi has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

github-actions · 2023-12-28T09:46:12Z

This pull request was successfully merged by @kraenhansen in e85d51c.

^{When will my fix make it into a release? | Upcoming Releases}

facebook-github-bot · 2023-12-28T09:46:12Z

@cipolleschi merged this pull request in e85d51c.

Summary: Building native modules from source, may take a long time. Xcode already helps bring this down, by providing incremental builds, as long as the user doesn't delete their `ios/build` directory. But in some situations, i.e. when iterating the native code of an app or library or when the developer need to delete that `ios/build` directory, it's advantageous to use a compiler cache, such as ccache. This is already outlined in our ["Speeding up your Build phase"](https://reactnative.dev/docs/build-speed#xcode-specific-setup) guide. But setting up an Xcode project to use Ccache with the correct configuration, isn't trivial in a way that doesn't require symlinking `clang` and `clang++` or passing configuration via environment variables on every `npm run ios` invokation. This PR takes its inspiration from the existing guide on [setting up Ccache for Xcode](https://reactnative.dev/docs/build-speed#xcode-specific-setup), but applies the build settings only if an installation of `ccache` is detected and the feature is explicitly opted into via an argument to the `react_native_post_install` function or a `USE_CCACHE` environment variable. It uses two shell scripts to wrap the call to `ccache`, which both injects a default `CCACHE_CONFIGPATH` environment variable (i.e. it won't override this if already provided, to allow for customisations on CI), pointing to a `ccache.config` which works well with React Native projects (it has the same values as the guide mentions). For context, I posted about this change in the ios channel of the contributors Discord server, where I discussed it with cipolleschi and saadnajmi ### Additional output printed when running `pod install` #### When `ccache_available and ccache_enabled` ``` [Ccache]: Ccache found at /opt/homebrew/bin/ccache [Ccache]: Setting CC, LD, CXX & LDPLUSPLUS build settings ``` #### When `ccache_available and !ccache_enabled` ``` [Ccache]: Ccache found at /opt/homebrew/bin/ccache [Ccache]: Pass ':ccache_enabled => true' to 'react_native_post_install' in your Podfile or set environment variable 'USE_CCACHE=1' to increase the speed of subsequent builds ``` #### When `!ccache_available and ccache_enabled` ``` [!] [Ccache]: Install ccache or ensure your neither passing ':ccache_enabled => true' nor setting environment variable 'USE_CCACHE=1' ``` #### Otherwise If the user doesn't have ccache installed and doesn't explicitly opt into this feature, nothing will be printed. bypass-github-export-checks ## Changelog: [IOS] [ADDED] - Added better support for `ccache`, to speed up subsequent builds of native code. After installing `ccache` and running `pod install`, the Xcode project is injected with compiler and linker build settings pointing scripts that loads a default Ccache configuration and invokes the `ccache` executable. Pull Request resolved: facebook#42051 Test Plan: I've tested this manually - would love some inspiration on how to automate this, if the reviewer deem it needed. To test this locally: 1. Install Ccache and make sure the `ccache` executable is in your `PATH` (verify by running `ccache --version`) 2. Create a new template app instance and apply the changes of this PR to the `node_modules/react-native` package. 3. Set the `USE_CCACHE` environment variable using `export USE_CCACHE=1`. 4. Run `pod install` in the `ios` directory. 5. Check the stats of Ccache (running `ccache -s`). 6. Run `npm run ios` or build the project from Xcode. 7. Check the Ccache stats again to verify ccache is intercepting compilation ("Cacheable calls" should ideally be 100%). 8. To check the speed gain: a. Delete the `ios/builds` directory b. Zero out the ccache stats (by running `ccache -z`) c. Run `pod install` again (only needed if you ran the initial `pod install` with new architecture enabled `RCT_NEW_ARCH_ENABLED=1`). d. Run `npm run ios` or build the project from Xcode. e. This last step should be significantly faster and you should see "Hits" under "Local storage" in the ccache stats approach 100%. Reviewed By: huntie Differential Revision: D52431507 Pulled By: cipolleschi fbshipit-source-id: 6cfe39acd6250fae03959f0ee74d1f2fc46b0827

Summary: Building native modules from source, may take a long time. Xcode already helps bring this down, by providing incremental builds, as long as the user doesn't delete their `ios/build` directory. But in some situations, i.e. when iterating the native code of an app or library or when the developer need to delete that `ios/build` directory, it's advantageous to use a compiler cache, such as ccache. This is already outlined in our ["Speeding up your Build phase"](https://reactnative.dev/docs/build-speed#xcode-specific-setup) guide. But setting up an Xcode project to use Ccache with the correct configuration, isn't trivial in a way that doesn't require symlinking `clang` and `clang++` or passing configuration via environment variables on every `npm run ios` invokation. This PR takes its inspiration from the existing guide on [setting up Ccache for Xcode](https://reactnative.dev/docs/build-speed#xcode-specific-setup), but applies the build settings only if an installation of `ccache` is detected and the feature is explicitly opted into via an argument to the `react_native_post_install` function or a `USE_CCACHE` environment variable. It uses two shell scripts to wrap the call to `ccache`, which both injects a default `CCACHE_CONFIGPATH` environment variable (i.e. it won't override this if already provided, to allow for customisations on CI), pointing to a `ccache.config` which works well with React Native projects (it has the same values as the guide mentions). For context, I posted about this change in the ios channel of the contributors Discord server, where I discussed it with cipolleschi and saadnajmi ### Additional output printed when running `pod install` #### When `ccache_available and ccache_enabled` ``` [Ccache]: Ccache found at /opt/homebrew/bin/ccache [Ccache]: Setting CC, LD, CXX & LDPLUSPLUS build settings ``` #### When `ccache_available and !ccache_enabled` ``` [Ccache]: Ccache found at /opt/homebrew/bin/ccache [Ccache]: Pass ':ccache_enabled => true' to 'react_native_post_install' in your Podfile or set environment variable 'USE_CCACHE=1' to increase the speed of subsequent builds ``` #### When `!ccache_available and ccache_enabled` ``` [!] [Ccache]: Install ccache or ensure your neither passing ':ccache_enabled => true' nor setting environment variable 'USE_CCACHE=1' ``` #### Otherwise If the user doesn't have ccache installed and doesn't explicitly opt into this feature, nothing will be printed. bypass-github-export-checks ## Changelog: [IOS] [ADDED] - Added better support for `ccache`, to speed up subsequent builds of native code. After installing `ccache` and running `pod install`, the Xcode project is injected with compiler and linker build settings pointing scripts that loads a default Ccache configuration and invokes the `ccache` executable. Pull Request resolved: facebook/react-native#42051 Test Plan: I've tested this manually - would love some inspiration on how to automate this, if the reviewer deem it needed. To test this locally: 1. Install Ccache and make sure the `ccache` executable is in your `PATH` (verify by running `ccache --version`) 2. Create a new template app instance and apply the changes of this PR to the `node_modules/react-native` package. 3. Set the `USE_CCACHE` environment variable using `export USE_CCACHE=1`. 4. Run `pod install` in the `ios` directory. 5. Check the stats of Ccache (running `ccache -s`). 6. Run `npm run ios` or build the project from Xcode. 7. Check the Ccache stats again to verify ccache is intercepting compilation ("Cacheable calls" should ideally be 100%). 8. To check the speed gain: a. Delete the `ios/builds` directory b. Zero out the ccache stats (by running `ccache -z`) c. Run `pod install` again (only needed if you ran the initial `pod install` with new architecture enabled `RCT_NEW_ARCH_ENABLED=1`). d. Run `npm run ios` or build the project from Xcode. e. This last step should be significantly faster and you should see "Hits" under "Local storage" in the ccache stats approach 100%. Reviewed By: huntie Differential Revision: D52431507 Pulled By: cipolleschi fbshipit-source-id: 6cfe39acd6250fae03959f0ee74d1f2fc46b0827 Original: facebook/react-native@e85d51c

Summary: Building native modules from source, may take a long time. Xcode already helps bring this down, by providing incremental builds, as long as the user doesn't delete their `ios/build` directory. But in some situations, i.e. when iterating the native code of an app or library or when the developer need to delete that `ios/build` directory, it's advantageous to use a compiler cache, such as ccache. This is already outlined in our ["Speeding up your Build phase"](https://reactnative.dev/docs/build-speed#xcode-specific-setup) guide. But setting up an Xcode project to use Ccache with the correct configuration, isn't trivial in a way that doesn't require symlinking `clang` and `clang++` or passing configuration via environment variables on every `npm run ios` invokation. This PR takes its inspiration from the existing guide on [setting up Ccache for Xcode](https://reactnative.dev/docs/build-speed#xcode-specific-setup), but applies the build settings only if an installation of `ccache` is detected and the feature is explicitly opted into via an argument to the `react_native_post_install` function or a `USE_CCACHE` environment variable. It uses two shell scripts to wrap the call to `ccache`, which both injects a default `CCACHE_CONFIGPATH` environment variable (i.e. it won't override this if already provided, to allow for customisations on CI), pointing to a `ccache.config` which works well with React Native projects (it has the same values as the guide mentions). For context, I posted about this change in the ios channel of the contributors Discord server, where I discussed it with cipolleschi and saadnajmi ### Additional output printed when running `pod install` #### When `ccache_available and ccache_enabled` ``` [Ccache]: Ccache found at /opt/homebrew/bin/ccache [Ccache]: Setting CC, LD, CXX & LDPLUSPLUS build settings ``` #### When `ccache_available and !ccache_enabled` ``` [Ccache]: Ccache found at /opt/homebrew/bin/ccache [Ccache]: Pass ':ccache_enabled => true' to 'react_native_post_install' in your Podfile or set environment variable 'USE_CCACHE=1' to increase the speed of subsequent builds ``` #### When `!ccache_available and ccache_enabled` ``` [!] [Ccache]: Install ccache or ensure your neither passing ':ccache_enabled => true' nor setting environment variable 'USE_CCACHE=1' ``` #### Otherwise If the user doesn't have ccache installed and doesn't explicitly opt into this feature, nothing will be printed. bypass-github-export-checks ## Changelog: [IOS] [ADDED] - Added better support for `ccache`, to speed up subsequent builds of native code. After installing `ccache` and running `pod install`, the Xcode project is injected with compiler and linker build settings pointing scripts that loads a default Ccache configuration and invokes the `ccache` executable. Pull Request resolved: facebook/react-native#42051 Test Plan: I've tested this manually - would love some inspiration on how to automate this, if the reviewer deem it needed. To test this locally: 1. Install Ccache and make sure the `ccache` executable is in your `PATH` (verify by running `ccache --version`) 2. Create a new template app instance and apply the changes of this PR to the `node_modules/react-native` package. 3. Set the `USE_CCACHE` environment variable using `export USE_CCACHE=1`. 4. Run `pod install` in the `ios` directory. 5. Check the stats of Ccache (running `ccache -s`). 6. Run `npm run ios` or build the project from Xcode. 7. Check the Ccache stats again to verify ccache is intercepting compilation ("Cacheable calls" should ideally be 100%). 8. To check the speed gain: a. Delete the `ios/builds` directory b. Zero out the ccache stats (by running `ccache -z`) c. Run `pod install` again (only needed if you ran the initial `pod install` with new architecture enabled `RCT_NEW_ARCH_ENABLED=1`). d. Run `npm run ios` or build the project from Xcode. e. This last step should be significantly faster and you should see "Hits" under "Local storage" in the ccache stats approach 100%. Reviewed By: huntie Differential Revision: D52431507 Pulled By: cipolleschi fbshipit-source-id: 6cfe39acd6250fae03959f0ee74d1f2fc46b0827 Original-Commit: facebook/react-native@e85d51c

facebook-github-bot added CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Shared with Meta Applied via automation to indicate that an Issue or Pull Request has been shared with the team. labels Dec 22, 2023

kraenhansen commented Dec 22, 2023

View reviewed changes

packages/react-native/scripts/cocoapods/utils.rb Show resolved Hide resolved

kraenhansen commented Dec 22, 2023

View reviewed changes

kraenhansen commented Dec 23, 2023

View reviewed changes

cipolleschi approved these changes Dec 27, 2023

View reviewed changes

kraenhansen added 7 commits December 27, 2023 19:47

Adding a default ccache.conf

3aae6f5

Adding ccache wrapper scripts, which sets a config path env variable

e870264

Adding post install to set compiler and linker paths if ccache is ava…

9f03a27

…ilable

Adding a link to ccache docs

e9c3024

Make feature opt-in via ccache_enabled

0ff36d3

Adding comment to the template's Podfile

9b7c407

Incorporated feedback

3cc7550

kraenhansen force-pushed the kh/default-ccache-config branch from e79ead9 to 3cc7550 Compare December 27, 2023 18:49

Simplified the formatting a bit

8243976

facebook-github-bot closed this in e85d51c Dec 28, 2023

facebook-github-bot added the Merged This PR has been merged. label Dec 28, 2023

kraenhansen deleted the kh/default-ccache-config branch March 22, 2024 10:05

kraenhansen mentioned this pull request Apr 23, 2024

Backport fix for Ccache from 0.74 to main #44089

Closed

kraenhansen mentioned this pull request Jun 10, 2024

Enables Ccache for our RNTA on iOS realm/realm-js#6712

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Detect `ccache` and provide a default configuration #42051

Detect `ccache` and provide a default configuration #42051

kraenhansen commented Dec 22, 2023 •

edited

Loading

Saadnajmi commented Dec 22, 2023

kraenhansen Dec 22, 2023

tido64 Dec 23, 2023

kraenhansen Dec 23, 2023

analysis-bot commented Dec 22, 2023 •

edited

Loading

kraenhansen Dec 23, 2023 •

edited

Loading

tido64 Dec 23, 2023

kraenhansen Dec 23, 2023

kraenhansen Dec 23, 2023 •

edited

Loading

cipolleschi left a comment

cipolleschi Dec 27, 2023

cipolleschi Dec 27, 2023

kraenhansen Dec 27, 2023 •

edited

Loading

facebook-github-bot commented Dec 27, 2023

facebook-github-bot commented Dec 27, 2023

github-actions bot commented Dec 28, 2023

facebook-github-bot commented Dec 28, 2023


		ccache_path = `command -v ccache`.strip

		if !ccache_path.empty?

Detect ccache and provide a default configuration #42051

Detect ccache and provide a default configuration #42051

Conversation

kraenhansen commented Dec 22, 2023 • edited Loading

Summary:

Additional output printed when running pod install

When ccache_available and ccache_enabled

When ccache_available and !ccache_enabled

When !ccache_available and ccache_enabled

Otherwise

Changelog:

Test Plan:

Saadnajmi commented Dec 22, 2023

kraenhansen Dec 22, 2023

Choose a reason for hiding this comment

tido64 Dec 23, 2023

Choose a reason for hiding this comment

kraenhansen Dec 23, 2023

Choose a reason for hiding this comment

analysis-bot commented Dec 22, 2023 • edited Loading

kraenhansen Dec 23, 2023 • edited Loading

Choose a reason for hiding this comment

tido64 Dec 23, 2023

Choose a reason for hiding this comment

kraenhansen Dec 23, 2023

Choose a reason for hiding this comment

kraenhansen Dec 23, 2023 • edited Loading

Choose a reason for hiding this comment

cipolleschi left a comment

Choose a reason for hiding this comment

cipolleschi Dec 27, 2023

Choose a reason for hiding this comment

cipolleschi Dec 27, 2023

Choose a reason for hiding this comment

kraenhansen Dec 27, 2023 • edited Loading

Choose a reason for hiding this comment

facebook-github-bot commented Dec 27, 2023

facebook-github-bot commented Dec 27, 2023

github-actions bot commented Dec 28, 2023

facebook-github-bot commented Dec 28, 2023

Detect `ccache` and provide a default configuration #42051

Detect `ccache` and provide a default configuration #42051

kraenhansen commented Dec 22, 2023 •

edited

Loading

Additional output printed when running `pod install`

When `ccache_available and ccache_enabled`

When `ccache_available and !ccache_enabled`

When `!ccache_available and ccache_enabled`

analysis-bot commented Dec 22, 2023 •

edited

Loading

kraenhansen Dec 23, 2023 •

edited

Loading

kraenhansen Dec 23, 2023 •

edited

Loading

kraenhansen Dec 27, 2023 •

edited

Loading