Feat/Add a React Native LLaMA demo app for iOS #7134
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/7134
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 New FailureAs of commit f7aec5d with merge base a347665 ( NEW FAILURE - The following job has failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
facebook-github-bot
added
the
CLA Signed
|
@shoumikhin let me know if this is something that would be useful / any feedback on it. I know the Android side would be an useful thing to add but did not have time to look into it yet. |
|
Wow, this looks awesome @hietalajulius! |
|
@shoumikhin has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@pytorchbot label "topic: react native" |
|
Didn't find following labels among repository labels: topic: react native |
|
@hietalajulius do you think it's possible to add some info on t/s performance and model load time similar to the iOS app, maybe in a follow up PR? |
|
@pytorchbot label "release notes: react native llama demo app for ios" |
|
Didn't find following labels among repository labels: release notes: react native llama demo app for ios |
|
@pytorchbot label "topic: not user facing" |
@shoumikhin absolutely, let me add those + fix the linting 👍 |
@shoumikhin I added the model load time + tokens/sec info now (screen recording in PR description updated to showcase). The newlines that the linting was complaining about are also removed now, there does not seem to be a way to trigger the job on my end to make sure it passes but the errors were quite trivial. |
|
@shoumikhin has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@hietalajulius thanks for the update! |
@shoumikhin thanks! I should have taken a closer look at the contributing guide earlier, the |
|
@shoumikhin has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
Hmm looks like still some permission issue with unittest-arm / linux-job, not entirely sure why |
Differential Revision: D66644092 Pull Request resolved: #7134
|
@hietalajulius many thanks again for the great contribution! Out internal vulnerability checker has flagged a potential threat from one of the dependencies, can please take a look?
Thanks! |
@shoumikhin thanks for the note! Let me check it out, looks like a transitive dependency of |
@shoumikhin I posted a PR here: #7218, let me know if that works 👍 |
Summary
rnllama.mov
Add React Native demo application for running LLaMA models
This PR introduces a new React Native mobile application demo that showcases running LLaMA language models on iOS devices using ExecuTorch. Key features include:
The demo app serves as a reference implementation for developers looking to integrate ExecuTorch and LLaMA models into React Native mobile applications. It complements the existing native iOS and Android demo apps by demonstrating cross-platform capabilities.
Release notes: examples
Test plan
Tested model initialization and chat functionality on: