Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PoC: Add Qualcomm mobile SoC native backend for GGML #149

Merged
merged 44 commits into from
Apr 23, 2024
Merged

PoC: Add Qualcomm mobile SoC native backend for GGML #149

merged 44 commits into from
Apr 23, 2024

Commits on Apr 6, 2024

  1. doc: update software architecuture diagram

    @zhouwg
    zhouwg committed Apr 6, 2024
    Configuration menu
    Copy the full SHA
    a2cf3a6 View commit details
    Browse the repository at this point in the history

  2. project: fix issue in merge

    @zhouwg
    zhouwg committed Apr 6, 2024
    Configuration menu
    Copy the full SHA
    ad5bc94 View commit details
    Browse the repository at this point in the history

Commits on Apr 7, 2024

  1. project: refine project

    @zhouwg
    zhouwg committed Apr 7, 2024
    Configuration menu
    Copy the full SHA
    aa40397 View commit details
    Browse the repository at this point in the history

  2. doc: update software architecuture diagram

    @zhouwg
    zhouwg committed Apr 7, 2024
    Configuration menu
    Copy the full SHA
    f68e711 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into kantv-poc-with-qnn

    @zhouwg
    zhouwg authored Apr 7, 2024
    Configuration menu
    Copy the full SHA
    850f01d View commit details
    Browse the repository at this point in the history

  4. github-issue: fix #103 finally by santiy check with a new clone

    @zhouwg
    zhouwg committed Apr 7, 2024
    Configuration menu
    Copy the full SHA
    75799df View commit details
    Browse the repository at this point in the history

  5. build: fix minor issue in build system

    @zhouwg
    zhouwg committed Apr 7, 2024
    Configuration menu
    Copy the full SHA
    1cbb98f View commit details
    Browse the repository at this point in the history

  6. PoC-S26: offload simple f32 2x2 matrix addition operation to QNN CPU … 

    …backend --- done but not perfect
    @zhouwg
    zhouwg committed Apr 7, 2024
    Configuration menu
    Copy the full SHA
    4e08bd4 View commit details
    Browse the repository at this point in the history

Commits on Apr 8, 2024

  1. PoC-S27: offload simple f32 2x2 matrix addition operation to QNN GPU … 

    …backend --- done
    @zhouwg
    zhouwg committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    5df736b View commit details
    Browse the repository at this point in the history

  2. PoC-S29&S30&32&33: mapping ggml_tensor to QNN tensor -- done

    @zhouwg
    zhouwg committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    c0dd9e2 View commit details
    Browse the repository at this point in the history

Commits on Apr 9, 2024

  1. project:fix conflict in merge

    @zhouwg
    zhouwg committed Apr 9, 2024
    Configuration menu
    Copy the full SHA
    a6469c6 View commit details
    Browse the repository at this point in the history

  2. project: sync with upstream llama.cpp/ggml

    @zhouwg
    zhouwg committed Apr 9, 2024
    Configuration menu
    Copy the full SHA
    23c03c1 View commit details
    Browse the repository at this point in the history

  3. ggml-jni: better method to keep sync ggml_op between Java/JNI/ggml.h

    @zhouwg
    zhouwg committed Apr 9, 2024
    Configuration menu
    Copy the full SHA
    336f63f View commit details
    Browse the repository at this point in the history

  4. project: fix merge conflict

    @zhouwg
    zhouwg committed Apr 9, 2024
    Configuration menu
    Copy the full SHA
    ed92fa3 View commit details
    Browse the repository at this point in the history

  5. ggml-jni: build code skeleton of PoC-S35&S37:implement a complex/comp… 

    …licated compute graph in C/C++ and then mapping to GGML compute graph and then mapping to QNN CPU/GPU backend
    @zhouwg
    zhouwg committed Apr 9, 2024
    Configuration menu
    Copy the full SHA
    fcbe3c5 View commit details
    Browse the repository at this point in the history

  6. project:fix merge conflict

    @zhouwg
    zhouwg committed Apr 9, 2024
    Configuration menu
    Copy the full SHA
    9944269 View commit details
    Browse the repository at this point in the history

Commits on Apr 10, 2024

  1. ggml-jni: fix misunderstanding from "compute graph" to "computation g… 

    …raph"
    @zhouwg
    zhouwg committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    6900d7e View commit details
    Browse the repository at this point in the history

  2. project:fix merge conflict

    @zhouwg
    zhouwg committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    7f8d9ae View commit details
    Browse the repository at this point in the history

  3. project: enable APK running well on any mainstream Qualcomm mobile So… 

    …C based Android phone
    @zhouwg
    zhouwg committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    0fce196 View commit details
    Browse the repository at this point in the history

Commits on Apr 12, 2024

  1. PoC-S41: HLD of ggml-qnn backend done --- apk bootstrap(with qnn back… 

    …end) successfully at the first time
    @zhouwg
    zhouwg committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    911157e View commit details
    Browse the repository at this point in the history

Commits on Apr 13, 2024

  1. PoC-S42: implementation of major GGML OP(mulmat...) using QNN SDK ---… 

    … data path works but ASR result is not correct
    @zhouwg
    zhouwg committed Apr 13, 2024
    Configuration menu
    Copy the full SHA
    21dea39 View commit details
    Browse the repository at this point in the history

  2. PoC-S45: validate PoC-S42 by llama.cpp --- not work

    @zhouwg
    zhouwg committed Apr 13, 2024
    Configuration menu
    Copy the full SHA
    2b00eb1 View commit details
    Browse the repository at this point in the history

Commits on Apr 14, 2024

  1. PoC-S42:prepare for implementation of GGML OP using QNN SDK

    @zhouwg
    zhouwg committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    375a5b4 View commit details
    Browse the repository at this point in the history

Commits on Apr 15, 2024

  1. PoC-S43: implementation of major GGML OP(mulmat) using QNN CPU backend

    @zhouwg
    zhouwg committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    ce44da6 View commit details
    Browse the repository at this point in the history

  2. project: regular refine file and directory

    @zhouwg
    zhouwg committed Apr 15, 2024
    1 Configuration menu
    Copy the full SHA
    dcb0bcf View commit details
    Browse the repository at this point in the history

  3. project: fix typo

    @zhouwg
    zhouwg committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    390fddd View commit details
    Browse the repository at this point in the history

Commits on Apr 16, 2024

  1. PoC-S49: implementation of other GGML OP(non-mulmat) using QNN API

    @zhouwg
    zhouwg committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    7cb1960 View commit details
    Browse the repository at this point in the history

  2. PoC-S44:implementation of datapath using QNN GPU backend: UI <---> Ja… 

    …va <---> ggml-jni < ---> whisper.cpp < ---> ggml <---> ggml-qnn.cpp < ---> QNN(GPU backend) < ---> CPU done with a workaroud method
    @zhouwg
    zhouwg committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    3957c89 View commit details
    Browse the repository at this point in the history

Commits on Apr 17, 2024

  1. PoC-S48:validate PoC-S42/PoC-S44/PoC-S46(aka QNN backend) by whisper.… 

    …cpp asr benchmark
    @zhouwg
    zhouwg committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    62831a6 View commit details
    Browse the repository at this point in the history

Commits on Apr 18, 2024

  1. PoC-S48:validate PoC-S42/PoC-S44/PoC-S46(aka QNN backend) by whisper.… 

    …cpp asr benchmark
    @zhouwg
    zhouwg committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    083ad08 View commit details
    Browse the repository at this point in the history

  2. PoC-S48:fix a typo in UT for PoC-S49:implementation of other GGML OP(… 

    …non-mulmat) using QNN API
    @zhouwg
    zhouwg committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    87a1ef9 View commit details
    Browse the repository at this point in the history

  3. ggml-qnn: toggle between ggml and QNN CPU backend

    @zhouwg
    zhouwg committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    5991f6c View commit details
    Browse the repository at this point in the history

Commits on Apr 19, 2024

  1. ggml-qnn: qnn htp(aka dsp) backend works as expected on Xiaomi14

    @zhouwg
    zhouwg committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    bb31748 View commit details
    Browse the repository at this point in the history

  2. ggml-qnn: qnn rpc mem works in jni function qnn_matrix

    @zhouwg
    zhouwg committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    0ca0852 View commit details
    Browse the repository at this point in the history

  3. ggml-qnn: enable real QNN backend with GGML_OP_ADD/GGML_OP_MUL/GGML_O… 

    …P_MUL_MAT
    @zhouwg
    zhouwg committed Apr 19, 2024
    1 Configuration menu
    Copy the full SHA
    c78e6ea View commit details
    Browse the repository at this point in the history

Commits on Apr 20, 2024

  1. ggml-qnn: fix compatible issue on Qualcomm SoC based low-end Android … 

    …phone
    @zhouwg
    zhouwg committed Apr 20, 2024
    Configuration menu
    Copy the full SHA
    e8a1918 View commit details
    Browse the repository at this point in the history

  2. ggml-jni: add automation UT for ggml ops --- op_add,op_mul,op_mulmat

    @zhouwg
    zhouwg committed Apr 20, 2024
    Configuration menu
    Copy the full SHA
    2c9b8f6 View commit details
    Browse the repository at this point in the history

Commits on Apr 21, 2024

  1. ggml-qnn: add resource management of internal QNN resource and refine… 

    … ggml_qnn_op_xxx functions in ggml-qnn.cpp
    @zhouwg
    zhouwg committed Apr 21, 2024
    1 Configuration menu
    Copy the full SHA
    8a73098 View commit details
    Browse the repository at this point in the history

Commits on Apr 22, 2024

  1. PoC-S61: refine code(remove dependence......) --- prepare remove LOGG… 

    …D LOGGI LOGGW LOGGE
    @zhouwg
    zhouwg committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    83aa795 View commit details
    Browse the repository at this point in the history

  2. PoC-S52: multithread supportive using QNN CPU backend in ggml-qnn.cpp

    @zhouwg
    zhouwg committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    0e0033d View commit details
    Browse the repository at this point in the history

  3. PoC-S53: fix stability issue during toggle between different backend(… 

    …QNN CPU/GPU/DSP backend, ggml...)
    @zhouwg
    zhouwg committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    74a57c3 View commit details
    Browse the repository at this point in the history

  4. ggml-qnn: refine internal doc in ggml-qnn.cpp

    @zhouwg
    zhouwg committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    b59482c View commit details
    Browse the repository at this point in the history

  5. PoC-S54: validate with llama.cpp using QNN backend

    @zhouwg
    zhouwg committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    f290eb9 View commit details
    Browse the repository at this point in the history

Commits on Apr 23, 2024

  1. project: fix merge conflict

    @zhouwg
    zhouwg committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    f759d4b View commit details
    Browse the repository at this point in the history