Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move file prep from build.sh to cmake #3014

Merged
merged 3 commits into from
Mar 12, 2024
Merged

Move file prep from build.sh to cmake #3014

merged 3 commits into from
Mar 12, 2024

Conversation

mreso
Copy link
Collaborator

@mreso mreso commented Mar 12, 2024

Description

This PR moves file preparation for cpp backend into cmake which will enable removal of build.sh.

Type of change

Please delete options that are not relevant.

  • New feature (non-breaking change which adds functionality)

Feature/Issue validation/testing

Please describe the Unit or Integration tests that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced.
Please also list any relevant details for your test configuration.

  • cd cpp && rm -rf _build && ./build.sh
    Logs for Test A
Running main() from /home/ubuntu/serve/cpp/_build/_deps/googletest-src/googletest/src/gtest_main.cc
[==========] Running 49 tests from 12 test suites.
[----------] Global test environment set-up.
[----------] 1 test from BackendIntegTest
[ RUN      ] BackendIntegTest.TestOTFProtocolAndHandler
[       OK ] BackendIntegTest.TestOTFProtocolAndHandler (117 ms)
[----------] 1 test from BackendIntegTest (117 ms total)

[----------] 8 tests from OTFMessageTest
[ RUN      ] OTFMessageTest.TestRetieveCmd
[       OK ] OTFMessageTest.TestRetieveCmd (0 ms)
[ RUN      ] OTFMessageTest.TestEncodeLoadModelResponse
[       OK ] OTFMessageTest.TestEncodeLoadModelResponse (0 ms)
[ RUN      ] OTFMessageTest.TestUTF8EncodeLoadModelResponse
[       OK ] OTFMessageTest.TestUTF8EncodeLoadModelResponse (0 ms)
[ RUN      ] OTFMessageTest.TestRetrieveMsgLoadGpu
[       OK ] OTFMessageTest.TestRetrieveMsgLoadGpu (0 ms)
[ RUN      ] OTFMessageTest.TestRetrieveMsgLoadNoGpu
[       OK ] OTFMessageTest.TestRetrieveMsgLoadNoGpu (0 ms)
[ RUN      ] OTFMessageTest.TestEncodeSuccessInferenceResponse
[       OK ] OTFMessageTest.TestEncodeSuccessInferenceResponse (0 ms)
[ RUN      ] OTFMessageTest.TestEncodeFailureInferenceResponse
[       OK ] OTFMessageTest.TestEncodeFailureInferenceResponse (0 ms)
[ RUN      ] OTFMessageTest.TestRetrieveInferenceMsg
[       OK ] OTFMessageTest.TestRetrieveInferenceMsg (0 ms)
[----------] 8 tests from OTFMessageTest (0 ms total)

[----------] 11 tests from ModelPredictTest
[ RUN      ] ModelPredictTest.TestLoadPredictBabyLlamaHandler
[----------] 11 tests from ModelPredictTest                                                                                                                                                                                                                                          [76/71365]
[ RUN      ] ModelPredictTest.TestLoadPredictBabyLlamaHandler
Total number of tokens generated: 332
Achieved tok per sec: 249.812
Generated String:  Hello my name is
The little girl, who was three years old, was playing in the garden. She saw a big, red tomato and wanted to pick it. She reached out her hand and grabbed it.
Suddenly, a voice said, "Hey! That's my tomato!"
The little girl looked up and saw a big, angry bird. She was scared and started to cry.
The bird said, "That tomato is mine! I'm going to eat it!"
The little girl was very scared and ran away. She was so sad that she didn't get to eat the tomato.
The bird flew away and the little girl was left alone in the garden. She was very sad and cried all the way home.
<s>

Generated String:  Hello my name is
The little girl, who was three years old, was playing in the garden. She saw a big, red tomato and wanted to pick it. She reached out her hand and grabbed it.
Suddenly, a voice said, "Hey! That's my tomato!"
The little girl looked up and saw a big, angry bird. She was scared and started to cry.
The bird said, "That tomato is mine! I'm going to eat it!"
The little girl was very scared and ran away. She was so sad that she didn't get to eat the tomato.
The bird flew away and the little girl was left alone in the garden. She was very sad and cried all the way home.
<s>

[       OK ] ModelPredictTest.TestLoadPredictBabyLlamaHandler (1349 ms)
[ RUN      ] ModelPredictTest.TestLoadPredictAotInductorLlamaHandler
[       OK ] ModelPredictTest.TestLoadPredictAotInductorLlamaHandler (2653 ms)
[ RUN      ] ModelPredictTest.TestLoadPredictLlamaCppHandler
/home/ubuntu/serve/cpp/test/examples/examples_test.cc:53: Skipped
Skipping TestLoadPredictLlamaCppHandler because of missing file: _build/test/resources/examples/llamacpp/llamacpp_handler/llama-2-7b-chat.Q5_0.gguf
[  SKIPPED ] ModelPredictTest.TestLoadPredictLlamaCppHandler (0 ms)
[ RUN      ] ModelPredictTest.TestLoadPredictAotInductorBertHandler
[       OK ] ModelPredictTest.TestLoadPredictAotInductorBertHandler (268 ms)
[ RUN      ] ModelPredictTest.TestLoadPredictAotInductorResnetHandler
[       OK ] ModelPredictTest.TestLoadPredictAotInductorResnetHandler (60 ms)
[ RUN      ] ModelPredictTest.TestLoadPredictBaseHandler
[       OK ] ModelPredictTest.TestLoadPredictBaseHandler (13 ms)
[ RUN      ] ModelPredictTest.TestLoadPredictMnistHandler
[       OK ] ModelPredictTest.TestLoadPredictMnistHandler (13 ms)
[ RUN      ] ModelPredictTest.TestBackendInitWrongModelDir
[       OK ] ModelPredictTest.TestBackendInitWrongModelDir (0 ms)
[ RUN      ] ModelPredictTest.TestBackendInitWrongHandler
[       OK ] ModelPredictTest.TestBackendInitWrongHandler (0 ms)
[ RUN      ] ModelPredictTest.TestLoadModelFailure
[       OK ] ModelPredictTest.TestLoadModelFailure (3 ms)
[ RUN      ] ModelPredictTest.TestLoadPredictMnistHandlerFailure
[       OK ] ModelPredictTest.TestLoadPredictMnistHandlerFailure (14 ms)
[----------] 11 tests from ModelPredictTest (4379 ms total)

[----------] 1 test from DLLoaderTest
[ RUN      ] DLLoaderTest.TestGetInstance
[       OK ] DLLoaderTest.TestGetInstance (0 ms)
[----------] 1 test from DLLoaderTest (0 ms total)

[----------] 1 test from JsonTest
[ RUN      ] JsonTest.TestParsingAndGetValue
[       OK ] JsonTest.TestParsingAndGetValue (0 ms)
[----------] 1 test from JsonTest (0 ms total)

[----------] 2 tests from LoggingTest
[ RUN      ] LoggingTest.TestIncorrectLogInitialization
[       OK ] LoggingTest.TestIncorrectLogInitialization (0 ms)
[ RUN      ] LoggingTest.TestFileLogInitialization
[info] 0312 01:23:28.650 /home/ubuntu/serve/cpp/test/utils/logging_test.cc:38] TestBody Test
[       OK ] LoggingTest.TestFileLogInitialization (0 ms)
[----------] 2 tests from LoggingTest (0 ms total)

[----------] 6 tests from TSLogMetricTest
[ RUN      ] TSLogMetricTest.TestCounterMetric
[info] 0312 01:23:28.651 /home/ubuntu/serve/cpp/src/utils/metrics/log_metric.cc:89] Emit [METRICS]test_metric.Milliseconds:1|#level:model,model_name:test_model|#hostname:ip-172-31-55-226,1710206608
[info] 0312 01:23:28.651 /home/ubuntu/serve/cpp/src/utils/metrics/log_metric.cc:92] Emit [METRICS]test_metric.Milliseconds:3.5|#level:model,model_name:test_model|#hostname:ip-172-31-55-226,1710206608,test_request_id
[       OK ] TSLogMetricTest.TestCounterMetric (0 ms)
[ RUN      ] TSLogMetricTest.TestGaugeMetric
[info] 0312 01:23:28.651 /home/ubuntu/serve/cpp/src/utils/metrics/log_metric.cc:89] Emit [METRICS]test_metric.Count:1|#level:model,model_name:test_model|#hostname:ip-172-31-55-226,1710206608
[info] 0312 01:23:28.651 /home/ubuntu/serve/cpp/src/utils/metrics/log_metric.cc:89] Emit [METRICS]test_metric.Count:-2|#level:model,model_name:test_model|#hostname:ip-172-31-55-226,1710206608
[info] 0312 01:23:28.651 /home/ubuntu/serve/cpp/src/utils/metrics/log_metric.cc:92] Emit [METRICS]test_metric.Count:3.5|#level:model,model_name:test_model|#hostname:ip-172-31-55-226,1710206608,test_request_id
[info] 0312 01:23:28.651 /home/ubuntu/serve/cpp/src/utils/metrics/log_metric.cc:92] Emit [METRICS]test_metric.Count:-4|#level:model,model_name:test_model|#hostname:ip-172-31-55-226,1710206608,test_request_id
[       OK ] TSLogMetricTest.TestGaugeMetric (0 ms)
[ RUN      ] TSLogMetricTest.TestHistogramMetric
[info] 0312 01:23:28.652 /home/ubuntu/serve/cpp/src/utils/metrics/log_metric.cc:89] Emit [METRICS]test_metric.Count:1|#level:model,model_name:test_model|#hostname:ip-172-31-55-226,1710206608
[info] 0312 01:23:28.652 /home/ubuntu/serve/cpp/src/utils/metrics/log_metric.cc:89] Emit [METRICS]test_metric.Count:-2|#level:model,model_name:test_model|#hostname:ip-172-31-55-226,1710206608
[info] 0312 01:23:28.652 /home/ubuntu/serve/cpp/src/utils/metrics/log_metric.cc:92] Emit [METRICS]test_metric.Count:3.5|#level:model,model_name:test_model|#hostname:ip-172-31-55-226,1710206608,test_request_id
[info] 0312 01:23:28.652 /home/ubuntu/serve/cpp/src/utils/metrics/log_metric.cc:92] Emit [METRICS]test_metric.Count:-4|#level:model,model_name:test_model|#hostname:ip-172-31-55-226,1710206608,test_request_id
[       OK ] TSLogMetricTest.TestHistogramMetric (0 ms)
[ RUN      ] TSLogMetricTest.TestTSLogMetricEmitWithRequestId
[info] 0312 01:23:28.652 /home/ubuntu/serve/cpp/src/utils/metrics/log_metric.cc:92] Emit [METRICS]test_metric.Milliseconds:1.5|#level:model,model_name:test_model|#hostname:ip-172-31-55-226,1710206608,test_request_id
[       OK ] TSLogMetricTest.TestTSLogMetricEmitWithRequestId (0 ms)
[ RUN      ] TSLogMetricTest.TestTSLogMetricEmitWithoutRequestId
[info] 0312 01:23:28.653 /home/ubuntu/serve/cpp/src/utils/metrics/log_metric.cc:89] Emit [METRICS]test_metric.Milliseconds:1.5|#level:model,model_name:test_model|#hostname:ip-172-31-55-226,1710206608
[       OK ] TSLogMetricTest.TestTSLogMetricEmitWithoutRequestId (0 ms)
[ RUN      ] TSLogMetricTest.TestTSLogMetricEmitWithIncorrectDimensionData
[       OK ] TSLogMetricTest.TestTSLogMetricEmitWithIncorrectDimensionData (0 ms)
[----------] 6 tests from TSLogMetricTest (2 ms total)

[----------] 2 tests from TSLogMetricsCacheTest
[ RUN      ] TSLogMetricsCacheTest.TestInitialize
[       OK ] TSLogMetricsCacheTest.TestInitialize (0 ms)
[ RUN      ] TSLogMetricsCacheTest.TestGetMetric
[info] 0312 01:23:28.654 /home/ubuntu/serve/cpp/src/utils/metrics/log_metric.cc:89] Emit [METRICS]GaugeTsMetricExample.Count:1.5|#model_name:model_name,host_name:host_name|#hostname:ip-172-31-55-226,1710206608
[       OK ] TSLogMetricsCacheTest.TestGetMetric (0 ms)
[----------] 2 tests from TSLogMetricsCacheTest (0 ms total)

[----------] 3 tests from RegistryTest
[ RUN      ] RegistryTest.TestValidConfigFile
[       OK ] RegistryTest.TestValidConfigFile (0 ms)
[ RUN      ] RegistryTest.TestInvalidConfigFile
[       OK ] RegistryTest.TestInvalidConfigFile (0 ms)
[ RUN      ] RegistryTest.TestReInitialize
[       OK ] RegistryTest.TestReInitialize (0 ms)
[----------] 3 tests from RegistryTest (0 ms total)

[----------] 3 tests from UnitsTest
[ RUN      ] UnitsTest.TestGetExistingUnitMapping
[       OK ] UnitsTest.TestGetExistingUnitMapping (0 ms)
[ RUN      ] UnitsTest.TestGetNonExistentUnitMapping
[       OK ] UnitsTest.TestGetNonExistentUnitMapping (0 ms)
[ RUN      ] UnitsTest.TestGetEmptyUnitMapping
[       OK ] UnitsTest.TestGetEmptyUnitMapping (0 ms)
[----------] 3 tests from UnitsTest (0 ms total)

[----------] 10 tests from YAMLConfigTest
[ RUN      ] YAMLConfigTest.TestLoadValidConfigFrontendContext
[       OK ] YAMLConfigTest.TestLoadValidConfigFrontendContext (0 ms)
[ RUN      ] YAMLConfigTest.TestLoadValidConfigBackendContext
[       OK ] YAMLConfigTest.TestLoadValidConfigBackendContext (0 ms)
[ RUN      ] YAMLConfigTest.TestLoadMinimalValidConfig
[       OK ] YAMLConfigTest.TestLoadMinimalValidConfig (0 ms)
[ RUN      ] YAMLConfigTest.TestLoadInvalidConfigWithDuplicateDimension
[       OK ] YAMLConfigTest.TestLoadInvalidConfigWithDuplicateDimension (0 ms)
[ RUN      ] YAMLConfigTest.TestLoadInvalidConfigWithEmptyDimension
[       OK ] YAMLConfigTest.TestLoadInvalidConfigWithEmptyDimension (0 ms)
[ RUN      ] YAMLConfigTest.TestLoadInvalidConfigWithUndefinedDimension
[       OK ] YAMLConfigTest.TestLoadInvalidConfigWithUndefinedDimension (0 ms)
[ RUN      ] YAMLConfigTest.TestLoadInvalidConfigWithDuplicateMetricDimension
[       OK ] YAMLConfigTest.TestLoadInvalidConfigWithDuplicateMetricDimension (0 ms)
[ RUN      ] YAMLConfigTest.TestLoadInvalidConfigWithMissingMetricName
[error] 0312 01:23:28.655 /home/ubuntu/serve/cpp/src/utils/metrics/yaml_config.cc:203] decode Configuration for a metric must consist of "name", "unit" and "dimensions"
[       OK ] YAMLConfigTest.TestLoadInvalidConfigWithMissingMetricName (0 ms)
[ RUN      ] YAMLConfigTest.TestLoadInvalidConfigWithEmptyMetricName
[error] 0312 01:23:28.655 /home/ubuntu/serve/cpp/src/utils/metrics/yaml_config.cc:215] decode Configuration for a metric must consist of a non-empty "name"
[       OK ] YAMLConfigTest.TestLoadInvalidConfigWithEmptyMetricName (0 ms)
[ RUN      ] YAMLConfigTest.TestLoadInvalidConfigWithDuplicateMetricName
[       OK ] YAMLConfigTest.TestLoadInvalidConfigWithDuplicateMetricName (0 ms)
[----------] 10 tests from YAMLConfigTest (0 ms total)

[----------] 1 test from ManifestTest
[ RUN      ] ManifestTest.TestInitialize
[       OK ] ManifestTest.TestInitialize (0 ms)
[----------] 1 test from ManifestTest (0 ms total)

[----------] Global test environment tear-down
[==========] 49 tests from 12 test suites ran. (4503 ms total)
[  PASSED  ] 48 tests.
[  SKIPPED ] 1 test, listed below:
[  SKIPPED ] ModelPredictTest.TestLoadPredictLlamaCppHandler
  • pytest test/pytest/test_cpp_backend.py
    Logs for Test B
===================================================================================================================================== test session starts =====================================================================================================================================
platform linux -- Python 3.11.5, pytest-7.3.1, pluggy-1.0.0
rootdir: /home/ubuntu/serve
plugins: mock-3.12.0, cov-4.1.0
collected 1 item

test/pytest/test_cpp_backend.py .                                                                                                                                                                                                                                                       [100%]

====================================================================================================================================== 1 passed in 6.56s ======================================================================================================================================

Checklist:

  • Did you have fun?
  • Have you added tests that prove your fix is effective or that this feature works?
  • Has code been commented, particularly in hard-to-understand areas?
  • Have you made corresponding changes to the documentation?

@mreso mreso requested review from lxning and chauhang March 12, 2024 01:26
@lxning lxning added this pull request to the merge queue Mar 12, 2024
Merged via the queue into master with commit a73a08a Mar 12, 2024
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lxning lxning lxning approved these changes

@chauhang chauhang Awaiting requested review from chauhang

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@mreso @lxning