Process text and binary payloads #1642

Teagan42 · 2020-03-31T01:10:52Z

Convert string and binary payloads to SeldonMessage payloads

seldondev · 2020-03-31T01:11:08Z

Hi @Teagan42. Thanks for your PR.

I'm waiting for a SeldonIO member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

ukclivecox · 2020-03-31T06:46:28Z

/ok-to-test

ukclivecox · 2020-03-31T06:46:56Z

Thanks @Teagan42

seldondev · 2020-03-31T06:47:50Z

Tue Mar 31 06:47:49 UTC 2020
The logs for [lint] [2] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1642/2.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1642 --build=2

seldondev · 2020-03-31T06:47:57Z

Tue Mar 31 06:47:56 UTC 2020
The logs for [pr-build] [1] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1642/1.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1642 --build=1

Teagan42 · 2020-03-31T16:48:13Z

/assign @ryandawsonuk

ukclivecox · 2020-03-31T16:50:32Z

/test this

Teagan42 · 2020-03-31T16:51:47Z

/retest

seldondev · 2020-03-31T16:51:57Z

Tue Mar 31 16:51:56 UTC 2020
The logs for [pr-build] [3] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1642/3.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1642 --build=3

Teagan42 · 2020-03-31T17:45:28Z

/check-cla

ukclivecox · 2020-03-31T18:43:29Z

/test this

ukclivecox · 2020-03-31T18:44:22Z

Looks like the build is failing - have you tested locally with make build_jar?

seldondev · 2020-03-31T18:44:51Z

Tue Mar 31 18:44:50 UTC 2020
The logs for [pr-build] [4] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1642/4.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1642 --build=4

Teagan42 · 2020-03-31T18:52:34Z

@cliveseldon yes I've used the makefiles to build the JAR and custom image and it is running in my k8s. The link to the logs don't seem to give a lot of info

Teagan42 · 2020-03-31T18:55:35Z

@cliveseldon weird, I ran this yesterday fine and now it's failing. I'll look into it.

ukclivecox · 2020-03-31T19:40:00Z

@Teagan42 https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1642/4.log
shows

[INFO] Changes detected - recompiling the module!
[INFO] Compiling 66 source files to /workspace/source/engine/target/classes
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  31.892 s
[INFO] Finished at: 2020-03-31T18:47:16Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project seldon-engine: Fatal error compiling: CompilerException: NullPointerException -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
make: *** [Makefile:8: build_jar] Error 1
make: Leaving directory '/workspace/source/engine'

Which is indeed not clear! Any ideas @axsaucedo

seldondev · 2020-03-31T19:57:43Z

Tue Mar 31 19:57:42 UTC 2020
The logs for [lint] [6] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1642/6.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1642 --build=6

seldondev · 2020-03-31T19:57:44Z

Tue Mar 31 19:57:43 UTC 2020
The logs for [pr-build] [5] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1642/5.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1642 --build=5

Teagan42 · 2020-03-31T19:57:50Z

@cliveseldon I'm stupid today. Needed these changes for my job, but we only have enterprise github, so had to mirror it up to our repository and make changes. When I verified it all worked, I forked it from my personal public org and made the changes, but 1. I forgot to update the swagger as I had in the enterprise repo and 2. I missed two imports. Apologies for the brain fart

Teagan42 · 2020-03-31T20:08:39Z

@cliveseldon I diagnosed this by adding the -X -e flags to the maven invocation, it gave me a little better idea (the root exception was a MojoExecutionException) and those are thrown only for a few reasons, one being missing imports :-)

ukclivecox · 2020-04-01T07:27:04Z

@Teagan42 Looks great. Is it possible to add a couple of junit tests?
See for a previous PR for form data:

seldon-core/engine/src/test/java/io/seldon/engine/api/rest/TestRestClientController.java

Lines 311 to 337 in be662dc

    
             public void testPredict_multiform_strData_as_text() throws Exception { 
        
               final String metaJson = "{\"puid\":\"1234\"}"; 
        
               final MultiValueMap<String, String> paramMap = new LinkedMultiValueMap<>(); 
        
               paramMap.put("meta", Arrays.asList(metaJson)); 
        
               String strdata = "test data"; 
        
               paramMap.put("strData", Arrays.asList(strdata)); 
        
               MvcResult res = 
        
                   mvc.perform( 
        
                           MockMvcRequestBuilders.post("/api/v1.0/predictions") 
        
                               .accept(MediaType.APPLICATION_JSON_UTF8) 
        
                               .params(paramMap) 
        
                               .contentType(MediaType.MULTIPART_FORM_DATA)) 
        
                       .andReturn(); 
        
               String response = res.getResponse().getContentAsString(); 
        
               System.out.println(response); 
        
               Assert.assertEquals(200, res.getResponse().getStatus()); 
        
               SeldonMessage.Builder builder = SeldonMessage.newBuilder(); 
        
               ProtoBufUtils.updateMessageBuilderFromJson(builder, response); 
        
               SeldonMessage seldonMessage = builder.build(); 
        
               Assert.assertEquals(3, seldonMessage.getMeta().getMetricsCount()); 
        
               Assert.assertEquals("COUNTER", seldonMessage.getMeta().getMetrics(0).getType().toString()); 
        
               Assert.assertEquals("GAUGE", seldonMessage.getMeta().getMetrics(1).getType().toString()); 
        
               Assert.assertEquals("TIMER", seldonMessage.getMeta().getMetrics(2).getType().toString()); 
        
               Assert.assertEquals(strdata, seldonMessage.getStrData()); 
        
               Assert.assertEquals("1234", seldonMessage.getMeta().getPuid()); 
        
             } 
        
           }

Teagan42 · 2020-04-01T08:19:44Z

@cliveseldon Absolutely! I will do that in the morning.

I guess I misunderstood the PR instructions as optional (This is a simplified description of our full PR testing and merge workflow that conveniently forgets about the existence of tests, to focus solely on the roles driven by OWNERS files. Please see below for details on how specific aspects of this process may be configured on a per-repo basis.).

seldondev · 2020-04-02T18:54:27Z

Thu Apr 2 18:54:26 UTC 2020
The logs for [lint] [8] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1642/8.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1642 --build=8

seldondev · 2020-04-02T18:54:37Z

Thu Apr 2 18:54:36 UTC 2020
The logs for [pr-build] [7] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1642/7.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1642 --build=7

Teagan42 · 2020-04-02T18:55:32Z

@cliveseldon Usually unit tests are a must for me, was in a hurry I guess. But I added tests and everything passes. I may make a PR to update the PR documentation to explicitly state code should be unit tested as the instructions were a little misleading (but that's no excuse for not doing it anyway).

ukclivecox · 2020-04-02T19:29:28Z

@Teagan42 Thanks for this!

ukclivecox · 2020-04-02T19:29:34Z

/approve

seldondev · 2020-04-02T19:29:42Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cliveseldon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [cliveseldon]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

seldondev · 2021-05-13T06:33:54Z

failed to trigger Pull Request pipeline

failed to create agent
failed to calculate in repo config
failed to load trigger config for repository SeldonIO/seldon-core for ref b9e44eb
failed to find any lighthouse configuration files in repo SeldonIO/seldon-core at sha b9e44eb
failed to process repo SeldonIO/seldon-core refref b9e44eb
failed to list files in directory /var/tmp/gitrepo660600678/.lighthouse
open /var/tmp/gitrepo660600678/.lighthouse
no such file or directory

Process text and binary payloads

c48190d

seldondev added the needs-ok-to-test label Mar 31, 2020

seldondev added the size/M label Mar 31, 2020

seldondev added ok-to-test and removed needs-ok-to-test labels Mar 31, 2020

seldondev assigned ryandawsonuk Mar 31, 2020

Update Swagger

56fdab1

Missing imports

27e532d

Teagan42 added 7 commits April 1, 2020 15:48

Add unit tests

836f516

Import clean up

18a5911

Use ByteString

98ad70e

Send bytestring

ed8f6d7

Octet should accept byte array payload

c6a66e3

Fix print call

d6e5060

Array equals

b9e44eb

seldondev added size/L and removed size/M labels Apr 2, 2020

seldondev added the approved label Apr 2, 2020

seldondev merged commit 8ae203c into SeldonIO:master Apr 2, 2020

axsaucedo unassigned ryandawsonuk May 13, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Process text and binary payloads #1642

Process text and binary payloads #1642

Teagan42 commented Mar 31, 2020

seldondev commented Mar 31, 2020

ukclivecox commented Mar 31, 2020

ukclivecox commented Mar 31, 2020

seldondev commented Mar 31, 2020

seldondev commented Mar 31, 2020

Teagan42 commented Mar 31, 2020

ukclivecox commented Mar 31, 2020

Teagan42 commented Mar 31, 2020

seldondev commented Mar 31, 2020

Teagan42 commented Mar 31, 2020

ukclivecox commented Mar 31, 2020

ukclivecox commented Mar 31, 2020

seldondev commented Mar 31, 2020

Teagan42 commented Mar 31, 2020

Teagan42 commented Mar 31, 2020

ukclivecox commented Mar 31, 2020

seldondev commented Mar 31, 2020

seldondev commented Mar 31, 2020

Teagan42 commented Mar 31, 2020

Teagan42 commented Mar 31, 2020

ukclivecox commented Apr 1, 2020

Teagan42 commented Apr 1, 2020

seldondev commented Apr 2, 2020

seldondev commented Apr 2, 2020

Teagan42 commented Apr 2, 2020

ukclivecox commented Apr 2, 2020

ukclivecox commented Apr 2, 2020

seldondev commented Apr 2, 2020

seldondev commented May 13, 2021

Process text and binary payloads #1642

Process text and binary payloads #1642

Conversation

Teagan42 commented Mar 31, 2020

seldondev commented Mar 31, 2020

ukclivecox commented Mar 31, 2020

ukclivecox commented Mar 31, 2020

seldondev commented Mar 31, 2020

seldondev commented Mar 31, 2020

Teagan42 commented Mar 31, 2020

ukclivecox commented Mar 31, 2020

Teagan42 commented Mar 31, 2020

seldondev commented Mar 31, 2020

Teagan42 commented Mar 31, 2020

ukclivecox commented Mar 31, 2020

ukclivecox commented Mar 31, 2020

seldondev commented Mar 31, 2020

Teagan42 commented Mar 31, 2020

Teagan42 commented Mar 31, 2020

ukclivecox commented Mar 31, 2020

seldondev commented Mar 31, 2020

seldondev commented Mar 31, 2020

Teagan42 commented Mar 31, 2020

Teagan42 commented Mar 31, 2020

ukclivecox commented Apr 1, 2020

Teagan42 commented Apr 1, 2020

seldondev commented Apr 2, 2020

seldondev commented Apr 2, 2020

Teagan42 commented Apr 2, 2020

ukclivecox commented Apr 2, 2020

ukclivecox commented Apr 2, 2020

seldondev commented Apr 2, 2020

seldondev commented May 13, 2021