Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace Ingest ScriptContext with Custom Interface #32003

Merged
merged 9 commits into from
Jul 13, 2018

Conversation

original-brownbear
Copy link
Member

@original-brownbear original-brownbear commented Jul 12, 2018

@original-brownbear original-brownbear added the :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP label Jul 12, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra

@@ -58,7 +59,7 @@ public void testScripting() throws Exception {
doAnswer(invocationOnMock -> {
ingestDocument.setFieldValue("bytes_total", randomBytesTotal);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one thing we do not check here is that the Map passed in as the ctx is actually

IngestDocument#getSourceAndMetadata()

Do you think that would be useful here?

We do have checks in rest-api-spec/test/ingest/50_script_processor_using_painless.yml that we are actually modifying the document, so that may be sufficient. Seems like it would be nice to validate this from unit tests too

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@talevy makes sense + is trivial to do => sec adjusting :)

Copy link
Member Author

@original-brownbear original-brownbear Jul 12, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@talevy done in 89874c9 :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cooool. thanks!

@original-brownbear
Copy link
Member Author

@talevy seems ITs are broken, gotta give this another look.

@original-brownbear
Copy link
Member Author

@talevy should be all good now, just had to adjust the execute API in https://github.com/elastic/elasticsearch/pull/32003/files#diff-859b948216f97425b337faff8da3c0b8R61 to fix the IT.
Also removed the mocking from the processor test in fb9647d :)

Copy link
Member

@martijnvg martijnvg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

I think @rjernst should review this too before mergin.


import java.util.Map;

public abstract class IngestScript {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe add class level java docs here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good, added in 762d2f7 :)

Copy link
Member

@rjernst rjernst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

return params;
}

public abstract void execute(Map<String, Object> ctx);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While this will be needed for backcompat in 6.x, I think we want to expose IngestDocument as a variable directly here. But this can be done as a followup (along with deprecation control of the old ctx variable).

@original-brownbear
Copy link
Member Author

@rjernst thanks for the review!

@original-brownbear original-brownbear merged commit 3679d00 into elastic:master Jul 13, 2018
@original-brownbear original-brownbear deleted the new-ingest-ctx branch July 13, 2018 21:26
dnhatn added a commit that referenced this pull request Jul 13, 2018
* master:
  Replace Ingest ScriptContext with Custom Interface (#32003)
  Mute failing tests
original-brownbear added a commit to original-brownbear/elasticsearch that referenced this pull request Jul 14, 2018
* Replace Ingest ScriptContext with Custom Interface
* Make org.elasticsearch.ingest.common.ScriptProcessorTests#testScripting more precise
* Don't mock script factory in ScriptProcessorTests
* Adjust mock script plugin in IT for new API
original-brownbear added a commit that referenced this pull request Jul 15, 2018
* Replace Ingest ScriptContext with Custom Interface
* Make org.elasticsearch.ingest.common.ScriptProcessorTests#testScripting more precise
* Don't mock script factory in ScriptProcessorTests
* Adjust mock script plugin in IT for new API
martijnvg added a commit that referenced this pull request Jul 16, 2018
* es/6.x:
  Use correct formatting for links (#29460)
  Revert "Adds a new auto-interval date histogram (#28993)"
  Revert "fix typo"
  fix typo
  Adds a new auto-interval date histogram (#28993)
  [Rollup] Replace RollupIT with a ESRestTestCase version (#31977)
  [Rollup] Fix duplicate field names in test (#32075)
  [Tests] Fix failure due to changes exception message (#32036)
  [Test] Mute MlJobIT#testDeleteJobAfterMissingAliases
  Replace Ingest ScriptContext with Custom Interface (#32003) (#32060)
  Cleanup Duplication in `PainlessScriptEngine` (#31991) (#32061)
  HLRC: Add xpack usage api (#31975)
  Clean Up Snapshot Create Rest API (#31779)
@jimczi jimczi added v7.0.0-beta1 and removed v7.0.0 labels Feb 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP >non-issue v6.4.0 v7.0.0-beta1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants