Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HTTP implementation based on elasticsearch-nio #28898

Closed
Tim-Brooks opened this issue Mar 4, 2018 · 3 comments
Closed

HTTP implementation based on elasticsearch-nio #28898

Tim-Brooks opened this issue Mar 4, 2018 · 3 comments
Assignees
@Tim-Brooks
Labels
:Distributed Coordination/Network Http and internode communication implementations >enhancement v7.0.0-beta1

Comments

@Tim-Brooks
Copy link
Contributor

Tim-Brooks commented Mar 4, 2018 •
edited
Loading

This is related to #27260. We will need a working http plugin based on the elasticsearch-nio work. This should have feature parity with our other http implementations. This means supporting:
@Tim-Brooks Tim-Brooks added >enhancement :Distributed Coordination/Network Http and internode communication implementations v7.0.0 labels Mar 4, 2018
@Tim-Brooks Tim-Brooks self-assigned this Mar 4, 2018
@Tim-Brooks Tim-Brooks mentioned this issue Mar 4, 2018
Closed
12 tasks
@jasontedor
Copy link
Member

FYI @elastic/es-core-infra.

@spinscale
Copy link
Contributor

super minor, but just to add it here for the sake of completeness: usage of the X-Opaque-Id in Netty4HttpChannel, a header that is passed from the request to the response for identification purposes

@Tim-Brooks Tim-Brooks mentioned this issue Apr 4, 2018
Closed
@Tim-Brooks Tim-Brooks mentioned this issue Apr 18, 2018
Merged
Tim-Brooks added a commit that referenced this issue May 15, 2018
@Tim-Brooks
Add nio http server transport (#29587)
99b9ab5 
This commit is related to #28898. It adds an nio driven http server
transport. Currently it only supports basic http features. Cors,
pipeling, and read timeouts will need to be added in future PRs.
@Tim-Brooks Tim-Brooks mentioned this issue May 17, 2018
Merged
Tim-Brooks added a commit that referenced this issue May 22, 2018
@Tim-Brooks
Make http pipelining support mandatory (#30695)
31251c9 
This is related to #29500 and #28898. This commit removes the abilitiy
to disable http pipelining. After this commit, any elasticsearch node
will support pipelined requests from a client. Additionally, it extracts
some of the http pipelining work to the server module. This extracted
work is used to implement pipelining for the nio plugin.
@Tim-Brooks Tim-Brooks mentioned this issue May 23, 2018
Merged
Tim-Brooks added a commit that referenced this issue Jun 5, 2018
@Tim-Brooks
Add cors support to NioHttpServerTransport (#30827)
05ee0f8 
This is related to #28898. This commit adds cors support to the nio http
transport. Most of the work is copied directly from the netty module
implementation. Additionally, this commit adds tests for the nio http
channel.
Tim-Brooks added a commit to Tim-Brooks/elasticsearch that referenced this issue Jun 6, 2018
@Tim-Brooks
Cleanup nio http thread names
108c5c9 
This is related to elastic#28898. This commit adds the acceptor thread name to
the method checking if this thread is a transport thread. Additionally,
it modifies the nio http transport to use the same worker name as the
netty4 http server transport.
@Tim-Brooks Tim-Brooks mentioned this issue Jun 6, 2018
Merged
Tim-Brooks added a commit that referenced this issue Jun 6, 2018
@Tim-Brooks
Cleanup nio http thread names (#31148)
4158387 
This is related to #28898. This commit adds the acceptor thread name to
the method checking if this thread is a transport thread. Additionally,
it modifies the nio http transport to use the same worker name as the
netty4 http server transport.
Tim-Brooks added a commit to Tim-Brooks/elasticsearch that referenced this issue Jun 6, 2018
@Tim-Brooks
Add nio-transport as option for http smoke tests
3fad3f9 
This is related to elastic#27260 and elastic#28898. This commit adds the transport-nio
plugin as a random option when running the http smoke tests. As part of
this PR, I identified an issue where cors support was not properly
enabled causing these tests to fail when using transport-nio. This
commit also fixes that issue.
@Tim-Brooks Tim-Brooks mentioned this issue Jun 6, 2018
Merged
Tim-Brooks added a commit that referenced this issue Jun 7, 2018
@Tim-Brooks
Add nio-transport as option for http smoke tests (#31162)
237f9b8 
This is related to #27260 and #28898. This commit adds the transport-nio
plugin as a random option when running the http smoke tests. As part of
this PR, I identified an issue where cors support was not properly
enabled causing these tests to fail when using transport-nio. This
commit also fixes that issue.
@Tim-Brooks Tim-Brooks mentioned this issue Jun 13, 2018
Merged
Tim-Brooks added a commit that referenced this issue Jun 14, 2018
@Tim-Brooks
Extract common http logic to server (#31311)
fcf1e41 
This is related to #28898. With the addition of the http nio transport,
we now have two different modules that provide http transports.
Currently most of the http logic lives at the module level. However,
some of this logic can live in server. In particular, some of the
setting of headers, cors, and pipelining. This commit begins this moving
in that direction by introducing lower level abstraction (HttpChannel,
HttpRequest, and HttpResonse) that is implemented by the modules. The
higher level rest request and rest channel work can live entirely in
server.
@Tim-Brooks Tim-Brooks mentioned this issue Jun 14, 2018
Merged
Tim-Brooks added a commit that referenced this issue Jun 15, 2018
@Tim-Brooks
Add byte array pooling to nio http transport (#31349)
a705e1a 
This is related to #28898. This PR implements pooling of bytes arrays
when reading from the wire in the http server transport. In order to do
this, we must integrate with netty reference counting. That manner in
which this PR implements this is making Pages in InboundChannelBuffer
reference counted. When we accessing the underlying page to pass to
netty, we retain the page. When netty releases its bytebuf, it releases
the underlying pages we have passed to it.
@Tim-Brooks
Copy link
Contributor Author

Closing this as the main work has been completed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Tim-Brooks Tim-Brooks

Labels
:Distributed Coordination/Network Http and internode communication implementations >enhancement v7.0.0-beta1
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@colings86 @spinscale @Tim-Brooks @jasontedor