Http response thread pool

[heifner]

Change Description

• Thanks go to @bspark8 Enable websocketpp part threading #6678 for testing and verifying the improvement of processing/sending http request on a different thread.
• This PR adds a thread pool to the http_plugin used for processing/sending http requests.
• Remove auto load of http_plugin in nodeos main.cpp since not all nodeos need the http_plugin and the extra threads it spawns.
• Application logic is still processed on the application thread.
• See Enable websocketpp part threading #6678 for additional info

Consensus Changes

None

API Changes

None

Documentation Additions

• http_plugin has new option:
  --http-threads arg (=2) Number of worker threads in http thread pool

[bspark8]

Thank you for reflecting idea !!
The original intent of the previous PR(#6678) was to process threading both parts receiving HTTP requests and responding to the HTTP requests because both parts were consuming almost same amount of cpu.
Instead of thread pool, I created a new io_context to be threaded separately and passed it to the acceptor of websocketpp.
So I would like to add some additional threading for the part that receives the HTTP requests. I would appreciate your feedback.

[heifner]

I missed that you are passing the new io_context to init. Thanks for pointing that out.

[heifner]

@bspark8 I've added a new io_context for the websocketpp server as you suggested.

[bspark8]

@heifner Thank you for additional threading for the part that receives the HTTP requests.
In addition, I did the same test that I did in the previous PR(#6678) in the current patch. Below is the result.

Current patch

• About 1137 TPS (22% performance improvement)
• Each thread CPU usage
  main: 88%
  receiving HTTP requests: 52%
  responding to the HTTP requests: 14%

[brianjohnson5972]

So we still want to default to 2 threads, even though we can have 1?

[heifner]

Since we now only load http_plugin if we need it, sure.