Low-level APIs (#211)

- Generator for low-level methods
- Generated low-level methods
- Guide for low-level methods
- Samples for low-level methods

Signed-off-by: Theo Truong <theotr@amazon.com>

CHANGELOG.md

@@ -3,6 +3,8 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 
 ## [Unreleased]
 ### Added
+- Added `http.get`, `http.post`, `http.patch`, `http.put`, `http.trace`, `http.head`, `http.options`, `http.connect`, and `http.delete`  ([#211](https://github.com/opensearch-project/opensearch-ruby/pull/211))
+- Added a guide and a sample for `http` namespace ([#211](https://github.com/opensearch-project/opensearch-ruby/pull/211))
 ### Changed
 ### Deprecated
 ### Removed

USER_GUIDE.md

@@ -138,6 +138,7 @@ response = client.search index: index_name, body: search {
 - [Advanced Index Actions](guides/advanced_index_actions.md)
 - [Index Templates](guides/index_template.md)
 - [Transport Options](guides/transport_options.md)
+- [Custom HTTP Requests](guides/json.md)
 
 ## Amazon OpenSearch Service
 

api_generator/USER_GUIDE.md

@@ -3,10 +3,10 @@ This API Generator generates API actions for the OpenSearch Ruby client based of
 ---
 
 ### Usage
-This generator should be run everytime the OpenSearch API Specification is updated to propagate the changes to the Ruby client. For now, this must be done manually:
+This generator should be run everytime the OpenSearch API specification is updated to propagate the changes to the Ruby client. For now, this must be done manually:
 - Create a new branch from `main`
 - Download the latest OpenSearch API Specification from [The API Spec Repo](https://github.com/opensearch-project/opensearch-api-specification/blob/main/OpenSearch.openapi.json)
-- Run the generator with the API Spec downloaded previously (see below)
+- Run the generator with the API spec downloaded previously (see below)
 - Run Rubocop with `-a` flag to remove redundant spacing from the generated code `rubocop -a`
 - Commit and create a PR to merge the updated API actions into `main`.
 
@@ -22,27 +22,39 @@ require './lib/api_generator'
 generator = ApiGenerator.new('./OpenSearch.openapi.json')
 ```
 
-The `generate` method accepts the path to the root directory of the `opensearch-ruby` gem as a parameter. By default, it points to the parent directory of the folder containing the generator script. For example to generate all actions into the `tmp` directory:
+### Generate Spec Methods:
+
+The `generate_spec_methods` method accepts the path to the root directory of the `opensearch-ruby` gem as a parameter. By default, it points to the parent directory of the folder containing the generator script. For example to generate all actions into the `tmp` directory:
 ```ruby
-generator.generate('./tmp')
+generator.generate_spec_methods('./tmp')
 ```
 
 You can also target a specific API version by passing in the version number as a parameter. For example to generate all actions for version `1.0` into the `tmp` directory:
 ```ruby
-generator.generate(version: '1.0')
+generator.generate_spec_methods(version: '1.0')
 ```
 
 The generator also support incremental generation. For example, to generate all actions of the `cat` namespace:
 ```ruby
-generator.generate(namespace: 'cat')
+generator.generate_spec_methods(namespace: 'cat')
 ```
 
 To limit it to specific actions of a namespace:
 ```ruby
-generator.generate(namespace: 'cat', actions: %w[aliases allocation])
+generator.generate_spec_methods(namespace: 'cat', actions: %w[aliases allocation])
 ```
 
 Note that the root namespace is presented by an empty string `''`. For example, to generate all actions of the root namespace for OS version 2.3:
 ```ruby
-generator.generate(version: '2.3', namespace: '')
+generator.generate_spec_methods(version: '2.3', namespace: '')
 ```
+
+### Generate Static Methods:
+
+To generate static methods:
+
+```ruby
+generator.generate_static_methods
+```
+
+The static methods are independent of the spec. A change in the OpenSearch API spec will not affect these methods.

api_generator/lib/action_generator.rb

@@ -74,7 +74,7 @@ def http_verb
   end
 
   def required_args
-    @action.required_components.map { |arg| { arg: } }
+    @action.required_components.map { |arg| { arg: arg } }
            .tap { |args| args.last&.[]=('_blank_line', true) }
   end
 

api_generator/lib/api_generator.rb

@@ -12,6 +12,7 @@
 require_relative 'spec_generator'
 require_relative 'namespace_generator'
 require_relative 'index_generator'
+require_relative 'low_level_action_generator'
 
 # Generate API endpoints for OpenSearch Ruby client
 class ApiGenerator
@@ -22,11 +23,12 @@ def initialize(openapi_spec)
     @spec = Openapi3Parser.load_file(openapi_spec)
   end
 
+  # Generate API methods from the OpenSearch Specs.
   # @param [String] gem_folder location of the API Gem folder (default to the parent folder of the generator)
   # @param [String] version target OpenSearch version to generate like "2.5" or "3.0"
   # @param [String] namespace namespace to generate (Default to all namespaces. Use '' for root)
   # @param [Array<String>] actions list of actions in the specified namespace to generate (Default to all actions)
-  def generate(gem_folder = '../', version: nil, namespace: nil, actions: nil)
+  def generate_spec_methods(gem_folder = '../', version: nil, namespace: nil, actions: nil)
     gem_folder = Pathname gem_folder
     namespaces = existing_namespaces(gem_folder)
     target_actions(version, namespace, actions).each do |action|
@@ -37,6 +39,22 @@ def generate(gem_folder = '../', version: nil, namespace: nil, actions: nil)
     IndexGenerator.new(gem_folder.join('lib/opensearch'), namespaces).generate
   end
 
+  # Generate basic HTTP methods that are independent of the OpenSearch Specs.
+  # @param [String] gem_folder location of the API Gem folder (default to the parent folder of the generator)
+  def generate_static_methods(gem_folder = '../')
+    gem_folder = Pathname gem_folder
+    namespaces = existing_namespaces(gem_folder)
+    low_level_namespace = 'http'
+
+    NamespaceGenerator.new(gem_folder.join('lib/opensearch/api/namespace'), low_level_namespace).generate(namespaces)
+    LowLevelBaseActionGenerator.new(gem_folder.join('lib/opensearch/api/actions'), low_level_namespace).generate
+    IndexGenerator.new(gem_folder.join('lib/opensearch'), namespaces).generate
+
+    %w[head get post put patch delete options trace connect].each do |action|
+      LowLevelActionGenerator.new(gem_folder.join('lib/opensearch/api/actions'), low_level_namespace, action).generate
+    end
+  end
+
   private
 
   def target_actions(version, namespace, actions)

api_generator/lib/low_level_action_generator.rb

@@ -0,0 +1,27 @@
+# SPDX-License-Identifier: Apache-2.0
+#
+# The OpenSearch Contributors require contributions made to
+# this file be licensed under the Apache-2.0 license or a
+# compatible open source license.
+
+# frozen_string_literal: true
+
+require_relative 'low_level_base_action_generator'
+
+# Generate low-level API actions via Mustache
+class LowLevelActionGenerator < LowLevelBaseActionGenerator
+  self.template_file = './templates/low_level_action.mustache'
+
+  def initialize(output_folder, namespace, action)
+    super(output_folder, namespace)
+    @action = action
+  end
+
+  def lower_cased
+    @action.downcase
+  end
+
+  def upper_cased
+    @action.upcase
+  end
+end

api_generator/lib/low_level_base_action_generator.rb

@@ -0,0 +1,30 @@
+# SPDX-License-Identifier: Apache-2.0
+#
+# The OpenSearch Contributors require contributions made to
+# this file be licensed under the Apache-2.0 license or a
+# compatible open source license.
+
+# frozen_string_literal: true
+
+require_relative 'base_generator'
+
+# Generate low-level API actions via Mustache
+class LowLevelBaseActionGenerator < BaseGenerator
+  self.template_file = './templates/low_level_base_action.mustache'
+
+  def initialize(output_folder, namespace)
+    super(output_folder)
+    @namespace = namespace
+    @action = 'request'
+  end
+
+  def namespace_module
+    @namespace.camelize
+  end
+
+  private
+
+  def output_file
+    create_folder(*[@output_folder, @namespace].compact).join("#{@action}.rb")
+  end
+end

api_generator/templates/low_level_action.mustache

@@ -0,0 +1,22 @@
+{{{license_header}}}
+{{{generated_code_warning}}}
+
+# frozen_string_literal: true
+
+module OpenSearch
+  module API
+    module {{namespace_module}}
+      module Actions
+        # Make a customized {{upper_cased}} request.
+        #
+        # @option arguments [String] :url Relative path to the endpoint (e.g. 'cat/indices/books,movies') (*Required*)
+        # @option arguments [Hash] :params Querystring parameters to be appended to the path
+        # @option arguments [Hash] :headers Custom HTTP headers
+        # @option arguments [String | Hash | Array<Hash>] :body The body of the request
+        def {{lower_cased}}(url, headers: {}, body: nil, params: {})
+          request('{{upper_cased}}', url, headers: headers, body: body, params: params)
+        end
+      end
+    end
+  end
+end

api_generator/templates/low_level_base_action.mustache

@@ -0,0 +1,21 @@
+{{{license_header}}}
+{{{generated_code_warning}}}
+
+# frozen_string_literal: true
+
+module OpenSearch
+  module API
+    module {{namespace_module}}
+      module Actions
+        private
+
+        def request(method, url, headers: {}, body: nil, params: {})
+          body = OpenSearch::API::Utils.__bulkify(body) if body.is_a?(Array)
+          headers.merge!('Content-Type' => 'application/x-ndjson') if body.is_a?(Array)
+
+          perform_request(method, url, params, body, headers).body
+        end
+      end
+    end
+  end
+end

guides/json.md

@@ -0,0 +1,62 @@
+# Making raw HTTP requests
+
+The OpenSearch client implements many high-level REST DSLs that invoke OpenSearch APIs. However, you may find yourself in a situation that requires you to invoke an API that is not supported by the client. In this case, you can use raw HTTP requests to invoke any OpenSearch API. This guide shows you different ways to make custom API calls using the OpenSearch Ruby client.
+
+## Setup
+First, create a client instance with the following code to connect to a local OpenSearch cluster using default security credentials:
+
+```ruby
+require 'opensearch-ruby'
+client = OpenSearch::Client.new(
+  hosts: ['https://localhost:9200'],
+  user: 'admin',
+  password: 'admin',
+  transport_options: { ssl: { verify: false } }
+)
+```
+
+## The http Namespace
+
+The `http` namespace provides a method of every HTTP verb (GET, POST, HEAD...). Normally, to get a summary of all indices in the cluster, you would use `client.cat.indices`. However, you can achieve the same result using `client.http.get`:
+
+```ruby
+puts client.http.get('_cat/indices')
+```
+
+Of course, you can also pass query-string parameters, headers, and a request body to the `http` methods. For example, the following block of code creates an index named `movies` with 5 shards and 2 replicas, with explicit timeout of 30 seconds, and content-type of `application/json`:
+
+```ruby
+body = { settings: { number_of_shards: 5, number_of_replicas: 2 } }
+params = { timeout: '30s' }
+headers = { 'Content-Type' => 'application/json' }
+
+client.http.put('movies', body: body, params: params, headers: headers)
+```
+
+If you provide the http method with a string as body, it will be sent to the server as is. However, you can also provide them with a Ruby hash or an array of hashes, and the client will convert them to a JSON string or a newline-delimited JSON (NDJSON) string respectively. For example, to create two documents in the `books` index using the `bulk` endpoint:
+
+```ruby
+body = [{ index: { _index: "books", _id: 1 } },
+        { title: "The Lion King", year: 1994 },
+        { index: { _index: "books", _id: 2 } },
+        { title: "Beauty and the Beast", year: 1991 }]
+
+client.http.post('_bulk', body: body)
+```
+
+Let's clean up and delete the `movies` index:
+
+```ruby
+client.http.delete('movies')
+```
+
+The `http` namespace includes the following methods:
+- get
+- put
+- post
+- delete
+- head
+- options
+- patch
+- trace
+- connect

lib/opensearch/api.rb

@@ -79,6 +79,7 @@ def self.included(base)
                 OpenSearch::API::Cluster,
                 OpenSearch::API::DanglingIndices,
                 OpenSearch::API::Features,
+                OpenSearch::API::Http,
                 OpenSearch::API::Indices,
                 OpenSearch::API::Ingest,
                 OpenSearch::API::Nodes,

lib/opensearch/api/actions/http/connect.rb

@@ -0,0 +1,28 @@
+# SPDX-License-Identifier: Apache-2.0
+#
+# The OpenSearch Contributors require contributions made to
+# this file be licensed under the Apache-2.0 license or a
+# compatible open source license.
+
+# This code was generated from OpenSearch API Spec.
+# Update the code generation logic instead of modifying this file directly.
+
+# frozen_string_literal: true
+
+module OpenSearch
+  module API
+    module Http
+      module Actions
+        # Make a customized CONNECT request.
+        #
+        # @option arguments [String] :url Relative path to the endpoint (e.g. 'cat/indices/books,movies') (*Required*)
+        # @option arguments [Hash] :params Querystring parameters to be appended to the path
+        # @option arguments [Hash] :headers Custom HTTP headers
+        # @option arguments [String | Hash | Array<Hash>] :body The body of the request
+        def connect(url, headers: {}, body: nil, params: {})
+          request('CONNECT', url, headers: headers, body: body, params: params)
+        end
+      end
+    end
+  end
+end

lib/opensearch/api/actions/http/delete.rb

@@ -0,0 +1,28 @@
+# SPDX-License-Identifier: Apache-2.0
+#
+# The OpenSearch Contributors require contributions made to
+# this file be licensed under the Apache-2.0 license or a
+# compatible open source license.
+
+# This code was generated from OpenSearch API Spec.
+# Update the code generation logic instead of modifying this file directly.
+
+# frozen_string_literal: true
+
+module OpenSearch
+  module API
+    module Http
+      module Actions
+        # Make a customized DELETE request.
+        #
+        # @option arguments [String] :url Relative path to the endpoint (e.g. 'cat/indices/books,movies') (*Required*)
+        # @option arguments [Hash] :params Querystring parameters to be appended to the path
+        # @option arguments [Hash] :headers Custom HTTP headers
+        # @option arguments [String | Hash | Array<Hash>] :body The body of the request
+        def delete(url, headers: {}, body: nil, params: {})
+          request('DELETE', url, headers: headers, body: body, params: params)
+        end
+      end
+    end
+  end
+end

lib/opensearch/api/actions/http/get.rb

@@ -0,0 +1,28 @@
+# SPDX-License-Identifier: Apache-2.0
+#
+# The OpenSearch Contributors require contributions made to
+# this file be licensed under the Apache-2.0 license or a
+# compatible open source license.
+
+# This code was generated from OpenSearch API Spec.
+# Update the code generation logic instead of modifying this file directly.
+
+# frozen_string_literal: true
+
+module OpenSearch
+  module API
+    module Http
+      module Actions
+        # Make a customized GET request.
+        #
+        # @option arguments [String] :url Relative path to the endpoint (e.g. 'cat/indices/books,movies') (*Required*)
+        # @option arguments [Hash] :params Querystring parameters to be appended to the path
+        # @option arguments [Hash] :headers Custom HTTP headers
+        # @option arguments [String | Hash | Array<Hash>] :body The body of the request
+        def get(url, headers: {}, body: nil, params: {})
+          request('GET', url, headers: headers, body: body, params: params)
+        end
+      end
+    end
+  end
+end