Low-level APIs

- Generator for low-level methods
- Generated low-level methods
- Guide for low-level methods
- Samples for low-level methods

Signed-off-by: Theo Truong <theotr@amazon.com>

CHANGELOG.md

@@ -3,6 +3,8 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 
 ## [Unreleased]
 ### Added
+- Added `http` namespace with low-level APIs ([#211](https://github.com/opensearch-project/opensearch-ruby/pull/211))
+- Added a guide and a sample for low-level APIS ([#211](https://github.com/opensearch-project/opensearch-ruby/pull/211))
 ### Changed
 ### Deprecated
 ### Removed

USER_GUIDE.md

@@ -138,6 +138,7 @@ response = client.search index: index_name, body: search {
 - [Advanced Index Actions](guides/advanced_index_actions.md)
 - [Index Templates](guides/index_template.md)
 - [Transport Options](guides/transport_options.md)
+- [Custom HTTP Requests](guides/json.md)
 
 ## Amazon OpenSearch Service
 

api_generator/USER_GUIDE.md

@@ -22,27 +22,39 @@ require './lib/api_generator'
 generator = ApiGenerator.new('./OpenSearch.openapi.json')
 ```
 
-The `generate` method accepts the path to the root directory of the `opensearch-ruby` gem as a parameter. By default, it points to the parent directory of the folder containing the generator script. For example to generate all actions into the `tmp` directory:
+### Generate High-level API Actions
+
+The `generate_high_level` method accepts the path to the root directory of the `opensearch-ruby` gem as a parameter. By default, it points to the parent directory of the folder containing the generator script. For example to generate all actions into the `tmp` directory:
 ```ruby
-generator.generate('./tmp')
+generator.generate_high_level('./tmp')
 ```
 
 You can also target a specific API version by passing in the version number as a parameter. For example to generate all actions for version `1.0` into the `tmp` directory:
 ```ruby
-generator.generate(version: '1.0')
+generator.generate_high_level(version: '1.0')
 ```
 
 The generator also support incremental generation. For example, to generate all actions of the `cat` namespace:
 ```ruby
-generator.generate(namespace: 'cat')
+generator.generate_high_level(namespace: 'cat')
 ```
 
 To limit it to specific actions of a namespace:
 ```ruby
-generator.generate(namespace: 'cat', actions: %w[aliases allocation])
+generator.generate_high_level(namespace: 'cat', actions: %w[aliases allocation])
 ```
 
 Note that the root namespace is presented by an empty string `''`. For example, to generate all actions of the root namespace for OS version 2.3:
 ```ruby
-generator.generate(version: '2.3', namespace: '')
+generator.generate_high_level(version: '2.3', namespace: '')
 ```
+
+### Generate low-level API actions:
+
+To generate the low level API methods:
+
+```ruby
+generator.generate_low_level
+```
+
+The low level API methods are independent of the Spec. A change in the OpenSearch API Spec will not affect the low level APIs. So, only execute `generate_low_level` if you have made changes to the low-level APIs logic. 

api_generator/lib/action_generator.rb

@@ -74,7 +74,7 @@ def http_verb
   end
 
   def required_args
-    @action.required_components.map { |arg| { arg: } }
+    @action.required_components.map { |arg| { arg: arg } }
            .tap { |args| args.last&.[]=('_blank_line', true) }
   end
 

api_generator/lib/api_generator.rb

@@ -12,6 +12,7 @@
 require_relative 'spec_generator'
 require_relative 'namespace_generator'
 require_relative 'index_generator'
+require_relative 'low_level_action_generator'
 
 # Generate API endpoints for OpenSearch Ruby client
 class ApiGenerator
@@ -22,11 +23,12 @@ def initialize(openapi_spec)
     @spec = Openapi3Parser.load_file(openapi_spec)
   end
 
+  # Generate high level API methods from the OpenSearch Specs.
   # @param [String] gem_folder location of the API Gem folder (default to the parent folder of the generator)
   # @param [String] version target OpenSearch version to generate like "2.5" or "3.0"
   # @param [String] namespace namespace to generate (Default to all namespaces. Use '' for root)
   # @param [Array<String>] actions list of actions in the specified namespace to generate (Default to all actions)
-  def generate(gem_folder = '../', version: nil, namespace: nil, actions: nil)
+  def generate_high_level(gem_folder = '../', version: nil, namespace: nil, actions: nil)
     gem_folder = Pathname gem_folder
     namespaces = existing_namespaces(gem_folder)
     target_actions(version, namespace, actions).each do |action|
@@ -37,6 +39,22 @@ def generate(gem_folder = '../', version: nil, namespace: nil, actions: nil)
     IndexGenerator.new(gem_folder.join('lib/opensearch'), namespaces).generate
   end
 
+  # Generate low level API methods that are independent of the OpenSearch Specs.
+  # @param [String] gem_folder location of the API Gem folder (default to the parent folder of the generator)
+  def generate_low_level(gem_folder = '../')
+    gem_folder = Pathname gem_folder
+    namespaces = existing_namespaces(gem_folder)
+    low_level_namespace = 'http'
+
+    NamespaceGenerator.new(gem_folder.join('lib/opensearch/api/namespace'), low_level_namespace).generate(namespaces)
+    LowLevelBaseActionGenerator.new(gem_folder.join('lib/opensearch/api/actions'), low_level_namespace).generate
+    IndexGenerator.new(gem_folder.join('lib/opensearch'), namespaces).generate
+
+    %w[head get post put patch delete options trace connect].each do |action|
+      LowLevelActionGenerator.new(gem_folder.join('lib/opensearch/api/actions'), low_level_namespace, action).generate
+    end
+  end
+
   private
 
   def target_actions(version, namespace, actions)

api_generator/lib/low_level_action_generator.rb

@@ -0,0 +1,27 @@
+# SPDX-License-Identifier: Apache-2.0
+#
+# The OpenSearch Contributors require contributions made to
+# this file be licensed under the Apache-2.0 license or a
+# compatible open source license.
+
+# frozen_string_literal: true
+
+require_relative 'low_level_base_action_generator'
+
+# Generate low-level API actions via Mustache
+class LowLevelActionGenerator < LowLevelBaseActionGenerator
+  self.template_file = './templates/low_level_action.mustache'
+
+  def initialize(output_folder, namespace, action)
+    super(output_folder, namespace)
+    @action = action
+  end
+
+  def lower_cased
+    @action.downcase
+  end
+
+  def upper_cased
+    @action.upcase
+  end
+end

api_generator/lib/low_level_base_action_generator.rb

@@ -0,0 +1,30 @@
+# SPDX-License-Identifier: Apache-2.0
+#
+# The OpenSearch Contributors require contributions made to
+# this file be licensed under the Apache-2.0 license or a
+# compatible open source license.
+
+# frozen_string_literal: true
+
+require_relative 'base_generator'
+
+# Generate low-level API actions via Mustache
+class LowLevelBaseActionGenerator < BaseGenerator
+  self.template_file = './templates/low_level_base_action.mustache'
+
+  def initialize(output_folder, namespace)
+    super(output_folder)
+    @namespace = namespace
+    @action = 'request'
+  end
+
+  def namespace_module
+    @namespace.camelize
+  end
+
+  private
+
+  def output_file
+    create_folder(*[@output_folder, @namespace].compact).join("#{@action}.rb")
+  end
+end

api_generator/templates/low_level_action.mustache

@@ -0,0 +1,22 @@
+{{{license_header}}}
+{{{generated_code_warning}}}
+
+# frozen_string_literal: true
+
+module OpenSearch
+  module API
+    module {{namespace_module}}
+      module Actions
+        # Make a customized {{upper_cased}} request.
+        #
+        # @option arguments [String] :url Relative path to the endpoint (e.g. 'cat/indices/books,movies') (*Required*)
+        # @option arguments [Hash] :params Querystring parameters to be appended to the path
+        # @option arguments [Hash] :headers Custom HTTP headers
+        # @option arguments [String | Hash | Array<Hash>] :body The body of the request
+        def {{lower_cased}}(url, headers: {}, body: nil, params: {})
+          request('{{upper_cased}}', url, headers: headers, body: body, params: params)
+        end
+      end
+    end
+  end
+end

api_generator/templates/low_level_base_action.mustache

@@ -0,0 +1,21 @@
+{{{license_header}}}
+{{{generated_code_warning}}}
+
+# frozen_string_literal: true
+
+module OpenSearch
+  module API
+    module {{namespace_module}}
+      module Actions
+        private
+
+        def request(method, url, headers: {}, body: nil, params: {})
+          body = OpenSearch::API::Utils.__bulkify(body) if body.is_a?(Array)
+          headers.merge!('Content-Type' => 'application/x-ndjson') if body.is_a?(Array)
+
+          perform_request(method, url, params, body, headers).body
+        end
+      end
+    end
+  end
+end

guides/json.md

@@ -0,0 +1,62 @@
+# Making raw HTTP requests
+
+The OpenSearch client implements many high-level REST DSLs that invoke OpenSearch APIs. However, you may find yourself in a situation that requires you to invoke an API that is not supported by the client. In this case, you can use raw HTTP requests to invoke any OpenSearch API. This guide shows you different ways to make custom API calls using the OpenSearch Ruby client.
+
+## Setup
+First, create a client instance with the following code to connect to a local OpenSearch cluster using default security credentials:
+
+```ruby
+require 'opensearch-ruby'
+client = OpenSearch::Client.new(
+  hosts: ['https://localhost:9200'],
+  user: 'admin',
+  password: 'admin',
+  transport_options: { ssl: { verify: false } }
+)
+```
+
+## The http Namespace
+
+The `http` namespace provides a method of every HTTP verb (GET, POST, HEAD...). Normally, to get a summary of all indices in the cluster, you would use `client.cat.indices`. However, you can achieve the same result using `client.http.get`:
+
+```ruby
+puts client.http.get('_cat/indices')
+```
+
+Of course, you can also pass query-string parameters, headers, and a request body to the `http` methods. For example, the following block of code creates an index named `movies` with 5 shards and 2 replicas, with explicit timeout of 30 seconds, and content-type of `application/json`:
+
+```ruby
+body = { settings: { number_of_shards: 5, number_of_replicas: 2 } }
+params = { timeout: '30s' }
+headers = { 'Content-Type' => 'application/json' }
+
+client.http.put('movies', body: body, params: params, headers: headers)
+```
+
+If you provide the http method with a string as body, it will be sent to the server as is. However, you can also provide them with a Ruby hash or an array of hashes, and the client will convert them to a JSON string or a newline-delimited JSON (NDJSON) string respectively. For example, to create two documents in the `books` index using the `bulk` endpoint:
+
+```ruby
+body = [{ index: { _index: "books", _id: 1 } },
+        { title: "The Lion King", year: 1994 },
+        { index: { _index: "books", _id: 2 } },
+        { title: "Beauty and the Beast", year: 1991 }]
+
+client.http.post('_bulk', body: body)
+```
+
+Let's clean up and delete the `movies` index:
+
+```ruby
+client.http.delete('movies')
+```
+
+The `http` namespace includes the following methods:
+- get
+- put
+- post
+- delete
+- head
+- options
+- patch
+- trace
+- connect

lib/opensearch/api.rb

@@ -79,6 +79,7 @@ def self.included(base)
                 OpenSearch::API::Cluster,
                 OpenSearch::API::DanglingIndices,
                 OpenSearch::API::Features,
+                OpenSearch::API::Http,
                 OpenSearch::API::Indices,
                 OpenSearch::API::Ingest,
                 OpenSearch::API::Nodes,

lib/opensearch/api/actions/http/connect.rb

@@ -0,0 +1,28 @@
+# SPDX-License-Identifier: Apache-2.0
+#
+# The OpenSearch Contributors require contributions made to
+# this file be licensed under the Apache-2.0 license or a
+# compatible open source license.
+
+# This code was generated from OpenSearch API Spec.
+# Update the code generation logic instead of modifying this file directly.
+
+# frozen_string_literal: true
+
+module OpenSearch
+  module API
+    module Http
+      module Actions
+        # Make a customized CONNECT request.
+        #
+        # @option arguments [String] :url Relative path to the endpoint (e.g. 'cat/indices/books,movies') (*Required*)
+        # @option arguments [Hash] :params Querystring parameters to be appended to the path
+        # @option arguments [Hash] :headers Custom HTTP headers
+        # @option arguments [String | Hash | Array<Hash>] :body The body of the request
+        def connect(url, headers: {}, body: nil, params: {})
+          request('CONNECT', url, headers: headers, body: body, params: params)
+        end
+      end
+    end
+  end
+end

lib/opensearch/api/actions/http/delete.rb

@@ -0,0 +1,28 @@
+# SPDX-License-Identifier: Apache-2.0
+#
+# The OpenSearch Contributors require contributions made to
+# this file be licensed under the Apache-2.0 license or a
+# compatible open source license.
+
+# This code was generated from OpenSearch API Spec.
+# Update the code generation logic instead of modifying this file directly.
+
+# frozen_string_literal: true
+
+module OpenSearch
+  module API
+    module Http
+      module Actions
+        # Make a customized DELETE request.
+        #
+        # @option arguments [String] :url Relative path to the endpoint (e.g. 'cat/indices/books,movies') (*Required*)
+        # @option arguments [Hash] :params Querystring parameters to be appended to the path
+        # @option arguments [Hash] :headers Custom HTTP headers
+        # @option arguments [String | Hash | Array<Hash>] :body The body of the request
+        def delete(url, headers: {}, body: nil, params: {})
+          request('DELETE', url, headers: headers, body: body, params: params)
+        end
+      end
+    end
+  end
+end

lib/opensearch/api/actions/http/get.rb

@@ -0,0 +1,28 @@
+# SPDX-License-Identifier: Apache-2.0
+#
+# The OpenSearch Contributors require contributions made to
+# this file be licensed under the Apache-2.0 license or a
+# compatible open source license.
+
+# This code was generated from OpenSearch API Spec.
+# Update the code generation logic instead of modifying this file directly.
+
+# frozen_string_literal: true
+
+module OpenSearch
+  module API
+    module Http
+      module Actions
+        # Make a customized GET request.
+        #
+        # @option arguments [String] :url Relative path to the endpoint (e.g. 'cat/indices/books,movies') (*Required*)
+        # @option arguments [Hash] :params Querystring parameters to be appended to the path
+        # @option arguments [Hash] :headers Custom HTTP headers
+        # @option arguments [String | Hash | Array<Hash>] :body The body of the request
+        def get(url, headers: {}, body: nil, params: {})
+          request('GET', url, headers: headers, body: body, params: params)
+        end
+      end
+    end
+  end
+end