[Incremental Build] Differentiate the commits changes and finalize rebuild components

[zelinh]

This is a sub-issue from #3922.

When enable incremental build, the build workflow should be able to:

• Lock the references from input manifest to commits format.
• Compare the components from input manifest to the local build manifest from previous build, and identify new addition components.
• Find components with commit changes.
• Go through the input manifest with depends_on field to locate any affecting dependency components when rebuilding.

Acceptance criteria:

• Get a list of components for rebuilding.
• The components in the return list should include all modified plugins with commit changes, components depending on these modified component, and all plugin component if core(OpenSearch/OpenSearch-Dashboards) changes.

Exception:
For OpenSearch Dashboards, since we haven't enabled building components. #606. We would need to also include OpenSearch-Dashboards repo for rebuilding if any of the dashboard plugin changes.

[zelinh]

There will be essentially three parts to achieve the goal here:

1. Implement a logic to get all components depending on a given plugin in the input manifests.

   # Return a list of plugin that will depend on the given plugin
   def plugins_depend_on(self, plugin: str) -> List[str]:

2. Compare the input manifest and build manifest from previous build to find out new additional components and all components with commit changes.

3. Finalize components for rebuilding. This should include: new additional components; components with commit changes; all direct or indirect components that depend on previous two groups of components.

[zelinh]

For more exceptions I'm thinking about:

• If the build manifest is not found from local folder with path e.g. tar/builds/opensearch/manifest.yml, we would build core with all other plugins from the input manifest.
• The build manifest within local build folder can come from previous version. We also need a check to verify that the build version of current input manifest matching the version of the build manifest from previous build. Otherwise, we would build core with all other plugins.

[zelinh]

Closing this issue as it's completed.