Merge pull request #929 from mandy-chessell/code2024

Add content packs and new UIs

site/docs/concepts/integration-daemon.md

@@ -14,7 +14,7 @@ Inside the integration daemon are one or more [Open Metadata Integration Service
 
 ## Integration connectors
 
-The code that manages the specific APIs and formats of the third party technology is encapsulated in a special type of connector called an [integration connector](/connectors/integration-connector).
+The code that manages the specific APIs and formats of the third party technology is encapsulated in a special type of connector called an [integration connector](/concepts/integration-connector).
 
 The specific interface that the integration connector needs to implement is defined by the [integration service](/services/omis). This interface enables the integration service to pass a context object to the connector before it is started. The context enables the connector to register a listener with the associated access service's [Out Topic](/concepts/out-topic), or call its REST API, or to push events to the access service's [In Topic](/concepts/in-topic). By default, the context uses the integration daemon's userId for requests to the access service which means that the metadata created by the integration connector will be credited to this user. If you want to use a different userId for metadata from each connector, the server's userId can be overridden in the connector's configuration.
 

site/docs/content-packs/cim-content-pack/overview.md

@@ -0,0 +1,10 @@
+<!-- SPDX-License-Identifier: CC-BY-4.0 -->
+<!-- Copyright Contributors to the Egeria project. -->
+
+# Cloud Information Model (CIM) Content Pack
+
+The *CloudInformationModel* content pack is an extraction for a glossary for the Cloud Information Model's data model.  It is created by processing glossary from the Cloud Information Model's [JSONLD formatted model](https://raw.githubusercontent.com/cloudinformationmodel/cloudinformationmodel/master/dist/model.jsonld). The content covers basic commerce concepts such as Party, Product, Invoice and Shipping. The cloud information project has been archived so there will be no further updates. However, it still provides a useful starter set glossary.
+
+This archive is available on GitHub as `CloudInformationModel.omarchive` at [https://github.com/odpi/egeria/blob/main/content-packs/CloudInformationModel.json](https://github.com/odpi/egeria/blob/main/content-packs/CloudInformationModel.json).  It is also included in Egeria's standard `omag-server-platform` distribution built from the `egeria.git` repository.
+
+--8<-- "snippets/abbr.md"

site/docs/content-packs/coco-content-pack/overview.md

@@ -0,0 +1,23 @@
+<!-- SPDX-License-Identifier: CC-BY-4.0 -->
+<!-- Copyright Contributors to the Egeria project. -->
+
+# Coco Pharmaceuticals Content Pack
+
+The *CocoComboArchive* supplies metadata to support the [Coco Pharmaceuticals scenarios](/practices/coco-pharmaceuticals/). It is a useful content pack to load when experimenting with Egeria's capabilities since it provides examples of many types of open metadata.  In addition, this metadata is also available in the following archives that are used in the[Open Metadata Labs](/education/open-metadata-labs/overview/) where different subsets of this metadata are loading into each of the servers.
+
+* *CocoBusinessSystemsArchive* provides a catalog of the business systems and the lineage between them and the load of their data into the data lake.  This archive simulates the type of metadata expected from an ETL tool suite.  It is intended for **cocoMDS5** in the open metadata labs but can be used in any server.
+
+* *CocoOrganizationArchive* - provides the profiles, user identifies and team of the featured personas of Coco Pharmaceuticals.
+
+* *CocoClinicalTrialsTemplatesArchive* - provides the template assets used for onboarding weekly patient measurements during a clinical trial.
+
+* *CocoGovernanceProgramArchive* - provides the metadata to describe Coco Pharmaceuticals governance program.
+
+* *CocoGovernanceEngineDefinitionsArchive* - provides the metadata to describe Coco Pharmaceuticals three governance engines: `AssetGovernance`, `AssetDiscovery` and `AssetQuality`.
+
+* *CocoSustainabilityArchive* - provides the base definitions for Coco Pharmaceutical's sustainability initiative.
+
+* *CocoTypesArchive* - provides additional types for Coco Pharmaceuticals.  These are `BiopsyScope` Enum, `BiopsyReport` Entity, `BiopsySupportingEvidence` Relationship and `ReviewedByClinicalTrials` Classification.
+
+
+--8<-- "snippets/abbr.md"

site/docs/content-packs/core-content-pack/overview.md

@@ -0,0 +1,9 @@
+<!-- SPDX-License-Identifier: CC-BY-4.0 -->
+<!-- Copyright Contributors to the Egeria project. -->
+
+# Core Content Pack
+
+The *CoreContentPack* contains the connector definitions for the open connectors supplied in this distribution along with the valid metadata values for the technologies they support. This content pack is designed to provide a good starting point for a new Egeria deployment. It is loaded automatically in the *active-metadata-store* sample server.
+
+
+--8<-- "snippets/abbr.md"

site/docs/content-packs/index.md

@@ -0,0 +1,6 @@
+<!-- SPDX-License-Identifier: CC-BY-4.0 -->
+<!-- Copyright Contributors to the Egeria project. -->
+
+# Content Packs
+
+--8<-- "snippets/abbr.md"

site/docs/content-packs/simple-content-pack/overview.md

@@ -0,0 +1,21 @@
+<!-- SPDX-License-Identifier: CC-BY-4.0 -->
+<!-- Copyright Contributors to the Egeria project. -->
+
+# Simple Content Pack
+
+The *SimpleCatalog* provides an example of a database, an API and an event structure linked to a glossary term.
+It is loaded automatically in the *simple-metadata-store* sample server.
+SimpleCatalog is also supplied as four archives for use in a demo showing 4 metadata access servers connected together in a single cohort.
+The archives are each loaded into a different server.
+It is then possible to show how the cohort integrates metadata from different catalogs.
+These archives are used in the *Development labs* which are part of the [Open Metadata Labs](/education/open-metadata-labs/overview/).
+
+* *SimpleAPICatalog* - API metadata typically found in an API catalog.
+
+* *SimpleDataCatalog* - Database metadata typically found in an Data catalog.
+
+* *SimpleEventCatalog* - Event metadata typically found in an API catalog.
+
+* *SimpleGovernanceCatalog* - A glossary term linked to metadata elements in the API, Event, Data catalogs.
+
+--8<-- "snippets/abbr.md"

site/docs/content-packs/types-content-pack/overview.md

@@ -0,0 +1,8 @@
+<!-- SPDX-License-Identifier: CC-BY-4.0 -->
+<!-- Copyright Contributors to the Egeria project. -->
+
+# Open Metadata Types Content Pack
+
+The *OpenMetadataTypes* contains all of the [open metadata type definitions](/types)  provided by Egeria. It is supplied for external utilities since each [OMAG server](/concepts/omag-server) capable of being a [cohort member](/concepts/cohort-member) will load these types on start up.
+
+--8<-- "snippets/abbr.md"

site/docs/guides/index.md

@@ -7,15 +7,17 @@ Egeria's guides provide practical guidance on different aspects of using Egeria
 
 --8<-- "snippets/getting-started.md"
 
-* [Planning Guide](/guides/planning) - The planning guide describes how to plan your Egeria installation and the development of the metadata definitions that will drive its use.
-* [Integration Guide](/guides/integration) - the integration guides describes how to set up integrations between Egeria and third party technologies.
-* [Connector catalog](/connectors) - the connector catalog provides a list of connectors available for Egeria.
-* [Developer Guide](/guides/developer) - the developer covers how to build connectors, utilities that call Egeria and open metadata archives.
-* [Administration Guide](/guides/admin) - the administration guide describes how to configure the [OMAG Server Platform](/concepts/ompag-server-platform) and [servers](/concepts/omag-server) that run in it.
+* [Planning Guide](/guides/planning) - The planning guide describes how to plan your Egeria installation and the development of the metadata definitions that will drive its use.  Part of this planning process makes use of the following catalogs:
+
+    * [Content Pack Catalog](/content-packs) - the content pack catalog provides detailed information about the content packs available for Egeria.  These content pack provide metadata to bootstrap your deployment.
+    * [Connector Catalog](/connectors) - the connector catalog provides a list of connectors available for Egeria.
+
+* [Integration Guide](/guides/integration) - the integration guides describes how to set up integrations between Egeria and third party technologies.  It also makes use of the content packs and connector catalogs.
+* [Administration Guide](/guides/admin) - the administration guide describes how to mange Egeria's configuration.
 * [Migration Guide](/guides/migration/migrating-configuration-documents) - Egeria strives to maintain compatibility of APIs and file formats from release to release.  The migration guide covers techniques to migrate aspects of Egeria where compatibility between releases was not possible..
 * [Operations Guide](/guides/operations/overview) - Describes commands to control and monitor an Egeria deployment.
 * [Diagnostic Guide](/guides/diagnostic/overview) - Problem determination methods and guidance.
-* [User Interfaces](/user-interfaces) - Details of Egeria's user interfaces.
-* [Features](/features) - Provides a detailed guide to the features of Egeria.
+* [Developer Guide](/guides/developer) - the developer covers how to build connectors, utilities that call Egeria, and open metadata archives.
+* [Features](/features) - Provides a detailed guide to the governance features of Egeria.
 
 --8<-- "snippets/abbr.md"

site/docs/guides/planning/index.md

@@ -1,18 +1,30 @@
 <!-- SPDX-License-Identifier: CC-BY-4.0 -->
-<!-- Copyright Contributors to the Egeria project 2019. -->
+<!-- Copyright Contributors to the Egeria project 2019-present. -->
 
 # Planning Guide
 
-The *Planning Guide* provides information to help you plan the deployment of Egeria in your organization.  It is divided into two parts as follows: 
+Egeria is highly modular, allowing you to choose which parts to use as supplied, and which to remove, extend, or replace with something customized for your organization.
+Typically, Egeria's function is broader than you need, at least initially, so often the challenge is to know which pieces you need to do the things you want to do, and what you can safely ignore.
+
+There are four types of "content" from Egeria that you need to consider: Content Packs, Connectors, Configuration and the Runtime Libraries.
+
+![Four layers of content](four-layers-of-choice.svg)
+
+* The *content packs* contain open metadata types and instances.  Open metadata is used to define the people, processes, data and technology that make up your organization.  These are all linked together and stored in a knowledge graph to show how they work together.  Egeria's content packs provide sets of open metadata that provide basic definitions that help to bootstrap your knowledge graph.
+* The *connectors* provide plugins into Egeria's runtime libraries that either connect to a specific technology to exchange metadata, or implement a specific governance action.  Connectors are configured using open metadata and only run when they are explicitly activated.  One of Egeria's content packs called *CoreContentPack.omarchive* includes the configuration for the [standard connectors included in Egeria](/connectors).
+* The *configuration* controls the activation of Egeria's services in the runtime.  Related services are grouped together into what are called [OMAG Servers](/concepts/omag-server) so they can be activated and deactivated together.  Egeria's services are used to build out your knowledge graph.  They can be called from the connectors, or other tools and technologies using Egeria's clients, or directly via the REST APIs.  Managing configuration is described in the [Administration Guide](/guides/admin).
+* The *runtime libraries* provide the services to manage the connectors and the knowledge graph.  They are packaged into a choice of two runtimes.  The [OMAG Server Platform](/concepts/omag-server-platform) is the typical runtime to use, particularly initially, since it can run all functions using multiple OMAG Servers and its configuration can be dynamically changed making it easy to experiment with different settings.  The [OMAG Server Runtime](/concepts/omag-server-runtime) is a specialist runtime for a cloud-native deployment that only runs a single OMAG Server using a fixed configuration.  The specific software services included in either runtime can be both removed or added at build time to allow you to fully customize Egeria's runtimes. 
+
+The *Planning Guide* provides information to help you plan the deployment of Egeria in your organization. As a general rule of thumb, we suggest using the content as supplied and only customize it as you get more confident.  To this end, when you build the main egeria repository ([egeria.git]()), it creates a distribution folder containing content packs, connectors, sample configuration and the runtime libraries along with `README.md` files describing how to use this content.  The Egeria project also includes additional content, such as connectors, in other repositories that can be built and added to this distribution.  The distribution also includes a Docker script file so that, once you have all of the content you need assembled, it can be turned into a docker container for deployment.  Alternatively it can be run directly on a local machine running Linux or MacOS (bare metal).
+
+
+The *Planning Guide* is divided into two parts as follows: 
 
 * *Planning Deployment* focuses on the setting up of the Egeria software and its supporting technology
 
-     * [Planning Runtime](/guides/planning/runtime/overview) describes how to plan your Open Metadata and Governance (OMAG) platforms and servers.
-     * [Designing your security model](/guides/planning/security/overview)
-     * Planning User Interfaces describes how to plan for the two user interfaces that Egeria supplies.
-
-         * General UI: TBA
-         * [Ecosystem UI](/user-interfaces/ecosystem/ecosystem-ui-planning)
+    * [Planning Runtime Deployment](/guides/planning/runtime/overview) describes how to design your Egeria runtime deployment.
+    * [Designing your security model](/guides/planning/security/overview) describes how to build a threat model for Egeria.
+    * [Planning User Interfaces](/guides/planning/user-interfaces) describes how to plan for the two user interfaces that Egeria supplies.
 
 * *Preparing Metadata Ecosystem* describes the types of metadata that should be populated in your open metadata ecosystem to allow it to operate effectively.
 

site/docs/guides/planning/planning-guide.drawio

@@ -0,0 +1,25 @@
+<mxfile host="Electron" modified="2024-05-21T16:25:48.152Z" agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) draw.io/24.1.0 Chrome/120.0.6099.109 Electron/28.1.0 Safari/537.36" etag="tfgaJ9QGfUZQx60fws8g" version="24.1.0" type="device">
+  <diagram name="four-layers-of-choice" id="SAM_tC9CIVqK7PkyQRDT">
+    <mxGraphModel dx="1114" dy="941" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="827" pageHeight="1169" math="0" shadow="0">
+      <root>
+        <mxCell id="0" />
+        <mxCell id="1" parent="0" />
+        <mxCell id="9Qbzj7xQOk-ZIAOEFLsu-1" value="Content Packs" style="rounded=0;whiteSpace=wrap;html=1;align=left;spacingLeft=14;fillColor=#ffe6cc;strokeColor=#d79b00;" vertex="1" parent="1">
+          <mxGeometry x="230" y="310" width="390" height="60" as="geometry" />
+        </mxCell>
+        <mxCell id="9Qbzj7xQOk-ZIAOEFLsu-2" value="Connectors" style="rounded=0;whiteSpace=wrap;html=1;align=left;spacingLeft=14;fillColor=#fff2cc;strokeColor=#d6b656;" vertex="1" parent="1">
+          <mxGeometry x="230" y="380" width="390" height="60" as="geometry" />
+        </mxCell>
+        <mxCell id="9Qbzj7xQOk-ZIAOEFLsu-3" value="Configuration" style="rounded=0;whiteSpace=wrap;html=1;align=left;spacingLeft=14;fillColor=#dae8fc;strokeColor=#6c8ebf;" vertex="1" parent="1">
+          <mxGeometry x="230" y="450" width="390" height="60" as="geometry" />
+        </mxCell>
+        <mxCell id="9Qbzj7xQOk-ZIAOEFLsu-4" value="Runtime Libraries" style="rounded=0;whiteSpace=wrap;html=1;align=left;spacingLeft=14;fillColor=#d5e8d4;strokeColor=#82b366;" vertex="1" parent="1">
+          <mxGeometry x="230" y="520" width="390" height="60" as="geometry" />
+        </mxCell>
+        <mxCell id="9Qbzj7xQOk-ZIAOEFLsu-6" value="What you want to do&lt;div&gt;(use cases)&lt;/div&gt;" style="rounded=0;whiteSpace=wrap;html=1;fillColor=none;verticalAlign=top;dashed=1;dashPattern=8 8;" vertex="1" parent="1">
+          <mxGeometry x="365" y="260" width="120" height="360" as="geometry" />
+        </mxCell>
+      </root>
+    </mxGraphModel>
+  </diagram>
+</mxfile>

site/docs/guides/planning/user-interfaces/index.md

@@ -0,0 +1,14 @@
+<!-- SPDX-License-Identifier: CC-BY-4.0 -->
+<!-- Copyright Contributors to the Egeria project. -->
+
+# User Interfaces
+
+Most users will work with Egeria's open metadata through their existing tools.  These tools are integrated into the Open Metadata Ecosystem so that Egeria can add, update and delete appropriate content in these tools.  Integrating tools into the open metadata ecosystem is described in the [Integration Guide](/guides/integration).
+
+There are, however, some specialist user interfaces provided to support functions that are unique to Egeria.
+
+* Egeria's [General User Interface](/user-interfaces/general/overview) makes use of Egeria's federated view of your organization's metadata to provide an enterprise asset catalog search and lineage viewer.  This UI will aggregate search results from across the open metadata ecosystem.  The aim is to show a complete, up-to-date picture.  It may not operate as fast as a dedicated asset catalog that has been integrated into the open metadata ecosystem and is maintaining open metadata from Egeria in a local repository. 
+* Egeria's [Python widgets](/user-interfaces/python-widgets/overview) provide text-based displays of Egeria's runtime status, and displays of related metadata elements.  They are aimed at a technical user who is working directly with Egeria and wishes to verify that something is either set up correctly or operating as expected.
+* Egeria's [Explorers](/user-interfaces/brain-explorers/overview) provide a cloud-based graphical interface for exploring Egeria's [Open Metadata Type System](/types) and its [Content Packs](/content-packs).
+
+--8<-- "snippets/abbr.md"