feat: npm based 'new bot' flow behind feature flag (#5029)

* - Cloned creation flow
- Put new creation flow behind a feature flag

* added fetchTemplateV2 which grabs templates from public npm and NuGet feeds

* minor feed changes

* Adding passed feedUrl support for template fetch

* initial implementation of template picker design

* populate template description view with corresponding readMe pulled from npm and viewed as html

* package instantiated on creation from npm [partial]

* yeomen template generation working e2e for conversational core

* Cleaned up old template flashing bug, changed runtime template

* Adding featureflag data to ExtensionContext

* clean up

* Yeomen generator template working e2e with new runtime generator package

* -Added ability for variable runtime command in appsettings
-cleaned up var names

* Clean up to ensure new creation flow does not affect legacy creation flow

* Adding runtime template for C# adaptive runtime

* Removing undoable template sideloading conditional since extensions cant access feature flag state

* Adding path to yeomon repo to process.env

* ensure yeomen env intialized .yo-repository dir in appData directory

* Make dictionary for feeds that are called to populate template view. Clean up template tab useEffect.

* removing unused code and comments. Fixed double telem bug.

* moved template readMe into local component state as opposed to recoil app state

* removing run command string manipulation code as this is now handled in the yeomen template

* Making template feeds strongly typed and removing UI conditional

* minor bug fix, creationFlow V1 mounts and loads V1 templates regardless of new creation FF.
Adding conditional to prevent template load on mount.

* grabbing templates when app loads

* Adding back in PVA integration to new create flow

* adding null check and removing commented code

* Adding missing fields in creation flow UI to be used later with template creation

* fix linting errors

* removing unsed import

* - moved yeomen env instanse to assetManager
- fixed versioning for name validation regex

* Put yeomen env creation behind feature flag

* fix required fields

* updates to make package manager work

* Load yeomenEnv regardless of Feature Flag value

* adjust azurepublish to work with new runtime format

* Added additional error handling and cleaned up code

* removing yeomen constructor params

* fix repo instantiation

* removing unneeded condtional

* apply fix to exclude generated content from the schema merger

* fix issue where package manager would reload feed unnecessarily

* undoing autoformat changes

* interface to type conversion and clean up

* Moving server calls from asset controller to assetManager

* removing package-lock.json

* removing commented code

* Added template versioning

* Initial readme implementation

* Fininshing readme change

* removing auto format changes

* Adding remaining telem

* implementation of initial component unit test

* added unit test ffor new fetchtemplate server endpoint

* adding createProjAsync unit test

* increase timeout for creation test

* Fixing test expectation to account for undefined PVA variables

* minor clean up

* Fixed dir in which .yo-repository dir is generated

* macked long runnning async creation call to isolate createProjV2 test

* Adding test for template feed grabbing

* moved controller function to service class as it is not an api endpoint

* properly mocked createProjAsync for testing

* removing unused references

* Adding unit test for feed driven template  instantiation with mocked yeomen lib to avoid test api calls

* Adding test for fetch read me flow with mocked fetch

* Adding feature flag considarations to contribution documentation

* updating yarn.lock with latest output from yarn install

* minor PR changes

* Removing console.log that was used for testing

* removing unused reference

* Making PR changes
- removing console.logs()
- removing unneeded comments
- removing package.json
- changing string interpolation

* fixing unit test

Co-authored-by: Patrick Volum <pavolum@microsoft.com>
Co-authored-by: Ben Brown <benbro@microsoft.com>
Co-authored-by: Chris Whitten <christopher.whitten@microsoft.com>

CONTRIBUTING.md

@@ -9,6 +9,7 @@
 - Programatic testing for each change is a requirement. Comprehensive unit tests for every change is not. _"Write tests, not too many, mostly integration"_. We should be using [Jest](https://jestjs.io/) & [@testing-library/react](https://github.com/testing-library/react-testing-library) for this in the client. Code coverage benchmarks will be introduced and need to be met with each change (TBD).
 - Write code with Internationalization & Accessibility in mind. Every rendered string should be wrapped in an i18n API. Scrub through each UI change for keyboard-navigation and focus-indication. This will prevent most accessibility bugs.
 - Use `rebase` when merging changes into to the main branch. This can be done using the _“Squash and Merge”_ technique in the GitHub UI. Local branches will need to be updated using rebase as well. This will keep a clean commit history. Reach out to me if you need help understanding rebase.
+- Features pushed behind a feature flag (regardless if the feature flag is hidden or not) need to follow all traditional PR requirements to the main branch.
 
 ### Forking
 

Composer/packages/client/__tests__/components/CreationFlowV2/CreateOptions/index.test.tsx

@@ -0,0 +1,51 @@
+// Copyright (c) Microsoft Corporation.
+// Licensed under the MIT License.
+
+import * as React from 'react';
+import { fireEvent } from '@botframework-composer/test-utils';
+
+import { renderWithRecoil } from '../../../testUtils';
+import { CreateOptionsV2 } from '../../../../src/components/CreationFlow/v2/CreateOptions';
+
+describe('<CreateOptionsV2/>', () => {
+  const handleDismissMock = jest.fn();
+  const handleCreateNextMock = jest.fn();
+  const handleFetchReadMeMock = jest.fn();
+  const handleFetchTemplatesMock = jest.fn();
+
+  const templates = [
+    {
+      description: 'conversational core template generator',
+      id: 'generator-conversational-core',
+      index: 0,
+      name: 'conversational-core',
+      package: {
+        packageName: 'generator-conversational-core',
+        packageSource: 'npm',
+        packageVersion: '1.0.9',
+      },
+    },
+  ];
+
+  const renderComponent = () => {
+    return renderWithRecoil(
+      <CreateOptionsV2
+        fetchReadMe={handleFetchReadMeMock}
+        fetchTemplates={handleFetchTemplatesMock}
+        path="create"
+        templates={templates}
+        onDismiss={handleDismissMock}
+        onNext={handleCreateNextMock}
+      />
+    );
+  };
+
+  it('should save conversational core template id', async () => {
+    const component = renderComponent();
+    const conversationalCoreBot = await component.findByText('conversational-core');
+    fireEvent.click(conversationalCoreBot);
+    const nextButton = await component.findByText('Next');
+    fireEvent.click(nextButton);
+    expect(handleCreateNextMock).toBeCalledWith('generator-conversational-core');
+  });
+});

Composer/packages/client/__tests__/components/CreationFlowV2/DefineConversation/index.test.tsx

@@ -0,0 +1,69 @@
+// Copyright (c) Microsoft Corporation.
+// Licensed under the MIT License.
+
+import * as React from 'react';
+import { fireEvent, act, waitFor } from '@botframework-composer/test-utils';
+
+import { renderWithRecoil } from '../../../testUtils';
+import { StorageFolder } from '../../../../src/recoilModel/types';
+import { focusedStorageFolderState, storagesState } from '../../../../src/recoilModel';
+import DefineConversationV2 from '../../../../src/components/CreationFlow/v2/DefineConversation';
+
+describe('<DefineConversationV2/>', () => {
+  const onCurrentPathUpdateMock = jest.fn();
+  const onSubmitMock = jest.fn();
+  const onDismissMock = jest.fn();
+  const createFolder = jest.fn();
+  const updateFolder = jest.fn();
+  let locationMock;
+  const focusedStorageFolder: StorageFolder = {
+    name: 'Desktop',
+    parent: '/test-folder',
+    writable: true,
+    type: 'folder',
+    path: '/test-folder/Desktop',
+    children: [
+      {
+        name: 'EchoBot-0',
+        type: 'bot',
+        path: 'Desktop/EchoBot-11299',
+        lastModified: 'Wed Apr 22 2020 17:51:07 GMT-0700 (Pacific Daylight Time)',
+        size: 1,
+      },
+    ],
+  };
+  function renderComponent() {
+    return renderWithRecoil(
+      <DefineConversationV2
+        createFolder={createFolder}
+        focusedStorageFolder={focusedStorageFolder}
+        location={locationMock}
+        templateId={'EchoBot'}
+        updateFolder={updateFolder}
+        onCurrentPathUpdate={onCurrentPathUpdateMock}
+        onDismiss={onDismissMock}
+        onSubmit={onSubmitMock}
+      />,
+      ({ set }) => {
+        set(focusedStorageFolderState, {} as StorageFolder);
+        set(storagesState, [{ id: 'default' }]);
+      }
+    );
+  }
+
+  it('should render the component', () => {
+    const component = renderComponent();
+    expect(component.container).toBeDefined();
+  });
+
+  it('does not allow submission when the name is invalid', async () => {
+    const component = renderComponent();
+    const nameField = await component.getByTestId('NewDialogName');
+    act(() => {
+      fireEvent.change(nameField, { target: { value: 'invalidName;' } });
+    });
+
+    const node = await waitFor(() => component.getByTestId('SubmitNewBotBtn'));
+    expect(node).toBeDisabled();
+  });
+});

Composer/packages/client/__tests__/components/CreationFlowV2/index.test.tsx

@@ -0,0 +1,101 @@
+// Copyright (c) Microsoft Corporation.
+// Licensed under the MIT License.
+
+import * as React from 'react';
+import { render, fireEvent, act } from '@botframework-composer/test-utils';
+import { createHistory, createMemorySource, LocationProvider } from '@reach/router';
+import { RecoilRoot } from 'recoil';
+import { getDefaultFeatureFlags } from '@bfc/shared';
+
+import {
+  focusedStorageFolderState,
+  creationFlowStatusState,
+  dispatcherState,
+  featureFlagsState,
+} from '../../../src/recoilModel';
+import { CreationFlowStatus } from '../../../src/constants';
+import CreationFlowV2 from '../../../src/components/CreationFlow/v2/CreationFlow';
+
+describe('<CreationFlowV2/>', () => {
+  let locationMock;
+  const createProjectMock = jest.fn();
+  const initRecoilState = ({ set }) => {
+    set(dispatcherState, {
+      createNewBotV2: createProjectMock,
+      fetchStorages: jest.fn(),
+      fetchTemplateProjects: jest.fn(),
+      onboardingAddCoachMarkRef: jest.fn(),
+      fetchRecentProjects: jest.fn(),
+      fetchTemplates: jest.fn(),
+      setCreationFlowStatus: jest.fn(),
+      navTo: jest.fn(),
+      saveTemplateId: jest.fn(),
+      setCurrentPageMode: jest.fn(),
+    });
+    set(creationFlowStatusState, CreationFlowStatus.NEW_FROM_TEMPLATE);
+    set(featureFlagsState, getDefaultFeatureFlags());
+    set(focusedStorageFolderState, {
+      name: 'Desktop',
+      parent: '/test-folder',
+      writable: true,
+      children: [
+        {
+          name: 'EchoBot-0',
+          type: 'bot',
+          path: 'Desktop/EchoBot-11299',
+          lastModified: 'Wed Apr 22 2020 17:51:07 GMT-0700 (Pacific Daylight Time)',
+          size: 1,
+        },
+      ],
+    });
+  };
+
+  function renderWithRouter(ui, { route = '', history = createHistory(createMemorySource(route)) } = {}) {
+    return {
+      ...render(<LocationProvider history={history}>{ui}</LocationProvider>),
+      history,
+    };
+  }
+
+  beforeEach(() => {
+    createProjectMock.mockReset();
+  });
+
+  it('should render the component', async () => {
+    const {
+      findByText,
+      history: { navigate },
+    } = renderWithRouter(
+      <RecoilRoot initializeState={initRecoilState}>
+        <CreationFlowV2 location={locationMock} />
+      </RecoilRoot>
+    );
+
+    navigate('create/generator-conversational-core');
+    const node = await findByText('OK');
+
+    act(() => {
+      fireEvent.click(node);
+    });
+
+    let expectedLocation = '/test-folder/Desktop';
+    if (process.platform === 'win32') {
+      expectedLocation = '\\test-folder\\Desktop';
+    }
+    expect(createProjectMock).toHaveBeenCalledWith({
+      appLocale: 'en-US',
+      description: '',
+      location: expectedLocation,
+      name: 'generator_conversational_core_0',
+      schemaUrl: '',
+      templateId: 'generator-conversational-core',
+      templateVersion: '',
+      alias: undefined,
+      eTag: undefined,
+      preserveRoot: undefined,
+      qnqKbUrls: undefined,
+      templateDir: undefined,
+      urlSuffix: undefined,
+    });
+  });
+});

Composer/packages/client/package.json

@@ -35,6 +35,7 @@
     "@bfc/ui-plugin-prompts": "*",
     "@bfc/ui-plugin-select-dialog": "*",
     "@bfc/ui-plugin-select-skill-dialog": "*",
+    "@bfc/ui-shared": "*",
     "@botframework-composer/types": "*",
     "@emotion/core": "^10.0.27",
     "@geoffcox/react-splitter": "^2.0.3",
@@ -60,6 +61,7 @@
     "react-dev-utils": "^7.0.3",
     "react-dom": "16.13.1",
     "react-frame-component": "^4.0.2",
+    "react-markdown": "^5.0.3",
     "react-measure": "^2.3.0",
     "react-timeago": "^4.4.0",
     "recoil": "^0.0.13",

Composer/packages/client/src/components/CreationFlow/CreationFlow.tsx

@@ -7,8 +7,9 @@ import Path from 'path';
 import React, { useEffect, useRef, Fragment } from 'react';
 import { RouteComponentProps, Router, navigate } from '@reach/router';
 import { useRecoilValue } from 'recoil';
+import { csharpFeedKey } from '@bfc/shared';
 
-import { CreationFlowStatus } from '../../constants';
+import { CreationFlowStatus, feedDictionary } from '../../constants';
 import {
   dispatcherState,
   creationFlowStatusState,
@@ -17,6 +18,7 @@ import {
   currentProjectIdState,
   userSettingsState,
   filteredTemplatesSelector,
+  featureFlagsState,
 } from '../../recoilModel';
 import Home from '../../pages/home/Home';
 import { useProjectIdCache } from '../../utils/hooks';
@@ -32,14 +34,15 @@ type CreationFlowProps = RouteComponentProps<{}>;
 const CreationFlow: React.FC<CreationFlowProps> = () => {
   const {
     fetchTemplates,
+    fetchTemplatesV2,
+    fetchRecentProjects,
     fetchStorages,
     fetchFolderItemsByPath,
     setCreationFlowStatus,
     createFolder,
     updateCurrentPathForStorage,
     updateFolder,
     saveTemplateId,
-    fetchRecentProjects,
     openProject,
     createNewBot,
     saveProjectAs,
@@ -48,6 +51,7 @@ const CreationFlow: React.FC<CreationFlowProps> = () => {
   } = useRecoilValue(dispatcherState);
 
   const templateProjects = useRecoilValue(filteredTemplatesSelector);
+  const featureFlags = useRecoilValue(featureFlagsState);
   const creationFlowStatus = useRecoilValue(creationFlowStatusState);
   const projectId = useRecoilValue(currentProjectIdState);
   const storages = useRecoilValue(storagesState);
@@ -73,8 +77,8 @@ const CreationFlow: React.FC<CreationFlowProps> = () => {
       await fetchProjectById(cachedProjectId);
     }
     await fetchStorages();
-    fetchTemplates();
     fetchRecentProjects();
+    featureFlags.NEW_CREATION_FLOW.enabled ? fetchTemplatesV2([feedDictionary[csharpFeedKey]]) : fetchTemplates();
   };
 
   useEffect(() => {