Create a Jupyter "PowerToys" extension to enable experimentation of new features

[greazer]

Continuation of discussion that started internally.

[IanMatthewHuff]

Basics:

• Create Microsoft github repository (initially private)
• Creation of extension skeleton
• Trim unneeded fat from extension skeleton
• Basic tweaks to README, SUPPORT, ect...
• Check on requirements for Microsoft Publishing
• CG setup for extension to meet Microsoft requirements
• Basic CI for build / test (loose and unopinionated as possible for now)
• Access to the proposed API for this extension
• One test to test extension activates

Next:

• CI for publishing step (seems like there are a few ways to do this...)
• Bundle Extension
• Prettier / Linting (No specific requirements here, Don added basics to not get file merge changes)
• Seed with an existing example (run groups, Don's cell meta data editing)
• Basic Logging
• Basic Telemetry
• Make sure we have a good name and follow naming principals
• [ ] Third Party notices (Don't believe anything special needed for standard npm usage)
• Extension pack to include this with Jupyter
• Revamp README to set expectations for features
• Add usage to README for notebook run groups
• Unify settings with the same pattern
• Add basics of building / running / tests to contributing
• Settings outline for turning on / off features
• Smoke test the final release VSIX on stable / insiders
• Initial release

One of the trickier bits moving forward is how multiple folks can work on the extension together without treading all over each other's toes. To a large degree I'd like to keep this as open as possible. I think there is a lot of flexibility with how we could all operate and few points where there would actually have to be consensus. Part of the joy of working on separate extensions is to have more flexibility with how we work and code. It should be easy to check out powertoys, work for a while on an idea and bring it back in without major merge conflicts. My personal take is I'd prefer to err on the side of developer autonomy and freedom until actual issues crop up with performance / size / code repeats. Part of the only guidance here would for most features to be aware of what might need to change to bring them into the main Jupyter extension, given that might be the goal for many features here.

Some of the bits that I think do need common consensus would be release pipeline, settings to control which features show up, and telemetry both to make sure that we are good with GDPR and with making sure we know what powertoys are getting the most usage to consider what to bring into the main extension.

[DonJayamanne]

One of the trickier bits moving forward is how multiple folks can work on the extension together without treading all over each other's toes.

Suggestion: we could have each feature as its own folder within the src directory.
E.g. metadata editing can be in src/metadtaediting, kernel manangement in src/kernelmanagement
& the main entry file extension.ts or other will be modified to call one function from each of the individiaul extensions.
This way each sub extension will have their own code base (whether they want to use ioc or not, tests or not), etc
This isn't the best (not production quality), but its a very simple and quick way to prototype, allowing each extension to have its own style and the like, else we end up with styles, IOC, interfces, etc and more madness that comes with large code bases

Sample:
extension.ts

import {activateMetadataEditor} from './src/metadataEditor';
import {activateKernelManager} from './src/kernelManager';
activate(context) {
	if (workspage.configuration.get('enableMetadataEditing')) {
		activateMetadataEditor(context);
	}
	if (workspage.configuration.get('enableKernelManagement')) {
		activateKernelManager(context);
	}
}

The goal is to improve productivity by sacrificing consistent code styles, interfaces etc (that can come later when we want to pull a feature into a seprate extension or into Jupyter). However Package.json will have to be shared, I think thats not a a major concern.

Again, this is just one approach

this is kinda the style i used for my powertoys exension (which too had smaller features in a single extension)

[IanMatthewHuff]

@DonJayamanne Yup, that lines up with what I was thinking. I want freedom over "production ready" for this extension. IOC tests ect as you want. Just with the caveat if anyone gets too crazy they might have more work if that code needs to be brought into the main extension.

[DonJayamanne]

Basic Logging

Can we use something as simple as console.log or just write to the output panel. I feel the logging in our extension is a little too complex (I believe Python extensoin have also simplified it since)

Telemetry

I think when we implement this, we'll have to use something that's strongly typed (so we don't leak PII).
Is this a good place to start using the same declarative style as VS Code so that its self documenting & the tools can extract thsi as well. Perhaps we can do telemetry as a later step. I.e. after the MVP.
Just a thought, i.e. not having telemtry isn't the end of the world, at least not for the first couple of releases.

[IanMatthewHuff]

Yeah, telemetry was just something that we might have to have some common ground on, if only because this is MS published and GDPR. Need to make sure we are safe / secure even if we are all doing our own things. Given that we are starting from scratch seems like a good idea to start with what VS Code has as well.

Agree that MVP could be without, but telemetry is something that I'd want to come along sooner rather than later, just because I see looking at that as being clutch to tell what features might need to get moved up from PowerToys into main extension.

Tagging @rchiodo and @amunger to make sure they can drop thoughts as well on the PowerToys extension.

[IanMatthewHuff]

@DonJayamanne Since you've just been doing this work for the other extensions, do you have a landing page for the "how to get your extension MS publishing ready (component governance and all that)" docs? Want to make sure I get my i's dotted and t's crossed. Can just look at what the extensions are doing, but I'd like to look at the full picture myself.

[DonJayamanne]

your extension MS publishing ready (component governance and all that)" docs?

Yes, its here https://github.com/microsoft/vscode-engineering/wiki/Extensions-Build-and-Publish
Just create the release & pre-release yml files and thats it. That will take care of component governance & the likes.
You could just copy our pipelines. and you'll need to create them manually in here https://dev.azure.com/monacotools/Monaco/_build?definitionScope=%5CExtensions%5Cms-toolsai (click on the blue button New Pipeline on the top right).

I can show this, takes on 2-3 minutes to show this.

[rzhao271]

I'm wondering what some verification steps are for this issue, or whether there should be a TPI instead?

[rchiodo]

Test plan would probably be good.

Verification steps:

1. Install power toys, make sure it's available.

Anything else we're likely not going to fix.

[rzhao271]

I can install it, but I can't seem to run any commands. I tried running the following command in an ipynb file:

[rchiodo]

That works for me? @rzhao271 where were you testing this?

[amunger]

The kernel explorer and execution groups seem to be working. but that notebook: edit kernelspec fails for me with the same message

[rchiodo]

Oh I ran a different command. Focus on Kernels view. That just sounds like that command isn't implemented.

[rchiodo]

I entered a separate issue for that:
microsoft/vscode-jupyter-powertoys#34