GitHub Copilot

[wilberforce]

Have you seen this?

It uses AI to predict the code you are about to write, and then you auto-complete with the tab key... it's pretty cool.
https://marketplace.visualstudio.com/items?itemName=GitHub.copilot

Sign up for the waitlist

[phoddie]

Very spiffy. The docs say it is trained against a broad library of code. For JavaScript auto-complete that would be relevant. For specific embedded API patterns, I wonder if it has enough context to make a useful (and not distractingly irrelevant) suggestion. Any insight into that?

[wilberforce]

Yes it does - here is an example using manifest.json - it picks up context from the current file too, predictions within _ _

"modules": {
		"*": [
			"./main"
		],
		"middleware/*": "$(MODULES)/network/http/middleware/*",
		"model":"site/model"
		},
	"preload": [
		"model",
                _"middleware/static",_
	],

if I accept that and over type static with w it suggests websocket and then suggests the next line:
"middleware/http" with I update to staticzip

[phoddie]

If it isn't too much trouble, would you be willing to share a short screen recording that shows this?

[phoddie]

Impressive.

[phoddie]

This blog post from the Software Freedom Conservancy raises some very interesting questions about GitHub Copilot. In particular, it does seem possible that Copilot would recommend code which has licensing requirements without providing a notification of that requirement. That isn't a problem for experimentation and learning, but it is for any kind of distribution.

Moddable has always been diligent about making sure we know the origin and license requirements of every line of code in the Moddable SDK. Even code permissively licensed with Creative Commons Attribution license requires a notice. We require a Contributor License Agreement for contributions to help us document the origin of our code. We make this effort so that everyone – both our commercial clients and our community developers – using the Moddable SDK can be aware of their obligations. Use of Copilot seems to have the potential to make that more difficult. It does seem like Microsoft/GitHub has the ability to remedy that, since they do know the origin of all the fragments they trained on and which contributed to a given a suggestion. It will be interesting to see how this evolves.

[wilberforce]

Here you go:

[wilberforce]

Here is a poco example...

[phoddie]

The auto-complete style contextual changes are great, and don't raise obvious copyright / license concerns. The bigger the suggestion, the more chance for trouble there. It looks like you don't have to do anything to use this. No new work would be great. ;)

[wilberforce]

It looks like you don't have to do anything to use this

You just need to sign up for the beta program and wait. Then you enable the extension in vscode and link to your github account

[kairos0ne]

Regarding the privacy concerns:

[kairos0ne]

My personal opinion is: if you dont want people to see what you write, don't put it on the internet. Same applies for generative AI

[phoddie]

I think there's a valid concern for commercial developers. If an LLM proposes code that has IP obligations inherited from the sources it scraped, a developer incorporating that code into their product could be unknowingly creating IP entanglements for their product's code. That's a dangerous mess.

[kairos0ne]

Im not so sure about that, IP is only considered IP if its 60% original if you steal 60% of someones code and use it in your project, you deserve to go to prison. thats straight up theft. But if a ai model uses someones code base to suggest and adapted function thats not theft. Thats not even IP theft and IP law requires that at a minimum its a 60% a copy.

[phoddie]

The IP attorneys I've worked with don't cite a percentage (here's one explanation of why that probably makes sense). But assume there is a percentage: how do you know the LLM changed enough of the code to fall below the limit? The LLM doesn't show you the sources it learned from, so you cannot evaluate that yourself.

The risk isn't prison. The corporation won't go to prison. They may have to pay significant license fees and damages after-the-fact. if the offending code is GPL, there may be an obligation to release the entire product code base to the public under GPL.

Setting all that aside, the code the LLM generates might be entirely free of copyright infringement. But, it could still infringe on a patent. When open source code is used properly, by following all the rules of the license, there is often a no-cost license granted to use any patents that the licensor holds related to using that code. But, if you are not following the open source rules because you don't know the code's provenance, there is no patent license grant either.

Everyone has to evaluate the level of risk they are comfortable with. LLLMs provide code that appears to be unique but with no traceable provenance. It can be a great way to learn. It is not a concern for a hobby project. It is a legitimate concern for corporations that want to have a clear understanding of the ownership and obligations associated with the code they are using.

[kairos0ne]

I'll admit the figure I got there was given to me by the legal team at the advertising firm I worked for however I think was most likely in reference to creative assets. I'm actually not sure about code. However my point is the context of the new code preseeded by question and given a adapted outcome that suites example supplied is not IP theft in my opinion. There are too many variables there to say for sure this is copying proprietary code and thus no definitive way to prosecute it. At least my thoughts anyway, it will be interesting to see and I believe their is a strong case for the privacy concerns and only time will tell. We do this all the time as humans and in human behaviour - in fact newton famously said - I could only see so far because I was standing on the shoulders of giants.

[phoddie]

Agreed - there is a continuum between verbatim copying and creating something new based on insights from an accumulation of knowledge. The LLMs typically provide no information on the the sources used, so there's no practical way to know where a given piece of generated code is along that continuum. Risk-averse organizations will be very cautious in using such code. Of course, an LLM could be trained on information known to be free of IP obligations. That would lower the risk considerably. But, it doesn't seem to be what GitHub Copilot is doing.

[dtex]

I see I'm very late to this discussion, but I do have an update from MS to share. The gist is "...if a third party sues a commercial customer for copyright infringement for using Microsoft’s Copilots or the output they generate, we will defend the customer and pay the amount of any adverse judgments or settlements that result from the lawsuit...".

As an aside, it was on an embedded JS project that Copilot won me over. I typed CONST PCA9685 = and before I could type the opening bracket, it built out an object with all the memory locations for the device. I was all set to download documentation and manually copy-paste everything over, but like magic, it was already there.

I was stunned because I knew there was no equivalent object in JavaScript anywhere on Github. Still, I searched to see if I was wrong and if someone else was doing a PCA9685 implementation in JS. There was nothing to be found apart from some work I had done myself a few years earlier for Johnny-Five. That implementation only had a handful of memory addresses, whereas Copilot's code had them all. It had to have ported code from an implementation in another language into JS 🤯

[phoddie]

@dtex - that's a remarkable commitment from Microsoft. It'll be interesting to understand how they intend to track the provenance of code to support it. I don't imagine most programmers note in their code "this fragment based on GitHub Copilot." While the commitment for copyright appears strong, they are silent on the topic of patents. For small snippets of code, that is a low risk. But for larger chunks of code in heavily patented domains, it sees like a valid concern.

Great story on the PCA9685. That seems like a very practical use of "AI" -- basically a smart, contextual look-up of reference data. And that surely doesn't raise any IP concerns. ;)

[tve]

I've had a similar experience with a different device, it auto-completed the registers and bit fields just like that.

I've been using copilot for what now seems like a long time, must be at least a year maybe two, can't remember. It's not the kind of thing I like, sending my stuff to some big co's servers is against almost everything I do. I started using it to try it out 'cause I thought I should know about it. I just couldn't stop! When I'm editing code not in visual studio, for example the Node-RED editor, I often stare at my incomplete line in expectation of copilot to fill in the rest. Then the light-bulb goes on: "duh, I guess I'm gonna have to look this up the hard way". Of course it's not always right but it either jogs my memory or intellisense will complain...

I don't use copilot to produce large chunks of code, like writing comments for a function and then expecting the function body to show up. I use it as super-smart auto-complete. Sometimes I use larger code fragments for idiomatic stuff like APIs that have iterators and one has to check for error conditions and such. The code fragments often remind me that I need to check for corner cases I'd otherwise forget.