[BUG] Copy-PnPFile only copies folders even if file is specified

[pbjorklund]

Reporting an Issue or Missing Feature

https://docs.microsoft.com/en-us/powershell/module/sharepoint-pnp/copy-pnpfile?view=sharepoint-ps states that

PS:>Copy-PnPFile -ServerRelativeUrl "/sites/project/Shared Documents/company.docx" -TargetServerRelativeLibrary "/sites/otherproject/Shared Documents"

will

Copies a file named company.docx located in a document library called Shared Documents in the site collection project to the Shared Documents library in the site collection otherproject. If a file named company.docx already exists, it won't perform the copy.

In https://github.com/pnp/powershell/blob/dev/src/Commands/Files/CopyFile.cs it seems that only folder is passed along CopyToOtherSiteCollection(sourceUri, targetUri); and not the actual ref to the file meaning that the folder will always be passed?

This was changed in PnP-powershell in commit pnp/PnP-PowerShell@7f87352#diff-2a3f10f735507b7333a80eabec28831f5de366d962ad4637cff0326c3c80f718

 Uri sourceUri = new Uri(currentContextUri, sourceFolder);

vs earlier SourceUrl.

This issue in the old repo says the same thing. pnp/PnP-PowerShell#2796

With this change my old script that copied files one by one (need to process each file individually so we at the same time can download and save metadata to replicate file by file in target location)

Expected behavior

When running

$fileRef = "/sites/Investment/SmallerTEst/General/3 test.docx"
$newTargetFolderUrl = "/sites/TargetTest2/Delade Dokument/General"
Copy-PnPFile -SourceUrl $fileRef -TargetUrl $newTargetFolderUrl -OverwriteIfAlreadyExists -Force

This should copy the file 3 test.docx to /sites/TargetTest2/Delade Dokument/General/3 test.docx

Actual behavior

This copies the whole "General" folder into /sites/TargetTest2/Delade Dokument/General/General (notice double General)

Steps to reproduce behavior

See above

What is the version of the Cmdlet module you are running?

ModuleType Version    PreRelease Name                                PSEdition ExportedCommands
---------- -------    ---------- ----                                --------- ----------------
Manifest   0.3.39     nightly    PnP.PowerShell                      Core,Desk {Add-PnPClientSidePage, Add-PnPClientSidePageSection, Add-PnPCli…

Which operating system/environment are you running PnP PowerShell on?

• Windows
• Linux
• MacOS
• Azure Cloud Shell
• Azure Functions
• Other : please specify

[erwinvanhunen]

Hej Patrik,

The cmdlet used the CSOM version of a copyjob behind the scenes but it didn't specify a file name if provided. I have updated both Copy-PnPFile and Move-PnPFile cmdlets to use an updated version of the code to copy the files. I also updated the documentation at https://pnp.github.io/powershell accordingly. Some of the parameters have been renamed, but aliases have been provided where appropriate.

[pbjorklund]

Thanks for the quick response @erwinvanhunen.

It seems the sleep added in RestHelper checking on jobstatus is a bit over-zealous. Right now all copies with the nightly build take at least 5-10 sec compared to the avg of 0-2 with the legacy PnP-PowerShell. This would bump my total runtime of the script to over 14 full days.

How about perhaps either
a) use a backoff strategy for the sleeping so we can move on from quick copies ... quickly.
b) separate the invocation of the CopyJob from the completion monitoring. Fire off the CopyJobs as quickly as the API will let us and then run the followups in a separate thread which can report back errors which we can handle separately? (In my case I will log the failed copies and handle them in a batch post-run). A naive implementation would be to set up a queue to where we put the invocations and then process the response with a delay, if the processed item is still not finished we can re-queue it at the end and it will be picked up on the next passthrough?
It would then be
Create A,B,C, Receive confirmation for A, Receive confirmation for B, Create D,E, Receive confirmation for C etc etc. I do not know if there is any server-side limit on how many concurrent jobs you can have running without getting throttled, perhaps you have some insight?

I guess A is the easiest fix that would let me actually use the cmdlets again.

Thoughts? Let me know if you are pressed for time and I could take the time to set up a dev env for pnp-powershell to create a simple PR (it will just take me some time since I never develop cmdlets :))

Ref commit: 0ed8d91#comments is

[erwinvanhunen]

Fair feedback. We updated the cmdlets. Both Copy-PnPFile and Move-PnPFile now support a -NoWait parameter which will return the cmdlet immediately after the first progress poll. The cmdlet will then return a job object which can be used in combination with Receive-PnPCopyMoveJobStatus to retrieve the job status. See the documentation which will be updated tomorrow after the nightly build (or check the repo now). It will be available in build 1.1.3-nightly.