Mover and replot enhancements

This release focusses on bringing PlotoMover into the PlotoSpawnerConfig.json aswell and make it startable upon launch. Until now, it was necessary to start another PowerShell Session to fire up PlotoMover. Alongside that, it now also supports MountPoints and Network drives! It also brings some new capabilities within the replot mechanism. Ontop of all, it includes the new logo and new illustrations for the readme.

Added

• Support MountPoints as OutDrives; Ploto now support the use of MountPoints as outdrives and final drives for mover.
• Support of Network Drives as OutDrives; Ploto now support the use of Network drives as outdrives and final drives for mover
• "ReplotPlotsOlderThan": "20.6.2021 in config parameter. Allows you to define which plots shall be replotted and which not by specifying a date as seen above. This is a mandatory param if you want to replot, as its the only way to determine which plots shall be replotted
• Replot mechanism now picks those drives that have final plots on it for replotting. If there is no drive that has plots to replot, its chooses the best suited OutDrive by free space. If there is none, it aborts.
• Mover is now able to handle several final drives. If there are more than 1 final drives, it picks the one that has no BITS Transfers in progress. If there are non with BITS Transfer in progress, it selects the one with the most available space.

Changed

• PlotoMover cannot be started anymore with params. It uses definable Drives and Paths in config. closes #98
• Changed some outputs to proper meaning and source (function)
• Mover now solely relies on BITS to transfer plots
• Mover now transfer plots async and waits 30 minutes between each move of a plot
• mover ignores plotsolderthan to be moved if replot enabled
• reduced sleep of Watchdog intervall to check for jobs to 1 minute instead of 5

Fixed

• An issue where -p and -f were not ommitted correctly. closes #93
• A couple of potential bugs

How to update

1. Launch a PowerShell session

2. cd to downloaded source ( for example cd C:\Users\Tydeno\desktop\ploto )

3. Launch Install-Ploto.ps1 (As Admin)
   3.1 Update Module
   3.2 set a new config using Install-Ploto, or add the properties manually to your cfg, see beloe
   3.3 Let install script copy the config

4. In PowerShell launch Start-PlotoSpawns

If you use different Users for administrative Tasks and normal usage of your PC (restricted normal user under which context you run chia), you have to abort the script after you defined your config (Where its asks if you want to get it into production)
The values you defined are then saved in downloaded config in the source folder. Afterwards, simply copy the config from sourcefolder to
C:\Users\Tydeno.chia\mainnet\config

New properties in config

  "EnableMover": "",
  "PathsToMovePlotsTo": "",
 "ReplotPlotsOlderThan": "",

Change to DriveLetters and Paths

This release focusses on moving from the former Drive Denoms to using distinct driveletters and paths for your Temp, T2 and OutDrives.

Added

• Support for Ramdisks closes #88

Changed

• Now using Driveletters and or UNC paths instead of a denominator to specify your drives closes #94

Sample disk config:

   "DiskConfig":  [
                       {
                           "TempDrives":  "B:,D:,E:",
                           "Temp2Drives":  "A:,Y:,Z:",
                           "OutDrives":  "H:\\Plots,K:\\Archiv,I:,P:,Q:,R:,S:,T:,X:",
                           "EnableT2":  "true",
                           "ReplotDrives":  ""
                       }
                   ],

How to update

1. Make sure you flush/move your existing logs in your plotter folder (C:\Users\Tydeno.chia\mainnet\plotter)

2. Launch a PowerShell session

3. cd to downloaded source ( for example cd C:\Users\Tydeno\desktop\ploto )

4. Launch Install-Ploto.ps1 (As Admin)
   4.1 Update Module
   4.2 set a new config using Install-Ploto (Introduces some cfg changes)
   4.3 Let install script copy the config

5. In PowerShell launch Start-PlotoSpawns

If you use different Users for administrative Tasks and normal usage of your PC (restricted normal user under which context you run chia), you have to abort the script after you defined your config (Where its asks if you want to get it into production)
The values you defined are then saved in downloaded config in the source folder. Afterwards, simply copy the config from sourcefolder to
C:\Users\Tydeno.chia\mainnet\config

Support for madmax/stotik plotters

Adds

• Support for madMAx43v3r and stotiks blazing fast custom plotter. closes #86
• A few improvements on the installer
  • shows selected drives by denom in cfg mode
• A config handout when Ploto Spawns
• A cleaner config
• Notifications when Remove-AbortedPlotoJObs detects an aborted Job
• Add Size of plotfiles on t2 drive in Get-PlotoJobs

Changes

• Ommits -p and -f in output of Get-PlotoJobs and shows "YourKeys" instead for those params.

Fixes

• Completed Job is considered "Aborted" closes #90
• When T2 is same as TempDrive error on Remove-AbortedJobs closes #89
• and various more

How to update (make sure you follow that procedure)

1. Make sure you flush/move your existing logs in your plotter folder (C:\Users\Tydeno.chia\mainnet\plotter)
2. Launch a PowerShell session
3. cd to downloaded source ( for example cd C:\Users\Tydeno\desktop\ploto )
4. Launch Install-Ploto.ps1 (As Admin)
   4.1 Update Module
   4.2 set a new config using Install-Ploto (Introduces some cfg changes)
   4.3. Let install script copy the config
5. In PowerShell launch Start-PlotoSpawns

If you use different Users for administrative Tasks and normal usage of your PC (restricted normal user under which context you run chia), you have to abort the script after you defined your config (Where its asks if you want to get it into production)
The values you defined are then saved in downloaded config in the source folder. Afterwards, simply copy the config from sourcefolder to
C:\Users\Tydeno.chia\mainnet\config

How to use Stotik Plotter

If you want to use Stotik Plotter, make sure you set PlotterUsed to "Stotik" and that you specify the fullpath to the .exe

replot and installer

Added

• An Installer script that correctly installs/updates Ploto Module and guides trough the setup of the config.
  Also copies config to where it needs to be.
• Ability to launch Ploto commands globally without need to import first (Thanks to installer)
• Replotting capabilities; A watchdog that looks for jobs that are about to enter phase 4 and has IsReplot set to true. Then deletes a plot on defined RepotDrive closes #75
• Ability to Define ReplotDriveDenom

Fixed

• A bug that did not set -f param when usin a P2Singleton to plot

portable pool plot support and NVME first

Added

• Ability to pass -c param to Chia plots create
  Be advised, this is a beta. It is based on the demo we saw from Mariano at the pool operators call. Might not be working correctly yet. closes #78
• Ploto now checks for NVMEs to spawn the jobs on. If an NVME is found and defined as a TempDrive, it will be used first to spawn a plot on. closes #74

Changed

• Added an additional check for OutDrives, if they have JobsinProgress.

Removed

• ProgressBar for PlotoMover using Bits

Fy on launch/t2 removal

Added

• Ability to start PlotoFy when Spawner is launched
• Remove-AbortedJobs/Stop-PlotoJobs now also does clean up of temporary t2 files
• PlotoSpawner now analyzes the output of the function that loads the config for known errors. If an error is known (missing ", or , and or using \ instead of /) a hint is displayed
• More resilience for Remove-AbortedJobs/Stop-PlotoJobs trough less generalization
• Some more output for console

Changed

• Removed MaxParallelJobsInPhase1OnSameDisk. Its a useless param.
• Adjusted some Write-Host/Verbose outputs all around Ploto (except mover)
• Split of "plots create" from Get-PlotoJobs output
• JobLob generated by chia GUI or CLI Plotter are now ignored by Ploto (No need to delete)

Fixed

• fixed a bug that prevented spawning (MaxJobsOnDiskInP1 was calculated wrong)
• a bug where Get-PlotoOutDrives could not get the right amount of plots to hold
• surpressed error fom Stop-PlotoJob is its the known one

dynamic staggering/start early

Added

• Ability to define pool & farmer key
• Ability to define max parallel jobs in phase 1 per each disk #4
• Ability to define max parallel jobs in phase 1 on all disks #4
• Ability to launch a new plot when a given phase is completed (EarlyStart, set to 3 advised. So when a job completed phase 3, the next can be started) #4
• PlotJob Phase on Get-PlotoDisks (Temp/Out/T2) #59

Changed

• Waaay less noisy verbose. Finally is usable to trace what Ploto is doing #59
• Merged Spawner and Alert config in one #53
• The most appropriate OutDrive is now specified by the least amount of jobs it has #56

Fixed

• Improper Job Count when Alarms enabled #55
• Some bugfixes and refactoring

-2 drives and obey

Added

• Ability to use -"2" param for chia.exe (temp2 drives) #33
  Assumes dedicated temp2 drives. A raw implementation.
  If a t2 drive is available it uses it, if not, it tries to launch a job without it.

• Ability to define Window Style of spawned plot jobs (chia.exe)
  Cause I closed all parallel plots today trough fatfingering.

  Accepts values:

  • normal
  • hidden
  • maximized
  • minimized

Changed

• Switched from WMI to CIM. Should now work for a broader range of PowerShell versions without adjustments and also more reliable (Could not get TempDrive issue) #19

Fixed

• An issue where Ploto would not want to stop generate plots (ignoring "InputAmountToSpawn" param) #48
• Some other issues

Hotfix and move to configFile

Changed

• The core logic has been refactored which makes Ploto much more understandable, readable and ultimately; more reliable
  * It does not wait anymore for a given TempDrive to become plotable. It checks all TempDrives peridodically upon definable Intervall (default 5 min)
• Ploto now uses a ConfigFile instead of Parameters to control spawning of jobs (Yes, you can edit it on the fly, meaning when Ploto is running)
• Moved some Write-Hosts to Write-Verbose

Added

• Ploto now checks for abortedJobs and deletes them upon every iteration
• Some more detailled -verbose logging

Fixed

• #45
• #47
• And a myriad of undocumented issues

Known Bugs

• #48
  -InputAmountToSpawn is blatantly ignored for now. Why? I focus on successfully spawning jobs as fast as possible rather than complying with that param. Overcommit should not possible anyway due to checking OutDrives. And also If a jobs fills a drive, you may copy/move stuff from on that drive and te job can continue. Will be fixed though!

PlotoFy Status Summary Reports

Added

• Status Summary Reports in PlotoFy

Setup Discord Alerts for periodical summary of jobs completed and in progress

To configure the behaviour of periodical summary reports in discord, we also use the PlotoAlertConfig.json stored in C:\Users\YourUserName.chia\mainnet\config.

1. Edit PlotoAlertConfig.json
2. Set WebhookURL (Your Discord Servers Webhook). You may use the same or a different WebhookURL than what PlotoSpawner Alerts uses. This allows you to separate channels in Discord for various alerts. I use this separate for each plotter I have:
   
3. Set the intervall upon which you would like to receive the summary. This also affects the period PlotoFy uses to check for events. For example: If 1 is specified (1hr), PlotoFy will each hour lookup jobs that were copleted in the last hours, and all active jobs in progress and wrap this in a notification.
   If there are no jobs completed within the period, PlotoFy will only send the notification for the jobs in progress. If there are no completed and no jobs in progress, PlotoFy tells you with a notification, that Ploto seems not running.
4. Set the Path to Ploto Module (where its stored right now)
5. Make sure you copy/move the edited .json to the folder: C:\Users\YourUserName.chia\mainnet\config. If its not there, it wont work.
6. Open a new PowerShell Session.
7. Import-Module "C:\Users\me\desktop\Ploto\Ploto.psm1" (make sure you change the path to Ploto.psm1 to reflect your situation)
8. Launch PlotoFy using the cmd below:

Start-PlotoFy

As PlotoFy launched a backgroundJob that continously calls "Request-PlotoFySummaryReport", for almost infinity, you may wont to stop that job at some point. To do this, you can use the following commands:

1. Get-Job to get the current running PowerShell Jobs:

Get-Job

Id     Name            PSJobTypeName   State         HasMoreData     Location             Command
--     ----            -------------   -----         -----------     --------             -------
1      Job1            BackgroundJob   Running       True            localhost            ...

2. Stop the Job, using Stop-Job

Stop-Job -Id 1

3. And to clean up, we need to deleted the stopped job.

Get-Job | where-object {$_.State -eq "Stopped"} | Remove-Job