Releases: mj-xmr/SolOptXMR
Security fixes after a review (hashes)
This release only replaces fixed versions (tags) of the 3rd party software (monero-core
, xmrig
& p2pool
) with hashes of the same versions for security reasons.
What's Changed
- Doc: Star history by @mj-xmr in #145
- Using hashes of 3rd party software, rather than tags by @mj-xmr in #146
Full Changelog: v0.4.0...v0.4.1
First public release
v0.4.0 Disable donation (#144)
v0.4-rc2 Full automation (Milestone 4 of 4)
Full automation (Milestone 4 of 4)
Brief
This version brings full automation (after an initial setup investment) and a lot of stability via fast corner case testing, that allowed me to fix prevailing bugs and enables further, risky experiments.
Major changes:
- New console visuals: fast, dependency free and SSH-friendly.
- Testing corner cases: major stability improvement, that now allows to experiment freely with risky improvements, like a better optimization algorithm.
- Automation documentation and scripts: automating almost all that could be automated, including compilation of external tools:
p2pool
,xmrig
andmonero
daemon itself. - Computer schedule: visually showing the plan and producing commands to remotely control the mining rigs.
- Presenting multiple alternative solutions to choose from, to better match your daily habits.
- Wind turbines along the solar input and example config.
- LiFePO4 (dis)charge profiles and example config.
- Tons of other new documentation: I'm the most fond of the additions to safety.
- Switching off Inverter can now be modelled: example config.
- Temperature control script: Manipulates the CPU governor to reduce the maximal CPU frequency down to a level, that enables the CPU to cool off, after detecting its overheat.
Change log
- Opti: No early stop until 1st solution is found by @mj-xmr in #84
- Bugfix: no solution at initial overvoltage by @mj-xmr in #85
- Initial Wake-On-Lan by @mj-xmr in #86
- Battery one-liner ASCII plot & proper 'at' scheduling by @mj-xmr in #87
- Habits: added li-ion vacuum cleaner by @mj-xmr in #88
- Doc: Checkout the latest tag by @mj-xmr in #89
- Timeout for ssh by @mj-xmr in #90
- Doc archive voltages by @mj-xmr in #91
- CLI: Charge & discharge status for voltage input by @mj-xmr in #92
- pip: relaxing conditions by @mj-xmr in #93
- Lifepo4 based on docs by @mj-xmr in #94
- pip: add matplotlib by @mj-xmr in #95
- Pip requirements by @mj-xmr in #96
- Bump tsqsim by @mj-xmr in #97
- Ascii plots in lib by @mj-xmr in #98
- Python: testresources by @mj-xmr in #99
- Ascii plots for other series by @mj-xmr in #100
- Bump tsqsim with Ascii fluent API by @mj-xmr in #101
- Ascii plots colors sol distrib by @mj-xmr in #102
- Fixing master by @mj-xmr in #103
- Doc: OCR screenshots by @mj-xmr in #104
- Disabled slow cooker habit & disp input as yellow by @mj-xmr in #105
- Mac ci2 by @mj-xmr in #106
- Mac ci: disabled Matrix by @mj-xmr in #107
- Doc by @mj-xmr in #108
- dev by @mj-xmr in #110
- Comp schedule tests and fixes by @mj-xmr in #111
- Bump tsq by @mj-xmr in #112
- Dev by @mj-xmr in #113
- UX fixes by @mj-xmr in #114
- Wind input by @mj-xmr in #115
- Dev by @mj-xmr in #116
- Fix number of iterations by @mj-xmr in #117
- Tests by @mj-xmr in #118
- Algo: stability & 3 line battery plot by @mj-xmr in #120
- Refactor: String forward declarations by @mj-xmr in #121
- Algo: more tests and stabilisation fixes by @mj-xmr in #123
- Stability, visuals & multiple solutions by @mj-xmr in #124
- New undercharge tests and algo simplifications by @mj-xmr in #125
- At script and doc by @mj-xmr in #127
- Visuals by @mj-xmr in #128
- Inverter schedule by @mj-xmr in #129
- Doc by @mj-xmr in #130
- Doc by @mj-xmr in #132
- Cfg: max and cooled temp in system.cfg by @mj-xmr in #131
- Default max battery charge percentage of 90% by @mj-xmr in #133
- Doc safety by @mj-xmr in #134
- Update testing.md by @mj-xmr in #135
- Doc: use cases by @mj-xmr in #136
- Corrected discharge per hour by @mj-xmr in #137
- Fix hashrate sum by @endorxmr in #138
- Doc: automation by @mj-xmr in #139
- Update deps-pull.sh by @mj-xmr in #141
- Stability: replace Array with Vector for now by @mj-xmr in #140
- CI: Fixing OSX with a missing hdf5 library by @mj-xmr in #142
- Auto scripts by @mj-xmr in #143
Full Changelog: v0.2-beta...v0.4-rc2
v0.4-rc1 Full automation (Milestone 4 of 4)
Full automation (Milestone 4 of 4)
Brief
This version brings full automation (after an initial setup investment) and a lot of stability via fast corner case testing, that allowed me to fix prevailing bugs and enables further, risky experiments.
Major changes:
- New console visuals: fast, dependency free and SSH-friendly.
- Testing corner cases: major stability improvement, that now allows to experiment freely with risky improvements, like a better optimization algorithm.
- Automation documentation and scripts: automating almost all that could be automated, including compilation of external tools:
p2pool
,xmrig
andmonero
daemon itself. - Computer schedule: visually showing the plan and producing commands to remotely control the mining rigs.
- Presenting multiple alternative solutions to choose from, to better match your daily habits.
- Wind turbines along the solar input and example config.
- LiFePO4 (dis)charge profiles and example config.
- Tons of other new documentation: I'm the most fond of the additions to safety.
- Switching off Inverter can now be modelled: example config.
- Temperature control script: Manipulates the CPU governor to reduce the maximal CPU frequency down to a level, that enables the CPU to cool off, after detecting its overheat.
Change log
- Opti: No early stop until 1st solution is found by @mj-xmr in #84
- Bugfix: no solution at initial overvoltage by @mj-xmr in #85
- Initial Wake-On-Lan by @mj-xmr in #86
- Battery one-liner ASCII plot & proper 'at' scheduling by @mj-xmr in #87
- Habits: added li-ion vacuum cleaner by @mj-xmr in #88
- Doc: Checkout the latest tag by @mj-xmr in #89
- Timeout for ssh by @mj-xmr in #90
- Doc archive voltages by @mj-xmr in #91
- CLI: Charge & discharge status for voltage input by @mj-xmr in #92
- pip: relaxing conditions by @mj-xmr in #93
- Lifepo4 based on docs by @mj-xmr in #94
- pip: add matplotlib by @mj-xmr in #95
- Pip requirements by @mj-xmr in #96
- Bump tsqsim by @mj-xmr in #97
- Ascii plots in lib by @mj-xmr in #98
- Python: testresources by @mj-xmr in #99
- Ascii plots for other series by @mj-xmr in #100
- Bump tsqsim with Ascii fluent API by @mj-xmr in #101
- Ascii plots colors sol distrib by @mj-xmr in #102
- Fixing master by @mj-xmr in #103
- Doc: OCR screenshots by @mj-xmr in #104
- Disabled slow cooker habit & disp input as yellow by @mj-xmr in #105
- Mac ci2 by @mj-xmr in #106
- Mac ci: disabled Matrix by @mj-xmr in #107
- Doc by @mj-xmr in #108
- dev by @mj-xmr in #110
- Comp schedule tests and fixes by @mj-xmr in #111
- Bump tsq by @mj-xmr in #112
- Dev by @mj-xmr in #113
- UX fixes by @mj-xmr in #114
- Wind input by @mj-xmr in #115
- Dev by @mj-xmr in #116
- Fix number of iterations by @mj-xmr in #117
- Tests by @mj-xmr in #118
- Algo: stability & 3 line battery plot by @mj-xmr in #120
- Refactor: String forward declarations by @mj-xmr in #121
- Algo: more tests and stabilisation fixes by @mj-xmr in #123
- Stability, visuals & multiple solutions by @mj-xmr in #124
- New undercharge tests and algo simplifications by @mj-xmr in #125
- At script and doc by @mj-xmr in #127
- Visuals by @mj-xmr in #128
- Inverter schedule by @mj-xmr in #129
- Doc by @mj-xmr in #130
- Doc by @mj-xmr in #132
- Cfg: max and cooled temp in system.cfg by @mj-xmr in #131
- Default max battery charge percentage of 90% by @mj-xmr in #133
- Doc safety by @mj-xmr in #134
- Update testing.md by @mj-xmr in #135
- Doc: use cases by @mj-xmr in #136
- Corrected discharge per hour by @mj-xmr in #137
- Fix hashrate sum by @endorxmr in #138
- Doc: automation by @mj-xmr in #139
- Update deps-pull.sh by @mj-xmr in #141
- Stability: replace Array with Vector for now by @mj-xmr in #140
- CI: Fixing OSX with a missing hdf5 library by @mj-xmr in #142
Full Changelog: v0.2-beta...v0.4-rc1
Stability fix - no solutions during initial overvoltage & early stop limited
What's Changed
- Opti: No early stop until 1st solution is found by @mj-xmr in #84
- Bugfix: no solution at initial overvoltage by @mj-xmr in #85
Full Changelog: v0.2-beta...v0.3-beta
v0.2-beta Measurements; OCR; Stability fixes
Intro
This pre-release is one step before being able to automate the act of switching the mining on and off. It deals with supplying all the necessary measurements, yet still without the "Home Assistant" hardware, but rather the bare metal ones. It automates the part of acquiring the initial battery state - the only part that had to be supplied by hand to perform the simulation. Moreover, by adding a few more parameters to the battery.json
, described in the configuration documentation, the required conversions from voltage to charge percentage are being performed.
Major changes
Charging and discharging voltage modelling
Thanks to the voltage modelling, we can now translate the voltage readouts to battery charge in percent, yielding a linear calculation, ultimately being able to translate voltage to Ah (Ampere-hours): a unit of energy capacity, that's much simpler for calculations.
As described in the main documentation, the voltage input can be triggered with:
./soloptxmr.py --battery-charge-v 12.3 # Set the voltage of your battery to be converted to its current charge
OCR
Thanks to an incorporation of GasPumpOCR project, it's possible to reuse the existing examples of OCR modules or to write your individual ones. The right parameters of such a module can be learned with an interactive script called playground.py
of the GasPumpOCR project (example). See the new documentation. The OCR module can be generalized to just recognize certain rectangles, that could represent the battery charge in % already, rather than the raw voltages, like here . There's an example that does it as well.
Using the OCR module is done via:
./soloptxmr.py --battery-charge-ocr # Use image recognition to read the current battery voltage
Image capture
Closely related to OCR (and documented there) is a similarly interfaced image capture system, allowing you to write your own image capture models.
Temperature readouts
These will come very handy whenever there's a suspicion, that a given rig is overheating. In such an event the controlling PC will reduce the mining effort of the mining rig to let it cool down.
Optimization algorithm stability fixes
I noticed, that the algorithm behaved in an unstable way for a few reasons not worth mentioning. This part is simply being left in a working ad-hoc state for now, while I focus on the integration of many individual systems, like the ones mentioned above. There will come a time soon, when I'll focus on making the optimization algorithm work much better.
Full changes
What's Changed
- Random seed by @mj-xmr in #52
- Bump pillow from 9.1.0 to 9.1.1 by @dependabot in #53
- pip fallback by @mj-xmr in #54
- Config volatile by @mj-xmr in #56
- Mac fix cfg by @mj-xmr in #57
- Fixed dynamic linking of libopt by @mj-xmr in #58
- Image recognition by @mj-xmr in #59
- Discharge rates by @mj-xmr in #60
- Discharge rates in json by @mj-xmr in #61
- Charging above the minimum before mining by @mj-xmr in #62
- LCD OCR of a gas pump (not glowing) by @mj-xmr in #63
- Image tests for all supported images & scripts by @mj-xmr in #65
- Temperature readouts by @mj-xmr in #66
- Refactorings by @mj-xmr in #67
- Max charge voltage in the battery config by @mj-xmr in #68
- Unit tests: Habits by @mj-xmr in #69
- Voltage charge soc by @mj-xmr in #70
- Json reader refactoring by @mj-xmr in #71
- Modelling max charge voltage by @mj-xmr in #72
- Stability fixes by @mj-xmr in #73
- CI: segmentation of building and testing by @mj-xmr in #76
- Stability fixes by @mj-xmr in #75
- Production test separate by @mj-xmr in #77
- Voltage plots simplifications by @mj-xmr in #78
- Recognizing simple rectangles representing percentage by @mj-xmr in #79
- Image capture via SSH and a custom script by @mj-xmr in #80
- Docs - OCR & rectangle recognition by @mj-xmr in #81
- Doc: Rectangle recognition & Image capture module by @mj-xmr in #82
- Opti algo solution improvement by @mj-xmr in #83
New Contributors
- @dependabot made their first contribution in #53
Full Changelog: v0.1-beta...v0.2-beta
MVP: Optimization, weather, habits & hashrates
First release and an MVP
The 1st release concludes the 1st milestone described in the original description, plus a few very important features from the @endorxmr 's 2nd milestone, that also nears completition.
This together means, that except for the stub of the hashrate's seasonality, all the major components, that deal with physical properties of the system are being simulated and optimized. Additionally, the system helps in further leveraging the current hashrate situation, using Mean Reversion strategy, to further boost the collected hashes per unit of energy used for mining. The hashrate bonuses are a result of @endorxmr 's work, and my post-processing of what his module delivers. The post-processing is being done by tsqsim. Many further improvements can be still done on this front and this is what tsqsim
is about: squeezing more from the publicly available data, that still is able to give you advantage over the careless type of competitors :)
Collectively, scooping the hashes this way also improves and stabilizes the network's security.
As far as the 1st milestone deals with optimization of collecting hashes while preventing battery's over-/undervoltage, the 2nd milestone will deal with profitability of the same act, measured in FIAT currency terms. Please note however, that this will be only a derivative of the collected hashes, and can thus be treated just as an additional layer for a User, who is dedicated to mine Monero and assumes being rewarded in FIAT later, at a higher price anyway.
Major achievements
As a starter, please see the Screenshots section. The individual achievements will be explained below. Here are the quickstart instructions, if you'd like to play around with the release.
Weather predictions
The output of a PV system is greatly influenced by weather conditions. The simulation takes this into account and whenever a cloudy weather is predicted, the system behaves much more conservatively.
Optimization
The optimizer is able to deliver multiple solutions and presents the User the one, which achieves the best score, measured by the sum of collected hashes. Whenever in a given solution the battery would be over-/undervolted, such solution becomes penalized and will thus not be taken into account. The horizon of such optimization is set to 4 by default but can be increased up to 8 days.
Obviously the longer the horizon, the poorer the weather predictions shall become. However, nothing prevents you from rerunning the simulation again with a state (battery charge) achieved in future, in order to correct any mistakes done earlier.
Extra documentation
I felt like I needed to let it all out, what I exactly think about safety and economy of the operation we're dealing with. Please see this section, if you like a quick blog-like read.
Habits
As explained in the economy article, many times it will make sense to rather use the electricity for domestic purposes, as the reward is immediate and without any competition. This is especially valid, if your PV system is of a tiny size. The habits configuration allows you to declare your predictable habits and mine when only these higher-priority needs (as defined by you), are satisfied and there's still enough power stored for mining.
Mining rig definition
Very related to habits are the mining rig definitions, with the difference, that they are assumed to be able to be started and switched off by the system on demand. More information can be obtained in this section.
Modeling solar arrays
Before even spending a dime, you might want to make a good plan how many panels to buy and where to place them. You might want to resist the mainstream idea of placing every panel with the same azimuth of 180° (exactly south) and tilt of 45°, for more diversification stabilizes your output across the day or even the whole year. The following section explains how to perform such experiments.
Console UI (almost done)
There's a console UI in works, whose prototype you may already execute with the src/ui_curses_menu.py
script. In the future we'll present a GUI version of the UI, using exactly the same menu definitions, to avoid duplication, which are stored here.
What's Changed
- Add requirements and move parameters to config.json by @endorxmr in #1
- Implement basic profitability calculation and price fetching by @endorxmr in #2
- CI: Initial version - Ubuntu by @mj-xmr in #3
- UI: Asciimatics console by @mj-xmr in #4
- UI: Ptyhon curses by @mj-xmr in #5
- version by @mj-xmr in #6
- UI: Curses-menu by @mj-xmr in #7
- UI: Installation menu by @mj-xmr in #8
- Tests: full automation by @mj-xmr in #9
- UI: Menu adjustments by @mj-xmr in #10
- Add windows-curses as a windows-only requirement by @endorxmr in #12
- Initial simulator by @mj-xmr in #14
- Simulator with python by @mj-xmr in #15
- Multiple computers defined by a json file by @mj-xmr in #17
- Configurable battery by @mj-xmr in #18
- Python reads battery.json by @mj-xmr in #19
- Fetching weather by @mj-xmr in #20
- Caching weather download by @mj-xmr in #21
- Input initial battery state by @mj-xmr in #22
- Weather prediction n days ahead by @mj-xmr in #23
- Fix horizon indexing by @mj-xmr in #24
- Config templates by @mj-xmr in #25
- Geoip by @mj-xmr in #26
- Arrays definition by @mj-xmr in #27
- System voltage and wattage lot by @mj-xmr in #28
- Display hour fix by @mj-xmr in #29
- Individual system settings by @mj-xmr in #30
- Hashrate bonus ma by @mj-xmr in #31
- Habits json by @mj-xmr in #32
- Habits got infrastructure entries by @mj-xmr in #33
- Ci mac by @mj-xmr in #35
- Battery charge AH & V stub by @mj-xmr in #36
- Verbose functions by @mj-xmr in #37
- Runtime optimizations by @mj-xmr in #38
- Reenable tests by @mj-xmr in #39
- Enable local unit tests by @mj-xmr in #40
- Doc: safety by @mj-xmr in #13
- Array modelling by @mj-xmr in #41
- Doc: Economy by @mj-xmr in #42
- Arrays can be parametrised by @mj-xmr in #43
- [WIP] Historical price and difficulty fetching and saving by @endorxmr in #16
- Unit tests: Habits, etc. by @mj-xmr in #45
- Latest network diff taken into account by @mj-xmr in #46
- Hash for data by @mj-xmr in #47
- Simplified historical data fetching by @mj-xmr in #48
- Hash for data by @mj-xmr in #49
- Update linux.yml by @mj-xmr in #50
- Tuning: Don't mine on the last day to spare energy by @mj-xmr in #51
New Contributors
Full Changelog: https://github.com/mj-xmr/SolOptXMR/commits/v0.1-beta