Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More granular json-style and astyle #51731

Merged
merged 4 commits into from
Sep 23, 2021

Conversation

kevingranade
Copy link
Member

Summary

None

Purpose of change

Erk complainined that json-style (actually style-all-json, but whatever) is too slow, and he's right.

Describe the solution

Added idiomatic make rules for both astyle and style-json to enumerate the cpp/h and json files respectively as targets, and touch token files to cause make to ignore those files in the future unless they have been edited again.
This accomplishes two things. One, it instructs make to use whatever parallelism is authorized by the -j argument to run multiple instances of the linting commands. Two, it records timestampts so that when in a protracted edit cycle these targets won't waste time repeatedly re-checking files that have not changed since the last check.

Describe alternatives you've considered

Tons of options, this is pretty idiomatic and performant. Some kind of batching might be helpful to keep the system from executing these quite so many times, but if you throw -j 10 at it, it's already pretty fast.

Testing

Ran the commands, does a full re-test the first time through.
Ran commands with -j 10, with very, very noticeable improvement in speed.
Re-ran commands, observed instant completion with not files checked.
Edited some .json, .cpp and .h files and re-ran commands, only the edited files were re-checked and it was very fast.
Decent chance of this breaking on different platforms, so let the full test suite complete before merging.

Additional whatever

The -check and -all variants are untouched, there might be a good way to integrate them, but it seems complicated.
I double-checked that .gitignore does not need any new entries, as I stashed all the timestamp files under obj/ (or objwin/ on some systems? IDK if that even works).

Makefile Outdated Show resolved Hide resolved
@actual-nh actual-nh added <Bugfix> This is a fix for a bug (or closes open issue) [C++] Changes (can be) made in C++. Previously named `Code` [JSON] Changes (can be) made in JSON Code: Performance Performance boosting code (CPU, memory, etc.) Code: Tooling Tooling that is not part of the main game but is part of the repo. labels Sep 20, 2021
Makefile Outdated Show resolved Hide resolved
Makefile Outdated Show resolved Hide resolved
@kevingranade
Copy link
Member Author

Make is so weird sometimes, I had to declare a dependency on the json formatter exe from both the main target and the sub=target, because otherwise it will launch all the dependencies in parallel, which of course means it will try to simultaneously build the tool and execute it.

Makefile Outdated Show resolved Hide resolved
@kevingranade kevingranade merged commit 81960cd into CleverRaven:master Sep 23, 2021
@kevingranade kevingranade deleted the more-makey-styling branch September 23, 2021 16:33
I-am-Erk added a commit to I-am-Erk/Cataclysm-DDA that referenced this pull request Sep 23, 2021
* reorganized to support expansion

* reorganized to support expansion

* removed sentences that shouldn't be there

* fixed log stable appearing as kitchen construction

* Reworked to push expansions back to the top level + doc

* testing disallows base name recipe changes

* removal of characters in middle of token managed to offend secret json format mandate

* Adjusted to CleverRaven#46278

* incorporated some old reverse conflict changes

* reorganized the stuff added while this PR was suspended

* reorganized the stuff added while this PR was suspended

* Fix bike shop variant roof

Fix the bike shop variant 1 having an empty roof

* Change stairs to ladder

Changed stairs down to ladder down

* Update the table of contents

* Remove duplicate shortcut keys from interaction menu

Fixes CleverRaven#51772

Co-authored-by: David Seguin <davidseguin@live.ca>

* Jsonified dead scientists map extra

* Use string_id for update_mapgen and nested_mapgen

These ids were previously just using strings; better to use a dedicated
string_id type.

This required adding an actual type for the string_id to point to in
each case, wrapping the collection of mapgen_function_json objects.

* Update cooking_components.json

* Force recipes to provide (sub)category

As suggested in CleverRaven#50866, require all recipe definitions to provide a
category and subcategory when loading from JSON.

* Add subcategory to Magiclysm recipe

* Clarify description of mon_zombie_dog (CleverRaven#51801)

See <https://discord.com/channels/598523535169945603/598614717799596055/890213920957161472> on the dev discord.

* Test disabling expensive LGTM checks (CleverRaven#51806)

Disable one LGTM checks to see if there's hope of reducing the runtime
thereby.

* Mythos Mod Resubmission (CleverRaven#51796)

* Mythos Mod Resubmission

* Remove Occulti as maintainer

* Fix typo in "Storehouse survey" blueprint name

* Add dialog activity functions, and flag to prevent activity being interrupted (CleverRaven#51809)

* Add dialog stuff

* docs

* Update player_activities.json

* use snippets (CleverRaven#51663)

Co-authored-by: Saicchi <Saicchi@users.noreply.github.com>
Co-authored-by: Kevin Granade <kevin.granade@gmail.com>

* Blacksmithing tongs and cooking tongs are two diffrent things.  (CleverRaven#51672)

* Add blacksmithing tongs.

The tongs in game wouldn't be able to carry around a four kilo zweihander. They are designed for cooking and flipping food, so they are very lightweight and thing, they would most likely bend under the heat of the hot metal.

* Added the need to use blacksmithing (flatjaw) tongs for the Zweihander

Not using tongs for smithing is almost impossible.

* Change the blacksmithing toolset to include the tongs.

I also removed the tongs from the tools on the zweihander.

Co-authored-by: Anton Burmistrov <Night_Pryanik@mail.ru>
Co-authored-by: Kevin Granade <kevin.granade@gmail.com>

* More granular json-style and astyle (CleverRaven#51731)

* More granular json-style and astyle

* Melee practice and training dummies (CleverRaven#51598)

* Melee practice
Added some basic melee practice recipes, including 2 for training dodge.
* Training dummy for practice recipes
* Dummy recipe and resolve problems
* Intermediate recipes and heavy training dummy
* Changed the armored dummy recipe
The recipe is inspired by the scrap suit recipe.
* Added tanto as a possible weapon for stabbing training
* Punching bag

Co-authored-by: Marloss <78324429+MarlossCDDA@users.noreply.github.com>

* Write documentation for neighbor based chunk spawning (CleverRaven#51818)

* Resubmit Fix Tire Rims (CleverRaven#51798)

* Update goblin.json (CleverRaven#51799)

* Fix: Force stereo when opening audio device (CleverRaven#51754)

Co-authored-by: Kevin Granade <kevin.granade@gmail.com>

* Add support for weakpoint damage and crit multipliers (CleverRaven#51770)

* Weakpoints (Part 4.5): Add support for JSON inheritance and default weakpoints (CleverRaven#51822)

* Let contributors specify the default weakpoint, by providing an empty ID

* Add support for weakpoint inheritance.

* Reduce debug.log file size by folding consecutive identical errors (CleverRaven#51791)

* debug_fold_repetition: Fold identical errors to reduce log spam

* debug_fold_repetition: added timestamp and timeout

timestamp for repeated error folding group is that of the last folded
error
timeout is currently hardcoded to 100ms. If another repeated error is
registered that would be folded but is >100ms after the previous error
it starts a new folding group

* debug_fold_repetitions: bookended fold for readability, made output DRY

It's a little easier to read the log as `[First]<folded>[Last]` than
`[First]<folded with last timestamp>`
Need to test folding output during deinit and DebugLog, made a function
to remove repetition.

* debug_fold_repetitions: excess repetition error to forced prompt

As logged error the excessive repetition error causes breaks in folds
which make debug.log less clear while reading, and does not add information
Forced prompting once excess threshold is reached will prompt once per
excessive repetition set.

* Allow using any tool with the drilling quality to make a draw_plate (CleverRaven#51794)

Fixes CleverRaven#51688

* The (impotent) Sound and the Fury (CleverRaven#51803)

* update m231

Updated price(https://www.invaluable.com/auction-lot/incredibly-rare-colt-m231-port-firing-u-s-propert-2086-c-3574a02a1e), adjusted desc, set sight disp to that of other no-sight guns, adjusted mod locations.

* Rapidly changing light levels do not grant max vision to turrets (CleverRaven#51815)

* Update fake.json (CleverRaven#51819)

* Fix process effects crash (CleverRaven#51827)

* Add Complete missions dialog function (CleverRaven#51820)

* Complete missions

* Update npctalk.cpp

* Update npctalk.cpp

Co-authored-by: PatrikLundell <j.patrik.r.lundell@gmail.com>
Co-authored-by: SegaSaturnity <75910217+SegaSaturnity@users.noreply.github.com>
Co-authored-by: kevingranade <kevingranade@users.noreply.github.com>
Co-authored-by: UmbralReaper <67179462+UmbralReaper@users.noreply.github.com>
Co-authored-by: David Seguin <davidseguin@live.ca>
Co-authored-by: Valiant <Night_Pryanik@mail.ru>
Co-authored-by: John Bytheway <52664+jbytheway@users.noreply.github.com>
Co-authored-by: mythosmod <91185016+mythosmod@users.noreply.github.com>
Co-authored-by: Kevin Granade <kevin.granade@gmail.com>
Co-authored-by: actual-nh <74678550+actual-nh@users.noreply.github.com>
Co-authored-by: Angela Graves <rivet.the.zombie@gmail.com>
Co-authored-by: Binrui Dong <brett.browning.dong@gmail.com>
Co-authored-by: Eric <52087122+Ramza13@users.noreply.github.com>
Co-authored-by: Saicchi <47158232+Saicchi@users.noreply.github.com>
Co-authored-by: Saicchi <Saicchi@users.noreply.github.com>
Co-authored-by: GOFLUMPYOURSELFPORCAY <64309930+GOFLUMPYOURSELFPORCAY@users.noreply.github.com>
Co-authored-by: Termineitor244 <termineitor244@hotmail.com>
Co-authored-by: Marloss <78324429+MarlossCDDA@users.noreply.github.com>
Co-authored-by: Zhilkin Serg <ZhilkinSerg@users.noreply.github.com>
Co-authored-by: John Candlebury <johncandlebury@gmail.com>
Co-authored-by: The SzQ <37194372+SzQ1@users.noreply.github.com>
Co-authored-by: Joshua Chin <5422226+Joshua-Chin@users.noreply.github.com>
Co-authored-by: OrenAudeles <orenaudeles@gmail.com>
Co-authored-by: Tonkatsu <7764202+tenmillimaster@users.noreply.github.com>
Co-authored-by: Roy Berube <royberube1965@gmail.com>
Co-authored-by: ISuckM8 <87550905+ISuckM8@users.noreply.github.com>
Co-authored-by: ferociousdork <78301810+ferociousdork@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
<Bugfix> This is a fix for a bug (or closes open issue) [C++] Changes (can be) made in C++. Previously named `Code` Code: Performance Performance boosting code (CPU, memory, etc.) Code: Tooling Tooling that is not part of the main game but is part of the repo. [JSON] Changes (can be) made in JSON
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants