World at War Performance is not good

[DanVanAtta]

The map takes a noticeable time to draw. This was not the case before major updates. There were a number of new custom unit images added, perhaps the game engine is not handling those well (or maybe there is a pre-cache for normal units but not map custom..).

[simon33-2]

Other maps have been complained about too. WW2 Global 1940. Is it a map or engine problem?

[DanVanAtta]

Game engine seems to be doing something inefficient.

[DanVanAtta]

Global 1940 also had a number of custom unit images added? I think the game engine was doing something bad there, and now that we use that feature more, we are seeing the problem more accutely.

[Cernelius]

I think the issue is that the map is drawn progressively, when you scroll it.
You should modify the behaviour as to draw it all at once, and keep it in memory. You would have to wait some seconds more at start game, but then it would be totally smooth thereafter.

Loading all the map Tiles etc. at once, instead of waiting you scrolling around to do it, is a thing I wanted to suggest since a long time. Current way of waiting the user to move around before loading is really clumsy.

[DanVanAtta]

Some things we should know before deciding what a good fix would be:

• why the dramatic difference? The progressive rendering was not updated, so why does it perform so relatively poorly now? It does seem like there is something that should be cached / loaded, that is not, since once everyone is loaded (by scrolling through the complete map), the performance seems to be there.

[Cernelius]

Of course, I've no clue about the dramatic difference you are talking about, since I've never played World At War, nor before nor after I modified it (playtest is all on redrum); maybe let's wait for redrum or somebody to confirm.

You can see for yourself that if you set "triplea.memory.Xmx" high enough (not much needed, default may be already enough, but I suggest you increase it to 2048, just to be more than safe), and, then, you open World At War and scroll around everywhere to uncover / load the whole map, you have the delays while you do it the first time, because it loads the stuff progressively, as you move your visual over it, but, then, you can keep moving around, and it is instantaneous.

You can also think in perspective, as a map like World At War, but sized for 4k monitors would be much more demanding.

Basically, if you change the behaviour to load all at start game, and you have enough memory assigned to handle it, you would just have to wait some seconds at start game, and then no problems thereafter, until you decide to like change the Flag Display Mode or something, but no problems again, once all is loaded anew.

The problem now is just that you have to scroll around the whole map manually, once, to load it fully. Not really any problems, if all would be loaded at start, and you have enough memory assigned for it (of course, the delay is only when you load the images from the hard disk or something the like).

Side note, a bit of memory is sucked out by the world_at_war_250.gif image, at start of Notes, that it is quite heavy, being an action GIF of 7.6 MB and 360 looping frames at 20 ms intervals, but it shouldn't impact much (it doesn't for me, at least). You can try to remove that image, to see if it changes anything for you.

Also, try to remove the misc/displayImage.png, and let me know if that changes anything, for you.

[DanVanAtta]

No, not a memory issue, same behavior as with 3072m of memory. Let's not speculate too much on what the problem is until we've isolated the problem a bit more. Meanwhile, @ Cernelius your input is needed on how to reproduce the horizontal line problem over in: #1056

[Cernelius]

I believe it can only be the displayImage.png; as I did that thing in a way that should not be done (having a decoration covering the whole map is too extreme, and surely not the proper way to use that feature).

Did you try to remove it? Does it make it faster?

If you think it is really unbearable, I can integrate that in the reliefTiles, but that would make them harder to modify in the future; this is why I wanted to keep them clean (the current baseTiles and reliefTiles are bot clean of drawn borders, and this makes them much handier to modify).

In case we go this way, I suggest to save the current reliefTiles, without the added borders, currently kept in the displayImage, somewhere.

I would really like to avoid having the borders in the reliefTiles, as it is the norm in TripleA, as that would make much harder (close to unfeasible) to modify any such borders in the future.

Sadly, TripleA lacks a way to have borders as their own thing; this is why mapmakers normally integrate them in the reliefTiles, which normally means you can't touch the map anymore, without trashing and redoing the whole relief.

[DanVanAtta]

yes, it was the displayImage.png. After removing it, the game seemed to play fine. Why is it necessary?

[Cernelius]

Well, as I said, it is not necessary, from a user standpoint (assuming you will never modify World At War again), as I can integrate it in the reliefTiles and the baseTiles, and would show up exactly like now, and be faster.

But this would imply that, then, changing the graphic again would be a pain and close to impossible.

Thus I wanted to keep the baseTiles and reliefTiles clean (current WAW is clean, because all the borders are the black standard; like you don't have the nice sea zone borders NWO style, but they are just the black default stuff).

I can reintroduce the thing of having pointless sea zones for all lakes, to display the black land borders, and limit the displayImage to the sea borders only, see if that way it can be more bearable.

I would really like to keep the baseTiles and reliefTiles clean of borders, in case anyone would need to change the map in the future (for example, with no borders in baseTiles and reliefTiles, it is very easy to revert back the split sea zones in pacific).

It would be cool if it would exist a borderTiles dedicated for the borders only, since the load delays are just because we have a single image for the borders, instead of a lot of small tiles, but of course this is not possible for the next release, and would need a developer coding it. Without it, you are obliged to draw the borders in the baseTiles and reliefTiles, making them much harder to modify.

Or I can split up the displayImage into a lot of small decoration Tiles, like the same way the reliefTiles are handled.

[Cernelius]

If you think the delay is too bad, I can just integrate them in the reliefTiles (as it should be normally done), thus getting rid of the displayImage.png, and redrum can save anyway the reliefTiles without the borders somewhere for future mapmaking use. Question is, where the relief image without added borders should better be saved, without obliging the user to download it each time, since now all the master is downloaded?

Let's wait too see what @ron-murhammer thinks?

[Cernelius]

I've also the multilayer GIMP image that would be good to be saved somewhere, if anyone wants to restart working on the map's graphic.
It's a 99.2 MB file you can open only with GIMP; where should it be saved, in case?
If that is saved somewhere, then it is no problem to integrate the borders or whatever, since you can always go back directly to the GIMP image source of everything.
Having that saved somewhere in GitHub, and integrate the borders, getting rid of the useless displayImage.png, would probably be the best move, if you think that the delays it gives are really that bad when gaming (I don't know, seemed only slightly slower to me, and once it is loaded it is loaded).

But I believe that if you make all the graphic loading at once, whatever kind of heavy graphic would be no problem, cept waiting some seconds more at start.

[Cernelius]

The issue with the lake borders, to be clear, is that TripleA doesn't allow you to draw territories with circles inside. And this, in turn, is the main reason you have that thing that territories ending in "Sea Zone" have lower priority over the other ones. This is why Sieg made all those pointless lake territories, as sea zones, most likely.
I preferred to do it with a decoration, instead of creating lake sea zones of no gameplay sense, but it can be done on the relief and base, as well (and it is usually done that way).

[DanVanAtta]

I'm not sure I understand correctly, the map renders the same with or without this file, and referring to this file - "it is not necessary, from a user standpoint"?

[Cernelius]

Yes, I can modify the map as to look exactly like now and be just as fast as when you remove that file.

Only drawback is that if anyone would want to modify the map in the future (like changing the sea zone borders around) it will be much harder, and not really feasible.

But this can be solved by just saving up the 99.2 MB original "GIMP Image", which would be good anyway, as the starting tool for whatever other future changes. But where to upload it?

That thing to have the borders as a single decoration is surely not a thing that should be done; I did it just to keep the tiles clean of borders, in case the map has to be worked on again in the future, and just hoped it would not slow the game down too much (yes, a single decoration covering the whole map is very inefficient, this is why the relief and base are split into small tiles, instead of having two massive images).

Any opinions where the 99.2 MB GIMP image with all the stuff can be saved, for future use, if merging the borders over the relief?

[Cernelius]

Or, instead of saving the GIMP image, we can just archive somewhere the reliefImage without borders and the displayImage, for future use, in case of any more changes. But where?

Now that I think about it, GitHub keeps everything in memory right? So, no problem; the next version I integrate all in the reliefTiles, and if you need to go back changing anything, you can download the release before the 2.1 version (which will be uploaded in like a couple days from now).

What I'm going to do now is merging the displayImage over the reliefTiles, obtaining new reliefTiles with the borders already on them (as it is normally done; that massive border decoration was basically just a hack to keep the relief clean, but not needed at all).

I still believe it would be cool if all graphic would load at once at start, if the user has enough memory to handle it, instead of waiting for you to move around and see all the tiles loading.

[Cernelius]

Also, where has the PNG optimizer gone? Was that abandoned because it gave problems? Should the mapmakers apply it themselves?

[Cernelius]

For example, you can see that, without that displayImage, all the borders of the islands (like Newfoundland), are not shown, and it is not possible to show them, due to the engine limits I previously mentioned.

In the original, Sieg got around it by using the cut-the-sea-zone hack (like you can see that in old a line connects Newfoundland with E.Canada). And he created all those pointless mini sea zones for the lakes, too. I got around it, in both cases, with the displayImage decoration.

But, no problem, I can just merge the displayImage over both the reliefImage and the baseImage, and this will solve without delays, or maybe create a massive territory for all borders.

Either ways, since you says the delays of that displayImage are too bad, I will remove it in the next, and it will be as fast as it is now without that decoration.

Would be nice to save the baseImage and reliefImage without the added borders somewhere, tho. Any ideas of a place where to archive the development only stuff for specific maps?

[Cernelius]

Yes, it is much faster now.

Without the displayImage now loads in less than half the time than latest. Strange noone else complained. Or maybe the delays are not much felt, since, anyway, once it is loaded it stays loaded.

I'm still using the decorations, but only to show the blue dots of the canals, individually.

I think the displayImage without the blue dots of the canals will be merged over the current relief, while the black of the polygonsImage will be merged over the current base. A blue_dot image will take the place of the displayImage, in misc.

I still believe it may be good to save the baseImage_noBorder and reliefImage_noBorder somewhere.

[Cernelius]

By the way, I also partially explained some of the reasons of that decoration in the changelog of the version you are testing:

• Change the sea border from side black and inside clear (so, displaying the sea underneath) to side "7e9d9d" (equal to the base image sea color) (dark turquoise) and inside "bacfcf" (bright turquoise) (about the same as the base image sea color next to the land territories) (to do so, add "displayImage.png" image of land and sea borders above the map as decoration (fully overlapping the respective displayed borders))

...

• Add all canal dots above the map as decoration (fully overlapping the canal dots in relief)
• Change canal dots color to much darker and slightly more saturated blue (-85 lightness, +17 contrast, some sharpening) (change applied to decoration only, not relief)

Anyway, I'm removing that decoration, as I said, pasting it on the reliefTiles, instead, except the blue dots canals, that stay as decorations (a single one, drawn multiple times).

[simon33-2]

So this problem is fixed? In what release?

[Cernelius]

As far as the map is concerned, I believe it will be solved when the 2.1 is uploaded in GitHub (which should be a matter of days) (because the displayImage will be no more).
Or it can be solved by having the entire map loading at start, instead of waiting to scroll around, to load the stuff (which requires loading the displayImage each time).
As now, having very big decorations is inefficient, and this was a relatively light one.

References:
http://tripleadev.1671093.n2.nabble.com/WORLD-AT-WAR-tp5862407.html

[Cernelius]

Anyway, to me the delays seemed very minor, especially since it stays loaded. Also, nobody else complained.

Now that I think about it, maybe a gif instead of a png might be light enough?

Try to do this (substituting the displayImage.png with an identical gif):

• Paste this in world_at_war/misc:
  
• Substitute the world_at_war/decorations with this:
  decorations.txt

Does it get better? Would it now be bearable?

Please, @DanVanAtta , do the above steps and let me know if you think it is acceptable for you now. If it is not, I'll go ahead with just removing the displayImage and integrating something like that in base and relief (which surely solves and it is the fastest).
Also, @ron-murhammer , test it too, and let us know.

[Cernelius]

To me it seems that if the displayImage is a gif, then the added delay is not much noticeable, but waiting for opinions (as I said, this is an unintended way to use decorations (no maps I know do this); I just did it to keep the base and relief clean, so that the map is the easiest to modify for mapmakers).

Is the gif one acceptable or not?

[Cernelius]

But I see the displayImage gives significant delays even as gif.
Since this displayImage thing for borders+canal is an unintended, inefficient and pointless solution, from an users' standpoint, probably better going the normal way.

Anyway, I cannot upload that, because it tells me limit is 10 MB.

So, just confirm that the previous solutions were indeed too slow to be acceptable, and we can stick with regular (you can just compare by removing the displayImage), and feel the difference.

Again, any ideas about a place where to save the map development stuff, for future mapmaking use, like the base and relief images without the added borders, and maybe also the relief image before the reduction of the opacity to 0.8 (that would be really good to save), or even the original GIMP image, stuff like that?

[DanVanAtta]

Thanks @Cernelius for hopping on this. For file upload:
https://github.com/ -> new repository button

Create a repo named perhaps 'world_at_war_assets', or similar. The 10MB restriction is just the web interface, there is no restriction when using a git client.. On the plus side git clients have gotten very good of late :)

The easiest I've seen so far is the one github released not long ago, after creating your new repo, in the 'download or clone' button, there is a 'open in github desktop' option which then guides you through. The UI will then basically help you mount a folder where you can files or drag and drop in new ones, and the updates and new files can then be 'committed' and 'pushed' back up to the github repo. Finally, this repo can be 'given' to triplea-maps org from the repository settings menu.

@Cernelius I think you would make an ideal map admin. In this case you could join that team and as an admin would have write access to all maps. In that case you could create the assets repo directly in the triplea-maps org without having to 'gift' the repo to a different org (note, github has a concept of org, and individuals. Triplea-maps is a different org than triplea-game, and both have a number of repos. We wanted to keep the mess of dozens and dozens of map repos out away from the game code)

To finish the topic, it was intended that content parallel to the 'map' folder was going to be a place to warehouse any additional content. It turns out though in practice the map zip option from the 'download or clone' button seems to have been easiest for everyone. No more confusion on finding and clicking 'releaseses', and no need to update the maps yaml config on every map update. So I think we are left with the parallel 'assets' repo as a strategy for hosting content. We've done so in the triplea-game repo for the game installer files and rule manual: github.com/triplea-game/assets

[Cernelius]

I want to make clear that I'm not the owner nor the admin of World At War (if I were, Chunking would be now called Upper Burma, not Yunnan), nor want to (that map is just too far away from my views). The current owner is @ron-murhammer . I'm just helping him out and giving him suggestions. Thus, whatever World At War related stuff should be owned by him, as well, or, anyway, not by me.

What I was saying is that having the originals, in particular the multilayer Gimp image or images for each of its layers (in this case, probably two images for the relief; one the basic and the other one at opacity 0.8) would make the map much more feasible to re-modify in the future.

My decision to apply the borders as a decoration, instead of directly in relief and base was in order to make the map easier to modify, in case we are not saving the basic stuff anywhere.

Anyway, I will ask redrum in the WAW thread about this.

Generally, I think having a map folder and a development folder (I would surely not call it assets, because they are not really assets) is too much. I was rather thinking about a single repository, owned by all people owning any repositories in triplea/maps, in which any of them can go there backing up their development stuff, used in the building of the final assets. But if such a repository would be limited to 1 GB, that might be bad.

Anyway, probably better I go talk with redrum in the WORLD AT WAR thread about it. Depends if he is interested in the idea or not, because it will be up to him to do the uploads.

I tend to think it is anyway better the map repository being used only to store the actual (final) assets and game info.

I still have to even learn how to use GitHub for uploading new maps. I've some (3) mods of 270BC I intend to upload soon in a new 270bc_variants repository, but I've still to learn the process.

[DanVanAtta]

I suppose you didn't quite grasp the role of a generalized map admin. That role is not something where you can just start changing territory names on any map.

The naming is inconsequential, there are not many other file hosting strategies other than hosting repos.

RE: single repo as dumping ground for extra images - since not everyone will have write access for it, it's a huge bottleneck. The many people writing to it will make the git flow difficult, and considering the total size limit, not a workable answer.

[DanVanAtta]

@Cernelius the file that is slowing things down can be removed, and we'll just lose borders? So we can fix the map, seems like we should do that while you are working on the updated images and reliefs.

[Cernelius]

Did you try to swap the png image for the gif one I posted? Does it improve performances? Not relevant, but just curious.

Anyway, I hope to be able to give redrum the new WAW 2.1 (without said decoration) within 24 hours; so I don't think it would make sense to make any changes meanwhile, but yes removing that decoration will have the only major consequences that:

• Most fake sea zones will not have borders (obtained by making 1 actual sea zone for each one, in the old WAW).
• All islands won't have intended borders (obtained via sea zone cutting, in the old WAW).
• You lose the turquoise colourisation for the sea to sea borders (absent, in the old WAW).

You can either remove that file or the decorations.txt or both, but I suggest you don't, since I don't see the reason to make another release with the right one just coming.

[Cernelius]

Just as a curiosity about having a 3rd Tiles folder, showing up above baseTiles and reliefTiles, you can see in total_world_war that the misc folder is used as a 3rd Tiles folder, showing up above baseTiles and reliefTiles (the coordinates are in decorations). This is sort of a hack, like my displayImage (but better working, cause split into many tile-like parts), since this is not the way decorations are supposed to be used (they should be used for something like the "Teams" and "TripleA_ancient" of 270BC). However, I don't want to go that way, as it doesn't make sense in my mind to use the "misc" folder as a 3rd "tiles" folder, as requiring all those coordinates in decorations is kind of silly.
Maybe there should be an actual optional 3rd Tiles folder (in addition to the baseTiles and reliefTiles), to avoid mapmakers using the misc for that, like in total_world_war.
It would make some good sense, as you may want to have some relief-like graphic that it is not merely decorative, and should not be switched off, by untiking Show Map Details.

[Cernelius]

Another example of the potential usefulness of a general storage for maps' development images is saving the patterns used in the maxing of the relief.

For example, all TripleA WWII, from v2 to v6, were made with the same relief patterns.

Veqryn has them, and I don't know if they are saved somewhere, but if nobody has them, it won't be feasible to adopt the same style for a future WWIIv7.

I'm under the impression that if such a general storage would exist, very few would use it, if any (people generally only care for making their own maps, not for helping unlikely future people modifying them).

Anyway, as I said, if such a storage would exist, it would be up to redrum to upload the stuff there, for WAW.

Example:
http://tripleadev.1671093.n2.nabble.com/Can-I-to-adapt-WW2v3-s-Relief-to-The-World-at-War-Ezell-or-Global-War-tp7593448.html;cid=1473928873659-26

[DanVanAtta]

@Cernelius , create a new repo to hold the offline assets, upload files there. Then please update the README of the main map repo to give a link to the offline assets repo.

[Cernelius]

I just sent redrum the new WAW. I leave to @ron-murhammer to make a new WAW related repo or update anything WAW related in GitHub.

[Cernelius]

@DanVanAtta is this release:
https://github.com/triplea-maps/world_at_war/releases/tag/0.25
solving this issue for you?
And I want to underline that noone else playtesting the new WAW, comprising @ron-murhammer , noticed this issue or expressed any concern over it. From my experience, the delays seemed quite minor, and just irrelevant after you scrolled (loaded) the whole game's image (but I notices you have them back if you change flag display mode, which is annoying).
But I agree that the faster the better, thus the change.
A benefit of the new version is that it allows you not having the turquoise sea to sea borders, by switching details off.

[ron-murhammer]

The performance is noticeably better now. It wasn't that bad for me before which is why I didn't say anything. But I think we can close this once we get one other person to confirm.