Skip to content

Development Testing Tools

Jump to bottom Edit New page
Oleg Agafonov edited this page Aug 3, 2024 · 37 revisions

Testing XMage

There are two methods to test app:

  • manual testing from xmage application (server test mode);
  • automatic tests from IDE (unit tests);

Also you can use additional launcher/java commands:

  • client: -Dxmage.badconnection - simulate bad/slow network connection with a server (useful to test game's GUI bugs);
  • tests: -Djava.awt.headless=true - run tests in headless mode without GUI access (travis CI’s run tests in that mode);
  • images: -Dxmage.scryfallEnableBulkData=false - run app without scryfall’s bulk data usage (example: testing api links generation on images download);

Server Test Mode

XMage has a test mode, that helps you with testing implemented cards or other changes. Therefore you have to start the XMage server in the test mode. The test mode will be active if you set the -Dxmage.testMode=true. You can use it even with release version (no need to install any developers tools or source code).

Test mode features:

  • adds three fast game buttons in tables list
    1. "Quick 2 player": 2 player commander game
    2. "Quick 4 player": 4 player commander game with limited range of influence (range = 1)
    3. "Quick start MCTS": 2 player duel with the MCTS AI.
  • add cheat commands button under avatar to setup battlefield and run any cheat commands;
  • disable deck validation;
  • simplify registration and login (no password check, e.g. login under any user);
  • add debug main menu for GUI and rendering testing.

Start server by launcher

Open launcher -> settings -> java -> server options and add additional param:

  • -Dxmage.testMode=true

Start server by IDE

If you run app from a source files then no needs in special argument - server will run in test mode by default. But it can be useful to disable test mode, e.g. for deck validation testing. How to do that:

  • For IntelliJ IDEA: open run configuration -> application (see IDE setup for details) and insert in program arguments field: -testMode=false or a java arguments: '-Dxmage.testMode=false'
  • For NetBeans: select the Mage Server project and use: Set Configuration -> Customize... -> Categories: -> Run -> Arguments: _ and insert -testMode=false

Start server by script

Create command file run-server-testmode.cmd in launcher folder and add text to it (for windows). Don't forget to fix version string to actual value:

set SERVER_DIR=.\xmage\mage-server
set JAVA_DIR=.\java\jre1.8.0_201\bin
pushd %CD%

cd /d %JAVA_DIR%
set JAVA_DIR=%CD%

popd
cd /d %SERVER_DIR%
set SERVER_DIR=%CD%

cd /d %SERVER_DIR%

%JAVA_DIR%\java.exe -jar %SERVER_DIR%\lib\mage-server-1.4.50.jar -testMode=true

How to test

Server test mode gives you on the games panel an additional button "Quick Start". With this button you can immediately start a game with lands only decks for each player:

5 [XLN:268] Swamp
5 [XLN:276] Forest
5 [XLN:264] Island
5 [XLN:272] Mountain
5 [XLN:260] Plains

Also you have an additional button (smiley) inside the small player status box to the left. If you push this button during a match -- you will see special dialog to choose test commands like adds cards to the hand, library or battlefield. That commands are defined in the file init.txt that has to be located in the Mage.Server/config (\xmage\mage-server\config) directory. This works also for matches not started with the "Quick Start" button. There already exists an init.example.txt file, that explains the needed format of the file. During a test game you can change the init.txt file, push the smiley button in XMage and get the cards you need in the zone you want to test your changes in.

Init.txt contain commands to run at any time. You can combine commands in groups by [group name]. Game will ask you what group of commands you want to execute. Also you can comment some lines by //. It's useful to have one file for different tests. Look at server logs to see any typo errors in command names. It can use default names: Human and Computer. If you run it for normal game then just replace it to real name too like user 1.

Init.txt file example:

// Allows run cheat commands in any games. How to use:
// * open launcher and add to server's command line: -Dxmage.testMode=true
// * rename that file to init.txt and put to server's config folder
// * activate smiley button on your player's panel

[init]
battlefield:Human:Forest:5
battlefield:Human:Plains:5
battlefield:Human:Mountain:5
battlefield:Human:Swamp:5
battlefield:Human:Island:5
hand:Human:Lightning Bolt:2

[quick game example]
graveyard:Human:Bloodghast:1
graveyard:Computer:Bloodghast:1
hand:Human:Bloodghast:1
hand:Computer:Bloodghast:1

[real game example]
graveyard:ReachyRichGuy:Bloodghast:1
graveyard:Computer 3:Bloodghast:1

[add tokens]
token:Human:TreasureToken:10
token:Human:ZombieArmyToken:3

[add planeswalker with loyalty]
battlefield:Human:Karn Liberated:1
loyalty:Human:Karn Liberated:20

[add emblem]
emblem:Human:ElspethSunsChampionEmblem:1

[add commander]
commander:Human:Silvercoat Lion:1

// @ref command example (all refs will be replaced by commands from ref's group)
[ref usage]
@init
hand:Human:Lightning Bolt:2

// real test example
[test karn restart]
@init
@bears
battlefield:Human:Karn Liberated:1
loyalty:Human:Karn Liberated:20
commander:Human:Silvercoat Lion:1
exiled:Human:Lightning Bolt:1
exiled:Human:Silvercoat Lion:1
exiled:Computer:Silvercoat Lion:1

// create any useful commands for fast usage
[clone]
hand:Human:Clone:3
[force attack]
hand:Human:Pit Fight:3
[exile]
hand:Human:Angelic Edict:3
[bears]
battlefield:Computer:Grizzly Bears:2
battlefield:Human:Grizzly Bears:2

// special command, see SystemUtil for more special commands list
[@activate opponent ability]

[diff set codes example]
battlefield:Human:XLN-Island:1
battlefield:Human:UST-Island:1
battlefield:Human:HOU-Island:1

A good first test is also looking at newly implemented cards in the deck editor. Check if the tooltip text is correct. In many cases this alone does point out problems of the card implementation. And don't forget to delete the Cards.h2 db files in client and server before you restart XMage after changes to get the tooltip text updated to reflect your last changes. Otherwise the old text is taken for tooltip generation in deck editor or for showing text of hand cards. The text in the database for already existing cards is only updated if you delete the cards.h2 db files of the server and client (for sure something that can and should be improved in the future).

Often it's helpful to start the client two times and create a match between two human players to test something. So you have the full control of playing something or attacking and blocking actions. As said before to add cards to the different zones of the players with the init.txt file works also in this kind of test matches.

AI testing

AI making useful logs for AI debugging (computer's decision process with possible commands and game score). Real games and maven/ci builds don't use AI logs. But you can enable it by log4j.properties setup (use log4j.logger.mage.player.ai=info to enable AI logs):

  • IDE and maven test runner: .\Mage.Tests\src\test\resources\log4j.properties
  • travis-ci test runner: .travis\log4j.properties
  • developer server: .\Mage.Server\src\main\resources\log4j.properties
  • local/real server: .\xmage\mage-server\config\log4j.properties
    • server must runs with additional command line -Dlog4j.configuration=file:./config/log4j.properties

Game simulations runs in different threads with termination by timeout. So you can't stop it to debug by default (debug session stops after 10-20 seconds). You must enable COMPUTER_DISABLE_TIMEOUT_IN_GAME_SIMULATIONS - it will disable timeout termination.

AI logs example with description: shot_240528_173154 shot_240528_174421

Unit tests from IDE

Tests can be created in the Mage Tests project, and can be run by building that project. If you make changes to the Mage core, we recommended you run these tests to see if your changes break any logic checked by the test cases in the Mage Tests project. It's also recommended that you create new tests for any errors not yet covered by existing tests. If you've created a new test that points out an error, running this test is an easy way to check if a change you made fixes it. This test also helps make other developers aware if changes they make cause this type of error again.

In case you use Debug mode in your IDE, you may start getting Mage.Client timeouts (mostly happens in IDEA, ok in Eclipse). To manage that use -Ddebug.mage VM argument when running Mage.Client.

Unit tests from command line

You must build full project first (one time only):

  • Open root folder and run mvn clean install -DskipTests

You must run all tests to create missing database files (one time only):

  • Open root folder and run mvn install

Possible problems:

  • If you catch JDK related error like No compiler is provided in this environment. Perhaps you are running on a JRE rather than a JDK then make sure that you setup environment variable JAVA_HOME to JDK folder

After that you can edit code and run single or multiple tests by commands like:

mvn install test "-Dsurefire.failIfNoSpecifiedTests=false" -Dtest=EquipAbilityTest#testEquipShroud
mvn install test "-Dsurefire.failIfNoSpecifiedTests=false" -Dtest=VerifyCardDataTest#test_showCardInfo

By default xmage's maven runner uses tree view reporter and shows logs from failed tests only. If you need full logs output then use treeViewRunnerShowAllLogs:

mvn install test "-Dsurefire.failIfNoSpecifiedTests=false" "-Dxmage.build.tests.treeViewRunnerShowAllLogs=true" -Dtest=VerifyCardDataTest#test_showCardInfo

Code coverage and static analyse

It's depends on your developer tools. If you need fast results then use jacoco's html report for code coverage and IntelliJ IDEA for code inspect.

Build in support

  1. Code coverage stats by jacoco tools:
  • build and run tests with coverage data: mvn install -Djacoco.skip=false -Dmaven.test.failure.ignore=true
  • it will generates two type of files:
    • exec-files can be used by IntelliJ IDEA in show coverage data dialog (see below): ./module_name/target/jacoco.exec
    • html-report can be used by devs for offline view: ./Mage.Reports/target/site/jacoco-aggregate/index.html shot_231030_084744
  1. Static code analyse by sonar cloud:
  • online version available at https://sonarcloud.io/project/overview?id=magefree_mage
  • core dev can run and upload it to the cloud by mvn -e sonar:sonar -Dsonar.projectKey=magefree_mage -Dsonar.organization=magefree -Dsonar.host.url=https://sonarcloud.io -Dsonar.token=xxx
  • warning, it's require 10+ Gb memory and 15+ hours to process whole analyse: shot_231030_084639

IntelliJ IDEA support

  1. Code coverage by IDE tools - just run test or app with "Run ... with Coverage" command: shot_231030_082216

  2. Code coverage import from jacoco report:

  • first run unit tests with enabled jacoco (see above);
  • open search panel and enter "Show Coverage Data" or <CTRL+ALT+F6>;
  • in dialog box choose exec-files for your module: ./module_name/target/jacoco.exec: shot_231030_082945
  • open search panel and enter "Coverage" to see code coverage panel;
  • open code editor to see coverage stats at the left: shot_231030_085305
  1. Static analyse by IDE:
  • open code editor and click to the right upper corner to see code problems: shot_231030_085501
  • open main menu -> Code -> Inspect code dialog to run same analyse for whole project (warning, it can take some time): shot_231030_090029
Add a custom footer

Home

Release notes (current)

Supported deck formats

Set implementation list

Project Overview

Setting Up your Development Environment

Development Workflow

Development HOWTO Guides

Development Testing Tools

Creating a new release

Mage Updater

Hosting an xMage server

Server configuration

Clone this wiki locally