Set up PHP testing framework and write initial tests

[tobyzerner]

When building this foundational prototype, I neglected to learn how to write tests in the interest of time. However, I realise that tests will be extremely important for the longevity of the project, and I am very keen to adopt a test-driven development approach for Flarum.

I personally need to read up properly about how to actually write and use tests in PHP/Laravel. If anyone would be willing to discuss/set up the necessary testing frameworks (again, I'm pretty uneducated as to what is what - unit? behavioural? acceptance?) that would be great. Don’t worry, I will be making an effort to learn soon!

• Set up backend testing framework(s)
• Write tests for current behaviour (domain and API)
• Proceed with test-driven development

[mikedugan]

Codeception is pretty beginner friendly in the testing regards and offers support for unit/functional/acceptance testing (all powered under the hood by PHPUnit).

You might want to look into behavior-driven development, I've found that (unit) test-driven development can be a bit pedantic when you're still discovering and knowledge crunching the domain.

[anuragsoni]

Codeception is pretty good unit testing framework and is also very beginner friendly. I started with codeception when i was teaching myself about unit testing in Laravel apps. I usually try to pair unit testing (for testing modules in an isolated manner) and Integration Testing to test the entire app and the database integration as a whole. I've found that following the repository pattern has made it easy to write test friendly code.

For integration testing i've used a tool called Testdummy.

[tobyzerner]

Starting to dive into this now. Just want to clarify: I should be writing unit and integration tests for the Flarum/Core layer (the domain), and functional tests for the API layer, yes? Will acceptance tests be useful for testing the API (I can't imagine how they would differ from functional tests in this case?)

Also, is it worth considering using a combination of Codeception for functional/acceptance testing and PHPSpec for unit/integration testing? PHPSpec's syntax seems a bit nicer in this regard, although I'm all for keeping the tooling as simple as possible.

@mikedugan @anuragsoni @DavertMik

[tobyzerner]

OK so I've made a start based on https://github.com/Codeception/sample-l4-app and I have a few more questions/thoughts:

• How in-depth should these API tests be? For example, when testing creating a discussion, we need to test validation and everything right? (Or does it belong with the integration tests?) Assuming so, I'm just wondering how the tests should be structured/grouped then, because it seems like DiscussionsResourceCest would get quite big.
• For the API tests, is the $I->haveRecord method of building up data OK, or should I be using factories?
• I was under the impression that Laravel4 module would run all db queries in a transaction, then roll them back at the end of the test, but this doesn't seem to be happening?

[jgknight]

I'd be interested in helping with writing tests. It is what I do in my day to day job - testing software/hardware though I'm not familiar with the Codeception framework.

[tobyzerner]

@jgknight Great! Codeception is really easy to get started with — just take a read through the docs. Any help I can get would be greatly appreciated!

[DavertMik]

@tobscure well, here is my experience on testing EmberJS app + backend.

I have acceptance tests running in Karma (see EmberJS docs). They test all user interactions and user interface. They run much faster than acceptance tests with Selenium, they also work well with async. I don't use Fixtures provider, instead I use real API calls to backend. Sure I perform cleanup after each test (this is done by API calls too).

API is tested with functional tests. I'd recommend to use Laravel4 module.

Unit tests are written as usual. Sure, there should be frontend and backend unit tests.

I was under the impression that Laravel4 module would run all db queries in a transaction, then roll them back at the end of the test, but this doesn't seem to be happening?

Yes, it is supposed to. I don't have an idea why not. If you are running it as part of acceptance testing, Laravel4 module may not connect to database to start a transaction. Anyway, Laravel4 is supposed to handle your primary database connection and turn it into transaction.

Please show me your config, so I could say something for sure

[tobyzerner]

@DavertMik Thanks for the info. I sorted out the transaction issue — turns out a couple of the database tables were MyISAM instead of InnoDB (for fulltext support). Switched them to InnoDB for now, but I guess we'll eventually need fulltext support at least on the posts table...

[DavertMik]

MyISAM is always not a good idea to use. For full text search better to use
Elastic or Sphinx.
If you fear of making a very complicated software, you can get it packed
into Docker.

2015-01-22 2:34 GMT+02:00 Toby Zerner notifications@github.com:

@DavertMik https://github.com/DavertMik Thanks for the info. I sorted
out the transaction issue — turns out a couple of the database tables were
MyISAM instead of InnoDB (for fulltext support). Switched them to InnoDB
for now, but I guess we'll eventually need fulltext support at least on the
posts table...

—
Reply to this email directly or view it on GitHub
#3 (comment).

[qrokodial]

relying on Elastic or Sphinx would remove the ability for this to be hosted on most shared or cloud hosting providers, as they are commonly not included.

[tobyzerner]

Yeah. Of course there will be a range of adapters for search - including MyISAM fulltext, Elastic, Sphinx, etc. I suppose we'll just manually purge the testing database before/after the tests for the fulltext adapter.

[franzliedke]

Once we have tests, this commit should be reverted again so that Travis runs our unit tests on all important PHP versions.

[franzliedke]

Also, for Travis: We need to consider testing against lowest and highest dependencies.

[franzliedke]

I've used PhpSpec to create a first test now. :)

@tobscure Can you set up Travis for the private repo?

[tobyzerner]

@franzliedke Sorry, Travis costs money for private repos :(

[franzliedke]

Ah, true, forgot about that. Never mind me then ;)

[franzliedke]

Also, we can enable the Composer cache.

[tobyzerner]

Closing in favour of flarum/issue-archive#424