Set up frontend testing framework

[tobyzerner]

With my limited knowledge, here's what I'm thinking:

• Unit tests for backend classes. I think phpspec is the preferred tool here – we already have one spec thanks to @franzliedke.
• Functional/integration tests for the JSON-API. (Not sure what tool would be best for this?)
• Functional/integration tests for the extension API. (Ditto?)
• Unit tests for front-end utils, components, helpers, and models. (Probably using QUnit?)
• Acceptance tests for the whole thing.

Thoughts?

[franzliedke]

From my perspective, the most important things are unit tests for important classes (which we'll have to go looking for) and acceptance tests (because those cover the other things).

[luceos]

I see there was a help wanted. If you'd like me to contribute with this I'd love to create the tests for the php side of things (laravel, eloquent, user creation etc).

[franzliedke]

Woah, I just played around with Codeception, and acceptance testing for the whole stack amounts to a whole lot of work. Therefore, I'd say we focus on the following for now:

• more PhpSpec unit tests
• integration (?) tests for the front-end components, one-by-one (cause that's where the magic is at)

We probably should add a few acceptance tests for the critical routes (such as registration and posting maybe), but limit it to a few...

What do you think?

[tobyzerner]

Sounds good to me. I'll work on setting up a front-end testing framework, you focus on the backend? :)

[tobyzerner]

What about acceptance testing the JSON-API? That would be easier than acceptance testing our whole stack, and is quite important since our app isn't the only one using the JSON-API.

@luceos Sorry, didn't see your comment above. I'm sure @franzliedke would greatly appreciate your help with backend testing!

[franzliedke]

Yeah, I'll have a look at how we can do JSON-API testing.

@luceos If you ever feel bored enough to help with tests, you could look at testing some of the central base classes, e.g. the Action class. ;)

[luceos]

I'll take a look at that asap @franzliedke , i have more experience with travis though, so you've decided for codeception, I'll read up into that then.

[franzliedke]

Nah, I won't use Codeception for now, that's only for the acceptance tests. For unit testing, we use PhpSpec. Take a look at our first (and only) test. :)

[kirkbushell]

I'm happy to contribute here. I have years of testing experience and really loving working with PHPspec. In fact, I'd be very happy to push forward with and champion the test strategy, beginning with the core library and feature set.

[kirkbushell]

One thing to add here - the dependencies and uses of Laravel in the project prevent phpspec from being used on a number of classes. I tried writing some specs for the database setting repository, and a couple of handlers, it was literally impossible. The lack of dependency injection (such as injecting the event dispatcher instead of using event()) is causing some problems.

Imho this highlights a possible design problem, and certainly a code smell. Having used those functions on other projects, they end up causing more problems than they're worth.

[kirkbushell]

Happy to close this one now? I'll continue to help and push forward with the tests.

[franzliedke]

I think we still need to sort the frontend

[kirkbushell]

Haven't looke at the frontend yet, but if it's written in a modular fashion happy to move ahead with that as well, using jasmine.

[tobyzerner]

Thanks for your work so far @kirkbushell, we're really lucky to have you pushing this forward!

That would be amazing if you lead the charge with the front-end tests too. I have virtually no experience with testing so I don't know how it will fare, but as far as I'm concerned it's pretty modular (split up into components/classes/helper functions). However, some of them rely on an app global... will that be problematic?

In any case, please take a quick look at the front-end codebase when you have a chance and let me know what you think :)

[kirkbushell]

Yup, anything global will be an issue for testing :)

I'll check it out once I get a handle on the server-side testing. Still a lot to do there. Also, we'll need to include tests and how to write them/what's expected as part o the contribution documentation.

[franzliedke]

Moving this back to beta.4 because we still have to sort out the frontend. :)

[tobyzerner]

Thinking about front-end testing... I'm in way over my head, but here are some ideas:

• Write tests in ES6 using Jasmine
  • e.g. import Button from 'flarum/components/Button'
  • Render the component to a staging area via Mithril and make sure DOM output matches component configuration
    • Or should we be mocking the DOM, like Mithril does in its own test suite? https://github.com/lhorie/mithril.js/blob/next/tests/mock.js
  • Fake interactions and make sure they behave as expected
  • Spy on app global/its properties, watching for the correct calls
• Compile tests into ES5 in the same way as the main app, output to the dist folder
• Set up Karma to run dist/tests.js against dist/app.js in PhantomJS or Chrome or whatever

@kirkbushell input?

[tobyzerner]

Another possibility for testing components: https://github.com/StephanHoyer/mithril-query

[franzliedke]

I like what Ember.JS suggests for components: integration tests. Example: https://github.com/openHPI/ember-form-components/blob/master/tests/integration/components/editable-field-test.js

Basically, you render a component, pass in any data you want, and then query the DOM for any changes you want. It's a reasonable and sensile way to test components work as expected; we still need to find a good setup for full acceptance tests, though...

I don't have any knowledge in regards to browser JS testing, though, so I can't make any recommendations on how to achieve that.

[luceos]

Revisiting this, I'm sure that Laravel Dusk can be a big life saver here for the Frontend testing. We would need Laravel 5.4 though.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. We do this to keep the amount of open issues to a manageable minimum.
In any case, thanks for taking an interest in this software and contributing by opening the issue in the first place!