Allow running of facts in parallel

[AlexBaranosky]

I haven't spent time thinking about how to do this. However, it would be a nice feature for teams with lots of unit tests.

@marick (when you have time - no rush) would you mind listing the hurdles to achieving this that are in the current implementation of the reporting?

[marick]

Midje's emphasis on treating facts as forms to be evaluated immediately would seem to make this harder.

I could imagine something like having each fact be a future, though that pushes off a lot of work onto the runtime.

The old XPer in me is inclined to say something like "if running your unit tests is taking too long, the solution is not to parallelize: it's to find out why you can't get reasonable assurance that your last small change didn't break something without requiring heroic efforts to run every last check you ever wrote."

[marick]

More on previous: the point of tests is to have intellectual control over what you're doing. My (and I'm not alone!) style is to aim for a sequence of small changes to code. What does it say about things like coupling or test coverage if a single small change to the code could break something that'd only be detected by running a large test suit?

Yes, it's wise to run all the tests every so often. But if you're dependent on running so many so often (or if each one is so slow you can't run many), you're doing it wrong.

[AlexBaranosky]

In general I agree. I might lean a little further in the direction of "if
the people wanteth, then it wouldn't hurteth", and think it might be good
for the design in general to not be coupled to time (a big 'might', I
admit).

If we ever snag some more contributors maybe this could make sense to do,
but until then there are more important chances I've got in mind.

On Fri, Feb 24, 2012 at 6:53 PM, Brian Marick <
reply@reply.github.com

wrote:

More on previous: the point of tests is to have intellectual control over
what you're doing. My (and I'm not alone!) style is to aim for a sequence
of small changes to code. What does it say about things like coupling or
test coverage if a single small change to the code could break something
that'd only be detected by running a large test suit?

Yes, it's wise to run all the tests every so often. But if you're
dependent on running so many so often (or if each one is so slow you can't
run many), you're doing it wrong.

---

Reply to this email directly or view it on GitHub:
#108 (comment)

[arohner]

IMO, facts should be functions. Several reasons

1. it makes parallelization possible

2. It makes other features, such as tags, easier (Feature request: tag tests with metadata, run all tests matching/not matching metadata #71)

3. it improves the interactive repl experience

Tags is an important feature for me. The best way to test code is not always with a unit test. Some tests are better expressed as integration tests, or selenium tests, or require interacting with 3rd party services. All of these take time, and can't be unit tested.

As to 3) I have emacs midje mode, and use it constantly, but having (run-tests 'my-ns) would still improve my REPL experience. I understand I can do (require 'my-ns :reload), but that's less pleasant.

As for running the whole test suite, I'm human. Sometimes even when I am making a series of localized changes, it can still have effects on tests in other namespaces that I should have remembered to run, but didn't.

The existing user semantics don't need to be changed to allow this. Have the current fact macro expand to something like:

(fact ...) => (let [f (fn [] (existing-fact-expansion))]
                      (def fn-name f)
                      (when *should-run-facts*
                        (f))

The def is important, because then I can attach metadata to the fact, and write a test runner that says "Go run all selenium tests", or "don't run the tests that cost money to run".

[marick]

On Feb 24, 2012, at 9:04 PM, Allen Rohner wrote:

(fact ...) => (let [f (fn )](def fn-name f)
(when should-run-facts
(f))

Where does the fn-name come from?

---

Brian Marick, Artisanal Labrador
Now working at http://path11.com
Contract programming in Ruby and Clojure
Occasional consulting on Agile

[arohner]

We can generate a name from the fact description. Strip out all characters that aren't legal in a var name, and then add gensym.

(fact "here is a simple test") => (gensym "here-is-a-simple-test") => here-is-a-simple-test371

[AlexBaranosky]

gen-sym?

Perhaps all we really need is the metadata (aka tags) on the fn, so having a gen-sym'd name could work.

[marick]

On Feb 25, 2012, at 4:10 PM, Allen Rohner wrote:

We can generate a name from the fact description. Strip out all characters that aren't legal in a var name, and then add gensym.

(fact "here is a simple test") => (gensym "here-is-a-simple-test") => here-is-a-simple-test371

Oh, I get it: you don't want the var for human use, but only so there's some handle upon which to hang metadata.

There's already namespace-specific (and also global I think) Midje-specific state. Maybe there should be some sort of registry that tools could query.

---

Brian Marick, Artisanal Labrador
Now working at http://path11.com
Contract programming in Ruby and Clojure
Occasional consulting on Agile

[marick]

This won't happen without someone stepping forward to do major work.