Schema validation quits on first error

[victorandree]

Given a simple schema:

schema = Schema({
    'name': And(str, lambda s: len(s) <= 50),
    'email': And(str, lambda s: '@' in s)
})

If I submit this with bad data for both fields, only one error will be reported:

>>> schema.validate({'name': 'Too Long Name', 'email': "invalid email address"})
SchemaError: key 'name is required'
<lambda>('Too Long Name') should evaluate to True

I understand that this might be intentional, but in my opinion it hampers the user experience since they have to resubmit and resubmit. Is there a way to get "all" validation errors at once?

[keleshev]

This is by design. I agree, schema needs to be changed to allow this. Do you have any implementation ideas?

All in all, do you like the error-handling API? Things like And(..., error='...') and the fact that it raises exception and that you need to do get exc.autos and exc.errors?

[victorandree]

From a quick glance at schema.py, I'd guess that the except/pass at https://github.com/halst/schema/blob/master/schema.py#L106 would be a suitable spot to collect the validation errors in a dict with keys for each field. Then, instead of raising on lines 115-119, you could do the required check and add the result (failure) to the dict. Repeat for all fields.

Then at the end, you see what you got and raise with the dictionary of validation errors if appropriate. The exception you get would have to make it possible to see what field was incorrect.

I appreciate the error handling API's simplicity. Some things I noticed when playing around a bit:

• It would be nice to know where a SchemaError came from; right now we don't know which part of the schema caused the error, it's just the data that failed to validate with possibly a user-friendly message about why.

• A wrapping function might be useful (but is trivial to implement yourself):

  def is_valid(schema, data):
      try:
          data = schema.validate(data)
      except SchemaError as e:
          return e
      else:
          return data
  

• I think the documentation could be a bit clearer, perhaps by providing a more complicated example. Would this, for example, be a good way of doing something a bit more complicated?

  # could arguably be just And()?
  username = Schema(
      And(
          lambda s: 8 <= len(s) < 16,
          lambda s: s.isalpha()
      ),
      error="username must be 8-16 characters in length, and [A-Za-z] only"
  )
  
  # Is there any real difference between Use and Schema here?
  email = Schema(
      lambda s: '@' in s,
      error="invalid e-mail address"
  )
  
  user_id = Schema(
      lambda n: 0 <= n <= 50000,
      error="valid user ids are in the range 0-50000"
  )
  
  # Or you could/should put most of this in the "external" definition, and just have "username": username
  schema = Schema({
      'id': And(Use(int), user_id),
      'username': And(str, username),
      'email': And(str, email),
  })
  

I know many larger validation libraries provide a lot of built-in validators with very granular error messages. It doesn't seem like schema is meant to provide lots of built-in validators (some other project could easily create such a library). However, it might be useful to show how one could build more complicated validation. Is is possible to implement a "confirm password field must match password" validation, for example?

[keleshev]

Maybe I can steal something more from voluptous, for example, how it constructs error-messages like:

 required key not provided @ data[0]['username']

Also, now that I look at voluptous I can't remember why the hell did I make another schema-validation library.

So, collecting errors in a dict is probably not the best idea, since the data could be anything, not just a dict.

A wrapping function might be useful

I never saw an exception being returned from a function. What is the benefit of it?

Is there any real difference between Use and Schema here?

Use applies a callable to a data to get the return value, where Schema checks if callable returns True or not.

You are very welcome to make a pull request with those examples into README, also you can try rewording things in README, if you think you can improve it.

a lot of built-in validators

I actually don't mind. It just happens I can validate all my use-cases with lambdas and types. (My use-cases are mostly docopt and yaml/json).

confirm password field must match password

Do you mean something like that?

lambda password: re.match('^[0-9a-zA-Z]{6,50}$', password)

Or maybe something database-related? Example with imaginary ORM:

lambda password: Users.objects.where(password_hash=sha1(password).hexdigest())

[victorandree]

So, collecting errors in a dict is probably not the best idea, since the data could be anything, not just a dict.

Well, the data could be, but the error messages are probably strings? Or do you mean that you might have nested things?

I never saw an exception being returned from a function. What is the benefit of it?

Nevermind :)

I actually don't mind. It just happens I can validate all my use-cases with lambdas and types. (My use-cases are mostly docopt and yaml/json).

Yes, I appreciate that! I came upon this project when looking to validate data for a rather big project, with SQLAlchemy models. It looks very nice in how it succinctly describes what's valid and what's not.

You are very welcome to make a pull request with those examples into README, also you can try rewording things in README, if you think you can improve it.

Maybe I will :)

confirm password field must match password

No, I mean if you have a user registration form and the user has to type in their password twice, to make sure that it's not mistyped. In general, is it possible to validate fields "together"? For example, if "a" is provided, "b" must also be provided and so forth (quite common in command line utilities).

[keleshev]

Well, the data could be, but the error messages are probably strings? Or do you mean that you might have nested things?

Yes, how do you see such a dictionary, say, for a list of dicts?

In general, is it possible to validate fields "together"?

Ah! Interesting... Need to think of API for that.

If you find yourself using the same validators again and again, I will consider including them

[charlax]

I might work on this as I need it. Any new idea on the implementation? Otherwise I'll try something.

[keleshev]

@charlax nice, go ahead!

[Firemango]

Did anyone ever come up with a fix or workaround that is plausible?

[jason-curtis]

Not with Schema. Note that voluptuous does have a decent pattern for this where a MultipleInvalid exception can wrap multiple Invalids.

[victorandree]

I doubt this is going to happen by now. As the author of the issue, I'm closing it.