Validation in Scala Part 1: The return type candidates

Introduction

In every project you need validation of user input, scala projects are no exception. In Java some people use Bean Validation to annotate their objects and have them validated by a validator instance.
So some months ago we also used Java Bean Validation in our Finagle based Scala project.
Unfortunately this wasn't a good choice for our distributed service architecture. We're using several services to do validations and we had to wrap these calls in Futures and had to "await" them inside our custom BeanValidations which causes threads to be blocked. Not a very reactive way for today!

Also the BeanValidation API is somehow not so nice to use, e.g. it forces you to annotate classes and we needed more flexibility than we expected, so to make a long story short: We moved away from it.

Now I've tried to move to a more functional and flexible solution, inspired by excellent posts like http://blog.lunatech.com/2012/03/02/validation-scala

What to use?

Keep it simple, use simple validation methods

We're back using simple validation methods like

  def validateUserRegistration(request: UserRegistration)

So the interesting part here is the return type to be used for returning either a success or a failure(s).

Here's my current choice list

• Option
• \/ (Scalaz Either)
• Validation (also from Scalaz)
• ValidationNel (a specialized Validation)
  

We will now look into them in a little bit more detail.

Option

Option is the simplest solution, it may contain either a None (representing e.g. something like "NotFound) or a Some(x) for the Success case. You may be tempted to use a Some(error) and a None for expressing "No failure" but I always regard this a little bit of irritating. Later in the conversion section we'll see that I'm not alone with this.
The nice thing is that Option is a Monad, you may use it in a for comprehensions like this:
You'll get either a Some if every step contains a Some or a None if any of the Options used in the for comprehensions are a None.

 

\/ (Scalaz's Either implementation)

As I already explained in another post we make use of Scalaz Either (written as \/ ), instead of the plain Scala Either, because you can combine them with other Monads (like Twitter futures) using Monad Transformers.

Besides that an \/ instance is either a Left (per convention) the bad case or a Right (per convention) the good case.
You create them like this:
Little bit funny at first that with ".left" and ".right" you'll specify the other type (right type for a ".left", left type for a ".right"), but it makes sense because you have to help the compiler with the non-present type, doesn't it?

Validation

Validation is pretty much like EitherZ but Success and Failure is not a convention any more but is expressed in the subtypes: Failure and Success.
Here's an example creating them:

ValidationNel

"Nel" stands for NonEmptyList, as the scaladoc says: A singly-linked list that is guaranteed to be non-empty. Why is this important?
Imagine the following:

  def registerUser(name: String,...): Validation[List[RegisterUserFailures], User]

The returned “List” of failures may be possibly empty with this signature. This is irritating, because what does this mean? So Scalaz helps us with a simple but nice trick:

Conversions between the above types

The nice thing that you may choose whatever best fitting type of the above for you validation method. If you later on compose several validation methods which use different return types of the above you may easily convert between them and use the best fitting type in your aggregation method:

The "fold" from Option to \/ is one of several ways to "enrich" the missing Left information in an Option. If you go back from \/ to Option with "toOption" the Left information from a -\/ is simply dropped and "mapped" to a None. So like I said above: "None" of an Option is usually considered an error case (without detailed information). The toOption method of \/ has the same viewpoint here.

In Part 2 of this blog we'll have a look at fail-fast strategy for validation or aggregating several validation failures.

Using Functors or How to use implicit conversions inside containers

Since I work with Scala in my current project, using implicit conversions is quite nice to do the usual mapping stuff 'behind the scenes', for example to map between the external representation of a time value (in thrift) and the internal one using Joda DateTime

Using this just requires it to bring it in scope, e.g. with an import:

Some weeks ago a colleague asked me why the Scala compiler is so stupid not to apply a implicit conversion in the following case:

But how should the compiler now how to convert an Option[JodaDateTime] to Option[ThriftDateTime]? Yes, there is a map() method in Option, but this is just some kind of usual convention, and it would be really dangerous if the scala compiler would automatically use a method map() if present.

At this time I just suggested to explicitly map or use an implicit conversion on Option:
But this is really cumbersome and you have to do it also for lists or any other container type, even if they all have a common map method with the same semantic.

Functors

It turns out that there is a functional class which describes the above situation, they are called functors.
A functor is simply a trait which specifies to supply a map() method with adheres to the functor laws.

The above functor definition is directly taken from Scalaz 7.
Using it for an implicit "container" conversion of our two classes means:


With that the conversions work in all "containers" for which scalaz defines functors:

..and if you want you can write your own functors for your classes.

Using Futures together with Either or My way to scalaz

Intro

We're using Twitter's Finagle stack in my current scala project. Finagle is Future based and the Future concept makes usage of exceptions easy. Using Exceptions for "infrastructural" problems (like when the network or the database is down) is perfectly ok, in Java you would use RuntimeExceptions for this which don't have to show up in the method signature. In Scala there's no difference any more between checked exceptions and runtime exceptions. In Java checked exceptions are often used for "business" exceptions, i.e. exceptions that steer the way your business logic works. For example, some months ago in Java projects I've often written API methods like this:

So the possible return types of this method is either a ArticleDetails object containing the information about the article or an Exception saying that the article could not be found.

Pros

• Method signature tells you everything about the return types

Cons

• Two different ways of returning information
• return new ArticleDetails(...)
• throw new ArticleNotFoundException(...) 
• This also means two different styles of handling this information at the caller side

Scala

In Scala there are no checked exceptions any more, so migrating the above java interface to Scala and still use an exception for the failure case would result in the following:
The method signature doesn't tell you anymore that the method could produce a separate exception output. You could use scaladoc or @throws to document this, but the compiler doesn't check this and this probably leads (after some evolution cycles of your class) to a method signature that doesn't document anymore what it is returning in case of expected business flow failures.

Either to the rescue

In Scala there's a more functional way to say what the method is returning: Either

The above method either returns an ArticleNotFound or an ArticleDetails object.
An Either has a left side, which is used per convention for the failure case or a right side, which represents the "right", the success case.

Using Futures 

As I said we're using Twitter Futures for our code, so if we're doing some blocking method call, we do it async and return a Future[RETURN_OBJECT] instead of the simple method return object to keep the current thread from being blocked. So the above method blocks because of a call to MySQL and would therefore look like this:

For Comprehensions

So far everything looks good, and it seems pretty easy to get away from business exceptions. But imagine now a second method which returns an articleId for an unique article name:

If someone has only the unique name and wants to retrieve the article details I would naively use the following for comprehension and would expect Left's (Failures) to fail fast and "bubble" out of the for comprehension:

Unfortunately the above doesn't compile. What we need is a way to compose the Future with Either. At this point I was very lost and asked for help in this stackoverflow question. Fortunately the help came at the Scala User Group Munich where Lars Hupel helped me a lot with this.

Monad Transformers or "My way to scalaz"

The way out of this problem is a Monad transformer. A simple mental model for  a Monad in Scala is a class which implements map, flatMap and withFilter and can therefore be used in a for comprehension.
Either and Futures are Monads and Monads don't compose out of the box together. There's even not a guarentee that mondas can be combined.

My first problem was now that scala.util.Either is not even a Monad. Only it's right or left protection is a Monad! Which makes it a little bit cumbersome to use. You'll have to say always .right in for comprehensions.

As I found out that scalaz has also an Either called \/ which is right biased and not neutral like scala.util.Either. The best: It has a transformer class EitherT to combine it with other Monads.

Converting the above example to \/ leads us to:

Pretty similar to the solution with Scala's Either, but without .right's, because of it's right biasing like in Haskell. To use EitherT with Twitter Futures we now only need to define the following implicit vals:

Then we can use EitherT together with Twitter Futures in our for comprehension:
The run method turns the resulting EitherT back to a Future[\/]. A EitherT in this case is an EitherT[Future, ArticleNotFound.type, ArticleDetails], so this could be used as alternate return type. Then you can omit the run call.

Here's the final complete example:

If a left happens on one of the calls in the for comprehension, it bubbles out in fail fast strategy. If there are only Right's than the article is yielded.

Slick in Scala: Combining multiple fields in one mapping case class field

Yesterday I've had a problem with an existing database schema. In one table there were several Boolean columns each expressing the existence of an user role.

Since we use Slick for MySQL access in our services, I wanted to combine these several columns in one single (enumeration) field of the mapping case class:
This can be achieved by providing your own constructor and extractor functions for User objects to Slick using the <> function on the * projection. Something like this:
Be sure that your own constructor and extractor functions have the apply/unapply style, e.g. extractUser should return an Option of a tuple, if it does not (extractUser returns a tuple instead of an option of a tuple) you get a hard-to-read error message from the compiler:

overloaded method value <> with alternatives:
  [R(in method <>)...
    userModeratorMajor ~ partner ~ premiumPartner ~ corporatePaid <> (constructUser _, extractUser _)

Thanks Mr. Vogt from Typesafe for the hint.