Developing for the web

Developing for the web

In this chapter, we recapitulate the web-specific constructions of the Opa language, including management of XML and XHTML, CSS, but also client-server security.

Syntax extensions

The syntax of expression is extended by the following rules:

expr ::=
| <xhtml>
| <ip>
| <id>
| css <css>
| css { <style-expr> }
| <action-list>
| <xml-parser>

action-list ::=
| [ <action>* sep , ]

action ::=
| <action-selector> <action-verb> <expr>

action-selector ::=
| <action-selector-head> <action-selector-property>?

action-selector-head ::=
| . <ident>
| . { <expr> }
| <id>

action-selector-property ::=
| -> <ident>
| -> { <expr> }

ip ::=
| <byte> . <byte> . <byte> . <byte>

id ::=
| # <ident>
| # { <expr> }

xhtml ::=
| <> <xhtml-content>* </>
| < <xhtml-tag> <xhtml-attribute>* />
| < <xhtml-tag> <xhtml-attribute>* > <xhtml-content>* </ <xhtml-tag>? >

xhtml-content ::=
| <xhtml>
| <xhtml-text>
| { <expr> }

xhtml-tag ::=
| <xhtml-name> : <xhtml-name>
| <xhtml-name>

xhtml-attribute ::=
| style = <xhtml-style>
| class = <xhtml-class>
| on{click,hover,...} =
| options:on{click,hover,...} =
| <xhtml-tag> = <xhtml-attr-value>

xhtml-attr-value ::=
| <string-literal>
| <single-quote-string-literal>
| { <expr> }
| <id>

xhtml-style ::=
| " <style-expr> "
| { <expr> }

xhtml-class ::=
| ' <xhtml-name>* '
| " <xhtml-name>* "
| <xhtml-name>
| { <expr> }

style-expr ::=
| ...

css ::=
| ...

xml-parser ::=
| xml_parser <xml-parser-rules>

xml-parser-rules ::=
| |? <xml-parser-rule>* sep | end?

xml-parser-rule ::=
| <xml-named-pattern>+ -> <expr>

xml-named-pattern ::=
| <ident> <xml-parser-suffix>?
| <ident> = <xml-pattern> <xml-parser-suffix>?
| parser? <parser-prod>+

xml-pattern ::=
| < <xhtml-tag> <xml-pattern-attribute>* />
| < <xhtml-tag> <xml-pattern-attribute>* > <xml-named-pattern>* </ <xhtml-tag> >
| _
| { <expr> }
| ( <xml-parser-rules> )

xml-pattern-attribute ::=
| <xhtml-tag> <xml-pattern-attribute-rhs>?

xml-pattern-attribute-rhs ::=
| <xml-pattern-attribute-value>
| ( <xml-pattern-attribute-value> as <ident> )

xml-pattern-attribute-value ::=
| <string-literal>
| { <expr> }
| _

xml-parser-suffix ::=
| ?
| +
| *
| { <expr> }
| { <expr> , <expr> }

Additionaly, some more directives are available.

And finally, the identifiers server and css have a special role at toplevel.

Xhtml, xml

Although xhtml and xml is not a built-in datastructure, there is a shorthand syntax for building it.

div = <div class="something">Hey</div>

which is a shorthand for the following structure:

(xhtml)
  { namespace: Xhtml.ns_uri
  , tag: "div"
  , args: []
  , specific_attributes: {
      some: {
        class: ["something"],
        style: [],
        events: [],
        events_options: [],
        href: {none}
      }
    }
  , content: [{ text: "Hey" }]
  }

The syntax only allows to build xhtml (not html, so there is no implicit closing of tags). On the other end, it is allowed to close any tag with the empty tag:

div = <div class="something">Hey</>

You can build fragments of xhtml with an empty tag:

<> First piece <div class="something">Hey</div> Third piece </>

Just like you can insert expressions into strings, you can insert expressions into xhtml. Here is the previous examples rewritten with insertions:

string1 = "First piece"
div = <div class={["something"]}>Hey</div>
string2 = "Third piece"
<> {string1} {div} {string2} </>

**NOTE: **

You can only insert expressions in the content of a tag or the value of expressions, you cannot insert
expressions instead of a tag name for instance.

tag = "div"
some_xhtml = <{tag}>Hello</> // NOT_VALID

In xhtml literals, the usual comments /* */ and // do not work anymore. Instead, you have xhtml comments <!-- -->, which really are comments and not a structure representing comments (so these comments will never appear at runtime, you cannot send them to the client etc.).

There are a few attributes that are dealt with specially in xhtml literals.

Style

The value associated to the style attribute should be of type Css.properties. For convenience, it can be written as a string, just like you would in html files, but it is not a string and it will be parsed.

Class

The value associated to the class attribute has a similar behaviour to the one of style. Its value has the type list(string) but it can be written as a string, that is going to be parsed.

Onclick, ondblclick, onmouseup, ...

The full list is actually the type of Dom.even.kind. The value of this attribute is a FunAction.t, ie Dom.event -> void.

Options:onclick, options:ondblclick, ...

The value of this attribute is a list(Dom.event_option).

Namespaces

By default, when you use the syntax for xml literals, you actually build xhtml (the default namespace is the one of xhtml, some attributes have special meaning, etc.). This can be disabled by putting a @xml around an xhtml literal.

_ = @xml(<example attr="value"/>)

You can build xml with any namespace, not necessarily only the empty one or the one of xhtml.

some_soap = @xml(
 <soap:Envelope
     xmlns:soap="http://www.w3.org/2001/12/soap-envelope"
     soap:encodingStyle="http://www.w3.org/2001/12/soap-encoding">
   <soap:Body xmlns:m="http://www.example.org/stock">
     <m:hello>Hello world</m:hello>
   </soap:Body>
 </soap:Envelope>
)

When you define a namespace with a xmlns attribute, the namespace is defined in the current litteral only. If you insert a piece of html, it must also defines the namespace:

body = @xml(
   <soap:Body xmlns:m="http://www.example.org/stock">
     <m:hello>Hello world</m:hello>
   </soap:Body>
) // this is not valid because the namespace soap is not in scope
some_soap = @xml(
 <soap:Envelope
     xmlns:soap="http://www.w3.org/2001/12/soap-envelope"
     soap:encodingStyle="http://www.w3.org/2001/12/soap-encoding">
   {body}
 </soap:Envelope>
)

To solve this problem and to factorize the namespaces, you can name them with a normal binding:

`xmlns:soap`="http://www.w3.org/2001/12/soap-envelope"
body = @xml(
   <soap:Body xmlns:m="http://www.example.org/stock">
     <m:hello>Hello world</m:hello>
   </soap:Body>
) // this not valid because `xmlns:soap` is in scope
some_soap = @xml(
 <soap:Envelope
     soap:encodingStyle="http://www.w3.org/2001/12/soap-encoding">
   {body}
 </soap:Envelope>
)

Parser

Xml is often used as an intermediate structure, and as such, it should be convenient to transform it into a less generic structure.

Opa features a special pattern matching like constructs that is meant for this:

my_parser = xml_parser {
    case <x>parser valx=Rule.integer</><y>parser valy=Rule.integer</>: {x:valx, y:valy}
}

It defines a parser that transforms xml such as

<x>12</x><y>13</y>

into

{x:12, y:13}

This syntax resembles the one of pattern matching, and it acts the same: The xml begin parsed is matched against the patterns in order until a match is found. The right hand side of the pattern is then executed.

We detail here how xml_parser patterns behave:

• One rule is composed of a list of patterns. Any xml matches the empty list of patterns. Note that the xml doesn't have to be empty. An xml matches a non empty list of patterns if the xml matches the head pattern and its siblings (if it is a fragment, or the empty fragment otherwise) match the remaining patterns.
• An xml matches the pattern parser <parser-prod>+ if its first node is a text node that is accepted by the parser. The bindings done in the parser are in the scope of the action.
• An xml matches the pattern x=<subrule> when the xml matches the and the result of is then bound to x.
• An xml matches the pattern x<suffix> when the xml matches the pattern x=_<suffix>
• An xml always matches the subrule <subrule>*. As many nodes as possible are matched, and the list of results returned by those matched is returned.
• An xml matches the subrule <subrule>+ when the xml matches at least once. As many nodes as possible are matched, and the list of results returned by those matched is returned.
• An xml always matches the subrule <subrule>?. A node is matched if possible, and its result wrapped into an option is returned. If no node is matched, {none} is returned.
• An xml matches the subrule <subrule>{min,max} when the xml matches at least min times. As many nodes as possible (but stopping at max) are matched, and the list the results of these matches is returned.
• An xml matches the subrule <subrule>{number} when it matches <subrule>{number,number}.
• An xml matches the subrule _ if it contains at least a node. This node is then returned.
• An xml matches the subrule {<expr>} when the calling the xml_parser accepts the xml. In that case, the result of the xml_parser is returned.
• An xml matches the subrule <tag attributes>content</> when the xml begins with a tag node and ** the tag of the node match the tag of the pattern ** the attributes of the node match the attributes of the pattern ** the content of the node match the content of the pattern
• A tag matches a tag of a pattern when they have the same namespace and the same name.
• A list of attributes matches a list of attributes pattern when each attribute ** has no counterpart in the list of attribute pattern ** or has a counterpart in the attribute pattern list and their value match (a counterpart being an attribute with the same namespace and name)
• An attribute value always matches _
• An attribute value always matches the absence of value pattern (<tag attr=1> matches the pattern <tag attr/>). The value of the attribute is bound to name of the attribute.
• An attribute value matches { <expr> } when the parser <expr> accepts the value. The return value is bound to the name of the attribute.
• An attribute value matches <string-literal> when it matches { Parser.of_string(<string-literal>) }. The return value is bound to the name of the attribute.
• An attribute value matches (<value> as <ident>) when it matches . In that case, the value of the matching is bound to <ident> instead of being bound to the name of the attribute.

**IMPORTANT: **

All whitespace only text nodes are discarded during parsing.

xml = @xml(<>"  "</>)
p = xml_parser {
      case parser .*: {}
    }

`p` would fail at parsing `xml`, because it behaves exactly as if `xml` were defined as

xml = @xml(<></>)

Ip address

You can write constant ip addresses with the usual syntax:

127.0.0.1

This expression has type IPv4.ip.

Directives

The set of expression directives is extended by the following directives:

• @sliced_expr :: Takes one static record with the field server and client (containing arbitrary expressions). Meaning described in the client-server distribution.
• @xml :: Takes an xml literal. The default namespace in the literal in the empty uri, not the xhtml uri.

The set of binding directives is also extended:

web_binding_directives

• public ::
• private ::
• both ::
• client ::
• server ::
• exposed ::
• protected ::
• async :: //Takes no argument, appear at the toplevel or inside toplevel modules. Meaning described in the client-server distribution.
• serializer :: Takes a typename. Overrides the generic serialization and deserialization for the given type with the pair of functions annotated.
• comparator ::
• stringifier ::
• xmlizer :: Takes a typename. Overrides the generic transformtion to xml for the given type with the function annotated.

Css

Opa allows you to define css (as a datastructure) using the syntax of css:

mycss = css
body {
  background: white none repeat scroll top left;
  color: #4D4D4D;
}

Now this is just a datastruture that you can manipulate like any other. To actually serve it to the clients, you need to register it, which is done by defining a variable with name css at toplevel.

css = [my_css]

The right hand side should be of type list(Css.declaration).

One exception is that it is allowed to say simply:

css = css
body {
  background: white none repeat scroll top left;
  color: #4D4D4D;
}

The right hand side of css is of type Css.declaration, but in the special case when it is a literal, it gets automatically promoted to a list of one element. It allows for a slightly lighter syntax when the css of your application is defined in one block.

To define css in opa, you simply declare a variable with name css at toplevel.

The style attribute of xhtml constructs is also parsed specially. Its content looks like a string but is actually a structure.

<div style="top: 0px; left: 29px; position: absolute; ">

This structure can be built in expression, you do not need a style attribute to build it:

css { top: 0px; left: 29px; position: absolute; }

The previous div was simply a shorthand for:

<div style={ css { top: 0px; left: 29px; position: absolute; } }>

Defining servers

The way to define a server in Opa is by mean of the function Server.start, where the first argument is the server configuration and the second one is a handler for the URLs (with many possible variants).

Server.start(Server.http,
  { title: "Hello"
  , page: function() { <h1>Hello World</h1> }
  }
)

Id

You can use

id = #main // as a shortcut to Dom.select_id("main")
// or equivalently
id = #{"main"} // you can of course put an arbitrary expression
               // (of type string) inside the curly braces

This syntax can also be used in xhtml attributes values:

html = <div id=#main>some text</>

which is really a way of saying

html = <div id="main">some text</>

except that the syntax makes it clear what the string will be used for since the definition and usage of an id share the same syntax.

Actions

Opa features list of actions, which is a small dsl to transform the dom conveniently. Note the syntax introduced below is really a structure, it does not execute anything. Dom.transform must be applied to a list of actions for the actions to be performed (so that you can build them of the server).

An action lists is just a list of actions, inside square braces and separated by commas (just like a list, except that it contains actions and not expressions). An action consist in a selector, a verb and an expression. The selector can be one of:

.some_static_class_name,
.{some_dynamic_class_name},
#some_static_id,
#{some_dynamic_id}

followed optionally by:

-> css // to select the style property
-> some_property // to select any static property (like style, value, etc.)
-> {e} // to select any dynamic property

The verb is either = for setting the value, =+ for appending to the value or += for prepending to the value.

The expression is simply the value that will be set, appended to prepended to whatever is selected. When the selector is not followed by ->, the expression should be convertible to xhtml. When the selector is followed by -> value, the expression should be convertible to string. In all other cases, the expression must have type string.

Here is an instance of an action list, that replaces the content of the element pointed to by #show_message by a fragment of html.

Dom.transform([#show_messages = <>{failure}</>])

If the list of actions to perform contains only one single element then the Dom.transform application can be ommitted and the above can be replaced with simple

#show_messages = <>{failure}</>

Client-server distribution --------------------------

This section details the distribution between client and server.

Slicing

Opa is a language that can be executed both on the client and on the server, but at some point during the compilation process, it must be decided on which side does the code actually ends up, and where there are remote calls.

This is the job of the slicer.

The slicer can put each toplevel declaration (or component of a toplevel module) either on the server, or on the client, or on both sides. The slicer will not divide the code at a finer (than a per-function) granularity.

The slicer can be told where a declaration should end up with the slicing annotations put before the function keyword:

• server :: The declaration is on the server (but it does not mean that it will not be visible by the client)
• client :: The declaration is on the client (but it does not mean that it will not be visible by the server)
• both :: The declaration is on both sides. Because a declaration can do arbitrary side effects, there are two possible meanings: either the side effect is executed on both sides or the side effect is executed once (on the server) and the resulting value is shared between the two sides.

By default, the slicer duplicate some side effects (printing for instance) and avoids to duplicate allocation of mutable structures.

For instance:

do println("Hello")

will print "Hello" at toplevel on both sides. On the other hand

s = Session.make(...)

will create one unique session shared between the client and the server.

• both_implem :: This directive behaves the same way as both, except that it explicitely forces the slicer to duplicate the declaration on both sides:

  both_implem s = Session.make(...)

This will create a session at the startup of the server and a session in each client.

Slicing annotations are not mandatory. When they are left out, the slicer decides where to place declarations: on both sides whenever it is possible, or on the only possible side when it has to.

When a slicing annotation is put on a (toplevel) module, it becomes the default slicing annotation for its components (and can be overriden by annotating the component with another annotation).

Now since everything cannot be executed on both sides, there are additional rules. Primitives that are defined on one side can only be placed on this side. When a primitive is server only, not only it is placed on the server, but it is implicitely tagged as server private, with the consequences explained below.

Whenever a declaration is tagged as server private, it cannot be called by the client (it is a slicing error), and any declaration using the current declaration becomes server private itself. Since the tag server private propagate, there is a directive to stop the propagation: it essentially says that a declaration is now visible by the client (possibly after some authentication mechanism, checking the input, or simply because you have a server only primitive that does not really need to be private to begin with). Note that a declaration that is server private can not be called by the client but can nevertheless call the client.

The relevant directive is:

• publish :: Stops the propagation of the server private tags. Note that the declaration annotated as publish are not the only entry point of the server: in server f(x) = x, f can be called from the client.

Finally, sometimes you will want to have a different behaviour on the server and on the client. This can be done, fairly simply, with @sliced_expr:

side = @sliced_expr({server: "server", client: "client"})
do println(side)

This will print "server" on the server and "client" on the client.

• @sliced_expr :: This is simply a static switch between client and server. It can appear at any place in an expression.

The dependencies of the code are not analysed. As a consequence, trying to call the client from the server at the toplevel will slice correctly but will generate a runtime error (because there is no client yet, the server has not even been started yet).

Serialization

Any native opa value can be serialized: integers, floats, strings, records and functions.

Naturally, integers, floats, strings and records are copied when they are send to the other side. Since these structure are not mutable, this duplication is not observable.

Function serialization can be done in two ways:

• either the side receiving the serialized function builds a function that will make the remote call when applied
• or the side receiving the serialized function actually already has this function in its code and can call the local function instead of the remote function

The only remaining types are external types. External types are not really serialized unless explicit serialization/deserialization functions are defined (with @serializer; see here). The default serialization generates an identifier and sends this identifier instead. When the side that generated the identifier unserializes it, it puts back the original structure in its place. The only thing that is not possible in this design is to manipulate an external type from a side where it was not created. As a consequence, if an external type can be manipulated by primitives from both sides, then explicit serialization and deserialization functions must be given.

Including external files in an Opa server

The syntax defines two directives for including the contents of one file or the resources of one file: syntax_keyword_static

• @static_content("foo.png") is replaced by a function that returns the content of compile-time foo.png;
• @static_resource("foo.png") is replaced by a resource foo.png -- with the appropriate last modification time, mime type, etc

Both directives support an additional argument for pre-processing the contents of the file before returning it.

Both directives have a counterpart that, instead of processing and returning one file, process a directory and return it as a stringmap: syntax_keyword_static_directory

• @static_content_directory("foo/") is replaced by a stringmap from file name to functions that maps key to the equivalent of @static_content(key). Of course, this stringmap is evaluated only once;
• @static_resource_directory is replaced by a stringmap from file name to functions that maps key to the equivalent of @static_resource(key). Here, too, the stringmap is evaluated only once.

Again, these directives support an additional argument for pre-processing the contents of the file before returning it.

Examples

The two typical scenarios are embedding one resource:

handler = parser {
  case "/favicon.ico": @static_resource("img/favicon.ico")
  case "/favicon.gif": @static_resource("img/favicon.gif")
}

and embedding many resources:

resources = @static_resource_directory("resources")
urls      = parser {
              case "/": start
              case resource={Server.resource_map(resources)}: resource
            }
Server.start(Server.http, {custom: urls})

For more details on parsers, see the related section. For more details on Server.resource_map, see the library documentation.

Relative paths are understood as starting from the project root.

See also

Resources embedded with these directives support runtime modification for debugging purposes. For more details, see the related section.

Runtime behavior

Release mode

Now, chances are that we want to secure these resources, e.g. to ensure that nobody will replace the nice MLstate logo with a not-quite-as-nice competitor logo. For this purpose, it is sufficient to compile your packages in release mode. A resource embedded by a package compiled in release mode is locked safely and can neither be dumped nor reloaded into the application by using --debug-* . Performance notes

All these directives are fast. Typically, @static_resource or @static_resource_directory will take a few milliseconds at start-up to determine whether they are executed in debug or non-debug mode, and there is no runtime performance loss in non-debug mode. When building resources, prefer these directives to @static_content or @static_content_directory are generally faster, as the final result will be a tad faster with @static_resource_*.

These directives interact nicely with zero-hit cache, provided that developers introduce resources in the zero-hit cache as follows:

resources = @static_resource_directory("resources")
urls      = parser {
              case "/": start
              case resource={Server.permanent_resource_map(resources)}: resource
            }
Server.start(Server.http, {custom: urls})

Of course, as usual with the zero-hit cache, you'll have to make sure that you are using URIs. For this purpose, as usual, you should take advantage of Resource.get_uri_of_permanent .