17.0.0 Release Notes

[hueniverse]

Sponsors

The hapi v17 release represents the next generation of node frameworks as the first enterprise-grade framework that is fully async/await end-to-end. It combines the latest technologies with a proven core that has been powering some of the world largest sites.

This release would not have been possible without the generous financial support of the following Featured Sponsors who have gone above and beyond to make this work successful and sustainable. I am extremely grateful for their support.

Lob enables you to seamlessly print and mail documents, postcards, checks, and more via an API. They have been an early hapi adopters and vocal supporters. If you are looking to take your hapi and JS skills to the next level, check out their career page for exciting opportunities.

Auth0 is the new way to solve identity. You can add authentication to your app or API by writing just a few lines of code. I have been consistently impressed by their products (which I have used for client work) as well as their talented team. If you are looking for a new opportunity, Auth0 is hiring for a variety of positions.

Condé Nast Technology powers many of the world's most iconic brands like WIRED, Vogue, Vanity Fair, GQ. hapi is a core part of the company's digital stack and drives just about every web request to their sites. I've had a long relationship with the team which includes some of my closest friends. Want to be hapi-er? Check out their Careers page.

This release effort required a significant investment of time and resources, surpassing $40,000 (and counting). If you or your employer has benefited from my work on hapi, especially if you are reading these notes in anticipation of migrating your applications to use the new v17 release, please consider supporting my work. If you would like to have your company included in the v17 materials and promotions, please consider becoming a Featured Sponsor.

Summary

hapi v17.0.0 is a major new version of the framework. It is among the top three major rewrites of the entire factor (after v2.0.0 and v8.0.0). In many ways, it is a new framework because it make fundamental changes to how business logic is interfaced with the framework.

The main change and the motivation for this release is replacing callbacks with a fully async/await interface. This is not merely an external, cosmetic change, but a deep refactor of the entire codebase, including most of the dependencies. With a handful of exceptions, there are no callbacks or closures used within the core module.

At the heart of the v17 release is the replacement of the reply interface (e.g. the reply() method passed to handlers, extensions, and authentication methods) with the new lifecycle method concept. The other major changes are the removal of multi-connections support and domain protection.

Note: hapi v17 requires node v8 and assumes a high proficiency with recent changes to JS. These notes and the hapi documentation do not go into any details about using async/await and promises as well as other topics such as symbols, sets, default values, etc. It is critical to have a strong understanding of the new flow controls introduced by async/await and the patterns around them before attempting to migrate to this new version.

Due to the nature of this release and the scope of changes, these release notes may be missing some details. It is recommended to read the full API documentation and please post any missing information in the comments so that we can keep this up to date.

• Upgrade time: high - a few days to a couple of weeks for most users
• Complexity: high - requires changes to anything that interacts with the framework with many changes in behavior of existing functionality
• Risk: medium - removal of domain protection could cause the process to exit due to existing application bugs that were previously handled as well as issues with mixing callbacks with promises if the application has legacy dependencies
• Dependencies: high - all plugins must be modified to work with the new version

Breaking Changes

• Removed all callbacks and replaced with async functions:
  • server.auth.test()
  • server.cache.provision()
  • server.emit()
  • server.initialize()
  • server.inject()
  • server.register()
  • server.start()
  • server.stop()
  • Plugin register()
  • Any methods that previously had reply interface argument
  • The after argument of server.dependency()
  • Server methods
  • Custom validation functions
  • Cache methods and the generateFunc option
  • The autoValue option of server.state()
• Removed support for multiple connections for a single server
  • The server.connection() method is replaced with options passed directly when creating a server object.
  • connection property is removed from all objects.
  • All connection methods moved to the server.
  • Removed support for labels and select() methods and options.
• Removed support for request tails
• Changed how errors and takeover responses are handled in the request lifecycle:
  • Instead of jumping to onPreResponse, jump to response validation first.
  • takeover() in handler will just to response validation (before it was ignored).
  • Authentication credentials validation split from access validation with a new onCredentials request extension point and a new request.auth.isAuthorized property. If a request failed access validation, the request.auth.isAuthenticated will be true in response validation and onPreResponse (previsouly was false).
• Replaced the reply() interface with a new lifecycle methods interface:
  • removed response.hold() and response.resume().
  • methods are async and the required return value is the response.
  • a response toolkit (h) is provided with helpers (instead of the reply() decorations).
• Removed node domains protection:
  • with async/await, most exceptions thrown are caught by the internal mechanism.
  • errors thrown in other ticks outside the internal async/await promises chain are no longer handled and will cause the process to exit if the application doesn't handle it explicitly.
• Response compression requires minimum size of 1024 bytes.
  • Default can be changed via the server compression.minBytes option.
• Moved request.id to request.info.id.
• Changes to internal logging:
  • Reduced internal request logging:
  • Only logs error. The initial stats log and the final response log are removed.
  • Removed the request.getLogs() method, replaced with direct access via request.logs.
  • request.logs are collected only if the route log.collect is set to true (false by default).
  • Combined 'request', 'request-internal', and 'request-error' into a single event and added channels support.
  • Replace the event.internal flag with event.channel.
  • When event data is an error, event.error is provided instead of event.data.
  • Event emitter interface changed to async/await and block option removed.
  • Moved all event emitters off the object and onto an events property:
    • server.events
    • request.events
    • response.events
• Remove handler and pre server method string shortcut support.
• Moved the failAction argument source into the error passed instead of a separate argument.
• Removed the option to set as default when calling server.auth.strategy().
  • Must call server.auth.default().
• Decoration changes:
  • Move server.handler() to use server.decorate() instead.
  • 'reply' decorations now use the new 'toolkit' decorations.
• Changed format of server.table() return value.
• Server methods always return the method value, regardless of caching is defined.
• Changed validation errors to exclude all validation information.
  • Use failAction to expose the information needed.
  • Note that the validation information no longer HTML escapes sensitive values that can be a security risk when sent back to the client.
• Removed server argument from 'route' event.
• Consistently set error to 413 when payload is too big.
• Changed plugin format from a function with properties to an object.
• Cookie autoValue methods are no longer executed in parallel.
• Renamed route config to options (config still acceptable but deprecated).
• Retain empty string as route.options.pre response as well as server.inject() response instead of casting to null.
• boom module no longer supports Boom.create() and Boom.wrap().

New Features

• Consistent failAction features - all failAction options now accept functions.
• A new onCredentials extension point and the ability to change the request credentials before authorization is validated.
• Expose compressor flush() method to response streams for better Server Sent Events support.
• Expose the full realm chain, not just the root server.
• Pass bind context as h.context in addition to this to better support arrow functions.
• Compression minimum response length.
• route.options.cors.origin can be set to 'ignore' which provides a CDN-friendly mode that ignores Origin headers and always responds with 'Access-Control-Allow-Origin' set to '*'.

Bug fixes

• Consistently execute response validation.
• Fix various issues with handling of promises and cached server methods.
• Handle parallel plugin registration.
• Exclude Connection header and hop-by-hop headers in response pass-through.
• Send 413 in all cases where request payload is too big.
• Send proper 400 HTTP response on node HTTP parser errors (clientError).

Updated dependencies

• accept v3.0.2
• ammo v3.0.0
• b64 v4.0.0
• boom v7.1.1
• big-time v2.0.0
• call v5.0.1
• catbox v10.0.2
• catbox-memory v3.1.1
• content v4.0.3
• cryptiles v4.1.0
• heavy v6.0.0
• hoek v5.0.2
• iron v5.0.4
• joi v13.0.1
• mime-db v1.31.0
• mimos v4.0.0
• nigel v3.0.0
• pez v4.0.1
• podium v3.1.2
• shot v4.0.3
• statehood v6.0.5
• subtext v6.0.7
• teamwork v3.0.1
• topo v3.0.0
• vise v3.0.0
• wreck v14.0.2

Migration Checklist

Callbacks

Any function that previously accepted a callback (either via callback or next) now returns a promise instead. With the exception of methods with a reply() interface (see lifecycle methods section below), all other methods remain the same and should be called with the await keyword.

For example:

server.start((err) => {
    if (err) {
        console.log(err);
    }
});

Is now:

try {
    await server.start();
}
catch (err) {
    console.log(err);
}

Checklist:

• Search for the following methods and replace the callback with await:
  • server.auth.test()
  • server.cache.provision()
  • server.emit()
  • server.initialize()
  • server.inject()
  • server.register()
  • server.start()
  • server.stop()
  • Plugin register()
  • The after argument of server.dependency()
  • Server methods
  • Custom validation functions
  • Cache methods and the generateFunc option
  • The autoValue option of server.state()

Multiple Connections

The server no longer supports more than one connection. All the options previously supported by server.connection() are now merged with the server object constructor. If your server calls server.connection() more than once, you will need to create another server object.

There are no simple instructions for implementing multiple connections with v17 because the needs vary too much. In its simplest form, you can just replicate the code that creates one server and create two, using different connection information for each. If you use labels to select connections within plugins, just register the plugins you want with the matching server.

If you need to share state between the different connections, consider using a shared server.app object or using a singleton pattern between the multiple servers.

Checklist:

• Replace each server.connection() call with a server instance configured with the options passed to both the server and connection.
• Check if any of the servers use more than one connection and refactor the code.
• Rework any plugin using server.select() to only register it against the desired servers.
• Remove the labels option.
• Remove references to connection and replace with server (e.g. request.connection).
• Remove the connections: false plugin option since it is no longer applicable. Plugins cannot set up connections since the connection is configured during server construction.

Lifecycle methods

A lifecycle method is an async function using the signature async function(request, h, [err]) and is used by pretty much every method passed to the framework to execute when processing incoming requests. This includes handlers, request extensions, failAction methods, pre-handlers, and authentication scheme methods.

With the move to async/await, the old reply() interface was no longer applicable as it was in practice a callback with a lot of special handling rules. Instead, the new lifecycle method is a much simpler interface. To set a new response, simply return that value. To set a new response and jump to response validation, use the takeover() response decorator. To continue execution without setting a response, return h.continue. The full list of options it listed in the API documentation.

For example:

// Before

const handler = function (request, reply) {
    return reply('ok');
};

// After

const handler = function (request, h) {
    return 'ok';
};

Checklist:

• Refactor every handler, pre, authentication scheme, failAction methods, request extensions, and any other method that previously accepted the reply argument. If your code uses the same argument names as the hapi convention, searching for (request, reply) is an easy shortcut to find most of the method you need to migrate.

• Remove response.hold() and response.resume() and replace with an async function or return a promise.

• In general:

  • Return the value you want to set the response as. This will continue to the next step in the request lifecycle.
  • Return the value with a takeover flag to set the response and skip to response validation and transmission. Use the toolkit h.response() helper to wrap a plain response in a response object to access the takeover() decorator.
  • Return a promise or declare the method as async for asynchronous operations.
  • throw boom errors to return an error response.

• In request extensions:

  • Return h.continue instead of reply.continue() to continue without changing the response.
  • Return h.response(result).takeover() to override the response and skip to validation instead of reply(result).takeover().
  • Return the value directly instead of reply.continue(result) in extension points after the handler.

• In authentication authenticate():

  • Return h.authenticated() or h.unauthenticated() for success and failure.

• If a route is configured with authentication and access rules (scope, entity) and the access validation fails, the request request.auth.isAuthenticated will be true (it was false in previous versions). This only matters if you check the flag in the onPreResponse step. If you do, check for request.auth.isAuthenticated && request.auth.isAuthorized instead for the same result.

• Note that errors and takeover responses now jump to the response validation step instead of directly to onPreResponse. If you have response validation configured, ensure it can handle these error and takeover responses or the validation will fail with a 500 error.

• Look for takeover() in handlers as it will now cause it to jump directly to response validation, skipping the onPostHandler step.

Events

In order to simplify and optimize logging, the request, response, and server emitters have been moved to use the events property instead of inheritance. In the case of the request and response emitters, if you never access them, they are not initialized, saving resources.

The three request event types ( 'request', 'request-internal', and 'request-error' ) have been merged into a single 'request' event. In addition, only internal error logging are emitted and collected.

Checklist:

• Replace:
  • server.on() with server.events.on().
  • request.on() with request.events.on().
  • response.on() with response.events.on().
  • Same for any other emitter method.
• If you rely on non-error internal logs, use request extension points (e.g. onRequest and onPreResponse or the 'response' event) to manually log the information you desire.
• Look for calls to request.getLogs() and replaced them with direct access to request.logs. You will also need to configure the route to collect logs by setting the route log.collect option set to true (false by default).
• To listen only to the previous request events use:
  • 'request' -> { name: 'request', channels: 'app' }
  • 'request-internal' -> { name: 'request', channels: 'internal' }
  • 'request-error' -> { name: 'request', channels: 'error' } (note that the listener signature is different and that it will pass a full event object instead of the previous err which can be accessed now via event.error).
• Replace access to request event.internal argument with event.channel (and check the value is 'internal' for the same result).
• When event data is an error, event.error is provided instead of event.data.
• If you used the podium block option, remove it and convert your listener to an async function.

Domains

Previous versions used the now deprecated node domains feature to protect application code from throwing errors synchronously or asynchronously. This has been a great feature for a long time as it captured many developer errors that made their way to production. Instead of crashing the application, a 500 error was returned and the error logged.

The problem was, domains didn't play well with promises and could instead swallow errors or produce unexpected results. In general, when an unhandled error is thrown, the server is considered to be in an unstable state. While it is better to return a 500 error than crash the process, it wasn't a perfect solution.

v17 removed domain support. It might come back in the future when node async hooks reach a stable place and an alternative solution is provided. The good news is that many of the common errors are already covered by the asyn/await error catching flow. The framework will continue to catch errors thrown synchronously as well as many of those thrown asynchronously (as long as they are thrown as part of the proper promises chain).

This is not as extensive as the domain support. Unfortunately, there isn't much you can do other than adding some global listeners.

Checklist:

• Listen to global events:
  • process.on('uncaughtException').
  • process.on('unhandledRejection').
• Migrate as much of your stack to use promises.
• Don't throw inside timers...

Plugins

In an effort to use more conventional patterns, the plugin function with object properties style has been replaced with a plain object.

Checklist:

• Replace the exports.register() and the matching exports.register.attributes with exports.plugin = { register, name, version, multiple, dependencies, once, pkg }.
• Remove the unsupported connections attribute.

Server Methods

All server methods must be full synchronous, an async function, or return a promise. When a server method is cached, the result no longer changes to an envelope with the result and ttl value.

Checklist:

• Migrate any callback method to async function.
• Remove the unsupported callback method option.
• If you need access to the cache result envelope information { value, ttl, report }, use the catbox getDecoratedValue option.

Misc

• Request tails are no longer supported. Lookup calls to request.tail() and any listeners to the 'tail' event and replace them with an application specific solution. The 'tail' event can be replaced with the 'response' event. You can use the request.app object to store local request state such as promises representing tail events and then use Promise.all() in the response event to wait for them to finish processing.

• Search for references torequest.id and replace them with to request.info.id.

• Look for handlers or pre methods that use a string as the handler and replace them with explicit calls to the server method used.

• If you use the failAction option in request input or response payload validation, the function signature has changed to a lifecycle method. The previous source argument is now available as a property of the error received.

• Look for calls to server.auth.strategy() and if they third argument is true or a string, replace that with an explicit call to server.auth.default().

• Replace server.handler(...) with server.decorate('handler', ...).

• Replace server.decorate('reply', ...) with server.decorate('toolkit', ...).

• Look for calls to server.table() and adjust the code to handle the new format which no longer returns an array or an envelope with a table property. Instead the table value is the direct return value.

• Input validation errors are no longer passed directly from joi to the client. Instead, a generic 400 error is returned which simply indicates which input source failed validation (e.g. 'query'). If you want to keep the original error, set a failAction validation option such as (request, h, err) => throw err. Note that unlike previous versions, the error message is on longer HTML escaped to prevent echo attacks. You must perform the applicable error string escaping to prevent exploits. In general it is best practice to never echo back the the client anything sent that could be injected with a script or other content.

• Look for listeners to the 'route' event and remove the second server argument.

• Ensure your clients do not rely on receiving a 400 error code when the payload is too big. Previous versions sent a mix of 400 and 413 errors based on the payload parsing rules. This will consistently set errors to 413 when the payload is too big.

• According to compression best practices, there is no reason to compress payloads under 1kb in size because the payload already fits within a single packet. In that case, compression wastes CPU resources and time for no benefit. This release changes the default compression behavior of response payloads smaller than 1kb to not compress. This should work correctly for most applications. To change this and restore the previous behavior, set the server compression.minBytes option to a smaller number or to 1.

• If you use the state autoValue option, note that if there are multiple cookies set, each with an autoValue options, these methods are now called in serial, not in parallel. This matters if you are making network calls which can cause the overall response time to increase. However, it is very unusual to make network calls when processing cookies for transmission. If you must, move that logic to another spot in the request lifecycle.

• When lifecycle methods returned an empty string, the response was converted to null. This was changed to retain the empty string. It will have no impact on the HTTP response payload which is still empty. It will affect the value of request.pre properties ('' instead of null) as well as the value of res.result in server.inject() which will also be ''.

• While at it, replace config with options when adding routes. config will still work but will go away in the future.

• • boom module no longer supports Boom.create() and Boom.wrap(). Use Boom.boomify() instead. You can also use new Boom() instead of Boom.internal() for a full replacement of new Error().

[vladikoff]

Did anyone have to solve the lack of security and CORS headers on non-defined routes (404) when migrating to v17? Ref: #3792

[devinivy]

I suggest defining a catch-all route that handles 404s however you'd like. There's a short section on this in the API docs.

const Boom = require('boom');

server.route({
    method: '*',
    path: '/{p*}',
    options: {
        cors: true,
        handler() {

            throw Boom.notFound();
        }
    }
});

[vladikoff]

@devinivy I really appreciate your response, thank you!

[bmgdev]

@devinivy this solution doesnt appear to work for me, where are you placing this route?

[marcuspoehls]

@bmgdev Hey Bradley, you can register this route wherever you want. Do you receive an error message?

It's important that you don't have another catch-all route, like your base handler that passes requests through to a client-side framework.

Maybe this tutorial on how to handle 404 responses helps.

[dwelle]

There's a breaking change not noted here, and that is setting a global route validation rule for params now throws for routes that do not specify any dynamic param in their path.

const server = Hapi.server({
    routes: {
        validate: {
            params: {} // <-- cannot be specified in v17
        }
    }
});

AFAIK, this isn't specified in API, either.

[fgalan]

With regards to:

'request-internal' -> { name: 'request', channel: 'internal' }

I'm getting this error:

    ValidationError: Invalid event listener options {
      "name": "request",
      "listener": function (request, event, tags) { ... },
      "channel" [1]: "internal"
    }    

I understand this is a typo, as using channels (with additional 's') doesn't produce that error. However, it'd suggest to fix it in the release notes (note that API reference at https://github.com/hapijs/hapi/blob/v17/API.md#server.events.request use 'channels' )

[lock]

This thread has been automatically locked due to inactivity. Please open a new issue for related bugs or questions following the new issue template instructions.