Releases: observablehq/framework
v1.13.0
New features
- DuckDB 1.1.1 and DuckDB-Wasm 1.29.0! 🐤 #1734
- Support for, and self-hosting of, DuckDB extensions! 🐤 #1734
- Observable Runtime 6.0.0 for faster async (e.g., DuckDB) reactivity 🏃💨 #1748
- Support secure sharing of private exported modules on Observable Cloud 🔒 #1731
- Support analytics of parameterized routes & exported modules on Observable Cloud 📊 #1776
- Deprecate cleanUrls in favor of preserveIndex and preserveExtension to support S3-like hosting #1784
Bug fixes and other improvements
- Fix automatic cleaning of
dist
when building on Windows #1792 #1804 - Add explicit
<html>
,<head>
, and<body>
tags to generated HTML #1656 #1732 - Suppress checkboxes on tables in SQL code blocks #1740
- Fix resolution of absolute paths in SQL front matter #1796 #1797
- Avoid a top-level await bug on Safari in SQLite #1750 #1759
- Fix importing of
npm:sql.js
by adding an ES module export shim #1759 - Fix resolution of absolute global imports #1755 #1758
- Fix deduplication of resolved imports #1718 #1719
- Fix parameterized routing when a parameter value is its name in brackets #1760 #1767
- Support the
NO_COLOR
environment variable (via picocolors) #1769 - Remove automatic redirect to strip basic authentication #1733
- Fix updating the app title when deploying to Observable Cloud #1715
- Various documentation improvements #1765 #1722 #1729 #1738 #1736 #1803 #1783 #1785
New Contributors
- @angrytongan made their first contribution in #1750
Full Changelog: v1.12.0...v1.13.0
v1.12.0
New features
- Exported modules for embedded analytics! 🎉 #1084 #1637
- Exported files for hotlinking images (and more) 🖼️ #1637
- JSR (
jsr:
) imports for the JavaScript Registry 📦 #956 #957 - New home config option for more sidebar customization, such as a logo 🏠 #1689
- Redesigned top-fixed header 💅 #1682
- Redesigned center-column layout on wide screens →← #657 #1023 #1682
- Enable CORS on the preview server with
--cors
#1583 #1637
Bug fixes and other improvements
- Allow
import.meta.resolve
to be used for local files #1008 #1696 - Favor
observablehq:stdlib
overnpm:@observablehq/stdlib
#1670 - Detect broken links during build #363 #1683 #1698
- Detect missing files during build #1650 #1651
- Detect file name conflicts during build #1367 #1374
- Fix crash during preview with temporary files (e.g., Vim) #1498 #1700
- Fix content hashes for modules that import builtins #1688
- Fix reactivity during preview for data loaders from modules #1661 #1662
- Fix reactivity during preview for parameterized modules 8cb2877 #1660
- Use registry.npmjs.org instead of data.jsdelivr.com to resolve npm package versions #1697
Full Changelog: v1.11.0...v1.12.0
v1.11.0
New features
- Parameterized routes for dynamic routing! 🎉 #245 #1523
- Page loaders for server-side rendering! 🚀 #931 #1216 #1523
- TypeScript fenced code blocks and modules! 🤓 #79 #129 #1632
FileAttachment
now exposes afile.size
property. #1608- The new globalStylesheets config option gives more control over styling. #1597
Bug fixes and other improvements
- The
build
command now empties the output root before building. #1568 - The
convert
command now outputs to the source root by default. #1579 - Fix rendering of command-line select prompts with many choices. #1618
Inputs.file
now supports thefile.href
property. #1608- Pages now include a
<meta name="generator">
tag. #1609 - Use
CREATE TABLE
for small Parquet files withDuckDBClient.of
. #1617 - Fix building when SQL front matter refers to a remote file. #1635 #1636
New examples and documentation
- New guide on converting notebooks and the
convert
command! #1576 #1580 loader-bigquery
- Load data from Google BigQuery (Thanks, @dirkbosman!) #1581datawrapper-api
- Embed a Datawrapper chart (Thanks, @palewire!) #1585 #1611
New contributors
- @vorburger made their first contribution in #1572
- @CobusT made their first contribution in #1576
Full Changelog: v1.10.1...v1.11.0
v1.10.1
New features
- Allow opt-out of link normalization via
<a href rel="external">
. #1560
Bug fixes and other improvements
- Fix crash attempting to preserve expanded state. #1559 #1561
- Fix content hashes of global imports in
_observablehq
,_npm
, and more. #1555 #1557 - Optimize
resize
to ignore height changes for unary render functions. #1563 #1564 - Improve slugify algorithm used for Markdown header anchors. #1552 #1562
Full Changelog: v1.10.0...v1.10.1
v1.10.0
New features
- Add
FileAttachment.arquero
for loading an Arrow, Parquet, CSV, TSV, or JSON file as an Arquero table. #1509 Inputs.table
now supports the select option to suppress user selection (checkboxes). #1541- In narrow windows, you can now close the floating sidebar with Escape. #1526
- The inspector now preserves the deep expanded state for dynamic values. #1458 #1475
Bug fixes and other improvements
- Fix a bug where expanded inspected values would not be cleaned up on invalidation. #1456 #1457
- Fix a crash when expanding inspected values while certain text is selected. #1475
- Fix a spurious error when using Control-C to terminate the preview server. #1459 #1460
- Fix
modulepreload
for local components during preview. #1537 - Optimize imports of
npm:react-dom
. #1467 - Tweak format of project identifiers in prompts during deploy. #1451
- Fix spinner rendering when not in a TTY. #1446 #1447 #1473
New examples
New showcase examples:
hotel-bookings
- Resort hotel bookings #1337us-dams
- U.S. dam conditions #1350
New technique examples:
loader-rust-to-json
- Generate JSON from Rust #1452loader-python-to-csv
- Generate CSV from Python #1461 #1466loader-airtable
- Load data from Airtable #1477loader-elasticsearch
- Load data from Elasticsearch #1490loader-census
- Load and transform shapefiles from the U.S. Census Bureau #1496loader-canvas-to-png
- Generate PNG using node-canvas #1514loader-python-to-parquet
- Generate Apache Parquet from Python #1422loader-r-to-zip
- Generate ZIP from R #1423
New contributors
- @walterra made their first contribution in #1490
- @taniki made their first contribution in #1505
- @palewire made their first contribution in #1516
- @gvanhorne made their first contribution in #1513
- @juba made their first contribution in #1530
Full Changelog: v1.9.0...v1.10.0
v1.9.0
JSX + React ⚛
Framework now supports JSX rendered with React, 🎉 providing a powerful new mechanism for implementing reusable stateful components. You can both write fenced code blocks (```jsx
) and import JSX modules (.jsx
). For example, to define a Greeting
component that accepts a subject
prop:
function Greeting({subject}) {
return <div>Hello, <b>{subject}</b>!</div>
}
You can then call the built-in display
function to render JSX content:
display(<Greeting subject="JSX" />);
The display
function in JSX code blocks uses React’s createRoot
to render the specified contents into the DOM. When you call display
again later, it applies React’s reconciliation algorithm to efficiently update the DOM.
Naturally, you can also use React’s built-in hooks such as useState, useEffect, and useRef. For example, here is a Counter
component whose count you can increment by clicking a button:
function Counter() {
const [count, setCount] = React.useState(0);
return (
<button onClick={() => setCount(count + 1)}>
You clicked {count} times
</button>
);
}
React
and ReactDOM
are now available by default in Markdown; if you prefer, you can also import them explicitly from npm:react
or npm:react-dom
.
Span-less inline expression rendering
Framework’s inline expressions allow you to interpolate dynamic content anywhere on the page using ${…}
. Previously, interpolated content was wrapped in a SPAN element; now Framework uses a comment (e.g., <!--:85902a01:-->
) to track where to insert displayed content. This enables a variety of new use cases. For example, you can now use inline expressions to populate the contents of a grid,
<div class="grid grid-cols-4">
${d3.range(4).map((i) => html`<div class="card">Hello ${i}</div>`)}
</div>
… or to generate table rows,
<table>
<thead><th>Index</th></thead>
<tbody>${d3.range(4).map((i) => html`<tr><td>${i}</td></tr>`)}</tbody>
</table>
… or even within SVG elements!
<svg width="640" height="120">
<text x="20" y="20">My favorite number is ${Math.random()}</text>
</svg>
As part of this change, we also fixed several edge cases in incremental updates during preview. (Previously we only tracked changes to top-level elements, but now we also track changes to top-level text nodes and comments.)
More robust inline expression parsing
We’ve also improved how we parse inline expressions: Framework now uses the HTML5 tokenizer algorithm (adopted from Hypertext Literal) to determine context, allowing Framework to ignore inline expressions in unsupported contexts such as attributes, coments, and raw text. This means that if you comment out a chunk of Markdown that includes an inline expression, Framework no longer runs the code (and no longer generates an error)!
<!-- ${"this code doesn’t run"} -->
In the future, we’d like to support interpolation into attributes <a href=${link}>
and possibly raw text <textarea>${1 + 2}</textarea>
; please upvote #32 if you’re interested in this feature.
Framework now also uses Acorn’s tokenizer to determine when an inline expression ends. Previously, Framework counted quotes and curly braces; by adopting Acorn’s tokenizer, Framework can correctly parse (and skip) JavaScript comments within inline expressions. For example, the following inline expression evaluates to 3
:
${1 + /* } */ 2}
Lastly, Framework now correctly handles backslash escaping of inline expressions within HTML blocks. To escape an inline expression, resulting in the literal text ${1 + 2}
, place a backslash \
before either the dollar sign $
or left curly brace {
:
<pre>\${1 + 2}</pre>
To instead show a literal backslash prior to the result of the inline expression \3
, use two backslashes \\
:
<pre>\\${1 + 2}</pre>
Or, to show a literal backslash followed by literal text \${1 + 2}
, use three backslashes:
<pre>\\\${1 + 2}</pre>
Together, these rendering and parsing improvements make Framework’s inline expressions feel more robust — they just work.
Other improvements
The FileAttachment
function now returns a canonical instance: calling FileAttachment
with the same name will return the same object, allowing easier comparison.
FileAttachment("foo.csv") === FileAttachment("foo.csv") // true
The Mod-Enter keyboard shortcut now opens search results in a new tab, making it easier to open multiple search results for the same query.
Sample datasets, such as penguins
and miserables
, are now self-hosted from npm:@observablehq/sample-datasets
, allowing you to work with sample data while offline.
The deploy command now supports the --deploy-config
command-line argument to specify an alternative path to the deploy.json
configuration file. (By default, this file lives in .observablehq/deploy.json
within the source root.) The deploy command now prints a better error message when attempting to deploy without authentication from a non-interactive terminal.
Examples improvements
We’ve added a variety of new technique examples:
loader-julia-to-txt
- Generating TXT from Julialoader-python-to-png
- Generating PNG from Pythonloader-python-to-zip
- Generating ZIP from Pythonloader-r-to-csv
- Generating CSV from Rloader-r-to-jpeg
- Generating JPEG from Rloader-r-to-json
- Generating JSON from Rcodemirror
- A text input powered by CodeMirrorinput-select-file
- Selecting a file from a drop-down menu
The examples are now searchable from the Framework documentation. The example config files have also been greatly simplified by removing shared boilerplate. Lastly, descenders in the hero text of the default template’s home page are no longer clipped.
Thanks @martinswan for contributing to the docs!
Full Changelog: v1.8.0...v1.9.0
v1.8.0
Sidebar and pager improvements
Sidebar sections can now have header links, which is useful for sections with main pages. Clicking on the section will visit the section’s main page while opening the section in the sidebar to show related pages. To give a section a header link, set the path option for a given section. See the pages config option for more.
export default {
pages: [
{
name: "Section 1",
path: "/s01/",
open: false,
pages: [
{name: "Page 1.1", path: "/s01/page1"}
]
},
{
name: "Section 2",
path: "/s02/",
open: false,
pages: [
{name: "Page 2.1", path: "/s02/page1"},
{name: "Page 2.2", path: "/s02/page2"}
]
}
]
};
The sidebar now supports links to external pages, which automatically open in a new window. (We use this feature in the Framework docs to link to examples.) The sidebar now renders sections correctly when there are no top-level pages.
Sidebar pages and sections can now use the pager option to define separate sequences for the next & previous links in the footer, or to opt-out of the pager for specific pages or sections. (We use this feature in the Framework docs to have separate sequences for inputs, and to suppress the pager on library pages.) The pager option can also be defined in a page’s front matter. If a page sets the pager front matter option to false or null, the pager is hidden.
Other improvements
The head, header, and footer config options can now be specified as functions, allowing dynamic footers. These functions are passed an object with the page’s title
, (front-matter) data
, and path
, and must return a string.
We’ve improved how data loaders are spawned, fixing a FATAL Error: fsync failed!
error with DuckDB data loaders that output directly to stdout. We also fixed the normalization of plain HTML links such as <a href="/test.html">hello</a>
; these are now correctly converted to relative links.
New examples! 🎉
We’ve been hard at work developing new examples.
geotiff
- Parsing GeoTIFF with geotiff.js, then visualizing with Observable Plotnetcdf
- Parsing NetCDF withnetcdfjs
, then visualizing with Observable Plotvega-dark
- Responsive dark mode in Vega-Litevega-responsive
- Responsive width in Vega-Lite using ResizeObserverloader-arrow
- Generating Apache Arrow IPC filesloader-databricks
- Loading data from Databricksloader-duckdb
- Processing data with DuckDBloader-github
- Loading data from GitHubloader-google-analytics
- Loading data from Google Analyticsloader-parquet
- Generating Apache Parquet filesloader-postgres
- Loading data from PostgreSQLloader-snowflake
- Loading data from Snowflakenetcdf-contours
- Converting NetCDF to GeoJSON withnetcdfjs
andd3-geo-voronoi
custom-input-2d
- A custom 2D input with bidirectional bindingmarkdown-it-container
- The markdown-it-container pluginmarkdown-it-footnote
- The markdown-it-footnote pluginmarkdown-it-wikilinks
- The markdown-it-wikilinks plugincustom-stylesheet
- Defining a custom stylesheet (custom theme)intersection-observer
- Scrollytelling with IntersectionObserverresponsive-iframe
- Adjust the height of an embedded iframe to fit its content
See all examples here: https://github.com/observablehq/framework/tree/main/examples
Please let us know which examples you’d like to see.
Full Changelog: v1.7.1...v1.8.0
v1.7.1
Page stats
The build
command now outputs summary page size statistics, including the weight of imported JavaScript modules and referenced files such as data. This helps you keep an eye on performance, encouraging you to reduce bloat by removing unnecessary imports or by optimizing data loaders to produce smaller files.
Bug fixes
Prevent the sidebar from closing on narrow windows when the search input is focused via Command-K and then a link is clicked. Avoid a rendering glitch in Safari with animated loading indicators ↻. Downgrade JSDOM to ^23.2.0 to avoid a regression (not a valid selector
error) in the selectors implementation. Automatically link H1, H2, H3, and H4 elements that have an id
attribute, enabling them to also appear in the table of contents. Fix table of contents highlighting when the heading id
contains non-ASCII characters.
Watch changes to static assets during preview, such as referenced stylesheets and images, so that changes to these files update automatically. Also populate FileAttachment.lastModified
on update during preview.
When deploying to Observable, correctly update titles when linking to existing projects.
And more…
The resize
function now implicitly awaits promises returned by the render
function; the render
function can also now return null to clear the container. Deprecate the scripts config option.
Full Changelog: v1.7.0...v1.7.1
v1.7.0
The default source root is now src
instead of docs
. For backwards compatibility, we’ll continue to default to docs
if it exists, but we now recommend that you specify the root config option.
We’ve added a few new markdown-it config options. The typographer option enables simple typographic replacements, such as smart quotes (replacing straight quotes with curly quotes based on context). You can customize the quotes using the quotes option. And the linkify option lets you turn off automatic linking of URL-like text in Markdown.
We’ve improved Node import compatibility by supporting packages that require or import JSON, such as arquero
. We also now better handle certain CommonJS packages (using requireReturnsDefault
) and certain transitive imports. We now use the latest version of npm:parquet-wasm
if no version range is specified, rather than defaulting to 0.5.0; thank you to @kylebarron for fixing parquet-wasm’s conditional exports in 0.6.0 to improve compatibility.
Deploying to Observable is now significantly faster when only some files have changed: rather than uploading every file, we now only upload files that have changed. We’ve also improved the error message when a file is too big or the file quota has been exceeded. We removed support for the deprecated deploy config option; the deploy configuration should instead live in .observablehq/deploy.json
within the source root.
Lastly, we fixed the scripts config option to correctly handle resolution of local modules; we also now recommend that you use the head config option instead.
Thanks to @scresawn, @declann, and @russbiggs for contributing to the docs!
Full Changelog: v1.6.0...v1.7.0
v1.6.0
Node imports 📦
You can now import from node_modules
! This lets you manage dependencies with a package manager such as npm or Yarn, import private packages from the npm registry, or import from a different package registry such as GitHub. For example, to use canvas-confetti, first install it (here using npm):
npm install canvas-confetti
Then import it:
import confetti from "canvas-confetti";
You can import named exports or import as a namespace, too. For example, here’s how you might import Apache Arrow:
import * as Arrow from "apache-arrow";
You can import specific entry points by adding the entry point subpath after the package name. For example, to import mime’s lite
entry point:
import mime from "mime/lite";
Note: not all Node packages are usable in the browser; Node imports are only supported for modules that do not rely on Node-specific APIs and that can be converted to ES modules via esbuild. If you have difficulty importing a module, please ask for help by opening a discussion.
We also made some improvements to npm:
imports: entry points without a file extension, such as npm:mime/lite
, now implicitly get /+esm
added to generate an ES module; an /+esm
import no longer collides with a non-/+esm
import of the same entry point; and undeclared dependencies are now correctly resolved, fixing certain imports.
More documentation 📚
We’ve overhauled and improved our documentation. Please take a look and let us know what you think. Thank you to @CAYdenberg for the thoughtful feedback, and @Stroked for helping to improve our wording. We also fixed a few links, added deep links to Observable where needed, added an interactive version of the mortgage rates dashboard, and clarified how to install markdown-it plugins.
And check out this new deck.gl example visualizing more than 100,000 reported collisions on roads in Great Britain for the year 2022:
It’s written using a data loader (dft-road-collisions.csv.sh
) that fetches almost 9M records from the Department for Transport and filters them using DuckDB, reducing the data from 1.2GB to 2.1MB. Here’s what it looks like if you display the entire dataset:
This data is a strong proxy for population density (and more specifically traffic density), so the major highways are strikingly visible.
Deploy improvements 🚀
We fixed a bug where observable deploy
could deploy a stale build if you recently modified your source files before deploying. Now observable deploy
will always ask if you want to build. For example, it might say:
◆ You built this project 6 minutes ago. Would you like to build again before deploying?
│ ● Yes, build and then deploy / ○ No, deploy as is
If you’ve not yet built your project, you will see:
◆ No build files found. Do you want to build the project now?
│ ● Yes, build and then deploy / ○ No, cancel deploy
In non-interactive terminals, observable deploy
will build automatically if the build is missing and otherwise deploy as-is. You can use the new --build
flag to force a fresh build on deploy, or the --no-build
flag to force a deploy of your existing build. These new flags replace the --if-stale
and --if-missing
flags. We also fixed a race condition that could cause deploying to Observable to fail.
Other improvements 🆕
Sidebar sections can now be marked as non-collapsible by setting collapsible: false
. Use this for a cleaner sidebar if you always want the section to be open. For example:
export default {
title: "Sample project",
pages: [
{name: "Page 1", path: "/page-1"},
{name: "Page 2", path: "/page-2"},
{
name: "Sample section",
collapsible: false, // always open!
pages: [
{name: "Item 1", path: "/section/item-1"},
{name: "Item 2", path: "/section/item-2"}
]
}
]
};
The new FileAttachment.dsv
method allows you to load delimiter-separated files like CSV or TSV, but with an arbitrary delimiter. For example, to load semicolon-separated values:
const capitals = FileAttachment("us-state-capitals.csv").dsv({delimiter: ";", typed: true});
The “empty” project template (which is used if you answer “No” to “Include sample files to help you get started?” during observable create
) is now emptier than before, with no included sample data. observable create
now sets a default favicon when creating a project, and survives errors that may occur during the initial build such as a network configuration problem.
There’s a new jQuery UI example. We fixed a bug where certain runtime errors would not be cleared in code blocks that don’t otherwise display. The head, header, and footer front matter options are now correctly suppressed when set to false.
Full Changelog: v1.5.1...v1.6.0