v1.5.1

Editing the project config (observablehq.config.js) now triggers an automatic reload over the preview socket. Changing the modification time (say via touch) of a data loader now immediately re-runs the data loader during preview. The preview server now correctly handles page names containing spaces, emoji, or other non-ASCII characters.

Full Changelog: v1.5.0...v1.5.1

v1.5.0

Live config

Changes to the project config (observablehq.config.js) now take effect immediately 🎉 on reload or navigation during preview, without needing to restart the preview server: the preview server now loads the latest config on request rather than preloading it on startup. Likewise, the default list of sidebar pages is now computed on request, and hence a server restart is no longer required when adding, removing, or renaming a page. We’ve also optimized parsing and caching to improve performance during preview.

The preview socket now automatically reconnects if the preview server restarts. No more broken sockets! 😌

External link treatment

External links how have target="_blank" and rel="noopener noreferrer" applied by default. This improves privacy by not sharing the (potentially sensitive) referrer when linking to a (potentially untrusted) external site. You can override this for a given link by setting the target and/or rel attribute explicitly.

Other improvements

The new FileAttachment.href property returns the absolute URL to the associated file. Use this simpler alternative to the async FileAttachment.url() method.

Safari 16 is now (again) supported. (We were unintentionally generating code with static initializer blocks which were only added in Safari 16.4.) Invalid Markdown front matter is now treated as page content, and logs a warning, instead of crashing. The default Mermaid theme is now consistent with the chosen page theme’s color scheme, rather than exclusively reflecting the user’s preferred color scheme (consistent with the built-in dark reactive variable). Static <script type="module"> are now only preloaded with <link rel="modulepreload"> if the script element does not have an async attribute (which we assume implies a lower-priority script, such as analytics).

The observable create command now uses the default path hello-framework instead of ./hello-framework, making it less confusing on Windows (where \ is the path delimiter instead of /). In addition, observable create now builds the project, ensuring instant initial page loads during preview since npm: imports and data loaders will already be cached. The observable logout command now logs a confirmation message on successful sign-out.

Full Changelog: v1.4.0...v1.5.0

v1.4.0

Improved SQL and DuckDB 🦆

SQL code blocks and DuckDBClient now support querying remotely-hosted data. This is especially useful for live, rapidly-changing data that you do not want to “bake-in” to your built site. For example, to load recent earthquakes from the USGS real-time feed:

---
sql
  quakes: https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_day.csv
---

To get the maximum recent earthquake magnitude, you could then query:

SELECT MAX(mag) FROM quakes

Note: for faster page loads and to avoid a runtime dependency on external servers, we recommend using data loaders when possible to generate static snapshots of data at build time, with continuous deployment to re-build your site as needed.

Framework also now supports DuckDB’s database file format. These files conveniently store multiple tables and may offer better performance on complex or repeated queries. DuckDB database files can either be static or generated dynamically by data loaders. For example, to load a database file named chinook.db:

---
sql
  chinook: chinook.db
---

Then, to query the tracks table:

SELECT * FROM chinook.tracks

The new DuckDBClient.sql convenience method lets you redefine the sql tagged template literal used by SQL code blocks. This allows you to load or create data sources dynamically, say in response to user interaction. For example, to allow the user to choose the desired USGS real-time feed from a drop-down menu:

const feed = view(Inputs.select(new Map([["M4.5+", "4.5"], ["M2.5+", "2.5"], ["All", "all"]]), {label: "Earthquake feed"}));

const sql = DuckDBClient.sql({quakes: `https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/${feed}_day.csv`});

SELECT * FROM quakes ORDER BY updated DESC

We also fixed a bug where DuckDB was being loaded during preview even when it was not needed, and fixed the implementation of db.describeTables and db.describeColumns now that db.query returns an Arrow table instead of a materialized array-of-objects.

More local assets

You can now reference local assets such as images, stylesheets, and scripts from the head, header, and footer config options. For example, to set the favicon to a favicon.png stored in the source root:

export default {
  head: `<link rel="icon" type="image/png" href="/favicon.png" sizes="512x512">`
};

Like file attachments, referenced assets can be either static or generated dynamically by data loaders, are included in the built site, and have a content hash applied to the file name for cache breaking.

The new FileAttachment.lastModified property exposes the modification time of the given file, represented as the number of milliseconds since UNIX epoch (like File.lastModified). This is useful for showing how fresh data is, for example. For files generated by data loaders, this stores the time the data loader completed.

FileAttachment("results.csv").lastModified // 1710809036000

new Date(FileAttachment("results.csv").lastModified) // "Tue Mar 19 2024 00:43:56 UTC"

Framework also now supports script elements with the src attribute in Markdown. This is especially useful for loading older, non-module JavaScript — it “just works”. We also fixed a bug resolving file attachment paths with a leading slash.

More flexible deploys

The build and deploy commands are now decoupled, offering more flexibility: you can modify a built site prior to deploying (say to add additional assets), and you can re-deploy a site without having to re-build it (say if your deploy gets interrupted). To support this, the deploy command has two additional flags, --if-stale and --if-missing, that indicate what to do when the build is either stale (more than five minutes old) or missing. Both default to prompt in interactive terminals, meaning the deploy command will ask how you’d like to proceed, and cancel in non-interactive terminals.

Our new Deploying guide shows how to automate deploys from GitHub Actions.

Deploys to Observable are also now significantly faster thanks to concurrent file uploads. 🚀

And more…

Self-hosted npm: modules have been renamed from +esm.js to _esm.js to avoid a quirk with how Amazon S3 interprets the + character in paths. The pages config option now properly normalizes page paths, fixing a bug where the previous & next footer links wouldn’t appear. The EIA example dashboard timeline chart now has better tooltips (thanks, @pettiross!).

Full Changelog: v1.3.0...v1.4.0

v1.3.0

Mosaic

Framework now has built-in support for Mosaic vgplot, an interactive grammar of graphics that helps you build coordinated views with fast interaction for millions of data points (or more). Framework’s Mosaic integration is powered by DuckDB-Wasm and Observable Plot. Data sources are configured using the sql front matter option — meaning you can query the same data using Mosaic and Framework’s SQL code blocks. See this interactive heatmap of one million New York City taxi rides for an example:

Speaking of DuckDB, our SQL code blocks and sql tagged template literal are now much faster thanks to an optimized DuckDBClient which uses connection.query instead of connection.send for non-streaming queries, and returns Apache Arrow tables rather than materializing array-of-objects. (You can use table.toArray or Array.from to materialize an array-of-objects if needed.)

All the interpreters

The new interpreters config option allows you to register additional interpreted languages for data loaders. 🎉 For example, to use Perl (.pl) and AppleScript (.scpt):

export default {
  interpreters: {
    ".pl": ["perl"],
    ".scpt": ["osascript"]
  }
};

We also now support Java (.java), Julia (.jl), and PHP (.php) data loaders by default.

Better links

The new cleanUrls config option, if set to false, preserves the .html extension on page links. This is useful for some static site servers that don’t support “clean” URLs. We’ve also enabled automatic “linkification” of links in Markdown such as https://observablehq.com and mailto:support@observablehq.com.

And more…

The new dark built-in reactive variable lets you more easily write code that adapts to light or dark mode. Our project templates now use a JavaScript config file by default, rather than TypeScript, to improve compatibility with Node 18. Deploying a project to Observable now prompts you for the desired access level rather than silently defaulting to private.

New Contributors

• @naltmann made their first contribution in #1004

Full Changelog: v1.2.0...v1.3.0

v1.2.0

SQL code blocks, powered by DuckDB 🦆

Framework now includes built-in support for client-side SQL powered by DuckDB. You can use the new SQL code blocks and sql tagged template literal to query data from CSV, TSV, JSON, Apache Arrow, and Apache Parquet files, which can either be static or generated by data loaders.

To use SQL, first register the desired tables in the page’s front matter using the sql option. Each key is a table name, and each value is the path to the corresponding data file. For example, to register a table named gaia from a Parquet file:

---
sql:
  gaia: ./lib/gaia-sample.parquet
---

To run SQL queries, create a SQL fenced code block (```sql). For example, to query the first 10 rows from the gaia table:

```sql
SELECT * FROM gaia ORDER BY phot_g_mean_mag LIMIT 10
```

SQL code blocks render results using Inputs.table:

To refer to the results of a query in JavaScript, use the id directive. (When an id is specified, the table is hidden by default, but you can display it using the display directive.) For example, to refer to the results of the previous query as top10:

```sql id=top10
SELECT * FROM gaia ORDER BY phot_g_mean_mag LIMIT 10
```

For dynamic or interactive queries that respond to user input, you can interpolate values into SQL queries using inline expressions ${…}. For example, to show the stars around a given brightness:

```js
const mag = view(Inputs.range([6, 20], {label: "Magnitude"}));
```
```sql
SELECT * FROM gaia WHERE phot_g_mean_mag BETWEEN ${mag - 0.1} AND ${mag + 0.1};
```

SQL fenced code blocks are shorthand for the sql tagged template literal. You can invoke the sql tagged template literal directly like so:

const rows = await sql`SELECT * FROM gaia LIMIT 10`;

For more, see the new SQL guide.

Other changes

Fix FileAttachment resolution in imported modules. Fix module preloads of transitive imports. Fix a race condition in display where stale values could be displayed after invalidation. Unpin the version of Apache ECharts (from 5.4.3; currently 5.5.0). Use files rather than built-in sample datasets for the default project templates.

New Contributors

• @elevenchars made their first contribution in #991

Full Changelog: v1.1.2...v1.2.0

v1.1.2

Fix yarn dlx @observablehq/create on Yarn Berry.

Full Changelog: v1.1.1...v1.1.2

v1.1.1

Adds missing dependency to fix Yarn 4.x support.

Full Changelog: v1.1.0...v1.1.1

v1.1.0

Windows support (and Node 18, too) 🎉

Framework now supports Windows! Thank you upvoting #90. 👍 Framework internally uses POSIX path delimiters (forward slashes /), and now converts to the Windows path delimiters (backward slashes \) as needed when reading or writing files. Please reach out if you encounter any issues.

We also now support Node 18. We previously had a runtime dependency on tsx, allowing us to ship Framework’s source as TypeScript; now we publish pre-built JavaScript. This reduces the number of moving parts and means we no longer depend on module customization hooks added in Node 20.6.

Self-hosted npm: imports 🎉

Framework now automatically downloads npm: imports from npm during preview and build! This means better performance and security for your users as your built site has no external code dependencies. For example, you can now enforce a content security policy on your deployed site that disallows cross-origin requests. It also improves performance during local preview since you only need to download libraries once, and means you can develop offline (say from an airplane without wi-fi). Downloads from npm are cached in the .observablehq/cache/_npm folder within the source root. You can clear the cache and restart the server to re-fetch the latest versions of libraries from npm.

Self-hosting of npm: imports applies to static imports, dynamic imports, and import resolutions (import.meta.resolve), provided the specifier is a static string literal. For example to load D3:

import * as d3 from "npm:d3";

const d3 = await import("npm:d3");

In both cases above, the latest version of d3 is resolved and downloaded from jsDelivr as an ES module, including all of its transitive dependencies. Implicit npm: imports, as when you use the built-in d3 provided by Framework’s standard library in Markdown, are also now self-hosted. (If you still want to load a library from the CDN at runtime rather than self-hosting, you can import a URL, but we don’t recommend this as it is slower and means your site could break when a new version of the library is published.)

Transitive static imports are also registered as module preloads (<link rel="modulepreload">), such that the requests happen in parallel and as early as possible, rather than being chained. This dramatically improves performance on page load. Framework also now preloads npm: imports for FileAttachment methods, such as d3-dsv for CSV files.

Import resolutions allow you to download files from npm. These files are automatically downloaded for self-hosting, too. For example, to load U.S. county geometry:

const data = await fetch(import.meta.resolve("npm:us-atlas/counties-albers-10m.json")).then((r) => r.json());

Framework automatically downloads files as needed for recommended libraries, and resolves import resolutions in transitive static and dynamic imports. For example, npm:@duckdb/duckdb-wasm needs WebAssembly bundles, npm:katex needs a stylesheet and fonts, etc. For any dependencies that are not statically analyzable (such as fetch calls or dynamic arguments to import) you can call import.meta.resolve to register additional files to download from npm.

Oh, and we’re working on jsr: imports for our next release, too. #956

CommonMark compatibility 🎉

Framework previously had a quirk with blank lines in Markdown. Now it doesn’t, so you can write bog-standard Markdown with no nasty surprises! 😌 In particular, you can now write Markdown within HTML blocks per the GitHub-flavored Markdown specification:

<div class="card">

## This is a card
### With a subtitle

And some *Markdown* text!

</div>

Content-hashed files names on build

Framework now inserts a content hash into published file names in _file and _import. This ensures cache-breaking on deploy and enables cache-control: immutable to further improve performance. When deploying to Observable, you’ll automatically get optimized caching.

Draft mode

The new draft option allows you to exclude certain pages from the build. This allows you to develop pages locally but prevent them from being published until you are ready to share. To mark a page as draft, use front matter:

---
draft: true
---

markdown-it plugins

You can now register additional markdown-it plugins using the markdownIt config option. For example, to use the markdown-it-footnote plugin:

import type MarkdownIt from "markdown-it";
import MarkdownItFootnote from "markdown-it-footnote";

export default {
  markdownIt: (md: MarkdownIt) => md.use(MarkdownItFootnote)
};

And many other improvements…

Apache ECharts is now available by default as echarts in Markdown. (Though we still prefer our own Observable Plot and D3 libraries for visualization!)

Framework now does live-updating of files referenced in static HTML, such as <img> and <video> elements, and stylesheets. Framework also does live-updating if you edit your custom stylesheet. Framework now skips unused built-in modules and minifies CSS during build.

Framework’s built-in search now correctly indexes text written in any Unicode writing system. Diacritics and HTML entities are normalized before indexing; for example, you can now type “manana” and retrieve pages containing “mañana” and “mañana”. Indexing no longer crashes when a title is not a string. The home page is now included in the index. The new keywords option allows indexing of additional boosted words in the front matter. And search results no longer show a spurious scrollbar.

FileAttachment.url() now returns an absolute URL rather than a relative path. FileAttachment.mimeType is now "application/octet-stream" instead of "null" when the MIME type is not known. Older browsers that do not yet support the CSS :has() selector are now supported. Framework no longer considers the HOSTNAME and PORT environment variables when starting the preview server; use the --host and --port command-line arguments instead. Markdown headers now have text-wrap: balance applied in default styles.

All this in less than three weeks since launch! 😅 Thanks for all the feedback and encouragement, y’all.

New Contributors

• @timmattison made their first contribution in #821
• @ikesau made their first contribution in #813
• @bertt made their first contribution in #858
• @marioangst made their first contribution in #919
• @imhalid made their first contribution in #893
• @huw made their first contribution in #964

Full Changelog: v1.0.0...v1.1.0

v1.1.0-rc.1

1.1.0-rc.1 (#973)

v1.0.0

What's Changed

• Change 'Observable Cloud' to 'Observable'. by @mcglincy in #775
• fail tests on lint warnings by @mythmon in #774
• no alt for default index by @mbostock in #779
• handle cloudflare 520 errors for file upload by @mythmon in #776
• improved plot example by @mbostock in #782
• mortage example edits by @mbostock in #785
• sort workspaces by owner (vs member) first, then alphabetical order by @Fil in #790
• ignore .env by @mbostock in #793
• search by @Fil in #543
• inputs edits by @mbostock in #792
• google-analytics edits by @mbostock in #759
• observable convert by @mbostock in #764
• File quota error message by @cinxmo in #797
• always exit with non-zero if a command throws an error by @mythmon in #799

Full Changelog: v1.0.0-rc.8...v1.0.0