Allow embedded SVG symbols via base 64 encoding

[nyalldawson]

QGIS Enhancement: Allow embedded SVG symbols via base 64 encoding

Date 2018/05/24

Author Nyall Dawson (@nyalldawson)

Contact nyall.dawson@gmail.com

maintainer @nyalldawson

Version QGIS 3.4

Summary

This proposal covers extending QGIS' support for SVG file handling (which currently supports file paths and http remote urls) to allow SVG files embedded within symbol definitions via base 64 encoding. This would allow SVG symbols to be embedded within QGIS projects, symbol libraries, QML files and QPT print templates.

Base64 encoding is a standard method for safely encoding binary data (such as PNG and SVG files) within an ascii string. Base64 encoding is notably used heavily within CSS for embedding PNG and SVG icons directly inside CSS text. These properties make it an ideal candidate for embedding image files directly inside QGIS symbol definitions/projects, and the widespread use of base64 mean there is a multitude of tools available for converting files to/from base64 representation. Additionally, the Qt QByteArray class already has full support for encoding/decoding base64, making implementation within QGIS simple.

Proposed Solution

Core changes

QgsSvgCache, which is the common class used for rendering all SVG images within QGIS (used across the application, including symbology, labeling, layouts, etc), will be extended to allow paths which begin with a base64: prefix.

The bulk of changes here will be modification of the QgsSvgCache::getImageData method, with the addition of the code:

  // maybe it's an embedded base64 string
  if ( path.startsWith( QLatin1String( "base64:"),Qt::CaseInsensitive ) )
  {
    QByteArray base64 = path.mid(7).toLocal8Bit(); // strip 'base64:' prefix
    return QByteArray::fromBase64( base64, QByteArray::OmitTrailingEquals );
  }

Appropriate unit tests will also be added to ensure that base64 decoding of SVG paths works correctly.

Impact on QGS/QPT/QML files

When a base64 encoded image is stored within a QGIS project, QML symbol definition, or QPT print layout template, the base64 content will be directly embedded inside the corresponding XML. E.g. Using a base64 encoded version of https://github.com/mapbox/maki/blob/master/icons/art-gallery-15.svg inside a SVG marker results in the following XML:

<symbol type="marker" clip_to_extent="1" name="0" alpha="1">
  <layer class="SvgMarker" locked="0" pass="0" enabled="1">
     ...
     <prop v="249,216,123,255" k="color"/>
     <prop v="base64:PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz48IURPQ1RZUEUgc3ZnIFBVQkxJQyAiLS8vVzNDLy9EVEQgU1ZHIDEuMS8vRU4iICJodHRwOi8vd3d3LnczLm9yZy9HcmFwaGljcy9TVkcvMS4xL0RURC9zdmcxMS5kdGQiPjxzdmcgdmVyc2lvbj0iMS4xIiBpZD0iTGF5ZXJfMSIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIiB4bWxuczp4bGluaz0iaHR0cDovL3d3dy53My5vcmcvMTk5OS94bGluayIgeD0iMHB4IiB5PSIwcHgiIHdpZHRoPSIxNXB4IiBoZWlnaHQ9IjE1cHgiIHZpZXdCb3g9IjAgMCAxNSAxNSIgc3R5bGU9ImVuYWJsZS1iYWNrZ3JvdW5kOm5ldyAwIDAgMTUgMTU7IiB4bWw6c3BhY2U9InByZXNlcnZlIj48cGF0aCBkPSJNMTAuNzEsNEw3Ljg1LDEuMTVDNy42NTU1LDAuOTUzOSw3LjMzOSwwLjk1MjYsNy4xNDI5LDEuMTQ3MUM3LjE0MTksMS4xNDgxLDcuMTQxLDEuMTQ5LDcuMTQsMS4xNUw0LjI5LDRIMS41QzEuMjIzOSw0LDEsNC4yMjM5LDEsNC41djlDMSwxMy43NzYxLDEuMjIzOSwxNCwxLjUsMTRoMTJjMC4yNzYxLDAsMC41LTAuMjIzOSwwLjUtMC41di05QzE0LDQuMjIzOSwxMy43NzYxLDQsMTMuNSw0SDEwLjcxeiBNNy41LDIuMjFMOS4yOSw0SDUuNzFMNy41LDIuMjF6IE0xMywxM0gyVjVoMTFWMTN6IE01LDhDNC40NDc3LDgsNCw3LjU1MjMsNCw3czAuNDQ3Ny0xLDEtMXMxLDAuNDQ3NywxLDFTNS41NTIzLDgsNSw4eiBNMTIsMTJINC41TDYsOWwxLjI1LDIuNUw5LjUsN0wxMiwxMnoiLz48L3N2Zz4=" k="name"/>
     ...
  </layer>
</symbol>

GUI Changes

In order to expose this functionality, the current button which is used for selecting an SVG file for the SVG marker/fill and generic SVG selector will be morphed into a QToolButton, with a new drop down menu accessible via an arrow on the side of the button:

Clicking this button will show a menu with the options:

• Select File (current behavior)
• Embed File (opens a file picker, and when a file is selected it will be base64 encoded into the svg path)
• From URL (opens a simple text input dialog asking user for a http:// URL - while it's possible to directly enter a http URL into the symbol path, this functionality is not widely known and explicitly exposing it as an option in the GUI will increase its discoverability)

Additionally, if the current SVG source is an embedded SVG, an "Extract Embedded File" action will be shown. Selecting this option allows users to save the current embedded file out to a standard SVG file in a location of their choosing.

Affected Files

• qgssvgcache.cpp
• widget_svgselector.ui
• qgssymbollayerwidget.h/.cpp
• qgssvgselectorwidget.h/.cpp

Performance Implications

None - QgsSvgCache caches svg renders, so there will be no significant performance cost associated with decoding base64 embedded images. The main cost will be the extra file size for the affected QGS/QML/QPT files, but this feature is entirely optional - so users can continue using file paths or remote urls as SVG symbol sources without change.

Backwards Compatibility

N/A

Issue Tracking ID(s)

https://issues.qgis.org/issues/11234

Votes

(required)

[peterisb]

Great feature! Would be complicated to implement feature able embed all used images with path or URL in project file, somewhere in project properties?

[nyalldawson]

@peterisb

It's not trivial! I'd suggest this could be a good candidate for a plugin as a proof of concept first.

[luipir]

@nyalldawson useful with low impact enhancement. Make sense to have a feature to decode/save an embedded svg as file e.g. for editing reason.

[nyalldawson]

Implemented here (no unit tests yet): nyalldawson/QGIS@2db28ae

The vast bulk of the changes are just hooking up the new gui actions - the actual core change is only 4 lines 😄

A follow up commit nyalldawson/QGIS@5d379cb uses this to automatically style ArcGIS feature server layers with embedded picture markers on load - converting them to an embedded svg file. It's a bit inefficient, because we have to take the original base64 encoded png marker from the server, and then embed this in an SVG document, and THEN base64 encode that svg! (When we get a proper "raster image marker" this would be much simpler!). End result is that loading a point marker from an AFS server results in QGIS showing the original point symbol pictures, no user effort required:

[peterisb]

@nyalldawson I fully agree - proof of concept plugin first.

[wonder-sk]

Really good stuff here!

This made me think how we would make it possible to embed SVG (and other) files inside project files (.qgz) and how users would deal with such embedded files, but I don't want to steer the discussion away from base64 encoding :-)

[nyalldawson]

@wonder-sk I thought the same, but in the end there's a strong use case for embedded files outside of projects. I'm thinking QML and symbol libraries, qpt templates, even plugins and scripts which have "embedded" symbols via the base64 strings.

Plus, it's considerably simpler than handling real embedded files ;)

[nyalldawson]

Implemented at qgis/QGIS#7433

[Vitruvius21]

Hello Nyall, there is an issue regarding complex SVG depiction. As you see in the gif when I embed "monuments SVGs" and then clicked on the base64 code it disappears. But in case of simple SVGs the issue is not actual. The same happens when I link base64 encoding from a layer data table.

[nyalldawson]

@Vitruvius21 that's fixed in 3.4.3

[Vitruvius21]

@nyalldawson I have the latest 3.4.3. If you wish I can give you exemplar of the svg to test.

[Vitruvius21]

@nyalldawson do you observe that issue on your pc?

[saberraz]

@Vitruvius21 Please consider filing a bug here:
https://issues.qgis.org/projects/qgis/issues

QEP is a place for discussions about implementation of a certain feature amongst the devs.

[nyalldawson]

Please test with 3.6 nightly - I'm confident this one is already fixed.