Denote Explore: Explore your Denote digital garden

The Denote-Explore package provides auxiliary functions to maintain and explore your collection of Denote files. Denote-Explore provides three groups of Emacs commands:

1. Summary statistics: Count and visualise notes, attachments and keywords.
2. Random walks: Generate new ideas using serendipity.
3. Janitor: Manage your denote collection.
4. Visualisations: Visualise your Denote network as a network graph.

This package exists because I started to move many files on my computer into Denote silos, leveraging the ingenious file naming convention and associated functionality. These functions helped me to add some structure to the organic chaos of my digital garden.

Read the denote-explore manual for a comprehensive explanation of all functionality (C-h R denote-explore).

This package is available on MELPA and is easily installed with the use-package macro.

The configuration below sets all available variables. The first two variables define where graph files are stored, sans extension. If the directory does not yet exist, then the package creates it when required. The third variable defines the output format and as such the file extensions. Possible options are graphviz, D3.js (JSON) and GEXF. You will need to install GraphViz and/or the R language to visualise Denote graphs.

(use-package denote-explore
  :custom
  ;; Location of graph files
  (denote-explore-network-directory "~/documents/notes/graphs/")
  (denote-explore-network-filename "denote-network")
  ;; Output format
  (denote-explore-network-format 'graphviz)
  (denote-explore-network-graphviz-filetype "svg")
  ;; Exlude keywords or regex
  (denote-explore-network-keywords-ignore '("bib")))

Alternatively, you can install it directly from GitHub:

(unless (package-installed-p 'denote-explore)
  (package-vc-install
   '(denote-explore
     :url "https://github.com/pprevos/denote-explore/")))

Change Log

Version 3.0

• Compatability with Denote 3.0
• Renamed barchart functions to all start with denote-explore-barchart-.
• Using universal argument with denote-explore-barchart-extensions visualises only attachment file types.
• All functions that rename files updated for compatability with Denote 3.0
• Enhanced info manual

Version 1.6

• New functions:
  • denote-explore-backlinks-barchart to view top-n notes with backlinks
  • denote-explore-random-regex to jump to random note matching a regular expression.
  • denote-explore-identify-duplicate-notes-dired: View notes with duplicates ID in Dired.

Version 1.5.1

• Fixed various compilation errors.
• Enhanced output for denote-explore-identify-duplicate-notes
• Update customization types for note ignore variables
• Updated defgroup definition and defcustom types

Version 1.5

• Published Info mode manual
• Clarified obsolete variables and functions
• denote-explore-network-keywords no longer interactive
• Updates to Denote functions

Version 1.4.2

• Geneal code refactoring with assistance from Prot
• Added ability to filter keywords network by minimum weight of edges
• Added ability for free text in denote-explore-network-graphviz-filetype
• GEXF format now includes the total degree for each node
• Enforced consistency with use of universal argument (its use will include attachments)
• denote-explore--retrieve-title deprecated. Now using Denote function.
• Updated manual with clarifications

Version 1.4.1

• Fixed bugs with random links and keywords from nil file-name
• New variable denote-explore-network-regex-ignore to ignore files matching a regular expression when creating neighbourhood or community graphs.

Version 1.4

• Network visualisation
  • SVG output includes links to source file.
  • Configure GraphViz output format with denote-explore-network-graphviz-filetype (SVG (default) or PDF advised).
  • Improved scaling of GraphViz graphics.
  • denote-explore-network-regenerate recreates the previous graph
• New functions:
  • denote-explore-isolated-notes to select Denote files without any links or backlinks.
  • denote-explore-degree-barchart draws the distribution of total number of (back)links per file (Degree distribution - Wikipedia).
• Updated functions:
  • denote-explore-rename-keyword allows renaming more than one keyword to a new version.
  • denote-explore-sync-metadata enforces renaming confirmation.
• Minor bug fixes.

Version 1.3

• Complete rewrite of the graph visualisation functionality. Now includes three formats (JavaScript, GraphViz and GEXF) and three graph types (notes that match a regular expression, note neighbourhood and keyword graph).
• Fixed bugs in denote-explore--retrieve-title and denote-explore--retrieve-keywords.
• Enhanced function and variable documentation.

Version 1.2

• Removed pandoc dependency in network generation.
• denote-explore-identify-duplicate-notes replaces denote-explore-identify-duplicate-identifiers. This new version either detects duplicate identifiers, or duplicate file names (using the universal argument). Comparing duplicate filenames ignores any duplicate identifiers caused by exporting Org mode files.

Version 1.1

• Added helper function to call R script for network visualisation.
• denote-explore-dashboard.el deprecated.
• Added to MELPA.

Version 1.0

This version is a complete rewrite of the code to improve coding quality and compatibility with internal changes since Denote version 2.2.

Also major changes to the network visualisation, which can now take a regular expression for partial networks. Most of the code is now undertaken in Emacs Lisp, which saves that Denote notes and attachment structures in JSON format.

The functionality for a Dashboard widget has been moved to a separate file to not enforce the requirements for those users not seeking to implement the widget.

New functionality since the previous version:

• denote-explore-identify-duplicate-identifiers: Provide a list of duplicate identifiers.
• denote-explore-single-keywords: Select a note or attachment with a keyword that is only used once.
• denote-explore-zero-keywords: Select a note or attachment without any keywords.
• denote-explore-sort-keywords: Order the keywords of all Denote notes and attachments alphabetically.
• denote-explore-rename-keyword: Rename or remove a keyword across the whole Denote collection.
• denote-explore-sync-metadata: Synchronise the filenames with the metadata for all Denote files.

Ideas for future development

• [-] Create a Denote widget for the Emacs Dashboard with links to statistics.
• [-] Functions to rename Denote attachments using EXIF metadata.
• [-] Jump to random note within a time frame

Visualising Denote

• [-] Ring of previously-generated networks instead of overwriting denote-explore-network-previous
• [ ] Dynamic org mode blocks to visualise the neighbourhood of a node
• [ ] Use org-graph-view for live neighbourhood view of the current buffer
• [ ] Community detection in GraphViz versions?
• [ ] Visualise signatures (example)
• [ ] Store the Denote metadata in a hash table that is updated after every modification to a Denote file for more efficient visualisation.
• [ ] The current network visualisation uses the networkD3 package of the R language as a detour to generate a HTML file with associated JavaScript code. I am slowly working my way to not have to use R, but I first need to learn more JavaScript and the D3 visualisation language.
• [ ] Scaling GraphViz text size based on number of nodes
• [ ] Use a plain text method to perhaps visualise the network neighbourhood. The hierarchy package can display information in a tree format, which might be helpful to visualise the local community of a Denote buffer, e.g. two links backwards and forwards.
• [ ] Plain text time series graph of the creation of new notes.