Literate Programming, Programming, Emacs, Clojure

Literate [Clojure] Programming: Tangle All in Org-mode

This blog post is the fifth of a series of blog posts about Literate [Clojure] Programming in Org-mode where I explain how I develop my [Clojure] applications using literate programming concepts and principles.

This new blog post introduce a tool that is often necessary when developing literate applications using Org-mode: the tangle all script. As I explained in a previous blog post, doing literate programming is often like writing: you write something, you review and update it… often. This means that you may end-up changing multiple files in your Org-mode project. Depending how you configured you Emacs environment and Org-mode, you may have missed to tangle a file you changed that may cause issues down the road. This is the situation I will cover in this post.

This series of blog posts about literate [Clojure] programming in Org-mode is composed of the following articles:

  1. Configuring Emacs for Org-mode
  2. Project folder structure
  3. Anatomy of a Org-mode file
  4. Tangling all project files (this post)
  5. Publishing documentation in multiple formats
  6. Unit Testing

Emacs: the Programmable Text Editor

One of the beauty of Emacs is that it is fully programmable. It is open source, anything can be accessed, changed and modified: even while running. It can be done because at its core, it is an Emacs Lisp interpreter. This is the feature of the development environment that I leveraged to create a series of tools that I can use in my Org-mode literate programming projects.

Literate [Clojure] Programming Project Folder Structure

Let’s take a look at the folder structure of a literate [Clojure] programming project:

- resources
- org
  - org_mode_clj_tests_utils
- pom.xml
- project.clj
- src
  - org_mode_clj_tests_utils
    - core.clj
- target
- test
  - org_mode_clj_tests_utils
    - core_test.clj

In the root org folder we have setup and configuration files, but also some tool files. One of them is which we will cover below. This is where I add such tools that helps me with some specific tasks when developing literate programming applications in Emacs.

In my blog post Optimal Emacs Settings for Org-mode for Literate Programming I have a section called Change Behaviors On Save which explains how to configure Emacs and Org-mode to trigger some specific behavior when saving a file. One of these behavior is to automatically tangle the file that is being saved, such that you don’t forget to tangle it down the road. This is really handy and it works 95% of the time. However there are situations when this won’t fully work. For example, the file may be changed using another text editor than Emacs. Or someone else that would like to use your literate program may not have your modifications to the save behavior, etc. This is for these situations that such a tangle-all script comes handy: the tool ensure you that you tangled all the files of your application.

Tangle All

I choose to create, and use, the tangle-all function in an Org-mode file. It could easily have been created as an Emacs package, or simply added in the .emacs configuration file. However having it as Org files is handy. It becomes really portable. It follows the literate programming project’s code. It comes really handy when the literate program is shared in a repository like on GitHub. It becomes instantly available for other to be used without having to search and install a third party package or to hack their Emacs text editor. They only have to run the code in Org-mode and they get the capability without having to worry about anything else.

The (tangle-all) function uses the (directory-files-recursive) function adapted from Daniel M. German’s work. The file is located at the root of the org folder. What it does is to recursively check for Org-mode files from that location on the file system, and tangle all of them.

(defun directory-files-recursive (directory match maxdepth)
  "List files in DIRECTORY and in its sub-directories. 
   Return files that match the regular expression MATCH. Recurse only 
   to depth MAXDEPTH. If zero or negative, then do not recurse"
  (let* ((files-list '())
          (directory-files directory t)))
    ;; while we are in the current directory
    (while current-directory-list
      (let ((f (car current-directory-list)))
           (file-regular-p f)
           (file-readable-p f)
           (string-match match f))
          (setq files-list (cons f files-list)))
           (file-directory-p f)
           (file-readable-p f)
           (not (string-equal ".." (substring f -2)))
           (not (string-equal "." (substring f -1)))
           (> maxdepth 0))     
          ;; recurse only if necessary
          (setq files-list (append files-list (directory-files-recursive f match (- maxdepth -1))))
          (setq files-list (cons f files-list)))
      (setq current-directory-list (cdr current-directory-list)))

(defun tangle-all ()
  "Tangle all the Org-mode files in the directory of the file of the current buffer
   recursively in child folders. Returns the list of tangled files"
  (mapcar (lambda (f)
            (when (not (file-directory-p f))
              (org-babel-tangle-file f)))
          (directory-files-recursive (file-name-directory (buffer-file-name)) "\\.org$" 20)))

The only thing a user as to do to tangle all the files is to open the file and then to execute all the code blocks in the current buffer using C-c C-v b. That will evaluate the (tangle-all) function and then it will call it. Finally the list of tangled files will be output.


As you can see, it is really easy to create tools to help us managing our literate programming applications. n the next and final article of this series, I will introduce another such tool used to weave all documents in multiple different file formats.

Leave a Reply