There is a wide range of tools and framework currently available for doing literate programming development. You have the ancestors like CWEB, NOWEB and nuweb. You have full editors like Leo. And then you have more modern approaches like nbdev, PyWebTool and FSharp.Formatting. 

However, most of them are specific to a programming language. Some of them are general like NOWEB, but they are lacking some kind of integrations in modern IDE environments.

For the last 8 years, I always fallback to the same: Org-Mode.

Org-mode

Org-mode is many things, but its most interesting feature has always been its code blocks to me. Org’s syntax is clean and powerful. Org is not specific to a particular programming languages: it supports tens of programming languages or other kind of configuration/scripting languages. Code blocks can be executed, tangled or weaved. 

Its drawback: the best (and frankly only) Org-mode implementation is in Emacs. Some, myself included, will say it is great because we love working with Emacs, and are happily willing to pay the cost. But we are not the norm, but the exception. Emacs is wonderfully different and it doesn’t appeal to all developers. I can understand that in today’s industry where the only incentive is to ship, ship, ship features.

But, how could we get the best of Org-mode without having to force people to use Emacs? One possibility could be to develop Org-mode plugins for other IDEs, the first on the list would most likely be VS Code. But this is not a small undertaking.

Org-mode CLI

There are some modules existing in other IDEs that support org-mode like on Vim, VS Code, etc. But those are mostly syntax highlighter, or implement some features mostly related to org-agenda and headings manipulation. It is a good start, but far from enough for a literate programming framework.

The goal of OrgWeb is to develop a simple tool that any developer could use to leverage the full power of doing literate programming using Org-mode and their preferred IDE. 

OrgWeb

OrgWeb is a simple CLI tool that can be installed using this command:

pip install orgweb

The tool only has four commands:

1. tangle: extract code from code blocks into their source files
2. detangle: sync source files back to their original Org-mode code blocks
3. execute: execute code blocks such that they produce their side effects
4. monitor: monitor local file system to tangle/detangle files automatically

The tangle, detangle and execute commands can be performed on a folder (recursively) or one or multiple specific files.

Note: I am not covering all the details of how we can use Org mode to do literate programming. You can search my blog which has plenty of posts about that, but also refer to the Org-mode documentation to read about all and every features available to you.

In addition to the orgweb CLI, you will need Docker available in your environment. If it is not already installed, you can follow those instructions to install it on your system. 

VS Code + Org-mode

For this blog post, I will cover how Org-mode can be used in conjunction with VS Code to develop an application using literate programming. To start, you can simply clone OrgWeb’s repository, install this Org-mode module in VS Code. The general development layout is:

In the bottom window, this is where we have the terminal instances. This is where OrgWeb commands happens. In the main edit window, this is where the Org files, or the tangled source files will be manipulated.

You can notice that the Org-mode VS code module does some basic syntax highlighting, even within the code blocks using Python’s syntax highlighter. This is far than enough to easily understand and follow the Org files.

Tangling

Tangling is the action of extracting code blocks from a literate file into its executable source code file.

Once ready to tangle the Org file, this command will tangle that specific file:

orgweb tangle . --file main.org

It asks OrgWeb to tangle the current directory . but to only tangle the main.org file. It will find all the Org files recursively, and tangle only the ones specified. If no files are specified, it will tangle all the Org files it finds. Then the main.py file will be generated from all the code blocks from main.org.

Detangling

Developers will often end-up working on the source files that have been generated from Org files. There are all kind of reasons for that, such as modifying a source file while debugging an application. When this happens, the literate Org files and the source files get desynchronized. Changes could be copy/paste to the Org files, but there is a much easier way to do it: detangling.

Detangling synchronize back any tangled code blocks from source files to their original Org file:

orgweb detangle . --file main.py

It asks OrgWeb to detangle the current directory . but to only detangle the main.py file. It will find all the Org files recursively, and tangle only the ones specified. Then the main.org file will be generated from all the code blocks from main.py.

Executing

The execute command is like the tangling command but instead of moving code in source files, it does execute the code blocks that needs to be executed. Code blocks that get executed produces side effects. It is those side effects that we want to force with the execute command.

One example are the PlantUML code blocks in the README.org file. When we execute them, the schema images will be generated.

Monitoring

This is all good, but it is still inconvenient to have to run commands in the terminal every time you want to tangle or detangle some files.

This is why there exists the monitor command:

The orgweb monitor . command will keep monitoring the specified folder. Every file that changes within that folder (recursively) will potentially be tangled or detangled by the running orgweb instance. If a .orgwebignore file exists in the target folder, then everything within that file will be ignores by the monitoring process.

In the envisioned development workflow, developers will simply run the monitoring in background such that every Org and source files automatically gets tangled and detangled every time they are saved. That way, developers will be sure that both files are always in sync.

How does it work?

As we know, orgweb is designed in a way that developers can use all the power of Org-mode, in any IDE they like, without having to rely on Emacs directly.

To do that, it leverages Docker to build an image where Emacs is properly installed and configured to implement the commands that are exposed via the command line tool.

If the orgweb docker image is not currently existing in the environment, then it will request Docker to build the image using the Dockerfile. The build process will install and configure all the components required to implement all orgweb commands.

orgweb check if it exists every time it is invoked from the command line. This process will happen any time that the image is not available in the environment.

If the image is existing in the environment, then the following will happen.

orgweb will ask Docker to create a container based on the image. Once the container is running, it will execute a command on the container’s terminal to run Emacs. Emacs is used directly from the command line by evaluating ELisp code it gets as input.

Every time a orgweb command line is executed, a new container is created and when the commands finishes, the container gets deleted:

Other possible avenues

OrgWeb works fine, but it won’t ever be as interesting as a proper IDE integration, like what is available in Emacs. Another interesting option worth investigating would be to use Emacs as a Org-mode backend of a LSP server. That way, IDE modules developers could more easily develop fully fledged Org-mode modules for specific IDE integration. That way, we could “easily” get the full Org-mode power within any IDE, being able to not only leverage code blocks but org-agenda, org-roam, tagging, date time, org-capture, etc.