{"id":3234,"date":"2016-06-21T15:59:25","date_gmt":"2016-06-21T19:59:25","guid":{"rendered":"http:\/\/fgiasson.com\/blog\/?p=3234"},"modified":"2017-02-17T10:39:20","modified_gmt":"2017-02-17T15:39:20","slug":"optimal-emacs-settings-for-org-mode-for-literate-programming","status":"publish","type":"post","link":"https:\/\/fgiasson.com\/blog\/index.php\/2016\/06\/21\/optimal-emacs-settings-for-org-mode-for-literate-programming\/","title":{"rendered":"Optimal Emacs Settings for Org-mode for Literate Programming"},"content":{"rendered":"

For some time I have been interested in using Emacs and Org-mode for developing Clojure in a Literate Programming way. I discussed the basic ideas<\/a>, some of the benefits of doing so<\/a>, etc<\/a>, etc<\/a>. It is now time to start showing how I am doing this, what are the rules of thumb I created, what is the structure of my programs, etc.<\/p>\n

However, before I start writing about any of this, I think the next step is to explain how I configured Org-mode to have a frictionless<\/i> experience to develop my applications in Literate Programming using Org-mode. Then in a subsequent series of blog posts I will explain how I structured my Clojure project, what is my development workflow, etc.<\/p>\n

Note that if you don’t have Emacs setup for Clojure\/Cider, I would encourage you to read this other blog post<\/a> which explains how to setup a Clojure environment in Emacs.<\/p>\n

This is the first post of a series of blog posts that will cover the full workflow. I will demonstrate how I do Literate Programming for developing a Clojure application, but exactly the same workflow would work for any other programming language supported by Org-mode (Python, R, etc.). The only thing that is required is to adapt the principles to the project structures in these other languages. The series of blog posts will cover:<\/p>\n

    \n
  1. Configuring Emacs for Org-mode<\/a> (this post)<\/li>\n
  2. Project folder structure<\/a><\/li>\n
  3. Anatomy of a Org-mode file<\/a><\/li>\n
  4. Tangling all project files<\/a><\/li>\n
  5. Publishing documentation in multiple formats<\/a><\/li>\n
  6. Unit Testing<\/a><\/li>\n<\/ol>\n

    <\/p>\n

    \n

    Packages<\/h3>\n
    \n

    Once Emacs is installed with Cider<\/a>, the first thing you have to do is to install all the packages that are required to use Org-mode and some of the tweaks that I am proposing. The packages that you have to install are:<\/p>\n

      \n
    1. org<\/a><\/li>\n
    2. adaptive-wrap<\/a><\/li>\n
    3. htmlize<\/a><\/li>\n<\/ol>\n

      They can easily be installed using the following commands:<\/p>\n

      M-x package-install [RET] org [RET]\nM-x package-install [RET] adaptive-wrap [RET]\nM-x package-install [RET] htmlize [RET]\n<\/pre>\n
      Additionally, you could have used M-x<\/code>package-list-packages<\/code>, then move your cursor in the buffer to the packages\u00e2\u20ac\u2122 line. Then press i<\/code> (for install) and once all the packages are selected, you could have press x<\/code> (execute) to install all the packages all at once.<\/p>\n<\/div>\n<\/div>\n
      \n
      Configure Org-mode Editor Behaviors<\/h3>\n
      \n
      Org-mode is installed with a series of default settings, but I don’t like all of them when come the time to do Literate Programming with it. The first thing I like doing is to hide some of the Org-mode markup and to replace it by the actual font. For example, I don’t want to see \/test\/<\/code> in my text, I want to see the (italized) version of test<\/i>. Same for bold, links, etc. This can be done by setting org-hide-emphasis-markers<\/code>.<\/p>\n
      Then I want to turn the Emacs visual line mode, but only for Org-mode. This basically wraps long lines without justifying it.<\/p>\n
      \n
      (<\/span>require<\/span> 'org<\/span>)<\/span>\n\n;; <\/span>Remove the markup characters, i.e., \"\/text\/\" becomes (italized) \"text\"<\/span>\n(<\/span>setq org-hide-emphasis-markers t)<\/span>\n\n;; <\/span>Turn on visual-line-mode for Org-mode only<\/span>\n;; <\/span>Also install \"adaptive-wrap\" from elpa<\/span>\n(<\/span>add-hook 'org-mode-hook 'turn-on-visual-line-mode)<\/span>\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n
      \n
      Configure Cider for Org-mode<\/h3>\n
      \n
      Since we want to do Literate Programming using (among others) Clojure, what we have to do is to configure Cider to be the Clojure backend to be used by Org-mode. Org-mode’s code block<\/code> feature supports a long list how programming language. However, each of these programming languages are supported by a backend except for a few like Elisp (which uses Emacs as the backend). This is what we are doing here for Clojure.<\/p>\n
      What we do here is first to specify to Org-mode which language we want to support with its code blocks. I only defined four for now: clojure<\/code>, sh<\/code>, dot<\/code> and elisp<\/code>.<\/p>\n
      Then we specify that the org-babel-clojure-backend<\/code> we want to use is Cider<\/code>.<\/p>\n
      \n
      ;; <\/span>Configure org-mode with Cider<\/span>\n\n;; <\/span>Configure Org-mode supported languages<\/span>\n(<\/span>org-babel-do-load-languages\n 'org-babel-load-languages\n '(<\/span>(<\/span>clojure . t)<\/span>\n   (<\/span>sh . t)<\/span>\n   (<\/span>dot . t)<\/span>\n   (<\/span>emacs-lisp . t)<\/span>)<\/span>)<\/span>\n\n;; <\/span>Use cider as the Clojure execution backend<\/span>\n(<\/span>setq org-babel-clojure-backend 'cider)<\/span>\n(<\/span>require<\/span> 'cider<\/span>)<\/span>\n<\/pre>\n<\/div>\n<\/div>\n
      \n
      Useful Key Bindings<\/h4>\n
      \n
      There are a few key bindings in Cider that are handy to have directly in Org-mode. We can easily add them like this. What I did here is to define two new: one for evaluating the last Clojure expression bound to C-x C-e<\/code>. Then the binding to get the documentation of a symbol bound to C-c C-d<\/code>.<\/p>\n
      If you wonder why this is necessary, it is because each Org-mode code block is not<\/b> running the Clojure major mode, so these key bindings are not available directly within a Org-mode file. This is why we have to remap the ones we want to use directly within Org-mode.<\/p>\n
      \n
      ;; <\/span>Useful keybindings when using Clojure from Org<\/span>\n(<\/span>org-defkey org-mode-map \"\\C-x\\C-e\"<\/span> 'cider-eval-last-sexp)<\/span>\n(<\/span>org-defkey org-mode-map \"\\C-c\\C-d\"<\/span> 'cider-doc)<\/span>\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n
      \n
      Configuration of Custom Features<\/h4>\n
      \n
      In my Using Clojure in Org-mode and Implementing Asynchronous Processing<\/a> I discuss some limitations of the current implementation of org-babel-clojure<\/code>. In that blog post I show how these limitations can be overcome. One of the setting I need to change is to remove the timeout of the nrepl<\/code> middleware. It can be done that way:<\/p>\n
      Only set that setting if you updated org-bable-clojure<\/code> with the information from that blog post.<\/p>\n
      \n
      ;; <\/span>No timeout when executing calls on Cider via nrepl<\/span>\n(<\/span>setq org-babel-clojure-sync-nrepl-timeout nil)<\/span>\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
      \n
      Configure Code blocks Behaviors<\/h3>\n
      \n
      The code block in Org-mode has their own behaviors that can be modified like the default number of spaces for indentations, if you can use the shift<\/code> key, the use the mode’s (Clojure in this code) native tab behavior and fontification (syntax highlighting), etc.<\/p>\n
      \n
      ;; <\/span>Let's have pretty source code blocks<\/span>\n(<\/span>setq org-edit-src-content-indentation 0\n      org-src-tab-acts-natively t\n      org-src-fontify-natively t\n      org-confirm-babel-evaluate nil\n      org-support-shift-select 'always)<\/span>\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n
      \n
      Change Behaviors On Save<\/h3>\n
      \n
      The most important configuration I did related to use Org-mode as a Literate Programming environment are all the behaviors that occurs when I save a org-mode document.<\/p>\n
      The purpose of Literate Programming is to write a computer software while writing about its development process, its purpose, its implementation, etc. However, every time I save a literate file, I want the literate environment to extract (tangle<\/i>) the code from that file into a [Clojure] source file that can be executed. What I did to enable this behavior is to add a function to call for Emacs’ after-save-hook<\/code>. What the function does is to make sure that the buffer being saved is a Org-mode buffer. If it is, then I run Org-mode’s tangling procedure that will tangle the Org file on save.<\/p>\n
      However, given the nature of Literate Programming in Clojure, it is often the case that you will have another buffer where the tangled source file is open. What this means is that if you change some code in a Org file that get tangled on save, then the buffer where this code file is open won’t automatically be refreshed with the newly tangled code. To fix this issue, I set the global-auto-revert-mode<\/code> which means that as soon as a file changes on the file system, if it is open in an Emacs buffer, then this buffer will be refreshed with that content.<\/p>\n
      Finally, because Org-mode is not only about code blocks, I also enabled a final behavior when I save a Org file. What I often do is to leave TODO<\/code> tasks at different places in my Org file to tell me what some work needs to be done at that place. However, once you start developing multiple projects with Org-mode, and when you start using Org-mode for others of its features, there is no way to track where you left TODO items across your entire computer system (and not just programing projects!). This is why Org-mode created a global list of TODO items via its agenda<\/code> feature. To see the list of all the TODOs across all the Org files you created, you can access it using: M-x org-todo-list<\/code>. However the problem here is that each of the Org file you want to have accessible in your agenda, you have to push<\/i> it to the agenda system. It is not a problem in itself, but it becomes a problem is you forget to push<\/i> each relevant Org file to the agenda. This is why I choose to automatically push<\/i> any Org file to the Org agenda every time a Org file is being saved. That way I don’t have to worry when I check the global list of TODOs, I am sure that all of them are there.<\/p>\n
      \n
      ;; <\/span>Tangle Org files when we save them<\/span>\n(<\/span>defun<\/span> tangle-on-save-org-mode-file<\/span>()<\/span>\n  (<\/span>when<\/span> (<\/span>string= (<\/span>message \"%s\"<\/span> major-mode)<\/span> \"org-mode\"<\/span>)<\/span>\n    (<\/span>org-babel-tangle)<\/span>)<\/span>)<\/span>\n\n(<\/span>add-hook 'after-save-hook 'tangle-on-save-org-mode-file)<\/span>\n\n;; <\/span>Enable the auto-revert mode globally. This is quite useful when you have <\/span>\n;; <\/span>multiple buffers opened that Org-mode can update after tangling.<\/span>\n;; <\/span>All the buffers will be updated with what changed on the disk.<\/span>\n(<\/span>global-auto-revert-mode)<\/span>  \n\n;; <\/span>Add Org files to the agenda when we save them<\/span>\n(<\/span>defun<\/span> to-agenda-on-save-org-mode-file<\/span>()<\/span>\n  (<\/span>when<\/span> (<\/span>string= (<\/span>message \"%s\"<\/span> major-mode)<\/span> \"org-mode\"<\/span>)<\/span>\n    (<\/span>org-agenda-file-to-front)<\/span>)<\/span>)<\/span>\n\n(<\/span>add-hook 'after-save-hook 'to-agenda-on-save-org-mode-file)<\/span>\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n
      \n
      Export Configurations<\/h3>\n
      \n
      There are a few things you can do regarding how you export your org files. One thing I like to do is to set org-html-htmlize-output-type<\/code> to css<\/code> (default is inline-css<\/code>) such that it does not include the CSS in the exported HTML. I prefer using the CSS that comes with the HTML themes I use. However, when I need inline CSS (like when I export HTML to be displayed elsewhere, i.e. on my blog) then I simply define a elisp<\/code> code block to set org-html-htmlize-output-type<\/code> back to inline-css<\/code> to reverse that behavior for that special usecase.<\/p>\n
      \n
      ;; <\/span>make sure that when we export in HTML, that we don't export with inline css.<\/span>\n;; <\/span>that way the CSS of the HTML theme will be used instead which is better<\/span>\n(<\/span>setq org-html-htmlize-output-type 'css)<\/span>\n<\/pre>\n<\/div>\n<\/div>\n
      \n
      Enable External Exporters<\/h4>\n
      \n
      There are tons of Org-mode export plugins, but not all of them are enabled by default. For contributed exporters, you will have to get it from the contrib\/lisp<\/code> folder on the Git repository<\/a> and save it in your else repository in [home]\/.emacs.d\/elpa\/[org-20160623]\/<\/code> Here is an example of how you can enable a new one which is the exporter for Confluence. Then you can export typing M-x org-confluence-export-as-confluence<\/code><\/p>\n
      \n
      ;; <\/span>Enable Confluence export<\/span>\n(<\/span>require<\/span> 'ox-confluence<\/span>)<\/span>\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
      \n
      Dire Configuration<\/h3>\n
      \n
      I personally use Dire a lot. However it displays everything by default which may not be optional, particularly when working within Org related directories with all the auto-save files that get generated. This is why I like to filter out a few things such that everything is not being displayed in the Dire buffer.<\/p>\n
      \n
      ; <\/span>Remove autosave and other unnecessary files to see in Dire<\/span>\n(<\/span>require<\/span> 'dired-x<\/span>)<\/span>\n(<\/span>setq-default dired-omit-files-p t)<\/span> ; <\/span>Buffer-local variable<\/span>\n(<\/span>setq dired-omit-files \"^\\\\.?#\"<\/span>)<\/span>\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n
      \n
      Spell checker<\/h3>\n
      \n
      It is always convenient to have a spell checker in Org-mode. Right now I am using ispell<\/code> along with flyspell<\/code>. That works fine, but I don’t like the fact that the last aspell version for Windows is about 14 years old! Any idea to improve this situation would be greatly welcome!<\/p>\n
      The first step that is required to enable this feature is to download GNU Aspell (in my case, for Windows)<\/a>. Then we have to instruct Emacs where the aspell dictionary is located, and then we have to enable Flyspell<\/code> for text modes. Finally, make sure to install the appropriate language pack as well.<\/p>\n
      \n
      (<\/span>custom-set-variables\n '(<\/span>ispell-program-name \"c:\\\\Program Files (x86)\\\\Aspell\\\\bin\\\\aspell.exe\"<\/span>)<\/span>)<\/span>\n\n;; <\/span>Enable Flyspell for text modes<\/span>\n(<\/span>add-hook 'text-mode-hook 'flyspell-mode)<\/span>\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n
      \n
      DOT support<\/h3>\n
      \n
      DOT<\/a> is like a markup language for describing graphs. It is really simple to use and generate effective graph images that can easily be embedded into your Org-mode files.<\/p>\n
      The first step is to install Graphviz<\/a> on your computer. This is the library that will be used to generate the images from the DOT specification. The only thing you have to do is to make sure that Graphviz’s bin<\/code> directory is in the Path<\/code> environment variable and you are done.<\/p>\n
      Once Graphviz is installed and configured, restart Emacs and start using it right away, no other configuration is required. Here is an example of a class hierarchy created using DOT:<\/p>\n
      \n
      digraph {\n  soloist -> \"musical performer\"<\/span>;\n  \"musical performer\"<\/span> -> musician;\n  musician -> artist;\n  artist -> person;\n  person -> human;\n  author -> artist;\n  \"scifi writer\"<\/span> -> author;\n  journalist -> author;\n  correspondent -> journalist;\n}\n<\/pre>\n<\/div>\n
      \n
      \"actors-authors-humans\"<\/p>\n<\/div>\n<\/div>\n<\/div>\n
      \n
      Inline Images Display<\/h3>\n
      \n
      One essential feature of Org-mode to make it a useful Notebook application is to be able to have inline images (that we generate from code blocks or that are somewhere on the file system) directly in Emacs. Depending on your Emacs distribution, you may require to download and install a few libraries in order to make this working properly (at least on Windows).<\/p>\n
      The first step is to make sure that Org-mode does display inline images by default. If you don’t want this behavior, you can always use the key binding C-c C-x C-v<\/code> to toggle this behavior. If you want to enable this by default when Emacs enter in Org-mode, then you have to add the following to your .emacs<\/code> file:<\/p>\n
      \n
      ;; <\/span>Enable inline image when entering org-mode<\/span>\n;; <\/span>Make sure you have all the necessary DLL for image display<\/span>\n;; <\/span>Windows ones can be downloaded from: https:\/\/sourceforge.net\/projects\/ezwinports\/files\/<\/span>\n(<\/span>defun<\/span> turn-on-org-show-all-inline-images<\/span> ()<\/span>\n  (<\/span>org-display-inline-images t t)<\/span>)<\/span>\n\n(<\/span>add-hook 'org-mode-hook 'turn-on-org-show-all-inline-images)<\/span>\n<\/pre>\n<\/div>\n
      It is possible that you get the following error message in your mini buffer<\/code> if you type C-c C-x C-v<\/code>:<\/p>\n
      “no images to display inline”<\/p><\/blockquote>\n
      What this probably means is that you are lacking the libraries to display these type of images. What you should do is to run this elisp code to see the expected library files each supported file format and the expected library files names:<\/p>\n
      \n
      (<\/span>print image-library-alist)<\/span>\n<\/pre>\n<\/div>\n
      ((xpm \"libxpm.dll\" \"xpm4.dll\" \"libXpm-nox4.dll\") (png \"libpng16.dll\" \"libpng16-16.dll\") (tiff \"libtiff-5.dll\" \"libtiff3.dll\" \"libtiff.dll\") (jpeg \"libjpeg-9.dll\") (gif \"libgif-7.dll\") (svg \"librsvg-2-2.dll\") (gdk-pixbuf \"libgdk_pixbuf-2.0-0.dll\") (glib \"libglib-2.0-0.dll\") (gobject \"libgobject-2.0-0.dll\") (gnutls \"libgnutls-28.dll\" \"libgnutls-26.dll\") (libxml2 \"libxml2-2.dll\" \"libxml2.dll\") (zlib \"zlib1.dll\" \"libz-1.dll\"))\n<\/pre>\n
      Then for each of the format you want to report, get the library file and for add it in your [...]\/emacs\/bin\/<\/code> folder. On windows, you can find all these DLL<\/code> from the EzWinPorts<\/a> project repository.<\/p>\n<\/div>\n<\/div>\n
      \n
      Language Specific Libraries<\/h3>\n
      \n
      For a few tasks I simply use external libraries to get the job done instead of Emacs\/Org-mode specific plugins or functionality. I will refer to Clojure external libraries, but the same kind of libraries could be used in any other programming languages.<\/p>\n
      For example, if I want to output tabular information in Org-mode, then I normally use the Clojure Table<\/a> application which takes multiple different kind of Clojure data structure and turns them into well-formatted tables in the resultsets. This is really handy for that kind of operation.<\/p>\n
      Otherwise I use Incanter<\/a> a lot to generate effective graphs, charts of plots that I save as PNG and that I display inline in Org-mode. However, if I have a graph or flow chart to create, then I will use the DOT plugin since it is really easy to use not to use it within Org-mode.<\/p>\n
      Basically anything that output some text or some image could be used within Org-mode, but for the kind of software I develop and the kind of data analysis tasks I am doing, these are the two bests in my toolset for the moment.<\/p>\n<\/div>\n<\/div>\n
      \n
      Helpful Keys for Working With Org-mode<\/h3>\n
      \n
      There are a few key bindings in Org-mode that really make your life easier when come the time to do Literate Programming in Org-mode.<\/p>\n
      If you are using Clojure in your Org file, then the first thing to do is to start Cider. I bound cider-jack-in<\/code> to F9<\/code>. Once Cider is started, then you will be able to run Clojure code within your Org file.<\/p>\n
      The most obvious key binding is C-x C-s<\/code> which will save the Org file. At the same time, it will do all the things described in the section Change Behaviors On Save<\/code> described above.<\/p>\n
      Then we have C-c C-c<\/code> that will execute a specific code block and show the results. Note that the cursor needs to be somewhere within the code block (including the header and footer) to execute that block with that key binding.<\/p>\n
      When you open an existing Org file with a lot of code blocks, you often want to run all the code blocks at once. It can easily be done using C-c C-v t<\/code> which will do exactly that.<\/p>\n
      We have to remember that it is not the Clojure major mode that we use directly in Org-mode. However, it is often handy to be able to switch to Clojure’s major mode from a Org file (to get auto completion, etc.). It can easily be done with C-c '<\/code> which will open a new buffer with the code in Clojure’s major mode. Then if you modify that buffer and save it using C-c C-x<\/code> then the Org file will be updated with the changes as well. To switch back to the Org file, then you simply has to hit C-c '<\/code> again.<\/p>\n
      There are a few key bindings quite handy to work with the structure of the document. We often endup writing big Org file with a lot of headers and level of headers. It is quite handy to be able to focus on specific regions in a Org outline. This can easily be done using C-x n s<\/code> which will focus on a particular region (only the content of that region appears in the buffer). Then you can use C-x n w<\/code> to unfocus a focused region (everything surrounding that region will reappear in the buffer).<\/p>\n
      There are tens of other key bindings that you will endup using in Org-mode for doing Literate Programming, but these are the ones I most often use when writing a Org file.<\/p>\n<\/div>\n<\/div>\n
      \n
      Conclusion<\/h3>\n
      \n
      As you can see, there are quite a lot of things that can be configured in Org-mode. This is even just the tip of the iceberg in fact. However, these are the main features I use to do Literate Programming and to create data analysis notebooks. Now that we have Emacs configured<\/a>, and that we have Org-mode configured, my next step will be to write about how I do organize my Clojure applications to write Literate programs.<\/p>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"
      For some time I have been interested in using Emacs and Org-mode for developing Clojure in a Literate Programming way. I discussed the basic ideas, some of the benefits of doing so, etc, etc. It is now time to start showing how I am doing this, what are the rules of thumb I created, what […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[251,279,277],"tags":[252,274,276],"class_list":["post-3234","post","type-post","status-publish","format-standard","hentry","category-clojure","category-emacs","category-literate-programming","tag-clojure-2","tag-literateprogramming","tag-orgmode"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/fgiasson.com\/blog\/index.php\/wp-json\/wp\/v2\/posts\/3234","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/fgiasson.com\/blog\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/fgiasson.com\/blog\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/fgiasson.com\/blog\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/fgiasson.com\/blog\/index.php\/wp-json\/wp\/v2\/comments?post=3234"}],"version-history":[{"count":9,"href":"https:\/\/fgiasson.com\/blog\/index.php\/wp-json\/wp\/v2\/posts\/3234\/revisions"}],"predecessor-version":[{"id":3618,"href":"https:\/\/fgiasson.com\/blog\/index.php\/wp-json\/wp\/v2\/posts\/3234\/revisions\/3618"}],"wp:attachment":[{"href":"https:\/\/fgiasson.com\/blog\/index.php\/wp-json\/wp\/v2\/media?parent=3234"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/fgiasson.com\/blog\/index.php\/wp-json\/wp\/v2\/categories?post=3234"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/fgiasson.com\/blog\/index.php\/wp-json\/wp\/v2\/tags?post=3234"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}