The next goal I had is to try to make Org-babel-clojure asynchrone. What I wanted is to be able to get, somehow, was the output of a Clojure procedure when that procedure was outputing something to the REPL. My second journey started after reading John Kitchin’s blog post about Asynchronously running Python code into Org-mode code blocks. What I found out is that Python code was run via a sub-process which run the Python interpreter. John’s solution was to use a local file to write what the interpreter is outputing and then to feed that output to a new window that got created by John’s function.
I took that example as a given, and then I tried to implement the same solution, but for Clojure (without knowing what I was really doing). It is in this process that I found that the Clojure solution to that problem would be quite different than John’s. There is an asynchronous API in nREPL, it is just that it is not used in Org-babel-clojure. What I ended-up using from John’s example is not his code, but his core idea: using a new window to output the asynchrone process and then to kill it once the processing is finalized and before populating #+RESULSTS section of the Org-mode file.
After much testing and debugging I ended-up with the following solution to my problem:
(defun org-babel-execute:clojure (body params)
"Execute a block of Clojure code with Babel."
(lexical-let* ((expanded (org-babel-expand-body:clojure body params))
; name of the buffer that will receive the asyn output
(sbuffer "*Clojure Sub Buffer*")
; determine if the :async option is specified for this block
(async (if (assoc :async params) t nil))
; generate the full response from the REPL
(response (cons 'dict nil))
; keep track of the status of the output in async mode
status
; result to return to Babel
result)
(case org-babel-clojure-backend
(cider
(require 'cider)
(let ((result-params (cdr (assoc :result-params params))))
; Check if the user want to run code asynchronously
(when async
; Create a new window with the async output buffer
(switch-to-buffer-other-window sbuffer)
; Run the Clojure code asynchronously in nREPL
(nrepl-request:eval
expanded
(lambda (resp)
(when (member "out" resp)
; Print the output of the nREPL in the asyn output buffer
(princ (nrepl-dict-get resp "out") (get-buffer sbuffer)))
(nrepl--merge response resp)
; Update the status of the nREPL output session
(setq status (nrepl-dict-get response "status")))
(cider-current-connection)
(cider-current-session))
; Wait until the nREPL code finished to be processed
(while (not (member "done" status))
(nrepl-dict-put response "status" (remove "need-input" status))
(accept-process-output nil 0.01)
(redisplay))
; Delete the async buffer & window when the processing is finalized
(let ((wins (get-buffer-window-list sbuffer nil t)))
(dolist (win wins)
(delete-window win))
(kill-buffer sbuffer))
; Put the output or the value in the result section of the code block
(setq result (nrepl-dict-get response
(if (or (member "output" result-params)
(member "pp" result-params))
"out"
"value"))))
; Check if user want to run code synchronously
(when (not async)
(setq result
(nrepl-dict-get
(let ((nrepl-sync-request-timeout
org-babel-clojure-nrepl-timeout))
(nrepl-sync-request:eval
expanded (cider-current-connection) (cider-current-session)))
(if (or (member "output" result-params)
(member "pp" result-params))
"out"
"value"))))))
(slime
(require 'slime)
(with-temp-buffer
(insert expanded)
(setq result
(slime-eval
`(swank:eval-and-grab-output
,(buffer-substring-no-properties (point-min) (point-max)))
(cdr (assoc :package params)))))))
(org-babel-result-cond (cdr (assoc :result-params params))
result
(condition-case nil (org-babel-script-escape result)
(error result)))))
The first thing this code does, is to expose a new #+BEGIN_SRC option called :async. If the new :async option is specified in a block code for the Clojure language, then that code block will be processed asynchronously. What this means is that a new window will be created in Emacs, it will be populated with anything that is outputted to the REPL and then it will be closed once the processing will be finalized.
Here is an example of a code block that would use that new option:
#+BEGIN_SRC clojure :results output :async
(dotimes [n 10]
(println n ".")
(Thread/sleep 500))
#+END_SRC
This code would output “1. 2.” etc into a new window and would close that window when it reaches 10 and then populate the #+RESULTS section with the output of the code.
This code works with the :results options output, value and silent. If output is specified, then everything that was outputted into the window will be added into the results section of the code block. If value is specified, then all output will still be displayed into the window, but only the resulting value will be added to the results section of the code block. If silent is specified, then all the output will still be displayed into the window, but nothing will be displayed in the results section of the code block.
If the :async is omitted, then the normal behavior of Org-babel-clojure will be used, with the new timeout setting org-babel-clojure-nrepl-timeout.