{"id":3724,"date":"2023-08-30T12:20:34","date_gmt":"2023-08-30T17:20:34","guid":{"rendered":"https:\/\/fgiasson.com\/blog\/?p=3724"},"modified":"2023-09-01T12:12:53","modified_gmt":"2023-09-01T17:12:53","slug":"literate-programming-in-python-using-nbdev","status":"publish","type":"post","link":"https:\/\/fgiasson.com\/blog\/index.php\/2023\/08\/30\/literate-programming-in-python-using-nbdev\/","title":{"rendered":"Literate Programming in Python using NBDev"},"content":{"rendered":"\n<p>Donald Knuth <a href=\"https:\/\/www.youtube.com\/watch?v=bTkXg2LZIMQ\">considered that, of all his work on typography, the idea of literate programming had the greatest impact on him<\/a>. This is a strong and profound statement that seems to be underestimated by history.<\/p>\n<p><a href=\"https:\/\/fgiasson.com\/blog\/index.php\/2023\/08\/28\/what-is-literate-programming-why\/\">Literate programming has grown on me<\/a> in such a way that I now have a hard time developing in a framework that is not literate. I need to be able to organize my ideas, my code, and its documentation the way I want, not in the way the programming language or library designers intend. I need that flexibility flexibility to be as effective as possible in my work; otherwise, I feel that something is missing.<\/p>\n<p>Since 2016, I have been practicing literate programming using <a href=\"https:\/\/orgmode.org\">Org-Mode<\/a> within <a href=\"https:\/\/www.spacemacs.org\">Emacs<\/a>. As of today, I have not yet found another tool as powerful as Org-Mode within Emacs for developing literate applications. It employs a simple plain text format with clean markup, making it easy to commit and suitable for peer review. However, when used in Emacs\/Org-Mode and enhanced with <a href=\"https:\/\/orgmode.org\/worg\/org-contrib\/babel\/intro.html\">Babel<\/a>, developers end up with one of the most robust notebook systems imaginable, capable of facilitating effective literate programming.<\/p>\n<p>However, the challenge lies in the tooling, particularly Emacs. I have been fortunate enough to build teams that worked with Emacs, allowing us to undertake projects in a literate manner. Yet, this was the exception rather than the norm.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Exploration<\/h3>\n\n\n\n<p>I recently invested time in exploring the latest developments in the Literate Programming tooling space. I aimed to find a solution that would bring me closer to the experience of Org-mode + Emacs, but without the friction associated with Emacs for general developers.<\/p>\n<p>In 2016, all my development work was conducted in <a href=\"https:\/\/clojure.org\">Clojure<\/a>. Clojure developers naturally gravitated toward Emacs due to <a href=\"https:\/\/cider.mx\">Cider<\/a>. Nowadays, I work extensively with Python and configuration files. Consequently, I began researching the current state of the literate programming ecosystem. My search began with two keywords: <code>Python<\/code> and <code>VS Code<\/code>.<\/p>\n<p>This research led me to discover a relatively new project (initiated a few years ago) called <a href=\"https:\/\/github.com\/fastai\/nbdev\">nbdev<\/a>, developed by <a href=\"https:\/\/www.fast.ai\/\">fast.ai<\/a> (<a href=\"https:\/\/jeremy.fast.ai\/\">Jeremy Howard<\/a>, <a href=\"https:\/\/hamel.dev\/\">Hamel Husain<\/a>, and a few other contributors).<\/p>\n<p><code>nbdev<\/code> is an incredibly intriguing project. It leverages several existing open-source projects to build a new literate programming framework from the ground up: it employs Jupyter notebooks as the format for writing software (in contrast to a plain text format like Org-Mode). The Quarto tool is used to generate documentation from the codebase. Additionally, <code>nbdev<\/code> provides a range of tools for running tests, creating vanilla GitHub projects with built-in actions for automated deployment, and more. Due to its reliance on Jupyter, this literate workflow is Python-centric and can be developed using a simple browser or VS Code, complemented by the constantly improving <a href=\"https:\/\/marketplace.visualstudio.com\/items?itemName=ms-toolsai.jupyter&amp;ssr=false\">Jupyter extension<\/a>. There&#8217;s even an experimental <a href=\"https:\/\/marketplace.visualstudio.com\/items?itemName=HamelHusain.nbdev\">nbdev extension<\/a> available.<\/p>\n<p>For this blog post, I will convert the <a data-pjax=\"#repo-content-pjax-container\" data-turbo-frame=\"repo-content-turbo-frame\" class=\"color-fg-default\" href=\"https:\/\/github.com\/fgiasson\/en-fr-translation-service\">en-fr-translation-service<\/a> project I <a href=\"https:\/\/fgiasson.com\/blog\/index.php\/2023\/08\/23\/how-to-deploy-hugging-face-models-in-a-docker-container\/\">recently blogged about<\/a> to use <code>nbdev<\/code>. Finally, based on my experience with Org-mode, I will propose some potential improvements to the project.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Creating a Vanilla nbdev (Notebook Dev) Project<\/h3>\n\n\n\n<p>The first step is to create a new vanilla <a href=\"https:\/\/github.com\/fgiasson\/literate-en-fr-translation-service\">literate-en-fr-translation-service<\/a> GitHub repository and follow <code>nbdev<\/code>&#8216;s <a href=\"https:\/\/nbdev.fast.ai\/tutorials\/tutorial.html\">End-to-End Walkthrough<\/a> to create the literate version of the project. After installing <code>jupyterlab<\/code>, <code>nbdev<\/code>, and <code>Quarto<\/code>, I cloned the new repository locally and executed this command in my terminal to initialize the <code>nbdev<\/code> project:<\/p>\n\n\n\n<div class=\"hcb_wrap\"><pre class=\"prism line-numbers lang-bash\" data-lang=\"Bash\"><code>nbdev_new<\/code><\/pre><\/div>\n\n\n\n<p>This command generated several new files in the repository:<\/p>\n<ul>\n<li><code>.github\/workflows<\/code>: two GitHub actions<\/li>\n<li><code>literate_en_fr_translation_service\/<\/code>: New module<\/li>\n<li><code>nbs<\/code>: where all literate notebook files reside<\/li>\n<li><code>settings.ini<\/code>: nbdev&#8217;s core settings file<\/li>\n<li>&#8230;and various other auto-generated files<\/li>\n<\/ul>\n<p>Once the nbdev vanilla project is complete, simply commit and push the changes to the GitHub repository:<\/p>\n\n\n\n<div class=\"hcb_wrap\"><pre class=\"prism line-numbers lang-bash\" data-lang=\"Bash\"><code>git add .\ngit commit -m&#39;Initial commit&#39;\ngit push<\/code><\/pre><\/div>\n\n\n\n<p>After pushing the changes to the repository, the final step is to enable <code>pages<\/code> <a href=\"https:\/\/nbdev.fast.ai\/tutorials\/tutorial.html#enable-github-pages\">in your GitHub repository<\/a>. Then you can verify the proper functioning of your <a href=\"https:\/\/nbdev.fast.ai\/tutorials\/tutorial.html#check-out-your-workflows\">workflows<\/a>.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Development Process<\/h3>\n\n\n\n<p>The literate programming development process is straightforward yet requires a mindset shift. In the following sections, I will focus on nbdev&#8217;s specific process, which is not substantially different from other literate programming frameworks.<\/p>\n<p>The entire application is developed directly within Jupyter notebooks. Each notebook defines both the application&#8217;s code <em>and<\/em> its documentation. When preparing the application, the documentation will be <em>weaved<\/em> from the Jupyter notebook and hosted as a set of GitHub Pages. Subsequently, the code will be <em>tangled<\/em> into source code files within the module&#8217;s folder:<\/p>\n<p><a href=\"https:\/\/fgiasson.com\/blog\/wp-content\/uploads\/2023\/08\/tangle_weave2.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/fgiasson.com\/blog\/wp-content\/uploads\/2023\/08\/tangle_weave2.png\" alt=\"\" width=\"463\" height=\"373\" class=\"size-full wp-image-3746 aligncenter\" srcset=\"https:\/\/fgiasson.com\/blog\/wp-content\/uploads\/2023\/08\/tangle_weave2.png 463w, https:\/\/fgiasson.com\/blog\/wp-content\/uploads\/2023\/08\/tangle_weave2-300x242.png 300w\" sizes=\"auto, (max-width: 463px) 100vw, 463px\" \/><\/a><\/p>\n<p>Documentation is intertwined among code boxes, and each code box has tangling instructions (indicating whether it should be part of the codebase or documentation, etc.). All the nbdev directives are <a href=\"https:\/\/nbdev.fast.ai\/explanations\/directives.html\">accessible here<\/a>.<\/p>\n<p>The first step involves writing the <code>nbs\/index.ipynb<\/code> file, which serves as the project&#8217;s readme. It introduces the project&#8217;s purpose, usage instructions, and more. This file becomes the initial page of your documentation.<\/p>\n<p>Next, start organizing your application into different <em>parts<\/em>. In nbdev, a part is equivalent to a <em>chapter<\/em>, and a <em>chapter<\/em> is numbered. This numbering is a naming convention specific to nbdev. For our simple application, we&#8217;ll create two chapters: <code>nbs\/00_download_models.ipynb<\/code> and <code>nbs\/01_main.ipynb<\/code>. As you can see, the files are prefixed with numbers, acting as &#8220;chapter numbers.&#8221; These numbers help order the generated documentation&#8217;s index and provide clarity regarding the repository&#8217;s file flow.<\/p>\n<p>The final step is to write each of these notebooks, focusing on both documentation (the <em>why<\/em>) and code (the <em>how<\/em>). This will be the focus of the upcoming sections.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Developing en-fr-translation-service as literate-en-fr-translation-service<\/h3>\n\n\n\n<p>The first step I took was to copy over the <code>requirements.txt<\/code> and <code>Dockerfile<\/code> to the root of the repository. Since nbdev currently only supports Python files, only that part of the application will be literate (more about this limitation later). The only change required is adjusting the paths of some files in the <code>Dockerfile<\/code> because nbdev creates a <code>module<\/code> for our application:<\/p>\n\n\n\n<div class=\"hcb_wrap\"><pre class=\"prism line-numbers lang-plain\"><code>COPY literate_en_fr_translation_service\/main.py .\nCOPY literate_en_fr_translation_service\/download_models.py .<\/code><\/pre><\/div>\n\n\n\n<h4 class=\"wp-block-heading\">nbs\/index.ipynb<\/h4>\n\n\n\n<p>The initial step is to create the <code>index.ipynb<\/code> file. This serves as the entry point for the generated documentation and also becomes the <code>README.md<\/code> file of the repository after running the <code>nbdev_readme<\/code> command.<\/p>\n<p><a href=\"https:\/\/github.com\/fgiasson\/literate-en-fr-translation-service\/blob\/main\/nbs\/index.ipynb\">This file<\/a> is a simple Jupyter notebook containing a single Markdown cell where we provide an introduction to the project.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">nbs\/00_download_models.ipynb<\/h4>\n\n\n\n<p>The next step involves creating the <code>00_download_models.ipynb<\/code> file. This file contains all the code and documentation related to downloading the ML models required for the translation service. Since the first task the Docker container performs upon running is downloading the translation model artifacts, I&#8217;ve prefixed the file with <code>00_<\/code> to signify it as the first chapter of the application.<\/p>\n<p>At the top of the file, a Markdown cell should be created for the <code>default_ext<\/code> directive. This directive informs nbdev which module file the code from subsequent <code>export<\/code> and <code>exports<\/code> directives should be woven into:<\/p>\n\n\n\n<div class=\"hcb_wrap\"><pre class=\"prism line-numbers lang-python\" data-lang=\"Python\"><code>#| default_exp download_models<\/code><\/pre><\/div>\n\n\n\n<p>In this case, all code from subsequent Python cells will be placed in the <code>literate_en_fr_translation_service\/download_models.py<\/code> file.<\/p>\n<p>Next, we add the import statements:<\/p>\n\n\n\n<div class=\"hcb_wrap\"><pre class=\"prism line-numbers lang-python\" data-lang=\"Python\"><code>#| exports\nfrom transformers import AutoTokenizer, AutoModelForSeq2SeqLM\nimport os<\/code><\/pre><\/div>\n\n\n\n<p>The difference between <code>export<\/code> and <code>exports<\/code> is that <code>exports<\/code> exports the code to both the code file and the documentation (the code will be displayed in a code box in the documentation). In contrast, <code>export<\/code> only adds the code to the code file and won&#8217;t appear in the documentation. For this case, we want the exports to be displayed in the documentation.<\/p>\n<p>Following this, we define the <code>download_models()<\/code> function:<\/p>\n\n\n\n<div class=\"hcb_wrap\"><pre class=\"prism line-numbers lang-python\" data-lang=\"Python\"><code>#| export\ndef download_model(model_path: str, model_name: str):\n    &quot;&quot;&quot;Download a Hugging Face model and tokenizer to the specified directory&quot;&quot;&quot;\n    # Check if the directory already exists\n    if not os.path.exists(model_path):\n        os.makedirs(model_path)\n\n    tokenizer = AutoTokenizer.from_pretrained(model_name)\n    model = AutoModelForSeq2SeqLM.from_pretrained(model_name)\n\n    # Save the model and tokenizer to the specified directory\n    model.save_pretrained(model_path)\n    tokenizer.save_pretrained(model_path)<\/code><\/pre><\/div>\n\n\n\n<p>In this case, we don&#8217;t intend for the code to appear in the documentation. Here, nbdev will document the function in textual form without directly including the code in the documentation.<\/p>\n<p>Finally, we proceed to download the actual model artifacts:<\/p>\n\n\n\n<div class=\"hcb_wrap\"><pre class=\"prism line-numbers lang-python\" data-lang=\"Python\"><code>#| exports\n#| eval: false\n\ndownload_model(&#39;models\/en_fr\/&#39;, &#39;Helsinki-NLP\/opus-mt-en-fr&#39;)\ndownload_model(&#39;models\/fr_en\/&#39;, &#39;Helsinki-NLP\/opus-mt-fr-en&#39;)<\/code><\/pre><\/div>\n\n\n\n<p>This last code block is an interesting one that shows the flexibility of the code block directives, and their importance in the development flow.&nbsp;<\/p>\n<p>First, we do export the code to the codebase, and we show the two line of code in the documentation to help the user to understand how it works. But then we added an&nbsp;<code>eval: false<\/code> directive. Why? This is used to tell nbdev to&nbsp;<em>not<\/em> evaluate this code block when it tangles and weave the notebook file. Otherwise, this code would be executed, and the models artifacts would be downloaded which would add a lot of processing time and spend unnecessary bandwidth on the network. However, we want this code to appear in the codebase since the container will run that file to initialize the service with all the right models artifacts.<\/p>\n<p>The result is a <a href=\"https:\/\/github.com\/fgiasson\/literate-en-fr-translation-service\/blob\/main\/nbs\/00_download_models.ipynb\">very simple and clean notebook<\/a> that is easy to understand:<\/p>\n<p><a href=\"https:\/\/fgiasson.com\/blog\/wp-content\/uploads\/2023\/08\/download_models.jpg\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/fgiasson.com\/blog\/wp-content\/uploads\/2023\/08\/download_models.jpg\" alt=\"\" width=\"2050\" height=\"1436\" class=\"alignnone size-full wp-image-3749\" srcset=\"https:\/\/fgiasson.com\/blog\/wp-content\/uploads\/2023\/08\/download_models.jpg 2050w, https:\/\/fgiasson.com\/blog\/wp-content\/uploads\/2023\/08\/download_models-300x210.jpg 300w, https:\/\/fgiasson.com\/blog\/wp-content\/uploads\/2023\/08\/download_models-1024x717.jpg 1024w, https:\/\/fgiasson.com\/blog\/wp-content\/uploads\/2023\/08\/download_models-768x538.jpg 768w, https:\/\/fgiasson.com\/blog\/wp-content\/uploads\/2023\/08\/download_models-1536x1076.jpg 1536w, https:\/\/fgiasson.com\/blog\/wp-content\/uploads\/2023\/08\/download_models-2048x1435.jpg 2048w\" sizes=\"auto, (max-width: 2050px) 100vw, 2050px\" \/><\/a><\/p>\n\n\n\n<h4 class=\"wp-block-heading\">nbs\/01_main.ipynb<\/h4>\n\n\n\n<p>The subsequent chapter is the core file of the translation service. It&#8217;s where the web service endpoints are defined, model file selection occurs, and the service&#8217;s entry point is specified.&nbsp;<\/p>\n<p><a href=\"https:\/\/github.com\/fgiasson\/literate-en-fr-translation-service\/blob\/main\/nbs\/01_main.ipynb\">You can access the notebook here to see the result<\/a>. I won&#8217;t elaborate on each section since the directives used are the same as in the previous chapter.<\/p>\n<p>However, one difference lies in the addition of tests after the endpoint creation:<\/p>\n\n\n\n<div class=\"hcb_wrap\"><pre class=\"prism line-numbers lang-python\" data-lang=\"Python\"><code>assert is_translation_supported(&#39;en&#39;, &#39;fr&#39;)\nassert is_translation_supported(&#39;fr&#39;, &#39;en&#39;)\nassert not is_translation_supported(&#39;en&#39;, &#39;es&#39;)\nassert not is_translation_supported(&#39;es&#39;, &#39;en&#39;)<\/code><\/pre><\/div>\n\n\n\n<p>Those assertions are defined in their own code block. <a href=\"https:\/\/fgiasson.com\/blog\/index.php\/2016\/05\/30\/creating-and-running-unit-tests-directly-in-source-files-with-org-mode\/\">This demonstrates a crucial aspect of literate programming that I wrote about in 2016<\/a>. This kind of workflow enables developers to:<\/p>\n<ol class=\"org-ol\">\n<li>Create a series of unit tests directly where it matters (right below the function to test).<\/li>\n<li>Run the tests when it matters (continuously while developing or improving the tested function).<\/li>\n<\/ol>\n<p>The developer can run that code cell within the Jupyter notebook to ensure that what they just wrote is functioning as expected. They can also execute the <code>nbdev_test<\/code> command-line application to run all the tests of an nbdev application. Finally, it will also be picked up by the <code>tests GitHub workflow<\/code>. This aspect of the development process is extremely important and powerful.<\/p>\n<p>Everything is contextualized in the same place; there&#8217;s no need to look at 2 or 3 different places. This makes PR reviews much more effective for the reviewer: the documentation, the code, and its tests will all appear more or less on the same screen. If any of those elements are missing, the reviewer can easily address it in a comment.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Wrap-up<\/h3>\n\n\n\n<p>So, what does it look like in the end? Here are the references to each component of the literate application:<\/p>\n<ul>\n<li><a href=\"https:\/\/github.com\/fgiasson\/literate-en-fr-translation-service\">GitHub repository of the literate application<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/fgiasson\/literate-en-fr-translation-service\/tree\/main\/nbs\">Literate notebooks where all the application code, documentation and tests are defined<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/fgiasson\/literate-en-fr-translation-service\/tree\/main\/literate_en_fr_translation_service\">Tangled source code from the notebooks<\/a><\/li>\n<li><a href=\"https:\/\/fgiasson.github.io\/literate-en-fr-translation-service\/\">Weaved documentation as GitHub pages<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/fgiasson\/literate-en-fr-translation-service\/actions\">Created CI\/CD workflows<\/a><\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Possible nbdev Improvements<\/h3>\n\n\n\n<p>The fast.ai team has done excellent work with nbdev. I can clearly sense the same literate process that I experienced using Org-mode+Emacs, but with a completely different toolbox, which is refreshing to experience!<\/p>\n<p>Here is a series of potential improvements I considered while testing nbdev. These could eventually become proposed PRs for the project when I find the time to work on them.<\/p>\n<h4>Save Jupyter notebook as <code>Markdown<\/code> or <code>py:percent<\/code>\u00a0instead of JSON<\/h4>\n<p>Since I used Org-Mode, I believe that all notebook formats should be plain text with some markup. One issue I have with Jupyter is its default serialization format, a very complex and large JSON file.<\/p>\n<p>While not a problem itself, it becomes one when reviewing notebook PRs. Therefore, whenever I had developers working with Jupyter notebooks, I always asked them to export their notebooks to <code>Markdown<\/code> or <code>py:percent<\/code> formats before committing to GitHub. This way, the notebook can be easily diffed on GitHub, and inline comments from PR reviewers can be added. Without this, you&#8217;d need to use a service like <a href=\"https:\/\/www.reviewnb.com\/\">ReviewNB<\/a>, which adds unnecessary complexity in my opinion.<\/p>\n<p>I suggest that nbdev could leverage Jupyter&#8217;s internal Markdown export functionality to export each chapter into its own Markdown or py:percent file, which would then be part of the literate GitHub repository.<\/p>\n<p>Another possibility without touching anything to the nbdev workflow could be using <a href=\"https:\/\/github.com\/mwouts\/jupytext\">jupytext<\/a> to manage the synchronization.<\/p>\n<h4>Add <code>.ipynb<\/code> Files to <code>.gitignore<\/code><\/h4>\n<p>Assuming nbdev exports all notebooks as Markdown or py:percent files, I would consider adding <code>.ipynb<\/code> files to the repository&#8217;s <code>.gitignore<\/code>. This simplifies the repository&#8217;s content (containing only plain text files) and avoids duplicates. This is possible since Markdown files can be used to recreate the original JSON Jupyter files.<\/p>\n<h4>Ignore All Files Generated by a Notebook During Export<\/h4>\n<p>If all notebooks are in Markdown format, there&#8217;s no need to commit all the exported content to the repository either.<\/p>\n<p>Since everything is in these notebook files, any developer can generate all the artifacts by:<\/p>\n<ol>\n<li>cloning the repository<\/li>\n<li>exporting the notebook files<\/li>\n<\/ol>\n<p>This would generate all the necessary files for the application&#8217;s functionality. The advantage is a streamlined repository with a collection of literate notebooks.<\/p>\n<h4>Support Beyond Python<\/h4>\n<p>This is where Org-Mode+Emacs shines. In a single notebook, I could incorporate code from various languages and formats, such as <code>Clojure<\/code>, <code>bash curl<\/code> commands, <code>JSON<\/code> outputs, <code>Dockerfile<\/code>, etc. This flexibility was possible due to <a href=\"https:\/\/orgmode.org\/worg\/org-contrib\/babel\/intro.html\">Babel<\/a>.<\/p>\n<p>It might be possible to achieve this in Jupyter (consider <a href=\"https:\/\/github.com\/n-riesco\/jp-babel\">jp-babel<\/a>), or even in VS Code&#8217;s Jupyter extension. Nevertheless, nbdev would need updates to enable this.<\/p>\n<p>Currently, nbdev assumes everything is Python. This is why the directives like <code>#| export foo<\/code> create a file <code>foo.py<\/code> in the module&#8217;s folder.<\/p>\n<p>My proposal is for the <code>export<\/code> and <code>exports<\/code> directives to accept a path\/file as a value, rather than a string used to create the target path and file. This would make the directive more verbose, yet considerably more flexible.<\/p>\n<p>If it worked this way, I could have all my Python code interwoven into one or multiple places in the repository. Additionally, in the same notebook file, I could have multiple code blocks for creating my <code>Dockerfile<\/code>, which would then export to <code>\/Dockerfile<\/code> in the repository. I would treat the <code>Dockerfile<\/code> like any other code source in my project.<\/p>\n<p>This aspect is crucial to me, particularly for Machine Learning projects, as they often involve diverse configuration files (<code>Docker<\/code>, <code>Terraform<\/code>, etc.) that should be managed in a literate framework, similar to traditional source code files.<\/p>\n<p>This aspect is more important than having a Babel in Jupyter (and we are lucky since it is way simpler to implement!)<\/p>\n<h4>New export-test directive<\/h4>\n<p>Having tests in the notebooks, along side the code it tests is very valuable. However, I would think they should be tangled as well, just like any other piece of the code base. We could think about different design, two that come in mind are:<\/p>\n<ol>\n<li>If\u00a0<code>export<\/code> and\u00a0<code>exports<\/code> end-up supporting a path\/file argument, then we would use that new behaviour to specify where the tests goes (i.e.\u00a0<code>\/tests\/test_foo.py<\/code>)<\/li>\n<li>A new directive like\u00a0<code>export-test<\/code> could be created where the test would be created in the <code>\/tests\/<\/code> folder like:\u00a0<code>\/tests\/test_<em>[default_ext]<\/em>.py<\/code><\/li>\n<\/ol>\n<p>I think I prefer (1) since it is more flexible and could be used for other scenarios, like the ones mentioned above.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">References<\/h3>\n\n\n\n<p>Lastly, I&#8217;ve compiled a list of excellent references about nbdev for anyone interested in trying it out:<\/p>\n<ul>\n<li><span><a href=\"https:\/\/www.youtube.com\/watch?v=l7zS8Ld4_iA\">Great introduction tutorial video with Jeremy and Hamel<\/a><\/span><\/li>\n<li><span><a href=\"https:\/\/github.com\/fastai\/nbdev\">nbdev repository (GitHub)<\/a><\/span><\/li>\n<li><span><a href=\"https:\/\/forums.fast.ai\/c\/nbdev\/48\">nbdev forums (community)<\/a><\/span><\/li>\n<li><span><a href=\"https:\/\/nbdev.fast.ai\/getting_started.html\">nbdev Documentation<\/a><\/span><\/li>\n<li><span><a href=\"https:\/\/github.com\/fastai\/nbdev-vscode\">nbdev-vscode (GitHub)<\/a><\/span><\/li>\n<li><span><a href=\"https:\/\/quarto.org\/\">Quarto<\/a><\/span><\/li>\n<li><span><a href=\"https:\/\/fastcore.fast.ai\">Fastcore<\/a><\/span><\/li>\n<li><span><a href=\"https:\/\/docs.fast.ai\/dev\/style.html\">Fast.ai Coding Style<\/a><\/span><\/li>\n<li><span><a href=\"https:\/\/www.reviewnb.com\">ReviewNB (Rich diff of Jupyter notebooks, integrated with Github)<\/a><\/span><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Donald Knuth considered that, of all his work on typography, the idea of literate programming had the greatest impact on him. This is a strong and profound statement that seems to be underestimated by history. Literate programming has grown on me in such a way that I now have a hard time developing in a [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[277,309],"tags":[274,312,314,313],"class_list":["post-3724","post","type-post","status-publish","format-standard","hentry","category-literate-programming","category-mlops","tag-literateprogramming","tag-mlops","tag-nbdev","tag-python"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/fgiasson.com\/blog\/index.php\/wp-json\/wp\/v2\/posts\/3724","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/fgiasson.com\/blog\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/fgiasson.com\/blog\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/fgiasson.com\/blog\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/fgiasson.com\/blog\/index.php\/wp-json\/wp\/v2\/comments?post=3724"}],"version-history":[{"count":30,"href":"https:\/\/fgiasson.com\/blog\/index.php\/wp-json\/wp\/v2\/posts\/3724\/revisions"}],"predecessor-version":[{"id":3764,"href":"https:\/\/fgiasson.com\/blog\/index.php\/wp-json\/wp\/v2\/posts\/3724\/revisions\/3764"}],"wp:attachment":[{"href":"https:\/\/fgiasson.com\/blog\/index.php\/wp-json\/wp\/v2\/media?parent=3724"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/fgiasson.com\/blog\/index.php\/wp-json\/wp\/v2\/categories?post=3724"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/fgiasson.com\/blog\/index.php\/wp-json\/wp\/v2\/tags?post=3724"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}