Faster Feedback with IPython autoreload

Many of the advancements in software development over the decades have been about speeding up feedback and tightening feedback loops:

• Switching from batch computing to interactive computing meant that you could do an action and promptly see the result, instead of waiting hours or overnight.
• REPLs let you interactively experiment with language features and functions without the overhead of writing a whole program.
• Syntax highlighting’s primary benefit is better communication of the code’s structure, but a secondary advantage is highlighting syntax errors as you write code, instead of making you wait until you try to compile or run the program.
• Newer IDEs and text editors can perform semantic checks, not just syntactical checks, showing red squigglies for undefined symbols, mismatched arguments, and so on.
• Unit tests check for bugs whenever you run them, instead of making you wait for QA.
• Continuous integration performs merging, building, and testing on every commit.

And, of course, various agile, lean, and iterative development methodologies are all about shorter, tighter feedback loops.

Taken to the extreme, this suggests the ideal is that, the instant you write a piece of code, you get immediate feedback on whether it will work as you intended. That’s obviously not possible, but current tools can get us surprisingly close.

For example, suppose you have some Python code that isn’t doing what you want. There are a few options for fixing it:

1. The manual approach: Tinker around to set up an environment for testing the entire program. Run it. Observe what breaks. Edit the code. Rerun it. See if things are better. Repeat as needed.
2. The automated testing approach: Write some unit tests to validate the code’s behavior. Run the tests and see if any fail. Edit the code. Rerun the tests. Repeat as needed.
3. The exploratory approach: Load the code in a REPL. Play around with it and observe what breaks. Edit the code. Reload it in the REPL and rerun it. Repeat as needed.

How does “faster feedback” apply to each of these?

The manual approach (#1) is right out. It doesn’t provide fast, automated feedback at all.

Automated testing (#2) fares much better. pytest-watch can automatically execute your test suite whenever a file changes, so “Edit and save the code, rerun the tests from the command line, and see if any fail” becomes “Edit and save the code.” (And pytest is a drop-in replacement for unittest and nose, so it’s easy to get started.)

For now, though, let’s focus on the exploratory approach (#3). To use it:

1. Install IPython, if you haven’t already. Python has a built-in REPL with basic functionality, but IPython is excellent.
2. Make sure your Python code uses the if __name__ == "__main__": trick, if it doesn’t already, so that you can import your code without it automatically running.
3. cd to your Python program’s directory and run ipython.

4. Enable IPython’s autoreload feature. (See here or here.)

   %load_ext autoreload
   %autoreload 2
   

5. import your code. For example, if you have my_script.py, then execute import my_script within IPython.
6. Experiment with your code: set up variables, create class instances, run methods from my_script, etc.
7. Because autoreloading is enabled, whenever you can make changes to your code in a separate window or pane, they’ll immediately take effect for the next commands you run within your IPython session.

You can also modify your code to drop into IPython from anywhere in the code, instead of launching IPython and setting up execution that way.

For the fastest feedback, combine all of this with a good text editor or IDE. For example, if you use Vim, then installing flake8 plus either ALE (requires Vim 8) or Syntastic will detect many common problems within your editor.

What fast feedback techniques have you discovered for Python? Let me know on Twitter.