LISP - Variable Capture and Hygienic macro

Lisp - Variable Capture and Hygienic Macros

Macros are one of Lisp’s most powerful features, allowing you to write code that writes code, effectively extending the language itself. However, this power comes with a subtle trap: variable capture. This occurs when a variable introduced by the macro unintentionally clashes with a variable in the code where the macro is used. A macro susceptible to this is called an unhygienic macro.

This chapter explores what variable capture is, how to identify it, and how to write hygienic macros—robust macros that avoid this problem using standard Lisp tools.

The Problem: An Unhygienic Macro

Let’s demonstrate variable capture with a classic, albeit contrived, example: a macro to swap the values of two variables. The built-in rotatef is the correct tool for this job, but swap serves as an excellent illustration of the problem.

main.lisp

;; An unhygienic macro to swap the values of two places.
(defmacro unhygienic-swap (x y)
  `(let ((temp ,x))   ; This 'temp' is the source of the problem.
     (setf ,x ,y)
     (setf ,y temp)))

;; Let's try to use it with variables named 'a' and 'b'.
(let ((a 1) (b 2))
  (unhygienic-swap a b)
  (format t "a: ~a, b: ~a~%" a b))

;; Now, let's see what happens when one of the arguments is named 'temp'.
(let ((temp 10) (my-var 20))
  (format t "Before swap: temp=~a, my-var=~a~%" temp my-var)
  (unhygienic-swap temp my-var) ; Uh oh! This is where it breaks.
  (format t "After swap: temp=~a, my-var=~a~%" temp my-var))

Output

a: 2, b: 1
Before swap: temp=10, my-var=20
After swap: temp=20, my-var=20

Explanation: What Went Wrong?

The first swap with a and b worked perfectly. The problem arose in the second case. To understand why, we need to look at the code after the macro is expanded. You can do this in your Lisp environment with macroexpand-1.

;; The expansion of (unhygienic-swap temp my-var) looks like this:

(let ((temp temp))  ; <-- The capture happens here!
  (setf temp my-var)
  (setf my-var temp))

• The macro introduced its own local variable named temp.
• The code that called the macro also used a variable named temp.
• In the expanded code, (let ((temp temp))...) creates a new temp that shadows the outer temp. The inner setf calls only modify this new, inner temp.
• The outer temp (with the initial value of 10) is never assigned to my-var. The result is that my-var gets assigned the new value of the inner temp, which was also set from my-var. In effect, my-var is assigned to itself, and the original value of the outer temp is lost.

The Solution: Achieving Hygiene with gensym

To fix this, we must ensure that any temporary variables our macro creates have unique names that can never clash with user code. Common Lisp provides the gensym function for exactly this purpose. gensym generates a new, unique symbol each time it’s called.

main.lisp

;; A hygienic macro using GENSYM to create a unique temporary variable.
(defmacro hygienic-swap (x y)
  (let ((temp-sym (gensym))) ; 1. Create a unique symbol, e.g., #:G1023
    `(let ((,temp-sym ,x))    ; 2. Use that unique symbol in the generated code.
       (setf ,x ,y)
       (setf ,y ,temp-sym))))

(let ((temp 10) (my-var 20))
  (format t "Before swap: temp=~a, my-var=~a~%" temp my-var)
  (hygienic-swap temp my-var) ; Now it works correctly!
  (format t "After swap: temp=~a, my-var=~a~%" temp my-var))

Output

Before swap: temp=10, my-var=20
After swap: temp=20, my-var=10

Explanation: Why It Works

Let’s look at the expansion of the hygienic macro:

;; The expansion of (hygienic-swap temp my-var) will look something like this:
;; Note: #:G1023 is just an example of a symbol GENSYM might create.

(let ((#:G1023 temp))
  (setf temp my-var)
  (setf my-var #:G1023))

• (let ((temp-sym (gensym))) ... ): This part runs when the macro is being defined. It creates a new symbol and stores it in the temp-sym variable.
• ** (let ((,temp-sym ,x)) … ) **: Inside the backquoted template, temp-symis unquoted. This injects the *unique symbol itself* (e.g.,#:G1023`) into the code that the macro generates.
• No Conflict: The generated code now uses a variable #:G1023 which cannot possibly clash with the user’s variable temp. The swap logic now works as intended.

Best Practices and Debugging

• Use Built-ins When Possible: For swapping, the correct, idiomatic, and hygienic way is to use (rotatef place1 place2). Always prefer a built-in function or macro to writing your own.
• Always Use gensym for Macro Temporaries: Any variable introduced inside a macro that is not one of its arguments should be created with gensym to guarantee hygiene.
• Debugging with macroexpand-1: Your most important tool for debugging macros is macroexpand-1. It shows you exactly what code your macro is generating, which almost always reveals the source of the problem.