Historical Problems with Closures in JavaScript and Python

A closure is an inner function has access to the variables defined in the environment of its outer function.

Closures are necessary features for supporting the functional programming paradigm. They can be found in almost all modern dynamic programming includes including JavaScript, Python and Ruby; however, Python and JavaScript have both made their own mistakes in the initial implementation of the feature.

Scope of Closures

Closures are composed with two environments: the inner and outer. The inner environment is where new, local variables are defined. The outer environment is where variables live that existed during the creation of the closure itself.

Closure Scopes

Closures have access to both environments and thus have access to variables from both the inner and outer scopes. In pseudocode, it may have the following structure.

procedure outer():
    procedure inner():
        # Do stuff
    return inner

The problems identified here are specific to the scope of closures.

This JavaScript Problem

I first encountered this problem while sifting through Douglas Crockford’s book, JavaScript the Good Parts, in the section regarding function invocation. He explains that the this keyword of functions are automatically bound to the global scope, the window object, when the function is not a method.

This is troublesome because closures cannot have access to the outer function’s this keyword without some hacking.

var obj = {};
obj.outer = function() {
	function inner() {
		console.log(this);
	}
	return inner;
}

var fn = outer();
fn();

With the above code, the console unsurprisingly logs the Window object. This strange functionality is due to the ECMAScript 5 specification on function calls.

in the HTML document object model the window property of the global object is the global object itself.

So, it isn’t necessarily incorrect that JavaScript binds the this keyword to window, for calling the function fn() is equivalent to its dependency injection variant, fn.call(window). For closures, however, we should not automatically override the scope of variables with the global scope, especially this of the outer function.

The quick and classical fix to this problem simply involves aliasing the object to a local variable of the outer environment instead.

var obj = {};
obj.outer = function() {
	var that = this;
	function inner() {
		console.log(that);
	}
	return inner;
}

var fn = outer();
fn();

The output now returns the Object as expected.

Python’s Lexical Scope Problem

Closures in Python 2.x could not change nonlocal variables. It was strange that variables in the outer scope could be accessed, but they could not be changed. This was an inherent problem in the lexical scoping rules of Python where names could only be bound to the local scope or global scope.

Since Python 3, this problem was fixed though it’s still noteworthy of discussion. See PEP 3104.

Consider the following example which increments a counter whenever the function is called.

def counter():
    count = 0;
    def inner():
        count += 1
        return count
    return inner

The above example will yield an error when executed:

>>> countr = counter()
>>> countr()
UnboundLocalError: local variable 'count' referenced before assignment

Again, the error occurs due to Python’s lexical scoping. The local variable, count is not initialized in the inner scope. We expect the variable to be initialized in the outer scope instead; however, this is not the case and the variable remains unbound.

Fortunately, Python 3 added the nonlocal keyword which enables closures to change variables in their outer environment.

def counter():
    count = 0;
    def inner():
        nonlocal count
        count += 1
        return count
    return inner

Now, the counter() works as expected:

>>> countr = counter()
>>> countr()
1
>>> countr()
2

Final Thoughts

These problems only serve to highlight the inherent difficulties with language design. Although it’s a craft tempered throughout the years, programming languages are still far from perfect. It is especially difficult with multiparadigm languages such as JavaScript and Python because of they must cater for each programming paradigm with precision.

I have no complaints for Ruby because I’m not as familiar with it as these two languages, though if anyone else has opinion on it, please free free to share.