Page 1 of 1

Scope, Globals, Nested functions and Nonlocals

#1 DK3250  Icon User is offline

  • Pythonian
  • member icon

Reputation: 233
  • View blog
  • Posts: 771
  • Joined: 27-December 13

Posted 25 October 2016 - 10:53 AM

Scope, Globals, Nested functions and Nonlocals

This tutorial is inspired by a recent question in the general Python forum.

The first part of this tutorial to some extend overlaps an old tutorial from December 2008, but I believe this new one is more extensive; furthermore, the last part of this tutorial, about nested functions and nonlocals, builds on the first half.

Functions and scope

Books can be written (and has been) about functions in Python. Here I will only briefly cover the subject and only enough to explain 'scope'.

A function in python is defined by the keyword 'def' followed by a name, a set of parenthesis (with optional arguments) and finally a colon. The function block appears indented below the definition line.
def my_function(my_argument):
    #code here

Variables used inside a function can be locally assigned or globally assigned. Look at this
x = 5
y = 6

def add():
    y = 2
    return x + y
print(”res =”,add())
print(”x =”, x, ”y =”, y)

The output is
res = 7
x = 5 y = 6
In the main code y is set to 6, but inside the function another variable also named 'y' is set to 2. This locally assigned variable only exists inside the function; the function is said to have a separate scope. Interestingly, however, when no assignment takes place in the function, the variable will be looked for in the main block. This is why the function knows about the value of x in the example above.
Now, look at this:
x = 5

def add():
    print(x)
    x = 4
add()
This will raise an error: ”local variable 'x' referenced before assignment”. This is because the compiler realizes that an assignment is made within the scope, and then the variable name is local within the full scope – even if it to the naïve human eye seams that print(x) happens before assignment and could be expected to output the global value.

So each function constitutes a scope, a limited amount of 'code volume' where assigned variables are separated from the main. A variable assigned within a scope is local to this scope.

Globals

A variable used (but not assigned) in the scope must be assigned in an outer function or in the main (or be a build-in of Python).
Now, what if you want to change (re-assign) an outside defined variable from within the function. The simplest way is to use the 'global' keyword.
x = 5

def add():
    global x
    print("Inside 'add':", x)
    x += 2
add()
print("In main:", x)
The output is
Inside 'add': 5
In main: 7
Use of the global keyword ensures that the variable inside the scope is the same as outside.
Use of the 'global' keyword is much disputed and generally considered 'bad' programming. There are may reasons for this judgment. Three reasons, that I fully support, are re-use of code, lack of overview and modularization:

Why globals are bad

As you write more and more programs, you will often come to situations where you can re-use a function from an earlier program. This is more than ok, it is downright recommendable. Re-use of debugged and commented code written with carefully chosen names for functions and arguments will shorten the code development and make the programs easier to read (known stuff is always easier to read than new). But if the code was written using globals, then re-use is much more difficult – it is not just 'plug and play' but requires proper handling of the globals, loosing much of the re-use idea.

As the code volume in larger programs grows, it becomes very difficult to maintain an overview of all variables used. Especially if the code needs a revision moths or years after it was written. Globals can be quite tricky in such de-bugging situations.

If a program evolves in such a way that modularization is relevant, then globals cannot be used. So, to avoid blocking for your later self, don't use globals in the first place.

This tutorial is not primarily about functions, but what you can do if re-assignment is relevant, is to bring the variable to the function as argument, and back to main as a return value; this is actually the usual textbook recommendation:
x = 5

def add(x):
    print("Inside 'add':", x)
    x += 2
    return x
x = add(x)
print("In main:", x)


A last comment about 'global's: If 'global's are so bad, why are the keyword at all included in Python? I'm not sure here, but I think it is largely a remnant from the early Python versions or even from the languages preceding Python. Much like the lambda function that now is obsolete due to the modern comprehension style introduced with Python 2.0 (Sticking out my neck here..)

Nested functions and nonlocals

Let's look at a construct that I believe is quite overlooked by many: Nested functions. Look at this code:
def primary():
    def power(x):
        x = x ** 2
        return x

    result = []
    for x in range(5):
        result.append(power(x))

    return result

print(primary())
Here the small function 'power()' is local to the 'primary()' function.

Such a construction is relevant if your primary() function is the interphase to the main, but the primary() function itself is complicated enough (or repetitive enough) to justify a number of local functions. The local functions can be placed in the main, but if they really are only used from within a single primary() function, then they should only live here.
I don't see this code style too often, but I think it should be recommended. The most obvious disadvantage that I see is the use of 8 spaces at the start of each line, making it difficult to keep the line length to the recommended 80 maximum.

Now, what if you need to modify a variable from the primary() in such a sub-function. Well, we just did by use of the good old argument/return method. But sometimes the code grows messy, and with Python 3.x we can use the 'nonlocal' keyword instead
def primary():
    def power():
        nonlocal x
        x = x ** 2

    result = []
    for x in range(5):
        result.append(power())

    return result

print(primary())
'nonlocal' works a bit like 'global' but only within the scope of primary(). And this working makes sense; all functions under the scope of primary() are dedicated to primary(), nobody will ever misunderstand – so, havning a nonlocal variable swimming freely in the little lake of primary() should also not be misunderstood.
This is quite different from having a variable to swim in the huge ocean of 'main' by use of 'global'.
I hope you can accept my aphorism of 'swimming' variables.

As mentioned above, the 'nonlocal' keyword is only available in Python 3.x

The primary benefit from use of 'nonlocal' is cleaner code when modifying variables from lower levels of nested functions; 'nonlocal' doesn’t provide functionality that is not available with traditional coding.

The 'nonlocal' works not only on the first level, but further down:
def first():
    def second():
        def third():
            nonlocal x
            x = x ** 2
        third()
    for x in range(5):
        second()
        print(x)

first()
Admittedly, I have never used such construction in real code, but tried it for this tutorial.

For demonstrating purpose, I have made the small program below. It calculates and list the first 10, 20, 30,.. primes, up to the value specified in the main. It does so without re-calculating the lower primes, it just continues the 'prime' list already existing. It is a rather silly code, but it demonstrates how a nonlocal variable is available to the inner function without declaration and how the modified variable is available in the outer function without a return statement.
"""
Demo program for use of the 'nonlocal' keyword
Program made by DK3250, October 2016
"""

def prime_list(maxx):
    """This outer function generates the nonlocal list 'primes' that by
successive calls to the inner function is gradually made longer.

At each 10th value, the list increment is printed"""

    def find_prime(num):
        """This inner function *continues* the nonlocal list 'primes',
adding more primes until the list length is 'num' """
        nonlocal primes
        candidate = primes[-1]
        while len(primes) < num:
            candidate += 2
            for prim in primes:
                if candidate % prim == 0:
                    break
                if prim * prim > candidate:
                    primes += [candidate]
                    break

    primes = [2, 3]  
    for i in range(10, maxx+1, 10):
        find_prime(i)
        print(primes[-10:])

number = 100  # the number of primes in the final list
prime_list(number)



I hope this small tutorial contributes to explain:
* the concept of scopes
* avoiding 'global's
* use of nested functions
* understanding and use of 'nonlocal'

As always: Feedback is much appreciated.

Is This A Good Question/Topic? 1
  • +

Page 1 of 1