Advanced

Python is a very versatile programming language – there is a great deal you can accomplish with it. Useful as it may be, that versatility has the potential to bring about a bit of confusion with some of its implementations. Decorators are one such implementation of Python that if not explained properly, will probably do more harm than good.

This article aims to explain decorators so that you fully understand them and are confident enough to utilize them in your programs where appropriate.

NOTE: An understanding of functions in Python is required to grasp the following concepts.

What is a Decorator?

A decorator is a feature in python that allows you to add extra capabilities to your function, without having to change the inner workings of your function.

Some examples of uses of decorators include: determining the execution time of a function, determine which URL call should result in this function being called, adding logging messages when a function starts/stops. There are many uses for this.

More technically, a decorator is a function that takes another function as an argument, adds some functionality then returns another function. This happens without altering the original source code of the function that was passed.

A Practical Example of a Decorator

An example code to measure runtime

Rather than go into the technical explanation, let’s work on a practical example of a decorator. Suppose you had the following code and you wanted to determine which function is faster:

#decorator.py
import math, functools

def sqrt_by_loop( count ):
	total = 0;
	for i in range(count):
		total = total + math.sqrt(i)
	print(f"Running function sqrt_by_loop: {total}")
	return total

def sqrt_by_reduce( count ):
	total = functools.reduce( lambda running_sum, next_item: running_sum + math.sqrt(next_item), range(count) )
	print(f"Running function sqrt_by_reduce: {total}")
	return total

sqrt_by_loop(100)
sqrt_by_reduce(100)

The output for the above is as follows:

Method 1: Traditional method to measure runtime

To determine which function is faster, you can change the function code to track the start and end time, and then subtract the two to determine the runtime. So your code could be like this:

#decorator.py
import math, functools
import time

def sqrt_by_loop( count ):
	start = time.time()	# performance time trackingcode
	total = 0;
	for i in range(count):
		total = total + math.sqrt(i)
	print(f"Running function sqrt_by_loop: {total}")
	end = time.time()	# performance time trackingcode
	print(f"Runtime for sqrt_by_loop: {end-start}")

	return total

def sqrt_by_reduce( count ):	
	start = time.time()	# performance time trackingcode
	total = functools.reduce( lambda running_sum, next_item: running_sum + math.sqrt(next_item), range(count) )
	print(f"Running function sqrt_by_reduce: {total}")
	end = time.time()	# performance time trackingcode
	print(f"Runtime for sqrt_by_loop: {end-start}")
	return total

sqrt_by_loop(9145000)

sqrt_by_reduce(9145000) 

Output:

We ran this with a much larger input of 9,145,000 to see which method was fastest to determine the sum of the square root of all sequence of numbers. So interestingly, the reduce function was a little bit slower.

So this works, however, the problem is that you have to edit your original functions.

Method 2: Measuring with a method

Another way to do this would be to use a method in between to do the measuring. There are several ways to do this, so this is one variation:

#decorator.py
import math, functools
import time

def measure_runtime( func, param ):
	start = time.time()		# performance time trackingcode
	func( param )
	end = time.time()		# performance time trackingcode
	print(f"Runtime for {func}: {end-start}")

def sqrt_by_loop( count ):
	total = 0;
	for i in range(count):
		total = total + math.sqrt(i)
	print(f"Running function sqrt_by_loop: {total}")
	return total

def sqrt_by_reduce( count ):	
	total = functools.reduce( lambda running_sum, next_item: running_sum + math.sqrt(next_item), range(count) )
	print(f"Running function sqrt_by_reduce: {total}")
	return total

measure_runtime( sqrt_by_loop, 9145000) 
measure_runtime( sqrt_by_reduce, 9145000) 

Output:

So this is an improvement in that the original functions dont need to be changed, however the problem is that the calling functions have to be changed quite a bit. It’s also quite inflexible in that as the parameters differ, the measure_runtime would have to be different.

The challenge with both Method 1 and Method 2 is also that if you have lots of functions, then this is a lot of extra code you have to include.

This is where decorators can help

Method 3: Using decorators to measure runtime

The above problems can be all resolved with a decorator. We can enhance the original functions, by adding a timer function, without having to edit the original function, and without having to edit the calling code.

Yes, it is possible and one of the great things about python. The code looks like this:

#decorator.py
import math, functools
import time

def measure_runtime(original_func):
	def wrapper_func(*args, **kwargs):
		start = time.time()		 
		return_value = original_func(*args, **kwargs)
		end = time.time()		 
		print(f"Runtime for {original_func}: {end-start}")
	return wrapper_func
	
@measure_runtime
def sqrt_by_loop( count ):
	total = 0;
	for i in range(count):
		total = total + math.sqrt(i)
	print(f"Running function sqrt_by_loop: {total}")
	return total

@measure_runtime
def sqrt_by_reduce( count ):	
	total = functools.reduce( lambda running_sum, next_item: running_sum + math.sqrt(next_item), range(count) )
	print(f"Running function sqrt_by_reduce: {total}")
	return total

sqrt_by_loop( 9145000) 
sqrt_by_reduce( 9145000) 

Output:

As you can see, the only change was to add this @measure_runtime single line above each function, and you achieve the same thing!

Method 3: Using decorators to measure runtime – with averages

The useful thing about decorators, is that you can also do some advanced things such as run functions multiple times to get the averages as well. So with the decorator, we can make the call to the core function number of times and return the average speed since it may vary:

#decorator.py
import math, functools
import time

def measure_runtime(number_of_times=1):
	def decorator_func(original_func):
		def wrapper_func(*args, **kwargs):
			execution_time = 0
			for execution in range(number_of_times):
				start = time.time() 
				return_value = original_func(*args, **kwargs)
				end = time.time() 
				print(f"Runtime for [{execution}] for {original_func}: {end-start}")
				execution_time = execution_time + (end-start)
			execution_time = execution_time / float(number_of_times)
			print(f"***Average runtime for {original_func}: {execution_time} ***\n\n")

		return wrapper_func
	return decorator_func
		
@measure_runtime(3)
def sqrt_by_loop( count ):
	total = 0;
	for i in range(count):
		total = total + math.sqrt(i)
	print(f"Running function sqrt_by_loop: {total}")
	return total

@measure_runtime(3)
def sqrt_by_reduce( count ):	
	total = functools.reduce( lambda running_sum, next_item: running_sum + math.sqrt(next_item), range(count) )
	print(f"Running function sqrt_by_reduce: {total}")
	return total

sqrt_by_loop( 9145000) 
sqrt_by_reduce( 9145000) 

See how the decorator @measure_runtime was called with a parameter of ‘3’ which was to make the call to that function 3 times to get the average runtime.

The Concept Behind Decorators

So the above should explain the power of decorators. So how does it actually work? Well it’s a combination of a few interesting features in python:

• Functions are First-Class Objects: this means you can assign functions to variables, pass them as arguments, and return them as values from other functions.
• Scope / Closure: this specifically refers to how a nested function that has access to variable(s) from its enclosing function.

It’s these two things that make it possible to have decorators. This is a diagram that shows what happens conceptually. The actual call to the original function is circumvented to call the decorator which encapsulates the call to the original call with some additional functionality. That’s where the magic lies

How Function References Work

The above example for the performance measurement showed you how to call and crate a decorator, but let’s spend some time going through that in more detail so you can understand what is happening and decorators are even possible.

Here’s a simple example to a function call:

message = 'Hello world'
def main_func():
  print(message)
   
main_func()   

Output:
Hello World

The output will obviously show: ‘Hello world’. The reason is that the ‘main_func’ can see the variable ‘message’.

That’s easy enough. Now consider this example:

message = 'Hello world'
def main_func():
	print(message)
 
temp  = main_func
print("Temp:" + str(temp))
temp()   

Output:
Temp:<function main_func at 0x7f3d429551f0>
Hello World

What you see above is that the “main_func” is just a variable like any other variable (in fact, it’s an object), and that it has an address for the actual function. You can also assign it to other variables and call it.

This is a more complex example of using addresses inside a function:

def outer_func():
	message = 'Hello world'

	def inner_func():
		print(message)

	print( "1. Inside outer_func - address of inner_func:" + str(main_func) )
	return inner_func

print( "2. In main code - address of outer_func:" + str(outer_func))
my_func = outer_func() 
print( "3. In main code - address of what outer_func returns:" + str(my_func) + "\n\n")
print( "4. Now calling my_func()")
my_func()

Output:
2. In main code - address of outer_func:<function outer_func at 0x7f6d692d71f0>
1. Inside outer_func - address of inner_func:<function outer_func.<locals>.inner_func at 0x7f6d691deee0>
3. In main code - address of what outer_func returns:<function outer_func.<locals>.inner_func at 0x7f6d691deee0>


4. Now calling my_func()
Hello world

What’s happening here is the following:

• The call to outer_func() returns a reference to the inner function called “inner_func()”. This is stored in the variable “my_func”
• Next, when my_func is called through my_func() it will execute the code under “inner_func()”

And here’s a final example, but with data passed as variables:

def outer_func(msg):
    def inner_func():
        print("Inner Function:" + msg)
    return inner_func

hi_func = outer_func('Hi')
bye_func = outer_func('Bye')

hi_func()
bye_func()

Output:
Inner Function:Hi
Inner Function:Bye

As you can see, the messages were also accessible by the inner functions.

The code we have just seen is the basis of how you start to understand decorators.

How to Write a Decorator

Now let us look at an example that illustrates how a decorator works.

# Python - Decorators

# decorator function taking function as argument
def decorator_func(original_func):
    def wrapper_func():
        print( f'A. wrapper executed this before function [{original_func.__name__}]' )
        # return passed function and execute
        return original_func()
    # return wrapper function waiting to be executed
    return wrapper_func

def display_func():
    print('B. [display_func] function ran')

# decorate display function with decorator function 
decorated_display = decorator_func(display_func)

decorated_display()

Output:
A. wrapper executed this before function [display_func]
B. [display_func] function ran

The best way to explain this code is starting at the end:

• We create a variable decorated_display and assign it decorator_func() with the display function passed to it
• We execute decorated_display and that:
  • runs the wrapper_func() which prints out a statement and returns the display function we passed
  • we return wrapper_func which is waiting to be executed
    • its execution happens when we call decorated_display

Normally, you won’t find decorators written as in the code above. Instead, you will something like this which is the shorthand version:

# Python - Decorators

# decorator function taking function as argument
def decorator_func(original_func):
    def wrapper_func():
        print( f'A. wrapper executed this before function [{original_func.__name__}]' )
        # return passed function and execute
        return original_func()
    # return wrapper function waiting to be executed
    return wrapper_func

@decorator_func
def display_func():
    print('B. [display_func] function ran')

display_func()

The output is the same because @decorator_func is the same as decorated_display = decorator_func(display) . But note, that in this version, you just call the original function name “display_func()” – the @decorator_func does the reshuffling automatically. The code becomes much easier to read and maintain!

Adding Arguments to Decorators

The example we have is good for illustration but does not have any use beyond that. If we wanted to pass parameters to our decorated function it would not work and we would run into errors. Let’s how we can correct this with a different example

NOTE: When you want to pass arguments but don’t know how many they are going to be, you can use *args and **kwargs for the parameters of that function.

def decorator_func(original_func):
    def wrapper_func(*args, **kwargs):
        print('wrappper executed this before {}'.format(original_func.__name__))
        return original_func(*args, **kwargs)
    return wrapper_func

@decorator_func
def display_info(name, age):
    print('display_info ran with arguments ({}, {})'.format(name, age))


display_info('James', 23)

Since we want to pass parameters to the function we decorate we would add *args and **kwargs both to our wrapper function and the function returned within it.

We have managed to decorate display_info() and pass parameters to it

Decorators with parameters

In the above example, the decorator was used to change the behaviour of the original function. However, you can also pass parameters so that your decorator can be even more powerful.

Suppose in the previous example, you also wanted to add a custom title text each time the decorated function is called. You can do the following:

def decorator_func(header_message):
	def main_decorator(original_func):
		def wrapper_func(*args, **kwargs):
			print(f"##### {header_message} #####")
			print('A. wrapper executed this before {}'.format(original_func.__name__))
			return original_func(*args, **kwargs)
		return wrapper_func

	return main_decorator

@decorator_func(header_message='My Title')
def display_info(name, age):
	print('B. [display_func] function ran with arguments ({}, {})'.format(name, age))

display_info('James', 23)

Output:
##### My Title #####
A. wrapper executed this before display_info
B. [display_func] function ran with arguments (James, 23)

It does look rather complex due to the multiple nesting, but it’s there to capture the arguments.

• The decorator_func is called with the argument, and it returns the main_dectorator decorate.
• The main_decorator() when called, in fact returns a call to wrapper_func which encapsulates a call to the original function display_info()!

You can use the above template to guide your decorators.

Conclusion

Decorators are a powerful tool to help simplify your code and also make it very extendible. There are many use cases you can use this for – some examples include:

• Measuring runtime of functions to assess performance (as above)
• Automatic logging when a function is called and storing arguments to support debugging
• Linking a website URL to execute a given function for that url (this is the famous @route decorator that you see in Django and Flask apps)
• Processing command line arguments with click
• Creating a plugin architecture for your code (see article here on Plugin articles with decorators)
• And there are many other use cases..

They are a bit intimidating at first, but once you set them up, then you put your decorator code in different package files to help make your code cleaner.

How have you used decorators in the past? Share you comments below!

Get Notified Automatically Of New Articles