124. Design Patterns

Design patterns are ways to structure code, particularity when using object-based programming. One or more design patterns may be used in any application.

Based on Mastering Design Patterns by Ayeva and Kaspamaplis

See also https://en.wikipedia.org/wiki/Software_design_pattern

Creational Patterns

  1. Factory centralises creation of object instances into a function(s), so all object instance creations can easily be found. Decouples generation of object from accessing objection.

  2. Abstract factory extends factory method (which centralises object instance creation). It contains a collection of object instance factories which may be chosen from (e.g. forms styled for Windows or Mac depending on OS).

  3. Builder constructs a complex object from component objects (often where order of construction is important) – e.g. build web page from component parts (objects). The Director controls the construction. Returns a single final object (built from component objects).

  4. Prototype creates objects by cloning an existing object. Can also be useful for create an archive copy of an object at a given point in time. Python has built in clone method for this.

  5. Singleton is a class which allows only one instance of the class (e.g. for storing the global state of a program, or for a coordinating object, or controlling accesses to a shared resource).

Structural Design Patterns

  1. Adapter is a class of objects for creating an interface between two otherwise incompatible objects (e.g. interfacing between old and new software without needing to change either the old or new object architecture, or when using software where the source code is hidden). It is based on a dictionary that matches new_object.method() to old_object_method().

  2. Decorator extends the function of a class, object or method. Especially useful when many different functions require the same extension, e.g. timing function, validating inputs, monitoring/logging, adding GUI components.

  3. Bridge provides an interface between general code and specific use cases, e.g. processing data with a bridge that allows multiple sources of data, device drivers allowing generic output targeted to specific devices, or using alternative themed GUI implementations.

  4. Facade provides a simplified application interface to the client, hiding the underlying complex application. Client code may then call just one class/method. Alternatively, in a complex system, parts of the system may communicate with each other through a limited number of facades. Facades can also help to keep client cod access independent of underlying code.

  5. Flyweight minimises memory usage by sharing resources as much as possible. A flyweight object contains state-independent immutable data (intrinsic data) that is used by other objects. Flyweight objects should not contain state-dependent mutable data (extrinsic data). Look for what data is common across objects and extract that to a flyweight object.

  6. Model-View-Controller (MVC) uses the principle of Separation of Concerns (SoC) where an application is divided into distinct sections. MVC describes application of SoC to OOP. 1) Model: the core of the program (data, logic, rules, state). 2) View: visual representation of the model, such as a GUI, keyboard or mouse input, text output, chart output, etc. 3) Controller: the link between the model and the view. MVC allows separation of view and model via different controllers, allowing for different views in different circumstances. MVC may also make use of different adaptors.

  7. Proxy uses a surrogate object to access an actual object. Examples are 1) remote proxy which represents an object that is held elsewhere, 2) virtual proxy which uses lazy implementation to delay creation of an object until and if actually required, 3) protection proxy controls access to a sensitive object, and 4) smart (reference) proxy which performs extra actions as an object is accessed (e.g. counting number of times object accessed, or thread-safety checking).

Behavioural Design Patterns

  1. Chain of Responsibility is used when it is not known which object will respond to a request. The request is sent to multiple objects (e.g. nodes on a network). AN object will then decide whether to accept the request, forward the request, or reject the request. This chain continues until all required actions are completed.The Chain of Responsibility creates a flow of requests to achieve a task. The flow of requests is worked out at the time, rather than the route being pre-defined. The client only needs to know how to communicate with the head of the chain.

  2. Command encapsulates all required actions in a command, often including ability to undo. GUI buttons may also use command to execute optoin or display tool tips. Macros may be an example of command: a sequence of steps to perform.

  3. Observer (or Publish/Subscribe) defines a one-to-many dependency between objects where a state change in one object results in all its dependents being notified and updated automatically.

  4. State allows an object to alter its behavior when its internal state changes. The object will appear to change its behaviour. For example in a game amonster might go from sleep to wake to attack. Python has the module state_machine to assist in state machine programming.

  5. Interpreter is a simple langauge to allow advanced users of a system more control in the system.

  6. Strategy defines a family of algorithms, encapsulates each one, and makes them interchangeable. Strategy lets the algorithm vary independently from clients that use it. For example Python may select a specific sort method depending on the data.

  7. Memento supports history and undo, and enables restore to a previous state.

  8. Iterator Provide a way to access the elements of an aggregate object sequentially without exposing its underlying representation.

  9. Template Define the skeleton of an algorithm in an operation, deferring some steps to subclasses. Template method lets subclasses redefine certain steps of an algorithm without changing the algorithm’s structure.

Microservices and patterns for cloud

  1. Microservices breaks applications down into smaller applications, each of which may be developed and deployed separately. Frequently each microservice will run in a container that also has all the required dependencies.

  2. Retry is common in micrososervices, and allows for temporary unavailability of a dependent services. The process may be retired, either immediately or after waiting a few seconds.

  3. Circuit Breaker monitors a service and shuts down a service if reliability is too low.

  4. Cache-Aside holds commonly used data in memory rather than re-reading from disc.

  5. Throttling limits the number of requests a client can send.

121. More list comprehension examples

Example 1 – double the numbers

Standard loop approach:

foo = [1, 2, 3, 4]
bar = []

for x in foo:
    bar.append(x * 2)
    
print(bar)

Out:

[2, 4, 6, 8]

Using list comprehension:

foo = [1, 2, 3, 4]
bar = [x * 2 for x in foo]
print(bar)

Out:

[2, 4, 6, 8]

Example 2 – convert Celsius to Fahrenheit

This example calls a function from within the list comprehension.

Define the function:

def convert_celsius_to_fahrenheit(deg_celsius):
    """
    Convert degress celsius to fahrenheit
    Returns float value - temp in fahrenheit
    Keyword arguments:
        def_celcius -- temp in degrees celsius
    """
    return (9/5) * deg_celsius + 32

Standard loop approach:

#list of temps in degree celsius to convert to fahrenheit
celsius = [39.2, 36.5, 37.3, 41.0]

#standard for loop approach
fahrenheit = []
for x in celsius:
    fahrenheit.append(convert_celsius_to_fahrenheit(x))

print('Using standard for loop: {}'.format(fahrenheit))

Out:

Using standard for loop: [102.56, 97.7, 99.14, 105.8]

Using list comprehension

fahrenheit = [convert_celsius_to_fahrenheit(x) for x in celsius]
print('Using list comprehension: {}'.format(fahrenheit))

Out:

Using list comprehension: [102.56, 97.7, 99.14, 105.8]

Example 3 – convert the strings to different data types

This example also make use of the zip function. Zip allows you to iterate through two lists at the same time.

inputs = ["1", "3.142", "True", "spam"]
converters = [int, float, bool, str]

values_with_correct_data_types = [t(s) for (s, t) in zip(inputs, converters)]
print(values_with_correct_data_types)

Out:

[1, 3.142, True, 'spam']

Example 4 – Using if statements within a list comprehension

The example filters a list of file names to the python files only

unfiltered_files = ['test.py', 'names.csv', 'fun_module.py', 'prog.config']

# Standard loop form
python_files = []
# filter the files using a standard for loop 
for file in unfiltered_files:
    if file[-2:] == 'py':
        python_files.append(file)
        
print('using standard for loop: {}'.format(python_files))

#list comprehension
python_files = [file for file in unfiltered_files if file[-2:] == 'py']

print('using list comprehension {}'.format(python_files))

Out:

using standard for loop: ['test.py', 'fun_module.py']
using list comprehension ['test.py', 'fun_module.py']

Example 5 – List comprehension to create a list of lists

list_of_lists = []

# Standard loop form
for i in range(5):
    sub_list = []
    for j in range(3):
        sub_list.append(i * j)
    list_of_lists.append(sub_list)

print(list_of_lists)


# List comprehension
list_of_lists = [[i * j for j in range(3)] for i in range(5)]

print(list_of_lists)

Out:

[[0, 0, 0], [0, 1, 2], [0, 2, 4], [0, 3, 6], [0, 4, 8]]
[[0, 0, 0], [0, 1, 2], [0, 2, 4], [0, 3, 6], [0, 4, 8]]

Example 6: Iterate over all items in a list of lists

The code converts a list of lists to a list of items
We call this flattening the list.

list_of_lists = [[8, 2, 1], [9, 1, 2], [4, 5, 100]]

# Standard loop form
flat_list = []
for row in list_of_lists:
    for col in row:
        flat_list.append(col)

print(flat_list)

# List comprehension:
flat_list = [item for sublist in list_of_lists for item in sublist]
print(flat_list)

Out:

[8, 2, 1, 9, 1, 2, 4, 5, 100]
[8, 2, 1, 9, 1, 2, 4, 5, 100]

118: Python basics – saving python objects to disk with pickle

Sometimes we may wish to save Python objects to disc (for example if we have performed a lot of processing to get to a certain point). We can use Python’s pickle method to save and reload any Python object. Here we will save and reload a NumPy array, and then save and reload a collection of different objects.

Saving a single python object

Here we will use pickle to save a single object, a NumPy array.

import pickle 
import numpy as np

# Create array of random numbers:
my_array= np.random.rand(2,4)
print (my_array)

Out:

[[0.6383297  0.45250192 0.09882854 0.84896196]
 [0.97006917 0.29206495 0.92500062 0.52965801]]
# Save using pickle
filename = 'pickled_array.p'
with open(filename, 'wb') as filehandler:
    pickle.dump(my_array, filehandler)

Reload and print pickled array:

filename = 'pickled_array.p'
with open(filename, 'rb') as filehandler: 
    reloaded_array = pickle.load(filehandler)

print ('Reloaded array:')
print (reloaded_array)

Out:

Reloaded array:
[[0.6383297  0.45250192 0.09882854 0.84896196]
 [0.97006917 0.29206495 0.92500062 0.52965801]]

Using a tuple to save multiple objects

We can use pickle to save a collection of objects grouped together as a list, a dictionary, or a tuple. Here we will save a collection of objects as a tuple.

# Create an array, a list, and a dictionary
my_array = np.random.rand(2,4)
my_list =['A', 'B', 'C']
my_dictionary = {'name': 'Bob', 'Age': 42}
# Save all items in a tuple
items_to_save = (my_array, my_list, my_dictionary)
filename = 'pickled_tuple_of_objects.p'
with open(filename, 'wb') as filehandler:
    pickle.dump(items_to_save, filehandler)

Reload pickled tuple, unpack the objects, and print them.

filename = 'pickled_tuple_of_objects.p'
with open(filename, 'rb') as filehandler:
    reloaded_tuple = pickle.load(filehandler)


reloaded_array = reloaded_tuple[0]
reloaded_list = reloaded_tuple[1]
reloaded_dict = reloaded_tuple[2]

print ('Reloaded array:')
print (reloaded_array)
print ('\nReloaded list:')
print (reloaded_list)
print ('\n Reloaded dictionary')
print (reloaded_dict)

Out:

Reloaded array:
[[0.40193978 0.55173167 0.89411291 0.84625061]
 [0.86540981 0.27835353 0.43359222 0.31579122]]

Reloaded list:
['A', 'B', 'C']

 Reloaded dictionary
{'name': 'Bob', 'Age': 42}

98: Brief examples of applying lambda functions to lists, and filtering lists with list comprehensions, map and filter

These examples are intended as a reminder of how to use list comprehensions, map and filter. They are not intended to be an exhaustive tutorial.

Let’s start with a list of numbers, and we wish to:

1) Square all numbers
2) Find even numbers
3) Square all even numbers

That is where list comprehensions, map and filter come in. There is significant overlap between these methodologies, but here they are.

Define my list of numbers

my_list = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

Square numbers with list comprehension

answer = [x**2 for x in my_list]
print (answer)
Out:
[1, 4, 9, 16, 25, 36, 49, 64, 81, 100]

Map a named lambda function to square the numbers

We use a one-line lambda function here, but a full function with define and return could also be used.

sq = lambda x: x**2
answer = list(map(sq, my_list))
print (answer)
Out:
[1, 4, 9, 16, 25, 36, 49, 64, 81, 100]

Map a lambda function directly to square numbers

answer = list(map(lambda x: x**2, my_list))
print (answer)
Out:
[1, 4, 9, 16, 25, 36, 49, 64, 81, 100]

Use a lambda function in a list comprehension

sq = lambda x: x**2
answer = [sq(x) for x in my_list]
print (answer)
Out:
[1, 4, 9, 16, 25, 36, 49, 64, 81, 100]

Filter a list with list comprehension to find even numbers

For even numbers x%2 will equal zero:

answer = [x for x in my_list if x%2 == 0]
print (answer)
Out:
[2, 4, 6, 8, 10]

Filter a list with a named lambda function

Full functions call also be used. The function (full or lambda must return True or False)

is_even = lambda x: x%2 == 0
answer = list(filter(is_even, my_list))
print (answer)
Out:
[2, 4, 6, 8, 10]

Filter a list with a lambda function applied directly

answer = list(filter(lambda x: x%2 ==0, my_list))
print (answer)
[2, 4, 6, 8, 10]

Combine squaring and filtering with a list comprehension

List comprehensions may filter and apply a function in one line. To get the square of all even numbers in our list:

answer = [x**2 for x in my_list if x%2 == 0]
print (answer)
Out:
[4, 16, 36, 64, 100]

Or we may apply lambda (or full) functions in both the operation and the filter ina list comprehension.

sq = lambda x: x**2
is_even = lambda x: x%2 == 0
answer = [sq(x) for x in my_list if is_even(x)]
print (answer)
Out:
[4, 16, 36, 64, 100]

Combine squaring and filtering with map and filter

filter and map would need to be used in two statements to achieve the same end. Here we will use named lambda functions, but they could be applied directly as above.

sq = lambda x: x**2
is_even = lambda x: x%2 == 0

filtered = list(filter(is_even, my_list))
answer = list(map(sq, filtered))
print (answer)
Out:
[4, 16, 36, 64, 100]

 

 

92. Queues

Python queues allow objects of any kind to be added to and removed from queues.

Here are three basic types of queue:

  1. FIFO (First In First Out)
  2. LIFO (Last In First Out)
  3. Priority (each item is assigned a priority in the queue)

Note: Python queues are not iterable; we cannot look through objects in the queue. It may at times be useful to maintain a separate list of queued items.

FIFO (First In First Out) queue

Here we will add numbers to a queue, but any Python object may be added to a queue.

# FIFO (First In First Out) queue

import queue
print ('FIFO (First In First Out) Queue')

q = queue.Queue()

# Add items to queue (add numbers 0 to 4)
for item in range(5):
    q.put(item) 

# Retrieve items from queue using a loop. 
# See additional method below of continually retrieving
# until queue is empty.
    
for item in range(5):
    x = q.get() # put item in queue
    print (x, end = ' ')

OUT:
FIFO (First In First Out) Queue
0 1 2 3 4

LIFO (Last In First Out) queue

# LIFO (Last In First Out) queue

import queue
    
print ('\nLIFO (Last In First Out) Queue')

q = queue.LifoQueue()

for item in range(5):
    q.put(item) # put item in queue
    
# Alternative way of emptying queue

while not q.empty():
    x = q.get() # put item in queue
    print (x, end = ' ')

OUT:
LIFO (Last In First Out) Queue
4 3 2 1 0 

Priority

In a priority queue an object is passed to the queue in a tuple. The first item of the tuple is the priority (lower number is higher priority by default), and the second item is the object to be queued. By default items of equal priority are handled as FIFO.

# Priority queue
import queue
import random
print ('\nPriority Queue')

q = queue.PriorityQueue()

for i in range(5):
    priority = random.randint(1,100)
    name = 'Item ' + str(i)
    item = (priority, name) # A tuple is used to pass priority and item
    q.put(item)

while not q.empty():
    x = q.get()
    print (x, end = ' ')

OUT:

Priority Queue
(15, 'Item 2') (47, 'Item 1') (52, 'Item 0') (54, 'Item 3') (81, 'Item 4')

83. Automatically passing unpacked lists or tuples to a function (or why do you see * before lists and tuples)

Imagine we have a very simple function that adds three numebrs together:

In [1]:
def add_three_numbers (a, b, c):
    return a + b + c

We would normally pass separate numebrs to the function, e.g.

add_three_numbers (10, 20, 35)
65

But what if our numbers are in a list or a tuple. If we pass that as a list then we are passing only a single argument, and the function declares an error:

In [3]:
my_list = [10, 20, 35]

add_three_numbers (my_list)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-900947bff326> in <module>()
      1 my_list = [10, 20, 35]
      2 
----> 3 add_three_numbers (my_list)

TypeError: add_three_numbers() missing 2 required positional arguments: 'b' and 'c'

Python allows us to pass the list or tuple with the instruction to unpack it for input into the function. We instruct Python to unpack the list/tuple with an asterix before the list/tuple:

add_three_numbers (*my_list)
65

 

 

19. Accessing date and time, and timing code

Accessing date and time

It may sometimes be useful to access the current date and/or time. As an example, when writing code it may be useful to access the time at particular stages to monitor how long different parts of code are taking.

To access date and time we will use the datetime module (which is held in a package that is also, a little confusingly called datetime!): Continue reading “19. Accessing date and time, and timing code”