98: Brief examples of applying lambda functions to lists, and filtering lists with list comprehensions, map and filter

These examples are intended as a reminder of how to use list comprehensions, map and filter. They are not intended to be an exhaustive tutorial.

Let’s start with a list of numbers, and we wish to:

1) Square all numbers
2) Find even numbers
3) Square all even numbers

That is where list comprehensions, map and filter come in. There is significant overlap between these methodologies, but here they are.

Define my list of numbers

my_list = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

Square numbers with list comprehension

answer = [x**2 for x in my_list]
print (answer)
Out:
[1, 4, 9, 16, 25, 36, 49, 64, 81, 100]

Map a named lambda function to square the numbers

We use a one-line lambda function here, but a full function with define and return could also be used.

sq = lambda x: x**2
answer = list(map(sq, my_list))
print (answer)
Out:
[1, 4, 9, 16, 25, 36, 49, 64, 81, 100]

Map a lambda function directly to square numbers

answer = list(map(lambda x: x**2, my_list))
print (answer)
Out:
[1, 4, 9, 16, 25, 36, 49, 64, 81, 100]

Use a lambda function in a list comprehension

sq = lambda x: x**2
answer = [sq(x) for x in my_list]
print (answer)
Out:
[1, 4, 9, 16, 25, 36, 49, 64, 81, 100]

Filter a list with list comprehension to find even numbers

For even numbers x%2 will equal zero:

answer = [x for x in my_list if x%2 == 0]
print (answer)
Out:
[2, 4, 6, 8, 10]

Filter a list with a named lambda function

Full functions call also be used. The function (full or lambda must return True or False)

is_even = lambda x: x%2 == 0
answer = list(filter(is_even, my_list))
print (answer)
Out:
[2, 4, 6, 8, 10]

Filter a list with a lambda function applied directly

answer = list(filter(lambda x: x%2 ==0, my_list))
print (answer)
[2, 4, 6, 8, 10]

Combine squaring and filtering with a list comprehension

List comprehensions may filter and apply a function in one line. To get the square of all even numbers in our list:

answer = [x**2 for x in my_list if x%2 == 0]
print (answer)
Out:
[4, 16, 36, 64, 100]

Or we may apply lambda (or full) functions in both the operation and the filter ina list comprehension.

sq = lambda x: x**2
is_even = lambda x: x%2 == 0
answer = [sq(x) for x in my_list if is_even(x)]
print (answer)
Out:
[4, 16, 36, 64, 100]

Combine squaring and filtering with map and filter

filter and map would need to be used in two statements to achieve the same end. Here we will use named lambda functions, but they could be applied directly as above.

sq = lambda x: x**2
is_even = lambda x: x%2 == 0

filtered = list(filter(is_even, my_list))
answer = list(map(sq, filtered))
print (answer)
Out:
[4, 16, 36, 64, 100]

 

 

92. Queues

Python queues allow objects of any kind to be added to and removed from queues.

Here are three basic types of queue:

  1. FIFO (First In First Out)
  2. LIFO (Last In First Out)
  3. Priority (each item is assigned a priority in the queue)

Note: Python queues are not iterable; we cannot look through objects in the queue. It may at times be useful to maintain a separate list of queued items.

FIFO (First In First Out) queue

Here we will add numbers to a queue, but any Python object may be added to a queue.

# FIFO (First In First Out) queue

import queue
print ('FIFO (First In First Out) Queue')

q = queue.Queue()

# Add items to queue (add numbers 0 to 4)
for item in range(5):
    q.put(item) 

# Retrieve items from queue using a loop. 
# See additional method below of continually retrieving
# until queue is empty.
    
for item in range(5):
    x = q.get() # put item in queue
    print (x, end = ' ')

OUT:
FIFO (First In First Out) Queue
0 1 2 3 4

LIFO (Last In First Out) queue

# LIFO (Last In First Out) queue

import queue
    
print ('\nLIFO (Last In First Out) Queue')

q = queue.LifoQueue()

for item in range(5):
    q.put(item) # put item in queue
    
# Alternative way of emptying queue

while not q.empty():
    x = q.get() # put item in queue
    print (x, end = ' ')

OUT:
LIFO (Last In First Out) Queue
4 3 2 1 0 

Priority

In a priority queue an object is passed to the queue in a tuple. The first item of the tuple is the priority (lower number is higher priority by default), and the second item is the object to be queued. By default items of equal priority are handled as FIFO.

# Priority queue
import queue
import random
print ('\nPriority Queue')

q = queue.PriorityQueue()

for i in range(5):
    priority = random.randint(1,100)
    name = 'Item ' + str(i)
    item = (priority, name) # A tuple is used to pass priority and item
    q.put(item)

while not q.empty():
    x = q.get()
    print (x, end = ' ')

OUT:

Priority Queue
(15, 'Item 2') (47, 'Item 1') (52, 'Item 0') (54, 'Item 3') (81, 'Item 4')

83. Automatically passing unpacked lists or tuples to a function (or why do you see * before lists and tuples)

Imagine we have a very simple function that adds three numebrs together:

In [1]:
def add_three_numbers (a, b, c):
    return a + b + c

We would normally pass separate numebrs to the function, e.g.

add_three_numbers (10, 20, 35)
65

But what if our numbers are in a list or a tuple. If we pass that as a list then we are passing only a single argument, and the function declares an error:

In [3]:
my_list = [10, 20, 35]

add_three_numbers (my_list)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-900947bff326> in <module>()
      1 my_list = [10, 20, 35]
      2 
----> 3 add_three_numbers (my_list)

TypeError: add_three_numbers() missing 2 required positional arguments: 'b' and 'c'

Python allows us to pass the list or tuple with the instruction to unpack it for input into the function. We instruct Python to unpack the list/tuple with an asterix before the list/tuple:

add_three_numbers (*my_list)
65

 

 

19. Accessing date and time, and timing code

Accessing date and time

It may sometimes be useful to access the current date and/or time. As an example, when writing code it may be useful to access the time at particular stages to monitor how long different parts of code are taking.

To access date and time we will use the datetime module (which is held in a package that is also, a little confusingly called datetime!): Continue reading “19. Accessing date and time, and timing code”

18. Python basics: Lambda functions (one line functions), and map/filter/reduce

Lambda functions are a Pythonic way of writing simple functions in a single line.

They may be applied to a list using the map statement.

The function filter offers a concise way to filter out all the elements of a list based on the results of a function.

The function reduce continually applies a function to a sequence until a single value is returned. Continue reading “18. Python basics: Lambda functions (one line functions), and map/filter/reduce”

17. Python basics: functions

Functions are (or should be) short sections of code that may be called from within the main programme, or from another function. They usually take some inputs (arguments) and return one or more results.

Functions may be used for two purposes:

1) To avoid repeating code when the same code needs to be run in two or more locations in the main programme.

2) To structure the code, breaking the code down into smaller pieces. In functional programming the whole programme would be broken down into functions Continue reading “17. Python basics: functions”

16. Python basics: Read and write files

For data analytics we will usually be using two libraries called NumPy and Pandas which have their own simple methods for importing data, but here is the standard Python method. This standard method may sometimes have an advantage that data may be written and read one line at a time without having all data held in memory. This may be useful for very large data files. Continue reading “16. Python basics: Read and write files”