Lambdas Forever

Ahhh Python, what a splendid scripting language. One of the most likeable features is the anonymous functions, aka Lambda. The lambda is actually a way to write/implement a simple one liner function in-place. The official docs says:

“Lambda forms (lambda expressions) have the same syntactic position as expressions. They are a shorthand to create anonymous functions; the expression lambda arguments: expression yields a function object.”

Instead of implemented the damned comparison function for sorting, you probably all know what I’m talking about:
def ComparisonProc(x, y):
    return  y – x

We can simply do:
list.sort(lambda x, y: y – x) # Descending
and voila.

This is a very simple example. Lambdas are really handy when you want to do one liner devices. Some of them which you manage to stuff in one line and some which you just can’t. However, without lambda it wouldn’t have been possible in the first place.

There are many samples in the Internet. I came up with something, hopefully even useful. Let’s say you want to print all .JPG files on your c:\, including subdirectories. So we have to scan the h.d for all files, then filter those with .JPG extension and afterwards print the result. :) Yes this is all possible in one-liner, let’s see the original code first:

for root, dirs, files in os.walk(‘c:’):
    for i in files:
            if i[-4:] == “.jpg”:
                    print i

The one-liner version:

print filter(lambda name: name[-4:] == ".jpg", reduce(lambda x,y:x+y, [i[2] for i in os.walk('c:')]))

See? Easy :)

Actually now, I have to explain a few more things.
1) We are only interested in the Files list from os.walk, therefore we take the third entry in the result, that’s i[2].

2) The i[2] itself, is a list, and we cannot filter a list of lists with the file names, therefore we have to flatten the lists to a single list containing the file names. This is where the reduce comes in, it will return the accumulated result of all lambdas – each time calling the lambda with the accumulated x and supplying the next item, y. Thus, adding the lists extends the resulting list and flatten them…

3) Now that we the a single list with all file names in the system, we need to filter out the files which are not .JPG. So yet again we use a lambda that checks the last 4 characters in the file name and assures whether it is a .JPG, all other files will be removed from the resulting list.

4) Lastly, print the result. Actually you can use pretty print (module pprint) to print it prettier :)

 Yes, Python’s crazy!

So what’s the annoying things with lambdas? They are slow relatively to list comprehensions (which we used to get all lists of file names above). But again, if we are using scripting – are we after speed? I am not sure. Another irritating thing about lambdas is that you cannot assign inside the expression, but then you have reduce.. :)

The worst thing about lambdas is when you use global variables, and let me explain. Since lambdas are evaluated at runtime (I hope I am right here) if you access some variables outside of the lambda, they will get re-evaluated everytime with the lambda itself. Now think that you wanted the lambda to have a specific value when you created the lambda, and then when you really call the lambda, that value was already changed and your result is screwed.

Enough words, let’s see the problem with some code:

>>> x, y = [lambda z: 5+z+i  for i in xrange(2)]
>>> x(0)
>>> y(0)

Oh uh, we’re in trouble!

Do you notice we get the same result for both functions? This is incorrect because they are not supposed to return the same value. Note this:

>>> i

So now when both lambdas, x and y, are evaluated they use i as 1. Sucks huh? 

>>> i = 3
>>> x(0)
>>> y(0)
“Use Nested Lambdas, Luke”

x, y = [(lambda v: lambda z: 5 + v)(i) for i in xrange(2)]
>>> x, y = [(lambda v: lambda z: 5 + v)(i) for i in xrange(2)]
>>> x(0)
>>> y(0)

The outter lambda is get evaluated immediately and thus leaves the value, and not the pointer to the value, in the code. Next time when the inner lambda is evaluated it uses the value-of(i) and not the value-from-pointer-of(i).

This surely will help someone out there :) And then they say Lambdas are going to be deprecated in Python 3000…


Thanks to Kasperle, here’s another solution to the nested lambdas:

x, y = [lambda z, dummy = i: 5 + dummy for i in xrange(2)]

The drawback is that x and y can now get a second parameter which you potentially let the caller override…Or if we’re talking about Python 2.5 you can use functools.partial.

7 Responses to “Lambdas Forever”

  1. Kasperle says:

    With list comprehensions, you can often also do without lambda expressions:

    print [name for name in reduce( list.__add__, [i[2] for i in os.walk( ‘c:’)]) if name[-4:] == ‘.jpg’]

    Another construct from functional programming languages was introduced with python2.5: partial function application.

    Partial function application allows you to “bind” an argument to another function, resulting in a function that accepts one argument less:

    >>> def foo( x,y):
    … return x + y

    >>> foo( 1, 2)
    >>> bar = functools.partial( foo, 2)
    >>> bar( 1)

  2. lorg says:

    I usually prefer using nested functions for solving this problem. I dislike the dummy=i solution, seems ugly to me.
    This works just the same for regular closures, which are still there even if you use list comprehensions (which I do).

  3. lorg says:

    Another thing…
    Instead of the specific reduce you used, you can do sum(x,[]) where x is the list you ran reduce on. example:
    In [1]: l = [[1,2],[3,4]]

    In [2]: sum(l,[])
    Out[2]: [1, 2, 3, 4]

  4. arkon says:

    Ahh nice ;) The big question is how to flatten a more than 1 level of nested lists…

  5. lorg says:

    I think I once gave it out as a riddle:
    Write a function to flatten a nested list,
    for example,
    [[[1,2],3],4,5] -> [1,2,3,4,5]
    (for the keen of eye, this is just a walk on a tree).

  6. arkon says:

    for that you need recursion..

  7. Erez says:

    I have some comments about all these code samples running around here.

    1. i[-4:] == “.jpg” would really be better as i.endswith(‘.jpg’) . Not only is it prettier and less prone to mistakes, it’s faster.

    2. lambda x,y:x+y should be replaced by operator.add. Again, not only prettier, but also faster.

    3. do not use “sum” for lists. In fact, don’t use it for anything aside from immutables, for it does not make good use of mutability (is that a word?), and you get O(n**2) by using it.
    def list_sum(*lists):
    res = []
    for l in lists:
    res += l
    return res

    is much much faster (even faster than using itertools.chain, which is still better than sum).

    4. Sounds like you would really have a good time with a true functional programing language, such as Lisp or Haskell. There, you can do all the lambdas you’ll ever want :)

Leave a Reply