[HN Gopher] Toolz: A functional standard library for Python
___________________________________________________________________
Toolz: A functional standard library for Python
Author : optimalsolver
Score : 84 points
Date : 2021-01-21 08:10 UTC (14 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| lysium wrote:
| I prefer functional style whenever I can. I noticed that this
| style slows Python programs down. It seems function calls are
| rather expensive. Does anybody have a similar experience?
| vmchale wrote:
| They have to check a recursion limit, for one.
| AlexCoventry wrote:
| It's been a while since I knew those kinds of details, but it
| certainly been the case in the past.
|
| https://stackoverflow.com/questions/22893139/why-is-a-functi...
| heavyset_go wrote:
| PEP 590[1] makes calling callables less expensive. It was
| finalized for Python 3.9.
|
| [1] https://www.python.org/dev/peps/pep-0590/
| thisiszilff wrote:
| Yes, I suspect it is a pitfall of a multi-paradigm language
| that cannot assume so much about code in order to optimize.
| Opinionated functional languages (ie clojure, haskell) can have
| a lot more guarantees about what is going on in order to
| optimize all those function calls, lexical bindings, etc.
| vmchale wrote:
| There's a bunch of oddities in Python, e.g. recursion limits.
| vorticalbox wrote:
| I've been writing functional code for a year or so now at work
| with the help of ramda in nodejs, there is also a port for
| python.
|
| My colleagues don't like becuase it's so different to what they
| are used to but I am putting out code faster and with less bugs
| so I'm not going to stop.
| archarios wrote:
| Hell yeah me too! Some of my coworkers like it, but there are a
| few out there who basically don't want to learn something new..
| I have been using Ramda professionally for about two years now
| and I love the heck out of it. My org is using more and more
| TypeScript which makes using Ramda a little harder but it still
| works pretty well in this context too. I have been working on a
| test data inventory project with Ramda recently and it has made
| the implementation so much nicer than it would have been
| otherwise. I started using pipeP (by defining it with pipeWith)
| and omg it's so great for dealing with a mix of async and non
| async calls.
| [deleted]
| zelphirkalt wrote:
| Why use a library for it? Can it not be done with just Python?
| And if using the library is much different from normal Python,
| does it mitigate Python's problems with functional programming?
| (For example one expression only lambdas and no TCO.)
|
| I also do use some functional concepts in my Python work, but
| do not use a library for it. Only procedures or functions. No
| additional dependencies.
| gugagore wrote:
| I think `grouby` is a compelling example:
|
| https://toolz.readthedocs.io/en/latest/control.html#other-
| pa...
|
| > Most programmers have written code exactly like this over
| and over again, just like they may have repeated the map
| control pattern. When we identify code as a groupby operation
| we mentally collapse the detailed manipulation into a single
| concept.
|
| If you don't use a library, then you have to re-write
| something like groupby many times, I would expect. Or WORSE,
| you don't even use the pattern, writing "code exactly like
| this over and over again".
| zelphirkalt wrote:
| Huh, interesting. This does not remind me of the database
| groupby operation, but rather of partition, like in SRFI-1.
| I mean in natural language it is clear to me, why they'd
| name it groupby, but in programming terms I think partition
| is more appropriate, as groupby is already "blocked" by the
| database operation.
|
| Often one only needs one of the partitions though, which is
| when filter is sufficient. Otherwise I guess one can easily
| write partition oneself and then use that function over and
| over again, without resorting to a library.
|
| But perhaps it is a good example, so that you do not have
| to write partition in every project and if the additional
| dependency is OK to have, why not, if it is indeed a good
| one.
| seertaak wrote:
| Functional programming in Python is severely hampererd by
| Python's lambda keyword, which - simply put - sucks.
| samasblack wrote:
| I like the way the Functional Programming HOWTO in the Python
| docs (https://docs.python.org/3/howto/functional.html#small-
| functi...) puts it: Fredrik Lundh once
| suggested the following set of rules for refactoring uses of
| lambda: 1. Write a lambda function. 2. Write a
| comment explaining what the heck that lambda does. 3.
| Study the comment for a while, and think of a name that
| captures the essence of the comment. 4. Convert the
| lambda to a def statement, using that name. 5. Remove the
| comment.
| cabalamat wrote:
| Given that Python allows unicode characters in source, I'm
| surprised it doesn't allow lambda to be replaced by the l
| character.
| JNRowe wrote:
| For those, like me, who get excited by this note that there
| are restrictions on the Unicode categories that are allowed,
| see the supported characters1 and gory details2. It is often
| enough to write math-y code in a usable way, but occasionally
| you'll find you can't use the character you want.
|
| 1 https://docs.python.org/3/reference/lexical_analysis.html#i
| d...
|
| 2 https://www.python.org/dev/peps/pep-3131/
| alrs wrote:
| I'm surprised that you're surprised.
| vmchale wrote:
| Also the recursion limit.
| pansa2 wrote:
| And no tail-call optimization.
| xapata wrote:
| Intentional to improve tracebacks. There's trade-offs with
| every design decision.
| heavyset_go wrote:
| Yeah, Python would really benefit from multi-line anonymous
| functions.
|
| There was a limitation in Python where spacing and indentation
| gets ignored between parentheses, which makes it impossible to
| pass a multi-line lambda as an argument to a method or
| function. However, given the new parser, that limitation might
| be able to be mitigated.
| pansa2 wrote:
| I don't think this limitation is likely to change - the only
| ways to allow whitespace-sensitive statements inside an
| argument list are all really ugly.
| scott_russell wrote:
| Agreed. I suspect that most uses of python decorators become
| moot with proper multi-line anonymous functions. I assume
| some would argue that decorators create more readable code
| but they seemed like a syntax hack to me.
| fwip wrote:
| You can do multiple lines, just not multiple statements:
| >>> f = lambda x: x * \ ... 20 >>>
| >>> f <function <lambda> at 0x7f8b69f2c6a8>
| >>> f(2) 40
| [deleted]
| pansa2 wrote:
| In fact, no statements at all - just a single expression.
| tutfbhuf wrote:
| But you can define functions inside of functions
| (recursively). def function1():
| print ("hello outer") def function2():
| print ("hello inner") function2()
| function1()
|
| Output: hello outer hello inner
| throwaway894345 wrote:
| But you can't do that (or try/except) in an expression such
| as a list comprehension or similar.
| robertlagrant wrote:
| Yes, because gross.
| mumblemumble wrote:
| I can't agree with this any harder.
|
| I've spent the past few months familiarizing myself with
| a legacy codebase in a functional language. If I could
| pick a single thing about the code that is just the
| absolute worst stumbling block, and that has made
| learning understanding the code unreasonably difficult,
| it's multiline anonymous functions. In general, if the
| function is doing something complex enough to need
| multiple lines, it's doing something complex enough to
| deserve a name.
| 0x2c8 wrote:
| Can you elaborate on this? What is wrong with Python's
| `lambda`?
| rustyminnow wrote:
| Like others have said:
|
| - Can only have one line
|
| - Can only use expressions, not statements. E.g. `print`s,
| loops, conditionals are out.
|
| - Overall just kinda clunky
|
| Here's an SO post about lambdas where the answer is "Use def
| instead." https://stackoverflow.com/questions/14843777/how-
| to-write-py...
| bobbylarrybobby wrote:
| The lambda calculus is Turing complete, so in theory,
| Python's lambdas should suffice...
| jolux wrote:
| Turing completeness is completely unrelated to whether or
| not anonymous functions are easy to use in Python.
| rustyminnow wrote:
| I mean... You're right. They suffice. They're just less
| friendly (i.e. useful) than lambdas in other languages.
| zelphirkalt wrote:
| Which of course says nothing about usability and
| readability.
| heavyset_go wrote:
| print() is a function now and you can use it with lambdas.
| diarrhea wrote:
| This works for conditionals: In [4]: f =
| lambda x: "Yes" if x else "No" In [5]: f(True)
| Out[5]: 'Yes' In [6]: f(False) Out[6]:
| 'No'
|
| It's Python's version of ternary operators, so not sure if
| that counts as a "true" conditional; but it is one.
|
| Loops don't work, but list comprehensions do, and they are
| definitely the way to go here. Multi-line loops deserve a
| `def`.
| seertaak wrote:
| Exactly. Kind of an own-goal too, from none other than ex-
| BDFL. While not totally obvious, it's not impossible to
| widen the syntax to allow multi-line lambdas, you just need
| to ditch the stack-based lexer-integrated whitespace
| sensitivity behaviour.
| VWWHFSfQ wrote:
| The use of lambda is becoming somewhat of a smell in Python
| in general. PSF's own black code formatter will complain
| about using it and pretty much always says to just use a def
| instead
| kissgyorgy wrote:
| It can only have one line for example.
| pansa2 wrote:
| Python's `lambda` can only contain a single expression.
|
| There's no good way to add support for full anonymous
| functions to Python's grammar. One of the rules that makes
| significant-whitespace work elegantly is that statements can
| contain expressions, but never vice-versa.
| ausjke wrote:
| since lambda is for simple and short anonymous functions most
| of the time why do I need type the whole word each time? can
| they also do what javascript does(or similar):
| x => x * 2
|
| instead of lambda x : x * 2
| cabalamat wrote:
| I like: lx: x*2
| pansa2 wrote:
| Until very recently, Python's grammar was strictly LL(1),
| so the parser couldn't handle, for example, `(x, y) => x *
| y`.
|
| Perhaps with the move to a PEG parser, this syntax could
| now be supported?
| AlexCoventry wrote:
| Also by the lack of persistent data structures with structural
| sharing. That forces you to take an expensive copy whenever you
| want a modified version of some data, or bash it in place.
| heavyset_go wrote:
| There's Pyrsistent[1], which provides persistent data
| structures.
|
| [1] https://github.com/tobgu/pyrsistent
| Iwan-Zotow wrote:
| Isn't it a true functional way?
| iimblack wrote:
| I could definitely be wrong but I think most functional
| data structures aren't fully copied. Instead, they'll
| utilize something like a pointer to the data that stayed
| the same so only the part that changes is "copied".
| alentist wrote:
| There are smarter ways to deal with data structures in the
| functional setting than copying the entire structure over
| and over again.
|
| See https://en.wikipedia.org/wiki/Purely_functional_data_st
| ructu... and Okasaki's book _Purely Functional Data
| Structures_.
| gameswithgo wrote:
| I imagine it is problematic that Python is already very slow,
| and then to pile up idioms with performance downsides on top of
| that is kind of rough.
| jolux wrote:
| Functional programming doesn't have an inherent performance
| downside, but it depends on specific optimizations in
| compilers and runtimes to make it fast. Python doesn't really
| optimize anything from what I know.
| bj0 wrote:
| This is one of the biggest reasons I really like (the python
| superset) Coconut: https://github.com/evhub/coconut
| refactor_master wrote:
| It looks interesting, but I'm not entirely sold. All the
| examples are about math functions, which is kind of stupid to
| implement in Python.
|
| How does this benefit "plumbing"?
|
| My biggest problem with Python is that I either have to write
|
| list(function1(*function2(len(obj.method())))
|
| Or try to avoid all the variables by using classes... which
| is fine... if it wasn't for self taking up roughly a third of
| the word count.
| Grimm1 wrote:
| I had a bit of a derp here at first, I was like what, python
| already has a functional (as in working) standard lib, then I
| read the docs and was like ohhhhh that functional.
|
| I think python made some half measures in regards to functional
| programming, not a bad thing since python tends to blend the
| different styles decently but at least for me it would be nice to
| have a good library to extend the functional side a little more,
| hopefully this scratches the itch.
| jonmoore wrote:
| Toolz, like seemingly everything Matt Rocklin is a major
| contributor to, is something of a model library: cleanly designed
| and coded, with strong documentation.
|
| Although Python is not going to match a full Lisp, Haskell or ML
| in all their strengths, using a functional style can be useful
| and expressive. The toolz docs give some relevant background at
| https://toolz.readthedocs.io/en/latest/heritage.html .
|
| At a language level, Peter Norvig gave a lengthy comparison of
| Python and Lisp at https://norvig.com/python-lisp.html in 2000.
___________________________________________________________________
(page generated 2021-01-21 23:01 UTC)