[HN Gopher] Problems once solved by a metaclass can be solved by...
___________________________________________________________________
Problems once solved by a metaclass can be solved by
__init_subclass__
Author : imaurer
Score : 153 points
Date : 2022-01-05 16:35 UTC (6 hours ago)
(HTM) web link (til.simonwillison.net)
(TXT) w3m dump (til.simonwillison.net)
| imaurer wrote:
| URL is to a "Things I Learned" by Simon W, but the title comes
| from tweet by David Beazley:
|
| https://twitter.com/dabeaz/status/1466731368956809219
|
| Which is a warning that people shouldn't buy his new book if they
| want a deep dive on Metaprogramming:
|
| https://www.amazon.com/dp/0134173279/
|
| Simon does a nice job demonstrating how "the __init_subclass__
| class method is called when the class itself is being
| constructed."
| captainmuon wrote:
| I've only once used metaclasses but I think it was pretty nifty.
| The use case was with Python and Gtk+ (both version 2), if you
| had your UI in a glade file you could define a corresponding
| class, and it would automatically create members for all your
| named widgets. It made it a bit like coding in VB:
| class MainWindow(gtk.Window): __metaclass__ =
| MetaDescribedUI("MainWindow.ui") def
| __init__(self): # now you can use e.g.
| self.lvFiles or self.btnOK pass
|
| Although writing it now I could probably do it differently
| without metaclasses.
| chadykamar wrote:
| "Metaclasses are deeper magic than 99% of users should ever worry
| about. If you wonder whether you need them, you don't (the people
| who actually need them know with certainty that they need them,
| and don't need an explanation about why)."
|
| -- Tim Peters
| tybug wrote:
| I realize this quote isn't meant to be taken literally, and
| it's a nice one-liner encapsulation of metaclasses, but it
| always bothered me. The people who "actually need metaclasses"
| were, at some point, learning about metaclasses for the first
| time and were "wondering whether they need them".
|
| This quote ignores the middle ground of users who have a valid
| use case, but don't yet understand metaclasses. Not great
| advice for people who are trying to learn metaclasses.
| jcranmer wrote:
| I do sympathize with this sentiment, but there are times when
| "if you're wondering if you need it, you don't need it" is a
| very true statement, and Python metaclasses are in that boat.
| To see why, let me explain in more detail:
|
| There's a "basic" [1] understanding of Python's object model,
| one which understands that most of Python is actually
| syntactic sugar for calling certain special methods. For
| example, the expression a[b] is "really" just calling
| a.__getitem__(b). Except that's not actually true; the "real"
| object model involves yet more dispatching to get things to
| work. Metaclasses allow you to muck with that "yet more
| dispatching" step.
|
| So when do you need metaclasses? When you need to do that
| low-level mucking--the kind of mucking most won't know about
| until actually needed. If all you know about metaclasses is
| kind of what they are, then you very likely haven't learned
| enough about them to use them to actually need them.
| Conversely, if you've learned those details well enough to
| need to muck with them, then you've also learned enough to
| the point that you can answer the question as to whether or
| not you need metaclasses.
|
| [1] I suspect most Python users don't even have this level of
| understanding, which is why I put it in scare quotes.
| nas wrote:
| I suspect Tim's meaning was that ordinary users don't need to
| know how metaclasses work in detail. It is enough that people
| know they exist (i.e. do something at time class is defined).
| If you need them, you are doing deep enough magic that you
| can take the time to learn how they work.
|
| Maybe he was also suggesting that people should generally not
| be using them. In my experience, it is exceedingly rare to
| need them. In 29 years of writing Python code, I think I've
| used them once or twice.
| ajkjk wrote:
| For all of Python's magic hackery that it lets you inject it
| feels like there are still some obvious 'holes'.
|
| The one that bites me a lot is: I want to easily be able to write
| a decorator that accesses variables on a particular _instance_ of
| a class (such as a cached return value), and I want to take a
| Lock when I do it.
|
| But since decorators are defined on the class method, not the
| instance, it has to do a bunch of work when the function runs:
| look at the instance, figure out if the lock is defined, if not
| define it (using a global lock?), take the lock, then do whatever
| internal processing it wants. It feels like decorators should
| have a step that runs at `__init__` time to do per-instance setup
| but instead I have to figure it out myself.
| Spivak wrote:
| I'm so confused. from functools import wraps
| from threading import Lock def my_decorator(f):
| # Do whatever you want to set up the lock. lock =
| Lock() @wraps(f) def wrapper(self,
| *args, **kwargs): # Do whatever with the lock.
| lock.acquire() # Do whatever you want with
| the instance. print(self.greeting)
| return f(self, *args, **kwargs) return wrapper
| class MyClass: def __init__(self):
| self.greeting = "Hello World!" @my_decorator
| def my_method(self): print("And all who inhabit
| it.") >>> MyClass().my_method() Hello
| World! And all who inhabit it.
|
| Edit: Wait do you want the decorator, before it ever runs, to
| add a property self.lock? Are you sure you don't want this to
| be a class decorator that messes with __init__? Because you can
| 100% wrap init to do that kind of set up.
| ajkjk wrote:
| The problem is that `lock = Lock()` is shared across all
| instances of the class, which creates contention if there are
| many of them. To solve this you need to save a per-instance
| lock, so yes to "Wait do you want the decorator, before it
| ever runs, to add a property self.lock".
|
| An example usecase is writing a @cached decorator that
| behaves well in multithreaded programs and can be used on
| classes that have many instances, with the caching happening
| per-instance.
| wheelerof4te wrote:
| Python is slowly going in the direction of C++. Sad to see,
| really.
| ajkjk wrote:
| I don't think it's nearly as bad. The magic nonsense is fairly
| well-constrained inside the OOP system and there is, imo, a
| certain coherent logic to it.
| ChuckMcM wrote:
| Not particularly slowly, as fast as it can manage :-).
| sam0x17 wrote:
| I love that Python is unique and strange enough that merely
| seeing "__init_subclass__" in this headline is enough to know
| exactly what language it is talking about ;)
| Naac wrote:
| For those curious, here is a great explanation of what
| metaclasses are in python: https://stackoverflow.com/a/6581949
| matsemann wrote:
| How does this play with typing and other tools? I hate the use of
| all the magic stuff in Python making it needlessly hard to use
| lots of libraries.
| sillysaurusx wrote:
| It's fine. https://news.ycombinator.com/item?id=29812639
| joebergeron wrote:
| (Shameless self-plug incoming...)
|
| For those curious about what metaclasses actually are, I wrote up
| an article with some motivating examples about this a little
| while back. Might be worth a read if you're interested!
|
| https://www.joe-bergeron.com/posts/Interfaces%20and%20Metacl...
| kkirsche wrote:
| Even using Python regularly I haven't run into a need for meta
| classes. I'm sure there is a valid reason, but what is it?
| throwaway19937 wrote:
| (https://www.linuxjournal.com/article/3882) has a dated (2000)
| example of metaclass usage in Python 2.
| enragedcacti wrote:
| More generically, Metaclasses are primarily useful when using
| (abusing?) the Python type system to represent other type
| systems. The most common example of this is ORMs like
| SQLAlchemy, where the Python type system is used to represent
| tables in a relational database.
|
| Another example is the builtin "Typing" module where the type
| system is used to provide a language for representing itself.
| "Union" itself is a type, and metaclasses allow you to set
| "__getitem__" on the "Union" type such that Union[Int|Str] is
| valid and represents a new type. Not that I don't know if
| metaclasses are actually used to build this system internally,
| but you could use it for many of the features implemented by
| Typing.
|
| There are alternative ways to go about doing that because
| Python is so flexible, but IMO metaclasses (or
| __init_subclass__ as seen in this post) are the least "magic"
| of the magic ways to tackle problems like these.
| dragonwriter wrote:
| The need _isn 't_ common, but there are some real uses. The top
| answer of this SO thread (I didn't read the rest, so there may
| be more goodies) has some good discussion of use cases
| including comparison to non-metaclass approaches to the same
| problems.
|
| https://stackoverflow.com/questions/392160/what-are-some-con...
| fdomingues wrote:
| Is useful to do code generation/transformation, for example in
| this utility[1] it uses the information provided on the
| declaration of the class to generate the methods required by
| the base class.
|
| The user of the utility write this: class
| Book(objects.Object): title: Optional[str]
|
| And the generated class code is: class
| Book(Base): __slots__ = ('title', )
| title: typing.Optional[str] def
| __init__(self, title=None): self.title = title
| @classmethod def from_data(cls, data):
| title = data.get('title', None) return
| cls(title=title) def to_data(self):
| data = {} if self.title is not None:
| data['title'] = self.title return data
| @classmethod def from_oracle_object(cls, obj):
| return cls(title=obj.title) def
| to_oracle_object(self, connection): obj =
| connection.gettype('Book').newobject()
| obj.title = self.title return obj
|
| [1]https://github.com/domingues/oracle-object-mapping
| Sohcahtoa82 wrote:
| I recently used them to create a framework for handling plugins
| to an IRC bot.
|
| By creating an IrcMessageHandler metaclass, I can easily
| subclass it to encapsulate functionality into different classes
| without having to add boilerplate to load each plugin. There's
| a Plugins directory with a __init__.py that automatically loads
| all .py files, and the main bot just has to do `from Plugins
| import *` to load them all. Each plugin script merely needs a
| `from ircbot import IrcMessageHandler`, then creates a
| subclass. So there's a Calc class that adds a "!calc" command,
| Seen class that adds "!seen", and more. Each of them merely
| needs to include an "on_message" function that parses IRC
| messages.
| tybug wrote:
| The most salient example I can think of is tracking all
| subclasses of a class. This is how django tracks `Model`
| subclasses to create a table for each model, for instance. I
| wrote a blog post on this topic:
| https://tybug.github.io/2021/11/06/tracking-all-subclasses-o...
|
| I've also used metaclasses in the past to apply a decorator to
| certain methods of all subclasses, without needing to specify
| that decorator in the subclass declaration itself.
| dreyfan wrote:
| Python is quickly taking the crown for low-barrier to entry,
| slow, buggy code supported primarily by stackoverflow copy pastes
| from a never-ending supply of "data scientists"
| ransom1538 wrote:
| I can write php in any language.
| dreyfan wrote:
| modern php and js/ts are both rather impressive how far
| they've come compared to the jumble of crap we had a decade
| ago.
| sam0x17 wrote:
| I miss it. On the javascript side there was just jquery and
| a mountain of vanilla js that everyone could at least
| understand (and no npm dependency issues, ES6+ weirdness,
| etc) and on the PHP side, all built-in functions were in
| the global scope (easy to CTRL+F on a single page at a time
| when search endpoints were quite slow), there were
| (virtually) no web frameworks (at least I didn't use any
| until after 2009), and no package management to worry about
| unless you were doing something weird and/or wrangling with
| wordpress. As a tween and young teen I had plenty of small
| web design clients.. local businesses, photographers, etc.,
| and I could build sites 100% from scratch starting from a
| photoshop mockup to custom PHP/HTML/CSS/JS, 100% no
| packages or other people's code. Packages have done a ton
| for us, for sure, but I truly miss those days and feel like
| we've lost something we'll never get back.
|
| Back then it was actually quite possible to build an entire
| website or "app" as we call them now from first principles
| just banging your head against HTML/CSS/JS and whatever
| server side language you were using. Then the internet grew
| up, security issues became more prevalent, packages, other
| people's code, dependency hell, "futures", "promises",
| server state and client state management, shadow dom,
| reactivity, etc etc.
| ww520 wrote:
| VB6 of modern day.
| robervin wrote:
| Quickly seems like an overstatement; you've always been able to
| write low-barrier to entry, slow, buggy code. Is there a low-
| barrier language that doesn't end up with a lot of less-than-
| ideal code samples?
| shrimpx wrote:
| There's a lot of nice Python code out there, mostly written by
| disciplined and experienced systems people.
|
| But I agree about the hordes of data scientists making copy and
| paste spaghetti. This is evidenced by companies deploying
| Jupyter notebooks in production, which essentially witnesses
| the fact that they've given up on getting data people to write
| quality production Python, and instead they treat their
| prototype spaghetti as sandboxed blackboxes that can be tested
| on inputs and deployed if they perform well enough.
| reedf1 wrote:
| If you can write code twice as fast then you'll write twice as
| many bugs.
|
| Having been a C++ and python developer I see just as many
| footguns in both languages and have spent many many many hours
| dealing with slow, buggy code in either.
| ThePhysicist wrote:
| Meh, Python really shines at I/O and gluing together
| C/C++/Fortran code. It's reasonably fast for most tasks, and if
| it isn't there are tools like Cython that make it pretty
| trivial to write Python code that achieves C-like execution
| speed (because it effectively compiles to C code).
| bloblaw wrote:
| I agree that Python is a great glue language (so are Perl and
| Python).
|
| However, Python really doesn't shine when compared with other
| memory safe languages like Go, Rust, and Java.
|
| Ultimately, Python is single-threaded by design. There are
| some hacks to get around this design contraint (multi-
| processing), but then you are adding an entire additional
| Python runtime to your system's memory for every "thread" you
| want to run. Even with asyncio, Python performs the worst
| when it comes to I/O.
|
| I love Python, but it's important to understand that due to
| specific design considerations it is not and will likely
| never be a performant language.
|
| reference: https://medium.com/star-gazers/benchmarking-low-
| level-i-o-c-...
| zbyforgotpass wrote:
| I wish there was a 'Modern Python' tutorial that would walk me
| through all the new python stuff like this or the type
| declarations or the new libs, etc.
| diarrhea wrote:
| I learned about metaclasses in "Fluent Python". An excellent
| book, but it predates typing, so that's not covered. Then
| again, I believe a second edition is on its way.
| cutler wrote:
| Evidence, if any was needed, that OOP, or at least the Python
| variant, was never designed for metaprogramming. Give me Clojure
| macros any day of the week over these ugly contortions.
| wheelerof4te wrote:
| A recent anegdote with regards to typing in Python.
|
| Let's say you have this code:
|
| class Creature: def __init__(self, name: str,
| hp: int, attack: int): ... def
| attack(self, other: Creature): ...
|
| Guess what? Since Creature is not yet defined, your very smart
| type-checker can't see the "other" parameter. Happened to me in
| VS Code.
|
| That is the problem with features bolted on to a language where
| they perhaps don't belong. Now, I know that typing helps when you
| have simple, functional code. So, this code will work:
|
| from dataclasses import dataclass
|
| @dataclass
|
| class Creature: name: str hp: int
| attack: int
|
| def combat_between(attacker: Creature, defender: Creature): ...
|
| I was really surprised with this. We need better review in
| regards to adding additional features that may be broken on
| certain setups.
| orf wrote:
| Simply add quotes around "Creature" and it works.
|
| > We need better review in regards to adding additional
| features that may be broken on certain setups.
|
| What does this even mean. The documentation explains that you
| need to do this, and it's fairly obvious why you would.
|
| Type hints are one of the best features added to Python.
| wheelerof4te wrote:
| Yeah, "simply add quotes", indeed.
|
| Except, it is not that simple. It's not even intuitive.
| Typing is a great feature on paper, but in practice, it is
| just something we don't need in a scripting and glue
| language.
| orf wrote:
| Except it is simple and it is fairly intuitive. It is also
| pretty awesome. See fastapi, pyndantic, dataclasses, etc
| etc.
| tda wrote:
| That's why you can add quotes to types: def
| attack(self, other: 'Creature'):
| wheelerof4te wrote:
| Thanks, I did not know about this.
| wheelerof4te wrote:
| Isn't it ironic that one of the most readable procedural
| languages today has almost unreadable OOP syntax?
|
| People unironically created interfaces and various overloaded
| operations using dunder methods. Not only do you need to keep a
| list of them under your pillow, but you must also keep notes on
| how to construct the interface for each one.
|
| And they look ugly as hell. In a language that prides itself on
| readability and user friendlines.
|
| My suggestion is to ditch the OOP syntax completely and make a
| built-in analogue to a C struct. Take inspiration from
| dataclasses syntax. OOP in a dynamic glue language is a silly
| idea anyway. Of course, you would have to bump the version to
| 4.0.
| amelius wrote:
| __This__ __looks__ __very__ __interesting__!
| pengwing wrote:
| I feel jumping right into __init_subclass__ without explaining
| why metaclasses exist and what problems they typically solve
| excludes the majority of devs on HN. Thereby limiting any
| discussion only to the advanced Python developer echo chamber,
| while discussion across very different devs is usually far more
| interesting.
| floober wrote:
| Note that this doesn't appear to have been submitted by the
| post's author, and HN may not have been the intended audience.
| abdusco wrote:
| Simon keeps his TIL (today I learned) blog as a way to take
| notes mainly for himself, from what I understand.
|
| I like the way he documents everything in Github issues[0]. I
| learn a new thing or two with every release of Datasette,
| because there's so much troubleshooting full of relevant
| links and solutions. Same with his TIL blog.
|
| [0]: https://github.com/simonw/datasette/issues/1576
| DonaldPShimoda wrote:
| I've not seen this before but it's so cool! I've lost track
| of all the ways I've tried to keep track of things I've
| learned. Maybe this would be a good way to do it.
|
| Link to the GitHub repo for those interested:
| https://github.com/simonw/til
| godelski wrote:
| Could you, or someone else, clue us in?
|
| Edit: another user linked their blog
| https://news.ycombinator.com/item?id=29813349
| Macha wrote:
| One use is for e.g. binding of database models for an ORM.
| class FooModel(metaclass=Base): x =
| Field(type=int, primary_key=True) y =
| Field(type=reference(BarModel))
|
| The Base metaclass will set it up to implement methods like
| save() by inheriting from parent classes, but it would also
| be nice for the library to have a list of all model types
| without the library user having to call a method like
| FooORM.register_type(FooModel). So the metaclass is being
| used in these classes to build up a dictionary of models when
| the class definition is encountered.
|
| The metaclass is basically a class that itself builds
| classes, which means it can be syntactically convoluted.
|
| However, with __init_subclass__ you can write a thing that
| looks like a regular class with regular parent methods, but
| instead just gets a method called each time the interpreter
| encounters a new subclass, which lets you do things like
| build up that dictionary for your ORM.
| diarrhea wrote:
| What classes are to instances, metaclasses are to classes.
|
| This is from someone who has only ever read about, never used
| metaclasses, because they are widely regarded similar to git
| submodules. If you cannot really assert that you need them,
| you don't. They solve very specific problems, mostly found in
| library, not user code. A library can then allow user classes
| to be modified comprehensively. If you control the classes in
| the first place (not library code), you probably can do
| without metaclasses.
| dragonwriter wrote:
| > What classes are to instances, metaclasses are to
| classes.
|
| Given the popularity of this construct for analogies and
| metaphorical comparisons, it should be noted that this is
| _strictly literal_. In Python classes are objects, and the
| classes whose instances are classes are called
| "metaclasses" (and they are subclasses of the class
| "type".)
| tshaddox wrote:
| Is this definition recursive? Meaning that classes whose
| instances are metaclasses are also metaclasses?
| [deleted]
| xapata wrote:
| Check the type of `type` in Python ;-)
| dragonwriter wrote:
| Because metaclasses are classes, yes, classes whose
| instances are metaclasses are also classes whose
| instances are classes and thus are also metaclasses. But
| you could distinguish this subset of metaclasses as
| "metametaclasses" if you wanted to, to distinguish them
| from more general metaclasses just as metaclasses are
| distinguished from more general classes.
|
| But AFAIK no one has come up with a distinct application
| for custom metametaclasses which would make having
| terminology to discuss them necessary or useful other
| than for entertainment.
| fdgsdfogijq wrote:
| I once heard a quote that described David Beazely as:
|
| "The Jimi Hendrix of programming"
| zzzeek wrote:
| Yeah this is great, __init_subclass__ comes along to make dynamic
| injection of methods and attributes on class creation easier,
| just as the entire practice is fast becoming fully obsolete
| because type checkers like pylance and mypy report these
| attributes and methods as errors.
| robertlagrant wrote:
| Yes, this. This is where type checking becomes potentially
| harmful: you have to start creating loads of explicit
| subclasses instead of one dynamic class. And then you're
| writing slow Java.
| shrimpx wrote:
| That's why I resist python types. A lot of the usefulness and
| interestingness of Python is in its dynamic features which aid
| in rapidly developing complex systems. Hard to imagine a
| strictly typed future for Python, when you then have little to
| trade off for its bad performance and lack of real concurrency.
| JavaScript has seen success as a typed language because it has
| being "the exclusive frontend language" going for it, and v8 is
| high performance. And the dynamic capabilities of JavaScript
| were never that interesting or powerful.
| ryanianian wrote:
| > Hard to imagine a strictly typed future for Python
|
| On the contrary, the dynamic nature of python can make it
| seem like inaccessible black-magic where you don't even know
| where to look in the code to see what's even possible.
| Stronger types will empower users to understand their
| ecosystems.
|
| I've spent countless hours in rails/django land trying to
| figure out what's even possible with an object that the ORM
| or whatever gives me. The human-flavored docs kinda outline
| it, but if you're in IDE-land, context-switching to prose
| isn't productive.
|
| Types help humans develop and debug, but they make library
| authors jump through acrobatics to express what their magic
| does. This is a limitation of the type-checker
| implementation(s) not having a strong syntax or notion to
| capture the runtime-dynamic polymorphism, but it doesn't mean
| that the concept of types is a flawed idea or not worth using
| when appropriate.
|
| Don't throw the baby out with the bath-water.
| shrimpx wrote:
| I like types, but I have a hard time seeing them in Python.
| If I was forced to use only typed Python, I'd just use
| golang and get a ton of performance and concurrency that I
| could never get in typed Python.
|
| Good point about Django though. Django has so much
| historical stickiness that types probably look attractive
| for entrenched corporate projects that are hard to port to
| high performance natively typed languages.
| dragonwriter wrote:
| > Yeah this is great, __init_subclass__ comes along to make
| dynamic injection of methods and attributes on class creation
| easier, just as the entire practice is fast becoming fully
| obsolete because type checkers like pylance and mypy report
| these attributes and methods as errors.
|
| The Python community and the set of people that treat mypy and
| pylance as dictators rather than tools to be used where
| appropriate and turned off where not are...not the same thing.
| (And the latter very much depends on tools built by the rest of
| the former that internally are wild and woolly, even if they
| present a nice cleanly typed interface to the consumer.)
| zzzeek wrote:
| sure...im coming from the library producer perspective. it's
| not very easy for us to say, "just turn off the type checker
| when you use our library" :)
| dragonwriter wrote:
| > it's not very easy for us to say, "just turn off the type
| checker when you use our library" :)
|
| Sure, and that is a problem with exposing certain kinds of
| dynamism when your customer base is the kind that is
| inflexible about typing.
|
| OTOH, a lot can still be done with dynamism on the inside
| and a typed public interface, and also there's a
| significant part of the community that accepts selective
| disabling of typechecking as an acceptable when dynamism
| provides adequate benefits.
| j4mie wrote:
| Python is the most popular language on the planet _because_
| it's a dynamic language, not in spite of it. If the type
| checker makes dynamic features difficult, ditch the type
| checker, not the dynamic features. I can't wait for this
| pendulum to swing back the other way.
| Tanjreeve wrote:
| Was there ever a point people were successfully building
| "backbone" systems in dynamically typed languages? I thought
| the pythons and Perl's of the world were always mainly doing
| scripts/interface applications.
| stevesimmons wrote:
| See [1] for a PyData London talk I did on "Python at
| Massive Scale", about JPMorgan's Athena trading and risk
| management system, used for FX, Commodities, Credit,
| Equities, etc trading. At that time, Athena had 4,500
| Python developers doing 20,000 commits a week.
|
| Also see one recent HN discussion on "An Oral History of
| Bank Python" [2].
|
| [1] https://www.youtube.com/watch?v=ZYD9yyMh9Hk
|
| [2] https://news.ycombinator.com/item?id=29104047
| ThePhysicist wrote:
| That's my observation as well. If you write typed Python code
| (or Typescript) you need to constrain yourself to a small
| subset of what the dynamic language can do. In some cases it's
| beneficial but often it can be quite stifling, so not sure I'm
| a big fan of gradual typing anymore.
| sillysaurusx wrote:
| There are a few ways to deal with that. One is to declare the
| properties directly in the subclass: class
| Inject: def __init_subclass__(cls):
| cls.injected = 42 class Child(Inject):
| injected: int print(Child().injected) # 42
|
| This technique isn't too useful in practice, though it has its
| place (e.g. when creating ctypes.Structures). But there's a
| clever way to simplify this. Remember that the subclass is a
| subclass -- it inherits from Inject: import
| typing as t class Inject: injected:
| t.ClassVar[int] def __init_subclass__(cls):
| cls.injected = 42 class Child(Inject): pass
| print(Child().injected) # 42
|
| (ClassVar[int] ensures that type checkers know it's a variable
| attached to the class, not each instance.)
| zzzeek wrote:
| yes you can do that, but if you want to do say, what
| dataclasses does, where it collects all the Field objects and
| creates a typed `__init__` method, typing can't do that
| without plugins. all the current typecheckers hardcode the
| specific behavior of "dataclasses" without there being any
| pep that allows other systems (like ORMs) to take advantage
| of the same thing without writing mypy plugins or simply not
| working for other type checkers that don't allow plugins.
| sillysaurusx wrote:
| Yeah, that's true. Dataclass support is severely lacking.
| Is there a pep on the horizon for this?
| zzzeek wrote:
| im not really sure (highly doubtful, much simpler ideas
| like being able to make your own Tuple subclass are still
| not ready for primetime) but this would be a pretty tough
| pep to allow the general "dataclass" concept to be
| available to other libraries that want to use the same
| pattern, but not use dataclasses.
| VWWHFSfQ wrote:
| There are bugs in this code and I'm glad that I'm not the only
| one that has done it! graph = {
| key: { p for p in
| inspect.signature(method).parameters.keys() if p
| != "self" and not p.startswith("_") } for
| key, method in cls._registry.items() }
|
| The first parameter to a bound method does not have to be called
| `self`. It's conventional, but not required. Is there a better
| way in the inspect module to filter these parameters? This comes
| up more often with classmethods where the naming convention `cls`
| is most common but I see `klass` somewhat frequently as well.
| 3pt14159 wrote:
| I'm working on a project right now that uses __init_subclass__
| and it works great, but takes some tricks (like having classvars
| with types set) in order to get mypy to cooperate. But it's way
| easier to reason about than metaclasses.
| echelon wrote:
| I'm at a point in my career where I see "magic" and wince hard.
|
| This is so hard to reason about and fix at scale. If you let
| other engineers run wild with this and build features with it,
| the time will eventually come to decom your service and move
| functionality elsewhere. Having to chase down these rabbits,
| duplicate magic, and search large code bases without the help of
| an AST assisted search is slow, painful, and error prone.
|
| I spent several years undoing magic method dispatch in Ruby
| codebases. Tracing through nearly a thousand endpoints to detect
| blast radiuses of making schema changes impacted by CRUD ops on
| lazily dispatched "clever code".
|
| I'm sure there are valid use cases for this, but be exceedingly
| careful. Boring code is often all you need. It doesn't leave a
| mess for your maintainers you'll never meet.
|
| Python users tend not to behave this way, but seeing posts like
| this requires me to urge caution.
| radicalbyte wrote:
| ..and there was me thinking that it was only Java which
| suffered from that problem.
|
| That kind of code is great if:
|
| (a) you wrote it
|
| (b) you knew what you were doing when you wrote it and
|
| (c) you also wrote an extensive set of tests to cover the
| entire domain of whatever you were doing.
| tentacleuno wrote:
| (d) and you used comments!
|
| Comments are underestimated. I'm not saying they should be
| used as an excuse to use "magic" esoteric code, but sometimes
| such solutions are needed and it's much better to document
| them for future maintainers.
| jhgb wrote:
| Wasn't the idea behind this to make _most_ code boring, and
| have only a small part of it exciting? Just like when
| structured programming was created, or even local variables
| introduced, etc. (Of course, the exciting part had to be moved
| into the compiler in those cases.)
| scottwitcher83 wrote:
| tsujamin wrote:
| I recently encountered __init_subclass__ for the first time in
| Home Assistant's custom integrations. Configuration handlers were
| registerd by "just declaring the class" like so and I couldn't
| grok how it was registered by the runtime class
| MyCustomOptionsFlow(OptionsFlow, DOMAIN="MYPLUGIN"):...
|
| A bit of digging showed that __init__subclass_ was being used
| effectively as a registration callback, adding the newly defined
| class/DOMAIN to the global CustomOptions handler at class
| definition.
|
| Very neat, not immediately obvious however :/
| lvass wrote:
| Am I just bad at OOP hackery or does anyone else look at this and
| think it'd be extremely hard to debug?
| foolfoolz wrote:
| i wouldn't let this past code review. it's magic and makes a
| great blog post about some feature but is not maintainable code
| draw_down wrote:
| pmarreck wrote:
| Yep. As a current functional-language guy, this merely confirms
| my suspicion that those crazy functional people were onto
| something.
|
| As it turns out, since the human mind is the real limitation
| here, anything that stops runaway complexity ("hiding"
| functionality behind "magic" is actually worse because you're
| just moving the complexity "away from your mental model" where
| it can fester and grow out of sight, even if what you see looks
| "simpler") results in fewer bugs, and more-debuggable bugs.
| jreese wrote:
| I'm not fond of the example given in the post, but I've very
| happily used __init_subclass__ in a few places where I
| previously would have needed a decorator. Eg, for "plugin-like"
| things where I would have otherwise needed a decorator to keep
| track, or iterated over subclasses, now can just "register"
| subclasses without any extra work on the child classes. Things
| that I normally wouldn't expect to change the behavior of the
| subclass, but where having a no-effort hook at subclass
| creation time can be extremely useful.
| sillysaurusx wrote:
| I used to think so, but cls.__subclasses__() makes it
| possible to just walk the type tree at some later point. I
| find the lack of state to be a feature. (Normally with a hook
| like you're talking about, you'd add the class to some list
| somewhere, which can get out of sync.)
| qsort wrote:
| Frankly I doubt you even need metaclasses in the first place
| (at least in non-library code).
|
| I get that it's a dynamic language but if I need metaclasses
| for something it's more likely than not to be a hidden
| technical debt generator.
| vosper wrote:
| I think SQLAlchemy makes good use of metaclasses. It's the
| only thing I've seen that used them in a way that I thought
| made sense and was well justified (but I have been out of the
| Python world for a while).
|
| In another language it would be done with code-gen, which
| would also be fine.
|
| But SQLAlchemy is library code.
|
| I took the time to understand how to use metaclasses, and
| concluded that I never would, especially not in code I
| expected another engineer to understand - I expect 90% of
| people who write Python haven't heard of metaclasses, and 99%
| have never written one.
| nomel wrote:
| There was a time, early on in my python career, that I
| tried all the different flavors of magic that python
| provides. These days, I tend to write code for junior
| developers. I don't use any magic at all, even when it
| could make sense.
|
| Once you start playing with magic, you have far fewer eyes
| and hands that are interested.
| nightski wrote:
| In my opinion it's not about dealing with junior
| developers or even having the ability to understand it.
| I've been programming for 25 years and to me it's about
| cognitive load. I despise tricks like this because they
| put a much higher strain on the programmer. Software is
| hard enough, let's not make it harder.
|
| If I can look at a piece of code and it's quickly obvious
| whether it is correct or not is my favorite type of
| code...
| formerly_proven wrote:
| Metaclasses / __init_subclass__ are generally only used if you
| try to define different / new behavior for how the body of a
| class is interpreted. So if you'd want to create e.g. an
| explicit interface system in Python, you might use them. If
| you're creating an ORM or some other kind of declarative
| property system, you might use them (but not necessarily, as
| descriptors are a thing). I think the only use in the standard
| library is in ABCs and probably the typing module.
|
| Edit: Enums use them as well (makes sense).
| klodolph wrote:
| It's used very sparingly in the wild, for cases where you need
| a bunch of classes that are set up similarly, and ordinary
| subclassing still results in too much boilerplate.
|
| For example... you might make a class for each HTML tag in an
| HTML processor, or each drawing command in an SVG processor. My
| experience is that it's not hard to debug, because all that
| you're doing is getting rid of some boilerplate.
| Traubenfuchs wrote:
| I am baffled that people make fun of Java and Spring and then
| use double underscore magic methods called __init_subclass__
| with a straight face.
___________________________________________________________________
(page generated 2022-01-05 23:00 UTC)