[HN Gopher] Reloadium - Advanced hot reloading for Python
___________________________________________________________________
Reloadium - Advanced hot reloading for Python
Author : kwazar90
Score : 142 points
Date : 2022-04-26 14:41 UTC (8 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| EgoIncarnate wrote:
| The source code on github seems incomplete. Is this a closed
| source project?
| uniqueuid wrote:
| Live reloading is difficult, but it has some use cases.
|
| There was some interesting discussion the last (?) time the topic
| was posted [1] and some links to similar projects.
|
| In general, the limitations seem to prevent me from using
| techniques such as this as a default.
|
| [1] https://news.ycombinator.com/item?id=26035623
| kwazar90 wrote:
| What limitations exactly do you have in mind? Reloadium does
| not cause issues with redefined objects ("is" operator issues)
| uniqueuid wrote:
| One thing that comes to mind is additional imports. For
| example, what would happen with Reloadium if an edit
| introduces a circular dependency? Will it resolve a tree of
| changes to multiple files?
| kwazar90 wrote:
| If you intruduce a circular dependency (or any Python
| execution error) it will rollback all already applied
| actions, highlight the issue and wait for you to fix it.
| Reloadium reloads modules across the whole project so it is
| possible.
| daenz wrote:
| Very cool, and no doubt a difficult to accomplish feat. But it's
| unclear how useful it is. The author has a comment in this thread
| that "in some situations a full reload still might be needed",
| which makes sense, but also severely damages confidence in the
| tool. It will be very difficult for a user to understand when a
| full reload is needed, and if they don't have confidence that the
| correct code is always running, it makes them less likely to use
| it.
|
| If you can make it crystal clear when full reloading is needed,
| so that users can gain an intuition about that, it would help
| this tool a lot.
| siilats wrote:
| It has a pycharm plugin too but need some wheels for M1 if anyone
| can help with the issue I wrote up, would be great ...
| haolez wrote:
| Very cool! Most of your social network icons are broken, though.
| kwazar90 wrote:
| True, the website needs some doing...
| aidenn0 wrote:
| I assume this also makes it possible to reload a function/file
| when not in the middle of a breakpoint? That is something that is
| super-useful in lisp (reloading under both situations that is),
| but AFAICT impossible in vanilla python due to the way imports
| work.
| kwazar90 wrote:
| Yes hot reloading also works when not suspended and without
| using debugger. Although you won't be able to reload the
| current function (currently executing on the call stack) until
| you suspend it with a breakpoint.
| digisign wrote:
| Outside of the Django/Flask use case, not sure what this does:
|
| "Reloadium will rerun current function when changed. Developers
| get instant feedback whether the code is working or not."
|
| How does it know what parameters to pass? Makes them up at
| random? Runs the whole file as a script? Does static analysis?
| Runs test suite?
| djstein wrote:
| this is really awesome! thanks for sharing. I was wondering what
| would happen if used in Django, and bam, there was the example.
| wrycoder wrote:
| Or, if you are developing, just use IDLE.
| toxik wrote:
| Please don't post low-effort dismissals of people's projects.
| Not only is it rude, but also insubstantial. You can just say
| "IDLE does something similar [...]", and substantiate the
| claim. FYI, IDLE does not do what this project aims to do.
| wrycoder wrote:
| IDLE restarts the entire computation in a subprocess every
| time you press "run". It is excellent for beginners and small
| projects, as no reloading is required when you change the
| code of an imported module.
| Shish2k wrote:
| That's great... but that's not what this module is for
| qorrect wrote:
| Can't wait to dig into this. I've been wanting this for python
| for a long time, love the PyCharm integration right out of the
| gate.
| tomatowurst wrote:
| interesting....how does it play along with type hints? btw, this
| is one area where I can't really find a comeback in argument
| against typescript.
|
| would be cool if this somehow let me know before running the
| python code whether I am passing in the wrong type or not.
|
| very nice work, i dont really understand whats happening
| underneath the hood but any productivity boost is welcome.
| uniqueuid wrote:
| There may be a completely new approach, by the way, in the very
| interesting unison language [1]. Because its code is content-
| addressed, reloading should be possible by swapping out function
| identifiers (which are hashes, not names). Because the AST refers
| to functions by ID not name or label, one could perform
| replacements/changes without interfering with any existing code.
|
| [1] https://www.unison-lang.org/
| kwazar90 wrote:
| Interesting, is it already supported?
| uniqueuid wrote:
| As far as I know not. Unison currently seems focused on the
| cloud angle (which is also cool and smart). The idea is that
| you can offload computation to some other server even when it
| does not have the code, since it will just fetch the
| necessary function implementations from a global register.
| hprotagonist wrote:
| In general this shouldn't be possible, due to how the import
| mechanisms of python operate. I wonder how easy it is to break
| this / stale state / the usual and cause truly incomprehensible
| state bugs?
| kwazar90 wrote:
| Yes it is very difficult, it took me one year and thousands of
| unit, integration, e2e tests for it to become usable. It is
| quite stable now although in some situations a full reload
| still might be needed.
| whalesalad wrote:
| Kudos for persisting through the muck and shipping.
| kapilvt wrote:
| Its not clear if this is functional versus aspirational, its
| missing any useful code minus dataclass model definitions, which
| makes it rather to hard evaluate if its either useful or safe.
| https://github.com/reloadware/reloadium/tree/main/src
| tracnar wrote:
| I've been using jurigged [0] for hot code reloading which is
| great. It works by replacing code objects in place at the
| function level which naturally handles many cases like applying
| to existing objects, avoiding loading a module multiple times,
| etc. As others noted the actual source of this one is not there
| so it's hard to know what it's doing. The published package
| contains a binary shared library so I suppose it's written as a
| CPython extension.
|
| [0] https://github.com/breuleux/jurigged
| breuleux wrote:
| Jurigged author here, just tried reloadium out of curiosity. I
| couldn't run it on my M1 Mac (unsupported, it seems), on Linux
| I was able to, and it seems to work similarly (identifying what
| needs to be updated and updating these things specifically).
| That's nice to see. There is one thing jurigged does that this
| doesn't, which is updating closures, and I also define a
| protocol for custom libraries to watch changes so that e.g. a
| multiple dispatch library can update its internal structures
| when code changes. There may be things reloadium does better,
| for example it looks like there is some special casing for
| Flask/etc.? Not sure how that scales. I didn't try the PyCharm
| stuff.
|
| Glad jurigged works for you. Coincidentally, I was working on
| an update today, and I am going to present jurigged at PyCon
| this weekend :)
| [deleted]
| ewuhic wrote:
| Would it perform IO in background if a line states to, say,
| delete a file?
| kwazar90 wrote:
| As in whether it tracks IO file operations, to know to rollback
| a file deletion?
| ewuhic wrote:
| As to not delete my home folder or issue DROP TABLES as I see
| it also works with SQLA.
| kwazar90 wrote:
| As long your code does not state that nothing will happen.
| About the SQL - the only thing it does is rolling back
| transaction if you make an error during reloading to
| recover the db state.
| ewuhic wrote:
| Sounds like a gun to the foot.
| kwazar90 wrote:
| How so? I might make this optional if it's misbehaving
| but so far it's been very useful.
| [deleted]
___________________________________________________________________
(page generated 2022-04-26 23:00 UTC)