[HN Gopher] Lua-RTOS: a real-time operating system for ESP32
___________________________________________________________________
Lua-RTOS: a real-time operating system for ESP32
Author : lnyan
Score : 153 points
Date : 2021-06-09 10:38 UTC (12 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| mijoharas wrote:
| This looks like a cool project but there's something I'm confused
| about.
|
| Is the goal of this project to allow people to run realtime
| software? if so, isn't using lua a problem because of it's memory
| management causing GC stalls?
|
| It doesn't appear to be addressed in the README of the github.
| One other explanation is that I'm missing the point, and it's
| just using RTOS as a small embeddable OS on which to run lua, is
| that the case?
| m00x wrote:
| GC stalls aren't as bad on a low memory chip since there's much
| less memory to clean up.
| bob1029 wrote:
| I do soft real-time in .NET5 without any problems.
|
| I find that if I completely abduct a high-priority thread and
| never yield back to the OS, things on the critical execution
| path run incredibly smoothly. I am able to execute a custom
| high precision timer in one of these loops while experiencing
| less than a microsecond of jitter. Granted, it will light up an
| entire core to 100% the whole time the app is running. But, in
| today's world of 256+ core 1U servers, I think this is a
| fantastic price to pay for the performance you get as a result.
| Not yielding keeps your cache very toasty.
|
| Avoiding any allocation is also super important. In .NET,
| anything touching gen2 or LOH is death sentence. You probably
| don't even want stuff making it into gen1. Structs and streams
| are your friends for this adventure.
| snarfy wrote:
| All the real time guarantees would only ever happen in the
| libraries you are calling out to, not in the scripting
| language. That's just glue code. If the scripting language has
| the equivalent of eval() there is no way it can be made real
| time anyway.
| admax88q wrote:
| I can't speak for this projevt specifically, but you can do GC
| in a real time system. IBMs metronome garbage collector is a
| real time garbage collector.
| pjmlp wrote:
| Lua is definitly not the best option, but GC isn't an issue
| when a real time aware implementation is used.
|
| https://www.ptc.com/en/products/developer-tools/perc
|
| https://www.aicas.com/wp
|
| https://www.microej.com/product/vee/
| gnzoidberg wrote:
| I don't think this is true. No current GC tech is fully hard
| real time. (I am happy to be corrected, as it'd make my life
| way easier)
| nix23 wrote:
| Not hard real time, but with a fixed, short bound:
|
| https://researcher.watson.ibm.com/researcher/view_page.php?
| i...
| dTal wrote:
| That's the definition of hard real time, if I'm not
| mistaken. Is there an even stricter guarantee than than
| that?
| nix23 wrote:
| True, above that would be "live".
| pjmlp wrote:
| Real time enough to control weapons in war scenarios.
|
| https://www.ptc.com/en/blogs/plm/ptc-perc-virtual-machine-
| te...
|
| https://www.militaryaerospace.com/defense-
| executive/article/...
|
| https://www.aicas.com/wp/products-services/jamaicavm/
| galangalalgol wrote:
| Those examples were naval ships. When you can float a
| huge data center in water and dump all the waste heat you
| want, it doesn't matter that the Perc vm used is 2.5x as
| slow as the sun vm. Or that it uses even more ram than a
| normal java vm which is already a lot. This is running on
| a microcontroller.
| pjmlp wrote:
| Moving goal posts?
|
| I thought we were talking about real time GC, besides
| only the first example is a naval ship.
| galangalalgol wrote:
| Both fair points. Hard real-time is absolutely doable
| with GC if it is deterministic. It has throughput and
| memory penalties to get low and predictable latencies but
| it is regularly done. I have written soft real time java
| myself. And other than avoiding garbage like you would do
| with any low latency code with GC, it was idiomatic. But
| that one change does increase cognitive burden to the
| point I didnt find it any more productive than C++. If I
| had needed reflection I might have felt differently.
| kaba0 wrote:
| > It has throughput and memory penalties to get low and
| predictable latencies
|
| Hard realtime C++ does as well.
| jaywalk wrote:
| How does any of that matter? The parent comment provided
| examples of GC being used in hard real-time when the
| claim was that it couldn't be done.
| ludamad wrote:
| You _could_ disable the Lua GC and mostly manage C buffers with
| Lua functions. There 's some precedent given that Lua has been
| used a lot in embedding
| mijoharas wrote:
| can you disable the GC? in my last role we had a large C++
| application that had embedded lua. I didn't touch it much,
| but I would have thought that while most of the stuff it did
| was calling out to our C++ api, the "lua objects" and tables
| e.t.c. would still be created and need to be garbage
| collected as normal.
|
| Can you entirely turn off lua's GC?
| wahern wrote:
| Yes: collectgarbage("stop")
|
| See https://www.lua.org/manual/5.4/manual.html#pdf-
| collectgarbag...
|
| For such a dynamic language Lua is quite good about
| avoiding hidden dynamic allocation. Creating a new closure
| (or plain Lua function), coroutine, string, or table will,
| of course, allocate a new object. But all of those are
| either explicit or fairly obvious. Lua's iterator construct
| is careful to avoid dynamic allocation--I believe it's one
| reason why iterators take two explicit state arguments. And
| Lua has a special type for C functions (as opposed to Lua
| functions), allowing you to push, pass, and call C
| functions without dynamic allocation. Likewise for
| lightuserdata (just a void pointer), and of course numbers
| (floats and integers) and booleans--no dynamic allocation.
|
| Nested function calls could result in a (Lua VM) stack
| reallocation. But Lua guarantees tail call optimization.
| And the default stack size is a compile-time constant.
|
| Finally, Lua is very rigorous about catching allocation
| failure while maintaining consistent state. Well-written
| applications can execute data processing code from behind a
| protected call or coroutines resume (which is implicitly
| protected) and still keep chugging along in the event of
| allocation failure anywhere in the VM. The core of the
| application, such as the event loop and critical event
| handlers, can be written to avoid dynamic allocation after
| initialization.
| gnzoidberg wrote:
| Embedded != real time
| mmoskal wrote:
| If your heap is on the order of 100kB the GC stalls may not be
| so bad. A bigger problem may be pulling your code from external
| SPI flash - typically you will need to put all your real time
| code in RAM and you have only so much of it.
| hikarudo wrote:
| > isn't using lua a problem because of it's memory management
| causing GC stalls?
|
| Perhaps even more important is the increased memory use due to
| Lua. Some devices have very little memory to begin with.
| caseymarquis wrote:
| I think anything truly real time would be interrupt driven and
| use C/assembly. This would act as the glue between that
| critical code. I don't feel like we've hit peak embedded
| development yet. I love the idea of rapid embedded development
| that deploys scripts over a network or RS232, but most
| scripting languages aren't great for this. I would like to see
| a statically typed actor system with no GC or optional per
| actor GC that is pre-allocated and can only cause a single
| actor to fail in a predictable way.
| cinntaile wrote:
| There are languages invented for this purpose, such as Ada.
| I'd look at that instead of C when you're doing real time
| systems. Although there are many different kinds of real time
| systems so it depends on the goal. Soft real time systems
| aren't as time critical as hard real time systems so the way
| you program and reason about them is a bit different.
| bluGill wrote:
| How real time do you need to be? some real time is you need
| to react within milliseconds or bad things happen, humans
| will notice if there is 10ms lag in real time audio. Some
| real time is you need to react within seconds. A machine
| might need several tens of seconds to get up to speed, so
| controlling the motor only needs to update a once a second or
| so for the speed to remain within tolerance. And everything
| in between.
|
| Depending on where you are on the need for control different
| technologies can work for you.
| detinho wrote:
| I'm probably wrong, but isn't real time not about how fast
| you can react but if you can react on a defined time
| constraint? Or is there a time threshold you can't consider
| real time any more like 1 day? Edit: just like you said,
| 10ms, 1s, etc.
| tdeck wrote:
| You're correct, the definition requires that operations
| be bounded, not necessarily "fast".
| fullstop wrote:
| Maximum tick time for RTOS are usually measured in tenths
| of seconds. The range is typically from microseconds to
| about 100ms.
|
| edit: updated to reflect that I meant "upper bound"
|
| edit2: completely reworded to be more clear.
| monocasa wrote:
| Time constraints on an RTOS are what you need them to be
| for the domain.
|
| On the RTOS I was the lead for, some constraints were
| measured in 100s of ns.
|
| Edit to address the edit: our upper bound in that case
| was single digit microseconds.
| fullstop wrote:
| My point was that once you pass the 100ms mark most would
| not consider it "real-time".
| monocasa wrote:
| Ah it reads backwards, that you're asserting no one
| actually deals with constraints tighter than 100 ms.
| fullstop wrote:
| We're coming from different directions. My upper bound
| was "after this point it won't be rtos", but I've
| reworded it entirely to be more clear.
| monocasa wrote:
| Upper bound is a specific term in real time also
| unfortunately.
|
| Might I suggest wording such as "upper bound constraints
| exceeding 100ms aren't typically addressed with real time
| methods".
| detinho wrote:
| For what I see from the discution it seems to be more a
| practical decision to not use real time resources and
| techniques for much higher times. So 1 day for example
| would be a waste of development time to try to use real
| time, you could use alerts and background jobs to fix
| issues. But when you reach shorter and shorter timeframes
| and bad things can happen a real time approach starts to
| make more sense.
|
| Now, some parent comment mentioned RTOS. Maybe for a real
| time OS there would be a practical hard upper bound. But
| for real time systems in general this upper bound would
| be totally domain specific.
| bluGill wrote:
| The time depends on your problem. The real time is more
| about bad things happen if you don't respond on time. If
| you click on a webpage and your browser times out loading
| it is a real time failure, but the failure isn't bad
| enough that anyone thinks of browsers as real time
| because the failure isn't really harmful.
| mastax wrote:
| "RTOS" has come to mean _a library os for embedded
| microcontrollers_. A lot of embedded projects don 't care about
| realtime-ness at all.
| [deleted]
| samatman wrote:
| Lua's garbage collector can be driven 'manually' quite easily.
|
| That is, one can start it, stop it, run it to completion, run a
| 'step', tune the size of the step and the aggressiveness of
| collection, all from within Lua.
|
| It's true that you can't get hard realtime guarantees while
| using Lua naively, you do have to be aware of what you're
| doing. If you need to be 'cycle perfect', probably use
| something else.
|
| But there are an enormous number of applications where what Lua
| offers is just fine, and there's no reason a Lua program should
| have GC 'stalls', if that means unexpected and lengthy pauses
| in execution.
|
| This is a really cool project imho.
| OskarS wrote:
| I'm curious about this as well. And it's not just GC: just
| allocating memory is not real-time safe unless you're using
| something like a bump allocator. Lua seems very much like the
| wrong language for this.
| fullstop wrote:
| Lua doesn't need to use malloc directly -- you can replace
| the memory allocation function with your own implementation
| which is real-time safe.
| TickleSteve wrote:
| The Lua interpreter is not a real-time interpreter and cannot
| give bounded response time.
|
| This is a non-realtime application running on top of a scheduler
| that is capable of supporting realtime applications.
|
| Just placing an application on top of an RTOS does not make it
| realtime.
| teleforce wrote:
| Very interesting project on RTOS and hopefully this can support
| the new RISC-V based ESP32-C3 MCU [1].
|
| I wonder if the performance can significantly be improved if this
| is ported to Terra language, a system programming language meta-
| programmed in Lua [2]. It's going to stable 1.0 version real
| soon.
|
| [1]https://www.espressif.com/en/news/ESP32_C3
|
| [2]https://terralang.org/
| antattack wrote:
| The project seems to have started in 2017 but it does not appear
| to be very popular. Whitecat IDE is still in alpha.
|
| https://ide.whitecatboard.org/
| Renaud wrote:
| I thought there was something wrong on that page:L I wanted to
| try the IDE but just got this video filling the page and that
| single red button: "sign in with your google account".
|
| Why does an IDE for microcontrollers requires a Google Account?
| Guess I'll never know.
| mastrsushi wrote:
| A language centric OS isn't a compelling idea to users outside
| enthusiasts of that language.
| samatman wrote:
| Most RTOS are 'language centric', it's just that the language
| is C.
| rainer42 wrote:
| https://github.com/crmulliner/fluxnode follows the same idea
| providing a JavaScript runtime for the application development,
| runs on ESP32 and supports LoRa. Fewer features as this is a
| hobby project.
___________________________________________________________________
(page generated 2021-06-09 23:01 UTC)