[HN Gopher] GNU lightning: generates assembly language code at r...
___________________________________________________________________
GNU lightning: generates assembly language code at run-time
Author : mpsq
Score : 93 points
Date : 2021-04-07 11:15 UTC (1 days ago)
(HTM) web link (www.gnu.org)
(TXT) w3m dump (www.gnu.org)
| analognoise wrote:
| Too bad it's GPL, is there something like this that's MIT, BSD,
| or Apache licensed?
| lucian1900 wrote:
| Cranelift is licensed Apache 2
| https://github.com/bytecodealliance/wasmtime/tree/main/crane...
| analognoise wrote:
| Awesome I'll check it out, thank you.
| peter_d_sherman wrote:
| https://www.gnu.org/software/lightning/
|
| >"GNU lightning is a library that generates assembly language
| code at run-time; it is very fast, making it ideal for Just-In-
| Time compilers, and it
|
| _abstracts over the target CPU_
|
| , as it exposes to the clients a standardized RISC instruction
| set inspired by the MIPS and SPARC chips."
|
| [...]
|
| >"GNU lightning is usable in complex code generation tasks. The
| available backends cover the aarch64, alpha, arm, hppa, ia64,
| mips, powerpc, risc-v, s390, sparc and x86 architectures."
|
| https://www.gnu.org/software/lightning/manual/lightning.html
|
| >"To be portable,
|
| _GNU lightning abstracts over current architectures' quirks and
| unorthogonalities._
|
| The interface that it exposes to is that of a standardized RISC
| architecture loosely based on the SPARC and MIPS chips. There are
| a few general-purpose registers (six, not including those used to
| receive and pass parameters between subroutines), and arithmetic
| operations involve three operands--either three registers or two
| registers and an arbitrarily sized immediate value.
|
| _On one hand, this architecture is general enough that it is
| possible to generate pretty efficient code even on CISC
| architectures such as the Intel x86 or the Motorola 68k
| families._
|
| On the other hand, it matches real architectures closely enough
| that, most of the time, the compiler's constant folding pass ends
| up generating code which assembles machine instructions without
| further tests."
|
| [...]
|
| >"3 GNU lightning's instruction set
|
| GNU lightning's instruction set was designed by deriving
| instructions that closely match those of most existing RISC
| architectures, or that can be easily syntesized if absent. Each
| instruction is composed of:
|
| an operation, like sub or mul most times, a register/immediate
| flag (r or i) an unsigned modifier (u), a type identifier or two,
| when applicable. Examples of legal mnemonics are addr (integer
| add, with three register operands) and muli (integer multiply,
| with two register operands and an immediate operand). Each
| instruction takes two or three operands; in most cases, one of
| them can be an immediate value instead of a register.
|
| Most GNU lightning integer operations are signed wordsize
| operations, with the exception of operations that convert types,
| or load or store values to/from memory. When applicable, the
| types and C types are as follow: _c
| signed char _uc unsigned char _s
| short _us unsigned short _i int
| _ui unsigned int _l long _f
| float _d double
|
| Most integer operations do not need a type modifier, and when
| loading or storing values to memory there is an alias to the
| proper operation using wordsize operands, that is, if ommited,
| the type is int on 32-bit architectures and long on 64-bit
| architectures. Note that lightning also expects sizeof(void*) to
| match the wordsize.
|
| When an unsigned operation result differs from the equivalent
| signed operation, there is a the _u modifier.
|
| There are at least seven integer registers, of which six are
| general-purpose, while the last is used to contain the frame
| pointer (FP). The frame pointer can be used to allocate and
| access local variables on the stack, using the allocai or allocar
| instruction.
|
| Of the general-purpose registers, at least three are guaranteed
| to be preserved across function calls (V0, V1 and V2) and at
| least three are not (R0, R1 and R2).
|
| _Six registers are not very much, but this restriction was
| forced by the need to target CISC architectures which, like the
| x86_
|
| [PDS: I think despite this trade-off, that this was a _good
| engineering decision(!)_ , otherwise x86 could not be targeted,
| and that would destroy a huge swath of functionality!]
|
| , are poor of registers; anyway, backends can specify the actual
| number of available registers with the calls JIT_R_NUM (for
| caller-save registers) and JIT_V_NUM (for callee-save registers).
|
| There are at least six floating-point registers, named F0 to F5.
| These are usually caller-save and are separate from the integer
| registers on the supported architectures; on Intel architectures,
| in 32 bit mode if SSE2 is not available or use of X87 is forced,
| the register stack is mapped to a flat register file. As for the
| integer registers, the macro JIT_F_NUM yields the number of
| floating-point registers.
|
| The complete instruction set follows; as you can see, most non-
| memory operations only take integers (either signed or unsigned)
| as operands;
|
| _this was done in order to reduce the instruction set_
|
| [PDS: Opinion: Any way that the instruction set can be simplified
| is a good thing!]
|
| , and because most architectures only provide word and long word
| operations on registers. There are instructions that allow
| operands to be extended to fit a larger data type, both in a
| signed and in an unsigned way."
|
| [...]
|
| >"8 Acknowledgements
|
| As far as I know, the first general-purpose portable dynamic code
| generator is DCG, by Dawson R. Engler and T. A. Proebsting.
| Further work by Dawson R. Engler resulted in the VCODE system;
| unlike DCG, VCODE used no intermediate representation and
| directly inspired GNU lightning.
|
| Thanks go to Ian Piumarta, who kindly accepted to release his own
| program CCG under the GNU General Public License, thereby
| allowing GNU lightning to use the run-time assemblers he had
| wrote for CCG. CCG provides a way of dynamically assemble
| programs written in the underlying architecture's assembly
| language. So it is not portable, yet very interesting.
|
| I also thank Steve Byrne for writing GNU Smalltalk, since GNU
| lightning was first developed as a tool to be used in GNU
| Smalltalk's dynamic translator from bytecodes to native code."
|
| [...]
|
| PDS: Overall, looks like a great idea!
|
| Also, I think that future compiler and VM writers and FPGA soft-
| CPU authors -- should target this "abstracted instruction set"!
|
| Keywords: Instruction Set Architecture, ISA, x86, RISC, RISC-V,
| Abstraction, Abstract Instruction Set, Instruction Set Subset,
| Generic Compatibility Layer
| MaxBarraclough wrote:
| > I think that future compiler and VM writers and FPGA soft-CPU
| authors -- should target this "abstracted instruction set"!
|
| GNU lightning succeeds in what it sets out to do, which is to
| offer a simple and minimal JIT code-generator. It offers
| nothing in the way of optimisation, by design. Most projects
| looking for a code-generator are looking for something with
| great optimisation built-in, so they're not wrong to go with
| LLVM or the JVM rather than GNU lightning (or something similar
| like Mir [0][1]). I don't think the average compiler would gain
| much by targeting GNU lightning.
|
| With all that said, GNU Guile, a Scheme interpreter, uses a
| fork of GNU lightning, insufferably named _lightening_. [3]
|
| [0] https://github.com/vnmakarov/mir
|
| [1]
| https://lists.gnu.org/archive/html/lightning/2020-02/msg0001...
|
| [2] https://wingolog.org/archives/2019/05/24/lightening-run-
| time...
| bonzini wrote:
| I'm not sure how well GNU lightning has aged, but considering
| that the above text was written by a pretty clueless 20 year
| old guy (me), that's a great compliment. Thanks. :)
| riffraff wrote:
| I believe Guile and GNU Smalltalk used to run on lightning, do
| they still do?
|
| What other projects use GNU lightning these days?
| sedachv wrote:
| GNU CLISP had GNU Lightning added as an option for compiling
| its VM bytecode in 2008. IMO a bad decision - more maintenance
| burden; code can now have subtle different bugs whether run in
| the CLISP interpreter, CLISP bytecode VM, or CLISP bytecode
| compiled with GNU Lightning; generated code is much larger than
| bytecode (small memory use was the one unique thing CLISP has
| over other Common Lisp implementations); performance is still
| low compared to other Common Lisp compilers.
|
| Today libgccjit seems like a much better alternative for this
| kind of application than GNU Lightning.
| bjoli wrote:
| Guile rund om a modified version of lightning 1 (called
| lightening for maximum confusion) due to the recent version
| being more heavy weight.
|
| It is a part of the recent JITted 3.0 release, which brought a
| lot better performance to the already completely OK 2.2 branch.
| snicker7 wrote:
| GNU Guile (known primarily for its Scheme implementation) uses a
| modified version Lightning under the hood.
| dannyobrien wrote:
| Does anyone know how this compares with libgccjit ? What are the
| difference in their use cases?
|
| (I note that Guile picked lightning, and emacs chose libgccjit
| for instance for what seems to be similar roles.)
| rekado wrote:
| Guile uses lightening (with an "e"), which is derived from GNU
| lightning: https://gitlab.com/wingo/lightening
| moonchild wrote:
| Libgccjit is based on regular gcc, which is a heavyweight
| compiler. The 'jit' is somewhat of a misnomer; you wouldn't
| want to actually use that for any of the same cases as a
| traditional jit (e.g. browsers). (LLVM jit has the same
| problems.)
|
| I think the _main_ point of libgccjit is actually to provide a
| cleaner interface than GIMPLE to gcc.
| zekrioca wrote:
| Documentation was last updated in September, 2019
| brudgers wrote:
| Stable software doesn't require modification.
|
| If a tool is intended to do one thing well, then it won't be
| constantly getting features.
|
| Or to put it another way, users have just been able to use it
| for almost two years without time spent upgrading to the latest
| release.
|
| Quicksort is still a useful algorithm after nearly sixty years.
| Merge sort after almost eighty.
| MaxBarraclough wrote:
| I like the GNU lightning project, but I wouldn't say it's so
| stable it doesn't need updating. Disassembly doesn't work
| properly on Ubuntu, which strikes me as a pretty serious
| problem, but which unfortunately they seem to be in no hurry
| to fix. [0]
|
| I think they'd do well to adopt a proper bug-tracker. Mailing
| lists are a good way to lose track of known issues.
|
| [0] https://lists.gnu.org/archive/html/lightning/2020-06/msg0
| 000...
| brudgers wrote:
| If it doesn't work on Ubuntu that seems like a fault with
| Ubuntu. That's where the breaking change is.
| MaxBarraclough wrote:
| I don't think the fault is with Ubuntu. _configure_
| scripts are responsible for coping with variations
| between different distros. lightning 's configure script
| fails to accommodate the way Ubuntu installs the
| _binutils-dev_ package. It works ok with (Red Hat based)
| Amazon Linux 2 on x86-64. It failed on Amazon Linux 2 on
| AArch64 though, with a seemingly unrelated issue, which
| was the second bug I reported.
|
| I don't have the specifics to hand. I also don't know
| autotools, so I'm not well positioned to write a fix.
| [deleted]
| vinceguidry wrote:
| There's an algorithm and then there's it's implementation. So
| much more goes into implementation. That said, September 2019
| is much much better than a lot of software I use.
| [deleted]
| zekrioca wrote:
| Sure, I agree. However, it is also a sign that no new
| features nor capabilities have been added.
|
| > Quicksort is still a useful algorithm after nearly sixty
| years. Merge sort after almost eighty.
|
| I am not sure specific algorithms really compare to a whole
| suit of algorithms and libraries (i.e., a tool) composing
| very specific versions of a software like GNU lightning, but
| sure, both Quicksort and Mergesort are useful since their
| inception. Try installing Windows 3.1 (don't use VMs please)
| on your current computer and see if we can really live
| without updating things..
|
| But I'd argue that if we follow your idea, we can say that
| the democratic concept is also useful since the 18th century.
| And still, it has to adapt throughout time and be somehow
| updated (e.g. allowing women and black people to vote or
| freely use public spaces like toilets and elevators).
| [deleted]
| 1vuio0pswjnm7 wrote:
| Love this comment.
| orf wrote:
| Has had 6 commits in the last two years. From the same person.
| 2.1.0 was released 6 years ago, the latest 2.1.3 release came out
| two years ago. Releases are available from some random FTP
| server.
|
| On one hand this could be software that is completely done and
| requires no more maintenance. On the other hand it could be
| another random half-baked GNU project with a single champion who
| has lost interest but is still saddled with a purist/terrible new
| contributor onboarding experience dooms it to an inevitable
| grave.
|
| You can be the judge of that.
| rekado wrote:
| "Random FTP server"? You mean the GNU FTP server?
| orf wrote:
| You speak of that like it's some bastion of software
| delivery. You mean THE gnu ftp server??
|
| It's a random FTP server. Could be sourceforge for it all it
| matters. Doesn't even support SFTP.
| rekado wrote:
| If you look closely you'll see that ftp.gnu.org is an HTTP
| server. But it seems that you just want to troll.
| orf wrote:
| And if you look closer you'll see that it's a FTP server
| with a HTTP interface (http, not https, by default mind
| you).
|
| ftp://ftp.gnu.org/
| rjeli wrote:
| Why does it only have six registers? The end user probably
| doesn't want to write a register allocator.
| tom_mellior wrote:
| "Six registers are not very much, but this restriction was
| forced by the need to target CISC architectures which, like the
| x86, are poor of registers; anyway, backends can specify the
| actual number of available registers with the calls JIT_R_NUM
| (for caller-save registers) and JIT_V_NUM (for callee-save
| registers)." (https://www.gnu.org/software/lightning/manual/lig
| htning.html...)
|
| There used to be a boardly similar library called libjit
| https://www.gnu.org/software/libjit/ which was slightly higher
| level: It offered an arbitrary number of virtual registers. It
| took care of register allocation, I don't recall how
| sophisticated it was. The docs call it "global" allocation,
| i.e., taking the whole function into account at once, as
| opposed to local allocation that considers blocks in isolation.
| AlexAltea wrote:
| Might not be the reason, but six registers make it trivial to
| write a backend for 32-bit x86 CPUs, since they offer 8
| general-purpose registers minus 2 typically used for stack
| pointer and base (ESP, EBP respectively). That leaves 6
| registers usable by compilers: EAX, EDX, ECX, EBX, ESI, EDI.
|
| To my knowledge, any other architecture targeted by GNU
| lightning offers more registers, so this magic number is
| effectively the largest number of registers one can offer where
| register allocation would become trivial for all architectures.
| tyingq wrote:
| There's both a COPYING file that has the GPLv3 in it, and a
| COPYING.LESSER file with the LGPL in it. I can't tell what's
| meant by that. Dual licensed? Or portions GPLv3 and portions
| LGPL?
| segfaultbuserr wrote:
| Since GPLv3, {L,}GPL have been "grand unified" to simplify
| things. Like code reuse, LGPLv3 is a just small patch on top of
| GPLv3 that says "given Condition X, you don't have to follow
| Clause Y in GPLv3", it's literally just a few short paragraphs.
| Without the original GPLv3 text, LGPLv3 is meaningless, like a
| running an executable without its shared library.
|
| Hence, although many people don't realize it, technically every
| project that licenses itself under LGPLv3 must contain a copy
| of GPLv3.
| elvis70 wrote:
| It's under LGPL. The LGPL allows you to relicense your work
| under the GPL, if you want.
| e12e wrote:
| Look at the actual files for copyright (or copyleft, if you
| will):
| http://git.savannah.gnu.org/cgit/lightning.git/tree/lib/ligh...
|
| Looks like the libraryis lgpl,I couldn't find licensefor the
| docs themselves (I assume they use document license
| (COPYING.DOC)).
|
| There seemsto be at least one example program that is gpl:
| http://git.savannah.gnu.org/cgit/lightning.git/tree/doc/body...
| tyingq wrote:
| Helpful, thanks. Feels like they should summarize it
| somewhere though, since GPLv3 and LGPL are wildly different.
| gspr wrote:
| The text of the LGPL relies on the text of the GPL.
| bonzini wrote:
| The LGPLv3 is implemented as a set of additional permission on
| top of the GPLv3. The library is LGPL.
___________________________________________________________________
(page generated 2021-04-08 23:00 UTC)