[HN Gopher] A statically typed scripting language that transpile...
___________________________________________________________________
A statically typed scripting language that transpiles to Posix sh
Author : TaKO8Ki
Score : 193 points
Date : 2022-03-25 13:54 UTC (9 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| bhussai20 wrote:
| Let me preface by saying this is pretty cool.
|
| I feel that this seems like a good idea on paper, and a bad idea
| in the wild.
|
| Last time I looked into shell syntax and such, I found out that
| most shells don't. They vary so greatly that very large context-
| free grammars have to be used to cover most of their context-
| sensitive rules. I don't think being POSIX compliant changes
| that.
|
| I suspect that if this got popular usage, the bugs would be quite
| numerous.
| goombacloud wrote:
| I also wonder if targeting bash which has things like pipefail
| and arrays could make things easier in the implementation side?
| remus wrote:
| I wonder whether there's a subset of shell syntax that's stable
| enough and powerful enough that it could be used as the
| compilation target whilst avoiding the incompatibilities you
| mention?
| tux1968 wrote:
| Would be nice if the language supported a #! bang line in order
| to make it a bit easier to execute, rather than manually typing
| "lic run X". Although of course on Linux you can always use
| https://en.wikipedia.org/wiki/Binfmt_misc without needing the
| bang line.
| perlgeek wrote:
| Nice, a language written a pretty obscure language (V) and using
| a pretty obscure build tool (z). I have a fondness for people
| using unusual tools for their hobby projects.
|
| POSIX shell is a good compilation target if you want to write a
| non-trivial build system, but be able to run it on some old or
| otherwise challenging systems.
| ducktective wrote:
| So it's APE (actually portable executable) but for POSIX systems
| which instead of binary uses scripting.
|
| nice
| nomemory wrote:
| Back in the day i had a similar idea, and i even started
| implementing it, but never had the resilience to finish. It waa
| too difficult for me at that time (probably even now). I am glad
| that finally someone had the courage to do it.
| JustLurking2022 wrote:
| Definitely a niche that needs a solution - cool to see one!
| beepbooptheory wrote:
| looks like the task runner part [1] has unfortunate name overlap
| with the family of programs that include z.lua[2] or zoxide [3]
| or z.sh [4].
|
| Not really that big of a deal, I just like tracking command line
| apps naming because it is an interesting exercise of being short,
| intuitive, and unique. When you decide a single letter is your
| name, surely you are going to run into this.
|
| 1. https://github.com/zakuro9715/z 2.
| https://github.com/skywind3000/z.lua 3.
| https://github.com/ajeetdsouza/zoxide 4.
| https://github.com/rupa/z
| throwaway894345 wrote:
| This seems neat. I love that there are examples front-and-center
| on the README page. This is exactly what I want and the first
| thing I would think to put when demonstrating a new language, but
| sadly this is rare.
|
| I'm not sure how I'd use something like this, but I've often
| found myself wanting simple statically typed scripting languages
| for generating YAML/HCL/etc configs. Dhall seems close, but
| config is something that every engineer has to touch and the
| burden of making everyone learn its syntax and functional
| programming idioms probably negates the advantages of using
| static typing. I've also looked into CUE, but it seems like it's
| largely neglecting the problem that I'm trying to solve DRY
| configs in favor of some problem I didn't know I had (unifying a
| bunch of layers of mostly-static configs?). This seems even worse
| than Dhall with respect to burden:reward ratio. I haven't yet
| checked out Nickel (the statically typed Nix language)--would be
| curious to hear from those who have.
|
| EDIT: just skimmed Nickel. It seems like it's trying to hew
| closely to Nix syntax and functional programming conventions,
| which suggests it probably shares many of the same problems as
| Dhall. Too bad. It seems like no one understands the assignment.
| sdevonoes wrote:
| What I miss in the README page is the output of the
| transpilation.
| JyB wrote:
| I guess the output is not really meant to be readable in any
| way
| em-bee wrote:
| if it transpiles to sh then the output should be an sh
| script. that should be at least somewhat readable.
| Brian_K_White wrote:
| It's possible to express many common features and
| functions of other languages in sh fairly well, so it's
| probably possible for a tool like this to output
| something not too bad.
|
| But even so, I would never expect any transpilation
| output to be more than technically readable.
|
| The part with the high level meaning is the part the
| human wrote. That's the entire point of any language.
|
| The output can not be considered source, regardless that
| it's possible to also write source in that same syntax.
|
| The only reason to even bother with transpiling instead
| of just requiring the necessary interpreter, is to ship
| something that the target already supports. It's very
| very little different than shipping a binary.
|
| The differences are just that a .sh is at least possible
| to read and hack vs not-possible. Not convenient or good,
| merely possible vs not-possible. And a .sh is more
| portable than a binary in that it doesn't care what the
| cpu or os is.
|
| If you want anything more high level meaningful instead
| of some 90's web page generator html output, you have to
| write that yourself. An automated tool cannot reverse
| engineer some code into an elegant expression of the
| original intent of that code. In that way a compiler or
| transpiler is not much different from a decompiler.
|
| Though, I admit there is room for levels. I don't know
| just how crude this tool's output is, but I know there is
| a range of what's possible, it does't _have_ to be
| ketralnis wrote:
| That's not a given, think typescript compiling to
| minified javascript. Some compilers like coffeescript do
| put in some effort to have the output be readable but I
| wouldn't assert that it should be a goal for everyone
| eyelidlessness wrote:
| > typescript compiling to minified javascript
|
| This is an odd example. TypeScript compiles to readable
| JavaScript for nearly everything except the odd runtime
| behavior (eg enums) and "downleveling", which is
| increasingly unnecessary. Minification is still a
| separate step AFAIK (if you're using tsc; obviously
| ESBuild and SWC are another story, as will be Bun).
| ketralnis wrote:
| You caught me, I've never written typescript. I don't
| think that changes my claim though
| xyzzy123 wrote:
| Yes although it might be readable in the same way that
| autotools configure scripts are readable -- technically
| you _can_ , but you probably don't want to, and if you
| _need_ to then something has gone badly wrong.
| xg15 wrote:
| I mean yes. Things do go wrong, especially with bleeding-
| edge projects such as this.
|
| You'll have to look at that output if you want to debug
| things, understand error messages.
|
| As such, I'd absolutely see it as a quality measure for a
| transpiler how readable the code it produces is.
| em-bee wrote:
| yeah, that's kind of what i meant. especially if the
| script has external commands you should be able to at
| least recognize something.
| [deleted]
| hnlmorg wrote:
| I think you can liken this to web assembly. The output
| might be text but it's not intended to be human readable.
| maggs wrote:
| I, too, missed that in the README. I was curious enough try
| it out in a vm. Here's the output for fib.li:
| https://termbin.com/pagk Or for something more complex,
| raytracing.sh: https://termbin.com/4rpd
|
| I was mostly wondering how/if it does any type checks for
| function input; it does not. Also raytracing.sh unfortunately
| fails to run on my system as it is.
| lakecresva wrote:
| The functional stuff in Nickel is really pretty vanilla, and
| the syntax seems much more inspired by ML than Nix. There's
| some info in the repo's `RATIONALE.md` that specifically
| answers "how is this different/better than
| Dhall/JSonnet/Cue/Starlark".
| em-bee wrote:
| _I love that there are examples front-and-center on the README
| page_
|
| except that those examples are to simple. for something that
| transpiles to sh i would like to use it to write better shell
| scripts. so i'd like to see examples that resemble actual shell
| scripts in that they call lots of external commands, do file
| operations, terminal interaction and other things commonly done
| with sh.
| smoe wrote:
| Felt the same. Most complete example seems to be the
| installer for the language which is written in it. This
| covers some of the things you mentioned
|
| https://github.com/cotowali/konryu/blob/main/konryu.li
|
| and the generated shell script
|
| https://raw.githubusercontent.com/cotowali/konryu/dist/konry.
| ..
|
| There is also the website, heavily under construction, but
| has some more documentation on the language than the readme
|
| https://cotowali.org/
| kazinator wrote:
| > _I 'm not sure how I'd use something like this._
|
| It seems that you could use this to program embedded systems
| that have a shell, yet which don't support any other usable
| language (due to flash space concerns, product management
| approval or whatever).
| mixedCase wrote:
| Alright, here's the hot take: The assignment is for any
| developer without basic knowledge of FP beyond .map and .filter
| to complete their intermediate-level programming education.
|
| All problem domains that involve declarative programming, such
| as configuration (Terraform, Kubernetes, Nix, many build
| systems), dynamic HTML, relational databases, and pretty much
| anything that is manipulating data in a pipeline fits
| functional programming like a glove and is hard to get right in
| a strictly traditionally imperative fashion. As an industry we
| should stop pretending FP isn't a necessary part of a
| professional programmer's education and that we can solve the
| problem by coming up with the umpteenth "we promise it's not
| FP" abstraction with its own take on fooling developers into
| writing functional code through an imperative-looking layer
| full of foot guns that subtly break imperative developer
| expectations, like React.
| Xavdidtheshadow wrote:
| Do you have resources to this end that you'd recommend?
| mixedCase wrote:
| The easiest hands-on approach I know of is to go through
| the Elm tutorial and build a simple CRUD app. The rest
| should flow from there.
|
| After that, https://fsharpforfunandprofit.com/series/why-
| use-fsharp/ and pretty much anything made by Scott Wlaschin
| is very accessible, and to the point.
| ModernMech wrote:
| Missed opportunity: fsharpforfunandfunds
| verall wrote:
| I think it's a super hot take because the real world has
| demonstrated (over and over again) that a huge portion of
| professional software developers have no interest in learning
| functional paradigms just to write some config files.
|
| Just like a huge portion of professional software developers
| have no interest in learning RTL or digital logic just to
| program an FPGA.
|
| HLS is a massively complex system to solve this problem.
| People will also build massively complex systems to solve the
| problem of "No FP here, please".
| mixedCase wrote:
| That'd be fine if it was just config files, but that's not
| the problem now isn't it?. It's an enormous amount of
| problem domains that FP handles trivially and is a complete
| pain to manage imperatively. We've had entire groups of
| people coming up with ridiculous ideas like replacing
| relational databases with key-value stores for relational
| data just because they find SQL "icky" because it fits a
| mental model they never developed (syntax issues with SQL
| aside) and it isn't until they're hand rolling data
| aggregation that they realize the mess they got themselves
| into.
|
| Which is why I stand by what I said: this knowledge is not
| optional, and engineers not willing to learn them are going
| to be stuck with sub-optimal tools, dragging the whole
| industry down with them. While I can't be confident this is
| the definitive solution, I believe that the best way to
| move forward is to start treating basic FP (purity, data
| transformation pipelines, immutability, favoring data
| structures over code) as a baseline that everyone needs to
| know to work as a developer, and stop treating it as if it
| is on a superior level of difficulty compared to learning
| whatever bullshit combination of half-templated server side
| rendering, 4D DOMs and incantations needed to upgrade
| Webpack we need to know to be "up to speed" nowadays.
| klekticist wrote:
| I agree with your point here, though imo it's worth drawing
| a distinction between the FP concepts the language
| designers / implementors must understand and the FP
| concepts the language users must understand.
|
| Ideally languages prevent bug classes like null deref, use
| after free while still being familiar enough for most
| programmers to use.
|
| It's analogous to the spectrum of SQL knowledge folks have.
| For instance knowing what a group by does vs how a group by
| reshuffles data on a single node / across nodes. Ideally
| the language design hides the hard details.
|
| A very tall order admittedly.
| staticassertion wrote:
| > (over and over again)
|
| Eh. I'm not saying one way or the other that FP is good or
| bad, but the software industry is incredibly young and
| changes a lot. Even in a relatively short period - 10
| years, let's say - things are pretty difference. 20 years,
| still a short time, and things are radically different.
|
| And on top of that, it is still the case that a lot of
| stuff takes a long time to get adopted. It's hard to build
| and popularize a programming language, regardless of how
| good or bad that language is, because they tend to lack
| interoperability with other languages if they're very
| different, which is usually a requirement if they want to
| be considered worth looking at.
|
| So I'd just as easily buy that people haven't used a lot of
| FP due to inertia, a lack of the right external inventives,
| etc, and not by any virtue of the paradigm.
| Willamin wrote:
| > simple statically typed scripting languages for generating
| YAML/HCL/etc configs
|
| Do you actually want a statically typed _imperative_ language
| for generating configs?
|
| Or do you want a sane, statically typed, declarative language
| for configs?
| kirbyfan64sos wrote:
| You might find Cue interesting: https://cuelang.org/
|
| It's a bit overambitious IMO, but it tries taking a relatively
| different approach to the problem that I find to be quite
| interesting.
| sidpatil wrote:
| Parent already mentioned their investigation of CUE.
| smasher164 wrote:
| Neat!
|
| A really cool enhancement to this would be if the compiler could
| target a polyglot script that is both Shell and Batch. For
| example: https://stackoverflow.com/a/17623721
|
| Then you'd get a single script that would automatically be
| portable and executable on pretty much every conceivable
| platform.
| jokethrowaway wrote:
| Why not just build a binary? What's the benefit of transpiling to
| sh?
| zakuro wrote:
| Binary files are different for each architecture, but shell
| script is not so. Most scripting languages require a
| interpreter to be installed. Output of Cotowali is shell
| script. so no need to install any interpreter.
| westurner wrote:
| Could this be done with LLVM?
| neoyagami wrote:
| nice. I do a lot of bash/sh things when need to automate some
| task, now they can be programmed in a propper lenguaje( sh can be
| painful sometimes) and just transpile it to sh
| kazinator wrote:
| There are no examples which show that the source language is
| good for shell scripting tasks. Making shell programming tasks
| nicer doesn't appear to be a goal.
| zakuro wrote:
| Although not yet in the sample, Cotwali is also intended to
| use for shell programming tasks.
| [deleted]
| Brian_K_White wrote:
| I see 2 possible rationales to transpile to posix sh:
|
| 1: To remove bashisms/kshisms/zshisms to create output that works
| on ash/dash/busybox/xenix/sun/... In which case, the more useful
| source language is bash or ksh etc, not a new invented one.
|
| 2: To generate a portable output like above from a better
| language than any *sh (*). In which case the source language
| should be something already common like python or perl or ruby
| etc (even if just a subset).
|
| In either of those two use cases, I don't think it makes a great
| deal of sense to make a new language, and especially not to write
| the compiler for it in V of all things.
|
| That relegates the whole project to being just a curiosity or a
| purely academic excercise.
|
| (*) I use every trick bash and ksh offer, so I know it's possible
| to do all kinds of stuff in some *sh's, but a lot of the features
| in them are encoded into practically inscrutable special syntax
| and rules and tricks, where more purpose built programming
| languages make all the features more explicit and spelled out ->
| readable. bash is a powerful enough language, but not a readable
| one, except for rairly stupid scripts that either don't do much,
| or do everything the hard way.
| maxbond wrote:
| What about, to gain saner semantics (static typing) while
| retaining the portability and versatility of shell scripts?
| Brian_K_White wrote:
| "What about, to gain saner semantics (static typing) while
| retaining the portability and versatility of shell scripts?"
|
| I think I was insufficiently critical in a misguided
| intention to be nice. Let me correct that a little :)
|
| If the need is great enough for a DSL, then the need is great
| enough for python or c or any of 100 other fully fledged
| universally available languages, and either require it's
| interpreter or ship a binary.
|
| However annoying it may be to require a ruby interpreter or
| something, there is essentially no case where the most
| sensible answer is this "Vala-for-sh".
|
| It's the worst of all worlds. Taking the whole life cycle
| into account, it's not saner or portable or versatile, it
| reduces all of those.
| chalst wrote:
| Typical shell programming idioms are quite dynamic. You can
| give saner semantics, allowing a fair amount of static
| checking, without abandoning this dynamicity.
|
| What Andy Chu has been doing with Oil Shell is promising in
| this respect. It supports optional type annotations. Cf.
| https://www.oilshell.org/blog/2021/01/why-a-new-shell.html
| [deleted]
| nerdponx wrote:
| I'm surprised to see that the compiler is written in V
| (https://vlang.io/). Last I heard, V was vaporware, and a lot of
| its claims about performance etc. ended up being "todo" items
| rather than facts about the language as it actually existed. But
| that was several years ago; has anything changed?
|
| Otherwise, fun project! I could see this actually being useful in
| some niche cases, e.g. generating scripts that need to work on
| some exotic architecture that has a POSIX Sh implementation.
|
| It might also be useful for writing interactive shell "plugins"
| that set up things like a fancy PS1 prompt.
| silisili wrote:
| V is a beautiful language, but got tons of flak for being
| overambitious if you're generous, a scam if you're being
| suspicious. They've been called both on this very site.
|
| I follow it, and love their ambition...but there are a ton of
| open issues that convince me it's not even alpha quality yet.
| If anyone has experience otherwise, would love to hear.
| yisonPylkita wrote:
| I would love to hear more about that. Does someone here have an
| experience with V lang and can tell us if its claims are at
| least close to truth?
| camgunz wrote:
| This was my impression too and it looks like they've made
| really impressive progress since I last saw them on HN. Their
| bug list [0] looks kind of scary, but that's maybe par for
| the course for a language? I remember the bad old days of
| Go's 32-bit GC not being wonderful for example.
|
| [0]: https://github.com/vlang/v/issues?q=is%3Aissue+is%3Aopen
| +lab...
| rurban wrote:
| looks very much like Rust's issue tracker to me
| johnisgood wrote:
| Agreed! Also... I was just going through smhasher. Thank
| you for it!
| KerrAvon wrote:
| Maybe? Some of those sound like unit tests should have
| caught them.
| jokethrowaway wrote:
| I generally prefer h https://h.christine.website/
| spacemanmatt wrote:
| h is the core framework language of my global
| infrastructure
| codeflo wrote:
| What's the main use case? I guess build scripts?
| zakuro wrote:
| Build scripts are one of the assumed use cases. Build scripts
| are one of the assumed use cases. The other use case is
| installation scripts. cotowali's installer is written in
| cotowali itself. https://github.com/cotowali/konryu
| metadat wrote:
| Awesome idea, but "cotowali" is a truly terrible gibberish
| sounding name, maybe even worse than klubernetes.
| adriangrigore wrote:
| Might as well check out my POSIX sh static site generator
| https://mkws.sh/ if you're into this stuff.
| Merad wrote:
| Neat idea, but it seems to me like a critical part of making a
| new language of any kind is documenting it so that others can
| learn it and use it. On that note, can anyone explain the syntax
| that seems to use pipes in a function declaration (i.e. every
| function in the readme example other than fib)?
| mbarbar wrote:
| It appears to me those are the types the functions expect to
| consume and output when used in a pipeline.
| norswap wrote:
| It's pretty nice! I'd love to see the compiled output of the
| examples though!
| Shadonototra wrote:
| V is such a nice and simple language, it's Go for system
| programming
| nixpulvis wrote:
| Unless I'm misinterpreting the example, it seems rather
| unfortunate that you need to create a sentential value before a
| `read` call to update some `&n` location. The `twice` function
| really could be just `return read() * 2` no?
|
| I guess I should look into how this is actually compiled to sh. I
| agree the README would benefit from adding some examples of this.
| reincarnate0x14 wrote:
| This is amazingly great for how simple the idea is, going to have
| to play with the output a bit to poke around for strange edge
| cases (it is sh, after all) but given all the random devices out
| there with Busybox or similar this could end up being
| significantly useful for configuration management, etc.
|
| Also seeing quick, readable examples right on the github landing
| page is such a small but wonderful thing.
| dogprez wrote:
| Wouldn't it just be better to make a new shell that has this
| syntax?
| hnlmorg wrote:
| There's already a plethora of shells that fit that domain:
| + Nu Shell + NGS + Powershell + Oil Shell
| + Elvish + Murex (which is one I'd written)
|
| Not to mention all the LISP, Python, and other such high level
| language REPLs come shells.
|
| But they solve a different problem. This seems to be more like
| what Typescript is to Javascript: a alternative "safer"
| language where your end point is limited to supporting a
| different language. So you need to compile down to a known
| portable base language.
| eyelidlessness wrote:
| Better in that you can use it directly, not necessarily great
| for interop. I love Fish shell, but it can be a pain when
| trying to use tools that expect posix syntax to work.
| hnlmorg wrote:
| Wow this is a great advert for V (the programming language it was
| written in). Having never written a line of V before I can still
| read and understand the code.
| IggleSniggle wrote:
| I had exactly the same reaction. Extremely digestible syntax.
| Didn't even notice until after looking it over that it had been
| written in a language I don't know.
| jpgvm wrote:
| Would appear that V isn't as vaporware as I initially thought.
|
| Going to give this a fairly through once-over because I think
| Posix sh is an under-appreciated transpilation target.
___________________________________________________________________
(page generated 2022-03-25 23:00 UTC)