[HN Gopher] Crush: A command line shell that is also a powerful ...
___________________________________________________________________
Crush: A command line shell that is also a powerful modern
programming language
Author : robin_reala
Score : 364 points
Date : 2020-08-07 06:35 UTC (16 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| niek_pas wrote:
| Has anyone got this to compile on macOS?
| sylr wrote:
| Nope, error: aborting due to 3 previous errors; 31 warnings
| emitted
| pritambarhate wrote:
| Yes, not working for me too. Ran the
|
| curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
|
| command also which upgraded by Rustc to rustc 1.45.2 (d3fb005a3
| 2020-07-31).
|
| This is one problem I have faced with Rust quite frequently.
| Try to clone and run a random (not famous) project from Github
| and it doesn't compile.
|
| Love Java when it comes to this. It's easy to see which version
| of the JDK is needed from the Maven file. If you have that
| version of JDK, the projects almost always run.
| snarfy wrote:
| The main innovation I see is tokenization into columns instead of
| character streams. If I want to do that in a traditional shell, I
| use awk.
| PaulHoule wrote:
| Looks like a unix-ish take on PowerShell.
|
| I love the idea behind PowerShell, but the verbosity turns me
| off.
|
| PowerShell, of course, has the pipelines but also COM
| integration, which is a big deal for Windows (you can script "the
| desktop" via COM; for instance you could create a new instance of
| Microsoft Word and call objects inside Word to do something to a
| document.)
|
| Unix has dbus and some similar things, but nothing quite so
| critical as COM is to Windows.
| AnIdiotOnTheNet wrote:
| The verbosity is one of PowerShell's best features. There is so
| much less guess work about what a command or argument does, or
| whether it is '-a' or '-A', or any of that garbage. You can
| usually guess how to do things because there's a reasonable
| consistency to the Verb-Noun structure. The extra typing is
| mitigated by everything being tab-complete including commands,
| arguments, and even variables. Not to mention you can just
| alias commonly used commands into shorter versions anyway!
|
| Did I mention it also makes readings scripts at least 100 times
| easier?
| AnIdiotOnTheNet wrote:
| I really want something like this as a replacement for sh in tiny
| busybox-backed use-cases, so I find it really annoying that
| developers insist I install rust and compile it instead of
| offering a statically compiled binary I can use.
|
| And if it can't do that because of a reliance on shared libs,
| then that's even worse!
| geophile wrote:
| The improved-shell-piping-objects space is getting crowded. My
| entry is marcel: https://marceltheshell.org.
|
| Trying to create a combined shell/programming language is a
| mistake, in my opinion. Marcel exposes bits of python on the
| command line, e.g. for writing predicates. For more serious
| programming, marcel offers a Python API that allows the use of
| shell commands inside Python, neatly integrated into the
| language. For example you can write a series of commands,
| connected by pipes, and the result defines a Python iterator so
| that you can use the pipeline results in a for loop.
|
| See the website for more details and discussions.
| black3r wrote:
| if the only problem with PowerShell is that it feels clunky and
| annoying, why not try to extend/improve it (like Kotlin over
| Java)? Why start a new project that will maybe catch up in X
| years?
| mehrdadn wrote:
| I can imagine lots of reasons. To build on PowerShell means
| either integrating your changes upstream (not likely, given
| fixing some of the clunkiness would require breaking changes)
| or maintaining a separate fork and incorporating patches from
| upstream constantly (a lot of work, and constant conflicts
| aren't fun).
|
| Plus the fact that PowerShell is .NET-based and not the most
| snappy shell out there either.
|
| Plus the fact that extending something requires investing a lot
| of work just understanding how things tie together, and then
| avoiding messing up all the hacks and idiosyncrasies added
| along the way instead of having a clean slate to build on.
| kanobo wrote:
| I'm not the author but it's probably for the simple reason that
| writing something from scratch without dependencies can be
| extremely rewarding and fun.
| kevin_thibedeau wrote:
| PowerShell is tied to the CLR APIs. You shouldn't have to run a
| Mono instance just to get basic shell behavior.
| pcr910303 wrote:
| While I do like the new wave of the structured shells(elvish,
| uxy, nushell, crush, etc...), I don't think anything will
| actually succeed unless there's a big change in the operating
| system.
|
| (I'm not suggesting these aren't valuable - these are valuable
| experiments, but I'm saying for them to succeed, a new OS is
| needed.)
|
| The native OS doesn't really embrace structured communication,
| and so commands will still have to treat the structured shell as
| a second-level citizen.
|
| In an ideal world, for the structured shell to work, 'npm list'
| should return a table (or whatever the native data structure is)
| and you should be able to do 'npm list | where {deduped==false}'.
| In the real world, 'npm list' won't support the structured shell,
| so you would be writing 'npm list --json | jq '<obscure
| commands>' | json:from | where {deduped==false}'. And then 'npm
| list | grep -v 'deduped' is more simpler, so you don't really
| feel the advantages of the structured shell.
|
| I think for this to be solved, the OS's primary communication
| system should be structured, so you can guarantee that every
| command will be a table. Then the programmers for the tools of
| that OS will output a table, and the users will be able to take
| advantage of this. Until then... it's basically a fun experiment.
| gbear0 wrote:
| I keep having the same thoughts, but I'm not sure it needs to
| be a full OS solution. Also a related problem that always
| frustrates me with input is what do I use for the help command?
| Is it '-h', '-help', '--help', or just 'help', or even man
| pages only. Each script/bin has its own way of doing things and
| this seems like something that should be standardized, but I
| realize there's no great way to do that currently when its just
| input to the program.
|
| I wonder if instead of an OS solution we can just add more
| metadata to the actual binary files we compile. For example, if
| we have a new ELF format that has an optional list of commands,
| options, full input/output types, etc, any one of the shells
| can start to provide better tooling. Output format types can
| then be passed along with the stream, and potentially checked
| that they work as the proper inputs for another command. Let
| the shell auto generate the simple cli help docs, or generate
| the bash completions, or figure out how to better convert data
| to desired types.
| gregmac wrote:
| Something relatively simple would be borrowing the idea of the
| HTTP "accepts" header, and standardizing on an environment
| variable. For example:
| STDOUT_ACCEPT=application/json,application/csv,text/plain
|
| I'm not sure how the application could easily communicate back
| the understanding of this, though -- basically the equivalent
| of HTTP Content-Type header. It would need to be done in a way
| that doesn't break other applications that aren't expecting it,
| perhaps such as being sent back on STDERR with an escape
| sequence of sorts?
|
| The other trick is knowing what an application will accept via
| STDIN - ideally at runtime, but before actually writing
| anything.
|
| If the shell understands all this, it could automatically
| format JSON/csv output for display, integrate jq for
| formatting, or even convert between formats (eg: when one app
| writes in yaml, the other understands JSON input).
| throwaway894345 wrote:
| I don't think you need an OS to pass structured objects, but
| rather simply a runtime that supports it, a la dotnet.
| Basically anything that can allow different programs to run and
| share memory as well as information about the shape of the
| objects in that shared memory. This can run on existing OSes
| and in the case of powershell, it already does. I think Go
| could probably do this as well via its plugin interface and
| runtime type system except that you aren't allowed to unload
| plugins (so every program you ever run hangs out in memory).
| hhas01 wrote:
| What you want is tagged and/or typed pipes. Once a pipe
| declares how its data is encoded, you can build all kinds of
| modern tooling and high-level automation on top of that.
|
| The whole "But everything's a text file!" is an outrageous lie,
| and always has been: just an excuse to cover up a massive
| omission in the design of the Unix file system (untyped,
| untagged "file" resources). Such egregious corner-cutting might
| have been understandable--even forgivable--back in K&R's day,
| when they were bootstrapping C and Unix on hardware with less
| processing power than a modern microwave oven controller. But
| we're decades on from that now and there's no excuse for
| perpetuating it: we _know_ how to build safe powerful modern
| static and dynamic type systems, and we're spoiled rotten for
| the hardware resources to support it.
|
| Fifty years ago, K&R built C and Unix, in a cave, with scraps.
| Nowadays you can bootstrap a complete hardware stack for less
| than the cost of a 360, and there are modern kernels (L4) and
| systems language (Rust) already available for free off the
| shelf. So pick a target market and build the userland for that.
| All that's missing is a modern-day Kernighan and Ritchie to
| step up and get on with it, so it'd be a sorry indictment of
| this generation if they do not exist.
| gen220 wrote:
| > untyped, untagged "file" resources
|
| I think the problem then, and the problem today, is that
| these "types" are a very very mobile target. How many
| implementations of JSON are there? XML? HTML? CSV?
|
| It doesn't make sense to put these types in the operating
| system, or else you'll start TypeError's every time you
| download anything. Not to mention the Computational costs of
| parsing the entire structure of a document every time you
| edit it to verify that it's "still of type X".
|
| I'm not at all sold on the idea that categorizing files and
| streams in this way is a step forward (in the shell, at
| least). This is what applications are for: to define their
| own formats, parsers, and generators. The point of the
| operating system is not to be one of these applications, but
| to facilitate their development and use. IMO, typed files and
| streams would actually impede more than enable this work.
| hhas01 wrote:
| "It doesn't make sense to put these types in the operating
| system"
|
| The OS doesn't need to hardcode individual types, and to do
| so would defeat growth. It only has to provide a standard
| mechanism by which files can declare the type of data they
| contain, and executables can declare the type(s) of data
| they consume and/or produce. BeOS, for instance, tagged
| files with MIME types. The original Mac OS used "four-char
| codes". Even DOS had file name extensions, as awful as that
| particular design choice was.
|
| Storing arbitrary data with no way to indicate how that
| data is encoded is rank insanity. Saying "it's the app's
| job to deal with encodings" is not wrong, but if apps have
| no way to know what encoding a given file was written in,
| how can they know how to decode it?
|
| Please do not rationalize away the longstanding and well-
| known deficiencies of Unix. These deficiencies do not aid
| progress; they hinder it. As do apologetics.
| gen220 wrote:
| > files can declare the type of data they contain [...]
| executables can declare the type(s) of data they consume
| and/or produce.
|
| I think you're conceding that the OS would not be
| responsible for __enforcing __data 's compliance with
| it's declared type.
|
| I'd be OK with this world, but I am not convinced that
| this hypothetical OS is _substantially_ better than the
| UNIX descendants we have today.
|
| > Storing arbitrary data with no way to indicate how that
| data is encoded is rank insanity.
|
| I think we all can agree on this... it's just a matter of
| how we encode the data type. UNIX descendants follow the
| pattern of encoding data types with filename endings.
| While I agree that this approach _feels_ primitive, it 's
| hard to argue that it doesn't work, because it does. This
| is fundamentally identical to MIME-types, imo, because I
| believe extensions (and mime-types!) are fundamentally
| "names", and don't have much semantic above that, given
| the diversity of "name"-implementations, as stated
| earlier.
|
| I don't believe that we gain a lot of practical value
| from compelling executables to register their accepted
| "names" with the Operating System: I'm trying to imagine
| what big value this gives you, beyond transforming
| `some_program --out_type=json | jq` to `some_program |
| jq`.
|
| I hope you understand I'm not trying to apologize for
| UNIX. I agree that many things about it should be better.
| But I _do_ think they got "everything is plaintext, from
| the OS's point of view" right.
|
| (I think it's good discussion and I want to learn from
| it)
| rootlocus wrote:
| > I think for this to be solved, the OS's primary communication
| system should be structured
|
| You mean the STDOUT format of all processes? I disagree. `npm
| list --json` already does what you want. Your only argument is
| that ` | jq '<obscure commands>' | json:from | where
| {deduped==false}` is too obscure. The solution is (probably) a
| simplified `jq` that has the query format you want. Problem
| solved. Keep it simple.
| smitty1e wrote:
| I kinda like the mystical feeling of jq.
|
| As with haskell, it'll feel as though I've arrived somehow,
| should I achieve real understanding.
| frou_dh wrote:
| To me it's a good example of Opportunity Cost. We could
| spend the time and effort memorising jq's special snowflake
| language, but that time and effort would almost certainly
| be better spent on something else. I deleted it a while
| back. Can't be bothered with it.
| klibertp wrote:
| > would almost certainly be better spent on something
| else.
|
| I'd would suggest to spend that time on learning how to
| effectively learn new languages. There's more than one
| way, and I guarantee that afterwards you'll conquer any
| given "snowflake" in no time at all.
| frou_dh wrote:
| I deliberately wrote "memorise" rather than learn in the
| usual sense, because there's a higher than normal degree
| of familiarity needed to get working code flowing from
| the fingers, when the venue is between a pair of single
| quotes in a shell.
| [deleted]
| gen220 wrote:
| I usually use jq just to format the blob, and then some
| unholy combination of grep -A/-B, tr, and sed to get the
| output I want. It's usually faster (for me) than
| relearning jq's query language.
| knome wrote:
| >jq's special snowflake language
|
| If you'll bother condescending to such, you'll find one
| of the most interesting languages I've encountered in
| years. jq's syntax and execution model is beautiful. It
| has some rough edges, leave no doubt. Some function
| names, date handling, and the obtuseness of some of the
| errors it returns can be frustrating. But its data
| streaming layers of nested closures are an extremely
| terse, readable, and powerful way of expressing
| operations over structured data.
| dkarl wrote:
| This is a very tempting feeling, but when I see someone
| else who feels this way about something that I _do_
| understand, I can see that it doesn 't serve them. In a
| tool that you use every day, you should be able to quickly
| and confidently demystify anything, or you're not really in
| command of the tool. Your temporary uncertainty should
| "bottom out" on something you don't have to think about
| further.
|
| There are plenty of mysteries "out there" in the problem
| domain; you don't need them in your tools.
| schemy wrote:
| The problem, as always, is that a table isn't the natural
| output for a computer. It's either serialized text, or a graph.
| Unix tools work ok-ish for serialized text and you need general
| recursive functions to deal with graphs. Something that all
| shell like languages are extremely bad at.
| hughw wrote:
| Microsoft went far down this road in the early 2000s with WinFS
| [1] but abandoned it. [Edit: WinFS and Longhorn together]
|
| [1] https://en.wikipedia.org/wiki/WinFS
| cnity wrote:
| It doesn't have to be this black and white, in my opinion. The
| most common data structures I've interacted with on the command
| line are _newline separated_ or _JSON_. What if your shell
| allowed parsing of JSON natively?
|
| npm list --json | ({ json }) => json.somePackage.author.name
|
| What if it allowed parsing of XML natively, allowing instant
| scrapers to be written in a single command?
|
| curl www.example.com | ({ xml }) =>
| xml.find('.importantInformation') > scraped_info.txt
| mywittyname wrote:
| jq makes handling json in existing bash scripts simple. It
| can even do some simple data wrangling via built-in functions
| like map. It's not a universally standard command yet, but
| it's available for most modern systems.
|
| I'm not sure about similar xml cli tools, but they probably
| exist.
|
| Point is, both of these should be cli tools, not shell built-
| ins.
| mdaniel wrote:
| > I'm not sure about similar xml cli tools, but they
| probably exist.
|
| I :heart: xmlstarlet (sometimes installed as that name,
| sometimes as just "xml"):
| https://formulae.brew.sh/formula/xmlstarlet#default =>
| https://xmlstar.sourceforge.io/
|
| The xsltproc that ships with libxslt1 is also pretty handy,
| although more verbose
|
| While not quite xml, pup is super handy for doing quick
| html selections:
| https://formulae.brew.sh/formula/pup#default =>
| https://github.com/EricChiang/pup#readme
| jodrellblank wrote:
| Newline separated _what_? Basically all Linux config files
| are newline separated lines, but that won't get you the
| equals separated data pairs, indented blocks, pretend-xml
| sections, ad-hoc boolean representations, INI-style sections,
| ad-hoc list-of-values style data out of them.
|
| > What if it allowed parsing of XML natively, allowing
| instant scrapers to be written in a single command?
|
| Powershell's Invoke-RestMethod parses xml natively to a
| structured output; it's useful but not enough to make XML fun
| or trivial to work with beyond root.node.foo.title simple
| scraping.
| cnity wrote:
| > Newline separated what?
|
| What I meant by this is that the newline character is the
| primary delimiter of the data, but I take your point. IMO,
| the best way to tackle the myriad ways of structuring data
| is to write parsers for them, not to rewrite the programs
| themselves to work with your shell's idea of structured
| data. That's a losing battle.
| jodrellblank wrote:
| This is how I felt powershell ought to go on Linux -
| parsers from every config format to a common intermediate
| and back, argument completers for every common shell
| command to a standard intermediate representation.
|
| I think that's a losing battle as well due to the brittle
| nature, any tweak to any command output could break
| something, meaning endless maintenance work on a large
| scale.
|
| Rewriting concepts into new commands at least bring
| consistency and a chance of shrinking maintenance work as
| it settles on a nice design - and if not the maintenance
| won't be endless rebuilds of the same parsers.
| AnIdiotOnTheNet wrote:
| Interesting note: newlines are perfectly valid characters
| in file names. In fact, the only byte that cannot be in a
| file name is '/'[0], which means that you're asking for
| trouble trying to parse the in-band signaling you have to
| do with a stream that's pretending to be structured.
|
| [0] with a hex editor it is possible to create a file
| system entry using '/' in its name. Linux does not handle
| this situation with grace.
| trenchgun wrote:
| Ooh that would be pretty damn sweet tbh.
| guerrilla wrote:
| Yeah this is probably it right here. Just being able to parse
| various common formats would give a lot of the structure
| people are looking for. The builtins can just operate on
| that. Maybe it could optionally detect the formats too. If
| such a shell became popular, it would also incentivize
| developers adding common formats as outputs to more programs
| too.
| dvh wrote:
| https://github.com/hikikomori82/StructurOS
| david_draco wrote:
| > The native OS doesn't really embrace structured communication
|
| OS system calls are highly structured, down to the argument
| position and data type.
| smitty1e wrote:
| Right, but the unix philosophy is that everything consumes
| and emits text.
|
| Having something more than a blob is great.
|
| Of course, there will be a performance tax.
| intelleak wrote:
| Somebody just give me a rust REPL already (that is not the slow
| as snails evcxr)
| cbreezyyall wrote:
| Doesn't AWK already do all of this?
| varbhat wrote:
| Your shell is not for me. Here's why.
|
| 1) I don't want shell to be full-blown high level programming
| language. POSIX, Bash scripts/languages are enough for my shell
| uses.
|
| 2) For anything high level or for tasks shell would become
| complex, We do have dozens of fully matured languages. I would
| rather use those instead of this/any shell script.
|
| 3) Features desirable for shell include syntax highlighting,
| autosuggestions,command-not-found helper,tab autocomplete, flag
| complete. My shell already has it.
|
| 4) I like bash and zsh as shell. Bash is simple,i like it,but it
| doesn't have few features that i mentioned. So,I don't use bash
| as my main shell. zsh also is compatible with most bash/POSIX and
| it has all tte features that i mentioned in (3).fish also has
| these features but it's syntax is different which i don't tend to
| use. So,i don't use fish. zsh has all features that i like in
| fish too(mentioned in (3)) .
| rsa25519 wrote:
| > Your shell is not for me.
|
| And that's okay. Personally, I'm thrilled by the idea of a new
| shell. I don't plan to use Crush in production (yet?!) but I
| definitely plan on installing it on my laptop.
|
| I'm excited to see how things go. I understand, for a while, it
| won't have all the bells and whistles of zsh. But that's okay.
| Everything has to start somewhere. I remember back when
| alacritty didn't even have easy scrollback!
|
| Best of luck to Crosh and its author, liljencrantz!
| mywittyname wrote:
| I'm curious if you've taken the time to learn Powershell?
|
| It's a supercharged version of bash that works with Windows,
| but instead of piping character-only data, it can pipe
| structured data between commands in the form of objects. The
| reason I ask is because, while it adds a ton of flexibility
| to shell scripting (to the point where I consider it to be a
| language), it is also quite complex.
|
| The benefit of shell scripting is that it is simple enough to
| have as a tertiary language that you can pickup and use after
| a six month hiatus. Once you get into more powerful language
| territory, there are a lot more components to memorize and it
| becomes difficult (for me at least) to keep those idioms in
| my head if they go unused for months.
| BeetleB wrote:
| > The benefit of shell scripting is that it is simple
| enough to have as a tertiary language that you can pickup
| and use after a six month hiatus.
|
| I definitely did not find that to be the case. I've learned
| and relearned shell scripting a bunch of times. I always
| forget it and need to relearn it. Worse than perl.
| ravi-delia wrote:
| Every time I learn more about Powershell I get more jealous
| of Windows users. Not so jealous I'd switch back, but _man_
| would I like some of those features on Linux. I know it 's
| ported, but it just doesn't have the tooling on Linux and
| I'll be damned if my standard scripting language doesn't
| run natively.
| asimilator wrote:
| > The benefit of shell scripting is that it is simple
| enough to have as a tertiary language that you can pickup
| and use after a six month hiatus.
|
| I've been using/abusing traditional shell scripting
| languages for over a decade now and I still struggle with
| the rough edges (especially quoting/escaping/argument
| passing).
| Nursie wrote:
| > Best of luck to Crosh and its author, liljencrantz!
|
| Crosh is the ChromeOS shell!
| rsa25519 wrote:
| Oh no! Too late to edit :-(
| kanobo wrote:
| A better PowerShell? I'll take it! What's the story behind the
| name 'Crush'?
| cies wrote:
| cRUSh shares a few letters with Rust (the prog lang of the
| project). The author made Fish before, a new shell+lang in C++.
| sitkack wrote:
| crush> ls | where {type == "directory"}
|
| Thank you!
| [deleted]
| betimsl wrote:
| What is wrong with: ls -l | grep '^d'
| quesera wrote:
| Outputs are not equivalent. But this will work:
| find . -type d
| smichel17 wrote:
| I've read most of the comments in this thread and will read
| TFA, but haven't yet.
|
| What's always bothered me about piping to grep is losing the
| original syntax highlighting. Particularly annoying when the
| first command, whose highlighting I want to preserve, is
| itself grep; this leads to awkward contortions like
| git grep "a" | grep -v "b" | grep "a"
| geodel wrote:
| It is minimal, so does not meet the modern aesthetics of
| computer world.
| Riverheart wrote:
| Until you want something more specific like listing
| absolute paths of those directories without the metadata
|
| ls -l | grep '^d' | awk '{print $9}' | xargs realpath
|
| Or
|
| ls -ld "$PWD"/* | grep '^d' | awk '{print $9}'
|
| Guessing Crush could just add another property called
| Fullpath you can echo in the next pipe after filtering.
|
| ls | where {type == "directory"} | select %Fullpath
|
| Maybe he adds a --dir flag
|
| ls --dir | select %Fullpath
| blame_lewis wrote:
| This looks absolutely brilliant.
| tomerbd wrote:
| for the people who use powershell on mac, and as someone who used
| neither, which one would you recommend to integrate into my
| workflow powershell or crush?
| viraptor wrote:
| Crush seems very alpha stage right now (no terminal support),
| or for actual usage PowerShell may be your only current option.
| qchris wrote:
| This looks to be in a similar vein to the Ion shell, which was
| developed for Redox [1]. Both written in Rust, both not
| (seemingly) perfectly POSIX-compatible, etc. I'm not sure I'd
| personally choose to go with either one over shell if I knew
| anyone else was going to need to use it, but it's cool to see
| iteration and experimentation on things that I usually consider
| to be the fundamentals of the computing environment.
|
| [1] https://github.com/redox-os/ion
| HelloNurse wrote:
| Publishing a project at this level of immaturity doesn't do it
| any good. While it definitely "works", it works as an experiment
| about techniques to write an interactive command interpreter in
| Rust, not as a shell for people to use.
|
| Currently there is no documentation beyond the introductory
| GitHub page. But this isn't a fork of sh that only needs
| documentation of the minor ways in which it departs from POSIX
| compliance: there is an unknown set of internal commands, each
| with a complex table format for its output (it is necessary to
| know all relevant column names to do anything), which depends on
| a large set of non-text types, which have complex object-oriented
| behaviour, with "serialization" to a variety of file formats,
| with an unspecified script and command line syntax.
|
| Such complex features, on par with serious frameworks in full-
| fledged programming languages, need to be defined and documented
| very thoroughly, or nobody will understand, trust and use them.
| Unfortunately, there are many hints that not only documentation
| is vastly behind schedule, but that important aspects and
| practical considerations haven't been addressed.
|
| For example: how can I quote file names containing nasty
| characters? How can I use environment variables? What is my home
| directory? Why is executing an executable program difficult, with
| complex and unclear mechanisms that mess with the command line?
| Given a very simple and very portable traditional shell script
| (e.g. ps -eaf | grep foo | sort) how can I do the same in Crush
| with minimal effort? If I load a very large file, does the shell
| choke attempting to load it all to memory, or does it process it
| in a proper streaming fashion?
|
| Other issues are more fundamental. How can I perform relational
| queries between multiple tables, e.g. a join between ls output
| and a file listing users? If I can't, what can I do with tables
| that cannot be done more easily and flexibly with awk, jq, and
| other established tools?
| appleflaxen wrote:
| I have zero interest in this project, but disagree that it
| "doesn't do any good" to share. People with similar
| inclinations and interests can collaborate on software that
| isn't ready, but that scratches their itch.
|
| Theres nothing wrong with that, as long as you don't create
| unrealistic expectations.
| HelloNurse wrote:
| The problem with novel shells, compared with other kinds of
| itch to scratch, is that existing shells have had about 50
| years of evolution: doing better in a few months of work is
| quite unlikely.
|
| A new shell must be similar to existing ones for
| compatibility with an enormous ecosystem of command line
| tools that work well in a typical shell; it must do almost
| everything that traditional shells do equally well (this
| includes documentation; documentation is non-negotiable) but
| it must also have some compelling advantage.
|
| The right way to collaborate on designing a shell is to
| discuss possible designs of command syntax and behaviour,
| then if some promising idea comes out of the brainstorming
| refine it to a full specification, then if the idea survives
| (without mutating into a less ambitious scripting language,
| or a library for an existing language or shell, or a small
| change to an existing shell) begin an implementation.
| Starting with a proof of concept without a well developed and
| well documented design is the wrong way to offer a "product"
| but also the wrong way to learn.
| da39a3ee wrote:
| You either have no idea about open source software, or you
| have no idea what it feels like to love programming, or
| both.
| da39a3ee wrote:
| Please define what you mean by "publishing". Where has the
| author said anything more than "this is a project on my Github
| and here's what it does"?
|
| Assuming the answer is nowhere:
|
| Please retract. You appear to be one of those people with no
| understanding of modern open source culture, and only the
| ability to criticize. Do you expect everyone to keep their
| projects private until they meet your personal standards? (a)
| why the fuck should they? (b) what about a Github user who
| simply lacks the skills to ever meet your standards? (Evidently
| not the case here.) It's a shame I'm being overtly aggressive,
| but I think it's commensurate with the less overt
| unpleasantness of your comment.
|
| I bet you're one of those people who thinks it's
| "irresponsible" to publish software on Github that could have
| bad consequences if someone were to rely on it in an important
| context. Repositories such as Github are free. They are a place
| for anyone to store whatever creations they come up with. It's
| a huge and uncuratable mixture of essential backbones of modern
| computing, art, utter incompetence, and technical genius, and a
| lot of stuff in the middle. How that is hard for people like
| you to understand I do not know. I guess you have a very boring
| job at a very boring company.
| chriswarbo wrote:
| Since there are lots of alternative shells being mentioned, I
| thought I'd give my tuppence. I want two things from a shell:
|
| - Easy process execution, including branching on exit codes,
| overriding env vars, providing a working directory, etc. This
| must support concurrent processes (e.g. pipelines).
|
| - Easy file and stdio management (i.e. piping). This must support
| streaming (passing along chunks of data as they arrive, rather
| than gathering them into big intermediate values).
|
| Scripting language REPLs provide everything else I care about
| (string handling, looping, variables, etc.). Their built-in
| support for these two things is usually really bad, but libraries
| can plug that gap. I've tried a few for Python, Haskell, Scala,
| etc. but the best experience I had was Racket's shell-pipeline
| library ( https://docs.racket-lang.org/shell-pipeline it's also
| used by the Rash shell that I've not used https://docs.racket-
| lang.org/rash ). This makes really good use of Racket features,
| e.g.
|
| - Quasiquoting for mixing commands with normal functions in a
| simple, sane way
|
| - S-expression syntax works well for shell-like code, e.g. no
| comma-separation, distinguishing between (unquoted) symbols for
| names, flags, etc. and quoted strings for data, allowing special
| characters in names (e.g. to allow things like > in our
| pipeline), etc. Of course, Racket can be written with
| I-expressions, sweet-expressions, etc. too, but I've never
| bothered.
|
| - Stdio redirection works with Racket's existing "ports"
| mechanism
|
| - Env vars can be managed using Racket "parameters" (dynamically-
| scoped variables)
|
| (Note that the API seems to have changed a bit since the version
| I used, so I'm not sure how the new stuff like laziness fits in)
| mongol wrote:
| We need SQshell, a shell that uses SQL for operations such as
| listing files, processes, all that stuff where a structured query
| language is useful. SQL join syntax is not that different from
| piping. In my opinion, SQL knowledge is timeless, most other IT
| knowledge seems to have a "best before" date, but SQL never
| expires.
| sumtechguy wrote:
| The thing is with SQL is many devs seem to be in one of 3
| states with it. Allergic to it, will not touch it. Enough to be
| dangerous, and make very bad decisions with it. Amazing at it
| and fixing poor decisions. I am not sure what it is about SQL
| but it really seems to rub people the wrong way. I think it is
| the order of the statements. I think if they had switched the
| from and select statement around people would 'get' it more.
|
| SQL is a neat tool for what it is. But command line bashing
| does not seem like a good fit to me. I get what you are saying
| though. I just would be kind of miffed to have to do something
| like 'select * from folders(/foldera/folderb/folderc) where
| lastcreatedate > getutcdate('1970') order by lastcreatedate
| desc' when I can currently do something like 'ls -ltar'. It is
| not that I can't do that. But it takes a cognitive load to do
| it for a command I use a lot. Of course I could make a program
| to do it. But then I am kind of back where I started? I could
| get behind SQL in addition to what I have. But in replacement I
| think that would be a tough sell.
| smichel17 wrote:
| I am definitely in the "enough to be dangerous" category, but
| isn't it technically possible to write SQL queries the way
| you describe? IE, the only reasons to write them in SELECT ..
| FROM order are (1) tradition / what others expect, and (2)
| optimizations made by the dbs that assume the traditional
| order? But in terms of returning correct results, FROM ..
| SELECT should work fine, I think.
| sumtechguy wrote:
| Having not written a SQL parser. But I would assume there
| is a grammar parser in there that puts the items into some
| IR like language or structure that runs it. The big ones
| call it a query plan. So you in theory could flip them
| around as long as the query plan was the same. You would
| just need to basically convince MS and Oracle and all the
| other DB projects to do/try it. Not sure they would see the
| benefit though. In my head they are backwards but in their
| design it may be fine. Ideally the keywords could be in any
| order. But that would probably break some implicit ideas
| that those keywords convey (looking at you having and group
| by).
| vthriller wrote:
| You can do that kind of things already, albeit not from within
| a single universal shell:
|
| - https://github.com/jhspetersson/fselect
|
| - https://github.com/osquery/osquery
| frou_dh wrote:
| Speaking of fancy alternative shells, I tried http://xon.sh/ for
| a while, whose selling point is that its language is a superset
| of Python.
|
| While it was neat for interactive use to have stuff like proper
| string manipulation and list comprehensions, writing scripts with
| it (.xsh files) was horrible, because the sorta-but-not-really-
| Python-ness meant that no tooling worked. Syntax highlighting was
| messed up, code autoformatters barfed, linters barfed,
| autocompletion didn't work.
|
| I actually find writing scripts in Bash, aided by the amazing
| linter https://www.shellcheck.net/ to feel almost like it's now a
| compiled language. It goes from being a fraught endeavor, to
| being kinda fun and educational. It's like pair programming with
| someone who has an encyclopedic knowledge of the language quirks.
| Hook it up to automatically run on-save in your editor!
| Skunkleton wrote:
| I check in on Xonsh every once in a while. The first test I do
| is try and suspend a sleep into the background. This has never
| worked on xonsh afaict. I am excited to give Xonsh a spin, but
| I can't until it gets its job control story sorted out.
| geophile wrote:
| Take a look at marcel: https://marceltheshell.org. It is also a
| shell that pipes objects, and is based on Python, but it sounds
| like scripting will be more to your taste. There is a
| marcel.api module that allows you to do shell-like commands
| neatly integrated into python code, e.g. from
| marcel.api import * for file, size in
| (ls('marcel', 'test', file=True,
| recursive=True) | map(lambda f:
| (f.path, f.size))): print(f'{file}: {size}')
| nathell wrote:
| For Clojure aficionados, there's a flavour of Clojure made for
| scripting and the command line: Babashka
| https://github.com/borkdude/babashka
|
| And a proper shell, Closh: https://github.com/dundalek/closh
| chrisweekly wrote:
| Curious how this compares to Xonsh (IIUC, pretty similar idea,
| tho I Xonsh is actual Python, vs Crush using its own new lang).
| onli wrote:
| Oil shell should be mentioned here, an alternative approach at a
| modern shell. It tries to be a (more sane?) upgrade path to bash:
| https://www.oilshell.org/
| chubot wrote:
| Thanks for the mention! That's an accurate way of describing
| it. I updated the home page with the one page description: _Oil
| is our upgrade path from bash to a better language and
| runtime._
|
| The ability to run existing POSIX and bash scripts is the
| biggest difference between Oil and other new shells (something
| I'm often asked). Unfortunately that's also why it's such a
| long implementation effort. But in reading the comments here, I
| still think that was the right choice.
|
| ----
|
| FWIW Oil's approach to structured data is to use interchange
| formats over pipes. You can already pipe JSON, HTML, and
| CSV/TSV over pipes, and people do. There are many command line
| toolkits that do this.
|
| https://github.com/oilshell/oil/wiki/Structured-Data-in-Oil
|
| However the shell is also missing few things which would help
| that approach, so I'd like to build some native understanding
| of those formats into it, without requiring any special "plugin
| interface". Plugins are simply _processes_ in Oil.
|
| For example, Oil already has a QSN "Quoted String Notation" to
| properly escape byte strings like filenames (this happens to be
| the format of Rust string literals, which are similar to C and
| JSON string literals). So you don't have to do parsing and
| splitting.
|
| The approach to tables will likely be built on QSN in TSV or
| QSV. And then you can enhance "npm list" and "dpkg" just to
| print that to stdout, without having to integrate into any
| special "plugin system". It just works with normal Unix
| processes and the kernel.
|
| https://github.com/oilshell/oil/wiki/TSV2-Proposal (not
| implemented)
| chubot wrote:
| Update: I just wrote a draft of "Why Use Oil?"
|
| http://www.oilshell.org/why.html
|
| Let me know what you think! (feedback link in the article)
| pjmlp wrote:
| > I also feel that tying a shell to COM objects is a poor fit.
|
| This is not correct. Powershell allows calling into COM, .NET or
| straight DLLs for that matter.
|
| It is also what I see as missing opportunity in most UNIX shells
| that failed to build up on Xerox PARC, Lisp Machines and ETHZ
| experiments.
|
| The ability to directly plug into shared libraries, UNIX IPC, and
| modern mechanisms like DBUS and gRPC, alongside live debugging
| capabilities, thus exposing the whole OS stack to the CLI, making
| it a proper REPL experience.
|
| Other than that, it looks quite cool.
| iflp wrote:
| Reminds me of elvish (elv.sh), which has a similar focus.
| gonzus wrote:
| I believe a good fit for Unix commands is using a pipeline where
| the output of your command (say, ls) is converted to JSON, so
| that later in the pipeline you can use jq to filter / count / etc
| in a structured, non-whitespace-dependent way. A combination of
| jc (https://github.com/kellyjonbrazil/jc) and jq fits the bill
| for me: ls -l /usr/bin | jc --ls | jq '.[] |
| select(.size > 50000000)'
| lordgroff wrote:
| I think this should honestly be higher. I'm not sure if jc is
| THE tool -- the Python dependency irks me slightly -- but I
| like the simplicity of the idea, wrapping around what is
| already available and functional, while improving my
| experience. I plan to play around with this idea.
| pedro84 wrote:
| $ ls -l /usr/bin | awk '$5 > 50000000'
|
| -rwxr-xr-x 1 root root 51859776 Jan 26 2018 pandoc
| Skunkleton wrote:
| Most commands that you would want to put into a pipeline
| already have some sort of machine parsable output flag
| (tabular, json, format specifier, etc). Still, there is not
| much reason to call ls from a script at all. In python you
| would be better off with the native libraries. In bash, you
| would probably be best off with `find /usr/bin -maxdepth 1
| -size +50M` in your usecase.
|
| IMO people have been trying to do the same thing with these
| tools since they were invented. Better to learn how to use them
| the way they were intended before reinventing them.
|
| Edit: re-read my comment. Seems a little negative. Sorry. To be
| clear, I think the jc/jq approach is much better than the roll-
| your-own parser in shell approach.
| dimator wrote:
| If you like the sql-like filesystem interaction, check out:
|
| https://github.com/jhspetersson/fselect
|
| I've integrated this into my workflow, it's a polished tool.
| codethief wrote:
| This is really cool! I've been waiting for something like this
| for a long time since I don't use `find` often enough to
| remember its command line arguments. Finally
| moonchild wrote:
| I think that these 'improved shell' projects (incl. crush and
| nushell) that try to copy powershell somewhat miss the point of
| the shell in the first place. If I want a high-level language
| that's tightly integrated with the unix environment, I already
| have perl and tcl. Little more power (maybe less integration, but
| greater capacity to build new integration) and I can use raku or
| lisp. The point of shell is that it interoperates freely with
| _all_ programs and _all_ programs share a language. If now,
| suddenly, I can only use commands that have been explicitly
| rewritten to support your environment; or regular commands are
| second-class citizens and need support code to interact with your
| environment 's higher-level functionality----
|
| Then your environment is useless to me. Because I have better
| programming languages which, despite being able to interact with
| the system and its commands, don't pretend to be one with the
| system. As a result, the core high-level language constructs mesh
| better, because they're not trying to serve two goals at once.
| It's not possible for something that's not a 'real' shell
| (including fish, zsh, even csh) to integrate with the system the
| way a real shell does, because the _system itself_ was designed
| around the shell.
| moonchild wrote:
| To clarify, I am not saying that everything except for bare
| /bin/sh without line-editing is Wrong. I wasn't clear enough
| here. I think that bourne shell, bash, fish, zsh, and csh are
| all able to interact with the unix system in a way that crush,
| nushell, and other powershell-alikes cannot.
| throwaway894345 wrote:
| Can you elaborate on that more of interaction? What can bash,
| etc do that crush, etc can't. Is this limitation inherent or
| easily amended?
| liveoneggs wrote:
| yeah perl with qx/cmd/ calls and similar is just really really
| good for "shell scripting".
| nixpulvis wrote:
| Exactly!
|
| This is why I started my shell. Eventually it will have the
| ability to run both POSIX and other shell languages in the same
| program.
|
| Assuming everything is a table is overly restrictive. Assuming
| everything is text is the UNIX way. It's then the job of the
| terminal to render it.
| chubot wrote:
| Right, I'm often asked this about Oil [1]: Why do you want to
| write programs in shell?
|
| That's not the idea of shell. The idea is that I write programs
| in Python, JavaScript, R, and C++ regularly, and about 10
| different DSLs (SQL, HTML, etc.) And I work on systems written
| by others, consisting of even more languages.
|
| I need a language to glue them together. A language to express
| build automation and describe deployed systems. Most big
| systems consist of more than one language.
|
| Shell is the best language for that, but it's also old and
| crufty, with poor implementations.
|
| When you program in shell, gcc, git, pip, npm, markdown, rsync,
| diff, perf, strace, etc. are part of your "standard library".
|
| [1] http://www.oilshell.org/
| chubot wrote:
| Update: also see the philosophy for structured data in Oil
| below: https://news.ycombinator.com/item?id=24083931
| pritambarhate wrote:
| Not many young people know Perl or Tcl (or even lisp for that
| matter).
|
| >> The point of shell is that it interoperates freely with all
| programs and all programs share a language.
|
| Also doesn't mean people should stop trying to come up with
| these new shells. May be some "new" shell will catch up and
| tomorrow's "all" programs will be compatible with that shell.
| For example, now on Mac OS X, now ZSH is the default shell.
| drbojingle wrote:
| when he says Perl or Tcl I think he means general purpose
| languages. It doesn't have to be Perl, or Lisp or Tcl, those
| are examples.
| mywittyname wrote:
| > Not many young people know Perl or Tcl (or even lisp for
| that matter).
|
| I don't know perl and I'm not young, but I do understand and
| appreciate the role perl -e plays as an extension of bash.
| When I run into an issue where I need more than bash, but
| less than python, I'll search around SO for a perl -e command
| for what I need.
|
| This is often how people decide to learn a tool: they have a
| problem for which it is the best solution. They get some
| exposure to the capability and slowly expand on their
| knowledge.
| esoterae wrote:
| I think that means you're using SO as a tool..?
| Shared404 wrote:
| Don't most of us? That doesn't mean he's not using Perl
| as well.
| _jal wrote:
| > Not many young people know Perl or Tcl
|
| There are more young people who know Perl or Tcl or Lisp than
| know Crush. I've never understood this reasoning - "Hardly
| anyone knows X. Let's replace it with something nobody
| knows." At most, you're making an argument about a lack of
| sunk cost.
|
| > May be some "new" shell will catch up and tomorrow's "all"
| programs will be compatible with that shell
|
| Think of it like transitioning to life in a wheelchair. You
| can't "catch up" with your house - it just wasn't designed
| for wheelchair use. Renovating all the narrow corridors,
| random steps, tight angles and vertical reaches probably
| costs more than rebuilding.
|
| In this comparison, the mobility mode is your shell.
| jwilber wrote:
| Crush has been around < 1 year. Perl has been around for
| 32. Of course more people know Perl.
|
| If I'm being honest, you're not adding anything
| constructive at all here - it seems you're in a competition
| with yourself as to how much you can degrade Crush for
| whatever reason.
|
| You act like it's a choice between Crush and Perl. People
| don't have to learn Crush. People don't have to learn Perl.
| Will picking between the two have any substantial effect on
| one's career outcomes these days? Probably not, as
| candidates will have already learned the important
| languages for getting a job anyway.
| _jal wrote:
| If I'm being honest, I have no idea where you got the
| belief that either I'm some big Perl partisan or that I
| have something against Crush, or why you're getting so
| personal.
|
| But it is cute to couch it in a complaint about not being
| constructive; points for shamelessness.
| Riverheart wrote:
| "Hardly anyone knows X. Let's replace it with something
| nobody knows."
|
| Because there's a value proposition. Just like TCL, Perl,
| and Lisp had value propositions over other available
| languages. It's fine if you don't see the value proposition
| in Crush but progress sometimes forces us to learn new
| things instead of sticking with what we have.
| TheDesolate0 wrote:
| Yes, but this is the role of other languages. Not the
| shell. I use Bash. I love bash. I code in bash for work
| daily. But I would never use it beyond simple stuff!
| That's not it's job. It's job is to play nicely with all
| my tiny little tools, allowing me to string them all
| together into compound tool statements, allowing me to
| use the OS and small apps to make the powerful tools
| WITHOUT needing to reinvent the wheel (use a real
| programming language).
|
| This is the roll of the shell.
| pritambarhate wrote:
| >> Hardly anyone knows X. Let's replace it with something
| nobody knows
|
| By that logic no one should make anything new to replace
| the incumbent.
|
| In case of Perl, PHP came and it replaced Perl as
| mainstream web development language.
|
| In sysadmin stuff where Perl used to be mainstream, now
| significant chunk is taken away by Python.
|
| These days JavaScript appeals to younger people more. Times
| change and with it the tools people use also change.
|
| There is no guarantee that any new project will succeed but
| you won't know unless you try.
| moopling wrote:
| The original point was "why would I learn a new thing, when
| I already know something which works". The response "some
| people don't have something they already know" is valid.
|
| You're really bringing up a separate point which is "should
| we be reinventing the wheel?", I think that's a fair point,
| but it's not a response to the original commentator.
| oblio wrote:
| It only took Zsh 29 years (initial release year: 1990,
| adoption in Mac OS: 2019) to become the default shell on one
| of the major _nixes, while being POSIX compatible and having
| just an extension layer on top.
|
| I wouldn't hold my breath for something else to replace
| sh/bash as the default shell on the major _nixes within this
| century. Pessimist? More like realist :-)
| robin_reala wrote:
| That wasn't even driven by zsh's actual benefits, but by
| the fact that bash changed to GPL3. If it had continued on
| GPL2 we'd 100% still have bash as a default.
| Fnoord wrote:
| And the fact Apple doesn't like GPLv3.
| TheDesolate0 wrote:
| Well, they hate open source in general. Apple is the,
| well, Apple of the 90's.
|
| God. I remember the 80's. Apple was the Hackers'
| computer! (well, if you didn't have access to a major
| corp or uni machine room). They were all into DIY! and
| getting kids to code!
|
| Bill Gates back then was all "neh! mine!" He was a
| SERIOUS tool! God he used to just fucking scream at
| people. awful. OMG. Young Bill Gates sucked donkey balls.
| Paul was paul. I kinda felt bad for him, actually.
|
| I dunno. then something happened. maybe it was the whole
| 6502 thing. memories are fuzzy, but Apple changed in what
| seemed like overnight.
| moonchild wrote:
| I explicitly included zsh. The point is not the language of
| the shell itself; that has never been the point. The point is
| that zsh interoperates with the other programs on your system
| in a way that crush doesn't and can't.
| viraptor wrote:
| > a way that crush doesn't and can't
|
| What do you mean by can't? If it misses some mode of
| interaction, that can be always added.
| ori_b wrote:
| Consistent behavior when piping through all commands. A
| lack of modes of operation to consider. Simplicity.
| viraptor wrote:
| Simplicity exists on more than a single dimension.
| Choosing one column from "ls" output for example is not
| simple, bordering on impossible using the consistent
| behaviour of standard pipes.
|
| Sometimes the inconsistent solution will be simpler by
| moving the complexity to the choice of the mode of
| interaction. Sometimes not.
| moonchild wrote:
| > Choosing one column from "ls" output for example is not
| simple, bordering on impossible using the consistent
| behaviour of standard pipes. ls -l | awk
| '{print $3}' ls -l | cut -f 3 -d ' '
| ori_b wrote:
| Maybe a shell is the wrong tool for that job, and hiding
| the modes that the language is operating in complicates
| things.
| ratboy666 wrote:
| You are correct. It is all about how you think about
| things.
|
| "ls" lists files. Sure. Simple interactive command. But,
| thinking about this, "echo _" also lists files. "ls"
| provides information about files. Sure. Simple
| interactive command. But, "stat" specifically provides
| information.
|
| So, I would reach for
|
| stat --format "%n %a" _
|
| To get name and octal permissions. You are correct, and
| wrong at the same time. Easy enough to accomplish -- but
| using * instead of ls. Note that the "space in filename"
| becomes the big issue (use %N in stat format).
|
| So your conclusion is 100% correct. Thank you for your
| post and insight
| heresie-dabord wrote:
| Many young people don't know history, science, other
| languages, or even the grammar of their native language. That
| is why we teach.
|
| The "ecosystem" of programming languages is healthy. There
| are many options, some to master, some to consider. But not
| knowing how to use a tool -- especially a widely-used and
| powerful tool -- is no badge of honour or discernment.
| cat199 wrote:
| > That is why we teach.
|
| but what if people don't know how to learn?
|
| instead of getting wrapped up in all of this 'knowledge
| preservation' nonsense, shouldn't we just mimic what they
| are already doing and assume it to be better, because the
| activity occurs in a year number that is higher than the
| previous behavior pattern?
|
| now _that_ would be real progress.
|
| or, not.
| heresie-dabord wrote:
| Sarcasm duly noted. (o;
|
| Societal results sadly noted.
| shawxe wrote:
| You really aren't kidding about Tcl. I recently discovered that
| with tclreadline, Tcl can be used completely in place of Bash
| in interactive mode, in a way that something like Python
| absolutely cannot.
|
| A lot of people seem to have negative opinions about Tcl (maybe
| because it's more of a LISP than its syntax seems to imply),
| but as someone who is just discovering it, I'm shocked at how
| powerful it is for how unpopular it seems to have become. It's
| like some kind of unexpected cross between Bash, LISP, and
| (with Tk) Visual Basic. There's something about it that's just
| kind of fun.
| hodgesrm wrote:
| Tcl is awesome. I used it constantly in the 1990s. It's
| outstanding for things like testing and build scripts.
|
| I think some of the pushback might be because people tried to
| use it to build large applications and failed. Languages like
| Java and (dare I say it) Visual Basic were more comprehensive
| and had good IDE support.
| cnity wrote:
| Your ideas are something I wrestled with while I was writing (a
| very similarly named) crsh[0]. I chose to avoid trying to
| reimplementing commands and instead stuck with an unstructured
| environment, but with the ability to parse output using an
| expandable set of parsers and use inline JavaScript to operate
| on it. I agree with what you say about perl, but the number of
| users with experience in perl is -- at least in my bubble --
| very small compared with more modern languages.
|
| [0] https://crwi.uk/2020/07/12/crsh.html
| dividedbyzero wrote:
| While I have some relatively extensive experience with Perl,
| I'd rather write assembler using morse than touch that ever
| again. The language is one guy's not-too-well-thought-through
| hack that gained enormous traction by filling a void, and
| from there grew far too organically. There are lots and lots
| of cryptic ways to do the most mundane things, lots of
| inconsistencies, the language really makes it hard to write
| software that isn't terrible to maintain.
| cnity wrote:
| Exactly. This is common with shell scripting languages.
| They feel like some awful duct tape, when in fact the
| perfect primitive for joining separate concerns already
| exists (the pipe). Using a nicer language (like modern JS)
| to perform procedural logic makes perfect sense to me.
| gridlockd wrote:
| Aren't we talking about "a better shell scripting
| language"? At the point where a Perl program becomes "hard
| to maintain", what would the bash alternative even look
| like?
|
| Perl code is often written once and then runs _for years_
| without issue. I have on occasion fixed broken Perl scripts
| despite having practically no experience with the language.
| It wasn 't a problem at all, because your average
| maintenance job is "something was moved" or "we're
| switching out X for Y", but not "we need to rewrite this
| complex system that is a Perl script for some reason".
| licebmi__at__ wrote:
| >It wasn't a problem at all, because your average
| maintenance job is "something was moved" or "we're
| switching out X for Y", but not "we need to rewrite this
| complex system that is a Perl script for some reason".
|
| I would intuitively agree with you as there are hundreds
| of unnoticeable perl scripts doing their job, but as
| someone with moderate experience with the language, I can
| say that when I did maintenance tasks, they were mostly
| the second kind.
| zeveb wrote:
| You are right, but none of those criticisms apply to Tcl,
| which is both beautiful and integrates really well with the
| Unix environment. I won't say that it is _perfect_ , but it
| is really, really good. A better choice than Perl where
| Perl is used, a better choice than Lua almost everywhere
| Lua is used, often a better choice than Python.
|
| Tcl is horribly, terribly under-appreciated. It is one of
| the great might-have-beens of modern computing, almost as
| much as is Lisp.
| yumaikas wrote:
| Eh, having used both Tcl and Lua, I definitely think Lua
| is the better of the two when it comes to a scripting
| engine for games.
|
| And I think Lua is more elegant than TCL overall. TCL's
| stringiness really gets clumsy in certain types of
| programs, and it's a much larger target than Lua overall.
|
| Not to say that TCL is bad, but I'm glad that dict-based
| languages prevailed over TCL's string-based-ness
| mhotchen wrote:
| This is a very interesting project. I really like the
| combination of languages and the way you've implemented it. I
| already see some opportunities to use it.
| cnity wrote:
| Thank you! It's pretty buggy at the moment, and maybe
| frustrating to use since it's missing some important shell
| features, but any feedback is always welcome.
| mhotchen wrote:
| I'm very impressed with what works so far by the way.
| mhotchen wrote:
| Yeah I've just installed it and seen eg. the
| autocompletion on ls isn't how I'm used to. I will keep
| playing around and watch for progress because I really
| like the idea.
|
| I especially see this being useful in my CI environment
| which has a handful of functions, some json and yaml
| parsing (using jq and python respectively), etc. It's a
| little cumbersome at the minute but not so much to switch
| from bash and the rest of the project is JS anyway so it
| would be good to be consistent.
| cnity wrote:
| Excellent. If there are things you notice, please feel
| free to raise as an issue, and pull requests are always
| welcome!
|
| > useful in my CI environment
|
| For sure. I've been meaning to play around with getting a
| docker image with the shell set as the default. It's
| definitely a use-case I had in mind.
| r-w wrote:
| It's my understanding that commands don't have to be specially
| written for Crush. As long as they speak an understandable
| format, they can be used.
| https://github.com/liljencrantz/crush#reading-and-writing-fi...
| kevincox wrote:
| This seems to strike a good compromise as "third party"
| commands are "lines" type which is basically what we are used
| to send and receive in unix tools. Of course a lot of tools
| already support emitting JSON so it seems the Crush is already
| more useful than the typical unix shell as you can still use
| "lines" for every program (and convert to a table by parsing if
| you want to use the more advanced features) and there are
| already a number of third-party programs that you take directly
| take advantage of these feature with even though they didn't
| know anything about this shell.
|
| I think the fact that you can already use this with most
| programs and it is never less powerful than bash seems like
| good indication that the system doesn't need to be reworked
| around this shell. It fits in cleanly to existing paradigms.
| hhas01 wrote:
| "somewhat miss the point of the shell in the first place"
|
| This presupposes that the Unix shell is the way it is because
| of careful, considered, forward-thinking design, rather than
| being a haphazard heap of mouldering hacks upon lazy corner-
| cutting upon quick-n-dirty bodges that should've been thrown
| out and replaced decades ago.
|
| I agree that trying to improve the *nix shell is a bad idea...
| because the _Unix shell itself_ is a bad idea. Fifty years is
| long enough. Instead of bending over backwards for that
| obsolescent garbage heap (for which no-one will thank you, as
| we see), study it, learn its lessons, and draw a line under it.
|
| Then go build a clean, modern shell architecture from scratch,
| using modern tooling and techniques, unhobbled by the myriad
| defects and deficiencies of the old. It'll take a tenth of the
| time and be ten times better (if not, you're still doing it
| wrong).
|
| See also:
|
| https://www.google.com/search?q=sunk+cost+fallacy
|
| https://homes.cs.washington.edu/~weise/uhh-download.html
|
| --
|
| p.s. As for driving early adoption, don't even think of trying
| to win over the Unix old guard (who will only bring regressive
| practices and complaints with them), but sell it hard to the
| new generations of users who are accustomed to modern UI/UX
| standards and much less impressed by what the "Old Unix"
| dogpile has to offer.
| joe_the_user wrote:
| _Because I have better programming languages which, despite
| being able to interact with the system and its commands, don 't
| pretend to be one with the system. As a result, the core high-
| level language constructs mesh better, because they're not
| trying to serve two goals at once._
|
| The power and usefulness of a given language is often the
| ability to serve multiple goals at once. Python as tool is cool
| specifically because of a large-ish but accessible-ish library
| covering a everything from regex-ex to a web-server.
|
| This project might not serve the goal of writing high level
| programs and interacting effortless with the command
| line/system. But I don't one should automatically dismiss the
| goal of making stuff-glued-together look like a coherent whole.
| jodrellblank wrote:
| > " _because the system itself was designed around the shell._
| "
|
| Windows wasn't designed around the command prompt, it wasn't
| designed around shell utilities. When I call "cat", it's not
| important that I'm writing those three letters, they are a
| library way of calling a set of file open/read OS functions.
| When I write "ping", it's not the ping tool I care about, it's
| the network api calls it does behind the scenes. The Unix/Linux
| shell is then seen like a "Python standard library" of ways the
| user can control the OS, and it's a hugely inconsistent,
| undesigned, ad-hoc agglomeration of names and parameters and
| bodges, which cheats out on dealing with the complexity and
| instead offloads that straight on to the user in a user hostile
| way.
|
| What powershell tries to do for Windows is make a somewhat
| designed, consistent, and more user-approachable set of
| wrappers around the Windows OS system calls, along with
| guidelines and support for vendors to build in that style, a
| human interface that is introspectable, built to be
| programmable, but still composable and modular.
|
| The objection that trying to do such a user-focused thing on
| Unix makes a "useless" system is weird because that - a user
| interface to tools/library of OS call patterns - is what the
| shell is. In what way is "resolve-dnsname" pretending to be one
| with the system that dig and nslookup aren't? They're names
| which trigger a pattern of network traffic and show a result.
| In what way is piping text from dig to sed "real" and piping
| resolve-dnsname to format-list "pretend"?
|
| In what way is running "docker compose" from sh real, but
| running it from zsh is pretend?
|
| In what way is it a better design to have (a pile of shell
| tools and Python library wrappers and Python wrapping shell
| tool calls), instead of (a Python based shell with transparent
| shell tool calls)?
|
| Like, "I want what I'm familiar with, too many things to
| relearn", or "too many things are awkward in this approach
| (cough PowerShell)" are reasonable objections, but "trying for
| a more consistent, more human friendly, interface with more
| high level language constructs at one's fingertips makes a
| thing _useless_ to me" feels heel dragging or Stockholm
| syndromeish. Presumably you don't object to there being a
| choice of multiple command line tools to do a task? Why object
| on principle to a choice of other OS "front end experiences"?
| Is there any version of things which you can imagine as being
| both "better" (less arbitrary and inconsistent than shell) and
| "not pretend"?
|
| > " _The point of shell is that it interoperates freely with
| all programs and all programs share a language._ "
|
| What language? Some treat piped bytes as bytes (e.g. piping a
| binary file), others as ASCII, others as UTF-8. All support
| different parameter options, names and values and ways of
| specifying them. Many handle output formatting in different
| ways, many trigger internal states from environment variables.
| If the point is a common shared language, the point doesn't
| seem to have been achieved from a user perspective. And if the
| point is that "I have to suffer a system that's tied to a
| shared language with an AIX box in a bank basement last touched
| by a human in 1997" the point is a bad one. I use my computer a
| lot more than I use AIX and I have more computing power to
| spare on human comforts and conveniences.
| jerf wrote:
| "What language?"
|
| The UNIX philosophy was to write a solid system by taking
| most of the hard problems in kernel space and kicking them
| into user space. This is not a slur, this is a description.
| Especially at the time, it wasn't at terrible solution. I
| kind of think of it the same way as many programming
| languages have observed that "the standard library is where
| things go to die"; the code in a system that is hardest to
| change, well, it doesn't change! So any bad decision that
| ever gets in there is almost permanent. Part of the reason
| UNIX won is that it enshrined fewer bad decisions in the
| kernel than its competition did. I think you could even argue
| it had more bad decisions overall than its competition, at
| least towards the beginning; it's just that far fewer of them
| were enshrined in the kernel.
|
| This is also the philosophy of the pipe. It disclaims all
| layout and moves that responsibility on to the consumers.
| There is no "shell language", at all. This is,
| simultaneously, a big strength and a big weakness.
|
| One aspect of that is precisely that the strength of this
| approach tends to inhibit anyone who tries to beat it with
| something like Powershell. All Powershell had to be was
| better than any shell on Windows. This was not hard. For
| something like Crush, to take off it has to be better than
| any existing shell on UNIX, and as annoying as unstructured
| pipes may be, the ecosystem as a whole still has a lot of
| positives about it, because it's been tuned for 50+ years.
| It's going to take a lot of work to match that, and so far,
| it doesn't seem like any of these projects are achieving
| take-off velocity.
|
| Arguably, the best thing to do would be for all these
| projects to get together and try to define a standard UNIX-
| level non-textual rich serialization of some sort, using
| lessons from Powershell and arguably things like D-Bus to do
| it, because none of these projects individually seem to be on
| a success trajectory.
| BeetleB wrote:
| > For something like Crush, to take off it has to be better
| than any existing shell on UNIX, and as annoying as
| unstructured pipes may be, the ecosystem as a whole still
| has a lot of positives about it, because it's been tuned
| for 50+ years. It's going to take a lot of work to match
| that, and so far, it doesn't seem like any of these
| projects are achieving take-off velocity.
|
| Who cares about "taking off"? If it's useful for 10s of
| users, it has taken off.
|
| Consider xonsh (https://xon.sh/). I don't think it will
| ever be "mainstream" the way bash or zsh are. I doubt it
| will ever come preinstalled in any major Linux
| distribution. But it's used by enough and has enough
| momentum that I don't worry it will die. That alone is
| enough for me to use it as my primary shell. I don't care
| that xonsh will never "take off".
| jerf wrote:
| Failure to take off means that instead of getting a long
| tail of things that work, you get a smaller island of
| reduced functionality, even if that is enough to keep you
| using it.
|
| It doesn't have to be the goal of the project or the
| author, but there are real advantages to achieving a
| takeoff, even to the users of a product.
| hodgesrm wrote:
| New shells seem unlikely to catch fire unless they:
|
| * Preserve compatibility with popular shells like bash.
|
| * Address some of the broken parts in a way that makes the
| world better. Quote handling is the original sin of *nix
| shells. If somebody fixed that you would have my full
| attention.
|
| Otherwise I can just stick with Bash + Python3.
| haecceity wrote:
| fish is pretty popular?
| hannasanarion wrote:
| Which is impossible, which is why no new shells have caught
| fire.
|
| In order to preserve compatibility, you _must_ handle quotes,
| namespaces, etc in the same way as Bourne-derived shells,
| because they will be expected to run scripts that use nested
| interpolation, or string declarations without quotes, or
| whatever. Shells that use a more robust syntax, like Fish or
| Powershell, can 't ever be compatible.
| majkinetor wrote:
| Not exactly true since PowerShell is doing quite good
| regarding popularity nowdays...
| mdaniel wrote:
| From my perspective, that's partially because CMD is so
| _stunningly_ incapable as what a reasonable person might
| think of as a shell. I 'd guess the other half is the
| amount of pressure Microsoft has placed upon management
| tooling using PS
|
| If Apple shipped some Swift-powered shell and all of
| their management documentation started using it, I'd bet
| a non-trivial portion of administrators would start to
| use AppleSwiftShell (although I do deeply appreciate
| that's a severely flawed comparison since macOS already
| has a sane functioning shell, but it's the best I could
| think of as a huge company that owns the platform pushing
| such a concept down)
| chubot wrote:
| It's possible, and http://www.oilshell.org/ has done it:
|
| http://www.oilshell.org/release/latest/doc/simple-word-
| eval....
|
| You can try it right now. Turn on 'shopt -s
| simple_word_eval' to get the better behavior.
|
| Or turn on the group 'shopt -s oil:basic' and get even more
| better behavior.
|
| When those options are off, Oil runs some of the biggest
| shell scripts in the world. It's the shell that's most
| compatible with bash by a mile.
|
| https://github.com/oilshell/oil/wiki/Shell-Programs-That-
| Run...
| hodgesrm wrote:
| I can't speak to the implementation details but there must
| be a middle ground where you can set compatibility using
| 'set -N' or some similar approach to enable new behavior.
|
| Scripting languages like Python, Ruby, Perl, etc., have
| dealt with upgrade many times. It has not always been
| pretty, but the alternative is stagnation.
| chubot wrote:
| Yup, it's already been done with 'shopt -s
| simple_word_eval' in Oil, see right above:
|
| https://news.ycombinator.com/item?id=24086213
|
| And yes I drafted a blog post called "Unix Shell Should
| Evolve Like Perl 5" which says exactly what you are
| saying:
|
| http://www.oilshell.org/blog/2020/07/blog-
| roadmap.html#the-u...
|
| And Oil has already done it. It needs to be faster, but
| that's an inevitability based on progress so far.
|
| _Why Use Oil?_ http://www.oilshell.org/why.html
| IshKebab wrote:
| You are forgetting about how shell scripts start. "I don't want
| to keep writing these 5 shell commands, I'll put them in a
| file. Ok that is useful, I'll add some parameters. This shell
| script works great, I'll add more functionality."
|
| And before you know it you have a buggy monster written in a
| batshit insane language that nobody is going to want to touch.
|
| If your shell is at least vaguely sane, the situation is a lot
| better. I don't really get why you would argue against
| something that fixes the many many many mistakes of Bash.
| lolc wrote:
| To a first approximation, people who are ok with Bash as a
| scripting language do not know enough Bash to safely use it
| as a scripting language.
|
| I see many insane constructs that accidentally work. I also
| know how much time it would take to explain expansion rules.
| And people don't even want to hear it because it shakes their
| simplistic (and inconsistent) understanding.
|
| So yes, I hope we'll see something like crush replacing Bash.
| But I don't see how it will happen.
| j88439h84 wrote:
| A similar idea is Mario, for Python code in shell pipelines. Its
| most novel feature is (optional) async commands, so you can pipe
| in a list of urls, make many concurrent requests, and then
| process them in a single line. cat urls.txt |
| mario async-map 'await asks.get ! x.json()' apply len
|
| https://github.com/python-mario/mario
|
| Nushell also seems cool, it's somewhat similar to Powershell.
| https://github.com/nushell/nushell
| danielbigham wrote:
| I love this. Yes, there are higher level languages that can be
| used for accomplishing these things, but people find themselves
| at the command line a lot, and having the ability to efficiently
| express your intent using a familiar language, without getting
| hand-cuffed by a poor underlying representation seems very
| useful.
| aniou wrote:
| Nice idea that corresponds with my idee fixe - a sane, usable
| output format for Unix utilities. There are differences, of
| course - I prefer a text-table format as universal panacea, but
| I'm glad that more and more people notice a mess in Unix
| environment ("simple" doesn't mean "consistent" and even Plan9
| suffers from that).
|
| See also https://github.com/aniou/cof/wiki/Draft
| Wandfarbe wrote:
| I'm struggling today as well with this shit:
|
| I just wanna write a few small lines of a shell script which
| downloads stuff;
|
| It is very very stupid through what loops you hvae to jump to do
| this with curl and bash script in a good way including
| readability and error handling.
|
| Is a perl script really better?
|
| Perl 5,6 and 7 did not make it very sane to me, managing and
| installing and working with perl is also less trivial then having
| a shell like bash, sh or dash available to you.
|
| Something inbetween bash and go is missing, i like the approach
| of crush. I think thats the biggest misstake happening in bash:
| playing around with strings and auto expansion and substition
| etc. Its weird, its error prone, it doesn't have a proper clean
| well defined ecosysem.
| adamnew123456 wrote:
| If you're open to learning it, Tcl is actually a good middle
| ground here. The best way I can describe it is "structured
| shell script" - still very string oriented, but the syntax and
| command set is much more well thought out, and its better for
| building libraries in (having namespacing, among other things).
| You even have a choice among object systems if you need one.
|
| You don't lose much in terms of shell interop though - the
| native exec command understands a lot of the basics (file
| redirection, pipelining, background execution). In the simplest
| case, it's just one more word: exec wget $URL
|
| You do have simple exceptions too, which terminate by default
| unless you catch them (unfortunately, the stack traces aren't
| really useful). AFAIK exec throws on a non-zero exit, and you
| can use it in your own code too: # Dead
| error "IO error" # Not dead if {catch [error
| "IO error"] error_or_result} { # The error puts
| "Caught: $error_or_result" } else { # The result
| puts "OK: $error_or_result" }
| robin_reala wrote:
| The author, Axel Liljencrantz, was also the initial developer of
| fish, so has form in shell development.
| cies wrote:
| Had a quick look: Fish is in C++, Crush is in Rust.
|
| I also feel Crush addresses some issues I had with Fish. Still
| on Zsh though (I like pastable scripts).
| gwd wrote:
| I was curious how crush compared to fish (having glanced
| cursorily once or twice at the latter). Want to expand on the
| "issues you had with fish"?
| frabert wrote:
| This sounds similar to the ideas found in PowerShell, which I
| really quite like in principle
| dimator wrote:
| Upon further thinking, I think the fact that running and
| integrating non-built-ins is so clunky makes this less a shell
| and more just a shell-like programming language, and at that
| point why not python?
|
| Like it or not, the Unix pipe model of command integration is
| what enables programs to "know" about each other without ever
| really knowing about each other. Crush looks like its foregoing
| that.
| mst wrote:
| The author was very clear that they're focusing on the
| programming language part first and the running non-built-ins
| part is just a proof of concept to be fleshed out later.
| caymanjim wrote:
| This is a shell for people who either don't know Unix or don't
| like Unix. The normal Unix tools do all these things, often with
| less typing, and they're a lot more flexible and transferrable to
| other environments.
|
| I get that many Unix commands are obscure, and the syntax is
| often inconsistent, but once you're familiar with them, you can
| do everything this shell does and a whole lot more. And for
| anything more complex you've got a million other languages to
| choose from.
| ashishmax31 wrote:
| Neat! Definitely going to try this out.
___________________________________________________________________
(page generated 2020-08-07 23:01 UTC)