[HN Gopher] Xplain - Explaining X11 for the rest of us (2017)
___________________________________________________________________
Xplain - Explaining X11 for the rest of us (2017)
Author : pcr910303
Score : 261 points
Date : 2020-08-18 11:44 UTC (11 hours ago)
(HTM) web link (magcius.github.io)
(TXT) w3m dump (magcius.github.io)
| jhardy54 wrote:
| This is great messaging. I'm happy to enable JavaScript when
| authors communicate why they use it.
|
| > This article relies a lot of interactive demos that use
| JavaScript. I understand you're disabling JavaScript because
| you've been burned by heavy scripts and tracking on other sites,
| but the article really doesn't work without the ability to play
| with it. I write every line of code in these demos from scratch,
| minus the snippet of Google Analytics that lets me know when
| people read these so I can join in the conversation. Feel free to
| block Google Analytics.
|
| This is a billion times better than a white screen with the words
| "JS required to use this app" when you're trying to read a simple
| document.
| bjarneh wrote:
| Very cool, hard to imagine the constraints they had when that was
| created. Reminds me of an interview I heard with Bill Joy a while
| back (creator of vi + worked on the Java language in its
| beginnings).
|
| "We solved problems that do not exist anymore" -- Bill Joy.
| DonHopkins wrote:
| "X is rasterop on wheels" -- Bill Joy in Unix Today 6/25 page
| 5.
|
| https://donhopkins.com/home/archive/NeWS/news-info.txt
|
| "..., Richie reminded the audience that Steve Jobs stood at the
| same podium a few years back and announced that X was brain-
| dead and would soon die.
|
| "He was half-right," Ritchie said. "Sometimes when you fill a
| vacuum, it still sucks."
|
| http://www.anvari.org/fortune/Miscellaneous_Collections/1332...
|
| https://news.ycombinator.com/item?id=22502125
| bitwize wrote:
| Thankfully, not only do we have a modern replacement for X in
| Wayland, we also have a modern replacement for NeWS. It's
| called "the browser"/Electron.
| moomin wrote:
| X also solved problems for a future that never happened. The
| whole client/server design was originally intended for a world
| where compute and storage were much more expensive than network
| bandwidth. I remember in the early 90s it was just kind of
| received wisdom that networks were going to get a lot faster.
| They didn't, processors did.
|
| Which isn't to say it isn't great that X makes it blindingly
| easy to put a window on someone else's screen.
| corty wrote:
| And yet, there is this newfangled streaming of videogames
| over the net...
| michaelmrose wrote:
| I guess the first question is how much people spend.
|
| Millennials spend a fair bit it seems.
|
| $112 a month
|
| https://venturebeat.com/2019/06/06/superdata-millennials-
| spe...
|
| This makes a console between 2 and 4 months of gaming
| budget.
|
| A network focused console still requires sone hardware or
| runs on a pc that costs more than most consoles.
|
| The big benefit would appear to be killing the secondary
| markets for used games dead which is a hard benefit to sell
| to your users. It also only saves you money up front as
| they pretty much must charge more than a console over time
| to cover their own costs.
|
| Then you have the problem of latency which can only be
| rendered slightly less crappy with substantial hardware at
| the isp, unpredictable demand hello covid, and annoying
| Netflix like account sharing.
|
| The real problem is that the more money your target
| audience has the better off they would be buying a desktop
| pc. Nothing up front sounds great to someone who can afford
| to pay 20 a month but then you can't eat. If someone can
| pay 99 why not buy a Playstation or pc?
|
| I predict it's a bad idea whose time will never come.
| izacus wrote:
| For which X is entrely inappropriate and needs a completely
| different approach.
| kjs3 wrote:
| Constraints? It was generally developed on '3-M' machines: more
| or less around 1MIPS of processing power (68k, VAX), 1M of
| memory, and 1M pixels (e.g. 1024x1024).
|
| While X11 is pretty grokky these days and definitely showing
| it's age, I'm not sure that the problems X11 solved don't exist
| any more. In particular, the idea of running an app on one
| machine and displaying it more or less transparently. It's just
| that with very performant desktops, this became the exception
| rather than the norm.
| pjc50 wrote:
| Solution: X11-over-websockets with display in Javascript.
|
| (Actually now wondering whether this nonsense already
| exists?)
| cycloptic wrote:
| Xpra has a way to do this:
| https://xpra.org/trac/wiki/Clients/HTML5
|
| There's a similar thing for Wayland too:
| https://github.com/udevbe/greenfield
|
| And GTK has yet another way to do this built-in, that uses
| its own protocol:
| https://developer.gnome.org/gtk3/stable/gtk-broadway.html
| Jasper_ wrote:
| OK, if I really want one meme about X11 to die, it's this
| "network transparency" one. Nobody has ever explained to me
| what it means, what its goals are, what constraints it has on
| the system or the protocol. I can't find any definition of it
| in any official X11 documentation, protocol or otherwise.
|
| The typical answer I see is "you can send the same bits
| locally that you do over the wire", but nobody ever explains
| why that's even desirable for a network protocol. Realities
| of distributed systems vs. local ones have a wide variety of
| tradeoffs. Transparent, distributed RPC was tried in the 90s
| and largely judged a failure. We know how to design much more
| resilient systems than that.
|
| Not to mention that ever since the introduction of the MIT-
| SHM protocol, which is used by most everything to transfer
| image data, it hasn't even been true in practice for the last
| 20 years. Most modern enhancements to X11 have been about
| taking the X server out of the loop for as much as possible.
|
| To make X11 usable over the network, even advocates of the
| principle usually use something like NX which is a network-
| designed fork of the protocol, which introduces a large
| number of new features aimed at network that wouldn't be
| required over a network, things like lossy compression,
| multiple independent streams so we don't have reach head-of-
| line blocking with roundtrips, and client-side rendering.
|
| Problems like remote desktop and streaming can be solved by
| other solutions than "shove the same bits over the wire and
| hope they're helpful on the other side".
| welterde wrote:
| NX partially solves the problem that applications don't use
| the X primitives anymore and are way to chatty with the X
| server (which leads to many unnecessary roundtrips which is
| the real killer for remote X11). Old applications work
| great even without NX or any hacks.. just anything modern
| is quite sluggish.
|
| And it's less about the actual protocol, but the desired
| behavior. I want to launch applications on a remote machine
| and have the windows appear on my local machine as if I was
| running them locally. And so far there is nothing except
| X11 that can really do that..
| cycloptic wrote:
| >And so far there is nothing except X11 that can really
| do that.
|
| In terms of Wayland there is Waypipe. And I believe
| Citrix has been doing this forever on Windows, although
| I've never used their products.
|
| In my opinion Xorg by itself can't really do it
| correctly, I find running remote clients to be basically
| unusable without Xpra. And in order to really use that
| effectively, you have to be running all your clients
| within Xpra from the start.
| tgot wrote:
| Network transparency meant that when I was doing Finite
| Element Analysis on the beefiest Sun workstation they sold,
| running a proprietary application, I could log in from
| home, run the application on the Sun, and see the results
| of the prior run.
|
| I could edit the model, regenerate the mesh if needed, and
| kick off another big batch run, and then drop the X11
| frontend GUI.
|
| Over _dial-up_ , from a Windows box to a Sun workstation,
| in the mid 90s. And it was could be secure, tunneled
| through SSH.
| cycloptic wrote:
| I'm assuming that was sending OpenGL 1.0 commands over
| the network, or doing some kind of equivalent. Using X11
| drawing commands for that would have resulted in
| miserable frame rates over dial-up if you ever tried to
| rotate or zoom the mesh. In any case that's not really
| network transparency nor is it really X11, the program
| likely had to be built in a very specific way using
| display lists.
| destructionator wrote:
| It is quite possible it just sent draw line requests as a
| list of vertexes after doing all the math on the client,
| which core x supports just fine.
| cycloptic wrote:
| That takes huge amounts of bandwidth. There's a reason
| display lists were necessary to get it to work on dial-up
| connection.
| ddingus wrote:
| CAD and Simulation Apps were written in those specific
| ways.
|
| You are right.
|
| Where that was done, performance was excellent.
| cycloptic wrote:
| It just seems wrong in that case to to say that X is
| network transparent. The real concern is that OpenGL 1.0
| was capable of running over the network, and in order to
| use it effectively application developers had to take
| network operation into consideration, and the server had
| to support the non-standard extension required to use it
| correctly. In some circumstances using display lists
| locally could actually reduce performance, so the
| application may not have wanted to take that path in all
| cases: https://www.opengl.org/archives/resources/faq/tech
| nical/disp...
|
| Generally if your application has any code that does
| this: if (client is remote)
| ... else if (client is local) ...
|
| Then I wouldn't say the protocol you're using is network
| transparent.
| ddingus wrote:
| Open GL could run over the network because X could.
|
| Sgi used X fully. They wrote the book on all that,
| including GLX, and it was flat out awesome.
|
| The Apps I used all ran display and pick lists. They
| worked very well over the wire and that use case for used
| a lot.
|
| The quibbles are really getting past the basic idea of
| running an app remotely on ones local display.
|
| That happened and worked well and had some advantages. I
| personally built some pretty big systems for modeling and
| simulation that were cake to admin and very effective on
| many other fronts.
|
| Notice I did not say network transparent in my comments
| above.
|
| Multi-use graphical computing is much more accurate when
| it comes to how X worked and what it delivered people.
| umanwizard wrote:
| It means that _today_, I can run an IDE on my 32-core
| home desktop and interact with it seamlessly on my
| 13-inch laptop at the office. I'm not sure whether this
| is possible with Wayland.
| mcguire wrote:
| https://wayland.freedesktop.org/faq.html#heading_toc_j_8:
|
| Is Wayland network transparent / does it support remote
| rendering?
|
| No, that is outside the scope of Wayland. To support
| remote rendering you need to define a rendering API,
| which is something I've been very careful to avoid doing.
| The reason Wayland is so simple and feasible at all is
| that I'm sidestepping this big task and pushing it to the
| clients. It's an interesting challenge, a very big task
| and it's hard to get right, but essentially orthogonal to
| what Wayland tries to achieve.
|
| This doesn't mean that remote rendering won't be possible
| with Wayland, it just means that you will have to put a
| remote rendering server on top of Wayland. One such
| server could be the X.org server, but other options
| include an RDP server, a VNC server or somebody could
| even invent their own new remote rendering model. Which
| is a feature when you think about it; layering X.org on
| top of Wayland has very little overhead, but the other
| types of remote rendering servers no longer requires
| X.org, and experimenting with new protocols is easier.
|
| It is also possible to put a remoting protocol into a
| wayland compositor, either a standalone remoting
| compositor or as a part of a full desktop compositor.
| This will let us forward native Wayland applications. The
| standalone compositor could let you log into a server and
| run an application back on your desktop. Building the
| forwarding into the desktop compositor could let you
| export or share a window on the fly with a remote wayland
| compositor, for example, a friend's desktop.
| cycloptic wrote:
| See waypipe for an example of how this can be done
| "natively":
| https://gitlab.freedesktop.org/mstoeckl/waypipe/
| toast0 wrote:
| > Nobody has ever explained to me what it means, what its
| goals are, what constraints it has on the system or the
| protocol.
|
| In today's world, what I think people want to do (and
| certainly what I want to do) is ssh to a machine, run a
| graphical program on that machine and see and interact with
| it in my graphical environment. X11 with ssh tunneling lets
| me do that. It's not a great experience, for all sorts of
| reasons, but it's an experience, and people who have been
| around for a while remember that it used to work better.
| It's usable enough on many applications to work in a pinch,
| or for ocassional work (I always run mythtv-setup via
| remote X, because it's inconvenient otherwise; sometimes I
| run a graphical browser over remote X to access
| localnetwork only servers from outside my home, but only if
| I can't get proxying to work, because browsers on remote X
| are awful in 2020 in ways they weren't in 2005)
|
| I haven't looked in a while, and maybe waypipe fills this
| gap now, but I think at least some people are frustrated
| because Wayland is supposed to replace X11, but it doesn't
| hit all their use cases, and they may not care that much
| about the things it does better because they were used to
| the old ways. (Of course, if those things include hardware
| support, that will be a forcing function).
|
| I certainly understand your frustration though. You built
| something new and better, and people won't use it because
| they're stuck on something that barely works.
| alerighi wrote:
| It's not a great experience, and it doesn't work at all
| with everything that uses video acceleration (like a
| modern browser, and as a corollary all Electron
| applications).
|
| In contrast Windows RDP protocol works much better, even
| if it's sending the whole desktop as a video stream, and
| works even on modest connections (like a 20mbit/s ADSL).
|
| In fact nowadays I would argue that the method used by
| Xorg is no longer ideal: the commands that needs to be
| sent between the application and the GPU are too big to
| be sent on a network connection. Instead we got much
| better video compression algorithms, so it's better
| instead of sending on the network the commands to draw
| the application to just have the server GPU render the
| application and then send a compressed video stream to
| the client.
| folmar wrote:
| > it doesn't work at all with everything that uses video
| acceleration (like a modern browser, and as a corollary
| all Electron applications).
|
| Works for me. 15 years ago it was a bit more problematic,
| but worked still.
|
| > works even on modest connections (like a 20mbit/s ADSL)
|
| This is not modest by any means. Vnc works good on 56
| kbit, and marginally at 24kbit.
| toast0 wrote:
| > It's not a great experience, and it doesn't work at all
| with everything that uses video acceleration (like a
| modern browser, and as a corollary all Electron
| applications).
|
| I'm posting this with Firefox 79 running on FreeBSD,
| displayed on Windows (VcXsrv) via remote X over ssh (gigE
| lan because I'm at home). Not sure how much modern I need
| to get. chromium-84 also installed and worked enough.
| Scrolling is glitchy, and I don't dare watch a video[1],
| but either would work enough if I need to use a graphical
| browser to administer some network device remotely (I
| _used_ to be able to use w3m for a lot of those)
|
| I'm sure RDP would be better, but it doesn't fit my use
| case. I don't want a whole desktop, just one window, and
| I do need it to tunnel through ssh, and it's nice when
| it's a simple setting, so I can just run a program and it
| shows up. If there's an easy way to tunnel RDP that I'm
| not aware of, I'm happy to learn, especially if the
| experience is better, but if it's super fiddly, I'd
| rather use what I know and what works, even if it's laggy
| and awful.
|
| If RDP worked like X, I'd love to setup a beefy windows
| server in my basement and play games on it via RDP from
| my PCs around the house. If anyone has tips on that,
| would love to read them. (Requirements: single server,
| three simultaneous users, single GPU ideal, but flexible;
| want to play Civ V etc with the family. Cost (including
| licensing) needs to be comparable with three low tier
| desktops (i3-8300 ))
|
| > In fact nowadays I would argue that the method used by
| Xorg is no longer ideal
|
| Again --- I know it's not ideal, it's just less fiddly
| and I already know how to use it, and it how to make it
| work hasn't changed significantly in the last 15 years
| (mostly the windows X servers are better and crash less),
| and desktop applications tend to be more synchronous and
| harder to use over high latency connections. I'm not
| looking to run all my applications from the other side of
| the country, I just want to occasionally run a couple
| things from home when I'm not there, or to setup my HTPC
| with a mouse and keyboard instead of a remote control.
|
| I think everyone will hate me for suggesting it, but I
| could imagine an X11 extension where if the client is not
| local to the server, it would switch to rendering
| everything locally and sending compressed video; I don't
| expect anybody to build that, but there's no reason you
| couldn't do it. It wouldn't solve important issues like
| head of line blocking, of course.
|
| [1] Ok, for giggles, I tried to watch something on
| youtube. It was surprisingly decent. Lots of tearing, and
| I don't know where the audio went (clearly, not over X
| which doesn't do that, not sure if it was playing on the
| X client host), and it took a long time to stop the video
| because I'm guessing a whole ton of frames were buffered.
| But, I got the idea of the video, and it didn't kill the
| session. If I were doing this over my WAN, it would be a
| lot worse, clearly.
| anthk wrote:
| Search for XPRA.
| toast0 wrote:
| XPRA sounds like screen for X? Plus maybe a betterish
| protocol, but also maybe a bit more fiddly?
| mmcgaha wrote:
| Back in the 90s, I had a serial cable under the carpet from
| my computer to my couch. My computer would access the
| internet over PPP and my laptop would PPP over to my
| computer. Then I would launch netscape and browse the web
| from the comfort of my couch.
|
| I have had a couple other use cases over the years like
| virt-manager at work, and a graphical management tool that
| I made for my son's minecraft server.
|
| Sure there were other solutions but X11 over the wire
| worked well.
| AshamedCaptain wrote:
| > Nobody has ever explained to me what it means, what its
| goals are, what constraints it has on the system or the
| protocol
|
| You answer it later on. They want something like NX. We can
| fill in the details as one pleases.
|
| > We know how to design much more resilient systems than
| that.
|
| I think you are overstimating the progress of
| computer/network engineering. Anyway, CORBA's
| "transparency" might be a thing of the past, but what's the
| point here? Unless you plan to make a graphics API
| explicitly asynchronous, what alternatives you have?
| Sending PostScript over the wire? :/
|
| The users want graphic primitives sent through the wire.
| There is not much alternative other than CORBA style RPC.
|
| > Not to mention that ever since the introduction of the
| MIT-SHM protocol, which is used by most everything to
| transfer image data, it hasn't even been true in practice
| for the last 20 years
|
| That is not true. See how many commercial Qt engagements
| were stuck with Qt 4 because Qt 5 initially forced to use
| Qt's local rendering and only pushing the final pixmap to
| the server. Multi-million CAD machines slowing down to a
| halt because the clients were suddenly sending megabits
| through the pipes * . Eventually they had to backpedal and
| Qt5 now offers coreX/Xrender rendering too. _Most_ toolkits
| _to this day_ still render with X by default, and people do
| notice when this changes (see Mate). Sure there are
| exceptions and likely amongside the most used software
| (e.g. browsers), but I'd hardly claim "anything but MIT-SHM
| is all that matters for the past 20 years".
|
| > is a network-designed fork of the protocol,
|
| Yet compatible (to a high degree, at least). Up to the
| point the "transparency" illusion is maintained.
|
| > Problems like remote desktop and streaming can be solved
| by other solutions
|
| The biggest problem is that once you drop the codeflow on
| most opensource programs that renders using X, it's code
| that's rotten forever, and this entire method of "network
| transparency" will never be workable again in the free
| desktop world. Everyone and their mother will draw in a
| different way.
|
| We'll have to resort to full video streaming or the like
| that have their own share of disadvatanges. There is
| absolutely no technical reason behind this, it's just a
| "code is done by volunteers" problem.
|
| * And please don't say VNC/video here, since this is CAD.
| It's mostly a several megapixel black bitmap with a couple
| million white lines rendered on it. Relatively easy for a
| "drawing primitives over the wire" solution, insane for
| anything that tries to encode it as realtime video.
| cycloptic wrote:
| >See how many commercial Qt engagements were stuck with
| Qt 4 because Qt 5 initially forced to use Qt's local
| rendering and only pushing the final pixmap to the
| server. Multi-million CAD machines slowing down to a halt
| because the clients were suddenly sending megabits
| through the pipes
|
| Is there a bug report for this in the Qt bug tracker that
| you can share?
|
| >It's mostly a several megapixel black bitmap with a
| couple million white lines rendered on it. Relatively
| easy for a "drawing primitives over the wire" solution,
| insane for anything that tries to encode it as realtime
| video.
|
| I think your math is off. In the X11 protocol, each line
| segment is transmitted as 8 bytes. Transmitting two
| million white lines uncompressed is going to take up 16
| megabytes. Do that 60 times a second and now you have a
| bitrate of 960MB/s. Compare this to youtube's
| recommendations which suggest a maximum bitrate of around
| 10MB/s for 4K HDR 60FPS video:
| https://support.google.com/youtube/answer/1722171
|
| To me, the compressed video solution is the clear winner.
| AshamedCaptain wrote:
| > Is there a bug report for this in the Qt bug tracker
| that you can share?
|
| "A" bug report? There is like tens of them. A 5 minute
| search reveals many:
|
| Main bug:
|
| * https://bugreports.qt.io/browse/QTBUG-56851
|
| Similar bugs:
|
| * https://bugreports.qt.io/browse/QTBUG-67170
|
| * https://bugreports.qt.io/browse/QTBUG-47069
|
| Bugs already filed back in qt4 release when the default
| was changed (but the x11 code could still be enabled):
|
| * https://bugreports.qt.io/browse/QTBUG-19636
|
| * https://bugreports.qt.io/browse/QTBUG-25761
|
| People complaining whenever the new X11 render path gets
| broken on qt5:
|
| * https://bugreports.qt.io/browse/QTBUG-70387
|
| In stackoverflow, plenty:
|
| * https://stackoverflow.com/questions/22017200/is-there-
| a-way-...
|
| * https://stackoverflow.com/questions/28782162/serious-
| perform...
|
| Bringing back the option to draw using X:
|
| * https://twitter.com/peha23/status/1037294277859131392
|
| * https://sourceforge.net/p/qwt/mailman/message/36418978/
|
| > compressed video solution is the clear winner.
|
| This is enlightening, but it is still comparing apples to
| oranges, or in this case, compressed to uncompressed. The
| cost of encoding video cannot be disregarded. And for
| starters, even if you use hardware encoding, they hardly
| do any type of variable bitrate; you are likely to get
| almost the same bitrate even if the screen only changes
| slightly (good luck sending partial damages to a hardware
| encoder). On the other hand, NX compresses all traffic by
| default, and so does SSH (but not by default).
|
| Not to mention that compressed video at this rate would
| show artifacts. Even at bluray rates artifacts can be
| seen in motion film; let's not even get started what you
| would see for this type of content.
|
| Do note that I am highly biased here since we (my
| company) constantly evaluates potential solutions almost
| every other week to do remoting (e.g. RDP/extremeFX,
| PCOIP, Blast, you name it) , and _every_ _single_ _time_
| customers end up falling back to an ANCIENT NX version
| (cause the newer ones also use video "sometimes") which
| is so full of bugs I consider it the a modern equivalent
| of torture. (By the way, xpra, freenx, and every other
| open source NX fork are also the same -- ancient NX 3.xx
| versions). I really would like to kill that with fire but
| there's absolutely nothing comparable in terms of
| (in)sensitivity to high-latency links, low bandwidth
| usage, image quality, and even economy (no GPU video
| encode farm thanks).
| cycloptic wrote:
| Thank you for those bugs. They seem to further the point
| that X11 is not network transparent, seeing as how the
| solution was for Qt to support two separate xcb backends,
| one for local rendering and one for remote rendering. And
| actually a third for local hardware rendering using GL.
|
| I have no comment on NX, I've never used it.
| AshamedCaptain wrote:
| Qt _needs_ a non-X11 backend, seeing that it runs on
| platforms where X11 is not prevalent. Network
| transparency is another story and not related whatsoever
| to this since even if you push bitmaps X11 is still
| technically network 'transparent' (the quotes are due to
| the increased bandwidth usage).
| kjs3 wrote:
| I'm not clear on what you're missing. What meme? The
| protocol is well documented. The basic abstraction is that
| there is a separation between the 'processing' part of an
| application and the 'display' part of the application and
| those two parts communicate using the X11 protocol. That
| conversation can happen over a local or a remote socket, or
| shared memory. The practical use cases are that I can have
| an application running on my local machine, or one on the
| other side of the world, and the 'display' portion is
| consistent. Not much different than, say how Citrix uses
| it's own protocol to draw remote displays.
|
| It allowed a 'mainframe' model where you could have a very
| fast, expensive application servers that could display to
| less expensive local resources, and mix the display with
| local applications. So my heavyweight applications could
| run on the monster Sun or Cray in the datacenter, but my
| clock, calc and notepad app ran locally, but all in the
| same display.
|
| Yes, over the years things like MIT-SHM and NX came along
| to paper over problematic issues in X11. And yes, there are
| other, possibly better ways so solve it now. But in the
| late 80s and through the 90s, it was pretty cool stuff.
| Jasper_ wrote:
| It's possible I'm a bit jaded because I spent a long time
| working on things like Wayland and next-generation
| graphics technology, where people would suddenly show up
| and name-drop the terminology "network transparency" as
| if it meant something, or solved useful problems.
|
| I just find it funny that CORBA-style RPC is now
| considered a dated, bad approach in the world of
| distributed systems, but the same thing in the X11 world
| is considered way ahead of its time and a design ideal.
| formerly_proven wrote:
| > I spent a long time working on things like Wayland and
| next-generation graphics technology
|
| I don't know you, so this might come off as overly
| aggressive. But very little that happens in the
| FOSS/Linux space and has to do with graphics I would
| describe with "next gen". Wayland for example is meant as
| the "ring to rule them all" as a way to drive displays,
| yet it doesn't even support color management, so it's
| increasingly useless in a world of non-sRGB displays. X11
| does color management (in a cludgy way), for example.
| Jasper_ wrote:
| Linux graphics is mostly driven by volunteer work. There
| is ongoing work to try and get color management done
| properly (though color management is an extraordinary
| difficult topic and you could make the argument _no_
| display system gets it correct)
|
| https://lists.freedesktop.org/archives/wayland-
| devel/2019-No...
| sadfklsjlkjwt wrote:
| > I spent a long time working on things like Wayland and
| next-generation graphics technology
|
| This is really sad. You are insulting and dismissing
| technology which did something well over 20 years ago
| that you think is not possible today. And instead of
| thinking "hmm maybe I should look at the older technology
| and see how it did it" you just dismiss it as a "meme"
| and tell people that the actual technology they used is
| impossible. So much effort is wasted in our industry
| because of NIH syndrome...
|
| I seriously hope that the Wayland people didn't just
| completely ignore the positives of Xwindows!
| Const-me wrote:
| > which did something well over 20 years ago
|
| Not sure that was ever true. 20 years ago, MS remote
| desktop (called "terminal server" at that time) was
| vastly superior to X11 for remoting.
| mcguire wrote:
| " _RDS was first released as "Terminal Server" in
| "Windows NT Server 4.0 Terminal Server Edition" a stand-
| alone operating system including SP3 and fixes built in.
| Starting with Windows 2000, it was an optional role and
| became a mainstay of the Windows NT family of operating
| systems and was improved with each version of Windows.
| The rename to "Remote Desktop Services" occurred with
| Windows Server 2008 R2 in 2009._"
|
| X predates Windows NT by more than 10 years and is a
| completely different beast than remote desktop systems.
| (Disclaimer: I used to log into a local box, run an
| unpopular window manager on a remote server, and then
| spread my workload over a number of remote machines.
| Terminals, email, gui apps, and yes, the cheesy web
| browser of the time. Seamlessly. (No desktop-in-a-window.
| Same files everywhere (Thanks, NFS!), same apps (more
| work to set up than in Plan 9, but didn't have Plan 9's
| disadvantages, either) everywhere. And that was half a
| decade before Windows NT.)
| wahern wrote:
| Sometimes I wonder if people who claim RDS is superior to
| X11 have even tried both.
|
| Of course, Terminal Server was more than just for
| display. It setup things like remote disk access, whereas
| on Unix NFS required separate configuration; likewise for
| Active Directory vs NIS, etc. In terms of ease of
| administration of remote desktop environments, I can see
| Terminal Server being superior for sysadmins. But nothing
| beats being able to do `ssh -X host application`. And
| while ssh deserves some credit, at a technical level
| (setting aside security cookies, which is a more recent
| thing), it's just a dozen or so lines of C code in the
| codebase to setup the X11 channel. But that says more
| about the design of SSH and X11 and the ease of creating
| arbitrary point-to-point channels. And of course on
| trusted LANs, where X11 was originally used, things were
| much easier, even in terms of administration relative to
| Terminal Server.
|
| More importantly for users, even over SSH X11 has
| _always_ been indisputably more responsive for me than
| RDS, let alone VNC. Even on low-latency links, unless you
| 're getting huge amounts of packet loss, RDS and VNC
| still suck in _all_ of my experiences compared to X11.
| (And I know because even in those jobs where I had to use
| a Windows laptop I always installed an X11 server for
| Cygwin apps and remote apps on my own server.)
| Jasper_ wrote:
| I wrote the linked article explaining X11's internals and
| design goals in a neutral manner. I've worked on X11 and
| Xorg ecosystem, worked on the server, I worked on the
| window managers, the toolkits, applications, compositors,
| you name it.
|
| I think I'm allowed to have opinions with 40-year
| hindsight saying that some design decisions have not aged
| well. I'm not saying X11 is a meme; I'm saying the words
| "network transparency" ( which never appears once in the
| documentation; seriously, try to find it https://www.x.or
| g/releases/current/doc/xproto/x11protocol.ht... ) are
| unhelpful, have unclear motivations and goals, and have a
| massive restraining influence on what a modern protocol
| can look like and do.
| mcguire wrote:
| Aaahh! I get it now.
|
| Network transparency: you can run a process remotely and
| see the same interface that you would have seen if you
| had (or had been able to) run it locally. Nothing more,
| nothing less. Mostly marketing.
| erik_seaberg wrote:
| We're so desperate for network transparency that we're
| abusing hypertext _web browsers_ as application
| platforms, because how else can a machine in a datacenter
| put a GUI on an end user 's screen today?
| michaelmrose wrote:
| https://www.omg.org/gettingstarted/history_of_corba.htm
|
| CORBA 1.1 (February 1992) This was the first widely
| published version of the CORBA specification. It closed
| many ambiguities in the original specification; added
| interfaces for the Basic Object Adapter and memory
| management; clarified the Interface Repository, and
| clarified ambiguities in the object model.
|
| https://en.m.wikipedia.org/wiki/X_Window_System
|
| X originated at the Project Athena at Massachusetts
| Institute of Technology (MIT) in 1984.[1] The X protocol
| has been at version 11 (hence "X11") since September
| 1987.
| mcguire wrote:
| I'm afraid I'm dreadfully confused by your comment.
|
| " _The typical answer I see is "you can send the same bits
| locally that you do over the wire", but nobody ever
| explains why that's even desirable for a network
| protocol._"
|
| Yes, you can use a network protocol to chat between local
| processes as well as between remote processes. Desirable?
| It's impossible to avoid...
|
| " _Transparent, distributed RPC was tried in the 90s and
| largely judged a failure._ "
|
| True (ahem[1]), but irrelevant. X isn't a remote procedure
| call system.
|
| " _Not to mention that ever since the introduction of the
| MIT-SHM protocol, which is used by most everything to
| transfer image data, it hasn 't even been true in practice
| for the last 20 years. Most modern enhancements to X11 have
| been about taking the X server out of the loop for as much
| as possible._"
|
| This is true; network speed hasn't kept up with graphics
| power and requirements for the most part, so processing has
| been pushed to the client. On the other hand,
| HTTP/HTML/JS/CSS can be seen as a poor-man's
| reimplementation of X (or more closely, of Plan 9 client-
| ui/server-processor protocols). And for the 20 years
| previous, clients were underpowered in comparison with a
| big honking machine you could log into remotely (or at
| least a dedicated machine that didn't have to bear the
| weight of a graphical interface).
|
| " _To make X11 usable over the network..._ "
|
| X11 has been usable over the network since 1987 (and X < 11
| before that) (https://en.wikipedia.org/wiki/X_Window_System
| #Origin_and_ear...).
|
| " _...even advocates of the principle usually use something
| like NX which is a network-designed fork of the
| protocol..._ "
|
| I'm not sure what you mean be NX. The...
|
| "NX technology, commonly known as NX, is a proprietary
| software application for remote access, desktop sharing,
| virtual desktop on Linux and file transfer between
| computers. It is developed by the Luxembourg-based company
| NoMachine which also lends its name to the software."
| (wikipedia)
|
| which has nothing to do with X other than being descended
| from one of the low-bandwidth X extensions. (Not the low-
| bandwith X, and low-bandwith means dial-up. Plain ol' X was
| pretty chirpy on a SPARCstation with 10MB ethernet.)
|
| " _...multiple independent streams so we don 't have reach
| head-of-line blocking with roundtrips..._"
|
| Welcome to the wonderful world of TCP.
|
| " _...and client-side rendering._ "
|
| Once upon a time, there was a thing called a Sun Ray
| (https://en.wikipedia.org/wiki/Sun_Ray), basically a
| (non-3d) graphics card glued to an ethernet card, that was
| used as a thin client for a Sun server. I didn't see one
| until after I'd used an IBM X-terminal (a smallish pizzabox
| that ran the X server and pointed everything else at a
| bigger IBM box) for several years and the latter was much
| better than the former.
|
| [1] Protobuffs, et al., notwithstanding.
| Jasper_ wrote:
| I'm saying that "a local protocol that can function as a
| networked protocol" isn't always an ideal design
| decision. We should consider designs that allow different
| local and remote protocols, and consider the tradeoffs in
| each space separately.
|
| > Yes, you can use a network protocol to chat between
| local processes as well as between remote processes.
| Desirable? It's impossible to avoid...
|
| Using a network protocol to chat between local processes
| comes with a lot of drawbacks. UNIX local sockets support
| features like datagrams and FD passing, and of course you
| can communicate resources in a shared namespace (e.g. a
| filesystem paths). So limiting yourself to the lowest
| common denominator of a TCP socket restrains the kinds of
| tools your protocol can make use of.
|
| > X isn't a remote procedure call system.
|
| It has requests, and it has replies. Xlib will block on a
| reply by default. That's the building blocks of a classic
| RPC system. It has all the messiness of distributed
| systems, but none of the innovations from the past 40
| years. Error handling is particularly gnarly in Xlib;
| since all resources are global, another client can stomp
| on or delete your window behind your back. You can't be
| sure of anything before you try, and get an error back.
| In theory, you can make a lot of requests without
| blocking, but that's a super annoying process and isn't
| easy to do in practice; lots of information requires
| getting a reply back from the server.
|
| > which has nothing to do with X other than being
| descended from one of the low-bandwidth X extensions
|
| It has a lot to do with X11! It's championed as an
| example of the success of X11's "network transparency",
| but instead is very much a custom design suited for the
| network.
| asveikau wrote:
| All of that said, "ssh -Y" works pretty well for me. It's
| pretty usable.
|
| And from a user perspective I personally like the
| "application connects to render" model much better than VNC
| or RDP style entire-desktop streaming.
| spanhandler wrote:
| IIRC it used to work a lot better (like, a _whole lot_
| better) but modern X apps mostly end up sending bitmaps
| over the wire, rather than native X rendering
| instructions.
|
| Agree that it works better than other remote desktop
| options on Linux. Overall though I get better performance
| with RDP to Win10 running Linux in Virtualbox (!) than
| any other option I've tried though, including Linux on
| bare metal with a variety of remote desktop options, and
| Virtualbox's remote desktop server.
| ddingus wrote:
| Not that anyone feels it is important today, but X11 is
| multi user graphical computing.
|
| A display server can display anything any user connected to
| it wants to display.
|
| Actual 90's era use cases:
|
| Application server. One copy of 3D CAD, one database, 30
| users all running on a beefy machine well up to the task.
|
| Each ran via X talking to the app server. Admin was cake,
| data big model access was fast, no user could actually
| touch the data except through use of the program.
|
| User having trouble, push a window onto my display. I fix
| it, close window.
|
| Display wall, conferencing. Big wall display, multiple
| users display data on it, one management user handling
| windows, etc...
|
| Display wall, many small displays, each running via
| workstation. App sends display data to all of them.
|
| Distributed computing resources:
|
| Once, just for fun, I had an application data shared via a
| machine running nfs, fonts served by another machine, yet
| another machine running the application, yet another
| handling window management, all displayed on yet another
| machine serving the display and taking input.
|
| (BTW, it was possible to literally kill the window manager
| and launch a different one and the display would shift a
| bit and the user continued along still running their app,
| but now using a different window manager!)
|
| User could click on an icon, all those machines contributed
| to the app experience. Was hard to tell the difference
| between that and it running all locally.
|
| Why?
|
| Linux user wanted to run CAD unavailable on Linux, was
| developer who wanted a Linux box. So I used X to push the
| whole experience to their machine and distributed it for
| fun once it was working well just to learn and really push
| things, frankly.
|
| While running, it was hard to tell the Linux machine was
| not an expensive commercial Unix machine.
|
| All that was over 10-T and 100-T ethernet BTW.
|
| Well designed X applications, particularly those using the
| SGI GLX extensions ran very well on local LANs. Poorly
| designed ones were nowhere near the quality of experience.
|
| That skill was expensive. I think this is a big part of why
| so many struggled with X. A browser, for example, would not
| be comparable, pretty terrible.
|
| Displaying web pages sucked, yet modeling a freaking car
| was amazing!
|
| At the time, the X implementation of multi user graphical
| computing allowed for these and other combinations of
| users, machines, displays and applications.
|
| It was well thought out multi user graphical computing!
|
| The users could be local, as in a multi head machine
| offering two sets of keyboard, mouse, display, or they
| could arrive over the network.
|
| Same as the Unix it was built on top of.
|
| I think that is what people really are trying to get at
| when they say "network transparency"
|
| Not all these use, or the set of possible use cases X made
| possible, make sense today.
|
| But, when it was done?
|
| Nobody knew, so they made a system as capable and that
| employed the best ideas in multi user computing, so that it
| was all possible.
|
| That's why people who got to understand and really use X
| appreciate it as we see them do today.
| david_draco wrote:
| I wish hacking on X and getting patches in had the same coolness
| factor as hacking on the Linux kernel. The hacking complexity is
| certainly comparable. X is used by a very large number of people,
| and contributions can make a big difference.
| badsectoracula wrote:
| FWIW i played around with Xorg xserver code a while some time
| ago and i found it quite easy to grasp things in it... or at
| least not any more difficult than any other relatively large
| codebase i've worked with (though the X server doesn't really
| have a large codebase and the entire Xorg project is broken
| into multiple small libraries that make things easier to
| grasp).
|
| My guess why you do not see as many people working on Xorg is
| that... it generally works for the majority of people.
| Personally i'm 100% certain i could fix any issue i have with
| it, but i never really felt anything missing from Xorg, it
| works perfectly fine for what i'm doing with it.
| AshamedCaptain wrote:
| The difference is that (sadly) no one seems to care much about
| desktop Linux. Only Android this Android that Android those...
| cycloptic wrote:
| There's still a good number of graphics hackers, but a lot of
| code has moved out of Xorg and as a result most of the hackers
| have moved on to working on Mesa, Wayland, and in-kernel GPU
| drivers. So there is some overlap there with the kernel
| hackers.
| [deleted]
| drewg123 wrote:
| I remember my undergrad graphics class in 1991 or so. We did it
| in C and X11 on Sun and DEC workstations, and we had to learn X
| as an implementation detail. Our final project was animated 3-d
| wireframe models of objects, written in Xlib primitives. No
| toolkits or modern libraries. One of the assignments was a
| "game", and I wrote a tetris-like typing game. The projects in
| that class were insanely hard compared to any other class I'd
| taken.
|
| The amazing thing is that, at least as of 10 years ago, my
| projects for that class _STILL_ compiled on 32-bit FreeBSD and
| Linux.
|
| Oh, and to learn X11, we used
| https://www.amazon.com/gp/product/0134999975/ref=x_gr_w_bb_s... I
| still remember sitting on a bench outside the college bookstore,
| cracking that book open and feeling both excited and hopelessly
| lost.
| formerly_proven wrote:
| > The amazing thing is that, at least as of 10 years ago, my
| projects for that class STILL compiled on 32-bit FreeBSD and
| Linux.
|
| i686 binaries from the 90s compiled for then-Linux and linked
| against then-xlib _still_ run on AMD64 Linux hosts (provided
| you have the necessary multilib libraries around).
| dang wrote:
| If curious see also
|
| 2019 https://news.ycombinator.com/item?id=21039848
|
| also 2019 https://news.ycombinator.com/item?id=21118633
|
| 2016 https://news.ycombinator.com/item?id=12042548
|
| 2014 https://news.ycombinator.com/item?id=8019346
|
| 2013 https://news.ycombinator.com/item?id=6978274
| SXX wrote:
| Really recommend that speech as well since it's give good
| understanding what's wrong with X11 and why Wayland was created.
|
| The real story behind Wayland and X:
|
| https://www.youtube.com/watch?v=GWQh_DmDLKQ
| Jasper_ wrote:
| Author here. I haven't updated this project in a few years. I
| burned out on it almost immediately. I wrote a little
| retrospective for it on here before, the last time it was posted:
| https://news.ycombinator.com/item?id=21041340
|
| In the days since, I've joined the games industry, fell in love
| with it there, and am working a bit more on more video game-
| focused content:
|
| https://noclip.website
|
| https://www.youtube.com/watch?v=mnxs6CR6Zrk
|
| https://www.youtube.com/watch?v=8rCRsOLiO7k
|
| If anybody has any questions about game graphics, Linux graphics,
| or X11, feel free to ask.
| simlevesque wrote:
| I had never heard of noclip.website... our office stopped for a
| couple minutes.
| pandatigox wrote:
| Amazing work! I can understand how you can burn out on it
| though. The explanations are very friendly and easy to follow,
| so don't be deterred if you want to create more EXPLANATION
| content!
| Jasper_ wrote:
| I have no desire to continue the Explanations series. The
| videos are where I'm spending my focus now.
| rpmuller wrote:
| Thank you for putting this together. This answered many of the
| questions I've had about X for multiple decades.
| dang wrote:
| What year do you suggest we put in the title above?
| https://news.ycombinator.com/item?id=21039848 said 2017 but
| clearly the project goes back earlier.
| Jasper_ wrote:
| The first article (X Window System Basics) was written in
| 2013. The last article (Basic 2D Rasterization) was written
| in 2017. Take your pick.
| dang wrote:
| I pick 2017.
| voltagex_ wrote:
| OT: Noclip is amazing, thank you. It's sorely needed in an
| industry that's (mostly unintentionally) very good at erasing
| its history.
| exikyut wrote:
| I just checked Xplain out and wow, it's a bit expanded from how
| I remember it from a few years ago, I'm definitely going to be
| fine-tooth-combing through it at some point, particularly the
| section on regions which looks like it'll help me crack the
| overlapping-window problem. The rasterization section also
| looks like it'll be fun to "??!?" through, thanks for making
| that!
|
| (Compositing was also interesting to look through; I always
| thought it was extremely curious how no X11 windowmanagers ever
| did hybrid compositing, where they only composited minimized
| windows into thumbnails but left actual open windows alone
| without inducing [?]1 frame of extra latency. I'm very curious
| if Wayland can avoid that, but that's just idle curiosity at
| this point.)
|
| NB. noclip.website is throwing "program not valid" from
| `this.gl.useProgram(e.gl_program)` (line 100,924 :D) with a HD
| Graphics 3000 on oldish i5-3360M (HP 8470p). Chances are it
| needs a beefier GPU.
| Crespyl wrote:
| Regarding "hybrid" compositors, I believe there used to be a
| few that had an "un-redirect fullscreen windows" feature that
| essentially did what you're suggesting. I think there was
| some sort of difficulty with doing this for regular/non-
| fullscreen windows and still having a consistent/smooth
| experience with the rest of the features you'd want out of a
| compositor.
|
| These days, at least in KWin land, there's just an "allow
| applications to block compositing" feature that automatically
| disables compositing (globally) when it makes sense. I'm not
| sure exactly what rules trigger it (simply putting, say,
| FireFox, into fullscreen doesn't do it, but Dota 2 does), but
| it seems to work pretty well.
| Jasper_ wrote:
| SDL2 sets this on windows by default, which is likely
| what's doing it:
|
| https://specifications.freedesktop.org/wm-spec/wm-spec-
| lates...
|
| Compositing is required for any sort of antialiasing,
| especially including window borders, so it's difficult to
| turn off without having a lot of artifacts. And it also
| solves vsync which a lot of raw X11 applications can't
| possibly render correctly with.
| Jasper_ wrote:
| Nothing I can do about old GPUs... I need a working WebGL
| implementation. You can try a simpler game like Mario Kart DS
| to see if that works.
___________________________________________________________________
(page generated 2020-08-18 23:00 UTC)