[HN Gopher] The Forgotten Operating System That Keeps the NYC Su... ___________________________________________________________________ The Forgotten Operating System That Keeps the NYC Subway System Alive (2019) Author : lproven Score : 112 points Date : 2022-10-07 22:02 UTC (2 days ago) (HTM) web link (www.vice.com) (TXT) w3m dump (www.vice.com) | tibbydudeza wrote: | If it takes years and billions to build a simple high speed rail | line from San Francisco to LA then NYC subway this will still be | around in it's current state. | haltingproblem wrote: | The articles makes vague complains and leaves many questions | unanswered. | | First, I don't get the primary complaint in the article. The MTA | chose OS/2 in the early 1990s, when there no other choices and it | has served the MTA for almost 30 years with minimal hassles of | upgrades or outages. I can't recall a single day when the | Metrocard was not working - sure there are problems but it is | more a UI/UX issue - a app that shows you Metrocard usage and | balance would be insanely useful but we lack small mercies but | hey you can pay with your smartwatch /s. | | A piece of technology that is stable for 30 years is remarkable. | Even if they had gone with Linux/NT the upgrade cycles themselves | would have wreaked havoc. OS/2 seems like a brilliant choice in | that regard. | | Questions left unaddressed: | | - What tech did they move to now with contactless payments? | | - The move to contactless payments is awesome but what happens to | people who lack smartphones or do not have credit cards or <big- | tech>-pay setup on their phones. Or just casual tourists visiting | from overseas with no US cards. How do they pay? Do they keep | supporting metrocards forever? Does that mean to support | Metrocards they need OS/2 and assorted new systems. | | - What is the mainframe part of the stack and what function does | it do. | cnorthwood wrote: | I was recently in New York as a Brit and was really happy to | see the OMNY system which just allowed me to use contactless to | traverse the turnstiles (for everything but the JFK SkyTrain) | with no issues, similar to how metro transport works in the UK | for example. I don't think there's a requirement for it to be a | US card, but instead any contactless card will work? | [deleted] | wikibob wrote: | As it turns out: | | "The OMNY system is designed by Cubic Transportation Systems, | using technology licensed from Transport for London's Oyster | card." | | https://www.cubic.com/news-events/news/cubic-wins- | contract-n... | em-bee wrote: | NT maybe, but linux not so much. that linux machine would still | be running. and don't argue security patches, because OS/2 is | not getting those either. | haltingproblem wrote: | The system went live in 1993 so what distro would they have | chosen in ~1990, when the system choice was made and is that | distro still around? How many ports would they have had to | make over the ensuing years. | | I love Linux and use it everyday but it would have been a | poor choice in 1990 and some would argue would still be a | poor choice today given other industrial OS-es on the market. | marcus0x62 wrote: | Seeing as Linux didn't exist in 1990, it would have been an | exceptionally poor choice. Early distros were pretty rough, | but honestly by 93/94 it was reasonably useful and, if you | didn't have defective hardware, which was a huge problem at | the time, quite stable. | 0x445442 wrote: | OS/2 failed because of IBM's abysmal marketing, not because it | was inferior tech. | readthenotes1 wrote: | And 18 floppy discs to install... | queuebert wrote: | Back then 18 floppies added gravitas. Only big important | things would take up this many disks. | Aaargh20318 wrote: | There was a time when OS/2 Warp was marketed heavily in my | country (the Netherlands) and there was quite a bit if hype | around it. If you bought a new PC at the time it came with both | OS/2 and Windows 3.11. | | The main problem with it was that is was basically unusable on | consumer hardware at the time. We got a new 80Mhz 486 with 4MB | RAM around that time and it was just incredibly slow, while | Win3.11 ran just fine. | | With hardware advancing at an incredible pace back then, it | might have had a better chance it they had delayed it by a | year. | throwawayForMe2 wrote: | Back when the company I was working for was trying to decide | between OS/2 and Windows NT we took a trip to both Microsoft | and IBM to get their pitch. | | Microsoft had us meet in Redmond, gave us a tour, free books | from their bookstore, brought in an Indian chef for lunch (our | boss was Indian), gave a big splashy presentation, etc. | | IBM met us in a dingy conference room in LA, with a whiteboard, | don't even remember any coffee or bagles. The two guys they | sent came across like bad car salesmen, and we were a big IBM | customer. | aidenn0 wrote: | I've noticed a pattern where some sales teams treat their | existing customers poorly because they assume they are | already locked-in. | tb_technical wrote: | That's flipping wild to me. I work in the evil defence | industry (some sarcasm here). | | They're so vigilant about bribery that when someone (a | general, or someone important) goes to a contractor | presentation it's *expected* that there's a little can they | put fifty cents or a few bucks into to pay for their share of | the bagels. | | And if this can doesn't exist? Everyone gets very | uncomfortable very quickly - _literal jobs are at stake_. | | Other industries are just wild, man. | walrus01 wrote: | apparently that DoD employee who went to boeing after | deciding the tanker contract didn't get that memo - as I | recall she later went to prison. | tb_technical wrote: | The revolving door problem is another unique form of | corruption that military contracting has to deal with. | | Somewhat related, to get my current job I had to sign a | bunch of conflict of interest forms because I jumped | ships from one side to the other. So some controls exist | for the behaviour you brought up! | | I'm not saying this stuff never happens, either, I'm just | remarking that the professional standards and | expectations are very different in military contracting | versus software. | ethbr0 wrote: | I think one of the reasons it's a bigger issue in future | military systems contracts is that they're usually art- | of-the-possible. | | So there's a lot of arbitrary requirement shifting | between actual requirements and as-bid program | requirements... which invites arbitrary requirements | adjustment on behalf of the bidders, but that can be | explained away as {insert semi-justified reason here}. | behringer wrote: | Yep that was my thought. This guy and his company fell for | literal bribery. | noirbot wrote: | I don't think you're wrong, but I do want to push back | against this sort of bribery being entirely illegitimate. | There's absolutely something to be said for a company | partner that cares enough to know how to bribe you and | cares enough about your business. | | Even if the financial part is 100% bribery, there's a | difference between "here's a check so you pick us instead | of the competition" and spending the effort to work out | what your company/decision makers care about and | providing that. | | It's like saying if I plan a date with someone at a | concert I know they're into that I'm bribing them into a | relationship. In a literal sense, yes, but having someone | care enough to cater to you isn't exactly meaningless in | terms of deciding who to pursue a longer-term | relationship with, personally or in business. | mcculley wrote: | My first professional job was working as a DoD contractor. | Having the anti-bribery training beaten into me did not | prepare me well for doing business with the private sector | and in other countries. I had to learn that there are | completely different rules and expectations. | GauntletWizard wrote: | That's exactly it - It's not illegal to take bribes as an | officer of a corporation; it's illegal to defraud others | while doing so. | | If you take a personal bribe to tank your business, | that's between you and your business partners - a civil | matter unless your business partners are John Q public | (Which is why the SEC exists). It is entirely legal to | enrich yourself at the expense of a business you wholely | own, with the obvious caveat that the courts will pierce | the veil if you've committed crimes as that company. | bombcar wrote: | Yeah, people assume that Microsoft just somehow "won" by | default, but they were young and VERY hungry back then, and | it showed. | ohjeez wrote: | If you want an in-depth examination of OS/2's history... | https://medium.com/@estherschindler/os-2-is-25-years-old-201... | russellbeattie wrote: | Hey now! You're talking about me! (Not really). My first job | out of college was as a temp for a marketing group in Boca | Raton, FL where I had moved briefly to get away from New | England weather. I was tasked to write "OS/2 Success Stories" | which were used to create white papers and other marketing | material. This was years before the OS/2 Warp release, which | was, I agree, truly abysmally marketed (that logo!!). The only | thing I can remember solidly was that Traveller's Insurance | used OS/2 and I wrote glowing details of how happy they were | with it. But other financial companies come to mind too. Chase | maybe? Oh, and that installing OS/2 in the days before CD ROMs | required a stack of easily 20 floppy disks or more... You'd pop | them in, one after another, listening for a beep to tell you | when to switch, for what seemed like hours. | | Even though it wasn't a technical job per se, I learned to | create a little Lotus Notes database to help us keep track of | the various companies and more importantly, got my own copies | of the rare developer manuals which were only available to | licensees of Notes. Between that small bit of knowledge and IBM | on my resume (a _very_ big deal even in the early 90s), I was | able to bald-face my way into getting a programming job and | starting my career. | pengaru wrote: | More like Microsoft had a stranglehold on the consumer PC | market already with MS-DOS and everything was so | broken/incompatible at the time it was practically a foregone | conclusion whatever GUI MS shipped in subsequent versions of | MS-DOS would own the PC market. | | OS/2 only made it worse for IBM by shipping DOS+Windows | compatibility, so basically zero developers wrote OS/2-native | applications. This gave Microsoft nothing but time to get their | Windows stability and multitasking performance up to snuff with | NT. | | In hindsight I feel like the dismal PC situation in terms of | constant crashes/need for reboots in pre protected-mode MS-DOS | and combinatorial explosion of incompatibilities created by all | the random ass hardware peripherals conditioned everyone to | fear more change and diversity. If the software being shipped | on the computers at the time couldn't make things work well, | and these were allegedly _the_ experts most qualified to do it | right, what will things be like with some alternative software | attempting to emulate the same stuff in a "compatible" way? No | way, OS/2 was doomed. And I'm saying this as someone who ran | OS/2 WARP for years before discovering an Infomagic 4-CD Linux | set in my teens. | bink wrote: | Back then the OS that shipped with the PC determined what 99% | of people would run (much like today). Microsoft used their | burgeoning monopoly to make sure that no one shipped a non- | Microsoft OS with any PC. OS/2 was so much better than | anything that shipped with any PC I bought at the time, but I | remember the cost being prohibitive for a student. | tzs wrote: | > OS/2 only made it worse for IBM by shipping DOS+Windows | compatibility, so basically zero developers wrote OS/2-native | applications. | | Practically zero developers wrote for OS/2 because IBM made | it hard to do so. I remember something Jerry Pournelle wrote | about his experience at a major trade show (COMDEX?) that IBM | and Microsoft both attended, after Warp was out and I think | when Windows 95 was still in beta. | | At the IBM booth he told them he'd like to do some OS/2 | development and asked how to get started. They gave him a | form to fill out to apply for their developer program. They | wanted details on what he wanted to develop, his business | plan, and stuff like that. If that was all acceptable to IBM | then they would let him to buy their expensive SDK. | | Then he went to the Microsoft booth and said he'd like to | develop for Windows 95. They handed him a CD-ROM with the | SDK, tools, and documentation. | | Later, after Windows 95 was released I remember going to a | retail software store (Egghead) and in the developer software | section there was nothing at all from IBM. Microsoft on the | other had everything one needed to start development there. | Watcom C/C++ was also there, which supported DOS, Windows, | OS/2, Netware, and major DOS extenders, so at least you could | get some OS/2 developer tools at retail. | | And if you did go through all the hoops and develop something | for OS/2, IBM didn't seem to care. Apple and Microsoft were | always on the lookout for 3rd party programs that could | promote their platforms and would run ad campaigns featuring | them. They'd pay stores to put such software in better | locations. | | (That same not bothering to market applied to the OS itself. | The reason when when you walked into a major software store | you would be greeted with a front of store Windows 95 display | while OS/2 was on a bottom shelf somewhere in the back of the | store most of the lights were burned out and the remaining | ones were flickering in a way that made you nauseous is that | the stores sell shelf space. Microsoft paid for premium | space. IBM did not). | | They also did a terrible job of hardware support, particular | video card support. I wanted to quad boot between Windows 95, | Windows NT, OS/2 Warp, and Linux. | | I had to go through something like half a dozen or more cards | video before I found one that worked well enough with OS/2 to | support a decent resolution and frame rate. I had expected | Linux to be the one to give me the most trouble, but it | wasn't too bad. | | IBM should have had its own team of video driver developers | writing drivers for all the common cards that didn't already | have good OS/2 support from the card vendors. | 0x445442 wrote: | The first paying developer job I had was writing apps for | O/S 2. But that software was for the US Navy. I think IBM | thought they could secure the "real" use cases and let | Microsoft have the mom, pop, buddy and sis market. Forward | thinking was not IBMs strong suit. | ethbr0 wrote: | In retrospect, it seems like IBM (tech strategy) was | always hamstrung by IBM (sales strategy). | | The latter being firmly in the "scale top-down" camp: big | customers are the only customers that matter. | | I've worked for both top-down and bottom-up marketed | companies, and the latter always seem to have better | products. | | I think it's because (a) there are so few whale | customers, which hyper-specializes your product to their | needs instead of getting a representative sample of the | actual market's needs, (b) you can do dumb shit like | limit your documentation to only customers, (c) it gives | you a convenient excuse to ignore customer input ("Oh, | they're not a customer that matters"), & (d) it slows you | down, because your customers' upgrade processes are | glacially slow, which removes pressure for you to be | fast. | johannes1234321 wrote: | Developer! Developers! Developers! | https://youtu.be/Vhh_GeBPOhs | | One can say a lot about Microsoft and Windows, but they | were always keen on building a platform which enabled | others to build upon. | jrockway wrote: | I don't know if we can take this conclusion away. For | example, Apple requires you to pay to be in the developer | program and buy their hardware in order to make an iPhone | app. They beat the crap out of Microsoft in mobile. | Microsoft was also in the mobile game way before Apple. | (Remember Windows CE? I had one of those right before the | first iPhone and Android phone came out.) | johannes1234321 wrote: | Many developers don't go through the steps to develop for | Apple since they like ObjC or Xcode or the review | process, but since that's where the users are. Apple | created great products for users, where the walled garden | reduces the risk of catching a bad app harming you. | | Microsoft on the other hand always tried to give | developers tools and access to it with quite good | documentation. Enabling people to build on top of their | platform. Making sure that other people can built upon | their platform. This brought Windows in a ton of embedded | worlds except the phone, where they missed the | reinvention by Apple. But from say ATMs to Trains Windows | is still ubiquitous. And that's the way they built Azure. | Want to offer your own Service? - You can integrate it | into Azures Cloud Console. And at the same time they | still foster developers, with GitHub, VisualStudioCode | and TypeScript. | ethbr0 wrote: | Apple beat the crap out of MS in mobile because they made | the more technologically correct choice: that mobile | ~2003 couldn't support desktop features and needed to be | something fundamentally different (and resource-lighter). | | Microsoft was sucked into the false idea that people | wanted desktops on their mobile devices, but hardware at | the time couldn't offer that in a cost-effective way. | | Apple was also cutthroat pragmatic in a way that I don't | think was ever in Microsoft's DNA -- specifically: | killing Flash support. | tengwar2 wrote: | What was in the SDK? I'm just curious because I developed | for OS/2 1.0, i.e. before the Great Schism, and before | Presentation Manager was launched. The SDK was MS C v5.2, | MASM, CodeView, some .lib files, and some seriously | inaccurate documentation. There wasn't much there by modern | standards - rather less than you would get for Petzold era | Windows development. | chasil wrote: | OS/2 was forced down to 80286 compatibility for version 1.1. | | That was major design damage, required by IBM because most of | the PS/2 line was on this CPU. | | Both Linux and NT required an 80386 with a flat memory model. | On capable hardware, this was far superior. | | Both Linux and NT quickly ran on multiple platforms besides | x86. I think that OS/2 was ported to Power, but nothing else. | tengwar2 wrote: | No, that was v1.0. v1.1 was incompatible with v1.1 in some | respects for that reason. I had some stuff that was literally | hard-wired with wire-wrap to do bit-shuffling from an image | acquisition system (assembly wasn't fast enough) and I would | have had to re-wire it to migrate to 1.1. There were also | some details that wouldn't have affected most people, like | moving from 8 byte entries to 16 byte entries in the | descriptor tables. Sorry if I'm vague on the details - it was | a while back. | chasil wrote: | Interesting, the wiki is not clear on this. | | "OS/2 1.x targets the Intel 80286 processor and DOS | fundamentally doesn't. IBM insisted on supporting the 80286 | processor, with its 16-bit segmented memory mode, because | of commitments made to customers who had purchased many | 80286-based PS/2s as a result of IBM's promises surrounding | OS/2. Until release 2.0 in April 1992, OS/2 ran in 16-bit | protected mode and therefore could not benefit from the | Intel 80386's much simpler 32-bit flat memory model and | virtual 8086 mode features." | | https://en.m.wikipedia.org/wiki/OS/2 | | The developer of busybox/toybox also had 80286 discussion. | | https://www.landley.net/history/mirror/os2/OS2Warp.html | markhahn wrote: | I remember 386 support connected with 1.3, but probably still | using descriptors. | | NSFT was still involved at 2.0 time, though DOS+Windows was | pretty obviously going to win, so to speak, and NT OS/2 was | still hush-hush. NT OS/2 ran on ia32, mips, alpha, and i860 | (wow, freaky to remember that!) at the time - the "new | technology" was as much the racy idea of portable OS code as | RISC and microkernels... | chasil wrote: | Cutler's design was VMS written in (portable) C. (The VMS | kernel was written in assembler.) | | VMS has great strengths, but hard file locks on OS | components require downtime for patching. Cutler didn't | foresee the avalanche of updates, and the availability | collapse of "patch Tuesday." | | POSIX opportunistic locking was a great gift to his | competition. | dflock wrote: | OS/2. Also used for Vancouver's SkyTrain system, afaik. | dtx1 wrote: | > The Forgotten Operating System That Keeps the NYC Subway System | Alive | | It's OS/2. Hardly forgotten. | doublerabbit wrote: | Give it a few more years. The iconic icon of a Floppy Disk has | almost faded out of existance. | manv1 wrote: | OS/2 was known to be rock-solid back in the day. It was the OS of | choice for a lot of financials. Everyone moved to Windows NT when | 4 came out, though, because of the way Microsoft did licensing. | not2b wrote: | "hilariously robust", the article says. | noodlesUK wrote: | How is it possible that the magnetic stripe encoding is a secret? | Surely you can just run the card through a reader before and | after taking some journeys and reverse engineer from there? Is | the balance actually stored on the card, or is it just basically | a primary key in a database somewhere that gets queried on each | ride? | walrus01 wrote: | they're in the process of replacing the whole magnetic stripe | system right now with a modern contactless system similar to | london, vancouver, seattle, etc. | crazygringo wrote: | It's both, and you can find the answer on Wikipedia. [1] | | > During a swipe, the MetroCard is read, re-written to, then | check-read to verify correct encoding... Each MetroCard stored | value card is assigned a unique, permanent ten-digit serial | number when it is manufactured. The value is stored | magnetically on the card itself, while the card's transaction | history is held centrally in the Automated Fare Collection | (AFC) Database... Whenever the card is swiped at a turnstile, | the value of the card is read, the new value is written, the | customer is let through, and then the central database is | updated with the new transaction as soon as possible. Cards are | not validated in real time against the database when swiped to | pay the fare. | | There's also an old NYT article [2]: | | > But since the main computer communicates regularly with both | the turnstiles and the booth computers, false cards have so far | all been invalidated after they have been used once... Some | hackers believe that because the turnstiles, the token booth | computers and the mainframe computer, respectively, upload and | download information at six-minute intervals, the length of | time that a passenger can transfer without paying may actually | be 18 minutes more than the advertised two hours, if the timing | is right... ''I believe that, fundamentally, a system as | complicated as the Metrocard cannot be absolutely secure,'' he | said. ''What we have to make sure is that the hack won't scale. | If a few people ride for free, that's no big deal.'' | | [1] https://en.wikipedia.org/wiki/MetroCard#Technology | | [2] https://www.nytimes.com/1997/08/11/nyregion/what-galls-a- | hac... | ardel95 wrote: | That's the thing with fraud. The goal is never to stop it | all, but to keep it at some local minimum. The rest is just | the cost of doing business. | VWWHFSfQ wrote: | Capitol One has been touting "With Capitol One, you're | never responsible for fraudulent charges on your account!" | | Yeah, no shit. Fraud is YOUR problem. Not mine. | roywiggins wrote: | In this case it just has to be harder to pull off than | jumping the turnstile. | kingkawn wrote: | Which is unfathomably easy, so much so that it doesn't | even require physically jumping | crazygringo wrote: | Cops were actually catching and fining people a ton for | this between about 2-4 months ago. First time I'd ever | seen them proactively enforce it. | | Haven't seen it in a few weeks though. It's expensive to | be posting two cops near every turnstile entrance. | tibbydudeza wrote: | Pretty neat to keep value on the card. | | EMV (chip) bank cards can default to local authorization mode | if the transaction value fits your profile if the pinpad | can't communicate upstream. | | Nothing worse than folks not able to buy/use your system | because it is "offline". | lupire wrote: | This is why swiping (read-write-read) MetroCard is much less | reliable than swiping a (read-only) credit card transaction, | and why you need to swipe a MetroCard slowly. | dehrmann wrote: | > Surely you can just run the card through a reader before and | after taking some journeys and reverse engineer from there? | | I doubt they do this, but if the data's encrypted with a good | cipher used correctly, all you'll be able to detect is that | it's changed. | someweirdperson wrote: | Chinas railway is running on flash player [0]. At least that | choice doesn't limit it to one specific OS. | | [0] https://news.ycombinator.com/item?id=25786670 | Meph504 wrote: | You would be shocked at the number of ATM machines that run on | OS/2, and though I don't advocate it, it makes sense in a way. | | When is the last exploit you saw published for OS/2, many of the | old OS/2 machines direct dial into an old modem bank (at a bank.) | | The thread window on these is tiny because of how far behind they | are. | jackblemming wrote: | How did this scale without kubernetes, docker, and microservices? | How does this work without transpiling from TypeScript to another | language? This wont scale without type safety!! Does it even have | a JIRA backlog and dedicated product manager? No wonder this | system is so mediocre. It probably wasnt even made following | Agile guidelines! | bpodgursky wrote: | The NY metro is infamously unable to scale, in large part | because of extraordinary tech debt! The cost of each new line | is stratospheric! They use century-old manual track routing | cables! | | You might think this is a clever take but it's really not far | from the truth. They NEED a revamp on a modern tech stack so | they can use the trains, electronics, and technology trains in | the rest of the world run on. | woodruffw wrote: | The subway's issues are mostly physical, not in software: the | city (really, the state) contracts virtually everything | around fare collection out to Cubic, and they do an okay | enough job. See for example the OMNY card, which rolled out | surprisingly fast. | | In other words: the things that are currently falling apart | (the physical relays, for example) are almost completely | disconnected from fare collection (which is what the | article's about). The latter runs pretty well; replacing it | with a modern tech stack rather than the current incremental | approach wouldn't have many benefits. | bontaq wrote: | A modern tech stack won't help that much, there are other | issues. | | > Most subway services cannot significantly increase their | frequencies during rush hours, except for the 1, G, J/Z, L, | and M trains (the L service already is automated with CBTC). | | > However, even without CBTC, the system is currently | retrofitted to operate at frequencies of up to 60 trains per | hour (tph) on the IND Queens Boulevard Line (30 tph on each | of the local and express pairs of tracks made possible by the | Jamaica-179th Street terminal, which has four sidings past | the terminal for each set of tracks) and 33 tph on the IRT | Flushing Line. | | https://en.wikipedia.org/wiki/Signaling_of_the_New_York_City. | .. | devwastaken wrote: | In 20 years all current software stacks will be legacy and | equally if not more unusable. We rely significantly more now | on tooling to just "do it for us". That tooling relies | heavily on the OS it runs on and makes assumptions. | chinabot wrote: | If you diddnt do updates I seriously doubt the stack would | survive 2 years. The joys of modern interconnected systems | nickbauman wrote: | I believe your comment might be a bit too broad for the | claims you're making. | | Arguably, the most advanced "tech stack" for moving trains | around rail systems today (at scale, with the highest safety | rating) is based around the Sicas ECC electronic interlocking | by Siemens. | | However, this article is merely about the technology used for | collection of fares for the NYC metro rail system. | supernova87a wrote: | If you want a story into the other "operating system" that keeps | the NYC subway system running (or from another point of view, | never advancing), see this: | | https://www.theatlantic.com/technology/archive/2015/11/why-d... | | "Why New York Subway Lines Are Missing Countdown Clocks" | | and how it's actually very about how managing a software/IT | project can go wrong. ___________________________________________________________________ (page generated 2022-10-09 23:00 UTC)