[HN Gopher] Taking over a Dead IoT Company
___________________________________________________________________
Taking over a Dead IoT Company
Author : pulisse
Score : 702 points
Date : 2023-01-10 14:52 UTC (8 hours ago)
(HTM) web link (blog.kchung.co)
(TXT) w3m dump (blog.kchung.co)
| sschueller wrote:
| I build something similar https://sschueller.github.io/posts/vbz-
| fahrgastinformation/ but with way less BOM and I keep getting
| asked that I should sell them. The primary reason I don't is
| because I don't want to support something like that for the next
| 10 years...
| [deleted]
| klinquist wrote:
| heh, same. I built a little custom sign for my airbnb. Part of
| my home automation system:
|
| https://imgur.com/a/8A5IKV6
| xd1936 wrote:
| Very cool, and well executed! Do you have build details
| anywhere?
| NotYourLawyer wrote:
| That BOM is crazy. It looks like (and is) a hobby project that
| never had any consideration given to manufacturerability.
| [deleted]
| speedgoose wrote:
| The way the hostname is set, allowing an incredibly simple shell
| injection, reminds me that the S in IoT stands for Security.
| [deleted]
| js2 wrote:
| > With full control of the domain, we can create a new API based
| on what the sign is expecting and revive all of the signs that
| are out in the field.
|
| Once the new server was up, did any signs from the field connect
| to it?
| ColdHeat wrote:
| Good question! No signs connected to the server until I reached
| out to some other sign owners to try out my instructions.
| wferrell wrote:
| Great post!
| Lucasoato wrote:
| > It seems that the company could remotely connect to a terminal
| on every sign.
|
| What a red flag... they basically had a backdoor to everyone's
| home network.
| [deleted]
| schubart wrote:
| I didn't quite understand the need for an injection attack. Once
| you had bought the domain and could respond to the devices' API
| calls, weren't you in the driving seat already?
| ColdHeat wrote:
| I actually had written more about the exploit & vulnerability
| in my original drafts but I cut it out because it was a bit
| boring to read.
|
| You are correct that with domain control I am able to serve
| content to any sign but the content will only be loaded once at
| boot time. Any future updates would have needed to come from
| their defunct AWS IoT connection (ignoring full restarts).
|
| Using the exploit I remove the connection to AWS IoT and update
| some of the code to better connect it to the recreated API so
| users can update their signs in mostly real time.
| wferrell wrote:
| Would you mind posting that content somewhere? I would find
| it very interesting!
| ColdHeat wrote:
| Sure I took out the relevant section and put it here:
| https://docs.google.com/document/d/e/2PACX-1vTYSTUp3eTjfD-
| hG...
| wferrell wrote:
| Thank you!
| karmanyaahm wrote:
| The author couldn't have accessed the original AWS account,
| which is what managed configuration updates.
| tyingq wrote:
| A guess, but...
|
| _" At boot time, the Config Server will pull the latest
| configuration from an HTTP server. In addition, the Config
| Server will connect to an AWS IoT Core endpoint to receive real
| time config updates from an MQTT server."_
|
| So, perhaps to bootstrap information they didn't have, like the
| current configuration? It sounds like they sold different types
| of signs with different resolutions, and also whatever train
| station setup config there was. Maybe a one-time exploit to
| upload that info from the sign itself.
| ColdHeat wrote:
| Yes the exploit removes the AWS IoT connection so that
| updates can come from the recreated API.
|
| There was only one type of sign but it did come in various
| different cases.
| neilk wrote:
| Maybe it's very North American of me but I cringe when the author
| says "bad team". Objectively, they didn't have the skills,
| experience, or advisors to do this thing. And if the money truly
| disappeared, then someone crossed over into "bad" territory.
|
| But hardware startups are brutal. Kickstarted hardware is
| torture.
|
| Even experienced and well capitalized tiny teams go through this.
| A friend did one, rapidly realized the only option was to somehow
| get Chinese manufacturers excited about a small run of a strange
| product. Then two years later was trying to QA injection molds
| from across the ocean and switched manufacturers two times.
|
| They delivered, but only after years of what must have felt like
| crawling over broken glass.
|
| Maybe the NYC Train Sign team realized they couldn't profit from
| the sign itself, but, unwilling to let the viral moment pass,
| were going to use it to establish themselves in consulting. You
| know... sell pickaxes.
| iamflimflam1 wrote:
| Indeed, I enjoyed the article, but felt the snarkiness was bit
| too much.
|
| Startups fail all the time, and hardware startups are a
| minefield.
|
| If they had succeeded then we'd be singing their praises on how
| they started off with a scrappy product built from off the
| shelf parts and then managed to productionise it and outsource
| the manufacturing to china...
| [deleted]
| closetohome wrote:
| I agree completely about the difficulty of a hardware startup,
| but being "good" means knowing in advance what kind of
| challenges you'll face, or at least knowing who to consult
| about it, and when to listen to their advice. I guarantee that
| at several points in the development of their product, someone
| pointed out the high BOM cost and the math required to become
| profitable. At this point Kickstarter has been around for long
| enough that even people without direct experience in hardware
| development should have enough case studies to know what
| they're wading into.
| klinquist wrote:
| Sounds like I need to partner with the author.... as I wrote my
| own predictive Caltrain late train model/alert site:
|
| https://caltrain.live
|
| (twitter @bettercaltrain)
| lxe wrote:
| Looks like they were trying to sell early prototypes as final
| products?
| moneywoes wrote:
| Your newsletter isn't working heads up
| ColdHeat wrote:
| Thanks for letting me know. I will have to take a look. I setup
| Ghost a long time ago and just really use it for the
| blogging...
| dom96 wrote:
| In 2020 I set out to build something very similar[0]. This was
| before I even knew NYCTrainSign existed. I started with zero
| electronics knowledge and learned a lot. My prototype has been
| running solidly for way over a year and while I've always hoped I
| would be able to turn it into an actual product I could sell...
| doing so proved challenging.
|
| Really what blocked me the most was CE/FCC conformance. I have
| done a lot of research into it and I couldn't figure out a way to
| get it without a massive investment upfront ($10k+). If anyone
| has experience in that area I would love to talk, my contact info
| is in my profile.
|
| Who knows, maybe I will get it to a point where it can be sold
| eventually. I'll certainly write up about my learnings from the
| project at the very least.
|
| 0 - https://twitter.com/d0m96/status/1427055272980328460
| [deleted]
| jareklupinski wrote:
| https://www.instagram.com/p/BVAIWfGBARk/
|
| my problem was squeezing the MTA API parser into the ESP32
| flash...
|
| as long as you use off-the-shelf / already conforming modules,
| like the esp wroom etc, the cost for FCC compliance testing
| goes down into the thousands. it's still a line item, but over
| a run of 1000-2000 units, it turns into something that costs
| about a buck per board :)
| elcritch wrote:
| Yah using the pre-canned esp32's make the cert process a lot
| easier.
|
| @dom96 thats cool! Ping me on the Nim discord sometime if you
| want. I've not done FCC but did do some hardware
| certification stuff. For low volume products there possible
| ways around full FCC certs I believe. IIRC, you can sell
| things as "kits". I wonder if theres also exemptions for
| "prototype builds" too.
| dom96 wrote:
| > my problem was squeezing the MTA API parser into the ESP32
| flash...
|
| Ahh, I wouldn't do that. The way I've done it is by pushing
| as much of the API handling logic off the ESP32 and onto a
| standalone server. The ESP32 is effectively a dumb client
| which takes in instructions like "WRITE westminster 5 min"
| from the server and draws the appropriate text/pixels/etc.
|
| > as long as you use off-the-shelf / already conforming
| modules, like the esp wroom etc, the cost for FCC compliance
| testing goes down into the thousands. it's still a line item,
| but over a run of 1000-2000 units, it turns into something
| that costs about a buck per board :)
|
| yeah... it's $1 per board unless none of your boards sell :)
| jareklupinski wrote:
| that's fair ;)
|
| when kickstarter was first taking off, I loved using it to
| springboard only the projects that would at least break
| even: if there wasn't enough demand, nothing happens, but
| as long as the minimum is high enough to cover all the
| costs, it worked out pretty well
| elcritch wrote:
| Crowdsupply is great for those things now. They're much
| better at helping get hardware projects off the ground,
| and they can handle S&H too.
| anigbrowl wrote:
| This was really interesting - a well deserved spot at the top of
| the front page. It's rare to see an article that combines the
| technical and business analysis so well.
|
| It's amazing to me how quickly the company loaded itself up with
| staff. Being kinda familiar with ESP32 development, I thought at
| the beginning that the company would have 4-6 people - 1-2
| technical, 1-2 production, 1 business/sales/marketing, 1 admin.
| It seems like they opted to target the luxury nostalgia market
| with the $600 price point rather than a more value-driven $300,
| and wrapped themselves in an aura of success rather than growing
| it naturally. The descriptions on the resumes of the former
| marketing staff juxtaposed with the business reality was...quite
| a contrast.
|
| It's especially sad as the basic idea was good and seems
| sustainable at a lower price point - with a metal/plastic housing
| it might have been an easy sell to businesses near subway
| stations, for example, and developing cosmetically different
| versions for other large cities would be quite feasible. I wonder
| what made them take the 'growth corporation' approach rather than
| crowdfunding the prototype > product route, which was fairly well
| established even 5 years ago. That offers a fairly clear roadmap
| for new ventures and is sufficiently familiar to consumers that
| innovators can do some market segmentation and have a cushion of
| patience to get them though the design to manufacturing
| transition.
|
| A great case study for anyone thinking about launching a product
| for fun or profit.
| mileza wrote:
| I think the 600$ price point was inflated with the Pi and the
| Arduino being used. Simply changing some parts for cheaper ones
| would have allowed to reduce the BOM cost and thus reduce the
| price. The author mentions this in his post.
|
| Also seemingly not a lot of people paid 600$ for the sign, with
| a lot going for a lot cheaper than that.
| FractalParadigm wrote:
| That's a part of what the article dives into. For example the
| LED matrix hat they were using from Adafruit was probably the
| most expensive option they could have used at $25/unit, and
| the article gave examples of much cheaper (i.e. 10% of the
| cost) parts that would have done the exact same job with a
| little extra engineering time/effort. Replacing the Pi with
| an ESP32 would have saved money, but like the article
| mentions using a Pi wasn't entirely a bad idea considering
| the free marketing they got from the foundation.
| [deleted]
| buescher wrote:
| Kudos to the author for a cool reverse-engineering job. Nice
| hack.
|
| The sign wasn't the most problematic "mininum viable product"
| I've seen. If it were a rapidly developed first prototype, I'd
| even be impressed.
|
| I would bet they spent significantly more than $3 on the
| packaging and miscellaneous BOM items. I was going to say the
| wiring harness alone would be much more than that, but it looks
| like most of it is included in the Adafruit panel kit.
| drewzero1 wrote:
| Often when showing off something cool I built for fun, people say
| "you should make those and sell them!"
|
| This company is a great example of why I don't.
|
| Mad respect for the author though. It'd be cool enough to just
| get one sign working, but to take over the API and domain is
| pretty awesome.
| [deleted]
| samwillis wrote:
| Seems down, mirror here: https://archive.is/dIbYd
| twawaaay wrote:
| I think RPi in this thing says it all. It screams it was built by
| somebody that does not know how to develop a physical product for
| mass market.
|
| There are only three parts in this product that should have any
| significant cost to them:
|
| * the display itself,
|
| * the wooden enclosure,
|
| * the packaging the product comes in,
|
| If there are switches, potentiometers or encoders I am really
| partial to well working ones and these are not so cheap. It is
| nice to have but at least they make the product _feel_ more
| expensive so it is not a total waste.
|
| Everything else can be done dirt cheap. I don't think chinese are
| the best way to learn product design but they are pretty darn
| good at saving every last penny (or yuan in their case). I found
| it illuminating to study some of the devices of chinese origin
| (as opposed to ones that are only manufactured in China).
| napolux wrote:
| Used to work on an iptv set top box user interface (html+js on
| opera embedded) back in the days. BOM for the board designed in
| my country... 150$. Chinese manufacturer proposed a redesign
| 100% compatible with the same features... BOM at 15$ IIRC
| dom96 wrote:
| How did you find a Chinese manufacturer to redesign it for
| you?
| napolux wrote:
| it was foxconn, they had some capacity :)
| samwillis wrote:
| Even the wooden enclosure could be done for significantly less.
|
| It's effectively a picture frame. There are plenty of standard
| box frame profiles they are very similar to what they have
| built. If it was me, I would have explored working with a
| picture frame company for final assembly, many are used to
| "assembling" products with extra components (think of stuff on
| Etsy). The frames are simply cut with a framing guillotine and
| stapled together.
|
| My estimate would be frame (including custom cut back panel),
| _and_ final assembly would be in the region of $10-$15 at a US
| /UK framers. (Excluding electrical BOM)
| dom96 wrote:
| Picture frames are indeed perfect for this. I built something
| similar and the frame cost me around PS37.50. My full BOM was
| around PS130 (and that's without any economies of scale).
| [deleted]
| not_the_fda wrote:
| You would be surprised at how often people with no experience
| try to build a physical mass market product that nothing more
| than an RPi in a box. And they often go under because the BOM
| is too high and the software is garbage.
| twawaaay wrote:
| That's why when I started with design I decided to go for
| niche products for businesses. Kind of thing that when they
| have to buy it they will not be looking at the price too
| much.
|
| Low volume (I was doing all soldering myself!), high sticker
| price, not especially profitable for me.
|
| But makes it so much easier to design when you don't have to
| compromise so much and you don't have to be very inventive
| trying to save on BOM. And it is fun to be able to just do
| whatever you fancy.
|
| I think starting mass market production for price conscious
| clients with no experience is just asking for problems.
| pettycashstash2 wrote:
| Wait, you are open sourcing code that you did not write but
| reverse engineered? It that legal?
| belval wrote:
| In the absolute no this is not legal, but in practice for a
| company that has been dead for a while after scamming a lot of
| people I wouldn't be overly concerned of legal repercussions.
| tyingq wrote:
| Google vs Oracle seems to say that offering up a compatible api
| with different internals is legal. Exploiting the signs seems
| risky though.
| hinkley wrote:
| Fujitsu vs IBM also says you can make compatible hardware and
| release it as well. And of course they lost other, similar
| cases during the PC Clone era.
| icapybara wrote:
| Nice analysis. I think the author really hit on the key problem:
| expensive BOM due to poor engineering choices.
| spfzero wrote:
| I wonder whether this particular unit was representative of the
| production target build. Maybe it was an early prototype?
| Certainly after 250K revenue (if you believe that number) there
| was funding for a design rev.
| luma wrote:
| The entire project looks like the result of a high school
| student's first Instructables. It's hilariously under-
| engineered and it's clear that these folks had absolutely no
| idea of how to build a product, IoT or otherwise.
|
| Hint: if your product plans involve a raspberry pi, you
| probably fucked up.
| canucklady wrote:
| I think dunking on the rpi is harsh - if it allows you to
| prototype and get your small-volume product out the door
| quickly, there's lots of other areas you could optimize the
| BOM first. Better to make a product with 75% gross margins as
| a single dev in 6 months than a product with 80% gross
| margins with a team of 6 in a year. Of course, they also
| overhired and wasted money on other BOM components
| luma wrote:
| The problem with rPi is storage and filesystem. You need to
| issue a shutdown command and wait for it to complete before
| removing power in order to help ensure that it will come up
| without problems. This will literally never happen with a
| consumer IoT device. Also, SD media is extremely prone to
| failure.
|
| It's fine for prototype at a very early stage, absolutely
| not fine for something you ship to a user.
| nereye wrote:
| On the other hand, see
| https://news.ycombinator.com/item?id=33892009,
| https://www.raspberrypi.com/success-stories/korg-
| synthesizer... for example on how some companies use
| Raspberry Pi compute modules in their products.
|
| Am assuming here that they're adopting strategies to
| minimize/work around the problems you mention.
| enlightens wrote:
| The computer module instead of the standard Pi is the
| first step of those strategies. It doesn't use the SD
| card (in fact the data sheet gives details on how to
| configure the OS if you won't be shutting it down before
| removing power) and the device as a whole requires
| smaller electrical current that the standard device.
| TickleSteve wrote:
| You would typically use a read-only filesystem in an
| embedded system, not the default raspbian style distro.
| This allows you to a) not require a clean shutdown and b)
| saves your physical media from writes.
| nyanpasu64 wrote:
| Ironically today, for embedded hardware projects, it
| would be a perfectly fine idea to use a Raspberry Pi...
| Pico microcontroller.
| sokoloff wrote:
| If your product needs to output HDMI, I can easily see an
| argument for using a Pi. If your product needs a couple
| GPIOs to drive an LED matrix? Using an RPi is pretty
| difficult to justify. You're likely going to spend more on
| the SD card for the Pi than you would on more suitable
| microcontroller (which can still be easily programmed and,
| in my experience, has a better [or at least no worse] story
| for GPIOs than the Pi)
| tyingq wrote:
| It's difficult to drive these LED matrix devices from a
| typical MCU, especially with chained panels, lots of
| LEDS, etc. Either due to the ~10Mhz+ output needed, or
| the memory to hold all the pixel data. Consider, for
| example, that each of the two panels in this sign has
| 32x64X3 (6144) leds. You have to send on/off for all of
| those, and then control brightness for each color by
| using PWM...meaning half-red brightness would be changing
| that pixel so it's on/off/on/off fast enough to beat
| persistence of vision.
|
| It's more common in the pre-built devices to have a
| controller that's using a main CPU plus a FPGA or CPLD.
| The better hobbyist drivers use things like a beaglebone
| black and it's onboard "PRU" that can drive real time
| from memory it shares with the ARM cpu and Linux.
|
| Also, since it's pulling train data, you need more smarts
| or cycles than a typical MCU to pull via WiFi/https,
| transform text into pixmaps, etc. Though there are
| examples of beefy MCUs doing all this well, like an
| ESP32. Or examples of small MCUs driving a small single
| color matrix.
| dom96 wrote:
| ESP32 is perfect for this application. RPI is far too
| beefy, unless you really want to do everything locally
| (without relying on an external server to package up the
| train data for you)
| tyingq wrote:
| Sure, though 240Mhz, 32bit, optional gobs of PSRAM, etc,
| aren't usually what comes to mind when you hear MCU.
|
| Edit: Yes, you could even use an ESP8266, though 32x128x3
| plus X bits of PWM brightness would mean being limited to
| drawing from storage rather than in-memory manipulation.
| I'd jump right to the WROOM type devices with PSRAM.
| anigbrowl wrote:
| You are quite right that the ESP is way over-specified
| for many tasks, but it's not like the uC is gonna quit
| and take a more fulfilling job elsewhere. You can get
| tiny boards for close to $5/ea, or under $5 if you don't
| need the USB connector. - and that's in single-digit
| quantities. The hardware configuration is very flexible,
| eg if power consumption matters you can run it a lot
| slower, switch off subsystems you don't need, use a
| watchdog timer to simplify exception handling etc..
| There's a rich and constantly expanding SW ecosystem so
| you can get to a working prototype _very_ fast.
| sokoloff wrote:
| If you need a microcontroller with networking (such as
| the train sign would), I think the ESP32 (or
| older/cheaper/less capable ESP8266) is one of the first
| that comes to mind for most engineers, especially those
| who aren't full-time firmware engineers.
| kjagiello wrote:
| RP2040 is a really fitting MCU for this use case thanks
| to its PIOs coupled with DMA. It got me some impressive
| refresh rates on a 64x32 HUB75 display - over 2 kHz in 24
| bit color mode.
| Existenceblinks wrote:
| How would you get a quick MVP of PCB + needed interfaces? I'd
| love to know if there is a PCB design as a service +
| manufacturer that allows small size of order (as a 3rd party
| integration). I think they definitely exist .. I just have to
| figure out who to talk to first.
| bjacobt wrote:
| I recently hired and working with someone for PCB
| design/manufacture low volume of IoT devices. As other
| threads mentions, you can always find folks on fiver and
| upwork.
|
| Hardware academy is also good, where you can ask questions
| and meet other folks who build electronics products.
|
| https://predictabledesigns.com/academy/
|
| Happy to talk and give pointers, email in my profile.
|
| Edit: I've no affiliation with hardware academy
| dom96 wrote:
| There are existing PCBs out there for driving HUB75
| displays (like the one used in this project).
| mcnugget wrote:
| You can contract the pcb work out on fiver and have the
| boards made at one of the Chinese board houses nowdays. I
| think I've seen people on Fiverr that provide turn key
| solutions you pay for it in turnaround spread and back and
| forth though.
| skazazes wrote:
| I don't know of any fully featured services, but something
| like Fiver (https://www.fiverr.com/) in combination with
| services like JLBPCB (https://jlcpcb.com/) or PCBWay
| (https://www.pcbway.com/) could act as an equivalent.
|
| An integrated combination or even a board design offering
| from either of the two mentioned manufacturers would be
| awesome and really help bring down the barrier to entry for
| hobbyists that want to try moonlighting as entrepreneurs
| HeyLaughingBoy wrote:
| I don't know of people who do PCB design, but I'm sure the
| usual suspects (Fiverr, Upwork, etc.,) will have them. I
| can do it, but I avoid that kind of work unless absolutely
| necessary.
|
| However, for small-quantity, high-quality PCBs at a great
| price, once you have the design done, it's hard to beat
| OSHPark and OSHStencils for solder stencils. I highly
| recommend both.
| dimmke wrote:
| Articles like this are part of why I love Hacker News so much.
| I've done a little investigation into producing hardware myself
| and concluded it was just too big a task for someone without the
| background who didn't want to put up a ton of capital or deal
| with overseas manufacturing. I loved reading this.
| jadtz wrote:
| I am new to hn, and really liked this article. If you know
| about other interesting articles like this, please mention
| them. I would love to read.
| _Microft wrote:
| I'm semi-frequently submitting electronics/hardware hacking
| articles here. Maybe browse my history of submitted articles
| to see if there is something among them. Here are some
| examples:
|
| - _" I hacked a Joy-Con controller to have a Capacitive
| Trackpad"_, https://news.ycombinator.com/item?id=34329927
|
| - _" Smartknob - an open-source input device"_,
| https://news.ycombinator.com/item?id=30646371
|
| - _" DVD Laser Scanner Microscope"_,
| https://news.ycombinator.com/item?id=26012652
|
| - _" A simple 11.2 GHz radio telescope"_,
| https://news.ycombinator.com/item?id=26078761
|
| - _" Home-Built Scanning Tunneling Microscope"_,
| https://news.ycombinator.com/item?id=26740968
| manv1 wrote:
| Taking over their DNS isn't going to help if they did things
| correctly. If they did things right each device is going to be
| doing cert verification with AWS IOT, and that verification will
| fail. That's client and server cert validation, unless they
| turned that off.
|
| On reset it should still verify the server cert when it tries to
| call home, but since you have access to the image you can replace
| that (if they didn't stash it somewhere securely, that is).
| ColdHeat wrote:
| The key point here would be "did things correctly" :)
|
| The sign did use AWS IoT for real time configuration updates
| however initial configuration was pulled from their HTTP
| server. Using the vulnerability I describe in the article I
| just remove the connection to AWS IoT.
| mynameisvlad wrote:
| I'm sorry, but did you _actually_ read the article? The one
| which details the process and shows a video of it working?
|
| This is just straight up gaslighting. "That thing you said you
| did isn't going to work".
| spuz wrote:
| Could you explain this further? How would ensure that when you
| connect to https://trainsignapi.com that it validates the
| response is coming from the expected server? Would you hardcode
| a certificate on the device? What if you needed to upgrade the
| certificate? I don't quite understand how AWS IoT fits into it
| either.
| toast0 wrote:
| If you're doing it "properly", you should have your own CA,
| not using public CAs, because public CAs will give
| certificates to anyone who controls servers on your domain or
| controls the domain; not just you. All it takes is someone to
| buy your expired domain, and they can get a new, publicly
| valid cert.
|
| If you needed to update the CA, you'd need a firmware
| upgrade, served with a certificate signed by something
| chained to the old CA. And then, more likely than not, you'd
| want to do further updates on a new hostname, because it's
| hard to do a single hostname and send the right certs to the
| right clients.
|
| This is a giant pain, and I don't blame people for using
| public CA infrastructure instead. Especially if your company
| goes bust, who cares?
| hnarn wrote:
| > Taking over their DNS isn't going to help if they did things
| correctly.
|
| If you had read the post, you would have seen that they did, in
| fact, not "do things correctly".
| IceWreck wrote:
| There are other ways to accomplish that (asymmetric
| cryptography says hi). You don't need AWS IoT for everything. I
| cringe when people's default solution is to make their company
| dependent on a niche product of another company, especially
| when its easily avoidable.
| college_physics wrote:
| Thoroughly enjoyable read.
|
| Maybe if i play devils advocate it will add some value: People
| have castigated the high RPi cost (among others) and they are
| probably right about it, but there is something remarkable about
| been able to ship even a small number of devices with a full
| blown computer and a unique UI.
|
| Maybe the more fundamental problem was not so much the hardware
| cost but not shipping enticing enough software to get people
| excited about the device?
| blakesterz wrote:
| This was a really good read. It's part IOT security, part story
| about startups gone wrong.
|
| The WayBack Machine has it
| https://web.archive.org/web/20230109144459/https://blog.kchu...
|
| As does Google
|
| https://webcache.googleusercontent.com/search?q=cache:71rSDb...
| [deleted]
| turbobooster wrote:
| This made me think of Chumby
| Neil44 wrote:
| App access could have been $1/mo. They could have expanded to
| other cities.
| Karsteski wrote:
| Very good read. The parts focusing on the cost of the BOM for the
| signs really shines a light on just the basic difficulties of
| hardware compared to software companies
| ahaucnx wrote:
| Very interesting read. Based on our experience developing and
| manufactoring air quality monitors, making a customized PCB
| should be done as early as possible in the developing process.
|
| With easy to use editors (e.g. easyEDA) and cheap and fast PCB
| production/assembly (even for small quantities), there is no
| reason to use overized MCUs/mini computers like the Pi and
| expensive modules e.g. from Adafruit.
|
| Also, BOM components selection is key. For the same
| functionality, e.g. a multiplexer there are many different chips
| available -often at very different price points. Availability in
| times of chip shortages are also key to keep an eye on.
|
| Technically this product is very simple and the BOM costs they
| had were a magnitude too high. Too bad because it seemed that the
| market was ready and they could have become a very successful
| business.
| [deleted]
| michaelmior wrote:
| > there is no reason to use overized MCUs/mini computers like
| the Pi and expensive modules
|
| There is if you don't have the expertise. Of course, arguably
| if that's the case, you shouldn't be developing such a product
| anyway. But as the post detailed, by switching to a Pi Zero and
| removing the Adafruit HAT, the BOM costs would already drop
| significantly. I think if you're smart about it, you absolutely
| can build a viable product this way. Certainly not as
| profitable, but possible.
| djhworld wrote:
| A few years ago I created something similar for the London Tube
| although it was much more crude, just a pi zero and a strip of
| RGB LED lights that represented the colours of the tube lines.
|
| The store that made the LED strip caught wind of it some how and
| they cut out a wooden enclosure for me to put the thing in, with
| the tube line names etched into the wood, which completely blew
| me away.
|
| My dad kept on telling me at the time to sell it as a product but
| even then I knew that not being a hardware guy and not really
| having any experience building products it would be a dumb,
| expensive move - and stories like this reaffirm that this was the
| right thinking.
|
| Not saying it can't be done, it's just someone with more
| electronics expertise is way more qualified than a guy throwing
| together a python script on a Pi zero with retail components.
| [deleted]
| jareklupinski wrote:
| there is an in-between: a write-up and a post on hackaday.com
| :)
|
| would love to see the Pi zero and hacked together code on
| there!
| michaelbuckbee wrote:
| Towards the end of the article the author mentions not wanting to
| pursue this as a business as there are so many cheap programmable
| signs out there already.
|
| As someone who would love to buy one of these and customize it
| for myself does anyone have experience or recommendations with
| any of these?
| bpicolo wrote:
| There are a lot of different sorts of programmable signs out
| there: https://tidbyt.com/ https://www.vestaboard.com/
| michaelbuckbee wrote:
| Thank you! The Tidbyt looks like the more approachable of the
| two. Less than $200 vs $3k for the Vestaboard and a it has a
| purposefully simplified Python like dev lang to pull data and
| shove it into widgets.
| jareklupinski wrote:
| adafruit sells most of the parts you need if you go the DIY
| route https://www.adafruit.com/product/5362
|
| great tutorials too https://learn.adafruit.com/rgb-led-matrix-
| cube-for-pi
| tyingq wrote:
| Some experience. The cheapest and easiest route is to buy a
| pre-made sign from AliExpress and just use the serial/usb
| interface to send data to the sign. That's somewhat limited, as
| you can't really control each pixel in a real time way, and
| you're limited to sending a series of "screens" as either text
| (in their limited choices of fonts) or images, defining some
| linger time, transitions, etc. With some visual disruption as
| you send new data to redefine what to display. But it's easy.
| There's also variations on this theme. Some signs, for example,
| have had people who reverse engineered the protocol so that you
| can send stuff from your own scripts instead of the supplied
| (usually crappy) software.
|
| The other path is to buy plain led matrix modules and use some
| SBC to directly control what's displayed. More fun, but more
| work. This particular sign uses qty 2, 4mm pitch, 32x64 pixel
| panels, side by side. Like this one:
| https://www.aliexpress.us/item/2251832064290423.html?spm=a2g...
| Lots of choices in pitch, single/three/full color, indoor or
| outdoor brightness, etc. Though you need to ensure your choice
| of driver (see paragraph below) supports it. They have
| different refresh rates, data sequence, and timing
| requirements.
|
| Then, you pick what to drive it with. Lots of choices there.
| There are libraries and hardware level-shifter adapters for a
| variety of things like a Parallax Propeller, Raspberry Pi,
| BeagleBone Black, etc. With various advantages and
| disadvantages. Fwiw, the Rpi is usually trickier as you're
| bitbanging from a non-real-time OS. The Parallax and BeagleBone
| have better ways to be quasi real time.
| michaelbuckbee wrote:
| Thank you for the very detailed answer. Though I think you
| may have just convinced me to stay with my current solution
| of Raspberry Pi's + 4k TVs for custom display signage in my
| office (but it's not nearly as cool).
| bshep wrote:
| You can use a colorlight card to control the panels, they can
| be had for $15 and connect to the nerwork using gigabit
| ethernet ( must be gigabit ). You then send then data using
| the windows software or you can use a Pi/computer running FPP
| software to send whatever you want.
|
| This would reduce cost to:
|
| 2x panels
|
| 1x colorlight card
|
| 1x power supply
|
| Misc cables / case
|
| Edit: The card can handle an 8x8 matrix of panels ( IIRC )
| and the cards can be chained together to make larger
| displays. I dont recall the max size. The biggest I've made
| is 5x5 for a xmas display.
| tyingq wrote:
| Interesting. Can you interact in a sort of live/api way, or
| is it limited to writing configuration and "launching" it?
| bshep wrote:
| So the configuration is a one time thing to set the card
| parameters, you shouldn't need to touch it once its setup
| for your panels and layout. As far as I know it can only
| be done using their widows app.
|
| The display is done by sending the card network frames (
| someone did a reverse engineering of the protocol, i dont
| know the details ). You can send the display anything you
| want.
| seanw444 wrote:
| Did you intentionally leave out Arduino from the "what to
| drive with" options?
| tyingq wrote:
| Assuming largish, multi-color matrixes, and/or chained
| ones, yes. A ~10Mhz+ pixel clock, double-buffering, or just
| the size of the pixel map would strain most of those. This
| sign is 32x128 pixels, with each pixel taking data for RGB
| (on/off), with the PWM rate controlling brightness for each
| of the three colors. Though there are examples of using
| something beefier/faster like an ESP32. There's also in
| this case, the assumed requirement of fetching train data
| and transforming text into pixels.
|
| The panels themselves don't take data and hold it. You have
| to refresh the display constantly, typically on a 1/8 or
| 1/4 duty cycle.
|
| You can drive smallish, single color matrix displays with a
| cheap MCU. Or there's also the approach of using
| addressable LEDs instead of these more "dumb" matrix
| modules, like arranging the smart strips into a matrix.
| Moves the refresh logic down into each LED.
| seanw444 wrote:
| I see now.
| explodingwaffle wrote:
| Woof. Adafruit + Raspberry Pi in a shipped product _screams_ to
| me that they did not have a real electronics guy on their team.
|
| https://ukdepartureboards.co.uk/ is the British equivalent of
| this, and it seems to do everything right (don't own one, but
| have seen on Twitter)- optional subscription, and nice looking
| hardware (seems like they contracted out for that). Though at the
| minute I suppose there are barely any trains running here :)
| dom96 wrote:
| > and it seems to do everything right
|
| Last time I looked they were missing CE conformance, which
| afaik is pretty illegal. Surprised they haven't been found out
| yet.
| [deleted]
| fmntf wrote:
| Additionally, the composition of certified products does not
| lead to a certified product. Even adding an enclosure around
| a certified board implies the recertification.
| [deleted]
| eschneider wrote:
| Usually yes, but it sometimes makes sense in a very low volume
| product.
| ChuckMcM wrote:
| This is all too common in my experience, "oh we could just use
| an arduino/pi and a hat and a peripheral" rather than "Design a
| system that does <x>". As an EE who spent their primary career
| programming (horrors I know) and in systems analysis, I see it
| as the logical extreme of "why use a 555 when I can program an
| ATTiny to be a timer and do other stuff too!"
|
| The allure of having one be able to add features "with just a
| bit of code" is catnip to product managers. They seriously
| cannot seem to resist going there.
| canadianfella wrote:
| [dead]
| ben_w wrote:
| Oof. My electronics stopped before GCSEs, and even I know
| about 555s.
|
| I can believe someone might prefer software for prototyping,
| but manufacturing? Well, I've seen prototypes enter
| production, so yes, but still, _ugh_.
| danhor wrote:
| A attiny might not make sense, but you can get something
| like the PMS150C for around ~5ct, where it makes more sense
| to use a single MCU than a 555 with all the extra needed
| BOM Items.
| fhars wrote:
| Well, you can always choose this:
| https://www.hwlibre.com/en/troll-duino/
| ryandrake wrote:
| The Code Quality section in the article also listed software
| red flags that point to the product being Amateur Hour. From
| the archive.org About Us link OP provided[1], there seemed to
| be plenty of "coders" but maybe not much embedded SW
| experience? They couldn't even agree on what language to use,
| whether to use tabs or spaces, and couldn't manage to produce a
| pristine Micro SD image without bash_history and a git
| repository littered onto it.
|
| Kind of reminds me of a small company I used to work for where
| I noticed shortly after joining that they 1. had no source
| control or reproducible builds, and 2. ship Debug builds with
| symbols and no optimizations to customers, because they
| couldn't get the Release configuration to run without crashing.
|
| 1:
| https://web.archive.org/web/20180107132644/https://www.devsh...
| adriancr wrote:
| This still seems like its fallen into same BOM trap.
|
| You could buy a phone (moto e20), same size as their small
| display at 90e vs 140e for the board above...
|
| Sell a nice android app, subscription and be done with it.
|
| Phone packs a lot more features.
| awkward wrote:
| That's an indescript rectangle that could just show anything
| and could be from anywhere. They are selling a specific shape
| of rectangle that shows stuff about London and aesthetically
| signals that it is from London.
| [deleted]
| ColdHeat wrote:
| Hello, author here. Happy to answer any questions!
|
| My apologies for the downtime, I wasn't expecting much traffic
| today since I submitted the post to HN yesterday but I've started
| scaling my server now!
| 1024core wrote:
| I'll ask the same question as asked by @js2 below:
|
| Once the new server was up, did any signs from the field
| connect to it?
|
| Do you know how many signs are out there actively looking for
| the mothership?
| ColdHeat wrote:
| Here's my reply to him from
| https://news.ycombinator.com/item?id=34328461
|
| > Good question! No signs connected to the server until I
| reached out to some other sign owners to try out my
| instructions.
|
| I do not know how many signs are out there. I imagine most
| people would have just unplugged their sign after the
| company's API vanished since any data would be stale and
| useless.
| is_true wrote:
| I think the other mark on the back was for the reset button.
| Thanks for sharing.
| ivraatiems wrote:
| I just want to say that I love that you did this. You have
| great skills and you used them for good. I've never seen such a
| clearly white-hat hack :)
|
| People's stuff will work again thanks to you and you should
| feel great about it!
| ColdHeat wrote:
| Thank you for the kind words! Made my morning!
| hahla wrote:
| Love this story. I did something similar with a website.
| Previous owner took it down, bought the domain from him and it
| organically grew to over 1m users per month with the same exact
| functionality.
|
| I wouldn't discount your ability to bring this back to market.
| Would consult a lawyer to see what implications taking over the
| branding and APIs have. But clearly this has a big market.
| MrGilbert wrote:
| I would really like to read more of this story. E.g. what did
| you do differently, did the original owner ever get in touch
| with you after it grew, stuff like that. Only if you can,
| ofc!
| Goz3rr wrote:
| Just curious what made you come up with at least $30 per LED
| panel, as these should run you about $10 in bulk.
| ColdHeat wrote:
| I had found some tweets by the company where they talked
| about using an Adafruit panel that was $40. The price on
| Amazon was about $30 so I figured I would go with the lower
| price. They may have switched to a lower cost panel but my
| guess is that didn't happen.
|
| EDIT: Here is the specific tweet:
| https://twitter.com/NYCTRAINSIGN/status/926106932573810688
| jamesmunns wrote:
| The generic name for these panels are "HUB75", based on the
| interface they use for driving them. I can confirm, these
| are $10-15/ea in small quantities on sites like Aliexpress.
| But you're likely right, they were probably buying retail
| from Adafruit.
| hef19898 wrote:
| Why would you buy retail if you are a company buolding
| and selling hardware to begin with?
| Nextgrid wrote:
| I suspect the intention was to get an MVP out the door
| and get some initial "growth & engagement" so they can
| attract VCs - cost-effectiveness wasn't a priority at
| this stage.
|
| It's a shame because there is a small but sustainable
| business here.
| mynameisvlad wrote:
| Everything in the article screams that they did not know
| what they were doing.
|
| Why would you think that a company using a $25 Adafruit
| HAT that could be found for a magnitude less elsewhere
| would be using cheap LED matrices?
|
| Put another way, what from the article gives you enough
| confidence to say that they would have used cheaper
| parts?
| tyingq wrote:
| "HUB75" refers to the 16 pin connector and specific
| structure of the square wave signals it's expecting (row
| order, pixel order, etc). This particular sign appears to
| use a 4mm pitch (space between each R/G/B led), indoor-
| level brightness SMD leds, and modules that are 32x64
| pixels, like this one on AliExpress: https://www.aliexpre
| ss.us/item/2251832064290423.html?spm=a2g... ( ~$8 + ~$9
| shipping to the US for single qty).
|
| There are various "HUB75" panels of different pitch,
| brightness, LED type, matrix size, etc. Also, there's
| variations like "HUB75E" with 5 address lines instead of
| 4. Address lines are typically A/B/C, A/B/C/D, or
| A/B/C/D/E. And there's also "HUB08", "HUB12" and other
| standards.
| anonymouse008 wrote:
| Sharp mind and entertaining writing!
|
| I've had a few projects like this where I list out the BOM then
| go for 'reasonable business success profit' and scratch my head
| at exactly this line:
|
| > At even just 60k per founder, with the profit per sign sold
| being maybe $400, you would probably need to sell ~600 signs at
| full price per year to create enough revenue to run payroll.
|
| Hardware is an exponent to the value of the internal reward,
| but the root of all business profit. We need a better funding
| mechanism for this style of development - more like the
| hardware lifecycle - wasn't Bolt doing something like that?
| csunbird wrote:
| Love the story! At one point, you are asking about the purpose
| of the giant hole in the backside of the display: The most
| plausible explanation is that, it is to allow air circulation
| within the device to prevent overheating.
| ColdHeat wrote:
| Another explanation that I saw [0] was that it was for people
| to pop the back panel out. I think this is the most likely
| explanation but it didn't occur to me while working with the
| sign. I feel like a little notch would have been more
| appropriate for an actual product.
|
| [0] https://hackaday.com/2023/01/09/iot-archaeology-leads-to-
| api...
| jedberg wrote:
| Have any signs in the field connected to your new API and
| updated?
| ColdHeat wrote:
| Yes I reached out to another owner who was able to connect
| the sign to the API. I've reached out to more people but
| haven't gotten too many responses. It's been 5 years after
| all. If you know anyone with one of these signs send them
| this post!
| 1B05H1N wrote:
| Awesome work!
| gilad wrote:
| Similarly, see https://tidbyt.com - RGB LED matrix desktop
| appliance to show weather, sports scores, etc.
| ilayn wrote:
| Not to take away anything from the article but what does this
| have anything to do with IoT?
| emef wrote:
| it was an IoT product (internet-connected LED sign)
| ilayn wrote:
| That's just a display not an IoT project. The display did
| nothing as a thing.
| m-p-3 wrote:
| IMO that Raspberry Pi 3 was way overkill for the project even
| back then and must have massively inflated the BOM.
|
| Nice article overall :)
| michaelt wrote:
| _" As we discussed earlier, even at $300 the product is too
| cheap. The sign should have likely been selling at $600 from the
| very beginning."_
|
| Even at $600 - there's a screenshot saying the company had three
| engineers and a marketer.
|
| If your 4 employees earn $25k per annum, with the $150 parts cost
| you'd have to sell 222 signs at $600. And if the employees were
| on $100k, that's 888 signs a year.
|
| That's a heck of a lot of signs, at the price. And you'd have to
| keep hitting those sales targets so long as you wanted the API to
| stay up.
|
| It would make much more sense if the business was a one-engineer
| passion project - someone who had a day job and did a few hours a
| week maintaining this as a side project would have much more
| achievable sales targets!
| [deleted]
| pdntspa wrote:
| Why does a project like this need _three_ engineers? Two, I can
| kind of see (a hardware guy and a software guy, though it
| doesnt look like the hardware guy was doing much here), but
| three?
| dinosaurdynasty wrote:
| People have to go on vacation sometimes
| pdntspa wrote:
| So you let the thing run and brief the other on what to do
| if SHTF. We're talking hundreds of units out in the field,
| not millions. Every aspect of that can be provisioned for
| ahead-of-time, and cheaply at that, and then the SWE can
| walk away for weeks at a time.
|
| If the HWE leaves for a bit, same thing. If he's making
| units production stops, but maybe the SWE can fill in since
| these dont look like difficult builds.
| hinkley wrote:
| I wish I could get contracting houses on board with the
| notion that there are many tasks that require 30-50% of a
| person but greatly increase the quality of the work. I almost
| convinced some people at the last place but convincing the
| customers is something else entirely. Code reviews by more
| expensive staff for instance. Architectural reviews. Security
| audits.
|
| One engineer makes a mess because there is nobody to push
| back. Two engineers either fight or achieve full echo
| chamber. Three balances some power dynamics, but you
| definitely don't need 3 FTEs. In fact for this project you
| probably find 3 of your friends' friends to sit together for
| 4 hours twice a week and slam out some code or designs, in
| exchange for a little money and some equity. You even get to
| write off food and beer as a business expense.
| jaclaz wrote:
| >but three?
|
| The third one designed the wooden case and the manufacturing
| process for it ... /s
| gdilla wrote:
| Or charge less and get a recurring fee for the API.
| brookst wrote:
| This. It is fatal to charge a one-time price for services
| that have perpetual operating expenses.
| michaelt wrote:
| Perhaps, but still tough to make the business plan work for a
| $100k wage bill.
|
| 833 subscribers at $10/month? 8,333 subscribers at $1/month?
| dom96 wrote:
| This makes me wonder how Tidbyt's business is doing. They seem
| to have a fair amount of employees.
|
| I actually built something very similar and all in my spare
| time. I've been wondering how realistic making a side-business
| out of it is, but if you are a single engineer that has to code
| the firmware, server, build the displays and market it all...
| it's pretty tough (especially in your spare time).
| rohansingh wrote:
| We actually only have 4 FTE's today and our business is both
| sustainable and growing. Our scale is also a couple orders of
| magnitude larger than the numbers from the article, and we
| have much better control over our BOM.
|
| Hardware is a tough business and managing cash flow &
| inventory requires you to be a lot more deliberate than a
| pure SaaS business. Fortunately we had a lot of examples
| going in of how not to do things.
| wferrell wrote:
| What a great reply. Thanks!
___________________________________________________________________
(page generated 2023-01-10 23:00 UTC)