[HN Gopher] Running a Unix-like OS on a home-built CPU with a ho...
___________________________________________________________________
Running a Unix-like OS on a home-built CPU with a home-built C
compiler
Author : abc_tkys
Score : 254 points
Date : 2020-10-04 16:17 UTC (6 hours ago)
(HTM) web link (fuel.edby.coffee)
(TXT) w3m dump (fuel.edby.coffee)
| peterburkimsher wrote:
| I did a course like this while on exchange programme to UCSB! In
| the ECE 152B Digital Design Methodologies class, teams of 2
| people had to design and build a CPU from scratch.
|
| We had to choose all our own chips and wire them together by
| hand. We had an FPGA for the Control Unit, but had to design the
| bus, pipelining, stack, carry-borrow, all the way up to running a
| demonstration program.
|
| We did that in teams of 2 people, with only 1 semester. I was
| broke, so we chose the cheapest chips (I built MUXes out of
| NANDs). I realised we couldn't finish on time. I wandered the
| hallways, saw a project poster from a previous year, took some
| photos, reverse-engineered and reimplemented their design. We
| didn't have the FPGA code, but knowing the breadboard layout
| helped enormously. A few more nights in the lab, working to
| exhaustion and sleeping under the desk, and we passed the class.
| If you think industrial espionage only happens in other
| countries, that taught me that it happens in the US too. Arguably
| we made enough changes that it wasn't total plagiarism, but it
| did help to have a known-working design to build on.
|
| The most lasting memory of that project, though, was when it
| worked fine during debugging, but failed whenever we removed the
| test probes. We were using 2 breadboards, but no common GND.
| During programming, they were grounded together through the GND
| on the PC's USB port. Always check for a common ground!
| saagarjha wrote:
| Very cool. I once taught a class similar in spirit to this,
| although it was aimed at first-year undergraduates with only a
| basic knowledge of programming, squashed into a single quarter,
| and designed on the fly, so we only got partway through
| implementing a C compiler. Instead of writing a full C compiler
| ourselves, we considered "cross-compiling" one by taking one that
| compiled to bytecode, and writing an emulator for that in
| assembly, taking that to be our first "compiler" and then
| progressively bootstrapping from there to a better compiler until
| it became feasible to start writing (or porting) a C-based OS.
| wrs wrote:
| I've always wanted to do this. I helped my housemates implement a
| CPU with custom ISA in college (this was before FPGAs, so there
| were a lot of chips in that CPU) but that's as close as I've
| gotten.
|
| I kind of feel like you can do something like this in college, or
| after you retire, but nowhere in between. :)
| ChuckMcM wrote:
| This was an excellent read. Congrats to the team on getting their
| port of Xv6 up and running!
|
| I had the opportunity to help with a project to port UNIX to a
| data general NOVA computer as an undergrad and the combination of
| "non toy" problems and results you can see is really fun. We
| didn't have to write the compiler though! That is an amazing part
| of this story too.
|
| Given the availability of RISC-V descriptions and FPGA boards
| that will host it with memory, this sort of project should be
| within reach of anybody willing to invest about $150 in parts and
| a lot of labor. It would make for an interesting basis for a demo
| contest!
| Teknoman117 wrote:
| That's a current project for me right now. When I took computer
| architecture in university, it was fairly light on hard details
| about things like superscalar and out of order designs. Sure
| there's the concepts, but there are so many interesting
| problems it didn't cover.
|
| E.g. how do you, at a hardware level, actually do the
| reservation stations for a out of order design. How do you
| actually implement a register file with enough read and write
| ports to satisfy such a design without taking up a whole die
| for it?
|
| I know there are a few Linux capable soft-core RISC-V designs
| out there (VexRisc, etc.) and microcontroller class ones
| (PicoRV32, etc.). If my goal was implement a system and it
| needed one of those things, sure, I'd use an off the shelf one.
| But I really want to understand how the CPUs work, and the best
| way to do that is doing it myself without looking at the answer
| key.
|
| Turns out register files are complicated and fascinating. I'd
| never come across "register file banking" in my architecture
| courses. Makes what I had to deal with in CUDA make a lot more
| sense now.
| ChuckMcM wrote:
| That sounds awesome! You should definitely post a Show HN on
| it.
|
| I am going to comment on this though: _But I really want to
| understand how the CPUs work, and the best way to do that is
| doing it myself without looking at the answer key._
|
| I am right there with you this, however with experience I've
| come to appreciate that there is a _lot_ of complexity in
| this topic and I personally have a limit on how steep a
| learning curve I 'm willing to climb in my spare time. As a
| result I've taken to trying to isolate topics to learn around
| things that are known to work. Here is an example ...
|
| In 2015 I discovered you could get software radios for $25
| and there was a bunch of open source software one could use
| to play with them. I wanted to write my own modulator and de-
| modulator code but kept running up against too many unknowns
| so having it be impossible to debug. Was it my code? Was it
| the radio setup? Was it the signal format?
|
| I didn't start making real progress until I got a real
| Spectrum Analyzer and Vector Signal Generator. This let me
| work from a "known good source" and generate signal that I
| could compare on my Spectrum Analyzer with the signal the VSG
| generates. THAT let me find bugs in my code and once I
| understood more of the basics of the DSP that was going on
| then I could branch into things like front end selectors and
| polyphase filtering.
|
| So I applaud your effort, and more power to you if you punch
| through and get this done. It will be huge. But if someone
| reading this were to think this is the only, or best, way to
| do something I would encourage them to recognize that one can
| break these problems apart into smaller, sometimes more
| manageable chunks.
| ChickeNES wrote:
| Another detail I never see covered is the implementation of
| variable length instruction decoders. Most every book seems
| to assume a classic RISC design with 32-bit wide
| instructions, single-issue and non-microcoded. Are there any
| advanced undergraduate or graduate level books that cover
| details like this?
| nkozyra wrote:
| Very interesting article. An ambitious project and I'm certain a
| great learning experience
|
| I see "cofee" written a bunch across the site is that some
| alternate spelling or just a typo?
| CyberRabbi wrote:
| Japanese people are incredible. Such an advanced race of people.
| I have so much respect for them.
| fancyfredbot wrote:
| Reminds me a little of Wirth's Project Oberon
| badsectoracula wrote:
| Indeed, while i was reading the blog post i had in my mind
| Wirth's Project Oberon which is also about implementing a
| custom CPU on FPGA and then a complete OS with its own
| compiler, tools, GUI, etc on it.
| duckfruit wrote:
| Jaw-droppingly impressive to have gone from nothing to a custom
| ISA, C compiler toolchain and a whole freaking OS with userland
| apps in six months!
|
| My favorite detail was that having spent all this time on extra
| credit, they only managed to complete their primary task an hour
| before the presentation - an unmistakable mark of a true hacker
| if there ever was one.
| unnouinceput wrote:
| They completed the primary task way before that. This was free
| time and they did it for fun. The fact that they re-implemented
| their initial task as a program in the OS itself was the cherry
| on top, to impress their teachers (and also to learn a heap
| load of stuff in the process).
| amelius wrote:
| > to impress their teachers
|
| If I was that teacher, I would have flunked them for "doing
| something just because you can" and not using their
| capabilities for a more useful goal. And also because I'm
| tired of yet another toy CPU and toy OS that's just like all
| the others.
| ylyn wrote:
| How ironic that you say this on Hacker News.
|
| That is one of the ideas at the very heart of hacker
| culture--doing things just because you can.
| amelius wrote:
| True, but these students were following an academic
| education, not "hacker university".
| ylyn wrote:
| Well, they also fulfilled the requirements for the credit
| (which was to get the raytracing program running bare-
| metal on their CPU), so if you want to be pedantic about
| them being in an academic course, then you should honour
| the original assignment requirements, right?
| unnouinceput wrote:
| It was their free time. You get the idea behind that
| concept, yes? As in do whatever they want. And they
| wanted an OS! opposite of let's say drinking in bars
| trying to pick girls.
| saagarjha wrote:
| I suspect you're failing to understand the goal of being a
| teacher, which is to nurture your students. Toy operating
| systems are a time-tested way to learn.
| amelius wrote:
| No, I would have taught them the valuable lesson of not
| trying to invent the wheel. These students are obviously
| smarter than the average student, and therefore I would
| expect more of them, like inventing new OS _concepts_
| instead of mimicking age-old designs.
| NobodyNada wrote:
| When I was 12 or so, I was fascinated by a video [0]
| visualizing the memory architecture of the Nintendo
| Entertainment System. I started researching the technical
| details of the console in order to try to make sense of
| what I was watching, and within a few hours I had decided
| to try to write a NES emulator.
|
| Now, the NES emulation scene is extremely mature, and so
| I quickly came across a number of warnings essentially
| saying "the world has too may NES emulators, please don't
| write another one." I decided to go ahead anyway, and
| after spending about 2 years' worth of afternoons and
| weekends, the result was just another bad emulator. It
| was slow, buggy, incomplete, and the code was a mess.
|
| By your standards, I wasted 2 years of my life making
| "yet another toy" emulator instead of "using [my]
| capabilities for a more useful goal." But I had a _ton_
| of fun, and the learning experience was absolutely
| invaluable. I learned in depth how a computer actually
| works at the hardware level (even though the NES is much
| simpler than a modern computer, the concepts and skills
| transferred over very quickly).
|
| As a result, my practical programming knowledge today is
| vastly improved over where it would have been had I not
| "wasted" all those afternoons. Many of the career
| opportunities available to me so far have been directly
| attributable to the skills I gained working on that
| emulator.
|
| [0]: https://www.youtube.com/watch?v=xI3xZAn7r2A
| MaxBarraclough wrote:
| There's obvious educational value in having students
| build tools like compilers and operating systems.
|
| > I would expect more of them, like inventing new OS
| _concepts_
|
| We're talking about undergraduates, not researchers.
| vlovich123 wrote:
| You can't build the next wheel without first getting a
| handle on how & why the existing wheels were built the
| way they were. Even CS undergrads start with toy
| operating systems that they fill in the parts for (adding
| a filesystem, writing a compiler for a toy language,
| etc). There's simply no better way to learn.
|
| > These students are obviously smarter than the average
| student, and therefore I would expect more of them, like
| inventing new OS concepts instead of mimicking age-old
| designs.
|
| You can disagree but here are the facts:
|
| 1. The students completed the task.
|
| 2. The professor had intentionally built slack into the
| schedule to let ambitious students take this further to
| play around with the tools & their knowledge to try doing
| creative things.
|
| 3. The students used this to recruit others students to
| form a larger team around a more ambitious end goal
|
| 4. Self-organized to distribute work & build a schedule
| that minimized interdependencies
|
| 5. Completed their goal.
|
| Aside from the technical stuff, these all sound like
| valuable soft skills that were learned/applied in
| addition to the technical achievement. I'd say both the
| professor & students did a good job here.
|
| What's hard for me to say is what year this is. In my old
| engineering school there was a final project in years 3 &
| 4. Your team would pick some kind of vague final project
| (with consultation from your teacher), you'd get a budget
| for materials + connections to companies/vendors for
| sponsorship, & then go about building your concept.
| That's a bit more complex than this but also lasts 1.5
| years and happens largely outside of school. This blog
| post is about a project done within the context of 1
| subject AFAICT. That makes it way more impressive.
|
| Students at this level simply don't yet (at least
| generally) have the experience nor a good understanding
| of the space to understand what are more significant
| problems to go tackle. From what I've seen you usually
| start that journey as a masters/PhD student.
| [deleted]
| codegladiator wrote:
| new doesn't come before knowing what exists and why, and
| to understand the latter making it is important. as a
| teacher if you fail to understand you should probably
| spend some time to read more why people study what exists
| in school and college for the better part of their life.
| Kuinox wrote:
| Yeah, a certain Linus should not have tried to invent the
| wheel.
| mlyle wrote:
| It's an undergrad class, and they sailed right past the
| expectations of the rubric and fit in with the culture of
| taking their project a bit further than what's required
| (but in a different direction than most.
|
| Perhaps at some point one would expect students to invent
| something new, but it's generally after the OS/arch
| classes where you do things like implement toy OSes or
| toy CPUs. :P
| Ericson2314 wrote:
| I'm sympathetic for your goals, if not your means. I
| would point out that they ported an existing OS: a sign
| that perhaps they wouldn't have had enough time to finish
| with a novel design.
|
| I hope that the code in the commons can continue to
| become more modular to the point that is _is_ practical
| to try out some fresh ideas with like a library OS /
| exokernel that provides all non-novel bits without
| constraining the design space.
| ryan93 wrote:
| Why do people even engage with this guy. Lol
| danparsonson wrote:
| > The original task of the CPU experiment was "Run the given
| ray-tracing program on your homebrew CPU". Now that you've
| got an operating system running on your CPU, you know what
| you're supposed to do, right? We decided to run the ray-
| tracing program "on the OS "on our own CPU. We had a few
| bugs, but we managed to finish it an hour before the final
| presentation.
___________________________________________________________________
(page generated 2020-10-04 23:00 UTC)