[HN Gopher] Show HN: I built a Guitar Hero robot [video]
___________________________________________________________________
Show HN: I built a Guitar Hero robot [video]
Author : nik12795
Score : 162 points
Date : 2021-04-17 12:47 UTC (10 hours ago)
(HTM) web link (www.youtube.com)
(TXT) w3m dump (www.youtube.com)
| nik12795 wrote:
| I was never great at Guitar Hero and couldn't fathom how people
| would play Through The Fire And The Flames, so I made a robot
| that could do it for me.
|
| The video explains some of the technical details but I'll
| highlight a few other key points.
|
| - My first attempt at note recognition was looking for specific
| pixels on the screen. The thought was that I would be able to
| detect if a pixel changed color and, depending on the color, it
| would know which note it was. The game outputs at 30 fps and on
| faster songs, it would miss nearly 30% of notes.
|
| - The note recognition was eventually built using computer vision
| from OpenCV and specifically blob detection. It took a fair
| amount of effort to hone in the perfect filtering to make note
| recognition was highly accurate.
|
| - Implementing the solenoids threw me for a loop. I ended up
| burning through a lot of them since I'm new to robotics and
| didn't understand the relationship between power, volts, and
| current. At one point I was giving these solenoids 4x the power
| it was expecting. Needless to say, they burnt out quickly. I
| eventually talked with a robotics guy and he gave me a 30 min
| crash course which straightened me out.
|
| - The whole project, from idea to the YouTube video I linked in
| the post, took 2 months. It probably cost nearly $1,000 for
| everything (raspberry pi, pvc, solenoids, AC/DC power converter,
| relay modules, paint, stickers, etc). That might seem high, but
| like I said, I was learning about volts/current and part of that
| learning process was destroying many solenoids (oops).
| fbelzile wrote:
| Very nice work! I noticed that if there were notes missed, they
| all seemed like the long hold ones.
|
| Was that a limitation of the AI software or was that by design
| to keep the strumming part easier to make?
| nik12795 wrote:
| There wasn't any limitation from the mechanical perspective.
| The AI did a nearly perfect job of recognizing notes. I don't
| know exactly why it missed some notes, but my hypothesis is
| that it was the speed of notes in quick succession. If I do a
| follow up video with Jon Bot Jovi, I'll dig into it more!
| utopcell wrote:
| you bring shame to the world of robots! there are humans that
| can achieve 100% notes. mere, humans!
| nik12795 wrote:
| Hahaha
| StavrosK wrote:
| Great work! Though, wouldn't it be easier to just open the
| controller up and use transistors to close the contacts?
| Transistors are cheaper!
| nik12795 wrote:
| Yes! But I don't think that'd be as cool. I really wanted to
| replicate the motion of having to press the buttons :)
| StavrosK wrote:
| Definitely agree that this way is much cooler.
| layer8 wrote:
| Nice! If you're looking for a follow-up project, you could pick
| up on this guy's work and build a Dance Dance Revolution bot:
| https://www.youtube.com/watch?v=M4DvlAb4eA4 (...and then
| challenge Boston Dynamics to a DDR duel ;))
| nik12795 wrote:
| Lol challenging Boston Dynamics would be awesome
| LocalH wrote:
| >The game outputs at 30 fps and on faster songs, it would miss
| nearly 30% of notes.
|
| That 30fps is interlaced though, so there _is_ 60Hz motion. All
| of the PS2 GH /RB games I know run at 60fps internally.
| freedomben wrote:
| Nice work man, and posting it to HN was a great idea to start
| gaining some subscribers ;-)
|
| Giving away my PS2 and Guitar Hero is one of the only things I
| regret by the way. If someone created a Guitar Hero that ran on
| Linux I would gladly pay.
| geococcyxc wrote:
| There's Frets on Fire: http://fretsonfire.sourceforge.net/
| LocalH wrote:
| https://clonehero.net/
| maratc wrote:
| Absolutely amazing. I probably spent more hours playing Rock Band
| than all other games combined.
|
| One thing was a bit unclear: is Raspberry Pi doing the blob
| detection, or is it running on the Macbook?
|
| Also, why only 97%?
| stefanmichael wrote:
| a couple of nuances make the remaining 3% a bit more difficult
| to get, two that I noticed is that the bot is bad at holding
| sustain notes and also tends to make mistakes during long
| hammer-on sections because it hits two notes at the same time
| instead of realizing that they are individual notes.
|
| the second one is probably easy to fix by spreading the notes
| out a bit (hyperspeed mode) but the first one is a bit more
| annoying to program, probably
| coverband wrote:
| This was great, thanks for the clear explanation and no-fluff
| presentation. On a different topic, based on your other video ---
| what was the result from your Shark Tank audition?
| nik12795 wrote:
| Haha thanks for checking out the videos! That company ended up
| failing. I was almost on Shark Tank again for my company
| DirtByMail.com (we were talking with the producers of ABC), but
| didn't make it on.
| swyx wrote:
| freaking love your persistence. please keep it up! you earned
| my subscribe.
| nik12795 wrote:
| Will do, thanks!!
| sodality2 wrote:
| This is incredible, hope this video gets more traction!
| Subscribed.
| brunojppb wrote:
| This is amazing work man! Creative and fun work while learning
| cool tech.
|
| Through The Fire And The Flames Is still in my playlist to this
| day and reminds me about how cool was to play my PS2 back then.
| nik12795 wrote:
| Thanks a lot!! Yeah it's an epic song and is so nostalgic.
| dom96 wrote:
| Nice! Big Mark Rober vibes in the video, was it a major
| inspiration for how you put this together? :)
| nik12795 wrote:
| Thanks! Yes I love Mark and his videos. Huge inspiration for
| sure :)
| exdeve wrote:
| This is so funny! Jon-bot-jovi rocks!
| nik12795 wrote:
| Haha thanks! Yeah it's pretty hilarious
| ziml77 wrote:
| I love that you actually made it press the buttons instead of
| wiring directly into the guitar. Get all that authentic clacking
| :) Looks really cool!
| nik12795 wrote:
| Thanks!! Appreciate it
| jbverschoor wrote:
| One minor nit pick: It only strums one direction
| ziml77 wrote:
| That's how I used to play. For some reason strumming in both
| directions really threw my timing off. Damn did that hurt my
| arm for fast sections though (if it got too painful I would
| change to strumming both ways simply because the alternative of
| slowing down would end up with me missing more notes).
| kodt wrote:
| Also it isn't necessary to strum for every note in Guitar Hero.
| kregasaurusrex wrote:
| Very cool mixture of software and hardware for your project! I've
| been wanting to build a Go-playing robot for the longest time,
| hoping the cost of 6DoF robotic arms comes down in the near
| future. Do you have future tweaks in mind for improving your
| score, like increasing the signal of the notes when they're first
| starting to appear on the screen?
| nik12795 wrote:
| A Go-playing robot would be sweet!
|
| I think the biggest improvement for score would be implementing
| logic to hold down long notes and then implement star power at
| the ideal time. That would make a dramatic difference!
| Baeocystin wrote:
| I love this kind of silly/fun stuff. Thanks for sharing your
| project with us, it's great.
| nik12795 wrote:
| You're welcome! Thanks for checking it out
| StavrosK wrote:
| I love this, especially the sound the solenoids make when hitting
| notes. Well done!
| nik12795 wrote:
| Super satisfying right?!
| StavrosK wrote:
| _SO_ satisfying! I 'm going to make one for my keyboard and
| drive my wife insane.
| nik12795 wrote:
| Lol she'll love that
| aledalgrande wrote:
| Interesting, was expecting RL, but computer vision works too!
| nik12795 wrote:
| Ah yes. And it worked beautifully!
| njacobs5074 wrote:
| Have so many great memories of playing this game with my family.
|
| What an absolutely fantastic project you made for yourself!
| nik12795 wrote:
| Thanks a lot!! Really glad you enjoyed
| kevinherron wrote:
| What a trip... I also built a Guitar Hero "bot" some 13 years
| ago.
|
| Mine was not mechanical like this. Instead I spliced into the
| guitar with a microcontroller that I controlled via serial port
| from my laptop. I wrote software that analyzed recordings of the
| note chart, saved them to a file, and could play them back over
| the serial port by telling my microcontroller program what
| buttons to press.
|
| I don't have any pictures of the thing any more, but some of the
| YouTube videos live on:
| https://www.youtube.com/watch?v=n1YE8Wwo2rs
| nik12795 wrote:
| Wow how crazy is that! That's really great. Obviously I had the
| privilege to leverage modern tech with CV, but amazing that you
| were able to accomplish it 13 years ago!
| kevinherron wrote:
| Your version is so much cooler. It's more like what I had in
| my head but wasn't able to accomplish :)
| nik12795 wrote:
| Thanks!! It was super challenging but it was a fun time and
| the payoff of it shredding was well worth it
| jmfldn wrote:
| This is awesome. I was heavily involved with designing the audio
| and gameplay for DJ Hero, the sequel to Guitar Hero. I'd love to
| see you do this for DJ Hero next!
| nik12795 wrote:
| Wow that's sweet! Glad you liked it. What was the process like
| to design the audio/gameplay? Sounds like a great gig
| jmfldn wrote:
| Hey there, nice to meet some fans! Yeah it did really well
| critically and we shifted a lot of units but the marketing
| investment killed the profit as we spent on it like it was
| going to be the next Guitar Hero. It was always going to be
| more niche than that so this was its achilles heel.
| Activision loved what we did almost too much and killed the
| profitability. When I started to see Eminem and Jay Z turning
| up to promo events I thought "wow" / "oh crap how much did
| this cost us?!"
|
| Working on it was amazing. My job was remixing music for a
| video game and turning it into gameplay so what was not to
| love! I prototyped some of the earliest mixes in the game so
| helped to get the project through it's earliest milestones. I
| met a lot of famous people and got to remix Daft Punk, DJ
| Shadow and a few other heroes of mine. My favorite moment was
| opening the Ableton project sent to us by Daft Punk
| containing their stems. The music nerd / fanboy in me
| couldn't believe I got to play around with these stems that
| very few people ever had access to. They basically don't work
| with anyone as a rule so the fact that they collaborated on
| our project was insane. We had their likenesses in the game
| and the big pyramid stage. Was surreal!
|
| It was a great gig and I feel we produced a cult classic game
| that stood the test of time. The financials just didn't stack
| up.
| adrianh wrote:
| Thanks for sharing this -- that is really cool!
|
| What are you up to now? Still in the music/tech space?
| jmfldn wrote:
| Actually not any more. I'm a software engineer these days
| in the energy sector so nowhere near as glamorous! Gaming
| industry is a young man's game so I decided to move on
| and scratch my programming itch instead! I didn't fancy
| being a game programmer.
|
| Music is back to being a hobby again. Was just a 3-4 year
| thing that I went into it professionally.
| nik12795 wrote:
| Wow man that's an awesome story. Sounds like it was quite
| the ride. Thanks for sharing!
| vuciv1 wrote:
| Wow, thanks for sharing. Very cool story :)
|
| You put together a lot of fun hours in my childhood
| Baeocystin wrote:
| Random fan here: DJ Hero was my absolute favorite of that era
| of music games. You did a stellar job figuring out how to make
| it really feel like you were doing the thing, and not just
| hitting buttons in time. Thanks for a great experience. :)
| Lammy wrote:
| Off-topic, but I feel like one of the few people who really
| really loved* this game, so thank you! It was interesting to
| see companies like FSG and SEGA try to make a DJ game without
| getting sued by Konami, and I don't know of any other games
| besides CRACKIN'DJ and DJHero that employ the track
| switching/fading mechanic.
|
| [*] And I would have 1000/1000 achievement points except for
| the one mix (Beats & Pieces) I was never able to five-star on
| Expert difficulty :argh:
| vuciv1 wrote:
| Great job. Very well presented.
|
| Keep it ul!
| nik12795 wrote:
| Thank you!!
___________________________________________________________________
(page generated 2021-04-17 23:00 UTC)