[HN Gopher] Shouting in the Datacenter (2008)
___________________________________________________________________
Shouting in the Datacenter (2008)
Author : mmastrac
Score : 269 points
Date : 2022-08-18 18:32 UTC (4 hours ago)
(HTM) web link (www.youtube.com)
(TXT) w3m dump (www.youtube.com)
| arberx wrote:
| Its a Solaris machine...not surprised
| rubiquity wrote:
| Can you point me to your references that indicate HDDs running
| Solaris are the only ones prone to loud noises disrupting their
| mechanical nature?
| arberx wrote:
| It's a joke
| anfractuosity wrote:
| There's an interesting little paper 'Acoustic Denial of Service
| Attacks on HDDs' where they use a function generator to find
| resonant frequencies which cause issues with spinning disks
| tomalaci wrote:
| I have a colleague who was on the on-call rotation for some of
| our systems hosted in a datacenter our company owned. During this
| rotation he got unlucky in that the datacenter detected fire and
| decided to disperse gas that displaces oxygen (I think it was
| Halon gas?). Those systems are no joke - you have a short warning
| before you are supposed to put on a mask or get out of the server
| room otherwise you will just suffocate.
|
| Anyways, no fire damage was detected but oh boy... he had to run
| around various tech stores left and right to buy about 50-100
| enterprise grade spinning disks before returning to the site. The
| fire suppression system somehow broke most of the hard drives in
| the server room.
|
| This was more than a decade ago, however. Most of the disks are
| replaced with SSDs and so far they seem to be much more resilient
| than the spinning ones.
| treffer wrote:
| I was once told by a datacenter operator that this is due to
| the pressure change when they pump the gas in and that we
| should plan on replacing all HDDs should the fire alarm go off.
|
| ...and I think it was Argon...
| tomalaci wrote:
| Thanks for jogging my memory!
|
| It was indeed Argon, not Halon, and the pressure issues that
| broke HDDs.
| treffer wrote:
| And the other comment made me google noise vs. pressure....
|
| Looks like Siemens did an investigation and tested various
| parameters (pressure, noise, vibration) independently.
|
| They point towards noise, not pressure. Really interesting
| as I am _sure_ I was told it is pressure. But that might be
| 100% wrong!
|
| https://virz.de/wp-content/uploads/2020/05/2015-White-
| Paper-...
| TylerE wrote:
| Noise really is just pressure, with a high frequency time
| component.
| jsty wrote:
| > The fire suppression system somehow broke most of the hard
| drives in the server room
|
| Possibly another case of loud noise when the gas is released -
| see e.g.
|
| https://www.theregister.com/2018/04/26/decibels_destroy_disk...
|
| https://www.bbc.co.uk/news/technology-37337868
| jtwaleson wrote:
| Oh man. Once, when I was a young padawan doing maintenance by
| myself on one of our three racks in a colocation DC, I dropped a
| 1U server from about 2m/6ft high. It was scary as hell. While
| trying to grab it I hit the off button on one of the other
| servers, causing an outage for about 50 customers. What's more,
| it fell on the floor and just missed the SAN with the spinning
| disk array at the bottom of the rack. Everything was fine,
| including the fallen server, but shit, that was stupid. The
| vibration alone could have caused a lot of data corruption.
| anonymousiam wrote:
| Same effect mentioned here:
| https://arstechnica.com/gadgets/2022/08/janet-jacksons-rhyth...
|
| and here:
| https://devblogs.microsoft.com/oldnewthing/20220816-00/?p=10...
| somecommit wrote:
| Yeah I noticed this phenomenon already, every times I'm shouting
| at my collaborators they start crying and procrastinating,
| classic.
| humanistbot wrote:
| What a classic! One of the best feature demo videos of all time.
| bitwize wrote:
| Fus ro data loss!
| SulphurSmell wrote:
| I recall this fondly when it came out. I had a few racks of Sun
| servers at that time...and anyone I showed the video too were
| convinced it was bullshit. April Fools anyone? However, Brendan
| and Bryan know their onions...and all the naysayers had to pause
| and think a bit. It too crazy to believe...and too crazy to not
| believe. I miss this stuff.
|
| Edit: This was done when everyone ran spinning media. I wonder
| how modern SSDs hold up?
| bayindirh wrote:
| This works. I tried this on a Sun ZFS7320 under synthetic load
| to utilize all disks.
|
| We all laughed to the video, too.
| gorkish wrote:
| I did it on our fishworks box as well, before we put data on
| it. That could have been an amazing product if not for
| Oracle.
| dmitryminkovsky wrote:
| I wouldn't want to spend too much time in there without ear
| protection! Is it common for data centers to be this loud?
| c0balt wrote:
| Yeah, the fan noise is that loud. You should use hearing
| protection in there, even when staying for a short amount of
| time. Though I have heard about some special DCs that use water
| cooling and as such can defer the noise away from the racks.
| kqr wrote:
| Sure -- especially in the hot aisles it can be hard to have
| conversations. I doubt it's much worse than a nightclub though.
| quercusa wrote:
| Eh? Certainly back when all the disks spun. That and the server
| fans and the A/C make quite the racket.
| yellowapple wrote:
| I've been in my share of datacenters and most of 'em are at
| least a little bit quieter now. Still loud, but not to the
| extent that you have to shout constantly.
| marcosdumay wrote:
| Does yelling at your disks shorten their lifespan?
| 0xbadcafebee wrote:
| Only one way to find out...
|
| _weeks later_
|
| "Yeah, we had to let him go. Too burned out. Caught him
| screaming at the SAN."
| willcipriano wrote:
| End up like this guy:
| https://m.youtube.com/watch?v=ghv31AUufNk
| dinom wrote:
| Lol, this was brought up before... at least one time:
|
| https://hn.algolia.com/?dateRange=all&query=tDacjrSCeq4&sort...
| mynameisvlad wrote:
| > Please don't complain that a submission is inappropriate. If
| a story is spam or off-topic, flag it. Don't feed egregious
| comments by replying; flag them instead. If you flag, please
| don't also comment that you did.
|
| https://news.ycombinator.com/newsguidelines.html
| annoyingnoob wrote:
| We do Vibration Testing where I work (think satellite parts,
| simulating launch, etc). We moved to SSDs a long time ago because
| some of our test gear would shutdown at certain
| frequencies/levels during vib tests.
| 1-6 wrote:
| Old man yells _in_ cloud.
| [deleted]
| ivandenysov wrote:
| Who's laughing now?
| j1br wrote:
| Amazon
| NegativeLatency wrote:
| Reminds me of the mummy shouting thing:
| https://www.youtube.com/watch?v=G8qhrURrQbI
| roxaaaane wrote:
| Always makes me laugh ! We need more funny content on the top
| page!
| dmazin wrote:
| In case anyone needs to know, the shouter in question is none
| other than Brendan Gregg (of flame graph, eBPF, and Systems
| Performance fame). And the guy who took the video is Bryan
| Cantrill of Oxide Computer, DTrace, and Joyent.
| devonkim wrote:
| I think Brendan is still peeved that this video is his most
| viewed video on Youtube even after all this time.
| Workaccount2 wrote:
| He shouldn't be too upset, views and video quality only have
| a very rough correlation if at all. It shouldn't be
| surprising that his most accessible video is his most watched
| one.
| sydney6 wrote:
| Yes, whilst he has given so many other valuable talks around
| systems profiling, etc..
|
| I firmly believe that this [1] still should be hanging in
| every system administrators bedroom around.
|
| https://www.brendangregg.com/Perf/linux_observability_tools..
| ..
| devonkim wrote:
| I do have it hanging in my office rather than my bedroom. I
| wouldn't recommend putting it in the bedroom unless one's
| spouse is also a sysadmin.
| twic wrote:
| Maybe he should instrument his talks to see how much
| shouting there is in them, and check how that correlates
| with view count.
| devonkim wrote:
| This is a flame graph I'd be interested in seeing from
| him on April Fool's Day
| tantalor wrote:
| NobodyNada wrote:
| HN readers may also recognize Byran Cantrill from his famous
| rant about Oracle, which is frequently cited here:
| https://youtu.be/-zRN7XLCRhc?t=1980
| walrus01 wrote:
| I have trained myself so that every time I see "Oracle", I
| mentally translate it as "Larry Ellison needs a bigger
| yacht".
| noSyncCloud wrote:
| >Don't make the mistake of anthropomorphizing Larry Ellison
|
| Absolutely incredible!
| bcantrill wrote:
| Pretty funny to see this video at the top of HN! My kids still
| think I'm an idiot for not having somehow parlayed the enduring
| popularity of this video into being a YouTuber -- despite my
| protestations that I was merely the videographer here. If anyone
| is curious about the video's origins, I had a fun conversation
| about it with Ben Sigelman a few years ago.[0]
|
| [0] https://www.youtube.com/watch?v=_IYzD_NR0W4
___________________________________________________________________
(page generated 2022-08-18 23:00 UTC)