[HN Gopher] Hire-to-fire at Amazon India?
___________________________________________________________________
Hire-to-fire at Amazon India?
Author : bobjones334
Score : 711 points
Date : 2021-06-20 13:09 UTC (9 hours ago)
(HTM) web link (leetcode.com)
(TXT) w3m dump (leetcode.com)
| imwillofficial wrote:
| AWS employee not in India, less than a year in. I haven't seen
| anything remotely like this. So far it's the dream job. I've been
| the one learning to mature, adjust, and up my game, no the job
| failing me.
|
| But everywhere is different. If any job is I packing your health
| and wellness, mental or physical. Get out while you can. Life is
| short! No matter what you do, there is work that will treat you
| well and value you.
| almog wrote:
| Not in India, but a good opportunity to share my experience
| interviewing for a Software Engineer role with a team in AWS that
| was part of a recent acquisition (CloudEndure):
|
| After the interview was scheduled, I had some questions but up to
| this point, I didn't have any human interaction with the
| interviewer. It took almost two weeks to schedule a call with my
| recruiter (mostly because she wasn't great at responding to
| emails). During that, she explained that this specific team
| (CloudEndure) had a different hiring process: I was to have two
| 90 (!) minutes interviews on separate days, after which it'll be
| decided whether they'd like to continue my interview process
| within that team, recycle me with another team or reject me. The
| content of each interview, she told me, was to include Amazon
| Leadership Principles, algorithms/data structures and possibly
| some system design (the main SDI interview is only in a later
| stage).
|
| While I was preparing for the LP principles and the more standard
| parts of the interview, another Amazon recruiter contacted me and
| scheduled an interview to another group. Few days later he told
| me that since I already had an interview with Cloudendure
| schedule, and sinc "all interviews in amazon are uniform and have
| the same 45 minutes format", he'll cancel the interview that he
| scheduled and we can talk after my interview with Cloudendure. I
| reached back and asked whether anything has changed regarding
| Cloudendure's interview format and he apologized, explaining that
| he didn't knew they had a different format.
|
| The interview itself: as the interview started I learned that the
| guy who was interviewing me wasn't the guy I thought was going to
| interview me but someone from his team who filled in for him
| (later I learned that day US-East had a major outage which I
| guess was the reason for this). The 2nd thing he mentioned was
| "we won't do any leadership principles on this interview things".
| At that point I realized I just spend few days working on my
| stories, linking them back with each LP, but I let it pass.
|
| He dedicated 30-40 min to explain what the team is doing, then
| 5-10 more minutes where I went over my experience with him.
|
| Then came the technical part:
|
| The algorithmic problem was rather easy and I was familiar with
| it so I let him know of the latter, allowing him to choose
| whether he want to hear the gist and switch to another question
| or let me solve it as if I never saw it before. He chose neither
| and instead questioned me as to who told me about this question,
| which was an awkward way to ask where I met this problem,
| regardless of the problem itself which is rather common (easy LC
| question). As per his instruction, I continued to solve that
| question, where the tricky part are the possible inputs for
| parsing a string, so I went over all the edge cases prior to
| writing any code, proceeded by explaining my idea of how I'd
| solve it, then coding it while explaining what I was doing, and
| finally traced an example input by hand.
|
| The interviewer then proceeded to the 2nd question which was more
| vague and consisted of a system given as a synchronous single
| machine, single threaded black box, on top of which I had to
| implement undo/redo. I won't go into all the details here but I
| went from a naive space inefficient solution to an optimal
| solution, making sure I don't cause any infinite feedback loops.
|
| The interviewer and I were discussing the possible solutions
| through out the 2nd question. He did provide me with hints seemed
| happy with my solution and by the end asked me if I could
| allocate few more minutes where we continued discussing different
| designs.
|
| The interview in total took 2 hours and 5 minutes (!!), 35
| minutes over time and I thought I did great, the interviewer
| seemed to like my approach to problem solving.
|
| Two days later I got an email that they "have decided to continue
| with other candidates". I was confused since the recruiter
| clearly mentioned that such decision was to be taken only after
| the 2nd interview, and so I emailed her, asking to schedule a
| call sometime that week. She didn't answer my (two) emails nor my
| two phone calls.
|
| I contacted the other group recruiter too. He answered promptly
| and told me that he'll try to check if he can see whether he can
| schedule another interview for me, but few days later he
| apologized, saying that he can't nor does he have access to
| anything within the team that I was interviewing with.
|
| I was very frustrated with that experience, most of all, by not
| knowing why I failed the interview. I have interviewed
| successfully and unsuccessfully with FAANG in the past, but never
| got zero feedback and complete ghosting from the recruiter.
| dang wrote:
| I've changed the title to try to make it less linkbaity and more
| neutral, in accordance with the site guidelines
| (https://news.ycombinator.com/newsguidelines.html). If anyone can
| suggest a better title--i.e. more accurate and neutral, using
| representative language from the article itself--we can change it
| again.
|
| Usually this sort of ambiguously-sourced riler-upper doesn't make
| for good HN discussions, but this thread is extraordinarily good,
| with many informed comments from all sides of the question, so I
| don't want to downweight it.
| buss wrote:
| I worked at Amazon years ago (2010-2012) on the retail website
| search navigation. I didn't like it, but it was nowhere near as
| bad as this hyperbolic post.
|
| Joining Amazon was the best thing I could have done with my
| career at that stage in my life. I learned a ton! But the culture
| does grind you down, especially when your alternatives are much
| more cushy.
| rejectedandsad wrote:
| I didn't get any cushier offers out of undergrad and have
| worked at Amazon since - do you think that reflects poorly on
| me? I self harm sometimes because many people think it does...
| fridif wrote:
| If you don't desire to seek medical help, the best thing you
| can do is stop self harming and stop caring about all the
| possible negatives you are imagining.
|
| I've gotten fat from all the stress of being a working man. I
| need to stop caring about all of the pitfalls and just live
| my life.
| [deleted]
| earth2mars wrote:
| Current AWS employee. Technical sales side.
|
| I don't disagree with what OP was saying. I have friends who
| experienced this. But my experience is no way closer to this.
| Think of this. The company have million+ people working. It got
| many companies in it (Acquisitions and independent business
| units). At least 30% are technical side. Out of it around 50%
| face this issue who are on development side. Which is still
| significant. Especially on call. other stuff like URA still
| applies for everyone. Work life balance is upto manager, team,
| business unit etc. Can't really generalize. You get to work with
| great companies and build great experience for your career if you
| got lucky. If someone doing their job,it's really hard to go
| things wrong way. Managers are always under hiring pressure and
| they tend to hire wrong folks most times. The interview process
| have lot of bias baked in. We miss lot of good folks and hire
| wrong folks too.
|
| But I agree this toxic culture need to change.
| throwaway2037 wrote:
| "million+ people working". Yeah, most of them are filling and
| taping boxes in warehouses. I think people are discussing
| software engineers which might only be 20K or so.
|
| I cringe when I see similar comments about Apple. The vast
| majority of their staff are not highly paid engineers that roam
| HN. Instead, they are Apple Store staff or phone technicians
| (is that still a thing in 2021?).
| amazon_throw wrote:
| Our technical staff is probably closer to 100k than it is to
| 20k. It was 20k when I started over 10 years ago.
| rejectedandsad wrote:
| I did digging on this a few weeks ago, this is correct if
| you add up SDE, SDM, TPM, Scientist etc roles.
| ixaxaar wrote:
| Ground reality in all big tech seems to be pretty much the
| same? Like 1% of the people hoard 99% of the good work?
|
| [Technically I imply a heavily skewed exponential distribution,
| maybe not exact numbers].
| [deleted]
| williesleg wrote:
| Fud. More union trickery.
| franczesko wrote:
| There are only two instances when you're happy at Amazon - when
| you start and when you quit. Doing the latter was the best
| decision I made, perhaps, in my entire career. I can honestly
| tell that those RSUs and bonuses weren't worth it.
|
| I can assure you that in the other areas of the company (non-SDE
| roles and not exclusively in India) things are WAY worse.
|
| Avoid it at all costs. You've been warned.
| angry_octet wrote:
| This is a great example of culture clash. In the US, being cut
| means nothing, you've always got to be ready for another job
| search. Only a few workplaces (Govt, places with strong unions,
| full professors) have any protection against retrenchment or
| arbitrary firing. But in India, jobs are careers for life, and
| your employer is part of your personal brand. The cachet of AMZN
| is great, until you realize they are going to fire 20% of you
| every year. People will say "they must have fired him for a
| reason" because they don't understand it's just about metrics.
|
| It's also common in India to solve problems by throwing people at
| it (and indeed the US if it involves minimum wage workers) rather
| than difficult tasks like fixing the root cause or improving the
| process. It's also super hierarchical (I recall watching the
| manager watching the engineer, who watched the junior engineer,
| who watched the tech type at the console, telling him what to
| do), encouraging lots of butt kissing. So overstaffed teams, 1/3
| of whom do not much, 1/3 (with replacement) who suck up
| ferociously to their manager, plenty of drudge work, and fear of
| the stigma of being fired. Great combination.
| blocked_again wrote:
| You theory does't makes much sense.
|
| Amzon USA - 71% software engineers recommends Amzon to a friend
| in Glasdoor. 87% Approves of CEO.
|
| Amazon India - 87% software engineers recommends Amazon to a
| friend in Glasdoor. 94% approves of CEO.
| brown9-2 wrote:
| Glassdoor is not reliable
| blocked_again wrote:
| > But in India, jobs are careers for life
|
| But remarks like this with no data whatsoever on the
| average turn over time of Indian vs US engineers are more
| reliable than Glassdoor rating of thousands of people?
| season2episode3 wrote:
| What are your go-to alternatives? Blind?
| perryizgr8 wrote:
| > in India, jobs are careers for life, and your employer is
| part of your personal brand.
|
| This might have been true at some point in the past, but it is
| increasingly changing. In Bangalore, especially in tech
| companies, it is not a big deal to get fired. You can easily
| get offers within a week if you are any good.
| angry_octet wrote:
| Yes it is changing fast, but that doesn't mean your parents
| understand, or mainstream companies.
|
| It's probably better to understand that these jobs are
| temporary and always be hunting for your next gig, but it is
| against the cultural norm, where your uncle works at the same
| PSU he started at after graduation. It probably doesn't help
| that every IIT graduate is expected to earn 1 Cr.
| nonamechicken wrote:
| > In India, jobs are careers for life, and your employer is
| part of your personal brand.
|
| This has not been the case for at least the last 10+ years. I
| cant speak of the 'product' companies such as Microsoft, Google
| etc. But for anyone working in the WITCH like companies, only
| way to get your salary changed is to switch companies. People
| change their jobs every 2 years or so, especially in their
| first 10 years. After that, it becomes less, probably because
| WITCHes feast on the young ones more, making more money from
| them. Openings for 10+ years are very less in these companies.
| WITCHes hire fresh graduates and pay them very less, hardly
| enough to survive as a bachelor in a big city like Bengaluru.
| So the only way to get your salary changed is by finding
| another job. When a person with 2 years experience change job,
| they often get double of what they were getting earlier. And
| every job change from then on comes with a 20-40% increase.
| Loyalty penalty is a very real thing in these companies.
|
| I don't think anyone in the Indian IT industry thinks jobs are
| careers for life. I feel like our shelf life is around 40 years
| of age. Only a few survive the industry after that. Sad thing
| is there is no social security like in US, so we are completely
| on our own. I was pleasantly surprised to find out that there
| were so many senior engineers working in US companies. In WITCH
| like companies, every single team I have seen are structured
| mostly with 0-3 years, a few 4-8 years. People with more
| experience expect more salary. Clients want cheap 'resources'.
| So, these companies hire mostly the cheaper ones.
| satyanash wrote:
| > WITCH
|
| I would presume: Wipro, Infosys, TCS, Cognizant, HCL
| solarmist wrote:
| What are WITCH companies?
| ixaxaar wrote:
| > But in India, jobs are careers for life, and your employer is
| part of your personal brand.
|
| In the last decade, I might have very rarely seen a resume with
| more than say 10 years of experience at a single place. The
| average would be somewhere in the range of 1-4 years.
| cuu508 wrote:
| Makes sense, people with 10+ years at one place are looking
| for a new job less often ;-)
| throwaway2037 wrote:
| I agree 100%. When we hire in or from India, the turnover on
| their CVs is incredible. Some people only stay one year, and
| then leave. How much impact can you really have after one
| year? It's bizarre, but I try very hard not to discriminate
| and just chalk it up to "local working culture".
| ixaxaar wrote:
| An average software engineer can traverse the entire SDLC
| of a major revision of a product in less than a year.
| Perhaps in big tech that would be ~2 years if not less.
| isuckatcoding wrote:
| Side note: I'd love to use a forum like this but filter out all
| the Indian specific things. Is that possible?
| cleandreams wrote:
| Anecdote. I used to work for Microsoft from the Bay Area and I
| was up in Seattle once for a business trip when I took an Uber
| somewhere. I struck up a conversation, "I bet you drive a lot of
| Microsoft employees around." The driver said, "Yeah. Also
| Amazon." I asked him if he observed any differences between these
| two groups. "Yeah," he said. "The Amazon employees are often
| crying."
| ignoramous wrote:
| > "Yeah," he said. "The Amazon employees are often crying."
|
| Ex-AWS. Can confirm. More often, those were tears of joy from
| rising stock prices. /s
| midhhhthrow wrote:
| I can't possibly imagine why anyone would ever go work at Amazon.
| How low would your prospects have to sink in order to make
| someone do that to themselves? I suppose if you're completely
| homeless and in danger of starvation. Personally, I'd rather eat
| the weeds that form on the sidewalk or probably go dumpster
| diving. That's much more dignified than working at amazon.
|
| Just imagine the damage you're doing to your resume. How do you
| explain having worked at the worst company in the world?
| rantwasp wrote:
| well... you're being downvoted.
|
| first: if you were truly homeless and in danger if starvation
| you'd work there. it's also not cool to diminish people because
| they're homeless.
|
| second: having Amazon on your resume is not bad. It shows that
| you can work hard and if you managed to survive in there you're
| probably thrive in other environments.
|
| third: overall I agree with the sentiment you are expressing
| throwaway666775 wrote:
| I currently work at Amazon as an engineer(NA), joined around 1
| year 5 months ago. My experience has been quite different than
| the one linked here and commonly talked about on the Blind
| community so I'm posting here to add a data point.
|
| Maybe I'm just lucky but having first hand experience around my
| team and the 5 teams surrounding it, things are very manageable.
| Our oncall is a bit hectic but you're not oncall that frequently
| for it to bother me personally. I work on projects that have a
| lot of impact and challenge me technically. The key here is to
| manage expectations very clearly before the beginning of a new
| project, this sets you up for success for the next -2-3 months in
| implementation.
|
| Overall, I feel like I don't overwork myself, my manager cares
| about me and my mental health and supports me in growing as a
| professional. The only time I've seen people work a ton of hours
| is if they slacked off , which then becomes a problem they
| brought onto themselves. If there is a crunch time that was a
| result of something out of our hands, my manager is empathetic to
| that and encourages us to take time off after the project is
| done.
|
| I'm a happy person to be working at Amazon, under my current
| manager and I see no reason to change that in the near future for
| me. Amazon is not for everyone. While my manager cares about me,
| he also has certain standards in the quality and timely delivery
| of value from me. Working 9-5 is entirely possible at least in my
| org, and most people that I know do work 9-5, but those 9-5 hours
| are going to be busy and you have to prioritize well.
|
| If you are a completely new in the field, want to coast or chill
| I think there are better places than Amazon to work.
|
| Edit 1: grammar/spelling
| whoknew1122 wrote:
| Current AWS employee. I haven't seen anything remotely like what
| is described in the OP. It strikes me as hyperbolic and a
| regurgitation of common posts on Blind.
|
| My work life balance is good. My manager is supportive of me and
| my org takes mental health seriously. We don't hire to fire; in
| fact we can't hire enough people to keep up with customer demand.
|
| Amazon is a huuuuge place with many different orgs. Maybe you can
| get stuck on a bad team in a unsupportive org. But to say every
| org operates as the OP alleges is simply not true.
|
| I'm sorry they had a bad experience at Amazon. But their
| experience is in no way indicative of every org as they allege.
|
| Also, I find it somewhat interesting that everyone who leaves
| Amazon after a negative experience blames URA and interoffice
| politics. It's almost as if it's never the person's fault they
| got fired.
| Immune wrote:
| How do you explain the high turn over rate that can be easily
| seen using the old fart tool?
| whoknew1122 wrote:
| I'm not sure that it's fair to ask a random employee (myself)
| to explain the employee churn for an employer that has over 1
| million employees. That being said, it is largely based on
| the org you're in.
|
| My org (AWS support) has a high churn rate because it's
| largely seen as a stepping-stone to other places in AWS.
| People get hired into my org, learn AWS really well, and then
| go one of a few different places:
|
| - AWS operations (e.g. SOC, NOC) - Service team SDEs -
| Technical account managers - Solutions architects - Training
| and certification
|
| Or they simply get enticed to work for someone who uses AWS.
| I've worked on literally thousands of support cases. I've
| gotten really good at diagnosing and fixing things on AWS.
| And as such, I have companies reaching out to me nearly daily
| with some job or another.
|
| Similarly, for SDEs, working for a FAANG can open lots of
| doors for you.
|
| The question can be similarly asked: If Amazon is 'lethal' to
| your career and health as the OP said, why are there so many
| boomerangs?
| harshalizee wrote:
| And yet, as a random Amazon employee, you feel empowered to
| dismiss other's cautionary tales as hyperbolic and
| regurgitation of blind posts?
| franczesko wrote:
| Please define 'many boomerangs' in the context of 1 million
| employees (FC workers shouldn't count).
| SilurianWenlock wrote:
| "The work here isn't technically challenging at all"
|
| Why isnt the work technically changing? What is this work likely
| to be?
|
| What sorts of CS work is the OP hoping for?
| amznthrow0000 wrote:
| I joined Amazon and was soon put on the devlist (i.e. preparatory
| chopping board) by a new manager.
|
| I only knew that I was at risk of losing my job after 4-ish
| months, at which point the secret deadline was very close.
|
| That period was extremely stressful and honestly left some scars
| (that have mostly healed).
|
| I saw friends with families and fresh mortgages get fired instead
| of me. I was essentially fighting against them (who were
| unbeknownst to them also on devlist) to showcase my deliverables
| to leadership and demonstrate I pass the bar.
|
| After that I moved to a team in AWS that I love. Management puts
| focus on our health, invests in operational improvements, and
| people work normal hours. Senior SDEs are vocal about taking it
| easy and taking care of ourselves when we need to.
| [deleted]
| curation wrote:
| Who are the people who still dream of careers? I don't know any.
| nkohari wrote:
| Hi, I'm Nate. Nice to meet you.
| alisonkisk wrote:
| Young ambitious people.
| asdev wrote:
| Young ambitious people are starting companies, not going
| corporate.
| SilurianWenlock wrote:
| How hard is it to become competent at full stack
| development without working for someone else first?
| f6v wrote:
| > Who are the people who still dream of careers?
|
| Those who like the paycheck attached to a career. 1 million in
| stock over 5 years is a good deal.
| benrbray wrote:
| For me, I'm not quite ready to give up on the dream having a
| job that I actually enjoy. I'm still young, so effort I put
| into my career now will pay dividends for the rest of my life.
| Giving up the most productive hours of my day to a company is a
| big deal, and I'd at least like the job to be intellectually
| stimulating, if not financially rewarding. At this stage in my
| life, showing up at work just for the paychecks seems like
| giving up on a brighter future.
| hinkley wrote:
| Despite ballooning tuitions, software development is still a
| career that people of modest means can aspire to.
|
| That we don't talk about it more here I think says something
| about how you only talk about escaping in certain company. You
| can't pass for upper class if people know where you come from.
| jwilber wrote:
| Oh man, these "I don't dream of labor" types are so fucking
| cringe, lol.
|
| Also condescending!
|
| A close friend of mine beat cancer as a child. Ever since then
| he's dreamed of working in oncology, and is finishing school
| just this year. He's very excited, and I'm happy to see his
| career goals coming to fruition.
|
| Other examples abound.
|
| I find, in my social circle anyway, the type who ask, "who
| dreams of careers?" are usually the type to have had cushy
| upbringings that afford them the ability to ignore dreaming of
| finding stable, well-paying careers. Let alone the mission-
| style careers I described earlier. Must be nice.
|
| In any case, I've worked on teams in both Amazon and AWS. Have
| not experienced the described culture. However, I've only ever
| worked in science roles/orgs, and I hear the difference between
| science and software (particularly product) can be quite stark.
|
| As others have mentioned: most variation is based on your
| manager and skip-level manager.
| approxim8ion wrote:
| Finding stable, well paying careers is more of a means to an
| end. I think some in this group of people (myself included)
| are closer to the "I don't dream of labor" type than the
| "mission style careers" type. That said, I am open to
| changing my view and don't resent anyone who has found a
| mission worth working towards.
| snarfy wrote:
| This article was on the front page for all of 30 minutes before
| being buried to oblivion.
| easton wrote:
| If the article has more comments than upvotes the algorithm
| uses that as a signal that a flame war is happening and buries
| it. If it gets upvotes it'll move back up.
| alwayshasbeen wrote:
| I was contacted by an Amazon recruiter some month back. The
| experience was so unpersonal, I felt like a soulless robot: I
| just got a link to a HackerRank test and solved some automated
| puzzles with no one but me and the clock ticking. How I
| understand coding tests is that there should be a human being
| that evaluates how you approach a problem, not a fully automated
| assignment.
| UK-Al05 wrote:
| That's just the initial filter. You'll eventually get a in
| person interview. Tbh its pretty standard.
| djmips wrote:
| Hire to fire might just be the sign of a healthy organization
| that only wants performant employees. You can't really tell how
| well someone is going to do just form an interview. Given a
| reasonable sample of work you'll know if you should keep them or
| not. Of course, human nature will lead to some of the perversions
| of the core idea and lead to good people being let go and buddies
| being kept on.
| nobleach wrote:
| I interviewed with Amazon and Facebook in the Spring/Summer of
| 2020. The difference was night and day. Amazon felt like I was
| part of a huge cattle herd. The recruiter contacted me and told
| me they needed to hire a bunch of people in my area. I was
| immediately given a course of study, link to common LeetCode
| questions and told I had a couple of weeks. That fine. I enjoy
| solving puzzles. Facebook felt VERY different. The recruiter had
| a conversation about career goals, told me how wonderful it is to
| work at FB, and then quizzed me on some basic CS concepts. The
| ensuing rounds for each got even more divergent. The Facebook
| interviewers constantly made me feel like my success was a "win"
| for them. It felt like a team! Amazon felt like they were waiting
| for me to screw up so they could disqualify me. In the end, I
| dropped out of Amazon because it felt disgusting. And I could
| tell, if this is how they bring me in, I can't see any reason why
| it'll magically get any better once I was "in".
|
| My feeling on those who use LeetCode as some sort of indicator.
| Great. You've found the people that do the brain-teaser puzzles
| at Cracker-Barrel. I'm one of those people. Some of the best
| people I've worked with, would fail those tests immediately...
| yet, they've built scalable, performant enterprise software. I
| now see those tests as nothing more than a way to reduce the
| number of applicants.
| baby wrote:
| I interviewed with google, fb, and amazon and fb really felt
| like the more human interview of the three. Now, do I like the
| leetcode/system designs interviews of fb? Nope. But definitely
| the least bad.
| nobleach wrote:
| The biggest problem I have with those "canned" systems, is
| that there's no real conversation. I have a REALLY hard time
| even understanding what's being asked. In a real environment,
| I can ask probing questions. I have had to do a TON of
| HackerRank, LeetCode, and Code Katas to get used to how the
| questions are framed. But, I definitely enjoy solving those
| types of puzzles.
| [deleted]
| noodle wrote:
| How long ago was this? Its been a while since I talked to
| either amzn or fb, but their processes were basically the same
| to me. Your description of amzn was almost exactly what I
| experienced with fb. It would be nice to hear hiring practices
| changing at a big tech co though.
| ucm_edge wrote:
| I had a similar experience at Amazon. About midway through the
| onsite at Amazon I kind of decided "I don't think I'd like
| working here and I already have other offers, so my brain will
| be going into neutral now." and kind of checked out. Felt very
| much like the interviewers didn't want to be in the room with
| me and just wanted me to fuck up so they could click "No hire"
| in the HR app and get out.
|
| Walked out figuring I probably wouldn't get an offer, but given
| they apparently needed a ton of people maybe they'd down level
| me and offer (I figured I'd checked out to the point there was
| no way I'd get that SDE III level offer).
|
| Recruiter actually sent me a very nasty email accusing me of
| wasting her time, I must be a bad engineer who over sold my
| experience, blah blah blah. Made it sound like I was blacklist
| for life. Three weeks later a hiring manager emails me and is
| like "So I found your interview notes in our internal database,
| you look like an amazing candidate and I want to fast track
| your hire onto my team."
|
| I have no idea what is going on Amazon, I mean they built AWS
| so this must work on some level, but I just have such a bad
| taste in my mouth from my experience I don't want in on it.
| Maybe it's working as intended in that they want a process that
| skews toward hiring certain personality types so having other
| folks self select out is a win for them. Although I get pinged
| by them 2x a month and regularly offered fast track hirings
| based on those notes from that interview I did 14 months ago
| which is also weird. Which makes me think it's more likely
| their hiring a mess and they survive via throwing money at the
| problem and big old RSU offers at candidates.
|
| e: Actually not 14 months, it was pre COVID lockdown, so even
| further back.
| soared wrote:
| Amazon has a hiring status that is basically "probably good
| to hire, but not a fit for my team". Other team managers can
| go check that status and look for candidates that have been
| through the process already and basically do 1 short
| interview then hire the person. I think it's called
| recycling?
| nobleach wrote:
| It's that subtle view into what your life might become that
| makes it so much easier to decline to move on. I'll admit,
| it's probably easier when I already had a job that paid well
| and I didn't hate...
|
| A recruiter that couldn't keep it professional... that is
| EXACTLY how I felt! I get that when you work for FAANG,
| you're used to the stigma your company carries. And you're
| used to people salivating over the idea of working for you.
| But there are some of us out here who already make a decent
| wage, and have been in the business long enough that we don't
| see having FAANG on our resume as the most important thing.
| I'm at the phase of life now where I really am interviewing
| YOU as much as you're interviewing ME. If I don't like what I
| see, I can walk away.
| TimPC wrote:
| If you bombed your interview amazing candidate could be code
| for they need someone to hire to fire.
| ryanSrich wrote:
| I interviewed for a Sr. Technical Product Manager role at AWS.
| This was shortly before Covid happened.
|
| Every interview seemed to go well until the on-site. It
| literally felt like I was wasting their time. They gave me very
| bad holier than though vibes. It seemed like every answer I
| gave was just given a head nod then on to the next question.
| They asked very basic level PdM interview questions. Things I
| would expect a first time PdM to know, which surprised me
| because I was interviewing for a Sr. role.
|
| I assume I didn't integrate their "principles" enough into my
| answers because I was rejected and put on a 6 month waiting
| list to reapply again.
|
| That ended up being the last straw for me. I was already on the
| fence about working at a BigTechCo based on previous bad
| experience. I ended up starting my own company. Best decision I
| could have made.
| SkyPuncher wrote:
| > My feeling on those who use LeetCode as some sort of
| indicator.
|
| I'm fine if you need me to at least demonstrate _basic_
| competency with a few simple Leetcode questions. There are a
| LOT of candidates who don't actually know what they're doing
| (either they're new or incompetent). It's okay for you to make
| sure that I have half a clue of what I'm doing. The problem
| becomes when Leetcode is used as an actual measurement of
| competency ceiling.
|
| The best engineers I work with would completely fail at
| Leetcode challenges because they've found easier, simpler ways
| to implement something.
|
| For example, anything that lodash provides is
| Leetcode/HackerRank under the hood (not a dig a lodash, just
| the type of problems Leetcode tends to ask about). I would
| expect a senior engineer to be able to replicate any of that
| code, but I'd first expect them to know that it's not worth
| their time to replicate that code. Instead, they should find a
| well tested tool instead.
| allenu wrote:
| I got a similar feeling when Amazon once reached out to me.
|
| I got a recruiter email and I wasn't super interested in
| working there, but was a little curious, so I responded. They
| wouldn't tell me what project it was or what I would be doing
| exactly, but said that once I was brought in for a full
| interview loop I'd sign an NDA and they'd tell me all about it.
| I said I wouldn't mind having a conversation on the phone just
| to learn what I could from the hiring manager since I wasn't
| comfortable spending time on a phone screen if I wasn't even
| interested in the job or the project. The recruiter set me up
| with a call with somebody on the team.
|
| So I had a phone conversation with somebody on the team,
| expecting it would be an informal chat about what they do and
| what they're looking for, but after about 20 minutes they
| started asking me design and behavioral questions (i.e. given
| situation X, tell us what you would do). I was really annoyed
| because I just wanted to get a feel for what the role entailed
| and what they were looking for, but they turned into a sort of
| interrogation to see if I had what it takes to join at all.
|
| I was already starting to look for a new job at the time and
| was familiar with Amazon's principles, so was sort of prepared
| for it and I answered all their questions quite well, and they
| seemed happy. I kept trying to steer it back to things I wanted
| to know, however, but it was hard.
|
| Overall, I felt like the call was a waste of my time since I
| didn't learn anything about the job. I guess you could say it
| saved me time later since I decided not to respond to any
| future emails that other recruiters sent me.
| [deleted]
| [deleted]
| blueblisters wrote:
| FWIW, the FB recruiter I spoke with also sent me Leetcode
| links. But the rest of the process was one of the best
| interview experiences I had, for a company of this scale. Not
| sure how well FB takes care of its employees, but they sure
| take good care of candidates.
| intricatedetail wrote:
| > I now see those tests as nothing more than a way to reduce
| the number of applicants.
|
| More like a way to discriminate neurodiverse people. This is
| probably illegal, but I have not heard of anyone challenging
| it. If you create a test that certain people fail, that would
| otherwise do their job fine, then this is likely testing for
| protected characteristic and disguised as competence test.
| mLuby wrote:
| Would you have to prove only that the test discriminates, or
| also that it was _intended_ to discriminate? The latter
| sounds nearly impossible.
| danbrooks wrote:
| I had a very similar experience with Amazon and Facebook
| interviews.
| nostrebored wrote:
| Honestly I've been to offices at Amazon in India, and this was
| not my impression. I would slack Indian coworkers and not hear
| back for ages even during normal working hours.
|
| At the end of the day the reality at Amazon is that your manager
| dictates your experience. If you join a bad team you will have a
| miserable time. The best work experiences of my life were also at
| AWS.
| [deleted]
| iJohnDoe wrote:
| I'm guessing another lousy side of this is the toxic HR as
| mentioned in the post. Let's say you lasted a year and left. Now
| you can put Amazon on your resume. However, HR is so toxic that
| they'll never give you a good reference if someone calls in. Now
| all that work and effort was really for nothing.
|
| Edit: I meant if someone calls into HR to verify employment. You
| run the risk of HR secretly bad mouthing you. Yes, it's probably
| illegal, but if they are doing all this other stuff, then would
| you be confident what they say or don't say?
| __derek__ wrote:
| > I meant if someone calls into HR to verify employment. You
| run the risk of HR secretly bad mouthing you. Yes, it's
| probably illegal, but if they are doing all this other stuff,
| then would you be confident what they say or don't say?
|
| Employment/salary verification is outsourced to The Work Number
| (an Equifax product). Nobody at Amazon is involved.
| oblio wrote:
| Do references really work like this? I've always given them the
| name of someone I worked closely with.
| ghaff wrote:
| HR at a large company basically never gives a good reference if
| someone calls. They'll acknowledge the dates you worked there.
| alisonkisk wrote:
| That makes no sense. HR gives employment confirmation.
| Individual people you know choose whether to give references.
| pureliquidhw wrote:
| Amazon HR won't give a reference, period. They'll verify
| employment dates, and I bet that's handled by a third party. I
| haven't had a reference check in the past 3 jobs over 5 years.
| loloquwowndueo wrote:
| " If you're someone who likes your 7 hours of sleep a day, stay
| away." how do people function long-term with so little sleep? I
| need at least 9 hours to not be a useless husk the next day.
| ricardo81 wrote:
| Used to pull a lot of all-nighters coding in my teens and
| twenties. Being twice as old now I'll regularly clock 8 hours
| sleep and doing an all-nighter never crosses my mind.
| markus_zhang wrote:
| Me too. I need some 6-7 hours in week days and 8 hours in
| weekends. I'd trade a few years at the end of my life (hope I
| don't die soon) for 2-2.5 hour reduction of sleep plus still
| keeping my sanity and productivity. It would be a good trade.
| perryizgr8 wrote:
| I usually average 4.5 hours a night. I am still living, but it
| is not a happy life. I slept 8 hours one night, felt like a new
| person the next morning. But I fall back into the habit again.
| vorpalhex wrote:
| Obviously everyones situation is different, but if you can
| manage more sleep you'll likely find the rest of your day is
| overall more productive despite being shorter.
| papito wrote:
| Just doing mundane things like decluttering and dishes at
| night helps relax the brain and wire it into the sleeping
| habit loop.
|
| Don't start anything major after 6, and definitely not
| "squeeze in" a movie or a show before midnight. That's how it
| happens.
|
| It's surprising how fast the time lapses if you do nothing
| but muck around the house getting ready for bed.
| Ozzie_osman wrote:
| I was the same then I had kids and now I probably get 6-7 hours
| per night, and they're fragmented. Your body adjusts. You use
| caffeine more effectively. One big learning for me is that i
| blamed a lot of being tired on lack of sleep, when in reality
| there are other causes (ie a big or greasy mid-day meal,
| sugar/caffeine crashes, etc).
|
| That said, i'm sure it has long term health consequences
| though.
| [deleted]
| nobleach wrote:
| Having worked in an environment where there was a constant
| onslaught of customer issues. I can tell you that even though I
| was offered more than 7 hours of sleep, my days were absolutely
| terrible. Fire-fighting all day, and never getting to work on
| the features that we promised to deliver led to me lying in bed
| awake. My mental health was extremely fragile. My physical
| health was worse. The promise of "Work/Life Balance" needs to
| be clarified. Does the "work" part bleed into your "life" part?
| Does the job make it possible to truly disconnect? Some of that
| is based on personality (I aim to provide value, and am truly
| jazzed when I know people are pleased with my performance), but
| some of that is based on what your daily grind looks like. If
| you are constantly dealing with a nightmare for even 8 hours of
| your day, it doesn't matter what the rest of your day is like.
| 63 wrote:
| I knew someone who went years with 5 hours a night. He was
| notoriously quick to anger and tended to over react to
| everything. Toward the end of my knowing him he started to take
| better care of himself and suddenly he was the nicest man I'd
| ever met. Sleep matters.
| ravenstine wrote:
| I knew a guy who claimed he only ever slept 3 hours a night.
| :) There was nothing obviously wrong with him at all. Older
| guy, too. His job was his passion, though, which I imagine
| helps quite a bit. I doubt he had to drag himself out of bed
| ever.
| nkozyra wrote:
| Some people have a weird tendency to talk up how little
| they sleep like it's some point of valor.
|
| Some people can do 6 hours a sleep a night for a long time
| and be fine but I don't think there's much support for
| maintaining health on 5 or fewer for long periods of time.
|
| Sleep is important and people need to find ways to get what
| they need. Even people with infants need outside support so
| they can get what they need. It's not something to be
| embarrassed about.
|
| Companies should be very cognizant of the types of
| constraints they may be putting on a work-life balance.
| Those that don't obviously treat employees as inherently
| disposable.
| saagarjha wrote:
| Some people just don't need much sleep.
| nkozyra wrote:
| Sure, and the short sleep gene people are brought up here
| in this thread a bunch.
|
| But that's not < 4 hours a night without naps for long
| periods of time.
| ravenstine wrote:
| Another unappreciated aspect of sleep is how much of it
| one needs depending on their age. As I've gotten older,
| I've noticed that I need far less sleep. I still try to
| get 8 hours if I can, but most of the time I get away
| with 6 or 7 just fine and I get up much earlier in the
| morning. I think adolescents need more like 9 hours of
| sleep, but for some reason we (in America anyway) make
| kids get up earlier than most adults and give them work
| to do at home so they have less time to get to bed early,
| giving them all the incentive to stay up later. Adults
| trivialize sleep, which is funny because in high school
| and college it seemed agreed upon by everyone that we
| actually needed that extra hour or two. Teachers and
| parents would tell us that we were merely "slacking off"
| if we slept for more than 8 hours and didn't get up at
| the buttcrack of dawn.
|
| Millennial parents undoubtedly have their own set of
| problems distinct from past generations, but I hope
| they've learned by now that their parent's views on sleep
| don't need to be repeated just as they're not repeating
| the 9-to-5 butts-in-seats mentality that is no longer a
| universal.
| igetspam wrote:
| This is me, in waves. When I become conscious of my crappy
| behavior, I take care to course correct. It's doesn't last
| though and I start to get burned out and stubborn and
| unpleasant. It's not a thing of pride. It's a known issue
| that I've only ever avoided while unemployed.
| [deleted]
| zrail wrote:
| I'm a parent to two small children. Sleep is incredibly
| important but I think I probably have another year at least of
| seven hours interrupted at least once as my normal.
|
| It sucks.
| wiredfool wrote:
| It was an amazing feeling when the interrupted nights
| stopped, after roughly 9 years over three kids.
| zrail wrote:
| Youngest is 2.5. Oldest quit interrupting on a regular
| basis when she was 3.5. Hoping youngest will follow trend
| but I have no expectations at this point.
| dgellow wrote:
| People have very different metabolisms and lifestyles. I know
| people who need less than 5h per night and are completely
| healthy. I myself feel like shit if I wake up before 9am.
| papito wrote:
| Do they "only" need 5 hours of sleep or do they _say_ they
| need it? Are they as productive as they could be when fully
| rested, or will they "sleep when they are dead".
|
| Arianna Huffington also thought she was being greatly
| productive at no sleep, until her body told her to fuck off
| and she collapsed in her office from exhaustion.
| Barrin92 wrote:
| I personally don't need much more than five hours. As in,
| even when I'm completely on my own free time and I don't
| set an alarm I'll go to bed at 24:00 - 1:00 and wake up at
| about 5:30 - 6:30. My father is the same, he even often
| sleeps only four hours, I guess it's genetic.
| klyrs wrote:
| I used to get by on about 5-6 hours. If I slept early, I'd
| wake up long before my alarm feeling well rested, alert,
| and ready to go. Something changed when I was around 30,
| and it took a few years to figure out that I wasn't getting
| enough sleep. But even so, it's rare that I sleep more than
| 7 hours.
| matz1 wrote:
| They only need 5 hours of sleep. Why is it so difficult to
| understand that not everyone the same ? There are a lot of
| variability in human.
| [deleted]
| tonyedgecombe wrote:
| _Why is it so difficult to understand that not everyone
| the same ?_
|
| Touchy. Perhaps you need some more sleep.
| azemetre wrote:
| Because these people are exceedingly rare. I get 5 hours
| of sleep a night because I have some pretty bad lifestyle
| choices. The few weeks where I have a normal sleep
| schedule I am absolutely a different person.
|
| I mean I can function on 5-hours, I'm able to hold a job
| and live a life; but my well being could be so much
| better if I got a normal nights sleep everyday.
| matz1 wrote:
| How rare?
|
| So you are not the one how can function with 5 hour sleep
| but doesn't mean other people can't.
| nzmsv wrote:
| Because this kind of reply gets posted every single time
| sleep deprivation is brought up.
|
| Needing this little sleep is usually a lie or a self-
| delusion. Perpetuating it has negative consequences for
| the rest of us. Do some of these unicorns exist? Sure.
| But they are nowhere near as common as this type of
| comment suggests. Most of the people in question would be
| better off with more sleep.
|
| Can they survive on 5 hours? Sure. Most of us can also
| survive on nothing but pizza. Doing this doesn't make you
| different, just unhealthy.
| matz1 wrote:
| How are you sure its a lie ?
|
| Not common? How do you know?
|
| Eating nothing but pizza is not healthy? I disagree,
| Pizza has carbohydrates, protein, vegetables, vitamin,
| etc. Not always but it can be healthy.
| klyrs wrote:
| Do you have data showing that it's "usually a lie or
| self-delusion"? Because there's a genetic explanation[1]
| for why folks need less sleep. The only way I can sleep 9
| hours in a night is if I work up a sleep debt.
|
| [1] https://www.nih.gov/news-events/nih-research-
| matters/gene-id...
| Apocryphon wrote:
| It's incredibly rare though.
| klyrs wrote:
| That one mutation is rare, but it's an advantageous one
| in a society of workaholics, so there's probably a
| sampling bias: we're more likely to encounter folks with
| that mutation when selecting for high performers. It's
| also a single mutation -- there may be others that
| provide a similar, if more moderate, effect.
| markus_zhang wrote:
| Do they take naps during day time? A half-hour nap can
| "reboot" one's body. But still, even if they do nap, I'm very
| jealous.
| browningstreet wrote:
| I can get by on less sleep but I still won't be much use to
| anyone working more than a usual hour load. There's more to
| life than work and sleep.
| bsenftner wrote:
| I'm "one of those" that actually has the low sleep gene. 5hrs
| is typical, 6 makes me feel groggy all day. However, this
| only works without alcohol; add booze and I need 8 hrs. So, I
| rarely if ever drink any alcohol.
| snarfy wrote:
| I've been functioning on ~3.5 hours sleep per night for about 5
| years now. You get used to it.
| PragmaticPulp wrote:
| You may get used to it, but that doesn't mean the negative
| effects disappear. 3.5 hours of sleep is well into the
| territory of chronic sleep deprivation, regardless of your
| genetic makeup.
|
| There is no known combination of genes that makes 3.5 hours
| of sleep acceptable. The "short sleep" genes don't shrink the
| sleep need window that much.
|
| We obviously can't know your personal circumstances, but I
| would caution that according to everything we know you are
| likely to pay a price for this chronic sleep deprivation,
| especially if it continues.
| wpietri wrote:
| You might consider the concept of "Normalization of
| Deviance":
| https://en.wikipedia.org/wiki/Normalization_of_deviance
|
| I agree people can get used to almost anything. We've all
| been living through a pandemic. Society made it through the
| black plague. But part of "functioning" through bad
| conditions is losing touch with the the possibility of
| something better.
| kzrdude wrote:
| Don't you have better days and feel smarter, when you have
| slept more? It's a night and day difference to me. With
| little sleep, work is a chore and spare time is unorganized,
| with enough sleep I handle both better.
| MaxBarraclough wrote:
| Apparently [0] some people are genetically disposed to
| require much less sleep than most, but no, you can't train
| yourself to perform normally when sleep deprived.
|
| [0] https://news.ycombinator.com/item?id=27471247
| gogopuppygogo wrote:
| I started today with 4.5 hours of sleep and I'll do just great
| with that. Once a week my body grabs 10-14 hours in a single
| night to "catch up".
| testmasterflex wrote:
| Why are you downvoting this? I also know people like this,
| however I personally need at least 7-8.
|
| The last weeks I have practised to not focus too much on if
| Ive had a sleep deprived night and that helps and works for a
| few days.
| z3ncyberpunk wrote:
| Yeah... that doesn't work how you think it does.. you're body
| and mind don't "catch up" on sleep. you had 6 terrible sleep
| days detrimentally effecting your brain and one "good" nights
| sleep. you do not biologically make up for lost sleep, you
| just walked around with poor mental function and toxic built-
| in up neurochemicals then probably masked it all by doing
| further damage and guzzling down some coffee and caffeine and
| sugar to force your body to stay awake.
| pc86 wrote:
| "Catching up" on sleep is a myth. If you're getting wildly
| different amounts of sleep night to night (even on a
| "schedule" like yours) it's a biological/physiological
| certainly that you're operating at a deficit. Maybe not
| cognitive, but there is a deficit there whether you realize
| it or not.
| yawaworht1978 wrote:
| I am not sure about this, i had periods of 5h a sleep days
| for a week or two and then slept 12 hours for a day and
| felt very fresh and carried on with 5h a day. Sometimes
| it's 7-8 hours a day for couple weeks, sometimes it's 5.
| Feels about equal to me, maybe it has to do with volatile
| working hours for me, not sure. All I know the worst is
| hangover days and on holidays I stay in bed for 10hours
| only to feel unusually tired. Maybe environment is a bigger
| factor than genetics.
| ThrowawayR2 wrote:
| I was under the impression that the idea that one can just
| oversleep to "catch up" was debunked already, e.g.
| https://www.scientificamerican.com/article/fact-or-
| fiction-c...
| martindbp wrote:
| The vast majority of people would be under performing without
| even noticing at 4.5 hrs. Yet somehow, everyone thinks
| they're part of the 1% that has some magic genetic fix for
| this. Sort of like how 80% of people think they're above
| average at driving.
| throwaway2037 wrote:
| The last sentence is brilliant! Sometimes, I feel I am the
| only person that I know who admits to being a terrible
| driver. I am so easily distracted by beautiful nature,
| construction sites, other car crashes, whatever... Thank
| goodness I never had an accident, but many close calls. As
| a result, I try to drive as little as possible.
| AgoRapide wrote:
| Average is not the same as mode. 99.9% of people (at
| least) have more arms than the average for instance.
|
| So yes, theoretically 80% may be better at driving than
| the average, it is just so that the other 20% are even
| worse at it.
| pc86 wrote:
| Given the average driver, imagine how bad that other 20%
| must be.
| adrianN wrote:
| Unless you're a genetic outlier it is likely that you're
| doing long-term damage to your health.
| bruce343434 wrote:
| like?
| exdsq wrote:
| High correlation between sleep deprivation and altzimers
| 2OEH8eoCRo0 wrote:
| I do 7 but 7.5 is my sweet spot. I pose the opposite: How do so
| many on this site need so much sleep?
| koolba wrote:
| Some of us have always been like this. Others have been doing
| it for so long that we don't even know what it feels to be
| fully rested anymore either. The mental fog of being
| continuously sleep deprived is all they remember.
| wpietri wrote:
| There's a lot of natural variance in this. There's no a
| priori reason that your 7.5 makes more sense than somebody
| else's 9. Or a house cat's 12-16. It's like asking whether 5'
| 8" or 6' 1" is the correct height. You roll the genetic dice
| and they land where they land.
| sampo wrote:
| The Chemical Worker's Song (Process Man)
| https://www.youtube.com/watch?v=GzcGOgxDoEk
| eric4smith wrote:
| Think before you join any large company. Nothing here is unique
| to AWS.
|
| We outside have the illusion that everything is perfect on the
| inside of these profitable and successful companies, but in most
| cases it's generally a lot of broken code and a boiler room.
| Staff is usually burned out after a few years - why do you think
| turnover in the tech industry is so high??
|
| The amount of bugs and issues in Facebook advertising is
| atrocious.
|
| Google is just getting some of their admin stuff right.
|
| The lesson to take away is that even if you have a small side
| business - perfection never usually makes money.
|
| It's more important to get an imperfect product to market that's
| held together on the back end by spit and strings.
|
| Or as my dad said, (he was an artist) - "I could continue working
| on this painting for months more, but I have you kids mouths to
| feed".
| Arainach wrote:
| We have more than enough data - be it Glassdoor, stories here,
| etc. - to say with certainty that this kind of thing is far
| more common at Amazon. Portraying it as "a thing everywhere" is
| deeply misleading.
|
| Many companies have teams that can push you hard (I burned out
| at Microsoft, who in general is fairly good for work/life
| balance), but none of them make it a core part of their
| identity. Amazon prides themselves on bare metal optimizations,
| tracking and micromanaging the smallest time quantums they can
| - it's why they have a fleet of warehouses that grind picking
| employees into a broken down mess, it's why their offices have
| almost no perks, and it's why there are so many stories of
| their codebase and process being a mess.
|
| Living in Seattle, I'm fortunate enough to have friends who
| work or have worked in all the big cloud orgs - AWS, Azure,
| GCP, and Oracle. The ranting, burnout, and raw shitshow
| quotient of Amazon is off the charts. It's the only company
| that any personal friends have ever warned me against joining,
| and ALL my friends that have ever tried to work there
| eventually gave me that warning.
| softwaredoug wrote:
| Not sure I agree. The tone from the top varies quite a lot
| across big tech companies, and sets the tone for the culture of
| the teams. Then within that context, a lot depends on your
| team, your manager, your ability to connect with stakeholders
| in the company, and the problem you're working on.
|
| I work at Shopify, and while we have had intense projects, I
| haven't experienced anything like what's described here about
| Amazon. Shopify, culturally, is quite different than other
| large tech companies I've worked at. It's opinionated about
| tooling instead of dozens of things that do the same thing. We
| emphasize sustainably doing work and not burning people out.
| After all, why would we want to burn people out when we want to
| retain them? We emphasize "building for the long term" and to
| do that we do seem to put people first. One example, a
| colleague of mine has been out nearly _10 months_ on parental
| leave. And nobody has batted an eye.
| brtkdotse wrote:
| > One example, a colleague of mine has been out nearly 10
| months on parental leave. And nobody has batted an eye.
|
| _laughs in Swedish 480 parental days_
|
| Seriously though, good to hear sane parental policies are
| spreading to more and more companies. Are those 10 months
| paid for by the employer?
| api wrote:
| Question: is it possible to achieve the heights of human
| achievement and capability / productivity without an abusive
| culture?
|
| It's a legitimate non-ironic question. I ask because I've vowed
| to try to build something great without that and I often wonder
| if it can be done.
|
| I don't see many examples. Most high achieving top of their field
| groups seem abusive and dominated by abusive personalities.
| datalus wrote:
| Is it ironic that Amazon uses leetcode for their programming
| screeners?
| sangnoir wrote:
| That may be the reason the author posted to leetcode.
| rantwasp wrote:
| tin foil hat on: that post is going to get removed.
| nkozyra wrote:
| Maybe in India but I don't recall leetcode being involved at
| all in my process.
| grumple wrote:
| I'm in the US and have been approached by multiple recruiters
| who explained the the first part of hiring was a leetcode-
| like online assessment, and a later part of the process was a
| full day onsite (which I understood to be leetcode-like, or
| systems design questions, just like other big tech
| companies). I did not actually go through the process.
|
| They might not be using leetcode.com itself, but they are
| doing similar things.
| nkozyra wrote:
| Well then the point about leetcode doesn't mean much here.
|
| Yes, most companies do coding assessments, but that doesn't
| mean it's ironic that the comment was posted on leetcode
| itself.
|
| My experience with Amazon had no self guided coding
| assessment, it involved a human guiding you through a
| problem or multiple problems.
| MattGaiser wrote:
| Hit and miss for me with Amazon recruiters. Some have a
| Hacker Rank and some say I can skip it.
| yoloyoloyoloa wrote:
| Funny to see people ask questions like LSE, lol you can probably
| figure out if someone on an alt account is a real amazonian or ex
| amozonian by asking them a few abbreviations.
|
| My experience is that everything this guy mentioned in this post
| is true and more worse things are true that are not mentioned.
|
| Working at amazon broke me. I had to go on Xanax after working at
| amazon and after quiting Xanax lost three jobs consequetively.
|
| I had to work so hard to regain my self respect and confidence.
|
| The situation described in this post is no different in Amazon
| Cape Town or any other place.
|
| Btw the internal dev tooling is absolutely terrible and a pain to
| work with. The dev tooling is a result of "not invented here"
| syndrome. Forget about using a good opensource library to do a
| given task if that library is a google open source library.
|
| It is hard to join amazon, but also to leave it. It operates a
| bit like a prison gang. Your phone tool icons are your prison
| gang tattoo's. Like a prison gang you will move up the ranks by
| doing things, sometimes even taking out members of your own gang.
| The longer you are part of the gang and the deeper you get...the
| harder it becomes to leave until it becomes impossible to leave.
|
| My two cents is that until amazon becomes customer obsessed with
| its "internal customers" aka the employees it is bound to fail.
| soared wrote:
| Similarly, oracle hates anything not built by oracle
| (especially google). They built their own version of google
| sheets, built their own outlook plugin to avoid ms entirely,
| etc. I'm not a dev but would have to call clients and say
| "please send me a blank google sheet, then I can use it to set
| up a doc we can use together".
| HenryKissinger wrote:
| What does Amazon need thousands of engineers for, exactly?
|
| Amazon.com is a fine website. It is finished. It is _complete_ ,
| perfect as it is. The website could remain the exact same for the
| next ten decades, with minor adjustments to the product menus to
| reflect new products, and it would serve its purpose perfectly
| without risking being dethroned by competitors.
|
| Why do millions of lines of code need to be written each month?
| It certainly isn't reflected in my browsing experience.
|
| The British cracked the German naval codes with no more
| mathematicians than can sit at a table. But Amazon needs
| thousands of engineers to run an e-commerce website (and its
| concurrent AWS, which could be run by less than 50 engineers)?
|
| (Not a software engineer or someone who's ever worked for $AMZN)
|
| Edit: Woosh, way too many people failed to understand that this
| post was mostly sarcasm. Cunningham's Law in action.
| jlund-molfese wrote:
| Heavily-utilized websites are like icebergs. You see the
| homepage, but what about the constant ongoing work to ensure
| sellers don't game your review system? What about emergent
| product categories? What about integration with your
| competitor's new smart home device? Amazon doesn't pay most of
| its engineers to sit around.
| lwhi wrote:
| Too many assumptions here.
|
| You need to consider the number of the tech products and the
| number of markets.
|
| The resulting number would be vast.
| [deleted]
| Tabular-Iceberg wrote:
| They have a big product portfolio, so less than 50 engineers is
| a big stretch even in the best of circumstances.
|
| But they are clearly in a very deep hole of technical debt that
| they will never be able to dig themselves out of, so all they
| can do is throw more people at the problem.
|
| I don't think this warrants being downvoted to the point of
| being barely readable. The estimate may be way off, but it's a
| perfectly good basis for a discussion. Please don't do this
| when you can use words instead.
| a4isms wrote:
| Consider this metaphor:
|
| You go on vacation. You stay at a hotel, not a "B&B." Your
| vacation, your money, so you find a place close to the things
| you want to see, with plenty of free amenities like free
| continental breakfast and free WiFi.
|
| It's somewhat reasonably priced, too.
|
| Now you go on a business trip financed by your employer, who
| books you into a "business class" hotel. It's also close to
| things you want to see, and it has a few nicer finishes than
| your vacation hotel.
|
| But the service is worse! You have to order WiFi, and they have
| some annoying provider that puts roadblocks between you and
| their slow WiFi. There's free coffee in the room, but if you
| want breakfast, you pay nosebleed prices for room service or
| you have to line up for a table in their brunch restaurant.
|
| And their "rack rate" for rooms is double what you paid on
| vacation! How could your employer be so stupid as to book you
| into this expensive hotel and force you to suffer worse
| service?
|
| Makes no sense. Ok, what's the connection?
|
| ------
|
| Well, WHY does your employer prefer the expensive place with
| bad service? The answer is that it provides good service for
| your employer, but not for you. YOU AREN'T THEIR CUSTOMER.
|
| The business-class hotel integrates with your employer's
| billing and expense systems. Your company can easily book
| dozens, hundreds, or even thousands of rooms when they're
| putting on a conference. They have incentive pricing that
| appeals to companies, not people.
|
| They have GREAT customer service when you're calling about a
| billing code, but not when you're calling about an extra pot of
| hot water for your tea.
|
| We don't see that as guests of the hotel. So if we were to
| discover that the big hotel chain has hundreds or even
| thousands of programmers, we'd ask, "What do they need so many
| engineers for? All they have is a crappy web site that can take
| a reservation, in a crappy way."
|
| We don't see the people putting in the work to integrate with
| systems we can't see, serving the needs of people and companies
| we don't even know exist.
|
| I betcha it's the same at Amazon. We can't judge the complexity
| of their operations looking at the page they serve us when we
| buy a book online. We have no idea what's involved integrating
| with the corporations that provide them with music, TV shows,
| and movies to stream.
|
| We have no idea how complicated it is to integrate with all the
| logistics systems around shipping products around the world, in
| real time.
|
| There's a massive machine we can't see. That's what all those
| programmers are building and maintaining.
| ghaff wrote:
| While I don't disagree with your general point, I'm not sure
| it broadly applies here.
|
| Honestly, a nice B&B is probably priced pretty similarly to a
| business-class hotel.
|
| Why do I usually stay in business class hotels in cities?
| Chain hotels like Marriott's brands are pretty consistent
| quantities. A random B&B really isn't. Business class hotels
| also tend to have 24-hour desks, I can leave my luggage after
| checkout, etc. If I don't really care much about the hotel,
| which tends to be the case when I'm traveling in a city on
| business, some mid-level chain hotel is just less mental
| overhead.
| a4isms wrote:
| All true, but I'm not comparing a business-class hotel to a
| B&B, I'm comparing one kind of hotel to another. All the
| major chains have offerings in both markets I describe:
|
| 1. Hotels that appeal to guests who pay their own way, and;
| 2. Hotels that appeal to companies with business travel
| needs.
|
| The second type of hotel has the massive machine hidden
| from guests.
| ghaff wrote:
| Certainly all the business class hotels have the ability
| to reserve room blocks, cater events, pre-book rooms,
| etc. But those are essentially additional profit centers.
| I'm just saying that many/most of us essentially deal
| with hotels 1:1 even for business travel.
|
| But, yes, there are some brands of e.g. Marriott that are
| in part oriented to corporate events and the like and
| other that are mostly for individual travelers whether on
| vacation or business.
| a4isms wrote:
| Of course we do. I'm just explaining why such a place
| might have many, many more programmers and other
| employees than we would guess are needed judging by our
| experience out at the periphery of their business.
| ghaff wrote:
| Absolutely, and I'm usually perfectly cool with some nice
| B&B having some janky third-party website or even, gasp,
| having to call them on the phone to make changes etc. I'm
| not fine with Marriott not having a streamlined mobile
| app, a loyalty program, 24-hour customer service,
| etc.There are certainly economies of scale with companies
| generally but there are also costs.
| cduzz wrote:
| My wife and I were recently escorted from a Ritz (a sub-
| brand of Marriot);
|
| We were paying in points (a _lot_ of points), they
| checked us in but still couldn 't figure out their back-
| office payment processing or something because they
| confronted us at 9:00pm and asked us to swipe a card
| because "they couldn't verify the certificate"
|
| After much back and forth between us and the front office
| and a customer support rep on the phone (who repeatedly
| suggested that the "certificate" was there and valid), we
| decided we didn't really want to deal with the hassle of
| staying where we were being treated like criminals.
|
| I went up to get our stuff from our room and my card had
| been deactivated; I had be escorted to the room by a
| member of the Ritz security staff to get our luggage.
|
| So, you're not really assured a good experience no matter
| where you stay.
| wdb wrote:
| I have to meet the first employer that does the hotel
| bookings for me. I always need to sort it myself and expense
| back. The only thing the employer did is arranging a
| discounted room price.
|
| Normally, meant that I spend at least one day a month doing
| my hotel/travel expenses. That's ~40-46 lost working days :)
| a4isms wrote:
| Pay yourself and expense it back is usually an SMB strategy
| for keeping costs down by offloading work onto employees.
| It's not just your time that's wasted, but all the work to
| organize expense reports, make sure you used the right
| codes, &c.
|
| It's a headache for you and the folks in accounts, but at
| small scale, that works. But as the company grows, this
| becomes harder to justify. At some point they centralize a
| lot of this stuff, and when they do, many choose to start
| booking people into hotels that cater to businesses who
| book people into hotels.
| ghaff wrote:
| Seems like you should get off your butt and create an AWS
| competitor with 50 engineers. You'll be able to underprice them
| by so much you'll completely eat their lunch.
| amazon_throw wrote:
| HAHAHAHHHHa hah... hah. ha hah _cough_
|
| Oh, wow, what an amazing take. You owe me a new keyboard.
| aetherson wrote:
| Most Amazon engineers do not work on the amazon.com front end.
| tequila_shot wrote:
| They have other suites of products apart from the AMZN.com. -
| AWS and the shitload of solutions on AWS come to mind. - A lot
| of internal tools for managing supply chain, financials etc
| also come to my mind.
| padastra wrote:
| You should ask yourself how you can be so self-assured yet
| wrong by several orders of magnitude. I'm not saying that as an
| insult -- it's probably worth re-evaluating your confidence :
| accuracy relationship.
| lucasyvas wrote:
| Indeed, a truly remarkable disconnect with reality that
| honestly makes me question the point of having an account on
| a site called "Hacker News."
| oschvr wrote:
| Imagine being this delusional
| luma wrote:
| AWS is a money printing machine in a very dynamic market. They
| need to keep the pace or MS will eat their lunch.
| hermannj314 wrote:
| Amazon also sells hardware (kindles, echo, sidewalk), has a
| media platform through Prime, has logistics software for vendor
| and order fulfillment, in addition you are underestimating the
| cost of running a global e-commerce site (legal complicance,
| security, privacy, accessibility, etc. are constant technical
| draws on even established products)
| 0kl wrote:
| Amazon web services runs a fair share of the internet. Around
| 33% as of Sept 2020 according to Forbes [^1].
|
| [^1]:
| https://www.forbes.com/sites/danrunkevicius/2020/09/03/how-a...
| nuclearnice1 wrote:
| Well this controversial opinion sure got pounded down.
|
| > The British cracked the German naval codes with no more
| mathematicians than can sit at a table.
|
| The Government Code and Cipher School employed almost 10k
| people during ww2.
| oblio wrote:
| Yeah, but you couldn't fit all those 10k people in a
| Hollywood movie.
| brazzy wrote:
| >AWS, which could be run by less than 50 engineers)?
|
| What the...
|
| > (Not a software engineer
|
| Ah, yes.
|
| It shows.
|
| 50 engineers is probably less than the size of some support
| teams within Amazon that handle a single large corporate AWS
| customer.
| middleclick wrote:
| This is a peak HN comment (the one you are replying to)! The
| other day on the Wikipedia thread, there was a data scientist
| who said he could run a website like Wikipedia all by
| himself. "How hard it is?". And here we have the same thing
| for Amazon..
| earth2mars wrote:
| AWS have close to 200 services. If the origina cmmenter js
| saying each service has 50 folks, it is 10k people. Which
| sounds about right the company have more than a million
| people working across all sub organizations. So you can
| imagine all kinds of work environments
| ghaff wrote:
| Admittedly I've even heard senior executives rhetorically
| ask "What do all those people at $LOCATION even do?" At
| scale, you need a lot to sell, support customers, put
| marketing programs in place, do developer outreach, test,
| be on call, etc. etc. But, yeah, you'd think it would
| obvious that even just talking about straight engineering,
| you need more than a fraction of a person to develop and
| enhance each AWS service.
| stavros wrote:
| Listen if you can't run 33% of the internet with 50 engineers
| what are you even doing?
| bennysomething wrote:
| How do you know it's "finished"? You could have said the same
| thing in 2006 and been completely wrong. What makes you think
| the customer front end of Amazon is where the bulk of the
| engineering is?
|
| Edit: engineering and maintaining the infrastructure of Amazon
| compared Vs cracking a code in world war two seems like a
| strange comparison.
| thesuperbigfrog wrote:
| >> Not a software engineer or someone who's ever worked for
| AMZN.
|
| Exactly. There is SO much more going on than you realize.
|
| The Amazon retail website gets hundreds of thousands of orders
| every second. New code gets deployed literally every night.
|
| The retail website is like the tip of an iceberg floating above
| the water. There are hundreds of web services underneath that
| provide the backend and data stores. Each one of those services
| has a team of engineers to maintain the service with one
| engineer oncall 24/7.
| terafo wrote:
| AWS alone requires thousands of software engineers. And there
| is Amazon Echo, Twitch, Prime Video, a lot of software for
| warehouses and logistics, their autonomous stores, game
| studios, Kindle, Amazon Fire devices, their app store, Audible
| and I barely scratched the surface.
| rytill wrote:
| > and its concurrent AWS, which could be run by less than 50
| engineers
|
| One engineer for every four offerings.
| intricatedetail wrote:
| Isn't that almost like modern slavery? Company makes billions,
| barely pay any tax and employees are exploited to the last drop
| of sweat. Disgusting company and I have no respect to developers
| who work there. Why won't developers unionise?
| amazon_throw wrote:
| Current (long tenured, moderately senior) AWS engineer here. I've
| been at the company long enough it's pretty clear that I'm a
| "good culture fit", so take what I'm saying with that in mind.
|
| While I absolutely believe that there are pockets of the company
| that work this way, more because of sheer scale than anything
| systemic, I have sat in the annual ratings meeting for engineers
| enough times, in enough organizations within the company, that I
| am pretty confident that this experience isn't universal.
|
| It sucks that this author had this experience, and I wish they
| had said which team that was, so that I could use what social
| cachet I have to steer people clear of it from inside. Nobody
| should have that experience.
| atopuzov wrote:
| I'll tell you mine, L7. Seen and experienced personally things
| the author describes.
| amazon_throw wrote:
| Like I said, I don't doubt that they happen, but given that
| you've also left the company and seem to have hit one of
| these areas of toxicity yourself, I'm not surprised you'd
| think so. Who was your last manager? I'm curious if it's
| anybody I know?
| mancerayder wrote:
| > Who was your last manager? I'm curious if it's anybody I
| know?
|
| Would you truly ask that? And have someone's personal name
| be searchable forever? What is the impetus?
| atopuzov wrote:
| Not gonna dox other people, you can easy figure out my name
| and reach out with a "real name".
| [deleted]
| bopbeepboop wrote:
| Amazon FinTech is such a toxic dumpster fire of
| unprofessional conduct, there's a chance not only will they
| significantly harm the company, they'll cause a legal issue
| with China, India, EU, or US by violating finance law.
|
| If they haven't already.
|
| Amazon is routinely fined $15M+ in tax audits because
| they're saving a few headcount on critical financial
| systems.
|
| Leadership doesn't care.
|
| So yeah, if all of FGBS counts as "parts" -- then sure,
| your comment may be technically correct.
| hitekker wrote:
| This went from "weird, defensive throwaway" to "possible
| Amazon HR agent" in 5 seconds.
|
| I think asking for the name of someone's former manager on
| a public forum without giving any details in return is
| pretty suspect.
|
| For those who aren't already aware, Amazon has a history of
| covertly paying its employees to "represent" the company in
| social media: https://news.ycombinator.com/item?id=26636021
| ixaxaar wrote:
| > It sucks that this author had this experience, and I
| wish they had said which team that was, so that I could
| use what social cachet I have to steer people clear of it
| from inside. Nobody should have that experience.
|
| "I have soft power". Suspicious indeed.
| amazon_throw wrote:
| No, far from it; I'm not HR, nowhere close, but I am
| senior enough that people sometimes listen to me. I'd
| like to make things better for people if I can, and part
| of that is "knowing where the problems are". Believe me
| or don't, it's no skin off my ass, but that's the
| opposite of my goal.
| Immune wrote:
| Like someone else has mentioned. The old fart tool is
| literal proof of the extremely high turn over rate. From
| my own experience checking the tool it showed that over
| %50 of people have been fired or left the company in just
| 1 year. I suspect that you're astroturfing.
| tmarthal wrote:
| > seem to have hit one of these areas of toxicity yourself
|
| Have you considered that the original poster's experience
| is actually the norm and that your experience is the one
| that is the anomaly? I was 1 for 2 in organizations with
| shitty leadership, and the organization that was run
| properly had zero open headcount. Everywhere people are
| hiring into is not one of the "good ones".
|
| Check out the old-fart tool, 85% of the company has been at
| Amazon for 3 years or less. Do you think that if the
| normal/average organization/team was a great place to be,
| that there would be so much attrition?
| amazon_throw wrote:
| I have considered it. I don't see the pattern widely, and
| I'm watching for it. I've seen teams implode because of
| it, and other toxic patterns, so it's not that they're
| not there... they just seem to be in the minority. There
| are teams I won't send friends to work for, for sure.
| kyawzazaw wrote:
| Have you visited teamblind.com?
| treis wrote:
| I've been doing software development for nearly a decade
| now and I've seen 0 teams implode. Nothing I would call a
| "toxic pattern" springs to mind either. If you've seen
| multiple occurrences of both at Amazon and think it's an
| ok place to work then I think you've just normalized the
| dysfunction.
| irateswami wrote:
| Found Bezo's account
| teawrecks wrote:
| This isn't the first time I've read about Amazon's 30%
| mandatory turnover rate, how it effects new hires mentally when
| they can't help but think they might be fired at any moment,
| and how it effects the ones who actually did get fired. This
| sounds like the definition of a systemic problem.
| rantwasp wrote:
| It's a thing. The author was not unlucky. It's actually a thing
| (I worked in AWS, I have heard this from several managers, some
| which actually had trouble struggling with how stupid the
| system is).
|
| You have a team of X engineers and you want to grow. You hire a
| couple more. The current engineers have 0 incentives to help
| the new ones. Most people don't have the chops (technical or
| emotional) to go up against a whole team.
|
| Come review time, what do you think is going to happen? As a
| boss will you let go someone who's been there for 5 years and
| knows the service inside out or the new guy who seems to be
| struggling.
|
| Not all people are jerks, and there are good pockets (but
| mostly the deck is stacked against you when you join).
|
| Also, IMHO Amazon is going to have a really hard time hiring
| people with the reputation they created for themselves.
| Popegaf wrote:
| > Also, IMHO Amazon is going to have a really hard time
| hiring people with the reputation they created for
| themselves.
|
| The majority cares about money and convenience. If people
| really cared about reputation then Riot Games, Microsoft,
| Tesla, Facebook, that big ride sharing company whose name
| slipped my mind, Shell, and lots of other companies, would be
| struggling to hire.
| rantwasp wrote:
| hmm. Do Microsoft or Tesla pay well? Last time I
| interviewed at M$ I had to turn down their offer because of
| how bad the comp was.
| throwaway2757 wrote:
| Microsoft really only pays comparatively at each end of
| the spectrum, so for new grads out of college and people
| who have Wikipedia articles.
|
| For the majority of people in their career (the Seniors
| and Principals/Staffs) MS pays significantly worse than
| other FAANG-type companies, often being well under half
| in terms of stock compensation for example.
|
| The company is well aware of this as it's constantly
| brought up in all-hands (both within individual
| departments as well as at the all-company level), and
| they always respond with "we've done research and we
| believe we actually do pay comparatively in this market
| segment".
| rantwasp wrote:
| lol. "i actually did my reservations and your
| compensation is not competitive"
| intricatedetail wrote:
| > "we've done research and we believe we actually do pay
| comparatively in this market segment".
|
| Sounds like market fixing and likely illegal, however I
| can imagine poorly paid employee wouldn't be able to
| afford a lawsuit.
| dralley wrote:
| I don't know about Microsoft, but Tesla is well known for
| having terrible compensation.
| raincom wrote:
| >The current engineers have 0 incentives to help the new
| ones. Most people don't have the chops (technical or
| emotional) to go up against a whole team.
|
| This is why companies want to hire rockstars who can be
| productive in a couple of months without being helped by
| colleagues. One can't get such rockstars just by leetcode.
| Maybe, these companies should pay $1M per annum for such
| rockstars.
| exikyut wrote:
| > _steer people clear of it from inside_
|
| What's really needed here is a way to maintain super-scale in a
| way that is, shall we say, "eventually morally consistent".
| Applying selection theory, if you create a pocket of badness
| wrapped in a function that reliably extracts all the good out
| of it... well all you'll be left with is a local maximum of
| even more suffering. Yeow.
|
| Of course, designing and maintaining such social structures
| seems close to P=NP in complexity...
| rantwasp wrote:
| tell me about a time you delivered awesome results while
| operating out of a pocket of badness
| [deleted]
| fridif wrote:
| >steer people clear of that org
|
| Um, shouldn't we fix the org instead!!!
|
| Very glad that I've been denied from Amazon final round twice
| now. It was great interview practice and nothing more
| bilater wrote:
| Somebody should forward this to Bezos with a question mark.
| aws_dub_temp wrote:
| Current AWS engineer here. I've been working here for ~2 years in
| a pretty critical service, in the Dublin office
|
| Back when I joined the company, you could already find dozens of
| online reviews talking about a toxic culture and awful management
| practices. I was about to turn down the offer because of that. 2
| years later, I have to admit there have been good and bad
| moments, but if I had to make the decision again today, knowing
| everything that I know after all this time, I would 100% still
| accept the offer.
|
| Amazon improved my life and my career in ways I would have never
| expected. When I look back, it really seems unbelievable to think
| that I've improved so much as a software engineer in that period
| of time. And not only on the technical side, but also on aspects
| like writing, caring about customers or thinking about
| operations: you realize that work is much more than coding or
| shipping new features. Besides, from its culture I've learned
| processes or ways of thinking that have incredibly helped me even
| on my personal life, like "one/two way doors", "working
| backwards" or "mechanisms over good intentions"
|
| I'm not going to pretend that all the other comments are lies and
| that all of that did not happen. Obviously there's a lot of
| people with really bad experiences at the company. But what I'm
| trying to say is that Amazon is a huge company, and you can find
| both great and awful experiences. If you're considering applying
| here, or even accepting an offer, don't get discouraged just
| because here you find mostly bad opinions.
|
| In case it helps, something that convinced me to join when I
| already had the offer and had to make the decision was talking
| with my future manager. I directly mentioned that I found reviews
| about a toxic culture and wanted to know his feeling about it.
| His answer was simply "Look, I cannot talk about other orgs, or
| even other teams. What I can say is that in my team we really try
| to create an inclusive and healthy environment, and that we
| really care for each other". He could have just lied and said
| that all the reviews were fake, that the culture was great and
| that all those problems did not exist, but he was honest and
| admitted that he could only talk about the areas that he knew
| about.
|
| To me that was a good reminder that even companies like Amazon
| are formed by normal people, and that while there will be people
| that only care about themselves and getting promotions, there are
| also some that really care about making the company a great place
| to work
| ALittleLight wrote:
| I spent about 10 years at Amazon (NA). Nothing as extreme as what
| the OP described for me, but the story seems plausible.
|
| In my opinion you need to both have the attitude and really
| believe that is fine to be fired. The company will take from you
| as much as you're willing to give and pressure you for more. If
| you aren't the kind of person who can and will resist that
| pressure, it probably won't be a good fit.
|
| I had a super capable coworker. Friendly, nice guy, always
| willing to help, joined as a senior engineer. He was more than
| willing to pitch in and work long hours in "crunch time". What he
| didn't get is that it is always crunch time. There is always some
| schedule we're behind on, some deadline the PMs care about, some
| presentation to some high level guy, a customer demo, whatever.
| Over the course of one or two years he basically transformed into
| someone grumpy, overworked, and mean. I introduced the new people
| I'd be a "buddy" to or my mentee to the guy, because I knew how
| much he had helped me, and they had very different experiences.
|
| For myself, on the other hand, I was never willing to pitch in or
| work long hours and I never did. I didn't care if we missed our
| dates. As far as I could tell, we always missed them anyway and I
| wasn't going to work late just so we could miss them by slightly
| less. My coworkers would message me at all hours of the day and
| I'd just ignore them until I was actually working.
|
| This approach isn't without cost. Some people did nag me about
| never replying to them - I continued to ignore them nagging me.
| People would complain to my manager that I didn't answer or even
| read their emails - "Sorry, must have missed that one" and
| continue to filter their emails. I had awkward conversations
| about why I didn't attend some meetings - "I went to the first
| couple and decided I had more productive uses of time available
| to me".
|
| On the whole though my reviews were positive. More people than
| not seemed to like working with me. And, I quit Amazon on my own
| timeline without getting fired. More importantly, while I did
| eventually get tired of working there, I never burned myself out
| in the way that the OP describes or that some of my coworkers
| did.
|
| I was never afraid of getting fired and I just don't have the
| personality that easily gets pressured into doing lots of work -
| I enjoy laying in bed and playing on my phone more than having a
| job. I think you need to have counterbalances like that to avoid
| getting consumed by Amazon.
| throwaway_32242 wrote:
| Current Amazon engineer (not AWS, not in NA or India) here. I
| feel very surprised to see folks in India having such a bad
| experience working in Amazon, because we're almost completely the
| opposite here. I feel bad for those who suffered.
|
| I've been working for 2 years here now, never heard about the
| "intent to fire" thing. Our oncall is not perfect but we're
| definitely working to improve service stability. Working hours
| have been worse than before since the start of COVID (working at
| home makes it easier to overwork), but it's still manageable, and
| when we were in the office it was mostly a 9-5 (or 10-6) job.
| Unless you're oncall and get paged, nobody would expect you to
| think about work from the moment you walk out of the office in
| the evening.
|
| Like people have said, managers really decide your experience.
| I've had some bad managers and did internal transfer to improve
| my experience. There were some projects where I worked with teams
| located in India, and sometimes I do feel that they're quite
| different. Some PMs and even leadership there would push really
| hard to get things done, to the extent that we could feel their
| pressure. That never happened in other projects we did before,
| and this post seems to give me some hints on why that happened.
|
| Update: minor edits on grammar
| bsenftner wrote:
| I read a telling point about Jeff Bezos yesterday, he believes
| people, all people, are inherently lazy and will work to avoid
| work. The description seemed to be trying to say, without saying,
| he's an Ayn Rand disciple, meaning he drunk the Kool-Aid to
| believe he's a John Gault or an Roark and we're just the peons
| preventing him from greatness and his destiny.
| papito wrote:
| Jeff Bezos wants to pay the least for the most amount of work,
| and the workers want the most money for the least amount of
| work. It sounds like both parties should meet themselves half-
| way as opposed to living in what is effectively modern-day
| slavery. Suggesting such a thing, however, is "radical" and
| "communist", and essentially one step away from a dystopian
| fascism hellscape something something ANTIFA.
| akarma wrote:
| > Jeff Bezos wants to pay the least for the most amount of
| work, and the workers want the most money for the least
| amount of work. It sounds like both parties should meet
| themselves half-way as opposed to living in what is
| effectively modern-day slavery.
|
| The way it currently works is what you describe here. Amazon
| pays the least they can for the most amount of work, and
| workers work the least they can for the most amount of pay.
| They meet at the equilibrium where Amazon receives adequate
| labor, and the workers receive adequate pay.
|
| There's nothing radical or communistic about that idea.
| pc86 wrote:
| This isn't specific to Bezos, and it's not specific to any
| particular class or type of "worker." If Bezos could double
| his money today with zero work he'd certainly do it. The
| family business down the road would cut every employees'
| salary in half tomorrow if it could.
|
| It's just the free market, nothing particularly interesting
| or scary about it. Everyone wants to get the absolute best
| deal for themselves, and some are more successful than
| others.
| burlesona wrote:
| There are a LOT of small business owners who choose to pay
| their employees more than they "have" to because they
| genuinely want to. There are a LOT of entrepreneurs who see
| "making my company a great place to work" as one of the
| core values of their job.
|
| I don't think this is so common when you were talking about
| mega companies, in part because the work of operating a
| mega company is a lot less fun than a smaller company, and
| so you have this selection pressure where (a) people who
| pursue that path are more likely to value wealth and growth
| over quality of life, and (b) ruthlessness seems to usually
| help companies compete and win in the market. Thus the
| biggest and most famous companies of the world are more
| likely to be focused on cutthroat efficiency and, as a
| result, miserable places to work.
|
| But that's no more a feature of capitalism than cancer is a
| feature of DNA. It's a pervasive malfunction, but I believe
| it's treatable, particularly through aggressive anti-trust
| and wealth taxes.
|
| Remember that the vast majority of capitalism is little
| businesses like your local veterinarian or florist, not
| FAANG.
| shadowlight wrote:
| >Remember that the vast majority of capitalism is little
| businesses like your local veterinarian or florist, not
| FAANG.
|
| That is the vast minority. Corporations dictate much of
| the business in the states and the world. It's really
| easy to see this without resorting to statistics.
|
| What is the ratio of your friends who work for
| corporations vs. the amount that own/work for small
| businesses? The anecdotal percentage here is a good
| indicator of the real percentage of economic output
| produced by corporations vs. small businesses.
|
| You will find that as how most of your friends direct
| their own economic output in service of corporations so
| does most of America.
| whakim wrote:
| This is a very myopic way of looking at things. There are
| plenty of family businesses that aren't only interested in
| capturing an ever greater share of profits.
| throwaway2037 wrote:
| Yes absolutely -- in many places, like Taiwan, Korea,
| Japan, Germany, Austria, France, Netherlands, etc.
| wpietri wrote:
| Nah. That's not "the free market". And that's not how most
| employers are. Wanting extra useless billions even if it
| immiserates others isn't commerce, it's sociopathy.
|
| Even sticking with industrial titans, look at Henry Ford,
| who insisted on paying his workers a living wage. That was
| revolutionary at the time, and put America on the road to
| having a significant middle class.
| papito wrote:
| This is not a fair fight, however. Do you know how often US
| lawmakers talk on the phone with a billionaire? About once
| a week. They get the preferential treatment, they get the
| laws passed, and if the "normals" want to do something
| crazy like organize to negotiate working conditions, then
| it's a code-red emergency in Washington the sky is falling
| somebody do something.
| alisonkisk wrote:
| Amazon is unapologetically Randian, and many senior people are
| proud of that. That doesn't have anything to do with the rest
| of your comment though.
| [deleted]
| Grakel wrote:
| You have a profound misunderstanding of Ayn Rand and, I
| suspect, Jeff Bezos.
| bsenftner wrote:
| I doubt that. She wrote impossible fantasy that convinced
| wealthy fools they were destined.
| ErikVandeWater wrote:
| Where did you read this? That's important context.
| wpietri wrote:
| There were a number of recent articles. Here's one:
|
| https://www.businessinsider.com/amazon-polices-based-jeff-
| be...
| imwillofficial wrote:
| Business insider is a lying trash rag.
| approxim8ion wrote:
| The primary source is the NYT, not Business Insider. They
| clearly name their source too, "David Niekerk, a former
| Amazon vice president who built the warehouse human
| resources operations"
|
| 1:
| https://www.nytimes.com/interactive/2021/06/15/us/amazon-
| wor...
| imwillofficial wrote:
| NY times I can get behind. Business insider is like
| citing the national enquirer.
| jmartrican wrote:
| Maybe Amazon just needs better SRE. They should read Google's SRE
| books.
| rejectedandsad wrote:
| The point of the system is that engineers operate what they
| own, that's fundamentally different from SRE. It allows for
| significantly greater organizational decoupling. At Google,
| they had to go through an entire ceremony just to get Rust into
| the monorepo (I actually don't know if they ever ended up doing
| that outside of the Fuschia repos) while at Amazon someone just
| created a build script that wraps cargo and now AWS heavily
| uses Rust for several control plane systems and Firecracker.
| This kind of agility is not easily possible with an SRE system.
| leros wrote:
| I work at a company that hires developers out of Amazon. We joke
| about having to deprogram them, but it's not really a joke. They
| bring a lot of emotional issues and toxic workplace habits with
| them. Don't get me wrong, the Amazon engineers are fantastic, but
| there is clearly something unhealthy going on at Amazon.
| Rebelgecko wrote:
| "Deprogram" is the same word I came here to use... I haven't
| seen it myself, but supposedly it's inevitable that you'll
| eventually have to have "the talk" with a former Amazon
| employee who tries to schedule a meeting at 7am
| nobleach wrote:
| I left a place that was full of death-marches. Those old habits
| really can stick with you. It took me months at my new job
| where I kept asking myself, "why do I still feel stressed? No
| one is behind me telling me I have to deliver" Because I was so
| used to that constant stress, I had begun to place those
| expectations on myself! What's worse, I was frustrated that no
| one else was on Slack on Saturday at 11am to look at my PRs...
| I've come down off of that lately. And it's such a relief.
| amznthrowaway8 wrote:
| I'm going to offer a different perspective here as someone who
| was lucky enough to get placed on a pretty good team within Prime
| Video. I just hit the 1 yr mark, fwiw. The engineering bar is top
| notch, lots of attention to code quality, and management seems to
| care and listen to us engineers, though we occasionally clash of
| course. With Amazon, YMMV, so try your best to choose a good team
| though it's a bit of a lottery ultimately.
|
| I've heard AWS can be super brutal, the consensus seems to avoid
| it.
| random_user_9 wrote:
| SDE3 here with 5+ years on the retail side. Listen up new SDE1.
| Here is the score when you get hired at Amazon. You have 2 to 3
| years to get promoted to SDE2 or you are gone. SDE2's you are in
| the same boat but with less risk of a PIP. You'll need to be
| making a case for SDE3 in 3 to 5 years or you are going to see
| you comp decrease over time. Once your at SDE3 you now are really
| Amazon employee instead of a trainee in the eyes of management.
| You can more easily push back on stupid management decisions at
| the L7+ level, except hiring, and use your years of history at
| Amazon as leverage, most managers are new anyway. Most SDE3s
| however have figured out that the path to Principal is easier by
| boomeranging. There are too few L7 level projects to go around.
| If you want to survive this climb then here are some tips.
|
| 1. Really own your product. Know it inside and out. Too many devs
| rely on tribal knowledge and out dated docs to tell them what
| they own. The code is the truth, read it, poke at it, review
| previous CR's and SIMs to piece together its history. You will
| become the master of the truth.
|
| 2. Drive your career. Understand what the moving to criteria for
| your level are and keep notes for how you are achieving them.
| Focus on taking on work that gives you more evidence that you are
| on a promo path. Write the documents that make the argument and
| iterate on them often. You need to actively manage this or it
| will manage you.
|
| 3. Pushback. Many managers at Amazon are just spreadsheet jockeys
| that are making things up as they go along. If something is
| stupid or determental to the product or team say so and have
| evidence for why. Make the argument and build consensus on the
| team for that argument. You won't win them all but you will erase
| from your manager's mind that your are a passive code monkey. The
| more you do this and can demonstrate results the more you'll have
| the ability to drive your work.
|
| If you want to be spoonfed a career don't work for Amazon.
| cduzz wrote:
| An important lesson I've learned is to ask in the interviews "how
| is employee performance evaluated?"
|
| If the response is blather about 360 peer review, be wary.
|
| Also scan all available information about if they perform any
| stack ranking.
|
| I worked at a place where managers would bring in other people
| just so they could meet their "must fire" quotas but keep their
| existing teams intact. I left shortly thereafter.
|
| An organization must be able to identify and cope with true "non-
| performers" but when it turns into a global quota for all teams
| you're in the hunger-games.
|
| The culture of a place radiates from upper management / C Suite;
| if they're sociopaths you end up with this sort of organization
| where certain types of people filter up.
| varispeed wrote:
| The problem is that this is a "factory worker" mindset applied
| to creative and IP based work.
|
| When a factory worker makes a chair, company does not profit
| from it indefinitely.
|
| When a developer creates code he or she gets paid once, but
| company profits forever.
|
| So even if they fire a developer, they still are profiting the
| from work they did. Question is, why developers sign IP
| transfer and royalty waiver in their contracts?
| visarga wrote:
| > When a developer creates code he or she gets paid once, but
| company profits forever.
|
| Forever usually being 1-5 years?
| varispeed wrote:
| It's a figure of speech, but I've seen 10 year old commits
| still doing well in production and some written before even
| git was a thing.
|
| Given how much value this work generates, it's time
| developers got together and put a pressure on companies to
| pay fairer and proportionally to profits they are
| generating.
|
| I'd even opt for a legislation that would make contract
| clauses about giving up royalties illegal.
| Copernicron wrote:
| 360 peer reviews are such bullshit. I've seen it happen where
| strong developers on a team of strong developers stagnate at
| the same level for years while mediocre developers surrounded
| by terrible developers get promoted. It all comes down to who's
| evaluating you and who you're in good with. It has nothing at
| all to do with how good you actually are.
| bitcoinmoney wrote:
| What is 360 peer review?
| cvrjk wrote:
| I believe it is when you review your colleagues, they
| review you and your manager collates all the reviews to
| access your performance. Some places allow you to nominate
| who you want to provide your review, perhaps people you
| worked closely with the last couple of months etc.
| amznthrowaway12 wrote:
| Current Amazon engineer.
|
| As others have said, management at Amazon will only direct focus
| on the development of new features or new services, at the
| complete detriment of improving existing services, or even the
| overall architectural design of a particular space of the
| business. The definition of completing a service or feature is
| only that a customer is using it without complaint. Management
| has no interest in technical reasoning, which causes the design
| decisions that rest at their level to be unreasonable. This leads
| to a few problems:
|
| 1. Services are only ever about 50% complete. Unit tests
| typically exist to some degree, but integration tests,
| documentation, complete monitoring and operational automation are
| rarely done. Services typically have numerous obvious bugs,
| grossly bad optimization, hideous over engineering, and sometimes
| design issues. Because the customer cannot detect these things
| when the first use the service (maybe it will reflect the second
| time as a bug, or as slow performance later down the line, or
| long times to develop new features), there is no interest in
| fixing them.
|
| 2. The graph of service dependencies is entirely unmanaged. Any
| service can depend on any other service, for any reason. This
| results in a massive, undesigned spaghetti of a system. Something
| like s3 or whatever will usually be supported in some way by a
| spaghetti built for s3, and if s3 fails, it is usually not
| immediately obvious which service in the spaghetti is
| responsible. It makes adding something new to the overall system
| take a very long time.
|
| 3. Even if a customer is encountering an acute problem, and
| management is asking for it to be fixed, if the problem is rooted
| at a system level outside the boundaries of a single service,
| thus at the level of management, management is unable to engage
| with any reasoning as to how it should be solved. Only management
| holds the keys to assigning work (senior or principal devs hold
| basically zero sway) and thus management must have the technical
| reasoning ability to make these decisions.
|
| 4. Management will sometimes intrude in service level problems
| and make unreasonable decisions. Examples: 4.1 I
| was told python is not performant (despite my history at the
| company having me deploy python code to every single physical
| host in the fleet) and asked to research and explain why it isn't
| scalable and we should switch off it. I declined to work on the
| issue 4.2 Management had an issue raised to them where
| a single user had sev2ed us because they couldn't paste into a
| field on our service. Investigation quickly revealed the user was
| trying to paste text with a space into a numerical field, and the
| browser was preventing it. The issue already had two solutions
| suggested: add highlighting to invalid inputs, and strip spaces
| from inputs. Despite this management decided a formal review was
| required, where somehow we would have to dig deeper than the
| existing explanation and explain how this happened and what
| should be done.
| salil999 wrote:
| Former AWS engineer here.
|
| I worked on a pretty critical product in AWS (big AWS service
| with lots of traffic) and I can safely say that it's totally up
| to your manager and pre-existing conditions which make up the
| job. My manager was great as a person but would always lack in my
| career-oriented goals (bigger projects, promotions, etc)
|
| But what really sucked for me was the pre-existing conditions.
| Our on-call was pretty bad (40-60 tickets a week) and there was
| very little investment being put in to improve it. We had a lot
| of little scripts here and there which would solve extremely
| specific situations but no focus was ever put on in building a
| general framework or trying to reduce the ticket count. This
| often led to engineers taking the day off after their on-call due
| to the load and honestly it made people quite grumpy. And upper
| management was always much more interested in feature delivery
| since the focus was always on promotions and the more you
| delivered the better it looked for your manager. So now you have
| engineers with such a terrible on-call load along with pressure
| to deliver new features and projects within the atrocious tight
| deadlines that would be set. It was, to be blunt, a shit show.
|
| Code quality was atrocious. We had one enormous Java method
| (>1000 lines) which would take care of nearly every single
| request coming into our service... With only about 7-8 unit
| tests. It was so difficult to get even basic things done to the
| point where any ticket that needed to be done would take a
| minimum of 4-5 days regardless of complexity. And of course
| managers and senior engineers would estimate small tickets to
| take around 1-2 days and then be shocked when 2 days later it's
| not even close to being finished. I will give Amazon credit that
| they do grill design reviews pretty harshly so those are done
| well in general. But code reviewers didn't care about quality or
| best practice. If it works then ship it.
|
| I'm just not 100% sure about the whole PIP scene. Our service was
| extremely critical and we were extremely understaffed. So I don't
| think it applied to anyone in our org but I know of other teams
| who would have no issues in taking in a fresh college grad,
| making them do work for 6-12 months and then just randomly
| putting them on PIP. Sad but I've seen it happen a few times in
| my time there.
|
| I'm glad I got the Amazon stamp on my resume and left. When I
| left, more than half my team and my manager quit around the same
| time too. It was definitely a wild experience.
| markus_zhang wrote:
| I'm always surprised why industry leads (like Amazon) sometimes
| treat their products as an amateur treats his/her weekend
| projects. This is definitely not the worst as I know one of the
| leading options marketmaker has been using a giant shit
| mountain of MS Access/Excel VBA code to run their system since
| the 90s. Last time I heard about it (a few years ago) they are
| planning to replace that shit mountain with something new but I
| don't know if it's done now.
| nobleach wrote:
| It's built into the culture to ship, ship, ship. Shipped code
| is better than good code, or clean code, or fast code. At my
| last job, the C-level was fascinated by Amazon success
| stories. They wanted to achieve the same success, so they
| urged us to ship, ship, ship. Unfortunately, we were all very
| seasoned engineers, and we knew the nightmare that would
| ensue if we purposely piled on the tech debt. The part of
| this article that refers to "every ticket taking 4 to 5 days
| regardless of complexity" should be a warning sign to ANYONE
| attempting to model their startup after Amazon.
|
| The message should be: You are not Amazon, and you will NOT
| get to Amazon scale by modeling their worst practices.
| dustingetz wrote:
| Shipping tech debt should be compared to the realistic
| alternative which is never shipping at all. The solution is
| not to ship slower, but to attract a better team and then
| retain them. They say the CEO's #1 job is recruiting and
| this is why. Actually more important is growing your
| revenue faster, if revenue compounds faster than debt then
| you're good!
| whateveracct wrote:
| I mean you can also ship a lil slower.
|
| Startup founders and management types are obsessed with
| optimizing dev time at a day/hour granularity though in
| my personal experience, so it's a lost cause.
| ako wrote:
| Can you really ship slower?
|
| In the end it's a winner takes all market, and AWS needs
| to out-innovate azure to win.
|
| And as customers of AWS we're all looking forward to the
| next re:invent for new features, and we're voting with
| our money buying huge amounts of AWS services.
|
| Also, as a potential employee, I would always sign with
| the company that has positive cashflow through customers
| that pay for shipped value, rather than a company with a
| great code base, but a bad sales track record.
| markus_zhang wrote:
| I think this culture is OK or even good to a company in
| start-up because you have to be quick. Once it grow into
| maturity those rules should be abandonned.
| nobleach wrote:
| There are a few problems with that approach. One is that
| there is rarely a good time to say, "ok, we've proven
| that this idea works, let's now go back and do it
| correctly". It's a constant stream of fixes on a system
| that "already works". Secondly, telling management that a
| team was able to go fast previously, but is now going to
| start being slow so that things can be done correctly, is
| a quick way to be shown the door. This is evidenced by
| Amazon and that thousand line Java method that's probably
| existed for 10 years. Finding that time where you're
| mature enough to switch gears never seems to happen in
| practice.
|
| I now advocate a "cut corners, cut scope, but do NOT cut
| quality" approach. Unit tests do not take much longer to
| write - but they pay dividends when it's time to
| refactor. I'm now back in a startup where the code was
| written with that "just ship it" attitude. The code is so
| terrible that it can take days to fix a bug. I can
| rewrite entire features (correctly) in that amount of
| time.
| gautamnarula wrote:
| This jives with what I observed as an intern some on an AWS
| team some years ago. The oncall rotations seemed absolutely
| brutal and the engineers were so busy and stressed out fighting
| fires that they barely noticed my existence (which I was okay
| with) and the tech debt kept accumulating between because
| between new feature launches and firefighting there wasn't much
| scope for anything else.
|
| My intern project was a fairly no brainer tech debt item that
| automated a lot of the deployment process and saved our lead
| engineer several hours a week in babysitting deploys. I
| resolved to never work on a cloud infra team after that --
| while the internship was fine, being a full time engineer
| seemed absolutely miserable.
| vishnugupta wrote:
| (Disclaimer: Was with Amazon for ~7 years a long time ago).
|
| I've been a customer of AWS across multiple startups and I've
| seen the overall quality of their products continuously degrade
| which complements your experience.
|
| While they continue to launch new products at a rapid clip you
| can see small cracks beginning to appear as the products age. A
| permission issue that's not documented, a cryptic error message
| etc., They aren't show stoppers on their own but if you use AWS
| long enough you will be worn down by the cumulative pain.
| awsthro00945 wrote:
| >Our on-call was pretty bad (40-60 tickets a week) and there
| was very little investment being put in to improve it. We had a
| lot of little scripts here and there which would solve
| extremely specific situations but no focus was ever put on in
| building a general framework or trying to reduce the ticket
| count.
|
| AWS engineer here and I confirm everything you say, but this
| quote _really_ struck home with me.
|
| The thing I've noticed at Amazon is that not only are the pre-
| existing conditions awful, but nobody has any interest or
| willpower to fix it. Everyone will happily vent to you and tell
| you how awful things are, but any suggestions to fix it or make
| things more efficient (even if the fix is very simple and
| requires low effort) will be met with hostility. And I'm not
| just talking tech issues, but also process/workload issues.
|
| I've worked across multiple teams and there is an
| "institutional ego" at Amazon where everyone, especially L7s+,
| think that Amazon is the best/smartest company in the world and
| have an attitude of "if Amazon, the best company ever, hasn't
| already figured out a way to solve this problem, then it must
| be an unsolvable problem and we won't even try". The thing is,
| a lot of these problems are in no way unique to Amazon and many
| other companies across the world have already found fantastic
| solutions to reduce things like on-call load. But adopting
| those solutions would require admitting that other companies
| were able to solve something that Amazon hasn't, which would
| hurt the ego.
|
| This all applies to the very issue being talked about in the
| OP, too. Even managers will vent to you about how their team
| goes through 50% attrition every year, and how everyone is
| overloaded and finding new engineers is hard. They just accept
| 50% attrition as "something that just happens every year" as if
| having such a shitty team is normal, and there is no movement
| at all to fix it.
| taway_zonian257 wrote:
| > The thing I've noticed at Amazon is that not only are the
| pre-existing conditions awful, but nobody has any interest or
| willpower to fix it. Everyone will happily vent to you and
| tell you how awful things are, but any suggestions to fix it
| or make things more efficient (even if the fix is very simple
| and requires low effort) will be met with hostility. And I'm
| not just talking tech issues, but also process/workload
| issues.
|
| SDE1 here. IMMV of course but on my corner of the org I've
| seen a bunch of team members raising concerns regarding tech
| debt for the L7 to shut it down as it got in the way of
| delivering the features he wanted to deliver.
|
| Also the elephant in the room is how the company relies on
| trial by fire as a form of performance evaluation, which
| involves inexperienced SDEs being pushed to deliver alone
| chunks of major projects in spite of lack of experience or
| insight.
| EliRivers wrote:
| Related to part of what you said. I've worked in a half dozen
| different industries; I've worked in little companies with a
| half-dozen employees and globe spanning companies with tens
| of thousands and employees. They all think that they have
| special unique problems that nobody else has - 90% of it is
| the same problem I've seen in other companies in different
| industries, with different industry specific acronyms and
| words.
|
| Every damn job, the same basic problems over and over, with
| the insistence that these problems are specific to the
| industry and usually to that specific company and that
| specific product.
| doggodaddo78 wrote:
| "NIH" duplication of effort with "IH" bugs.
|
| SMH. Will they ever learn?
| mirker wrote:
| Same is true in academic research (though old-timers catch
| it often). The common pattern is:
|
| * approach "A" was invented in 1970 or so and didn't work
|
| * "B" extends "A" in multiple ways and now works
|
| * noobs assume "B" invented "A" and treat "B" as the root
| of modern knowledge. "B" often has more market presence so
| noobs (without deep understanding) don't see the
| relationship to prior attempts.
|
| Examples:
|
| * AlexNet/deep learning/ML in general
|
| * MapReduce/databases/functional programming primitives
|
| * Docker/chroot "Jail" Containers
|
| * Bitcoin/90s coins/blockchains
|
| Ego and ignorance are rarely a great combination :)
| doggodaddo78 wrote:
| The Emperor's new clothes (TENC). Most people don't do
| history, especially the Dunning-Kruger afflicted.
|
| Docker (Linux containers) is, like jails: awful, leaky
| isolation pretending to be virtualization. If you want
| real resource and security containment, use
| virtualization. Docker is insecure in so many ways; it's
| like using PHP to write a TLS library.
| kablow wrote:
| > The thing I've noticed at Amazon is that not only are the
| pre-existing conditions awful, but nobody has any interest or
| willpower to fix it. Everyone will happily vent to you and
| tell you how awful things are, but any suggestions to fix it
| or make things more efficient (even if the fix is very simple
| and requires low effort) will be met with hostility. And I'm
| not just talking tech issues, but also process/workload
| issues.
|
| In my case, direct management seems interested in these
| issues and understand there are problems we need to fix, but
| ultimately the feature/product launches always make it into
| the sprint and the larger bug fixes never do. It's very much
| "actions speak louder than words".
| vlovich123 wrote:
| Can you share your knowledge or reading materials for how to
| reduce on-call load?
|
| I've worked at a number of big companies but all the problems
| driving the oncall load seemed, at best, domain specific if
| not application specific with highly variable fix times and
| unpredictable occurrence (eg started becoming more of a
| problem due to unrelated change X). As a result each team has
| to decide the cost of fixing the pain vs focusing on other
| things.
|
| If there's actually best-practices here that help that we're
| not already doing, I'd be extremely eager to learn about
| them. I'm not an Amazon engineer but I've been bitten by
| oncall stuff.
| wikibob wrote:
| There is a lot of SRE content about fixing on-call.
|
| My short summary is, fixing on-call is a HUMAN problem, not
| a technical engineering problem.
|
| Here's an excellent place to start:
|
| https://monitoring.love/articles/how-to-improve-on-call/
|
| There is a wonderful list of talks and resources there, and
| those will lead you to yet more concepts and ideas to
| research.
| awsthro00945 wrote:
| I don't have any particular reading material, and my
| example of the on-call load at AWS that I'm referring to is
| probably very basic to most people.
|
| On my team at AWS, leadership has given specific
| instruction that we do not believe in on-call runbooks or
| automation to triage issues, for example. Leadership's
| reasoning for this is that they think runbooks prevent
| engineers from applying personal judgement, and every
| single issue should be handled manually by an engineer on
| an ad-hoc basis.
|
| This leads to a significant amount of on-call time and
| cognitive load spent doing stuff like verifying the most
| basic of issues. Even if you have seen the same issue come
| up for the 1000th time, and even though the previous 999
| times it came up the answer was always the same, leadership
| still insists that the on-call engineer go through a full
| ad-hoc process of investigating the issue "just to be sure"
| that this time isn't different.
|
| It's a similar situation with documenting our integration
| guidance for other teams. Our leadership insists that any
| documented guidance be vague, and that whenever another
| team wishes to integrate with our software they _must_
| schedule meetings with us to discuss even the most basic of
| design questions. I 'm talking very simple stuff like
| "should you use the HTTPS endpoint to communicate with our
| service?" where the answer is "yes" 99.99% of the time, and
| could easily be included in some documentation. But
| leadership insists that we spend _multiple hours per week_
| in meetings to discuss this just in case that 0.01% design
| comes up.
| wikibob wrote:
| It is absolutely fascinating how wrong this approach is.
|
| Every single issue that comes up in on-call should be
| evaluated under the lens of "does fixing this absolutely
| require human judgement, or can it be automated, ideally
| by fixing the code in the main system. If it does require
| human judgement, are there ways to redesign so that is no
| longer true?"
| YZF wrote:
| There is something to be said for this approach. If the
| root cause is to be fixed someone needs to look at it in
| depth rather than running some play book procedure to
| recover. If you have too many problems though you're
| beyond the point where that helps. Let's say your
| software has worked flawlessly for a year, no issues, now
| an issue pops up, the engineers should definitely spend a
| lot of time understanding it, understanding why it popped
| up, fixing it properly and fixing the underlying
| process/org causes that made it pop up. It should not be
| "follow some playbook to recover". If issues pop up every
| week this is unsustainable, you're well beyond the point
| where stuff can actually be fixed. Automation has its own
| dangers, it is additional software to maintain, it has
| its own bugs etc. The right amount of automation makes
| life better for sure.
| awsthro00945 wrote:
| >If the root cause is to be fixed someone needs to look
| at it in depth
|
| The root cause has already been looked at in depth 999
| times when the same issue has come up. It's already been
| RCAed and the fix has been put in the backlog to be
| implemented sometime next year. In the meantime while we
| wait for the fix, we will continue to do a full, ad-hoc
| RCA _every time_ the exact same issue appears, with the
| _exact same results_ every time, because managers
| genuinely think it is a valuable way to spend our time.
|
| I understand your point, but the relative utopia of a
| team you're describing is not really the situation I'm
| talking about. We have on-call periods where the _exact
| same issue_ will appear 10-20 times per week, and _each
| and every time_ it is treated as a completely novel issue
| with an ad-hoc response, even though we already know
| beforehand what the root cause is and what the fix is. It
| 's an incredible waste of time and contributes
| significantly to on-call engineers being overloaded, and
| yet we continue to do it and then are baffled when all of
| our engineers leave the team due to being overworked.
|
| There's also nothing excluding runbooks and root cause
| analyses from existing together, either. In fact, most
| good runbooks specifically include steps to determine
| when an RCA is necessary and how to conduct one. There
| really is no excuse to not use runbooks as much as
| possible. If over-reliance on runbooks is having a
| negative impact due to engineers not applying personal
| judgement, then that is certainly an issue to be
| addressed, but the answer is almost never to completely
| abolish runbooks and documentation.
| YZF wrote:
| This does sound pretty dysfunctional. You'd think that
| for something that's causing 999 on-calls getting the
| root cause fixed would be a priority. What I described
| obviously falls apart when the team has no ability to
| actually fix issues. Perhaps the original intent was to
| get those issues fixed but that somehow got lost as the
| org grows larger.
| throwaway210620 wrote:
| > the exact same issue will appear 10-20 times per week,
| and each and every time it is treated as a completely
| novel issue with an ad-hoc response
|
| Yeah, this sounds like a very bad situation where
| management won't let you do something that reduces ops
| pain because it isn't the most desirable solution, but
| they won't let you prioritize the right solution either.
| The next thing that happens is that on-call folks develop
| ad-hoc quasi-runbooks and share them amongst a subset of
| people (or just keep them to themselves to make their own
| life easier) and those quasi-runbooks become critical to
| ops, but not documented or shared by everyone. It's pure
| dysfunction.
| throwaway210620 wrote:
| In my experience: 1) Make a goal: we
| should get paged for at most N incidents per week. That
| goal should be closer to 0 than 10 IMO. 2) Track
| stats on this goal, both in aggregate and broken down by
| cases you think you can address separately. Example:
| tickets from alarms vs tickets filed by people. Alarms
| having to do with external dependencies vs alarms caused by
| your own bugs. Don't just make this a "it seems like we've
| had fewer pages lately" thing. Real numbers. 3)
| Review these stats on a graph every week. Someone should
| have an explanation of why they have spiked, why they
| haven't dropped, the breakdown of problem type, etc. There
| should be congratulations when they drop and a request for
| plans when they don't. 4) have management that can
| communicate upwards to leadership that ops improvement is a
| priority for your team and that you ultimately won't be
| able to continue other feature development if you are
| always mired in ops pain and people are either busy with
| mundanity or driven to leave the team. 5) dedicate
| time in each sprint to working on the most recent
| identified target from plans made in #3.
|
| This isn't particularly complicated, and typing it out
| almost sounds like I'm giving you worthless common sense
| advice, but I think the key here is that multiple levels of
| your organization need to commit to making this important
| enough to spend time reviewing it, agree it is a priority,
| and put actual dedicated dev work into it.
|
| edit: formatted so indented list is readable on mobile. I
| have no idea how to do this without a code block. HN,
| please make this easier :)
| YZF wrote:
| I'm not the parent but lemme chime in on this topic. It's
| pretty simple, if you don't build crappy software you won't
| get a heavy on-call load. You're really asking how to build
| great software. Build strong teams, with experienced
| people, follow good practices, reward quality and stability
| and not features or lines of code, reduce complexity, etc.
| etc. I've worked on software used by millions of people
| with a very low problem rate and then I worked on software
| used by hundreds of people where nothing ever works. Often
| in the latter the team, through lack of experience or
| ability, assumes that this is just the way all software is.
| There's plenty of examples of widely used software systems
| that are generally quite reliable and well built, and
| there's plenty of examples of stuff that's garbage, held
| together by duct tape, works by chance.
| rossjudson wrote:
| This is oversimplification.
|
| One reality is more like "You are handed a system with a
| heavy on-call load. Make it better."
|
| The other is "The system you built was great with 1e9
| load, but now we're heading for 1e12 load. Make it
| better."
| rodgerd wrote:
| > The thing I've noticed at Amazon is that not only are the
| pre-existing conditions awful, but nobody has any interest or
| willpower to fix it.
|
| Jeff Bezos can afford a private space program because - in
| part - his Amazon retail model is based on working people
| until they are mentally and physically broken, while paying
| them a pittance, discarding them, and moving onto the next
| group of people. He would rather spend money on crying booths
| and astroturf campaigns and buying newspapers than change
| that.
|
| Why would he think about the expendable meat-units in AWS any
| differently?
| throwaway210620 wrote:
| The lack of investment into decreasing on-call pain is a real
| factor. I work on Oracle cloud (OCI) and at least some of the
| orgs (VP-down) have figured out that this is something worth
| focusing on, and the on-call gets better and better as a
| result. My original team had an average of something like 50
| pager-worthy (sev2) events per week until we got moved into a
| new org that had the right philosophy and we relentlessly
| drove that down because management realized that engineers
| made miserable by mundane ops fake-emergencies would
| eventually get fed up and leave, and that's not what they
| wanted (afaik, OCI has no such forced attrition). So we got
| put on a program of relentlessly tracking and categorizing
| the sev2 counts and committing to improving those numbers
| over a period of time. 25% of dev sprints were dedicated to
| improving ops (tools, better alarms, fixing long-backlogged
| bugs that led to pages), and now that team's ops are pretty
| easy and they are free to work on new features, which
| everyone prefers. I've since moved to another team whose ops
| had _already_ had this optimization done, and I 've never
| experienced a bad week of on call there.
|
| I won't pretend OCI is a panacea (lol google oracle cloud
| toxic work environment for latest stories) but at least they
| don't lack this particular piece of wisdom. The sheer number
| of regions they plan to operate doesn't really allow them to
| ignore dumb ops problems.
| rantwasp wrote:
| worked in AWS for a while, but it was 5 years ago.
|
| you're right that your manager can make or break your
| experience.
|
| For the first couple of years there it kinda sucked, mostly due
| to oncall (our ticket queue was at 3000 tickets at some point)
| and constantly being yelled at when things broke. We would
| basically only work on sev2s. Having the pager was super-
| stressful. We "owned" so much cruft/dead projects/experiments
| that a couple of time we were paged for something that we
| didn't know existed.
|
| After I build a little bit of leverage / gathered some
| political capital I somehow ended up in the position of "team
| lead" with I guess management's intent to move me to be a full
| time manager (after the team's manager was PIPed).
|
| I made a good case that half the team is going to leave, me
| included, if we don't do anything (6 people team).
|
| So what did we do? I introduced a "secondary" oncall. After you
| were oncall for a week, you were secondary for a week. While
| secondary you got the chance to try fixing some shit without
| worrying about being paged every hour. You were also motivated
| because you just got offcall. People went for annoyances that
| would generate a lot of busywork or for... fixing the alerting
| and monitoring (a lot of autocuts due to wrongly setup alerting
| thresholds or even alerts that should not have been alerts in
| the first place). After we exhausted the low hanging fruit, we
| put some effort into automating some tedious task that would
| take a lot of time but understood and never meant to be done
| manually at the scale we did them.
|
| Towards the end of the journey we aggressively
| deprecated/migrated the shit that was not used. By the end of
| this (took more than one year) we had an empty-ish oncall queue
| and for the first time in ages people coupd breathe (we now got
| a sev2 every other week - which in Amazon terms is freaking
| awesome).
|
| I wish this story had a happy ending. There was close to zero
| recognition for what happened there and most of the team
| migrated together, internally, to another opportunity after. I
| left Amazon 6 months after this migration. From what I hear
| from the people that stayed there, entropy took over and in
| another 2 years they were roughly in the same shitty place as
| far as oncall goes.
| bpicolo wrote:
| I use reducing the rate of incoming tickets as the primary
| OKR for on-call engineers. Has never failed to reduce that
| burden over time. Just like you say - I'm convinced it's the
| only correct strategy to make the problem go away.
| middleclick wrote:
| > Code quality was atrocious. We had one enormous Java method
| (>1000 lines) which would take care of nearly every single
| request coming into our service... With only about 7-8 unit
| tests
|
| AWS is pretty reliable for the most part so I am pretty
| surprise that the code quality is that bad.
| kottapar wrote:
| This is indeed surprising. Any time we have slowness issues
| the usual recommendation would be to throw resources at the
| problem; increase cpu, add more memory et al. We used to
| lament that we should spend time debugging the problem and
| fix the actual issue. We then used to say that probably at
| places like AWS and the other biggies they'd be following
| some excellent best practices and we should also strive to
| reach that level of excellence.
| bradleyjg wrote:
| > AWS is pretty reliable for the most part
|
| In telecom or traditional mainframes, for example, the
| compute unit itself was expected to be reliable. Individual
| elements of AWS are not pretty reliable in that context.
| Check out the single host EC2 SLA.
|
| However, today most large or even medium scale software
| assumes unreliable individual elements and has redundancy at
| the program level. For that purpose, AWS core services are
| pretty reliable.
| oceanplexian wrote:
| Way back I used to work in telecommunications at a place
| that provided POTS service. They are two very completely
| different worlds. Software engineers act as if 5 9's is a
| badge of honor, when really it isn't. When you are
| responsible for something that people use to dial 911 and
| can make the difference between life and death a few
| minutes of downtime doesn't cut it.
| bradleyjg wrote:
| Right, and even five nines would be impressive compared
| to:
|
| _AWS will use commercially reasonable efforts to ensure
| that each individual Amazon EC2 instance ("Single EC2
| Instance") has an Hourly Uptime Percentage of at least
| 90% of the time in which that Single EC2 Instance is
| deployed during each clock hour (the "Hourly
| Commitment"). In the event any Single EC2 Instance does
| not meet the Hourly Commitment, you will not be charged
| for that instance hour of Single EC2 Instance usage._
|
| This essentially forces the use of distributed computing
| for even small businesses.
| ev1 wrote:
| EC2 is absolutely not meant for this, though. Use an
| abstraction layer like Heroku if you're going to not
| understand what you're getting into.
|
| The amount of times I've had to 'advise' small businesses
| that are somehow running their small business site off a
| single EC2 instance's ephemeral boot volume is atrocious.
| bradleyjg wrote:
| I don't have any experience with huroku but what most
| small businesses need is a (perhaps simulated) reliable
| box on a fast network. As glorious as the paxos based
| present is, it's overkill to the point of distraction for
| most businesses. The whole attraction of the cloud for
| them is not needing to hire sysadmins. Replacing that
| requirement with needing a devops team is even worse.
| awsthro00945 wrote:
| AWS is very big, culturally, on making sure that all the
| bugginess from shitty code is not shown externally to the
| customer. Externally it might look like everything is fine to
| you, but internally AWS is a massive, leaky cargo ship with
| thousands of engineers running around 24/7 with duct tape and
| band-aids to plug the leaks.
| papito wrote:
| For something heavily used, like the EC2 and load-balancing,
| perhaps, but I am still experiencing PTSD from my last
| CloudFormation encounter.
| salil999 wrote:
| I think one thing I learned from AWS is that there's so much
| hidden away from the customer. There definitely were (and
| probably still are) many issues which the customers won't
| actively experience. Reliability doesn't necessarily equate
| to good standards and good practice.
|
| But yes, from a customer point of view, AWS is pretty nice.
| cpach wrote:
| Reminds me of that old quote by John Godfrey Saxe:
|
| _"Laws, like sausages, cease to inspire respect in
| proportion as we know how they are made."_
| markus_zhang wrote:
| "I just had one for breakfast." -- Jim Hacker
| Frost1x wrote:
| AWS has the advantage of having so many engineers behind the
| scenes available for firefighting with a culture of pushing
| more than is reasonable that as a customer, that would sort
| of disappear. They simply have to occasionally make trade
| offs between unrealistic development/feature request goals
| and firefighting whenever the firefighting is needed. This
| also acts as another form of pressure to work even more to
| meet timeline goals.
|
| Don't let your developers know that you're expecting them to
| always be behind, infinitely queued up with work, and
| constantly in emergency mode and they won't have much time to
| think about what's really going on and how efficiency is
| being pushed at the cost of their sanity.
| colde wrote:
| I think that highly depends on the service. The new App
| Runner service for instance is a wild ride of buggyness, lack
| of testing and incorrect documentation.
| wpietri wrote:
| > AWS is pretty reliable for the most part so I am pretty
| surprise that the code quality is that bad.
|
| I'm not totally surprised because of two factors: very stable
| product definitions and lots and lots of users.
|
| A number of years back, I was talking with people at a famous
| and popular site with a broad audience. I asked them how much
| unit testing they did. They said that particular isolated
| pieces sometimes had tests. But most of the user-facing stuff
| didn't because they had one-button rollout and one-button
| rollback. Instead of bothering with unit tests, they'd just
| frequently release changes, watch the metrics and the
| customer support queue, and quickly roll back if they'd
| introduced a bug.
| zorked wrote:
| For very, very popular services, a second of being live
| will exercise more code paths and edge cases than even the
| most dedicated testing team could ever dream of.
|
| We hear a hell of a lot about testing but the most
| fundamental piece of software quality nowadays is the
| release strategy: running on tee'd live production traffic,
| canarying, metrics and alerting, quick roll backs, etc.
| treis wrote:
| >For very, very popular services, a second of being live
| will exercise more code paths and edge cases than even
| the most dedicated testing team could ever dream of.
|
| Most of the code we care about is to handle anomalous
| situations. That AZ going down a week or two is a good
| example. It's when stuff like that happens that a bunch
| of code springs to life to keep things running. And
| indeed, things didn't exactly roll over just fine for us.
| notacoward wrote:
| That's an overly general statement. Can you do that for
| front-end code that stores all of its state elsewhere?
| Sure. Can you do it for a storage system? Absolutely
| freaking not. If you introduce a bug that loses or
| corrupts data, there's no going back. You will have
| committed the worst sin that somebody in that specialty
| can commit. Better to test as much as you can, at every
| level. Other kinds of code are often somewhere in
| between.
|
| Also, even if it's true that being live will exercise
| more edge cases etc., it's a terrible way to test changes
| during early development. For one thing, there's no
| isolation. It becomes harder to determine _which_ of
| several recent changes caused a problem, and that burden
| unfairly falls on the person who 's on call instead of
| the person who introduced the error. And decent
| unit/functional tests allow "dumb" mistakes (we all make
| them) to be caught _earlier_ than waiting in a deploy
| queue, allowing faster iteration. "Most recent change
| probably caused the problem" is a very useful heuristic,
| but the more low-assurance changes you allow in the less
| useful it becomes.
|
| To drive the point home even further: I have found data-
| loss bugs in focused testing that didn't show up in prod
| for _months_. I know because in many cases I was able to
| add logging for the preconditions when I fixed the bug.
| No logs for months, then some completely unrelated and
| completely valid change by another engineer tickles the
| preconditions and BAM. That would have been an absolute
| nightmare for other members of my team, possibly even
| after I was gone. Based on those experiences, I will
| _never_ believe that foregoing systematic early tests can
| be valid. The systems most of us work on are too complex
| for that.
|
| "Test in prod" only works for trivial code and/or trivial
| teams. Not in the grown-up world.
| ed_elliott_asc wrote:
| Everyone should be testing in prod, in that you release
| code and see metrics and monitoring to show that
| everything is working.
|
| Testing in production is not going "let's see if this
| will work" it is "we will release and validate that
| everything is working as expected"
|
| People need to get over the old school cowboys who jump
| on prod to see if something works.
| notacoward wrote:
| Yes, everyone should release code and watch metrics etc.
| but I think that's at the very edge of what "testing"
| encompasses. Between model checking, traditional forms of
| testing, and shadow-traffic testing (which can test
| _higher_ per-server load than prod), finding something
| after deploy should be like a parachute failure. Yes
| those happen, yes there should be a reserve, but if it
| happens more than once in a blue moon you have a process
| problem somewhere (quite likely between teams /services
| but still).
| mavelikara wrote:
| Tangential, where can I learn more about shadow-traffic
| testing? Books, blogs, tools etc.
| [deleted]
| joshuamorton wrote:
| Teeing/dark launch/dual write strategies solve most of
| the issue for databases. Sure you run into concerns when
| changing the framework that manages that, but that's
| usually a far smaller surface area than your entire
| storage layer.
|
| That said, you should have tests anyway.
| polotics wrote:
| Tell that to the millisecond of testing in production
| that makes the MRI fry the patient's brain, to the one
| that trades one trillion instead of one thousand naked
| puts, to the nuclear armaggedon launch check that
| canaries humanity...
| bradleyjg wrote:
| Depends on if you are serving ads over cat pictures or
| routing air traffic. Different solutions for different
| problems.
| sreque wrote:
| It's a very short-sighted view on testing, although I'm
| not surprised SREs would say it. The biggest problem with
| software deployment is that it is owned and managed by
| people who have no vested interest in developer
| productivity, including devops engineers.
|
| A major goal of any org should be developer productivity;
| otherwise you are just hemorrhaging money and talent.
| When I say developer productivity, I mean: How
| confidently and quickly can I make a shippable, rollback-
| free change to a unit of software?
|
| If you are the dos equis man of testing, "I don't always
| test my code, but when I do, I do it in production", then
| you can't confidently make any change without risking a
| production outage, so you play lots of games, like you
| mentioned, around canarying, rolling out to a small
| percentage of users, etc., but at the end of the day your
| developer productivity has absolutely tanked.
|
| The goal of any system maintenance should be that a
| developer can quickly make and test a change locally and
| be highly confident that the change is correct. The
| canarying, phased rollouts, and other such systems should
| not be the primary means of testing code correctness.
| hibikir wrote:
| If the release/rollback process is fast enough, and your
| detection of anomalies is fast enough, you can still have
| great productivity, and few relevant outages, when
| testing in production. Hell, there are situations where
| testing outside of production is never going to cut it,
| as generation of sufficient load of the right shape would
| take you a whole lot more of engineering time than the
| consequences of failure.
|
| That said, the tradeoffs are different for different
| companies, and different services in the same company:
| Within the same team at $large_company, I owned code
| where testing in production, via deployments and an
| amazing feature flag system, was better than unit tests,
| while there were other areas where the build system would
| dedicate many CPU-hours to testing before any release. To
| be able to have that flexibility though, you need to know
| your systems, know your problems, and have great tooling
| for both testing in production and extremely parallelized
| test suites. Small and medium sized companies might not
| have either alternative, and we had both!
|
| So what I'd say is that any general rule on what should
| be the primary means of testing code correctness is going
| to not lead to optimal productivity, and even more so if
| you don't have top quality of tooling across every
| possible dimension. It's perfectly OK to argue about
| specific examples, but without judgement of this kinds of
| things without having the entire story of what's there is
| just hubris.
| wpietri wrote:
| Yeah, I really appreciate excellent rollout strategies,
| although I suspect a lot of them are more developed out
| of self defense by SRE teams. I see it as a series of
| safety nets: I'm still going to write tests for my code
| so that I don't have far to fall if I make a mistake. But
| I also want a safe rollout so if I miss the first net I
| don't splatter on the pavement.
|
| And I totally agree with out about developer
| productivity. It's just not a consideration in most
| places. For example, in a factory or a restaurant,
| meetings are things that happen rarely and in constrained
| time slots, because everybody realizes that production is
| primary. But in most software companies, actually getting
| work done is second priority to meetings.
| sreque wrote:
| Agreed. I was an SRE for over a year and the philosophy
| is that anything that is shipped can be broken. SRE is
| all about detecting, limiting, and mitigating damage. I
| think this is the right philosophy for SREs but should
| not be the total picture in the org.
|
| I am also agreed in that I anecdotally often see a
| disregard for automated testing. I am still trying to
| understand how to eliminate this tendency. I know that in
| every software project I've had a major hand in building,
| I've helped ensure automated testing, with a heavy
| emphasis on unit testing, become a major part of team
| culture, and I've always felt the tests more than paid
| for themselves over time, even in the relative short-
| term.
| [deleted]
| muststopmyths wrote:
| >but I know of other teams who would have no issues in taking
| in a fresh college grad, making them do work for 6-12 months
| and then just randomly putting them on PIP.
|
| stack ranking came up in another thread a few days ago and this
| practice of forced attrition seems like just another way to do
| the same thing.
|
| I thought it was understood that this kind of structure just
| incentivizes internecine fighting and politics over and above
| any imagined positive effects.
|
| Mind-boggling that there still exists upper management that
| thinks otherwise.
|
| I mean, early Amazon must have hired a lot of ex-Microsofties
| with stack-ranking PTSD, being based in the same area. You
| would think they would know better.
|
| It seems like the more employees you have under you, the less
| you see them as human beings instead of inputs and outputs into
| a Rube Goldbergian machine you are trying to keep running.
| epolanski wrote:
| > It seems like the more employees you have under you, the
| less you see them as human beings instead of inputs and
| outputs
|
| I mean, there's a department in most companies called "human
| *resources*" to give away that employees are still resources,
| just human ones.
| jjtheblunt wrote:
| well said
| ignoramous wrote:
| > _Mind-boggling that there still exists upper management
| that thinks otherwise._
|
| Far too many Harvard MBAs calling the shots, I guess?
| [deleted]
| frozenport wrote:
| Was it Amazon US or Amazon India?
| smlss_sftwr wrote:
| Had a relatively similar experience (finally understood what
| drinking out of a firehose meant), the silver lining of the
| experience though is it finally dispelled any notion of
| imposter syndrome when I realized everyone was running around
| as much of a headless chicken as myself ahah
| JediPig wrote:
| Amazon begged to interview, so i did. My friend worked there,
| and thought I go call her and find out how things worked there.
| That is when I discovered this forced termination of 30%. I
| thought it was 15% , but who's counting. So I gave amazon a
| choice. 2 year contract forced pay contract, if they fired me
| for any reason, other than fraud, I would collect the full
| amount. I done this twice.
|
| It was a defunct mobile company. I had the contracting company
| put it in the paperwork, to my surprise they signed. cause they
| have multiple times let go of contractors to make C level
| "bonus". It was known for it. I got some push back, but they
| signed it, they needed an engineer who understood mobile
| routing records.
|
| 3 months into the contract, they apparently forgot, and let me
| and everyone go. Reason 'services no longer needed' aka CEO
| wanted to make bonus. I said ok, then called the contractor's
| legal dept. After 3 days, i got a call back. Told him look at
| page 4 line 30.. he read it. Long silence. 5 minutes of
| silence. "I never seen a clause like this. I will call you back
| with an answer." Next day, Both lawyers call, one asked would i
| be interested in returning. I said "fufill the contract terms".
| "Your check will be delivered in two weeks." Contracting firm,
| was getting paid too. they were happy and mad. Happy for $$$,
| mad that they probably lost a semi big client.
|
| I got my check, had a new contract before the 2 weeks.
|
| Fast forword to Amazon. They read it and said no. I declined.
| They offered little more, but I said I will not work for Amazon
| until they fix their corporate envirnoment. My friend worked
| there, she lasted 4 months more, after moving to Seattle, and
| finding out the hard way that amazon is pure corporate monster.
| M5x7wI3CmbEem10 wrote:
| how many months of work does it take to get the "stamp"? I'm
| assuming employers would be wary if it's only 6-12 months
| lucasyvas wrote:
| Yeah there's no way I'd put up with this. Trillion dollar
| company and still operating like it's amateur hour on many
| teams it seems.
| papito wrote:
| American capitalism is built on employee heroics, but at
| least you can get the AWS stamp on your resie, which, let's
| be honest, will make you the "it" girl of job hunting.
| tester756 wrote:
| but what's the point?
| ghaff wrote:
| I think you overestimate the appeal of a handful of large
| companies, taken by itself, on a resume if only because
| they come with downsides as well.
| papito wrote:
| The level of abuse and mismanagement I put up with before
| I got older is pretty shocking. Young people have a much
| higher tolerance for BS, in many cases because they don't
| have enough experience to know that their workplace is
| broken.
| mancerayder wrote:
| Or worse, they may internalize it as normal and
| unreflexively continue breeding the problem. Those
| olderish managers of young people were young people
| themselves, once.
| papito wrote:
| Yeah, it's _much_ easier to end up in a malfunctioning
| workplace than a good one. The number of healthy
| workplaces where everything seems to just work is
| shockingly small, and you are actually lucky to end up in
| ONE through an entire career. And then you can never go
| back.
| lumost wrote:
| This works when the company is hot, but eventually it's
| not. When the last time you saw HP, or IBM positively on a
| resume?
|
| Big companies breed big company problems that can leave
| engineers poorly equipped for the rapid delivery world of
| startups/scaleups.
| VRay wrote:
| "HP" or "IBM" carry a lot more weight on a resume than
| something like "BF Consulting Ltd"
| lucasyvas wrote:
| As someone with one of those two on his resume, I'd say
| you are not wrong.
|
| That said, it's a bit more nuanced. It still looks good
| if you can share experiences that prove why it's good.
|
| Ex. "I learned what it's like to be closer to the cutting
| edge in some respects, but I also learned how that should
| be secondary to delivering a good product and good
| customer service due to issues X, Y, Z observed. Also,
| using technology A is great, but it might not be worth
| your investment at current stage."
|
| This kind of statement illustrates that you learned
| multiple things of value, and hopefully avoided bad
| habits and are pragmatic and worth having. It's possible
| they can leverage your experience to avoid making
| mistakes in the next growth stage.
|
| So, yes, it doesn't always look good immediately. It's up
| to you to prove to a questioner why it was good and
| hopefully they also agree.
| SilurianWenlock wrote:
| "Big companies breed big company problems that can leave
| engineers poorly equipped for the rapid delivery world of
| startups/scaleups. "
|
| Why are big company employees not suited (or trained
| for?) to startup like environments?
| lumost wrote:
| There are several reasons big company experience doesn't
| translate.
|
| 1. Legacy tech, "cool" big companies have the latest
| stacks but over time these stacks become old and the dev
| practices atrophy. The biggest cases of this I've seen
| are engineers unable/unwilling to do their own QA as its
| "not their job". Or unwilling to adapt to new technology
| the startup may be using.
|
| 2. Politics: Big tech companies have major politics
| leading to pathological "not our problem" conditions.
| Efficient and successful startups/scale-ups need to
| minimize politics, and _some_ employees from big tech
| companies will have found this to be one of their primary
| skills.
|
| 3. Ambiguity/Timeline constraints/dealing with crap: At a
| big tech company everyone is expected to be at the top of
| their game and the company rarely faces deadlines that
| are not self-imposed. An engineer may expect 100% test
| coverage, crystal clear product requirements, and no risk
| of failure. Dealing with sub-optimal conditions is common
| in startups/high growth companies.
|
| 4. Definition of success: A big company may strongly
| value marginal contributions as prized wins. Shaving ms
| off of a frequent call can drive real monetary
| improvements when the company has hundreds of millions of
| customers. Making engineers marginally more productive
| has huge benefits when you have 50k+ engineers. Startups
| often just don't care about these things and simply won't
| value the skills necessary for this work in most cases.
|
| On the flip side there are great engineers/managers who
| learned what made the big company successful and how to
| navigate internal obstacles. These employees are likely
| to be gold to a startup, but they are also gold to a well
| capitalized company with vastly more data on just how
| effective they are than a startup has. Odds are, startups
| are interviewing/hiring the employees the big companies
| don't care about - something the hiring firm often
| implicitly knows.
| TedDoesntTalk wrote:
| Don't conflate corporate size with engineering competency.
| It's counter-intuitive, but oh so common to see this. That's
| why so many engineers want to work for start ups!
| tablespoon wrote:
| > Don't conflate corporate size with engineering
| competency. It's counter-intuitive, but oh so common to see
| this. That's why so many engineers want to work for start
| ups!
|
| I thought start ups were all about letting tech debt pile
| up like there's no tomorrow, since there might literally be
| no tomorrow for them.
| errantspark wrote:
| The truth of the matter is that I've never seen anywhere
| where this isn't the case to some degree. Long term
| thinking is rare.
|
| I think it's very safe to assume the level of technical
| rigor in a given undertaking just falls to the minimum
| required unless there's a very strong force keeping it in
| a higher state. Maybe places like NASA JPL or Apple
| manage to float above the minimum because of a really
| unified and powerful culture, but outside of that I'm
| thinking it's more or less universal. e.g. the 737 MAX
| debacle illustrates Boeing's stochastic search for the
| lower bound of technical rigor when it comes to flight
| control software quality.
| tablespoon wrote:
| > I think it's very safe to assume the level of technical
| rigor in a given undertaking just falls to the minimum
| required unless there's a very strong force keeping it in
| a higher state.
|
| It really depends on who's setting the tone. If it's an
| owner or manager taking an active interest, I think your
| observation is true.
|
| > Maybe places like NASA JPL or Apple manage to float
| above the minimum because of a really unified and
| powerful culture, but outside of that I'm thinking it's
| more or less universal. e.g. the 737 MAX debacle
| illustrates Boeing's stochastic search for the lower
| bound of technical rigor when it comes to flight control
| software quality.
|
| IIRC, Boeing has made a years-long effort to cripple
| their unionized engineering workforce. I don't remember
| exactly where I read this, but for a long time they had a
| very effective, rigorous organization, but management
| (from McDonnell Douglas. IIRC) make a lot of changes that
| messed it up.
| asveikau wrote:
| I had an experience at a startup where the team was
| mostly experienced people who had been at larger
| companies and didn't tend to cut corners. It was a pretty
| good balance. We focused on doing the job well but didn't
| have big company meeting culture, etc.
|
| I wouldn't want to work at a place like you described.
| pnutjam wrote:
| I think this is because Amazon loves to churn teams.
|
| My team, not Amazon, had alot of churn when I joined.
| Basically a full turn over within a year. We lost alot of
| institutional knowledge and had to reverse engineer stuff all
| over the place.
| cduzz wrote:
| The org is getting what it wants out of the team; why would
| they fix it?
|
| As long as you've got meat for the grinder, you can keep
| making sausage.
| rodgerd wrote:
| Also: the cruelty is the point.
| SheinhardtWigCo wrote:
| I don't get this. In this market, if you're good, why take a
| job with _any_ on-call work? It's thankless, shitty work. If
| you can deliver features, that's good enough to get paid
| extremely well at other Amazon-scale companies with dedicated
| SRE teams. What's making you stay?
| awsthro00945 wrote:
| Because changing jobs is really fucking difficult.
|
| I'm probably one of the biggest proponents of "quit your job,
| you deserve better" that you'll ever find, but even I have to
| admit that finding a new job is ridiculously hard. Even in
| "this market", even if you're a top engineer, it's still
| ridiculously hard to even get an interview, let alone get
| hired.
|
| There is only a limited amount of companies that will pay at
| the same level as Amazon, and those companies often have
| months-long interview processes with ridiculous requirements
| that, even if you are a top engineer, still require a lot of
| time and effort be set aside to prepare for the specific
| interview processes that the new company is looking for. And
| that's to say nothing of the nebulous "culture fit" that is
| just as likely to prevent you from getting an offer and is
| completely unaffected by how "good" of an engineer you are.
|
| Almost everyone I know at AWS is interested in switching
| jobs/companies, but it really is not just something you wake
| up one morning and decide to do. It's a long, perpetual
| process that can take up a huge amount of time and effort
| (stuff you don't have a lot of when you're working on-call at
| AWS anyway), not to mention has huge implications if you are
| relying on your job for something like a work visa.
| tolbish wrote:
| It sounds like the real reason is that the money is worth
| it.
| awsthro00945 wrote:
| The money is probably the thing that keeps most people in
| the job, but it's not like willingness to take a pay cut
| magically makes jobs fall into your lap, either. It
| expands the pool of companies you could possibly work
| for, sure, but everything else above that I said still
| applies.
|
| There's also rarely a guarantee that the new company
| you're trying to join is significantly better, and
| companies intentionally make it difficult to get the
| inside scoop on work-life balance/on-call
| responsibilities until after you're hired. Personally, I
| would love to find a new company to join... but my
| experience interviewing is that it takes months of effort
| for a potential pay cut and no guarantee that I won't
| just end up in another shitty situation and paid less, so
| I struggle deciding if it's actually worth it.
| pkaye wrote:
| > not to mention has huge implications if you are relying
| on your job for something like a work visa.
|
| Isn't it just apply for a new job and as part of the hiring
| process they handle the paperwork? I know people on work
| visas that have jumped jobs every two years.
| ab_testing wrote:
| Also about a third of Amazon's engineers are on a visa. You
| can transfer jobs but it is never smooth sailing and full of
| gotchas.
| rejectedandsad wrote:
| The answer is 1) many people at Amazon like me have inferior
| intellects and everyone knows it, so it's harder to get a
| better job and 2) there are many services that don't have
| dozens of sev2s per day because ultimately you do own what
| you build. It's a big reason why Lambda is used to heavily
| internally now - we cut out a class of operational scaling
| issues out by design.
| MuffinFlavored wrote:
| > I can safely say that it's totally up to your manager and
| pre-existing conditions which make up the job
|
| Isn't it like that at any job?
| hintymad wrote:
| What I don't get about Amazon is that AWS customers have way
| less oncall load, particularly Netflix, whose engineers could
| afford oncall 24x7 for weeks or months with perfectly normal
| work-life balance. Why can't Amazon, the pioneer of cloud
| computing, achieve the same level of effectiveness?
| tylersmith wrote:
| Maybe I'm misunderstanding you but AWS customer's eningeers
| have less oncall load because the AWS engineers are doing so
| much of the difficult and failure-prone work.
|
| That's largely what you're buying when paying for cloud
| services.
| hintymad wrote:
| What I meant was that Netflix built their platforms and
| services on top of AWS services, just like AWS teams. Yet
| AWS teams have brutal oncalls, while Netflix teams enjoy
| great work-life balance.
| kaesar14 wrote:
| But that's because the AWS teams are presumably
| shouldering the burden, then.
| hintymad wrote:
| I'm sure you're right for some services, especially infra
| ones, such as EC2. Some other services, though, should be
| built on top of EC2, EBS, Lambda, S3, and etc, in which
| case Netflix and AWS teams use the same infra, yet
| Netflix internal services require much less oncall
| kaesar14 wrote:
| No doubt Netflix is a much better run engineering org.
| dkubb wrote:
| Its interesting that Netflix built their systems in a way
| that _assumes_ the underlying platform is unstable. With
| Chaos Monkey and other systems they made sure things are
| resilient to flakey behaviour.
| aaronbrethorst wrote:
| Company culture and values flow from the top
| awscurrentdev wrote:
| Current AWS engineer here, can confirm. I'm absolutely broken.
| I'd second the point about the manager and pre-existing
| conditions making up the job. It's not clear to me if it's
| endemic, these are big orgs with teams run very differently.
|
| That being said, the on-call sucks. It's really awful, and
| something I've never seen before. It's also typically the
| primary cause of team churn for people in my org. This varies,
| as I've seen other teams stacked with L6 engineers with very
| little churn (4-8 years of tenure each). This is very much a
| pit of despair of our own making, but I still haven't figured
| out how teams like mine get out of it. My own view is that
| normalization of deviance means that engineers who've only
| worked at AWS just accept that getting paged many times in a
| week at awful hours for false positives is OK.
|
| There's certainly a view that the only way to get promoted
| (which is _incredibly difficult_ ) is to create new features or
| products. You read this of many orgs though, not just AWS. I'm
| not convinced it's fair to single out AWS. It can be endemic in
| some teams, and I've certainly worked with engineers who are
| clearly only focused on shining for promotion.
|
| The worst I've seen was a team go from 8 engineers with > 2+
| yrs tenure down to 4 people with < 6 months, over the course of
| a couple of months. This was for an enormous product. That team
| had a very tough 4th quarter.
|
| AWS does handle operations failure incredibly well. If you've
| hopped on a LSE call before, the execution to identify,
| mitigate and review correction of error (COE) is world class.
| Doc / design review is also very thorough.
|
| There's ample opportunity to learn a lot during your time at
| AWS, and many engineers have carved out incredible careers in
| this place. Just go in with your eyes wide open.
| imwillofficial wrote:
| I'm sorry you're having a rough time. Hang in there, and if
| you move on, good luck!
| imwillofficial wrote:
| I'm genuinely curious how saying good luck gets down voted.
| Fuck the HN crowd for this shit.
| xref wrote:
| Comments like "good luck!", "me too", and "hi James" will
| get downvoted because they don't contribute to the
| conversation
| imwillofficial wrote:
| When somebody is clearly in crisis, we stop sprerging for
| a moment and offer empathy and support. It's called being
| a human.
| sundarurfriend wrote:
| > we stop sprerging for a moment
|
| Yeah, a lot of empathy leaking out of your comment there.
| "It's called being human" ironic.
| awscurrentdev wrote:
| Thank you, I appreciate you saying this. I don't want to
| give the impression it's all awful though. We are
| reasonably well compensated (you do a lot better if you're
| based in certain countries than others).
|
| There are part of this job that were a lot harder than I
| could have imagined. The aim of my earlier comment was to
| make this clear to people considering AWS. When the hiring
| manager interviews you and mentions there's "an on-call
| component to this job", realise that it _can_ be severe.
| The on-call time is also unpaid; it's also very difficult
| to spend significant time on improving this situation, if
| that's possible at all. Other comments have done a better
| job of describing this.
|
| There are parts of the job that are fantastic. You have
| access to some outstanding engineers (this is also the case
| at many other companies though). Almost all of the
| principal engineers I've interacted with have been very
| generous with their time and knowledge. I thoroughly enjoy
| the Principals of Amazons talks, and subsequent
| discussions. I've also had the opportunity to be able to
| look very deeply at technical problems (this is a direct
| result of my manager). Having worked at a number of SMEs
| before, this wouldn't have been the case. You also work on
| systems being used by so many people (this is mostly
| wonderful in hindsight), which having worked on products
| that have evaporated into the ether in the past, is
| rewarding.
|
| It's not for everyone, and it's certainly not forever at
| AWS. My guess is I'll walk away with some scars and a much
| better idea of what I want I don't want to spend my time
| doing.
|
| You'll learn a lot and cry a lot.
| laegooose wrote:
| What is LSE?
| kablow wrote:
| "large scale event" or something to that affect - these
| issues are visible to everyone company-wide
| atopuzov wrote:
| Large scale event, something that has big impact on
| services.
| pyuser583 wrote:
| London School of Economics ;)
| exikyut wrote:
| Quote-Googling produced "Large Scale Event"
| doktorhladnjak wrote:
| > pit of despair of our own making, but I still haven't
| figured out how teams like mine get out of it.
|
| I've only seen two ways out: 1. Team implodes when everybody
| leaves, reorg follows making it some other team's problem 2.
| Management recognizes it's a problem, takes it seriously by
| staffing the team with enough people to sustainably address
| the tech debt/operational load AND build new features
| vr46 wrote:
| Sorry to hear about your awful job, is this only in
| engineering or does it affect professional services like
| solutions architects etc too?
| awsthro00945 wrote:
| ProServe and SAs have little-to-no on-call
| responsibilities, but the general workload issues and
| mindsets affect those teams as well. Just as an example,
| resource management (aka staffing) and project scoping are
| things that AWS Sales is _absolutely fucking awful_ at, and
| are things in particular that other consulting companies
| have figured out decades ago, but AWS does nothing to
| improve the shitty staffing processes because they have
| essentially just thrown up their hands and think the
| inefficient, inaccurate, and incredibly stress-inducing
| process is normal.
| ako wrote:
| What is really problematic is that from a management
| perspective they're doing really great: company is growing and
| extremely valueable, stock price is doing great, revenue and
| profits are doing great, and bezos is the richest person in the
| world.
|
| Management has zero incentive to change any of problems you
| signal, and probably don't see them as an issue. Probably the
| opposite, they see this as a winning system, and who can blaim
| them. AWS practices are often used as examples of best
| practices: you build it, you run it, 2 pizza teams, api first,
| etc. Survivorship bias and all, the probably regard the other
| characterestics of the current system as best practices as
| well.
| taway_zonian257 wrote:
| > Management has zero incentive to change any of problems you
| signal, and probably don't see them as an issue.
|
| SDE1 here, I might be completely wrong but I'm inclined to
| believe that upper management is kept practically blind about
| the real struggles and issues experienced on lower-levels,
| and this scenario is abused by managers to scapegoat their
| way out of problems they have to deal with, specially the
| ones they created. I'm not going to provide examples as they
| could be easily traced back, but I can say that I've
| witnessed a L7 overpromissing on a project in spite of
| callouts from lower levels, and once postponing the delivery
| of some milestones started to become inevitable then I've
| started hearing said manager frequently mention "firing
| offenses" on dailies.
| tablespoon wrote:
| > Management has zero incentive to change any of problems you
| signal, and probably don't see them as an issue.
|
| It sounds like management intentionally created these
| practices. This is from a recent NY Times article about
| Amazon warehouse workers, but it wouldn't surprise me if this
| attitude was applied to all workers in the company to some
| degree:
|
| https://www.nytimes.com/2021/06/15/us/politics/amazon-
| wareho...:
|
| > 5. Many of Amazon's most contentious policies go back to
| Jeff Bezos' original vision. Some of the practices that most
| frustrate employees -- the short-term-employment model, with
| little opportunity for advancement, and the use of technology
| to hire, monitor and manage workers -- come from Jeff Bezos,
| Amazon's founder and chief executive.
|
| > He believed that an entrenched work force created a "march
| to mediocrity," said David Niekerk, a former long-serving
| vice president who built the company's original human
| resources operations in the warehouses.
|
| > Company data showed that most employees became less eager
| over time, he said, and Mr. Bezos believed that people were
| inherently lazy. "What he would say is that our nature as
| humans is to expend as little energy as possible to get what
| we want or need," Mr. Niekerk said. That conviction was
| embedded throughout the business, from the ease of instant
| ordering to the pervasive use of data to get the most out of
| employees.
|
| In many cases software engineers overestimate their own value
| to the company, often to the point were they'll ape the
| attitudes of owners and management (e.g. rejecting the the
| idea of a union out of hand). But the fact of the matter is
| they're cogs just like warehouse workers, and in many
| situations competent management will exploit the hell out of
| them to extract maximum value for the shareholders.
| abnercoimbre wrote:
| When did humane treatment, job security, and the worker's
| collective voice leave the picture? Are we to simply praise
| everything laid out here?
| rodgerd wrote:
| Some time in the 1980s, when people were convinced that
| if they stabbed themselves in the back, they had a shot
| at being millionaires.
| kazen44 wrote:
| > Company data showed that most employees became less eager
| over time, he said, and Mr. Bezos believed that people were
| inherently lazy. "What he would say is that our nature as
| humans is to expend as little energy as possible to get
| what we want or need,"
|
| If this was truly the case, humans would have evolved very
| differently (and it's needs would be substantially smaller.
|
| Also, what about all those people working for little gain?
| WJW wrote:
| It's a pretty interesting take given that Bezos himself is
| the most entrenched person that Amazon has.
| ipaddr wrote:
| They are doing a great job from a management perspective but
| it is not good place on average for a developer. Both can be
| true.
|
| Management only needs to change if all developers go else
| where.
| jjtheblunt wrote:
| Current Amazon engineer: it's far and away the most
| incompetently run bureaucracy with self-defeating dysfunctions
| forced on the huge number of layers. The recent "leaks" to
| Business Insider, pointed out to me by coworkers, are exactly
| what we see.
|
| Everything the parent comment mentioned is exactly what we see.
| Laughably, our middle manager berates us as incapable noobs,
| entirely unaware that some of us at least actually have been
| very valued and trusted and dependable hard core results driven
| engineers for decades. It's like a failed brainwashing attempt,
| and it's embarrassing to be associated with such idiocy.
|
| I contrast it with excellent experiences at Sun, Motorola,
| Apple and others over the past 20+ years, where in some cases I
| had very high engineering ranks with fabulous results, in very
| well run and just healthy orgs.
|
| I do believe there's intentional treatment of engineers as
| fungible assets, because the engineering quality is so shoddy
| and the business plan prioritizes maintaining system uptime,
| with no true priority given to tech debt removal, which
| actually would in short order surface measurable benefits.
| rodgerd wrote:
| > I do believe there's intentional treatment of engineers as
| fungible assets, because the engineering quality is so shoddy
| and the business plan prioritizes maintaining system uptime,
| with no true priority given to tech debt removal, which
| actually would in short order surface measurable benefits.
|
| I mean this how Amazon treats all of its employees. It can't
| quite lock devs and ops people to a desk and have them piss
| in bottles, but they would if they could.
| lupire wrote:
| Only Apple is on Amazon's tier of technical success, and half
| of Apple's tech success is from non software.
|
| So on what basis ia Amazon incompetent at software? It sounds
| like Amazon is results driven, and pretty well-unit-tested
| code isn't critical for that.
| cerved wrote:
| OP made no mention of software and mentioned Sun and
| Motorola, so I don't think they made any point specifically
| about software.
|
| It may also be relevant to point out that Amazon is first
| and foremost a retailer, unlike the other companies
| mentioned.
| hn_throwaway_99 wrote:
| > Amazon is first and foremost a retailer
|
| I feel these days that it's more correct to say Amazon
| _was_ first and foremost a retailer, but these days they
| are just UberMegaCorp across so many businesses (Amazon
| site proper, AWS, WholeFoods, Kindle, Echo /Alexa, movie
| studios, etc. etc.) that it's hard to say they are
| foremost a retailer.
|
| I have a similar question, though, in that I've heard
| lots of nightmare anecdotes about Amazon code quality,
| but obviously whatever they are doing is working on some
| level.
| refactor_master wrote:
| Sure, why test code when you can throw expendable people at
| it instead?
| tazjin wrote:
| > I contrast it with excellent experiences at Sun, Motorola,
| Apple and others over the past 20+ years, where in some cases
| I had very high engineering ranks with fabulous results, in
| very well run and just healthy orgs.
|
| I can understand how junior devs end up in these positions
| for a while and stay on the ride, but what is making someone
| with your level of experience you stick to Amazon? Is the
| compensation that good?
| jjtheblunt wrote:
| Compensation seems good, but I'm there because I got fooled
| and plan to quit very soon, when graciously (not screw
| team-mates) feasible, because it's just a waste of time
| that could be spent helping positive efforts in healthy
| orgs.
| tablespoon wrote:
| > ...I'm there because I got fooled and plan to quit very
| soon, when graciously (not screw team-mates) feasible,
| because it's just a waste of time that could be spent
| helping positive efforts in healthy orgs.
|
| Quit now, and encourage your teammates to do likewise. I
| understand where you're coming from, but the attitude you
| have can be a source of exploitation (e.g. using loyalty
| to peers to keep you while the org screws all of you
| more).
| Retric wrote:
| I have felt that way before, but it's better just to quit
| ASAP. It's the company that's making your teammates
| suffer and their also better off just quitting.
| chipper02 wrote:
| I live by the motto "I work for money and appreciation,
| in that order. If you want loyalty, buy a dog!" It has
| served me well and removed a LOT of stress. I walked out
| of one crap-hole with a yelling management style on on
| week's notice. They told me it was unprofessional, and I
| laughed in the VP's face and told him he was lucky he got
| a week.
| johannes1234321 wrote:
| "Unique technical challenges"
|
| Some of the problems you can work on in such an environment
| are interesting.
|
| Combine that with the fact that you don't see this before
| joining and that some.manager s might be able to shield
| some of it you are where you are.
|
| And then throwing hands in the air and leaving is a step,
| as most other companies have "smaller" problems to solve
| and "also bad management" ...
| dnautics wrote:
| well the good news (for amazon) is that I used to work at a
| place where I was trying to build a cloud. And management
| suffered from EXACTLY THE SAME PROBLEMS (though for different
| structural reasons, there was no "build a feature"
| incentive). And then we hired someone who used to work on
| amazon cloud who came with rave reviews, even though he
| completely failed my interview, which basically tested "do
| you read the spec?". Then I told him to take notes while I
| was onboarding him, which he didn't do, and then I quit,
| because of the management problems.
|
| > the engineering quality is so shoddy
|
| What stuns me is just how well AWS works. I mean, did you
| ever try to use Azure in 2015? Hell, even GCP was worse in
| that era. Now, it's on-par, but IMO "more confusing" which
| beggars belief considering how confusing AWS is.
| blueblisters wrote:
| Yeah it's surprising that despite the negative comments in
| this thread, AWS works almost flawlessly for stuff that
| matters (uptime, reliability etc.). Of course it's painful
| to work with sometimes, but so are other cloud vendors.
| There's literally zero incentive to change things
| internally until external KPIs start turning red.
| hef19898 wrote:
| IMHO that's the down side of Amazon's customer obsession.
| As long as it works for the customer, nothing will
| change. Regardless of internal benefits. And that system
| quite obviously works.
|
| Disclaimer: Former logistics Amazonian.
| johndubchak wrote:
| I feel for all of you in this thread that work at Amazon
| and complain of the poor conditions. There are no shortage
| of stories that point to a very large problem in that
| company, for certain. However, I believe the underlying
| theme, if there is one, is incompetent Management and
| Leadership.
|
| This is systemic in Technology companies because, to me,
| the "bean counters" try to run the companies like a basic
| Manufacturing organization of commodities where line items
| and people are substitutable things and can be dialed and
| tweaked at the planning stage.
| taway_zonian257 wrote:
| > Everything the parent comment mentioned is exactly what we
| see.
|
| Current sde1 bluebadge. I also see exactly what the parent
| comment mentioned. I also saw a few nasty episodes such as L6
| managers quitting in less than a year in because they hated
| the pressure they were subjected to, and a yellowbadge say
| during a all-hands to a bunch of newly arrivals that he was
| at the company when they joined and he will still be at the
| company when they all left. I know of a team where SDE2s and
| SDE3s felt the need to pull in all-nighters and even work the
| entire weekend to meet the deadlines that their L7 leadership
| pulled out of their ass. I myself felt compelled to work
| close to 12 hours a day and weekends during a period just
| because my senior manager wanted to shine. And yes, I've
| already see a fair share of colleagues disappear.
|
| Of course YMMV but I'm baffled why I'm seeing accurate
| descriptions of what I'm personally experiencing on a daily
| basis being faced with such a sense of incredulity.
|
| > because the engineering quality is so shoddy and the
| business plan prioritizes maintaining system uptime,
|
| I've created this throwaway account just to comment on the
| issue of engineering quality being shoddy.
|
| It's true that engineering quality is shoddy but the problem
| is caused by Amazon's structure and not by the engineers
| themselves. For example, each and every single engineer is
| expected to deliver major tasks alone as part of their
| evaluation process, and this means that by design we are
| placed on tasks where we are way over our head and we are
| still expected to deliver things out of thin air. This sort
| of trial by fire is sold under the premise of allowing us to
| deliver consistently at the next level before being
| considered for a promotion, but in practice if you screw up
| that's expected to also be held against you.
| mavsman wrote:
| Former AWS/Alexa engineer. Totally forgot about the whole
| badge color thing. Can't stand that crap.
|
| Things are very dependent on team and manager and org but I
| will say that one of the things I'll never miss is our
| Re:Invent launch. That was horrible. Our team (startup
| acquisition) basically had a giant retro after the launch
| to figure out how we could avoid doing that again. Most of
| the startup members left the team of the company 2 years
| in. I learned a lot.
| kottapar wrote:
| > We had a lot of little scripts here and there which would
| solve extremely specific situations but no focus was ever put
| on in building a general framework or trying to reduce the
| ticket count.
|
| wow, honestly this is surprising. For me as an end customer I
| was always impressed with the way the services are engineered.
| Kudos to people like you for making this happen. But then I was
| also under the impression that AWS has very good best practices
| to take care of repeating issues.
|
| Wouldn't interim patch-ups cause stability issues in the long
| term?
| hardwaresofton wrote:
| > Wouldn't interim patch-ups cause stability issues in the
| long term?
|
| No matter what people tell you or put on their marketing
| blog, it feels like this is the state of play for 99% of
| software teams. The only time it doesn't end up like this is
| when you leave time up front to pay down technical debt (like
| Intel's tick-tock model), and almost no one does that.
| kazen44 wrote:
| my senior once said to me that the world is held together
| by two things.
|
| Ducttape and shellscripts. This was a while ago, but the
| older i get the more i tend to agree.
| kablow wrote:
| I'm in non-AWS and things aren't any better here. It is very
| much dependent on the manager and team, and I've been
| optimistic (~3 years now), but it gets harder as time goes on.
|
| > I'm just not 100% sure about the whole PIP scene. Our service
| was extremely critical and we were extremely understaffed. So I
| don't think it applied to anyone in our org but I know of other
| teams who would have no issues in taking in a fresh college
| grad, making them do work for 6-12 months and then just
| randomly putting them on PIP.
|
| Our team is over-worked and has a large ticket queue, constant
| sev-2 pages, understaffed, etc. - and yet they still PIP'd (and
| then fired) someone last year who didn't deserve it IMO.
| ipaddr wrote:
| Amazon has a lot of money to waste. It is cheaper to hire
| someone so they can fire them in a few months to keep the
| rest of the staff scared enough to overwork themselves so
| they can understaff.
|
| They play a good game.
| adwn wrote:
| > _Amazon has a lot of money to waste. It is cheaper to
| hire someone so they can fire them [...]_
|
| So... Amazon has a lot of money therefore they have to
| resort to money-saving practices? Or is the causality the
| other way around? How do these two sentences fit together?
| rejectedandsad wrote:
| I don't think they do, but it's a way to smear and make
| all Amazon engineers unemployable by proxy. I already
| have difficulties finding interviews at good companies
| because the Amazon name implies IBM level talent instead
| of Google level talent.
| ipaddr wrote:
| Amazon has a lot money. These practices make them
| additional money. If they didn't have a lot of money they
| couldn't afford to hire to fire.
|
| How can they do both? - They waste money when they hire
| to fire and constantly onboard. - They make money by not
| staffing enough resources but by using fear of being
| fired to force overtime.
|
| Maybe the hire to fire costs are justified because the
| savings they get by understaffing outweights the cost.
| IggleSniggle wrote:
| Wow that's a huge wtf. You don't have enough manpower, and
| then you take somebody who's contributing and fire them?
| Seems totally insane.
| PragmaticPulp wrote:
| I worked at a company that hired several people out of Amazon.
| Amazon is a big company, so there is significant variation from
| team to team.
|
| However, several of the ex-Amazon employees were clearly scarred
| from their time working at Amazon. Whenever something went wrong
| (delays, outages, missed deadlines) one of them went so far as to
| spend hours or days preparing documents showing how it was
| actually someone else's fault, not his, because he thought it was
| necessary to avoid being fired. He said his experience at Amazon
| was basically one large game of "not it" whenever it came time to
| assign blame for the latest issue, and everyone had to become
| very good at blaming someone else. The worst was when he was
| actual at fault for something, because he would panic and
| scramble to distract from the issue he caused by raising concerns
| about everyone else around him. Easily the most toxic person I've
| worked with in the past few years.
|
| On the other hand, I've known other ex-Amazon people who said
| their work was nothing like this, so YMMV
| amazon_throw wrote:
| That "CYA" response really doesn't feel like "us" to me, at
| least not the AWS side. I can't speak to the experiences on the
| store side, nor on the devices side. Over in AWS, we practice
| the "blameless postmortem" model you'll read about from time to
| time, and it is very rare to see any professional or personal
| consequences beyond some light joking come about from "breaking
| prod"
|
| To that point, I know personally the engineers who triggered
| several of the AWS outages you will have read about in the news
| over the last decade and more. Some of them are still with us,
| some have been promoted since then.
| itg wrote:
| Depends on the team. In my team in AWS, if something was your
| fault, the manager would call you out in front of the entire
| team. Then they wondered why attrition was so high.
| [deleted]
| delecti wrote:
| Your experience also matches mine from my time in the
| advertising org. There's no benefit to placing blame, just
| figure out what went wrong, how to fix it, and how to prevent
| something like it from happening again. In fact it was one of
| the few parts of the culture that I tried to take with me
| after leaving.
| teawrecks wrote:
| Reminds me of that classic parable about the CEO and the
| worker who just made a $100,000 error. The worker says, "I
| assume you'll be firing me now" and the CEO says, "Fire
| you?! I just spent $100,000 training you!"
| imwillofficial wrote:
| I'm agreeing with the "it's not us" vibe. I work on a team
| that has some planet wide scale software pipelines.
|
| I broke it, hard. I'm not a software engineer so I had no
| idea how to undo what I had done.
|
| People from 3 different teams, two of them experts from other
| teams who heard my call for help hopped on a call and spent
| all day helping me fix the issue I made and then some. This
| was a hard stop on a cortical package pipeline for 24 hours
| and at no point did I feel like my job was at stake despite
| it clearly being my fault, and due to me running a command I
| know I shouldn't have and cutting corners.
| pram wrote:
| I've had two job offers from AWS for SA and TAM and I turned both
| down. The actual position was always lower than what I was
| originally interviewing for (Senior instead of Staff)
|
| Literally the only company I've seen that does that. It's like a
| bait and switch. If I'm not actually qualified just say no lol
| tweenagedream wrote:
| I think down leveling is quite common, especially at higher
| levels as the expectations grow quite a bit between senior and
| staff. Depending on the position, actually filling it with +/-1
| of the target level is a win. Interviewing is expensive and if
| a candidate is good for the role, but perhaps just a bit below
| the bar, then a recommendation to hire at L-1 tries to keep the
| candidate and give them a career path.
|
| This is from my perspective at Google, I've never worked at
| Amazon.
| jelling wrote:
| My friend works there and said that virtually everyone gets
| dropped a rank on the way in. It's so common that people
| commonly introduce themselves as "I'm an X but pre-Amazon I was
| an X+1".
| Frost1x wrote:
| Sounds like an HR policy to reduce labor costs. As long as
| Amazon keeps a steady supply of labor behind you and at least
| some of them don't have other comparable options, they're
| going to agree to these terms and their hiring teams are
| going to continue this nefarious practice. Some may agree to
| it alone in an unconscious sunk cost fallacy.
|
| There really need to be laws with teeth about false
| advertisement from both employees and employers so we can
| skip this sort of non-sense false advertising.
| aix1 wrote:
| If someone was L+1 at a similar-tier company, why would they
| accept the downlevelling instead of a finding a suitable gig
| elsewhere? Is Amazon that appealing? Are openings that
| scarce?
| TulliusCicero wrote:
| Compared to smaller/less prestigious/non-tech companies, a
| certain level at Amazon/Google/FB will generally come with
| a higher expectation for technical competence, as well as
| more comp -- even an L4, mid level engineer at Google makes
| like 250k. So someone who's a "Senior Software Engineer"
| from some rando firm in the Midwest may still be doubling
| their salary or more.
|
| The extreme version of this is being CTO as a startup of 10
| people vs CTO at a company of 10,000.
| pc86 wrote:
| This might be a dumb question but if it's standard to drop
| everyone a level (I've heard this too) is it possible to
| interview for Principal or something where they would drop you
| back to Staff?
| f6v wrote:
| I'd interview for CEO instead and see where that lands me.
| FredPret wrote:
| Funny but also not a bad approach to life
| TulliusCicero wrote:
| Realistically, the expectations for technical competence for a
| senior engineer are going to be higher at a huge successful
| tech behemoth like Amazon or Google than most smaller companies
| (though maybe not tech start ups), especially non-tech ones.
| steelframe wrote:
| I'm at the point of my career where it's all about the manager,
| the hours, the comp, and the kind of work I'll be doing. I no
| longer care as much about the title or level. In fact if the
| level is lower, great -- I'll have an even easier time getting
| high perf scores.
|
| These days, when I negotiate for a new job in Big Tech, I first
| and foremost maximize base pay. That's [1] immediate (no
| waiting for vest or anything), [2] durable (carries forward
| into future jobs), [3] non-variable (doesn't arbitrarily change
| much year to year while at the same company), and [4] the basis
| for bonus.
|
| Then, I pay very close attention to who my manager will be. In
| fact I've once switched companies just to follow a great
| manager.
|
| After that, I make sure I won't be carrying a pager that can
| ever "go off" in the middle of the night. That's a non-
| negotiable for me at this point in my career.
|
| Then I negotiate up-front for a couple of pre-planned multi-
| week trips in my first two years to get around the bullsh*t of
| "you don't get real vacation time until you've earned it by
| staying at the company long enough."
|
| Finally I look at the small print in the non-compete and the
| benefits. Oh wait, you mean my long-term disability coverage
| won't actually become effective until I've been with the
| company for 12 months? Nope; it's effective immediately. Oh,
| you want 18 months of non-compete? Nope, you're doing what
| every other company does. 12 months.
|
| I'm finding that I'm in way too high demand among tech
| employers to put up with anything less. I am well aware that my
| demands are too much for places like Amazon, which is no skin
| off my back.
| Seattle3503 wrote:
| What did you do in your career to get to a point where you
| could ask for those things?
| mathgenius wrote:
| > switched companies just to follow a great manager.
|
| This is gold. I'm starting to realize (20 years in) that who
| you work with, and for, makes the difference. I really need
| to make this a priority for my next gig.
| zetsurin wrote:
| >Oh, you want 18 months of non-compete?
|
| Is this even enforceable? Maybe if your C level, and they pay
| you while you wait.
| chewzerita wrote:
| > After that, I make sure I won't be carrying a pager that
| can ever "go off" in the middle of the night. That's a non-
| negotiable for me at this point in my career.
|
| I'm currently in undergrad so starting my actual "career"
| still seems far enough off for me, but serious question how
| do people actually accept that? _Will I have to accept that_
| when I apply for junior /entry level positions? I don't think
| I'm asking too much if I want to have a full night sleep and
| a strong work/life divide. I might be a bit young and naive,
| but I hope I won't get comfortable with living at the beck
| and call of my employer. People are more than just their
| individual contribution to lining the pockets of their
| bosses, no?
| ev1 wrote:
| It's entirely dependent. This is also partially why "do you
| have a good manager" is included. I've had a few that
| absolutely will be in the line of fire first, and not make
| their team do anything they wouldn't do themselves.
|
| From experience: a good team/manager, when forced to do
| this, will often freely be first on call, give you a day
| off after if you had an incident at night or let you sleep
| in til noon or later without complaint, etc.
|
| If you are European or similar country with better working
| conditions/employee laws, it will be less painful. IIRC, in
| US salaried tech employees can effectively have unlimited
| unpaid overtime as a specific exemption.
| chewzerita wrote:
| Good advice about managers, it sounds like it may take a
| while to settle down and find a team that fits. It's good
| that you have been able to find a manager and team that
| actually works as a, well, team.
|
| > IIRC, in US salaried tech employees can effectively
| have unlimited unpaid overtime as a specific exemption.
|
| Yeah I see that daily with my dad working from home. He
| works at a local newspaper (wait those still exist?)
| doing stuff with maps, datavis and page layout/design. He
| works probably twice the amount of hours he gets paid for
| early morning until late at night, and can never take
| time off even with his measly "paid vacation time"
| allotment. He somehow manages to trudge along, which is
| unimaginable to me. Generational difference? Or maybe I
| just don't have the full picture yet
| kondu wrote:
| Unfortunately, you don't always know if on-call will be
| involved until you start at the job. Often many companies
| don't even hire you with a particular role in mind, instead
| you are matched to a team after you're signed on.
| (Especially for new grads). And sometimes on-call is
| introduced in an existing role, it can be difficult to
| refuse, especially for people in more junior roles.
|
| But definitely ask about on-call when interviewing, in case
| they say they have it you can bail out before you sign.
| mancerayder wrote:
| I ask about it repeatedly to all interviewers to the
| point that some companies disqualify me. Call it a
| survivorship bias in reverse. It doesn't always work,
| amazingly, but it always sends a strong signal.
| maxlamb wrote:
| It depends a lot on the company and specific role. As a
| junior person there is a higher chance that the position
| you apply for will require this kind of commitment but not
| all. Just make sure you ask during the interview process.
| __derek__ wrote:
| AFAIK, there is no Staff designation. On the IC track, L6 is
| Senior, L7 is Principal, and L8 is Senior Principal.
| pram wrote:
| Yes whatever is above Senior, I don't have every corporations
| IC structure memorized
| hn_go_brrrrr wrote:
| http://levels.fyi is a great reference for converting
| between levels at different companies. Cuts through a lot
| of the bullshit recruiters and HR will try to sell you.
| alisonkisk wrote:
| Article does not support headline. Even if you don't stay at
| Amazon for many years, that doesn't mean your career is over or
| worse than having not worked there.
|
| Also, it's about Amazon India, so take care before generalizing
| globally.
| DangitBobby wrote:
| You're ignoring the part about the impact the psychological
| toll will have on your ability to perform and the fact that you
| can't use your manager there as a reference since they've
| intentionally railroaded you and have fake peer reviews on file
| that do the same thing.
| mattacular wrote:
| Amazon has had this reputation for as long as I can remember.
| Sounds like they treat their employees like shit at every level
| and in every country. When I was looking for a new job it was not
| even a consideration to apply.
|
| The only defense of the company seems to be "well maybe you'll
| get lucky and work for one of the good managers who will treat
| you with a modicum of respect and they'll stay good as long as
| you're there" - not exactly the dice roll I'd aim for.
| sombremesa wrote:
| When I was at Amazon the org structure at my org changed about
| six times in half as many years. I'd say your chances of having
| the same manager for a while are not great.
| umanwizard wrote:
| Amazon is a loose federation of different teams. Some are fine
| and some are bad.
|
| My counter-anecdote: I worked at Amazon for two years and it was
| mostly fine. Normal boring job.
| devoutsalsa wrote:
| If the majority of stock options vest in years 3 and 4, may I
| ask why you left after 2 years?
| lumost wrote:
| You have the strongest negotiation power with other companies
| at year 2 assuming the stock has done well. Companies need to
| match/exceed your projected comp 2 years out, and for some
| reason are more willing to do so when someone else is paying
| you at a certain level.
|
| At Amazon you may face a cliff in year 5 if you haven't been
| performing at the 80%+ mark for your level and haven't been
| promoted in the last 4 years. At the higher levels where
| advancement is more difficult this increases the incentive to
| lock in a 4 year compensation package of your 3 & 4 year
| comp.
| oblio wrote:
| Random guess: new job with higher total compensation,
| especially big signing bonus and earlier vesting.
| asdev wrote:
| At some point you need to weigh work life balance, career
| fulfillment, responsibility(more at a startup) vs a FANG salary.
| Some people don't mind either working their ass off for a large
| salary or being a cog in a machine. But for most, the
| idealization of FANG is not what it's cracked up to be
___________________________________________________________________
(page generated 2021-06-20 23:00 UTC)