[HN Gopher] Ask HN: Has anyone worked at the US National Labs be...
       ___________________________________________________________________
        
       Ask HN: Has anyone worked at the US National Labs before?
        
       I have spent the last 10 years working for FAANG companies, but
       nowadays I find their performance-review and promotion obsessed
       cultures to be really draining. Worse, those negative feelings seem
       to be leaking into my personal life and slowly alienating friends
       and family.  Therefore, I've been pondering a change of pace. The
       classic HN answer is of course "create/join a startup", but I've
       also been looking at areas more adjacent to scientific research.
       One option that has come up is the US Department of Energy's
       national laboratory network[0]. From what I understand, the pay is
       33-50% of FAANG, but they do seem to have interesting projects
       (e.g. the nuclear fusion facility that was recently in the news).
       Has anyone here worked at one of them before? What is/was the day-
       to-day like?  [0] https://www.energy.gov/national-laboratories
        
       Author : science4sail
       Score  : 161 points
       Date   : 2023-01-17 16:27 UTC (6 hours ago)
        
       | schoen wrote:
       | I was a summer intern at LBNL as a student. Beautiful views and
       | very smart people (and very big computers).
       | 
       | The cafeteria is way less fancy than tech company cafeterias. :-)
       | 
       | As a few posts have pointed out, there are national labs that do
       | only unclassified work (LBNL is one). So you don't have to get a
       | clearance or be prohibited from accessing lots of places or
       | conversations, and you don't have to work on weapons. You _do_
       | still have to sign a loyalty oath as a state government employee
       | (the lab being managed under contract by the University of
       | California), something that became highly objectionable to me in
       | retrospect.
        
         | floren wrote:
         | > The cafeteria is way less fancy than tech company cafeterias.
         | :-)
         | 
         | The cafeteria at Sandia in Albuquerque always had extremely
         | good posole, though.
        
           | rheone wrote:
           | The posole is a thing of the past. The latest food contract
           | switch over didn't come with the recipe as I understand it.
        
         | gautamcgoel wrote:
         | I thought that loyalty oaths were not allowed in the UC system?
         | I recall there was a brouhaha many years ago where they were
         | requiring employees to disavow communism; several people
         | refused and were fired. As a result, the UC system adopted a
         | rule that political litmus tests would not be allowed when
         | making hiring/firing decisions.
        
       | maldev wrote:
       | Dont work directly for the national lab, work for the contracting
       | company. Pays alot better and same work.
        
       | tsbischof wrote:
       | I worked at LBNL, at the Molecular Foundry. The day to day was a
       | mix of typical nanoscience work (chemical synthesis, electron
       | microscopy, etc) and work in support of the user facility. In my
       | case that involved consulting on projects involving our users
       | (design of high-throughput screens, teaching spectroscopy, etc),
       | setting up and maintaining instrumentation, and developing
       | workflows for our chemical synthesis robots.
       | 
       | I liked the work and really enjoyed getting to be a consultant on
       | many projects. Turnover is massive among the researchers because
       | there are few permanent positions, and most groups are heavy on
       | postdocs since graduate students tended to be primarily on campus
       | (UC Berkeley).
       | 
       | If pay is a concern, look closely for the open databases of
       | salaries. At LBNL there is the "book of tears" at the library
       | under the cafeteria, listing every employee and their salary. The
       | exact amount you get varies wildly with the department: prior to
       | unionization in 2016, the range was from 20k to 125k annual
       | salary for postdocs. I hear they raised the floor to NIH levels
       | at least, but I assume they did not make NERSC take a paycut.
        
       | abadger9 wrote:
       | I'm surprised more people haven't brought up NASA? I was offered
       | 180k/yr salary in 2020 doing fullstack development. While the
       | offer was brought through a recruiting agency, I believe it was a
       | full time position as a NASA Employee - I was surprised by the
       | competitive salary, maybe worth looking into for others
       | interested in the space.
        
         | compsciphd wrote:
         | doubtful it was as a NASA employee (i.e. US Govt). It was
         | probably as a contractor. I believe (someone can correct me if
         | I'm wrong) that NASA employees are on the GS schedule, as GS-15
         | step 10 in the DC area, only gets ~176K (after cost of area
         | increase). i.e. a very senior (but not executive) manager with
         | many years at that level.
         | 
         | The value of working at a national lab (vs say a military one)
         | as an employee is that when one works at a military lab as an
         | employee, one is a govt employee and on the GS schedule (which
         | makes it hard for them to compete). When one works for a
         | "national lab", one is an employee of the lab which is under
         | contract with the US govt, but they are not paid according to
         | the GS schedule.
        
       | robg wrote:
       | There's a cool accelerator that is partnering with Oakridge and
       | out of TN that might be a good middle ground:
       | 
       | https://www.techstars.com/accelerators/industries-of-the-fut...
        
       | dopeboy wrote:
       | I have a friend who works at Sandia. I remember him saying the
       | pay is lower and there is way more work life balance. There is
       | less of a sink or swim attitude around perf there. I got the
       | sense you can come in, do your thing, and be out by 4 or 5 -
       | everyday.
        
         | dehrmann wrote:
         | My dad works at a national lab. The other side of this is that
         | underperformers stick around longer, and working with them can
         | be frustrating.
        
       | uberman wrote:
       | Just a heads up that most of the interesting work will require a
       | "current DOE security clearance". Many positions at places like
       | LLL or any DOE lab really are going to require the more intense Q
       | clearance.
       | 
       | Sometimes prior clearance can be negotiated and a well qualified
       | candidate who is likely to clear might be accepted and placed in
       | a holding pen until they clear but I'm not sure what the backlog
       | is now or even if they do that anymore. At the very least you
       | will almost certainly need to be a US citizen proper.
        
         | floren wrote:
         | When I worked at Sandia, most of us were hired without any
         | clearance but with the expectation that we would get one soon.
         | 
         | It is not difficult to get a Q clearance, just annoying. You
         | have to fill out a massive document listing everywhere you've
         | lived in the last 10 years (a real hassle for a recent grad)
         | and give all sorts of info about people you know. They will
         | drive out and interview people.
         | 
         | There was plenty of work that did not require a clearance, but
         | so much of the sites are cleared-only that it just makes your
         | life easier to have it.
         | 
         | edit: oh yeah good point made in the dead comment below, if
         | you've smoked weed in the last 7 (? something like that) years
         | you're gonna have to tell them. Even if it was legal in the
         | state where you did it. I've heard it's not a deal-killer these
         | days, but they want to know and _you will have to stop using
         | it_. There are not a ton of ways to lose your job at a National
         | Lab but failing a drug test is one. Do not toy around with it.
        
           | oppanoppen wrote:
           | > It is not difficult to get a Q clearance, just annoying.
           | 
           | Will be if: you do drugs (including pot), have debt problems,
           | have dual citizenship esp. if you have made use of it in some
           | way.
           | 
           | > You have to fill out a massive document
           | 
           | SF-86. https://www.opm.gov/forms/pdf_fill/sf86.pdf
        
             | SoftTalker wrote:
             | These are the forms that the OPM leaked some years ago,
             | BTW. So there's that risk also.
        
           | [deleted]
        
         | zhivota wrote:
         | Oh yeah this is the other thing for sure. In my lab I didn't
         | need a clearance for the work I was doing, but my boss had one
         | and his later grants were going to require it. He asked me to
         | come back and I declined because I didn't want a clearance.
         | 
         | It's not just annoying to get it, it's kind of intrusive to
         | keep it. You have to notify the government when you travel to
         | foreign countries, if you have contacts with foreigners of
         | certain nationalities when you are at conferences, you have to
         | be careful with your social media presence (many people with
         | clearances don't have one at all for this reason), no drug
         | usage even if it's legal in your state, etc. I don't like the
         | feeling of those kinds of restrictions so I said no to that
         | lifestyle.
        
         | justinzollars wrote:
         | Is this something you can apply for prior to getting a job?
        
           | dguest wrote:
           | Not that I've heard: it's not unusual to have the start of
           | the job delayed by a year or so while someone checks in with
           | everyone you've lived with / worked for for the last 10
           | years.
        
           | hakkoru wrote:
           | No, a company or organization must sponsor your clearance.
           | You cannot start the process on your own.
        
           | actinium226 wrote:
           | Certainly not. This is a government security clearance, not
           | an industry certification. Zero need to distribute these to
           | anyone who doesn't specifically require it.
           | 
           | That said, it stays with you afterwards, so you can take
           | another job that requires clearance at a different
           | organization/company (provided you have the right level of
           | clearance for the job). I think it expires after some time.
        
       | actinium226 wrote:
       | Just to add a couple places to your list that are similar to the
       | national labs:
       | 
       | Simons Foundation (offices in NYC and Berkeley) Allens Institute
       | (Seattle)
       | 
       | OpenAI would be more research focused as well
       | 
       | I have no affiliation with any of these nor have I worked at any
       | of them, but I'm also looking towards a career in scientific
       | research.
        
         | actinium226 wrote:
         | This thread is totally government/national lab heavy so I made
         | a separate one about private research institutes:
         | https://news.ycombinator.com/item?id=34417257
        
       | wanderingmind wrote:
       | As an aside, if you are an international PhD student, thinking of
       | doing postdoc, National labs provide much better pay and
       | work/life balance compared to Universities. However, you don't
       | get any priority or positive consideration in your Green Card
       | because you work in US National labs (EB1A). You will be treated
       | the same way as working/studying in a random foreign university.
        
       | funajoy wrote:
       | Worked at Sandia National Laboratories from 1990-2013 in ABQ/NM,
       | after 10 years in the Silicon Valley (CA Bay Area). I was
       | recruited as a PM in Nuclear Energy to lead joint, cost-shared
       | programs with the US Power Industry. Worked in tech. leadership
       | roles in many areas including Homeland security, Intelligence,
       | Nuclear Deterrence and Emergency Response Readiness, Global
       | Security and Leading a remote test site within a USAF Base. All
       | Missions of National Importance. Had opportunity to help shape
       | national policy and strategy, team with allies and work with some
       | of the smartest technical minds. Great R&D resources and funding.
       | It's world's largest engineering R&D organization with
       | currentannual budget of >$4 billion and workforce of ~15,000
       | professionals.
       | 
       | Having a MS or Ph. D. from top ten schools used to be requirement
       | for entry. Salary is competitive, with excellent benefits
       | including 10% 401K match since they no longer offer pension. No
       | stock options, profit-sharing and other tech industry perks.
       | However, excellent work-life balance, stability, working in
       | different tech. areas with best minds w/o having to relocate and
       | start over again, and wonderful quality of life, especially in NM
       | if you love outdoors.
       | 
       | Employees are not government employees. It's a government owned
       | contractor operated (GOCO) FFRDC, a non-profit. Like any big
       | organization, there is a fair amount of beaurocracy and people
       | issues to deal with. Having a team of former "A" students and
       | ranking them is not conducive to teamwork although for all large
       | projects or initiatives, it's a must. Multi-displinary teams
       | range in size from a few to 100's and are spread out in many
       | locations around the country. Opportunity to interact with
       | Wasington law makers and agency e ecutives. Recommend exploring
       | opportunities at Sandia.gov
        
       | dekhn wrote:
       | I worked at LBNL (in Berkeley) and it was great. It's like
       | academia but with no teaching responsibilities. Yes, the pay was
       | lower and the expectations were roughly the same as MAAA
       | (Microsoft-Amazon-Apple-Alphabet) but you didn't get fired if you
       | didn't meet expectations.
        
         | aksjdglkjlk wrote:
         | LBL is a special place. It's still managed by UC without the
         | involvement of any other orgs (aside from the DoE). It's
         | basically just another UC campus, but research oriented and
         | federally funded. The other labs can be quite different,
         | managed by LLCs that involve a few universities and some
         | private companies.
         | 
         | Things might have improved now that Bechtel is out of the
         | picture, but for many years LBL was hands down the best lab to
         | work for, purely because of the management situation.
        
           | dekhn wrote:
           | I wonder if the same thing can be said about Argonne (run by
           | U of Chicago)?
           | 
           | My manager at LBNL worked there because he was kicked out of
           | one of the other labs (run by Batelle or Bechtel or one of
           | those) for refusing to take a drug test. They said he'd fit
           | in well at LBL- and he did.
        
       | chemeril wrote:
       | Did some time at LANL as an R&D engineer in the non-global
       | security skunky areas, though wound up leaving for reasons not
       | pertaining to the work. Participated in several projects
       | involving Sandia and LLNL.
       | 
       | Pros: - Pay was excellent, especially for the area - Incredibly
       | beautiful country - Very interesting work - Infinite well of
       | taxpayer dollars for equipment and materials - The best job
       | security one can find - Crippling bureaucracy enforced a
       | remarkably safe work environment
       | 
       | Cons: - Crippling bureaucracy made it difficult to move quickly
       | and hit tight deadlines - Internal politics (intra-lab and inter-
       | lab) often adversely affected decision making and program success
       | - Living in a company town - An inability to remove demonstrably
       | problematic employees - A Q clearance limits certain
       | extracurricular activities
       | 
       | Personal experiences with LANL were all over the place and
       | highly, highly dependent on which group one works with. I was
       | very lucky to get in with a group of wonderful people and
       | immediate management that firewalled most adverse developments
       | from higher up the food chain. This is not a common experience
       | but organizational mobility is relatively free, so you can move
       | to work and groups that are attractive.
       | 
       | Worth noting for those coming from private industry: the national
       | labs are institutions first and foremost, not businesses.
       | Organizationally and operationally they exist in a very different
       | mindset and within very different value systems than FAANG-like
       | orgs. The adjustment can be a bit jarring.
       | 
       | My work at LANL will likely be the most interesting and most
       | fulfilling work I'll have done: every day was an adventure into
       | the unknown. The work/life balance was also excellent. If you're
       | a naturally curious person and have an inclination for basic
       | science I'd recommend taking a look at the labs. If you have
       | specific questions feel free to drop them here!
        
       | walrus01 wrote:
       | > US Department of Energy's national laboratory network
       | 
       | As someone who had all their personal information stolen in the
       | famously known OPM data breach, think twice about if you really
       | want to do the background check with SF-86 for a Secret or TS
       | clearance. It's a real pain in the ass. Even if you have an
       | absolutely squeaky clean record.
       | 
       | https://www.google.com/search?client=firefox-b-d&q=opm+data+...
        
       | mattpallissard wrote:
       | I worked at ANL, which didn't do any weapons research, and loved
       | it. Definitely had _a lot_ of bureaucracy so if you haven 't
       | worked enterprise be prepared. Things can move slow. I knew some
       | people at several other national labs and a few in DOD. I got the
       | sense that it was the same over there as well.
       | 
       | That said, it was hands down the most interesting place I've ever
       | worked. I don't think it would have been nearly as interesting if
       | I were a remote worker though. Half the fun was taking to the
       | "lifers" there. Learning the lore of old experiments and asking
       | them to show you around on lunch breaks.
        
       | evanb wrote:
       | I was a postdoc at Lawrence Livermore National Lab (LLNL) from
       | 2013-2016, working on computational particle physics (lattice
       | QCD). Pay was extremely high (for a postdoc) but not high by
       | comparison to other bay-area employers. What you get instead is
       | reliable job security, and some sense of civil service (as most
       | of the programs are federally funded).
       | 
       | LQCD is kind of funny because it doesn't YET have anything
       | practical to say about nuclear physics, which is what the lab
       | cares about, but it will someday. So I was pretty insulated from
       | all the weapons+complex integration stuff; my work was 'pure
       | research', which is not that common (though it is more common at
       | the postdoc level, which the lab views as a way of recruiting
       | talent). But unless you can find your own funding (usually from a
       | DOE grant), you're working on something that advances the lab's
       | programs. I can't bring myself to work on nuclear weapons, which
       | is why I didn't stay [there's a lot more to LLNL than that, of
       | course, but it's what my field funnels into, broadly speaking].
       | 
       | The computational expertise for HPC is really unparalleled,
       | especially at Livermore (and Oak Ridge, which I've only visited).
       | They're consistently pushing the envelope in terms of high-
       | performance machines which can address scientific questions that
       | require extremely tight coupling between computing resources,
       | rather than a cluster, and they have a lot of experimental
       | architectures and things like that. LLNL publishes a lot of open-
       | source software; if you've used a cluster in a scientific setting
       | you might be most familiar with SLURM or spack.
       | 
       | The day-to-day can be a bit surreal. At the defense labs people
       | with enormous machine guns thoroughly check your badge on the way
       | in. On your walk to the cafeteria you might pass a beach
       | volleyball court that's inside the superblock [an extra-high-
       | security area where they've got plutonium etc.], next to a
       | machine gun turret. Very few employers have teams that regularly
       | win SWAT competitions.
       | 
       | The food was fine. No luxuries like free snacks or anything else
       | I'd seen my tech-company friends enjoy. No dogs allowed. LLNL has
       | a lot of employee organizations for sports, charities, exercise,
       | etc. Transportation around the LLNL site is via sporadic shuttles
       | but more practically there's a bike share, which is just a bunch
       | of bikes you can leave anywhere (on the sidewalk / by a
       | building).
        
       | metalforever wrote:
       | I had a bad experience. I have top tech companies on my resume
       | and they made the role look cool and I was trying to slow down a
       | little . They instead put me into support . The codebase wasn't
       | in source control ; it was just a few scripts in any case . They
       | didn't have a sane deploy. The people that they had me working
       | under had been there for decades but were not good . They talked
       | down to me and did not give me tasks that aligned to my
       | experience. I thought I would ruin my career if I stayed so I
       | left. Much happier with a principal role where I'm at now .
        
       | aplsoftwaredev wrote:
       | I work as a software engineer at the JHU applied physics lab and
       | absolutely love my job.
       | 
       | * Pay is very comfortable to live on in the area
       | 
       | * The large majority of my teammates are self motivated and
       | driven which keeps me motivated and on my toes
       | 
       | * I get to constantly experiment with new tech, work on
       | prototypes, and pursue work I'm interested in.
       | 
       | * Almost all of my work is software development but it's rarely
       | pure software work. I'm almost always working with other SMEs and
       | helping them develop their ideas into code
       | 
       | If you're a curious, hard working person it (and I imagine other
       | UARCs) are great places to be
        
         | goldtownjac wrote:
         | I know a lot of people who work at APL (but not SWE), love it,
         | and have no plans to leave. Definitely seems like a great
         | option for someone in OP's situation.
        
       | xvedejas wrote:
       | I've known several people who worked at LANL, and they reported
       | you might sit around for six months or more waiting on
       | bureaucratic approval for your project. They all agreed: very
       | slow moving, lots of red tape, otherwise fine.
        
       | tinglymintyfrsh wrote:
       | Former neighbors had good things to say about Sandia. Albuquerque
       | isn't so great though.
        
       | scdlbx wrote:
       | The projects are more interesting and feel like they are actually
       | benefiting humanity, rather than making money for some random
       | company. Though there can be a lot more bureaucracy and friction.
       | Pay is certainly less, though sometimes the benefits can be
       | better.
       | 
       | There's also annoyances coming from political things, such as the
       | budget not being done on time so no one gets paid or there are
       | furlough days.
        
       | wedn3sday wrote:
       | I worked at LLNL for a little over 6 years, and recently took a
       | job at a "real" software company. Other peoples comments here are
       | completely valid so Im not going to rehash them all, but they all
       | seem to be focusing on the positive so here's a few take aways
       | from my time behind the curtain (taken with a grain of salt,
       | there was a reason I left):
       | 
       | 1. Extremely high bureaucracy, very poor facilities management.
       | Expect a terrible shared office in a temporary building with
       | asbestos. Dont expect to be reimbursed for your travel expenses
       | for like 3 months, and you're very likely going to have to travel
       | for conferences (i.e. fly to DC to make your sponsors happy).
       | 
       | 2. The old joke around LLNL was, "hey, you know how many people
       | work here? About half." Half the people you work with will be the
       | smartest most productive (and friendly!) people you've ever met,
       | the other half are also smart but have realized that their
       | productivity has no bearing on their advancement and so have
       | decided to not give a fuck. Since hiring is such a nightmare, its
       | very hard to get fired. From a program managers perspective,
       | they'd rather keep people around who massively underperform then
       | fire them since getting a replacement could take years. The
       | upside is that if you want to get paid to do nothing, this is an
       | amazing place to work!
       | 
       | 3. You dont need a clearance to get hired, although it helps, you
       | get to do the incredibly invasive FBI-agents-knocking-on-your-
       | moms-door clearance process after starting, and then again every
       | 5 years for the rest of your time there. The upside is that you
       | get to make really fun Qanon jokes with all your coworkers. Be
       | ready for random drug tests as this is a Federal facility and
       | Cali's pot laws have no affect. Eat a gummy at a party? kiss your
       | career goodbye.
       | 
       | 4. If you're cool working on nuclear weapons, you're set for
       | life. If you dont want to operate the gas chamber at Auschwitz
       | (how I see nuke people), then your funding will perpetually be in
       | danger, and you will likely spend more time chasing grants than
       | writing code. (Nearly) Everyone you work with will be totally OK
       | working on a tool with the express purpose of killing 100MM
       | people. I had friends/co-workers who couldnt talk to their
       | spouces about what they did during the day. The nuke people are
       | also very enthusiastically pro-america in a creepy way that
       | always set my teeth on edge. All diversity/inclusivity programs
       | here forcibly killed by the GOP, which leads me to:
       | 
       | 5. Politics affects everything. Government disfunction is
       | annoying enough, but if there's ever a fight over the federal
       | budget or a government lockdown expect it shutdown your work as
       | well. If you work on something politically sensitive (hello
       | climate program!) expect your funding to be on the chopping
       | block. Real shit, when Trump came into office my entire program
       | changed its name to exchange "climate" with "earth system
       | science" to try to run under the radar.
       | 
       | 6. (LLNL Specific) Livermore is very expensive (about the same as
       | the rest of the Bay, but still pretty expensive by anyones
       | measure) with none of the things that make the Bay nice. No BART
       | stop and massive traffic means more than 2 hour round trip to
       | Berkeley and back. Your Bay friends are not going to come out to
       | visit, and in terms of travel time Sacramento is closer than SF.
       | Unlike the rest of the Bay, its temperature is not regulated by
       | the ocean so it regularly hits 115 degrees for weeks on end
       | during the summer.
       | 
       | All that said, I still (mostly) enjoyed working there. The reason
       | I left is because during covid I moved to SoCal and after the
       | climate program got its funding reduced the HPC/Nuke people who
       | wanted to hire me onto their team wanted me to come back into the
       | office (there are certain terminals for accessing classified
       | material that are fixed in place and cant move).
        
       | oppanoppen wrote:
       | Have worked at Sandia for 12 years. Will probably retire in
       | another 20.
       | 
       | Pay is good. I make $145,000. That is low for Sandians with 12
       | years experience I think, but have had some atypically low points
       | in my Sandia career. You can make more in 'private industry'
       | (other defense contractors) even in Albuquerque but you will lose
       | work-life-balance
       | 
       | Benefits are very good. 3 weeks paid vacation. 2 weeks unpaid.
       | Flexible work schedule: normal hours, or 9/80, or 4/10. Generous
       | 401k match, plan supports roth mega backdoor, HDHP+HSA available.
       | Good WFH was slow to arrive, but corona fixed that. Nobody has
       | ever disturbed me on vacation or implied I should not take one.
       | 
       | Location is ok. Abq is high crime and NM is a poor state with
       | poor outcomes but it is very rugged beautiful. Don't knock LCOL,
       | it provides wonderful peace of mind, and the bad parts can be
       | easily avoided, but maybe you won't like it. Relative to NM, Los
       | Alamos is outlier with very good outcomes (crime nil, public
       | schools among best in nation), because it is a place that only
       | exists due to LANL. Sandia CA is option but while Sandia Abq pays
       | well for Abq, Sandia CA pays very bad for CA. Californians often
       | poached by FAANG.
       | 
       | Clearance means govt will look up your ass, and often. You must
       | report all 'meaningful' foreign interactions, including
       | friends/family. Investigations occur every five years. Random
       | drug tests: get a phone call that tells you to go to the medical
       | facility and piss today, or you're fired. 'Forgetting' to pick up
       | your phone only holds them off for so long. Some (rare)
       | clearances have worse requirements: must report all dual citizens
       | not just foreigners; random polygraphs; must ask permission to
       | leave the country; but again, these are rare. Easy to opt out of
       | such a clearance and it will hardly limit your opportunities at
       | all.
       | 
       | Project work varies. Nuclear is the mission but Sandia has
       | expanded to wider govt contracts. Nuclear weapons, non-nuclear
       | weapons/military, CIA/FBI/NSA partnerships, all are possible. If
       | this is against your morals, there are other options, but
       | probably better to work elsewhere. Would be like working for
       | Google while hating ads. Myself, I am okay with thise things. I
       | have not encountered govt abuses, nor any projects I consider
       | inherently immoral, but perhaps I am naive.
       | 
       | I have had good projects and bad projects. Sandia is broadly 10+
       | years behind the curve at software engineering. There are pockets
       | that are better, but eg version control is spotty in some places,
       | lots of crufty old codebases. I think a symptom of being
       | primarily an elec/mech/chem eng shop since the 50s. But there are
       | 12000 people and if you find the right department, combined with
       | the job benefits it is heaven. I have, and will stay till I
       | retire. It's the perfect job for me.
        
       | Huntsecker wrote:
       | tinfoil hat, but couldnt this be an easy way for foreign agencies
       | to easily find whos working in a sensitive industry
        
       | qooiii2 wrote:
       | I interned at Sandia Livermore and Los Alamos in college, then
       | worked at Sandia's main site for a few years before moving out to
       | the Bay Area to work in the more dynamic world of consumer
       | electronics.
       | 
       | The labs are not for everyone, but it's the perfect job for some.
       | If you want to work with fantastically smart people and don't
       | mind following a lot of arbitrary rules, it can be a lot of fun.
       | Most of my coworkers intended to spend their entire careers
       | there.
       | 
       | Just like anywhere else, a lot of the day-to-day experience
       | depends on the group you work with. In general, it's somewhere
       | between a university campus and a defense contractor, and the mix
       | is different for each project. The good part is that once you get
       | a security clearance and make some friends in other groups, you
       | can move around.
       | 
       | There might be some culture shock. Most employees have to be US
       | citizens, so the labs are probably less diverse places than you
       | might be used to. And you will really be hitting the brakes while
       | you wait for a security clearance.
       | 
       | I'd say look at the job postings and give it a try! It didn't end
       | up being for me, I don't regret the time I spent at the labs. And
       | it's tough to beat the work-life balance. You can't take a lot of
       | the work home, and most people take every other Friday off (9/80
       | schedule).
       | 
       | But do consider the location carefully. For example, Sandia and
       | Los Alamos are both huge and have a huge variety of projects, but
       | you're stuck in Albuquerque or Los Alamos which can be limiting
       | unless you really enjoy hiking.
        
       | chinchilla2020 wrote:
       | I'm not sure how you would end up on the fusion program as a
       | FAANG engineer.
       | 
       | Software engineering is vastly different than the physical
       | sciences (I've worked in both).
       | 
       | I worked a bit with the Sandia and some folks from national labs
       | on the corporate side in the energy industry. The only software
       | engineers I was aware of were ones that did IT work (Integrated
       | workday, salesforce, etc). The scientists and mech/chem/EE
       | engineers I met were the ones doing all the physical sciences
       | work.
        
       | ArtWomb wrote:
       | Fascinating thread. Love reading the oral histories. I heard some
       | tales of covered up nuclear accidents happening perilously close
       | to Manhattan at Brookhaven National Lab back in the day, and its
       | clear from the fusion results, government science is still the
       | largest research outlay. If you are comfortable with payscale,
       | lack of advance, essentially academic role in natsec context,
       | then its a dream job! I feel like the only people I know anymore
       | who have multi-decade careers for the same entity (DoE/PPPL) are
       | gov scientists ;)
        
       | imroot wrote:
       | I worked at LANL for a few years as a contractor, building out
       | their SCM tooling.
       | 
       | I loved every minute of it and, by proxy, got to see some really
       | really interesting projects and meet some really interesting
       | people.
        
       | kxyvr wrote:
       | I spent about a decade working for one of the NNSA labs. I think
       | they're a good place to work, but it's a very different work
       | environment than startup culture. Most of what I'm seeing in the
       | discussion is pretty accurate. I'll add a few things in case
       | they're deal breakers.
       | 
       | It's likely that you'll either be required to or pressured into
       | obtaining a security clearance. Have a look at Standard Form 86
       | (SF86) and see if you're fine with these kinds of questions.
       | Depending on how comfortable you are with this line of
       | questioning, do note there will be more digging especially if
       | you've lived abroad, had prior drug use, had prior legal issues,
       | or a variety of other items that pique their interest. Failure to
       | obtain a clearance is grounds for termination in most
       | circumstances.
       | 
       | A clearance comes with certain responsibilities. You will have to
       | report certain kinds of travel even if on vacation. You will have
       | to report certain kinds of contacts even if not work related. You
       | may not be allowed to freely publish even non-technical documents
       | or books without prior approval depending on the level of
       | clearance. There is a way to accommodate the lab in a way where
       | you can mostly live your life freely. Most of this is just
       | paperwork. However, you do give up some of your autonomy.
       | 
       | There is a proprietary innovation form they will push you to sign
       | that will assign to them all innovations even on your personal
       | time without company resources. Likely, this is not valid in
       | California. Not sure if this is currently negotiable. They were
       | pretty insistent in the past.
       | 
       | Pay for a starting Ph.D. was a little over $100k about 10 years
       | ago. It capped out for most people at around the $130-140k region
       | after 10 years of experience. Likely a bit different now, but not
       | tremendously so.
       | 
       | While it depends on the group, internal politics heavily favors
       | pedigree and degree level. Meaning, you are treated better if you
       | have a Ph.D. even if it's not really necessary for your position.
       | I'm not sure I would consider a job there with less than a
       | masters. Practically, this means you're more likely to be the PI
       | with a higher degree. More likely to be selected for promotion.
       | More likely to be able to move to management if that's of
       | interest. More likely to obtain internal research funding. To be
       | sure, I don't agree with this, but it's a reality of the culture.
       | 
       | Things like time card fraud are taken very seriously. Meaning, if
       | you work extra hours, you will be compensated for it either in
       | terms of flex time or pay with overtime. Management doesn't like
       | to pay overtime, so this means that you'll likely be out on time
       | most of the time.
       | 
       | It's a professional workplace where people come to work and then
       | leave. As such, no alcohol on the facility, no gaming tables, or
       | sleep cubicles or cutesy architecture. It's not that people are
       | completely serious the entire time, but they are professionals
       | and there is a pretty strict separation between what's considered
       | business activity and personal activity.
       | 
       | The research projects are great. There's easier access to grant
       | money than in academics and it provides the opportunity to work
       | in areas that are not necessarily commercially viable, but have
       | broader impact. Getting access to the money either means being
       | good at proposal writing, and having the right pedigree, or
       | making friends with someone who is and likes to farm out work to
       | you. When I say money here, you're not going to make anything
       | more on the grant. It means being able to buy out your time to
       | work on this particular project.
       | 
       | That's already probably too much, so I'll stop here. In short, I
       | think they're a great place to work if you can fit into the
       | culture with them. They tend to play by the book professionally
       | and tend to hold by their agreements with you. However, there is
       | a lot of paperwork and you will necessarily give up some of your
       | personal autonomy.
        
       | JackFr wrote:
       | I worked at National Center for Biotechnology Information at the
       | National Library of Medicine at the National Institutes of
       | Health. It was a nice entry level job, pay was OK, environment
       | was fantastic (low stress, extremely collegial), but without an
       | MD or a Phd (many had both) my path there was relatively limited.
       | 
       | One other thing, I was absolutely blown away by the number of
       | outrageously brilliant people I got to work with. Really some
       | next level people.
        
       | [deleted]
        
       | lanmthrwy wrote:
       | I've spent most of my career at them, with a bit of time in
       | industry in between time at the labs. (For context: me = ~20years
       | post Ph.D. in EECE, started career at one lab, later landed at a
       | second lab where I still reside.) Labs are awesome environments:
       | it's not too hard to get involved with projects that are pretty
       | cutting edge scientifically, which is exciting if you're a
       | science nerd like me. It's also nice not to have layers of
       | management over me dictating what I do - the scientists play a
       | major role in running the show, so you have much more control
       | over where you end up going. That said, they are an environment
       | where it's most easy to dictate research direction if you have a
       | PhD or in some cases, a masters + lots of experience.
       | 
       | The pay isn't competitive with the giant Silicon Valley
       | companies, and computing tends to be a little less bleeding-edge
       | than the other scientific domains. The only place computing is at
       | that bleeding edge is in the HPC world since the labs typically
       | have machines like nobody else, so there is a lot of research to
       | do in terms to utilizing them well and programming them.
       | 
       | The only other complaint I see for the tri-labs (SNL/LANL/LLNL)
       | is that pretty much everyone is expected to hold a Q clearance
       | (roughly equivalent to DoD TS + CNWDI). That can be an obstacle
       | for some people. Not a really difficult process - lots of
       | paperwork, interviews, patience while it goes through the system,
       | and then the periodic renewal process and occasional random drug
       | tests.
       | 
       | I personally love working at the labs and plan to stay for the
       | rest of my career. I don't optimize my career around maximization
       | of take-home $. For me, I want fair pay doing something I really
       | feel like I get excited about in an environment where my employer
       | treats me pretty well. The labs give me that.
        
       | UniverseHacker wrote:
       | I highly recommend it- the culture is much better than academia
       | or industry in my opinion, having experienced all three. You get
       | to work on big projects that directly target big problems in the
       | world, with the best equipment and facilities in the world. There
       | is less politics vs academia, and less careerism- people mostly
       | love what they are doing, and are excited about it. Generally
       | great benefits, pay, and work-life balance.
       | 
       | Also, the national labs model gives you a chance to work on big
       | teams with people from diverse backgrounds, which is a lot of
       | fun. For example, a software engineer may find themselves working
       | day to day with physicists, biologists, etc. and learning enough
       | about these fields over time to make novel contributions to them.
        
       | hypeatei wrote:
       | I know someone who works at LLNL and it sounds pretty typical but
       | he has to go on trips every so often that can span anywhere from
       | a week to a month for testing their designs and such.
        
       | nukenuke wrote:
       | I worked at LBNL as a research scientist and spun a startup out
       | of there. It was a great place to work in some respects, lots of
       | really smart people, awesome brainstorming, seeing Nobel prize
       | winners around. But if you like getting things done quickly it's
       | quite a challenge due the the bureaucracy (ex ordering simple
       | things could take an extra couple weeks going through lab
       | purchasing). I once described it to a friend who worked at a
       | FAANG company and they said "Oh so it's like working at a big
       | company but without the advantages of a big company".
       | 
       | Working at our startup almost feels like working at the lab (ie
       | we have scientists and are doing hard tech), but we can also move
       | fast and don't have the bureaucracy. So maybe consider working at
       | a hard tech startup with a heavy science base!
        
       | [deleted]
        
       | whalesalad wrote:
       | Reminds me that I started re-reading The Cuckoo's Egg over the
       | holiday break and need to keep up the habit.
        
       | mdmglr wrote:
       | Speaking from my experience. All comments here are my own.
       | 
       | - Exceptional work/life balance that you will not find anywhere
       | else.
       | 
       | - I started at 100k fresh out of masters program, at 5 YOE I was
       | 150k. Goes up steadily YoY.
       | 
       | - The labs operate like a variety of small businesses. This is
       | because there are many projects and funding sources from a
       | variety of customers.
       | 
       | - From what I hear some labs are super relaxed. Like Los Alamos.
       | The working environment is unlike any other.
       | 
       | - It is typical for teams to be in the same building but have no
       | idea about each other. Overlap and rework is common, but is
       | improving.
       | 
       | - Performance reviews depends on the lab and whatever review
       | process du jour HR wants. Where I am you are in competition with
       | your peers. Limited bonus money. So you'll need to go above and
       | beyond your peers to get it.
       | 
       | - day to day is: you work on one or more projects that last
       | anywhere from weeks to years and report to that projects
       | principle investigator. The PI will interface with the customer
       | and get funding.
       | 
       | - for software development we need to go through strict security
       | processes that dictate what libraries and dev tools we can use.
       | We use self hosted versions of popular tools like Mattermost and
       | Gitlab. No cloud. GovCloud is typically too expensive for most
       | customers unless your working on very well funded projects.
       | 
       | - Managers are hands off and mostly there to ensure corporate
       | compliance activities get done. E.g training, timesheet, perf
       | reviews,etc.
       | 
       | - High level of autonomy. So your expected to be knowledgeable in
       | your area, able to learn quickly and able to work with and
       | network with others to deliver results quickly to customers. For
       | example, you might be tasked with implementing an algo a staff
       | scientist came up with in C++ for an ARM board. It doesn't matter
       | if you haven't done that before. Your expected to learn C++, get
       | a demo out, and maybe you can pull in some colleagues you
       | previously worked with who are experts to help.
       | 
       | - There is a political structure in place and reputation is
       | important. While it may not be as intense as FAANG, and
       | underperforming is likely not to get you fired, if you
       | consistently underperform folks will remember and your reputation
       | will be permanently ruined. Which will result in not being picked
       | for more desirable projects. And likely shunned. I've seen it
       | happen a few times.
       | 
       | The lab was going to be a brief stint on my way to FAANG but will
       | likely turn into my career.
        
       | rqtwteye wrote:
       | I live in NM and meet quite a few people who work at Sandia Labs.
       | I am sure it's not perfect but they seem pretty content. It's
       | definitely not a high stress place. If anything, it's too slow
       | and relaxed for some people.
        
       | jvanderbot wrote:
       | I spent 7 years at JPL, which is run mostly like a national lab,
       | based on this comment
       | [https://news.ycombinator.com/item?id=34414812]
       | 
       | Is it a change of pace? Yes, but not in workload. You can easily
       | work yourself to death if you let yourself. And politics happens
       | everywhere -- usually in the form of missing the good projects.
       | But the promotion frenzy is minimized, there's a sense of greater
       | purpose in all the projects that is impossible to replicate
       | anywhere else, and the _technologists_ run everything. Managers
       | help connect, but they don 't typically determine your day to day
       | priorities. In that sense, you can continuously shop around for
       | good projects and teams without any formal change of position.
        
         | oso2k wrote:
         | I second this comment. I spent the first 8.5 years of my career
         | at JPL as well. My first project out of college was writing
         | testing support tools for the Mars Exploration Rovers (MER)
         | Project. I sat 10 feet from a small window into the Spacecraft
         | Assembly Facility (SAF Building 179). Got to watch the rovers
         | take their first steps. Got to do a ton of interesting things,
         | from Satellite Radio Software Stack Testing (QA), to Project
         | Planning Systems, to first steps into Cloud Computing. Got to
         | fill out forms that placed some of my work into the National
         | Archives.
         | 
         | Also don't forget there's other FFRDCs (https://en.wikipedia.or
         | g/wiki/Federally_funded_research_and_...) in the US.
        
       | mrpf1ster wrote:
       | I worked at Argonne National Lab for two years as a web developer
       | for a few projects (https://afleet.es.anl.gov/home/,
       | https://energyjustice.egs.anl.gov/).
       | 
       | The people I worked with were super smart in their fields, but
       | were pretty bad at writing code / handling data outside of Excel
       | so they usually hired interns to help with code-related stuff.
       | Some of the divisions had full-time software developer teams, but
       | I was the only software developer in mine.
       | 
       | The pace was extremely relaxed, deadlines were not tight at all.
       | 
       | I worked remotely, but came in to the office a few times a month.
       | The campus is beautiful, as it is right inside a nature preserve.
       | Everyone there is doing scientific work, so it feels like a real
       | scientific think tank atmosphere and I loved it.
        
       | throwawy_gfdh wrote:
       | Throwaway since I'm going to say some negative things.
       | 
       | Overall I agree with almost all the positive things people say in
       | other comments - the national labs have a lot of very smart and
       | kind people, there are interesting things (or at least
       | interesting ideas), and if you're at home at a university, you'll
       | find a lot of kindred spirits.
       | 
       | But I made the opposite move you're considering and quit my
       | national lab job and moved to FAANG. Why? Because I wasn't a
       | scientific superstar with a clear vision, and IMO my field
       | (applied math, computational science, etc.) seemed to asking (by
       | which I mean funding) people to do software engineering without
       | many engineers and at the same time still be academics:
       | scientists and mathematicians who spend a lot of time writing
       | grants and trying to publish papers, etc. This made me feel like
       | a liar, writing the grants, and a hack, writing the code. Not to
       | mention that like all academic-type and "interesting" jobs, you
       | are supposed to be happy with the idea that you're going to
       | expend a lot of your free time and energy, and perhaps not be
       | paid quite as much as you would if you weren't pursuing your
       | (supposed) passion.
       | 
       | Industry is for sure a whole other pile of bullshit, but don't
       | assume that the performance review and promotion stuff is any
       | more draining than the things you will have to do to get funding
       | (you'll likely either be spending a lot of time writing
       | proposals, or you'll be funded by weapons money). Don't make
       | trying to get away from money and politics your reason for
       | moving, though there are plenty of good reasons. Good luck!
        
       | eslaught wrote:
       | Experiences can vary a lot. For example, we're bootstrapping a CS
       | research group at SLAC; our goal is to do fundamental CS
       | research. Depending on where you land, that may or may not be
       | typical of your experience. A lot of labs are science-focused
       | (which they should be, but sometimes it comes at the cost of
       | awareness of the CS side).
       | 
       | (We're hiring by the way; a bit stale but [1] is still relevant.)
       | 
       | [1]: https://news.ycombinator.com/item?id=33424570
        
       | zhivota wrote:
       | I worked at Argonne as an intern, and then Idaho National Lab as
       | an intern and then as a full time research associate for a
       | further 18 months.
       | 
       | The pace is definitely much slower, and in fact you'll find a lot
       | of people just barely doing their part and kind of hiding in the
       | woodwork, so to speak, which annoys some people to see. You also
       | see that in big corps too of course.
       | 
       | I was given an enormous amount of responsibility for being right
       | out of college, which was basically just determined by my boss
       | who had won a grant to do some specific type of research, so he
       | pretty much decided how to spend the money. In retrospect with
       | more experience, I was underqualified for the job, but I had a
       | lot of fun and learned a ton.
       | 
       | My boss and his colleagues had to account for their time in
       | something like 15 minute increments, because their time was
       | billed out to their projects. Now, I don't think they actually
       | tracked their time that closely, they more realistically were
       | probably just billing time according to the money they had in
       | each pot to keep the projects going on time.
       | 
       | Which brings me to my next point - the labs as far as I could see
       | run as a form of contracting business. They compete for funding
       | from other agencies, like the military, DOE itself, BLM, etc., to
       | do either basic research or applied research. So grant-writing
       | and competition for the next grant is very important. Each
       | principal has to have many grants in the pipeline at all times to
       | make sure their funding doesn't dry up at an inopportune time.
       | This can be stressful I think for people, depending on your role
       | in this process and your personality.
       | 
       | In some cases you'd just be working for someone else who had
       | secured all the funding for many years, and in that case you'd
       | never have to worry about that aspect.
       | 
       | I see some other comments that not having a PhD means no one
       | listened to you... that wasn't my experience. My boss only had a
       | masters, I had a bachelors, and we pretty much did what we
       | pleased, and decided how to do it ourselves. I'm sure it totally
       | depends on the area you are in and your working group though. I
       | think our group was small enough that none of the PhD scientists
       | gave a damn what we were doing, plus my boss had gotten the
       | funding with his colleagues (all of whom were masters degrees, no
       | PhDs in the group at all), so no one could really say boo to
       | them.
       | 
       | I ended up leaving because if you don't at least get a masters
       | you get capped really quickly in what you can do. In my case they
       | didn't have a permanent role for me at all with a bachelors, I
       | was on some kind of extended internship of sorts. I decided to go
       | into industry instead of going for a higher degree.
        
       | 2OEH8eoCRo0 wrote:
       | FYI- there is also the US Digital Service. My interviewer was a
       | former SWE manager @ Microsoft who wanted to help people in a
       | "civic duty" type of manner.
       | 
       | https://usds.gov/
        
         | jhart99 wrote:
         | In addition to USDS(which has issues being under the White
         | House), there is 18F and PIF at GSA which can also be good
         | choices.
        
       | jlturner wrote:
       | My dad works in the Molecular Foundry division at LBNL / Lawrence
       | Berkeley Labs (Dept of Energy and UC Berkeley), and loves it. He
       | started there 40 years ago working in electron microscopy and
       | oversaw the transition to digital imaging (you'd be surprised how
       | much code they write). Good work life balance (he comes home for
       | lunch everyday), a pension (rare these days!). His favorite part
       | of the job is the revolving door of very smart people
       | using/visiting the lab and getting to interact with so many
       | ambitious (and not yet jaded) younger grad students from UC
       | Berkeley.
        
         | tcpekin wrote:
         | I am 99% sure I know him - shares your same first initial? Tell
         | him hello from me (TC Pekin), I always liked talking to him
         | during some down time, and the feeling was mutual. As a grad
         | student I always liked talking to him and hearing about all the
         | history and his time at NCEM!
        
           | N1H1L wrote:
           | Congrats on your Zeiss job!
        
       | kincl wrote:
       | Great thread! I noticed a bunch of the comments are from
       | Sandia/LLNL/LANL all of which are mostly focused on the National
       | Nuclear Security Administration side of the Department of Energy
       | which is focused on the various aspects of maintaining the
       | nuclear stockpile of the US.
       | 
       | I worked at Oak Ridge National Laboratory in High Performance
       | Computing and did not work with anything directly nuclear at all.
       | The HPC efforts of the DOE are under the Office of Science
       | (separate and at the same level as the NNSA) which is focused on
       | more wider scientific impact and application than just nuclear.
       | The Office of Science has a number of program offices that focus
       | on all different kinds of science from basic energy
       | sciences/physics to biological/environmental and scientific
       | computing (where HPC is funded in DOE).
       | 
       | I agree that the work/life balance is great and it is definitely
       | a slower pace than what you would find in industry. The lab
       | system is huge and there are plenty of opportunities but on the
       | Office of Science side I like to break it down between what I
       | think of as a research group and a user facility.
       | 
       | Working in a research group is much like academia, they mostly
       | require a PhD and from what I could tell performance is judged on
       | publication output. These folks also write grant proposals that
       | come from DOE program offices for funding their own research. In
       | some cases I have seen these groups employ non-PhDs to be
       | computational scientists and write code.
       | 
       | The user facilities are long-running projects funded by the DOE
       | at the labs to provide specific capabilities to researchers,
       | sometimes just for DOE scientists but a number of them are open
       | to scientific researchers all over the world. This is where I
       | have the most experience where I worked at ORNL's National Center
       | for Computational Sciences on the Oak Ridge Leadership Computing
       | Facility (OLCF). These projects are generally well funded and
       | have all kinds of interesting challenges to solve. For example,
       | the OLCF has consistently deployed the number one supercomputers
       | on the Top500 list and it offers those computational resources to
       | anyone through their allocation program INCITE which supports
       | many different computational modeling and simulation experiments.
       | Other examples of user facilities at ORNL are the Spallation
       | Neutron Source and the High Flux Isotope Reactor.
       | 
       | One thing I have noticed since moving from ORNL to industry is
       | that the sense of shared purpose does not extend as far in the
       | lab system as it does in company. What I mean is that with the
       | small research group and with a user facility like the OLCF there
       | is shared purpose with the people in those groups but it does not
       | go much beyond that. A lab is generally made up of lots of
       | different research groups and a few facilities but beyond the
       | drive for "Science!" there is not a lot of shared purpose or
       | collaboration at a macro level. The analogy I use is that a lab
       | is a bunch of small dinghy boats that are all generally moving in
       | a similar direction but a company is a single ship with a
       | specific purpose driving it forward.
       | 
       | Overall I loved my experience at ORNL, I learned so much working
       | with so many smart people and made friends that I will have for
       | life.
        
       | dlivingston wrote:
       | I worked at LANL [0] for five years and enjoyed the hell out of
       | it. I was a Research Technologist, which is basically an R&D
       | Software Engineer.
       | 
       | You will find a research group within a division to work for. For
       | example, mine was the Computational Earth Science group (since
       | renamed) within the Earth & Environmental Sciences division.
       | 
       | You will be working with a handful (3+) of research scientists as
       | their supporting engineer. On some projects, you may be doing
       | machine learning work in Julia. On others, you may be coding a
       | fluid dynamics simulation in FORTRAN or C++. On others, you might
       | be doing data analytics in Python. It's highly, highly variable,
       | depending strongly on the PIs you're working with, and can change
       | as frequently or infrequently as you wish (within reason).
       | 
       | Ultimately, I did the reverse: went from a DOE lab into a FAANG
       | company. My reasons are particular to me, but if you're at all
       | interested in a slower paced, more varied and collaborative
       | environment, you can't do much better than working for the labs.
       | 
       | For context, at LANL, I was making ~$100,000 / yr with 3 YOE
       | (circa 2019). This is in northern New Mexico, with such a low
       | cost of living that this amount of money goes about as far as
       | $150k+ up in the Boston area (where I am now).
       | 
       | [0]: https://www.lanl.gov/
        
         | ianai wrote:
         | It's much less affordable than it used to be. Lots of
         | transplants seem to be driving prices up.
        
           | dlivingston wrote:
           | If you're talking about housing and property prices, that's a
           | long-standing supply issue related to building a town on a
           | mesa...
        
             | ianai wrote:
             | That just makes it a factor which long term diminishes the
             | labs abilities and people. Consequences priced-in as it
             | were.
             | 
             | But the grocery stores and surrounding supportive
             | industries like medical/dental are also going to reflect
             | the towns population.
             | 
             | I do love Los Alamos but knowing what a large city is
             | capable of helps, too.
        
         | floren wrote:
         | LANL is also in an absurdly beautiful place, and Los Alamos is
         | a very pleasant little town.
        
           | jessriedel wrote:
           | Strong emphasis on "little". If you're married with kids, it
           | can be great. Much harder for single young adults, especially
           | men since the gender ratio at the lab is unbalanced. Most
           | people in town are employed by the lab or contractors for the
           | lab.
        
           | subsubzero wrote:
           | It is, I was just there this past weekend. The town itself is
           | very small, and like some have said its getting more pricey.
           | I can talk about this area for hours as I had a job offer
           | from a startup spun out of LANL a few years back, I ended up
           | not taking the job and sometimes regret it.
           | 
           | But for Los Alamos itself I would consider it on an "island"
           | so to speak as the surrounding communities(except santa fe)
           | are not great, see espanola. There are reservations around
           | the area and they have their own issues(see drugs alcohol etc
           | - the drive from the valley to Los Alamos requires you to
           | turn on daytime running lights due to many DUI's) but the
           | town is extremely friendly and safe just not alot to do if
           | its not outdoor related.
           | 
           | Like some have said some LANL employees commute in from Santa
           | Fe which is a nice town, it skews very old and rich(on the
           | north and east sides) and if you have a family is not
           | ideal(almost all the families live on the south side of
           | town). Overall the area has excellent food(northern new
           | mexican food is incredible!) and for outdoor enthusiasts it
           | really can't be beat. Home prices in Santa Fe have risen alot
           | in the past few years like most nice outdoor areas. But I
           | think you can't go wrong with the area if you don't mind the
           | few downsides.
        
           | dlivingston wrote:
           | Definitely. The town always gave me 'Stars Hollow' vibes, if
           | you've ever seen Gilmore Girl.
        
           | flatline wrote:
           | New Mexico can be a challenging place to live, Los Alamos
           | even moreso. Like someone else mentioned: married with kids?
           | Great place, but even then Los Alamos is isolated, it's a 45
           | minute drive to Santa Fe which itself is not huge. For the
           | national labs you will also need to maintain a clearance, and
           | the DOE is somewhat regressive in its policies around past
           | drug use and mental health treatment.
           | 
           | There is very little in the way of night life. There are
           | activities but this is a place where people live their whole
           | lives and you often have to know someone to get involved.
           | There is a paucity of health services. The airport is small
           | and has had its routes reduced dramatically over the last 15
           | years. Los Alamos real estate is dated and expensive.
           | 
           | On the other hand the state is beautiful and there are
           | endless outdoor activities.
        
         | ouid wrote:
         | >low cost of living
         | 
         | Except for the nightly drive to Santa Fe.
        
           | dlivingston wrote:
           | If you live there, sure. There is also Los Alamos, White
           | Rock, Jemez, Espanola, etc.
        
             | femto113 wrote:
             | It gets only a sliver of attention compared to Los Alamos
             | and Santa Fe, but Espanola is actually effectively another
             | LANL company town--the lab is the town's biggest employer
             | and a large portion of the lab's workforce lives there, so
             | e.g. sharing the commute up the hill would be easy. And
             | compared to either of the others (or any FAANG city) real
             | estate is cheap (like 3+BR 2000+sqft on half an acre for <
             | $500K).
        
       | q845712 wrote:
       | I interned at Sandia around '04-'05. That was a little bit before
       | the current "FAANG" thing we have now (IIRC amazon was turning
       | itself from a bookstore into an e-commerce startup, and google
       | was still not being evil) but certainly it was already less
       | glamorous than getting a Microsoft or even Apple internship.
       | 
       | My recollection is that the whole experience depended on which
       | group you were in, and mine was fortunately very chill. Smart,
       | friendly people who arrived and left more or less at the same
       | time every day. Lots of matrixing and loaning of people from
       | different orgs -- I had the feeling that if I were making a
       | career there I would wind up slowly drifting around between
       | projects.
       | 
       | The biggest surprise to naive me-in-my-early-20s was that
       | "Department of Energy" is a euphemism for "Department of Nukes."
       | Nuclear stockpile stewardship was a large portion of the activity
       | there, and so a lot of your colleagues will be people who are at
       | least vaguely comfortable with that.
       | 
       | There was a ton of "basic research" too -- some high-energy group
       | had a daily experiment that would deliver a "whomp" of a
       | shockwave around 3:15pm most afternoons, there was a room temp.
       | fusion group, lots of interest in assisted driving cars and
       | unmanned aerial vehicles... you just had to appreciate that all
       | the first applications of all this tech was going to be military.
       | 
       | Also the security clearances.... the joke was that the "L"
       | clearance stood for "Lavatory pass" because in our building until
       | you got one, you needed a line-of-sight escort at all times, even
       | in the bathroom. Even for the "L" the process was quite onerous,
       | and I understood that the 'Q' clearance held by nearly all full-
       | time staff was even more burdensome. I heard stories of people
       | waiting for their clearance getting stuck in rooms with nothing
       | to work on. One person in my group basically got sent offsite to
       | some "think tank" or something for several months while he waited
       | for his clearance - I only met him once the whole summer, at a
       | conference.
        
         | madcaptenor wrote:
         | _The biggest surprise to naive me-in-my-early-20s was that
         | "Department of Energy" is a euphemism for "Department of
         | Nukes."_
         | 
         | It also surprised Rick Perry when he got to be Secretary of
         | Energy (he thought it had to do with oil)
        
       | myself248 wrote:
       | Side question: How are these positions affected by things like
       | government shutdowns and other Washington bullshit?
        
         | Gh0stRAT wrote:
         | These days it seems like management tries to keep a buffer so
         | they can operate for a little while during a government
         | shutdown. That being said, there are a variety of funding
         | sources and it wouldn't be fair for some teams to be allowed to
         | keep working while others could not, so they'll typically shut
         | basically the whole lab down once they run out of extraordinary
         | budget measures.
         | 
         | Also, if there is an actual shutdown Congress usually
         | retroactively pays the federal employees who were furloughed
         | but it doesn't typically apply to contractors. (DOE labs are
         | staffed by contractors rather than government employees so that
         | they can pay more than the government pay scales)
        
         | ssully wrote:
         | It depends on how your projects/team is funded. I've been on
         | teams where a shutdown had a pretty immediate impact (people
         | had to be shuffled around to other projects in the short term).
         | I've also been on teams that get all their funding at the start
         | of fiscal, so a shutdown doesn't impact them until they need to
         | be funded again.
        
       | fryz wrote:
       | Haven't seen anyone mention a non-DOE lab, so figured I'd weigh
       | in.
       | 
       | I interned twice with MIT Lincoln Labs, which among other things,
       | helped build and deploy Radar for WWII which turned into
       | building/managing the technology for Air-Traffic Control, and
       | then turned towards space.
       | 
       | They are primarily a DOD-associated research lab (even located on
       | an US Air Force Base), and so most of the projects have some
       | military-oriented mission. Their mission is entrepreneurial-
       | minded (which I found cool), in that they do the "basic research"
       | and prototyping to prove viability and then the DOD turns over
       | the project to a contractor to make feasible.
       | 
       | While I was there I worked in their GeoIntelligence and Natural
       | Language groups, doing research which I'd ultimately come to
       | understand as being relevant for Project Maven (year 1) and PRISM
       | (year 2). While I'm sure as an intern my contributions weren't
       | directly related to or otherwise leveraged for these programs, in
       | hindsight it was clear that this was the bigger picture that the
       | work was contributing to. Take from this what you will.
       | 
       | Most of the anecdotes that I've read through in the comments
       | mirrors my experience. However, one thing I see missing was how
       | opportunity was "metered" out. Each group I was in was organized
       | like a research lab and the level of your academic progression
       | limited (or opened) your ability to get access to specific
       | projects/work. Their pay scale was also dictated based on this as
       | well. So if you have a BS, your ability to "move up", doesn't
       | exist, but it does if you have PhD.
       | 
       | Ultimately, I was given an offer to work there, but ended up
       | taking a SWE position in the Bay Area because I wasn't interested
       | in continuing my education and felt like my ability to have a
       | career progression at MITLL would have necessitated that.
        
         | neilv wrote:
         | The glass ceiling for non-PhDs sounds like a caste system, not
         | the meritocracy that the broader MIT-ish community sometimes
         | professes.
        
       | tcpekin wrote:
       | I worked as a grad student in LBL/UC Berkeley for 5 years. At the
       | time I didn't want to make a career of it, but if I had to go
       | back to a national lab, that might be the one. The culture was
       | department dependent, but at the Molecular Foundry was pretty
       | good. The campus is gorgeous, it's not isolated like other
       | national labs at all, pay is low compared to tech for the Bay
       | Area, but not pennies, and the conversations and people you can
       | meet are fantastic. Always lots of new faces with students and
       | visitors, but the core group that I interacted with were all very
       | kind, helpful, hardworking, and just fun to talk to about
       | science! I can definitely recommend it.
        
       | hprotagonist wrote:
       | Your experience will vary strongly depending on which PI you work
       | for.
       | 
       | Expect a very different work culture. If academia is cozy to you,
       | you'll fit in fine.
        
       | boringg wrote:
       | Always had a grass is greener mentality of the Labs but i suspect
       | there are certainly their own set of challenges to overcome. Also
       | i imagine if you are trying to level up competition is subtle but
       | fierce at the higher levels given that its zero sum for the
       | positions. I imagine multi year to decade long political
       | campaigns for top positions in the orgs.
       | 
       | Staff levels probably pretty good if your motivation isn't to
       | move upwards and focus on whatever your interests are.
       | 
       | These are my speculations.
        
       | floren wrote:
       | I worked for Sandia. Pay is pretty good by almost any standards
       | except FAANG. The glory days where every staff member got a real
       | office with a real door are over (shared offices are the norm)
       | but it's still a pretty decent work environment.
       | 
       | Things don't move fast, as another commenter said. In my area of
       | work, projects tended to last 1-3 years and you'd be on several
       | projects at any given time. In general, it is ICs rather than
       | managers who run the projects. Your manager might say "Bob over
       | in 9876 has a neat project that could use somebody like you, send
       | him an email if you're interested".
       | 
       | You have to acknowledge that the core mission of the DOE National
       | Labs is nuclear weapons. You might not ever come in contact with
       | the mission, but it is there. They have strong HPC programs--
       | because HPC as we know it is basically driven by the need to
       | simulate nuclear weapons. Some people have moral objections to
       | this, and that's fine!
       | 
       | I thought it was a good place to work, all in all.
       | 
       | Edit: I'd like to stress that probably the biggest advantage of
       | the labs is the opportunity for self-directed work. If you can
       | convince somebody (external sponsors, internal R&D funding
       | committees) to give you money, you can work on just about
       | anything. If you can't get funding of your own, you are still
       | more or less able to choose what you work on.
       | 
       | Your work environment will depend highly on which group you're
       | in. Some groups look like a university department without the
       | students: you work in the unclassified area, you publish papers,
       | you can even open-source software (with some effort). Other
       | people spend their whole day in a windowless SCIF working on very
       | sensitive stuff which they can never, ever discuss outside of a
       | SCIF -- but while their public visibility is nil, their impact is
       | arguably greater.
        
         | funajoy wrote:
         | Hacker News welcome | new | threads | past | comments | ask |
         | show | jobs | submit funajoy (1) | logout
         | 
         | Worked at Sandia National Laboratories from 1990-2013 in
         | ABQ/NM, after 10 years in the Silicon Valley (CA Bay Area). I
         | was recruited as a PM in Nuclear Energy to lead joint, cost-
         | shared programs with the US Power Industry. Worked in tech.
         | leadership roles in many areas including Homeland security,
         | Intelligence, Nuclear Deterrence and Emergency Response
         | Readiness, Global Security and Leading a remote test site
         | within a USAF Base. All Missions of National Importance. Had
         | opportunity to help shape national policy and strategy, team
         | with allies and work with some of the smartest technical minds.
         | Great R&D resources and funding. It's world's largest
         | engineering R&D organization with current annual budget of >$4
         | billion and a workforce of ~15,000 professionals.
         | 
         | Having a MS or Ph. D. from top ten schools used to be
         | requirement for entry. Salary is competitive, with excellent
         | benefits including 10% 401K match since they no longer offer
         | pension. No stock options, profit-sharing and other tech
         | industry perks. However, excellent work-life balance,
         | stability, working in different tech. areas with best minds w/o
         | having to relocate and start over again, and wonderful quality
         | of life, especially in NM if you love outdoors.
         | 
         | Employees are not government employees. It's a government owned
         | contractor operated (GOCO) FFRDC, a non-profit. Like any big
         | organization, there is a fair amount of beaurocracy and people
         | issues to deal with. Having a team of former "A" students and
         | ranking them is not conducive to teamwork although for all
         | large projects or initiatives, it's a must. Multi-displinary
         | teams range in size from a few to 100's and are spread out in
         | many locations around the country. Opportunity to interact with
         | Wasington lawmakers and agency e ecutives.
         | 
         | Recommend exploring opportunities at Sandia.gov
        
         | sytelus wrote:
         | Are you allowed to publish papers (for non-sensitive basic
         | science/CS)? Do you have to relocate?
        
         | scheme271 wrote:
         | Not all DOE national labs do weapons stuff. For example,
         | Fermilab (FNAL) is pretty much all open science research. FNAL
         | doesn't even have fences and the guard booth just checks to
         | make sure you have a drivers license if you're driving. Argonne
         | and Brookhaven are a bit more on the defensey side but not as
         | much as say oak ridge or sandia.
        
         | kbarros wrote:
         | I'm a computational physicist at Los Alamos and would echo
         | these sentiments.
         | 
         | Note that there are two main types of DOE labs: NNSA (Sandia,
         | Los Alamos, Livermore) and Office of Science (Brookhaven,
         | Berkeley, Oak Ridge, Argonne, ...). Although the former is more
         | focused on "nuclear weapons stockpile stewardship", there is
         | still much basic science at all DOE labs, especially where
         | computer science meets physics and other domain sciences.
         | 
         | Perhaps relevant to HN, I would mention the Applied Computer
         | Science group at Los Alamos, which is in hiring mode
         | (https://www.lanl.gov/org/ddste/aldsc/computer-
         | computational-...). Besides supporting computational physicists
         | in code development efforts, this group does a variety of
         | researchy things like designing programming model, doing
         | compiler development, building ML models, especially with an
         | eye towards large scale scientific computing. The pay at a DOE
         | lab is less than FAANG (PhD student interns might be around
         | $80k/yr and starting staff scientists maybe $130k/yr), but the
         | tradeoff for some people would be the research-flavor of the
         | work, and the flexibility. Many of the LANL codes being
         | developed are open source, for example. Other DOE labs have
         | similar computer science divisions. For example, Oak Ridge,
         | Argonne, and Berkeley all have "leadership computing"
         | facilities.
        
         | screwturner68 wrote:
         | I've worked at a few DOE sites albeit as a consultant not an
         | FTE, Fermi, Los Alamos and a couple others. You are correct
         | that the work is interesting and a like academic atmosphere, it
         | was the academic atmosphere part that I found off putting.
         | Where I worked there was very much a hierarchy and if you
         | didn't have a PHD your opinion didn't matter much, you just did
         | what you were told. Having a couple of decades of experience
         | and being brought in to spin them up on their system and being
         | talked down to on a daily basis like I was a Sophomore in
         | college was really annoying -that said I'm a consultant so I
         | get paid to be annoyed by the people who hire me. Aside from
         | that I liked working there, the tech was cool, as far as the
         | moral issues I don't ask nor do I want to know about what I'm
         | working on -I don't have a need to know.
        
           | neilv wrote:
           | Sounds similar to what I mentioned in comments a couple
           | months ago, about science organizations:
           | https://news.ycombinator.com/item?id=33673259
           | 
           | (I'm not familiar with DoE sites. And I actually had a good
           | experience (other than the pay), as a high-end federal
           | consultant doing challenging technical work, reporting to
           | operations research PhDs at the Director level, who respected
           | what I could do. Where I've seen and heard of problems is
           | other places.)
        
           | randcraw wrote:
           | Having worked in R&D at a big pharma for 17 years and in
           | university HPC groups for 10, I echo these comments. I've
           | found that corporate R&D is essentially investigative (rather
           | than production driven), and as an IC computational scientist
           | (or data scientist) you will move from project to project
           | over the years, usually working solo 95% of the time. You
           | will have some opportunity to propose projects, especially in
           | partnership with scientists (in pharma, those are chemists or
           | biologists or work to improve manufacturing). But without a
           | PhD, advancement along the technical track will be limited.
           | 
           | If you're not embedded in a science R&D group, you will be
           | lumped in with general IT staff where tech support, database
           | management, or software development of products are prized by
           | management, but tackling individual R&D questions is not
           | (though it is tolerated by IT brass since they know
           | investigation is critical to finding & improving drugs).
           | 
           | I found the same limitations when I worked in R&D-based
           | military contracting (or for US gov't FFRDCs for various
           | agencies). There it's more important to develop a strong
           | relationship with the gov't client, irrespective of the
           | academic degree you have.
        
         | rcpt wrote:
         | My friends who worked there say that the job was great but
         | living in a remote science outpost made dating unbelievably
         | difficult
        
           | closeparen wrote:
           | You'd think there would be plenty of pretty young Russian and
           | Chinese women looking to meet handsome American nuclear
           | scientists.
        
           | kbarros wrote:
           | The town of Los Alamos is beautiful, but it's small. It can
           | be a great place if you already a have family that enjoys
           | outdoors activities. Many people prefer to commute from Santa
           | Fe (45 min drive with mountain views, negligible traffic).
           | Certain groups allow a flexible hybrid home/office working
           | mode.
           | 
           | https://www.usnews.com/news/healthiest-
           | communities/articles/...
        
             | neltnerb wrote:
             | I was only a guest scientist at LBL, but LBL is practically
             | in Berkeley.
             | 
             | But yeah, generally I assume this is true, though you may
             | just find that you need a car and that if you have one it's
             | not so bad. For instance, NREL (where I was also briefly a
             | guest scientist) is in an incredibly gorgeous area near
             | Golden which isn't super small but everything is still very
             | spread out and it'd take a while to walk anywhere for sure.
             | 
             | I guess it also depends on your definition of "middle of
             | nowhere" I suppose. Golden is hardly Oakland, but I am
             | pretty sure I could find people to date as long as I
             | included Denver... and if you have a car in Colorado you
             | will find that Denver is considered close to a lot of
             | things you might not at first consider it close to. It's
             | _only_ an hour drive from Boulder (where I lived and drove
             | to NREL as needed).
        
           | screwturner68 wrote:
           | If you are talking about Los Alamos you can commute in from
           | Santa Fe, it wasn't bad, about 45 minuets each way. In my
           | opinion it's kind of expensive for being in the middle of
           | nowhere, you don't get much bang for your buck house-wise.
           | Social wise I couldn't imagine living in LA for an extended
           | period of time if I were single.
        
             | gautamcgoel wrote:
             | Took me a moment to realize that you meant LA = Los Alamos,
             | not LA = Los Angeles :)
        
         | gorgonical wrote:
         | I did an internship at Sandia this past summer, specifically on
         | the HPC programs. Being a PhD student, the pay for that
         | internship was excellent, and the cost of living in ABQ is low.
         | I lived somewhere that I could bike to my office. My manager
         | was excellent and a technical staff member and really
         | understood how to manage other technical staff; no pointless
         | meetings, hands-off, and tried at every turn to shield us from
         | boring administrative non-sense.
         | 
         | The work was interesting, though my experience was very skewed
         | as I was working on a solo project. I met weekly with my
         | "mentor" to discuss where I was and if I needed support, but I
         | was working nearly entirely solo. Like the parent comment here
         | says, from what I learned the norm is that you work on many
         | projects -- my mentor certainly was. Even in the department
         | meetings it was clear that while we were unified under a
         | general theme, each person in the department was working on
         | their own, many projects.
         | 
         | My work was entirely unclassified, and my understanding is that
         | most of the people in my department worked on projects with
         | similar levels of classification. My office building looked
         | like every other building and infosec and opsec requirements
         | were pretty mild; wear your badge, don't photograph things,
         | don't tell people any specifics about what you do.
         | 
         | I was offered to stay on as a full-time intern as part of the
         | hiring pipeline and if it weren't for that it's in ABQ I would
         | have strongly considered it -- the work was very interesting
         | and also like the parent comment says, you are largely in
         | control of what you do there. It's a lab first, not a defense
         | weapons company, so research is the name of the game for the
         | department I worked in.
        
         | funajoy wrote:
         | Hacker News welcome | new | threads | past | comments | ask |
         | show | jobs | submit funajoy (1) | logout
         | 
         | Worked at Sandia National Laboratories from 1990-2013 in
         | ABQ/NM, after 10 years in the Silicon Valley (CA Bay Area). I
         | was recruited as a PM in Nuclear Energy to lead joint, cost-
         | shared programs with the US Power Industry. Worked in tech.
         | leadership roles in many areas including Homeland security,
         | Intelligence, Nuclear Deterrence and Emergency Response
         | Readiness, Global Security and Leading a remote test site
         | within a USAF Base. All Missions of National Importance. Had
         | opportunity to help shape national policy and strategy, team
         | with allies and work with some of the smartest technical minds.
         | Great R&D resources and funding. It's world's largest
         | engineering R&D organization with currentannual budget of >$4
         | billion and workforce of ~15,000 professionals. Having a MS or
         | Ph. D. from top ten schools used to be requirement for entry.
         | Salary is competitive, with excellent benefits including 10%
         | 401K match since they no longer offer pension. No stock
         | options, profit-sharing and other tech industry perks. However,
         | excellent work-life balance, stability, working in different
         | tech. areas with best minds w/o having to relocate and start
         | over again, and wonderful quality of life, especially in NM if
         | you love outdoors.
         | 
         | Employees are not government employees. It's a government owned
         | contractor operated (GOCO) FFRDC, a non-profit. Like any big
         | organization, there is a fair amount of beaurocracy and people
         | issues to deal with. Having a team of former "A" students and
         | ranking them is not conducive to teamwork although for all
         | large projects or initiatives, it's a must. Multi-displinary
         | teams range in size from a few to 100's and are spread out in
         | many locations around the country. Opportunity to interact with
         | Wasington law makers and agency e ecutives. Recommend exploring
         | opportunities at Sandia.gov
        
         | dqpb wrote:
         | Is there a lot of work left in nuclear weapons?
        
           | exmadscientist wrote:
           | Plenty! Among other things, they literally decay over time,
           | and that's fundamental to and completely inseparable from the
           | way they work. Plus for some reason people always want them
           | to be smaller and cheaper and more reliable.
        
             | floren wrote:
             | Plus, we're not allowed to actually blow them up to test
             | any more, so they do all that in simulation on the world's
             | biggest supercomputers. This leads to work in HPC system
             | software, kernels, compilers, programming languages, HPC
             | libraries, and so on.
        
               | xxpor wrote:
               | ZFS-on-Linux (now OpenZFS) comes out of LLNL, for
               | example.
        
               | Barrin92 wrote:
               | probabl a lot of that stuff is very sensitive but are
               | there any good books people can recommend on that type of
               | work at the intersection of programming/ (nuclear)
               | weapons development?
        
         | MadVikingGod wrote:
         | I worked, as a civilian, for part of the R&D arm of the Navy,
         | but it has a very similar feel to what you have posted. I would
         | 100% agree with everything you have said, except the pay.
         | 
         | One thing I will add is getting tools and resources to do the
         | work you want to do can be a challenge. There can be times when
         | getting through the process to buy a $75 multimeter can be more
         | difficult then the $16,000 signal generator.
        
           | sthu11182 wrote:
           | I also worked as a civilian for the Navy as an EE (b.s.). I
           | think my pay was ~50k (in the mid 2000s). I don't know about
           | the national labs, but my experience working for the navy was
           | that funding was a fight, especially everything being
           | siphoned off for the Afghanistan war. The equipment was
           | pretty up to date (the computers were not the latest, but
           | decent), the furniture was a mismatch collection of liquated
           | stuff, lab reports were published to a confidential library,
           | and we were on flex time, which made the hours great.
        
       | bargle0 wrote:
       | There are also FFRDCs and other similar organizations outside of
       | the DOE. For example, JPL is a NASA FFRDC.
        
       | indigochill wrote:
       | I haven't, but outside the DoE (but still in national lab land)
       | my brother worked at APL and I interviewed there (after hearing
       | all the praise he had for it). I loved interviewing with them.
       | Everyone, even when they severely outclassed my own education
       | level (I have a bachelor's in journalism and am a self-taught
       | software engineer, and I was talking to a couple of PhDs with
       | many years of experience), were super personable, humble, and
       | passionate about their area of expertise. They made me feel like
       | I was already part of their team even when I was just
       | interviewing (they even went out of their way to make
       | international interviewing work since I live abroad). If they'd
       | made me an offer, I almost certainly would have taken it. Great
       | people and by far the best interviewing experience I've ever had
       | (though in retrospect, interviewing for a position requiring
       | clearance while living abroad was probably an uphill battle).
       | 
       | As others have said, the work was highly self-directed. As for
       | the need for software engineers, it was definitely there
       | according to my brother. The scientists he worked with were
       | capable in their field, but they needed someone capable of
       | translating their models into something that would execute on a
       | computer. I don't know what exactly he worked on, of course, but
       | he's an ML specialist and was pretty interested in CUDA
       | programming around that time, so maybe that's a clue what kind of
       | skills he was applying.
       | 
       | Anyway, maybe something to check out similar to the DoE network.
        
       | wpasc wrote:
       | I'm not sure my response qualifies, but I see no other responses
       | yet so:
       | 
       | I interned in high school at Brookhaven National Lab working on a
       | team that analyzed STAR (Solenoidal Tracker at RHIC) data from
       | RHIC (Relativistic Heavy Ion Collider). I didn't contribute all
       | that much as a high school intern but the program director said
       | at the end that he liked the high school program because he
       | wanted to help funnel and bring people back to help build up the
       | labs.
       | 
       | My experience was that everyone there was extremely smart, but
       | all post-doc and top scientists in their field (the team I worked
       | on was looking for Anti-Alpha particles from gold-gold particle
       | collisions that also helped create Quark Gluon Plasma). So I'm
       | not sure their relative need for regular software engineers.
       | 
       | In terms of bureaucracy, you're still working for the government.
       | The scientists all complained about the layers of government
       | bureaucracy but were mostly okay with it. High-tier science moves
       | at a pretty slow pace; coming from a tech background you might
       | not be used to the slow pace around the actual physical
       | construction of some of these devices, let alone the fund-
       | seeking, approvals, testing, runs, and data collection. and
       | 33-50% is a hopeful estimate. Let's say one is a 500k a year
       | senior/staff SWE at FAANG. at a similar level of experience,
       | one's pay would be very lucky to break 150k.
       | 
       | So fascinating science, layers of bureaucracy, slow moving stuff,
       | PhD's in their fields, and reduced pay. Again I was only a high
       | school intern, but I spoke with the scientists about their
       | experiences so take my recollection with massive salt. I walked
       | away from the summer fascinated by the work and I had a love of
       | physics at the time; but I also left (this was 2010 IIRC?)
       | watching the world of tech explode at a massive pace and thought
       | that I didn't like physics enough ( I had spent my junior/senior
       | year of high school doing a capstone project on theoretical
       | physics and having taken a lot of physics classes). When I went
       | to college the next year, I tried a few engineering courses, and
       | switched to CS. I'm glad I made the switch.
        
         | AlotOfReading wrote:
         | There's a pretty decent need for software engineers at national
         | labs and they occasionally do some cool stuff. If you've used
         | ZFS on Linux for example, you've used something produced by
         | LLNL. They do some pretty massive software projects and have a
         | huge number of software testers. The ones I've interacted with
         | were pretty exceptional.
        
       | devoutsalsa wrote:
       | What are the qualifications to get a software job at a national
       | lab?
        
         | wedn3sday wrote:
         | From what I remember from some old all-hands slides, in the R&D
         | groups (so not facilities, not administration etc) there was a
         | pretty even breakdown of 1/3 PhD, 1/3 MSc, 1/3 BS, and like 2%
         | with no degree.
        
         | shagie wrote:
         | Likely depends on the lab and the project.
         | 
         | For example, here's a software developer at Idaho National Lab
         | -
         | https://inl.taleo.net/careersection/inl_external/jobdetail.f...
         | which is bachelors + 5 years professional.
         | 
         | Same lab, Cybersecurity Researcher
         | https://inl.taleo.net/careersection/inl_external/jobdetail.f...
         | which has bachelors + 0-2 years.
         | 
         | Over at LLNL Embedded Software Developer
         | https://www.llnl.gov/join-our-team/careers/find-your-job/all...
         | is "just need a bachelors"
         | 
         | while Software Developer https://www.llnl.gov/join-our-
         | team/careers/find-your-job/all... asks for a masters.
        
       | altintx wrote:
       | I've personally done both Sandia (2004-05) and LANL (2006-10).
       | I'd do LANL again, but Sandia was very political with dull work.
       | Interesting ideas, but dull work. So much bikeshedding. So slow
       | moving.
       | 
       | I did workflow management systems, environmental controls in
       | labs, and lightning prediction software.
        
       | elil17 wrote:
       | I have heard universally positive things from my friends working
       | at NREL, Oak Ridge, and Argonne.
       | 
       | You might also want to check out the US Digital Service, which
       | might be more aligned with traditional SWE skills.
        
       | seanlane wrote:
       | It's been a minute since I was an intern there, but haven't seen
       | this one mentioned yet. I spent a summer at Pacific Northwest
       | National Laboratory (PNNL, https://pnnl.gov) and really enjoyed
       | my time. It seemed like compensation was pretty good, especially
       | considering the cost of living in the area. There were a number
       | of times when we needed some help from a subject matter expert,
       | and we could go down the hall or to another building and speak
       | with someone who recently published on the topic. There was a lot
       | of interesting work going on, from national security issues to
       | storing nuclear waste, etc.
       | 
       | The campus there was also different than many other national lab
       | campuses in that it's an open campus and doesn't have the
       | military entrances that others have. It felt like the culture was
       | much more laid back than the FAANG and other corporate cultures
       | that OP mentioned, but perhaps more bureaucratic as well. Again,
       | I was an intern, so didn't have much visibility into that aspect.
       | Overall, definitely a positive experience and I could see myself
       | there if things lined up right.
        
       | throwaway84592 wrote:
       | I've worked at Sandia Labs as a software developer my entire
       | career, so I can't compare to FAANG or SV in general. Obviously I
       | like it, or I wouldn't have been here for 20 years.
       | 
       | Their job classification system is such that you will want a
       | Masters degree. I joined with a BS in CS, and until I got my MS,
       | I was categorized as a technician and was paid the same as say
       | someone who soldered and assembled electronics - just over half
       | of what someone with masters in CS would get with simular
       | rankings. I've heard it has improved since then, but there is
       | still stong bias towards those with a masters. For many
       | engineering jobs this makes sense, but it is out-of-touch for
       | computer programmers and security researchers.
       | 
       | Apart from that while the pay is less than SV, all the labs
       | except LLNL are in parts of the country with much lower cost of
       | living as well, so the pay is pretty darn good for the area IMO.
       | Benifits are good, but not exceptional. Work-life balance is
       | great. I've hand a handfull of month-or-two long crunches in a 20
       | year career where I had to work 60 hours a week. The rest of the
       | time I work my normal 40 hour schedule and go home. They have
       | standard, 9-80, and 4-10 schedules as options (which nearly all
       | managers will approve). After years of having every other Friday
       | off it would be hard for me to go back to a normal schedule.
       | 
       | The actual work varies wildly with the project you are on.
       | Nuclear Weapons work is extremely slow and process heavy as you
       | might imagine, others are more nimble. Projects I've been on have
       | varied from solo development writing software for the engineering
       | next office over, to small agile teams on quarterly releases, to
       | 5-year waterfall development cycles which a huge team. I've done
       | everything from microcontroller software for sensor systems,
       | realtime streaming data processing, desktop data analysis
       | software, web tools for managing data stores, to pure algorithm
       | research. And that is just a small sample of projects going not
       | even touching supercomputer simulation or security work, that I
       | have no experience with. I feel like I have had a great balance
       | of interesting and stimulating technical work and necessary grunt
       | work. Needless to say it is hard give a single "this is what
       | working at the labs is like", and individual experiences will
       | vary.
       | 
       | There are some differences from industry that are independent of
       | the project, particularly around security. It is not uncommon for
       | software development to be done at the unclassified Official Use
       | Only level, but (production and test) data to be at the
       | classified level, which is done on separate networks (or stand-
       | alone computers). Moving between the two environments can be a
       | time sink. Getting approval to use third-party libraries on
       | classified systems can be a very slow process (weeks at a
       | minimum) depending on the network. If the generic security plans
       | won't work for your project developing a custom one can take the
       | better part of a year. There are many security processes for
       | which I completely respect the purpose, but am flabbergasted at
       | the inefficiency of the execution. Contributing patches back to
       | open-source projects is painful enough that it is rarely done.
       | There is some third-party software that is prohibited (like
       | JetBrains due to connections with Russia), and cloud based tools
       | (on the public internet) are obviously not allowed. You need to
       | be constantly mindful of what you type/say to maintain OPSEC and
       | avoid leaking classified onto OUO systems, or leaking OUO to
       | friends and family.
       | 
       | They are allowing WFH now, but most managers for most jobs will
       | want you start on-site to help acclimate to the security culture,
       | and to live in town to be able to come on-site to work in the
       | classified environments when needed. You will need to apply for a
       | security clearance once hired, and some projects are better than
       | others at finding meaningful work for you to do while waiting for
       | the security clearance to be granted.
       | 
       | As far as ethics go, on one hand you won't be asked to write
       | dark-pattern advertising-driven manipulative spyware. On the
       | other, most work will be related to defense applications directly
       | or indirectly. There are some projects related strictly to energy
       | generation and power-grid security and the like, but they are the
       | exception. The best way to advance your career in the Labs is to
       | move around between departments every several years, so you will
       | be limited in your options to do that if you have reservations
       | about defense work.
        
       | secabeen wrote:
       | The national labs are pretty good, as is Research Facilitation at
       | most of the R1 research universities. You might find good leads
       | through the Campus Research Computing Consortium. We're still in
       | the early days of developing these positions, but it is starting
       | to snowgball.
       | 
       | https://carcc.org/
        
       ___________________________________________________________________
       (page generated 2023-01-17 23:00 UTC)