[HN Gopher] Fixing for loops in Go 1.22
___________________________________________________________________
Fixing for loops in Go 1.22
Author : todsacerdoti
Score : 278 points
Date : 2023-09-19 19:34 UTC (3 hours ago)
(HTM) web link (go.dev)
(TXT) w3m dump (go.dev)
| assbuttbuttass wrote:
| for _, informer := range c.informerMap { informer :=
| informer go informer.Run(stopCh) }
| for _, a := range alarms { a := a go
| a.Monitor(b) }
|
| Not sure what the difference could be, but let me take a guess.
| In one case, the loop variable is a pointer, and in the other
| case a value. The method call uses a pointer receiver, so in the
| value case the compiler automatically inserts a reference to the
| receiver?
| Spiwux wrote:
| I have such a love-hate relationship with this language. I use it
| professionally every single day, and every single day there are
| moments when I think to myself "this could be solved much more
| elegantly in language X" or "I wish Go had this feature."
|
| Then again I also can't deny that the lack of ""advanced""
| features forces you to keep your code simple, which makes reading
| easier. So while I hate writing Go, I like reading unfamiliar Go
| code due to a distinct lack of magic. Go code always clearly
| spells out what it does.
| bvinc wrote:
| "Love-hate relationship" were the exact words that I used when
| I used go professionally every day.
|
| I could complain all day about things the language does
| obviously wrong, often in the name of simplicity. But after all
| my complaints I still admit it's a very good choice for certain
| kinds of software and software companies.
| [deleted]
| JyB wrote:
| The first point cannot bother you after you've correctly
| realized your second point. The more empathy you have for your
| future-self or your peers, the clearer it becomes.
| jamespwilliams wrote:
| Go is the worst language, except all the others
| noelwelsh wrote:
| This was also an issue in Javascript (e.g.
| https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...)
|
| It's somewhat amusing to see Go rediscover old ideas in
| programming language theory, given the stance against PLT that
| the Go developers took in the early years of the language.
| stouset wrote:
| The entire story of go seems to be learning through repeating
| the same mistakes as other languages, one at a time.
|
| Nil being another big one. Even more impressively they doubled
| down on this mistake with _typed_ nils. Even if you explicitly
| do a comparison with nil you can still shoot yourself in the
| foot because it was a different nil than the one you compared
| against.
| TwentyPosts wrote:
| Wait what? Do you have an example or more information on
| this?
| Thaxll wrote:
| op is confused with nil interface.
| gwd wrote:
| Basic example: func foo() *bar {
| // ... if something_wrong { return
| nil; } } var x
| interface{} x = bar() if
| x != nil { // Dereference x }
|
| This will crash if `foo()` returns nil, because it's
| checking if `x == interface{}(nil)`, which is false. What
| you wanted to check was whether `x == *bar{nil}` or one of
| the other nil types that implements the interface; which
| must be done with `reflect.ValueOf(x).IsNil()`.
|
| https://github.com/golang/go/issues/30865
| Spiwux wrote:
| e.g. var typeA Interface = (*TypeA)(nil)
| println(typeA == nil) // false
| println(typeA == (*TypeA)(nil)) // true
|
| Yes really https://go.dev/play/p/sz44kJW8OuT
| akira2501 wrote:
| Programming languages are an exercise in compromise, not pure
| application of theory. Well, except maybe Haskell and it's ilk,
| but this should be your expectation of most languages and
| generally not a surprise.
| noelwelsh wrote:
| There's no need for compromise in this case. This issue is
| something I learned about in the late 90s / early 2000s when
| I started reading about programming language theory. It's
| found in introductory textbooks.
| FridgeSeal wrote:
| I think OP's point is that given the go devs casual disregard
| for every development in PL theory and design over the last
| 30 years, it's amusing to watch them rediscover half the
| issues from scratch.
| skybrian wrote:
| This meme is commonly repeated on HN, but I think it's
| inaccurate or at least greatly exaggerated.
|
| There's nothing _casual_ about Go 's approach to language
| design, and I haven't seen any evidence that they're
| _unaware_ of what other languages do.
|
| I also haven't seen much criticism of languages other than
| C++ or Java.
| FridgeSeal wrote:
| Sum/Product types. Generics/type-parameters. Bizarre
| handling of nil in places. Error handling that's like
| some deliberately crippled version of a Result<T,E>. The
| absolutely unhinged decision about zero-value-defaults
| for types. I'm sure other people can think of some more,
| but that's the ones I can think of off the top of my
| head.
|
| Non PL theory but related: incorrect implementation of
| monotonic clocks, and then refusing to fix it because
| "just use google smear time bro".
| skybrian wrote:
| Everyone who disagrees with you is "unhinged." This is
| just name-calling.
| bobbylarrybobby wrote:
| If they're aware of how other languages either handle
| these issues or suffer the consequences of not handling
| them, then it sure is odd that they consciously decided
| to introduce the same issues into their own language,
| only to fix them down the road.
| skybrian wrote:
| Given Go's success, it seems like fixing certain footguns
| _much later_ actually worked out pretty well for them?
| That doesn't necessarily mean it was right, but it was
| perhaps not as big a deal as some people assume.
| doctor_eval wrote:
| It's also amusing to see someone talk about JavaScript and PLT
| in the same sentence, given the practical origins of JS.
| noelwelsh wrote:
| It's not making any claims about the quality of design in JS
| here. I'm literally not talking about them in the same
| sentence for this reason. I'm, instead, merely noting that it
| had the same issue. Looks like it was fixed in 2014:
| https://nullprogram.com/blog/2014/06/06/ C# also had the same
| issue, as noted elsewhere in the comments.
| [deleted]
| [deleted]
| anderskaseorg wrote:
| MDN goes into great detail, but the important point is that
| JavaScript fixed this with let and const. > a
| = []; for (i = 0; i < 3; i++) a.push(() => i); a.map(f => f())
| [ 3, 3, 3 ] > a = []; for (let i = 0; i < 3; i++)
| a.push(() => i); a.map(f => f()) [ 0, 1, 2 ] >
| a = []; for (i of "abc") a.push(() => i); a.map(f => f())
| [ 'c', 'c', 'c' ] > a = []; for (const i of "abc")
| a.push(() => i); a.map(f => f()) [ 'a', 'b', 'c' ]
| zeroimpl wrote:
| In contrast, Java would have a compile error so long as the
| variable was not declared final.
| ben0x539 wrote:
| But isn't it _completely wild_ that the `for (let i =`...
| version works like that? What does the loop 'desugar' into
| if written as a while-loop?
| biomcgary wrote:
| Is this loop variable problem really a theory issue (if so,
| does it have a label)? Or, primarily a practical one? Is there
| a database of known and historical programming language
| problems that are nicely tagged with "theory problem",
| "frequent foot-gun", etc?
|
| Could an LLM coupled with a constraint solver create the
| perfect language (for particular domains, e.g., performance)?
| Or, just use Rust ;-)?
| noelwelsh wrote:
| In contrast to the view of many, programming language theory
| is very closely intertwined with programming practice. It
| both drives practice and reacts to practice.
|
| In this particular case I imagine the issue was uncovered
| some time in the 1970s, which is when lexical scoping came to
| the fore in Scheme (in the US) and ML (in Europe). It's a
| fairly natural problem to run into if you're either thinking
| deeply about the interaction between closures (which capture
| environments) and loop constructs, or if you're programming
| in a language that supports these features.
| philosopher1234 wrote:
| What stance against PLT are you referring to?
| pseudonom- wrote:
| Probably quotes like:
|
| "It must be familiar, roughly C-like. Programmers working at
| Google are early in their careers and are most familiar with
| procedural languages, particularly from the C family. The
| need to get programmers productive quickly in a new language
| means that the language cannot be too radical."
|
| And not including sum types despite having a sum-type-shaped
| hole in the language (`if err != nil`).
|
| And some of the discussion about "why no generics" seemed
| kind of divorced from existing PL knowledge on the topic.
| tensor wrote:
| These were all intentional tradeoffs though, not any
| ignorance of theory. Also, it's pretty rich for someone to
| be complaining about Go while referencing Javascript of all
| languages. Javascript's design flaws are legendary. And I
| mean no disrespect to the creators of Javascript, they had
| to deal with some crazy last minute change requests to the
| language.
| kaba0 wrote:
| With all the crazy warts that JS has, it is at least a
| lisp-like very dynamic language if you squint (a lot) at
| it. Its greatest fault is probably leaving out integers
| (I can't even fathom why they decided on that, floats
| can't represent properly ints).
|
| Go is just simply badly designed, relying on hard-coded
| functionality a lot.
| TwentyPosts wrote:
| Ehhh, I see absolutely no evidence that the Go developers
| were particularly _aware_ of theory. It really feels more
| like they just were used to thinking in terms of C, and
| built a language which is kind of like C.
|
| Go also has some really weird stuff in it, such as named
| return values.
|
| Frankly, the lack of sum types hurts the most. The
| language would just be a lot better with a unifying
| Result type in the library. And don't give me any of that
| "oh, they tried to keep the language simple!" stuff.
|
| Intuitively, sum types are laughably simple. Everyone
| understands "It's one of these possible values, so you
| need to check which one it is and then handle that
| situation." They are more simple than enums on a
| conceptual level! Sum types are just not how
| C-programmers think about the world.
| scottlamb wrote:
| I've written a tiny bit of Go and am aware of the general problem
| this solves. I don't get their more subtle examples (the
| letsencrypt one or "range c.informerMap" vs "range alarms".
|
| When you do "for k, v := range someMap", is "v" of the map's
| value type (and one binding for the whole loop, copied before
| each iteration)? This would explain the problem, but I would have
| expected "v" to be a reference into the map, and I couldn't find
| the answer in a quick skim of the "For statements with range
| clause" in the spec. I'm probably looking in the wrong place
| because I touch Go rarely...
|
| [1] https://go.dev/ref/spec#For_statements
|
| edit: oh, the answer is in the "code block"-formatted table.
| Guess I had banner blindness. "v" is the copied value, not a
| reference. I'm surprised!
| tedunangst wrote:
| If you have a map of string to int, then v is of type int. It's
| a value. It's not pointer to int.
| scottlamb wrote:
| Is the expectation that you simply won't create a
| `map[...]expensivetocopyvalue`, but instead always do
| `map[...]*expensivetocopyvalue`?
| tedunangst wrote:
| The expectation is the compiler does what you tell it? If
| you want pointers in your map, you can do that too.
| Arnavion wrote:
| They're asking that, if the programmer wants the map to
| store expensivetocopyvalue semantically but also doesn't
| want to have iteration generate expensive copies, does
| the programmer have to change the map to store
| *expensivetocopyvalue instead?
|
| Anyway I believe the answer is that expensivetocopyvalue
| is not a type that exists in golang, because golang's
| "copy" operation is always a simple bitwise copy ala C
| struct copy / Rust's Copy trait, not like C++ copy ctor /
| Rust's Clone trait that can be arbitrarily expensive.
| morelisp wrote:
| In Go, `expensivetocopyvalue` can still be achieved via
| an enormous (e.g. multi-KB/MB) structure (which is most
| literally expensive to copy) or something containing a
| lot of pointers (which is not really expensive to copy
| but will start to pressure the GC).
| konart wrote:
| No expectations whatsoever. You can use pointer or a
| values. It's up to you.
| scottlamb wrote:
| That's silly. Language constructs and APIs are always
| made with expectations for how they're used, stated or
| not. You can write code that compiles without
| understanding and matching those expectations but it
| probably won't be good code.
|
| I'm asking because I think if it were expected that folks
| used large/expensive-to-copy map values, this construct
| would return a reference instead of copying. In Rust for
| example, the std library's "normal" [1] iterators return
| references.
|
| [1] not those returned by into_* or drain.
| konart wrote:
| Here an example: https://go.dev/play/p/He0lBEYZJ03
|
| Value is always a copy. Either a copy of a struct (in
| case of map[T]struct{}) or a copy of a pointer (in case
| of map[T]*struct{})
| morelisp wrote:
| Returning references to storage within the map would be a
| substantial footgun without borrowing.
| scottlamb wrote:
| Thanks, useful reply. I just realized that myself:
| https://news.ycombinator.com/item?id=37576558
|
| The peer comments along the lines of "the expecation is
| it does what it does" are not so helpful from a
| perspective of learning to write code that is in harmony
| with the language philosophy.
| [deleted]
| erik_seaberg wrote:
| Go doesn't support pointers to map keys or values. It does
| support pointers to array slots, but for-range copies each slot
| rather than giving you a pointer to it.
| scottlamb wrote:
| I suppose that makes sense when I think about it for a bit.
| My recent expectations come from work in Rust. There the
| language prevents you from mutating a map while holding a
| reference into it. Go doesn't have a mechanism to prevent
| that (except the one you said, simply not supporting those
| references at all). If you had a reference into a map that
| was resized because of a subsequent mutation, your reference
| would have to keep the whole previous map alive and point to
| different memory than a reference acquired since then. Both
| seem undesirable.
|
| With array slots, the same issue is present but is a bit more
| explicit because those resizes happen with `mySlice =
| append(mySlice, ...)`.
| erik_seaberg wrote:
| I think the slice append semantics are very error-prone,
| and it would have been better if a slice was a shareable
| reference to a single mutable thing, like a map (or a list
| from Python or Java or ...)
| parhamn wrote:
| > as a consequence of our forward compatibility work, Go 1.21
| will not attempt to compile code that declares go 1.22 or later.
| We included a special case with the same effect in the point
| releases Go 1.20.8 and Go 1.19.13, so when Go 1.22 is released,
| code written depending on the new semantics will never be
| compiled with the old semantics, unless people are using very
| old, unsupported Go versions
|
| How does this work? If I pull in a package that decided to pin
| 1.22 (as they should) and I compile with 1.18, would it compile
| or error that I need to use the 1.22 compiler?
| yankput wrote:
| They way I understand this, with go 1.18, 1.22 module as a
| dependency will compile and produce errorneous logic (!!) if it
| depends on this feature
|
| Thus it will be actively dangerous using go 1.18. I understand
| it like that.
|
| With go 1.19, you will get a compiler error.
|
| But as go is not fixing security bugs in old releases and std
| library, I think it is dangerous to use them anyway.
| jerf wrote:
| "But as go is not fixing security bugs in old releases and
| std library, I think it is dangerous to use them anyway."
|
| A bit of a harsh way to phrase that. In my experience, the
| backwards compatibility promises have been very good, and the
| way you stay up-to-date with security fixes and bugs in the
| standard library is to upgrade Go.
|
| I know that may strike terror in the hearts of developers
| used to the nightmare that major version upgrades can be in
| other languages, where a major version upgrade gets a multi-
| week task added into the task tracker, but it's completely
| routine for me to upgrade across Go major versions just to
| get some particular fix or to play with a new feature. I
| expect it to be a roughly five minute task, routinely.
|
| The only thing that has bitten me about it is arguably not
| even Go's fault, which is its continuing advances in TLS
| security and the increasing fussiness with which it treats
| things connecting with old-style certificates. I can't even
| necessarily disagree... I would also like to upgrade them but
| while it's my server, the clients connecting to it are using
| certs that are not mine and it's out of my control.
| yankput wrote:
| > A bit of a harsh way to phrase that. In my experience,
| the backwards compatibility promises have been very good,
| and the way you stay up-to-date with security fixes and
| bugs in the standard library is to upgrade Go.
|
| I don't think we disagree? There is no reason to use old
| version of go.
|
| I speak about grandparent comment who wanted to still run
| go1.18. It is not a good idea to still run go1.18, as it
| doesn't get security updates.
| packetlost wrote:
| Yes.
| jchw wrote:
| Interestingly, though, if you use Go 1.21 and a module declares
| a later version of Go, the default behavior is actually to go
| fetch a newer toolchain and use it instead[1]. It's a pretty
| cool feature, but I am a bit on the fence due to the fact that
| it is surprising and phones home to Google-controlled servers
| to fetch the binaries. That and the module proxy are for sure
| two of the most conflicting features in Go and I'd feel a lot
| better about them if Go was controlled by a foundation that
| Google merely had a stake in. Alas.
|
| edit: Actually, though, I just realized what I am talking about
| is different than what you and your quote is talking about,
| which is what happens when you have a dependency that declares
| a different version, not the current module. Oops.
|
| [1]: https://go.dev/blog/toolchain
| tedunangst wrote:
| You should get a compile error.
|
| But your code, when compiled with go 1.22, will still have go
| 1.18 semantics.
| krackers wrote:
| I think https://eli.thegreenplace.net/2019/go-internals-
| capturing-lo... describes the problem in more detail?
| davidw wrote:
| That's a much better explanation for someone like me who isn't
| very familiar with Go.
| msteffen wrote:
| This post was great--thanks for posting it!
|
| Interestingly, the reason the old i := i trick works is not at
| all what I thought!
|
| The trick, for reference: for i := 0; i < 5;
| i++ { i := i // the trick go func() {
| print(i) }() }
|
| What I assumed happened:
|
| - The escape analyzer sees that the new `i` is passed to a
| goroutine, so it is marked as escaping its lexical scope
|
| - Because it's escaping its lexical scope, the allocator
| allocates it on the heap
|
| - One heap allocation is done per loop iteration, and even
| though the new `i` is captured by reference, each goro holds a
| unique reference to a unique memory location
|
| What actually happens:
|
| - Go's compiler has heuristics to decide whether to capture by
| reference or by value. One component of the heuristic is that
| values that aren't updated after initialization are captured by
| value
|
| - The new i is scoped to the for loop body, and is not updated
| by the for loop itself. Therefore it's identified as a value
| that isn't updated after initialization
|
| - As a result, the Go compiler generates code that captures `i`
| by value instead of by reference. No heap allocations or
| anything like that are done.
|
| I recognize that the latter behavior is better, but if anyone
| with intimate knowledge of Go knows why the former doesn't
| (also) happen (is that even how Go works?) I would love to find
| out!
| xiaq wrote:
| I don't think it's so complex. Without i := i, there's only
| one i. With i := i, there's one i per iteration.
|
| Closure captures are always by reference.
|
| Heap vs stack allocation don't affect the language semantics.
| [deleted]
| MatthiasPortzel wrote:
| JavaScript had a very similar problem. If the loop variable is
| declared with the old `var` then it will not capture the
| variable. "New-style" variables declared with `let` are scoped
| to the loop. Although, I have to point JS started talking about
| making this change almost 20 years ago. As a JS developer, it's
| surprising to me to see Go having to make this change now.
| wheelerof4te wrote:
| Go never should have implemented closures.
|
| You add them and the next thing people want are objects.
| bagful wrote:
| All you really need in an language is the former - closures
| provide encapsulation (instance variables become bound
| variables) and polymorphism (over closures with the same
| function signature) all the same
| kubb wrote:
| Russ Cox and the Go team learned that the loop variable capture
| semantics are flawed not by reflecting about how their language
| works, but through user feedback.
|
| This could have been prevented by having one person on the team
| with actual language design experience, who could point this
| issue out in the design process.
|
| In this case, after 10 or so years, and thousands of production
| bugs, they backpedaled. How many other badly designed features
| exist in the language, and are simply not being acknowledged?
|
| If you point it out, and you're right, will you be heard if you
| don't have a flashy metric to point to, like a billion dollars
| lost?
|
| What if the flaw is more subtle, and explaining why it's bad is
| harder than in this very simple case, that can be illustrated
| with 5 lines of code? What if the link between it and its
| consequences isn't that clear, but the consequences are just as
| grave? Will it ever get fixed?
| AaronFriel wrote:
| Many languages have made this mistake, despite having engineers
| and teams with many decades or centuries of total experience
| working on programming languages. Almost all languages have the
| loop variable semantics Go chose: C/C++, Java, C# (until 5.0),
| JavaScript (when using `var`), Python. Honestly: are there any
| C-like, imperative languages with for loops, that _don't_
| behave like this?
|
| That decision only becomes painful when capturing variables by
| reference becomes cheap and common; that is, when languages
| introduce lightweight closures (aka lambdas, anonymous
| functions, ...). Then the semantics of a for loop subtly
| change. Language designers have frequently implemented
| lightweight closures before realizing the risk, and then must
| make a difficult choice of whether to take a painful breaking
| change.
|
| The Go team can be persuaded, it's just a tall order. And give
| them credit where credit is due: this is genuinely a
| significant, breaking change. It's the right change, but it's
| not an easy decision to make more than a decade into a
| language's usage.
|
| That said, there may be a kernel of truth to what you're
| alluding to: that the Go team can be hard to persuade and has
| taken some principled (I would argue, wrong) positions. I'm
| tracking several Go bugs myself where I believe the Go standard
| library behaves incorrectly. But I don't think this situation
| is the right one to make this argument.
| tuetuopay wrote:
| I hate to be _that_ guy but this would not be possible with
| rust, as the captured reference could not escape the loop
| scope. Either copy the value, or get yelled at the lifetime
| of the reference.
|
| This is one of the things the language was designed to fix,
| by people that looked at the past 50 years or so of
| programming languages, and decided to fix the sorest pain
| points.
| jeremyjh wrote:
| C# made the mistake not when they introduced loops, but when
| they introduced closures, and it didn't become evident until
| other features came along that propelled adoption of
| closures. Go had closures from the beginning and they were
| always central to the language design. C# fixed that mistake
| before the 1.0 release of Go. But the Go team didn't learn
| from it.
| MatthiasPortzel wrote:
| > JavaScript (unless using `for...of`)
|
| The change in Javscript doesn't have anything to do with
| for...of, it's the difference between `var` and `let`. And JS
| made the decision to move to `let` because the semantics made
| more sense before Go was even created (although code and
| browsers didn't update for another several years). That's why
| Go is being held to a higher standard, because it's 10+ years
| newer than the other languages you mentioned.
| AaronFriel wrote:
| Ah, thanks for the correction! That's right. But:
|
| 1. Did JS introduce this change after Go's creation? Yes.
| (And also after C#.)
|
| 2. Did arrow functions support precede let and const
| support? Yes.
|
| Answering the second question and finding the versions with
| support and their release dates answers the first question.
| Arrow functions let and const Firefox 22
| (2013) 44 (2016) Chrome 45 (2015)
| 49 (2016) Node.js 4 (2015) 6 (2016)
| Safari 10 (2016) 10 (2016)
|
| This places it nearly 10 years after the creation of Go.
| And with the exception of Safari, arrow functions were
| available for months to years prior to let and const.
|
| This is somewhat weak evidence for the thesis though; these
| features were part of the same specification (ES6/ES2015),
| but to understand the origin of "let" we also need to look
| at the proliferation of alternative languages such as
| Coffeescript. A fuller history of the JavaScript feature,
| and maybe some of the TC39 meeting minutes, might help us
| understand the order of operations here.
|
| (I'd be remiss not to observe that this is almost an
| accident of "let" as well, there's no intrinsic reason it
| must behave like this in a loop, and some browsers chose to
| make "var" behave like "let". Let and const were originally
| introduced, I believe, to implement lexical scoping, not to
| change loop variable hoisting.)
| adra wrote:
| This isn't a bug in java. Java has the idea of "effectively
| final" variables, and only final or effectively final values
| are allowed to be passed into lambdas seemingly to avoid this
| specific defect. Ironically, I just had a review the other
| day that touched on this go "interaction".
|
| The outcome of this go code in java would be as you'd expect,
| each lambda generated uses a unique copy of the loop
| reference value.
| AaronFriel wrote:
| Oh, today I learned. I think this was an issue in Scala
| (with `var`), but this seems like a great compromise for
| Java core.
|
| I suppose Java had many years after C#'s introduction of
| closures to reflect on what went well and what did not. Go,
| created in 2007, predates both languages having lightweight
| closures. Not surprising that they made the decision they
| did.
|
| Your comment inspired me to ask what Rust does in this
| situation, but of course, they've opted for both a
| different "for" loop construct, but even if they hadn't,
| the borrow checker enforces a similar requirement as Java's
| effectively final lambda limitation.
| ncann wrote:
| Newcomers to Java usually dislike the "Variable used in
| lambda expression should be final or effectively final"
| compiler error, but once you understand why that
| restriction is in place and what happens in other
| languages when there's no such restriction, you start to
| love the subtle genius in how Java did it this way.
| Macha wrote:
| Go, designed between 2007 and 2009, certainly had the
| opportunity to look at their introduction in C# 2.0,
| released 2005, or its syntactic sugar added in C# 3.0,
| released 2007.
| AaronFriel wrote:
| I think that's an ahistorical reading of events. They did
| have the opportunity, but there were very few languages
| doing what Go was at the time it was designed. My
| recollection of the C# 3 to 5 and .NET 3 to 4.5 is a bit
| muddled, but it looks like the spec supports a different
| reading:
|
| C# 3.0 in 2007 introduced arrow syntax. I believe this
| was primarily to support LINQ, and so users were
| typically creating closures as arguments to IEnumerable
| methods, not in a loop.
|
| C# 4.0 in 2010 introduced Task<T> (by virtue of .NET 4),
| and with this it became much more likely users would
| create a closure in a loop. That's how users would add
| tasks to the task pool, after all, from a for loop.
|
| C# 5.0 in 2012 fixes loop variable behavior.
|
| I think the thesis I have is sound: language designers
| did not predict how loops and lightweight closures would
| interact to create error-prone code until (by and large)
| users encountered these issues.
| omoikane wrote:
| This bug appears to be because Go captures loop variables by
| reference, but C++ captures are by copy[1] unless user
| explicitly asked for reference (`&variable`). It seems like
| the same bug would be visually more obvious in C++.
|
| [1] https://eel.is/c++draft/expr.prim.lambda.capture#10
| scottlamb wrote:
| Others have suggested that Rob Pike and Ken Thompson have some
| language design experience, to state it mildly. I also want to
| point out...
|
| > Russ Cox and the Go team learned that the loop variable
| capture semantics are flawed not by reflecting about how their
| language works, but through user feedback.
|
| I think "user feedback" isn't the whole story. It's not just
| the Go team passively listening as users point out obvious
| flaws. I've noticed in other changes (e.g. the monotonic time
| change [1]) the Go team has done a pretty disciplined study of
| user code in Google's monorepo and on github. That's mentioned
| in this case too. This is a good practice, not evidence of
| failure.
|
| [1]
| https://go.googlesource.com/proposal/+/master/design/12914-m...
| kaba0 wrote:
| > Rob Pike and Ken Thompson
|
| They are huge names in the field, but honestly, they just
| suck at language design itself.
| badrequest wrote:
| Are you suggesting Rob Pike has no experience designing a
| programming language?
| ziyao_w wrote:
| I think the parent was trying to imply that Ken Thompson had
| no experience in designing a programming language :-)
|
| Seriously though, "having experience" and "getting things
| right" are two different things, although Golang got a lot of
| things right, and the parent is being unnecessarily harsh.
| ranting-moth wrote:
| Or Ken Thompson?
| kubb wrote:
| I should say language design knowledge.
| randomdata wrote:
| _> Russ Cox and the Go team learned that the loop variable
| capture semantics are flawed not by reflecting about how their
| language works, but through user feedback._
|
| Since "Go 1" was deemed complete and the "Go 2" project began
| in 2018, the direction of the language was given to the
| community. It is no longer the Go team's place to shove
| whatever they think into the language. It has to be what the
| community wants, and that feedback showed it is what they want.
| jacquesm wrote:
| > This could have been prevented by having one person on the
| team with actual language design experience
|
| I thought you were serious, right up to that bit. Well played.
| I hope.
| alecbz wrote:
| > This could have been prevented by having one person on the
| team with actual language design experience, who could point
| this issue out in the design process.
|
| Instead of making a mistake, they could have simply not.
|
| See also RFC 9225: Software Defects Considered Harmful
| https://www.rfc-editor.org/rfc/rfc9225.html
| akira2501 wrote:
| > How many other badly designed features exist in the language,
| and are simply not being acknowledged?
|
| Very few.
|
| > If you point it out, and you're right, will you be heard if
| you don't have a flashy metric to point to, like a billion
| dollars lost?
|
| If you're right yet don't have a better idea then what do you
| expect to occur?
|
| > What if the link between it and its consequences isn't that
| clear, but the consequences are just as grave?
|
| The consequence is your developers must be careful with loop
| variables or they will introduce bugs. That's not particularly
| "grave" nor even especially novel.
|
| I'll admit, it's not a good ivory tower language, but then
| again, that's probably why I use it so often. It gets the job
| done and it doesn't waste my time with useless hypothetical
| features.
| [deleted]
| amomchilov wrote:
| I can't find it now, but I remember some joke about "it's an
| interesting language, but why did you ignore the last 50 years
| of programming language design?"
|
| I find Go quite frustrating in how it decries how over-
| complicated some features are, and slowly comes around to
| realize that oh, maybe people designed them for a reason (who
| woulda thunk it?).
| Patrickmi wrote:
| So whats your point ?, old ideas never die ?, language design
| is not language purpose and goal ?, they made a mistake
| creating Go ?, refusing to find something suitable or just
| break compatibility?
| rsc wrote:
| I answered some of this above:
| https://news.ycombinator.com/item?id=37577312
| dangoodmanUT wrote:
| Thank god!!!
| raydiatian wrote:
| Can somebody please explain to me why this doesn't constitute a
| major version due to a breaking change? Maybe I didn't read
| precisely enough but it sure sounds like a breaking semantic,
| esp. with the fact that "this will only work for versions 1.22
| and later." Sounds like a version upgrade trap to me? What am I
| missing?
|
| Or is it just because it's Golang and they're "we'll never
| release a go v2 even if we actually do release go v2 and call it
| v1.x"
| ben0x539 wrote:
| Seems like a pragmatic decision where the breakyness of the
| change is mitigated by the module versioning thing. Old code
| gets the old behavior, code written in newly created or updated
| modules gets the new behavior. Everybody is happy, compared to
| the alternative where this ships in a mythical go v2 which
| nobody uses while this sort of bug keeps sneaking people's
| actual work.
| fyzix wrote:
| This will result in more memory allocations but it's well worth
| it.
| orblivion wrote:
| For migrating, I wonder if there are any tools that could, let's
| say, go through your codebase and add a "// TODO - check" to
| every place that might be affected.
| [deleted]
| tgv wrote:
| I think there's a linter for it, perhaps more than one, in
| golangci-lint. It might be exportloopref and/or loopclosure.
| AaronFriel wrote:
| The C# language team encountered this as well, after introducing
| lightweight closures in C# 4.0 it quickly became apparent that
| this was a footgun. Users almost always used loop variables
| incorrectly, and C# 5.0 made the breaking change.
|
| Eric Lippert has a wonderful blog on the "why" from their
| perspective: https://ericlippert.com/2009/11/12/closing-over-the-
| loop-var...
|
| I had a bit of trouble finding the original C# 5 announcement;
| that's hopefully not been lost in the (several?) blog migrations
| on the Microsoft domain since 2012.
| hinkley wrote:
| Java also had this problem with anonymous classes. The solution
| is usually to introduce a functor. Being pass-by-value, it
| captures the state of the variables at its call time, which
| helps remove some ambiguity in your code.
|
| If you try to do something weird with variable capture, then
| any collections you accumulate data into (eg, for turning an
| array into a map), will behave differently than any declared
| variables.
|
| Go is trying to thread the needle by only having loop counters
| work this way. But that still means that some variables act
| weird (it's just a variable that tends to act weird anyway).
| And I wonder what happens when you define multiple loop
| variables, which people often do when there will be custom
| scanning of the inputs.
| [deleted]
| nerdponx wrote:
| Meanwhile Python has received this same feature request many
| times over the years, and the answer is always that it would
| break existing code for little major benefit
| https://discuss.python.org/t/make-lambdas-proper-closures/10...
|
| Given how much of an uproar there was over changing the string
| type in the Python 2 -> 3 transition, I can't imagine this
| change would ever end up in Python before a 4.0.
|
| Cue someone arguing about how bad Python is because it won't
| fix these things, and then arguing about how bad Python is
| because their scripts from 2003 stopped working...
| travisd wrote:
| It's worth noting that it's much less of a problem in Python
| due to the lack of ergonomic closures/lambdas. You have to
| construct rather esoteric looking code for it to be a
| problem. add_n = [] for n in
| range(10): add_n.append(lambda x: x + n)
| add_n[9](10) # 19 add_n[0](10) # 19
|
| This isn't to say it's *not* a footgun (and it has bit me in
| Python before), but it's much worse in Go due to the
| idiomatic use of goroutines in a loop: for
| i := 0; i < 10; i++ { go func() {
| fmt.Printf("num: %d\n", i) }() }
| hinkley wrote:
| Everyone else solved this problem by using list
| comprehensions instead. Rob has surely heard of those.
| sneak wrote:
| Somehow, Go managed to not break old code and also fix the
| problem.
|
| I think this is a good case of Python not fixing things,
| given that a fix exists that solves both problems.
| wrboyce wrote:
| By letting you specify a language version requirement? Not
| exactly backwards compatible (because it is explicitly not,
| as per the article).
|
| Python doesn't make breaking changes in non-major versions,
| so as mentioned by the upthread comment the appropriate
| place for this change would be in Python 4.
|
| Given the above, I'm really not sure what point you think
| you're making in that final paragraph.
| carbotaniuman wrote:
| This seems weird to given the number of breakages and
| standard library changes I seem to run into every
| version.
| wrboyce wrote:
| Really? I find that surprising. I don't write as much
| code as I used to but I've been writing Python for a long
| time and the only standard library breakages that come to
| mind were during the infamous 2 -> 3 days.
|
| What sort of problems are have you faced upgrading minor
| versions?
| pcl wrote:
| _> To ensure backwards compatibility with existing code,
| the new semantics will only apply in packages contained in
| modules that declare go 1.22 or later in their go.mod
| files._
| IshKebab wrote:
| Python could very easily have a similar mechanism. Hell
| even CMake manages to do this right, and they got "if"
| wrong.
|
| The Python devs sometimes seem stubbornly attached to
| bugs. Another one: to reliably get Python 3 on Linux and
| Mac you have to run `python3`. But on Windows there's no
| `python3.exe`.
|
| Will they add one? Hell no. It might confuse people or
| something.
|
| Except... if you install Python from the Microsoft Store
| it _does_ have `python3.exe`.
| wrboyce wrote:
| I've not run "python3" in years on my Mac, and I'm almost
| certain I never type it into Linux machines either;
| either I'm losing my mind, or there are some ludicrous
| takes in this thread.
| rat9988 wrote:
| You are surely losing your mind then. Python3 isn't
| something esoteric.
| wrboyce wrote:
| Entirely possible, but my point was I just type "python"
| and Python 3 happens. Do modern OS even come with Python
| 2 anymore?
|
| I'm not claiming any mystery about Python, just disputing
| how the modern version is invoked.
| nerdponx wrote:
| Right, and Go has the luxury of being a compiler that
| generates reasonably portable binaries, while Python
| requires the presence of an interpreter on the system at
| run time.
| orbisvicis wrote:
| Is it a bug? I've always depended on late-binding closures
| and I think even recently in a for loop, not that I'm going
| to go digging. You can do neat things with multiple functions
| sharing the same closure. If you don't want the behavior bind
| the variable to a new name in a new scope. From the post I
| get the sense that this is more problematic for languages
| with pointers.
| tester756 wrote:
| It is crazy that such behaviour even gets deployed
|
| It is so unintuitive...
| wahern wrote:
| It's unintuitive to users of the language, but it's very
| intuitive from the perspective of those implementing the
| language. Everybody seems to make this mistake. Lua 5.0
| (2003) made this mistake, but they fixed it in Lua 5.1
| (2006). (Lua 5.0 was the first version with full lexical
| scoping.)
| tester756 wrote:
| >It's unintuitive to users of the language, but it's very
| intuitive from the perspective of those implementing the
| language.
|
| It sounds like a lack of dogfooding, lack of review?
| catach wrote:
| To the degree the the implementers are also users they
| carry their implementer understanding into their use.
| Dogfooding doesn't help when your understanding doesn't
| match that of your users.
| gtowey wrote:
| It's not crazy. It's just the difference between a pointer
| and a value, which is like comp sci 101.
|
| I think the main things that make it such a trap is that the
| variable type definition is implicit so the fact that it's a
| pointer becomes a bit hidden, and that easy concurrency means
| the value is evaluated outside of the loop execution more
| often.
| tester756 wrote:
| >It's not crazy.
|
| No. Full disagree.
|
| Array represents a concept of holding multiple values
| (let's simplify) of the same type.
|
| Loop (not index based) over array represents concept of
| going *over* array's elements and executing some code body
| for each of it.
|
| Now, if the behaviour isn't that loop's body is executed
| for each array element (let's forget about returns, breaks,
| etc)
|
| then the design is terrible (or implementation, but that'd
| mean that it was a bug)
|
| I have totally no idea how can you design this thing in
| such a unintuitive way unless by mistake/accidentally.
| jrockway wrote:
| The loop semantics do not have anything to do with
| arrays. The point of confusion is whether a new slot for
| data is being created before each iteration, or whether
| the same slot is being used for each iteration. It turns
| out that the same slot is being used. The Go code itself
| is clear `for i := 0; i < 10; i++`. `i := 0` is where you
| declare i. Nothing else would imply that space is being
| allocated for each iteration; the first clause of the
| (;;) statement is only run before the loop. So you're
| using the same i for every iteration. This is surprising
| despite how explicit it is; programmers expect that a new
| slot is being allocated to store each value of i, so they
| take the address to it, and are surprised when it's at
| the same address every iteration. (Sugar like `for i, x
| := range xs` is even more confusing. The := strongly
| implies creating a new i and x, but it's not doing that!)
|
| Basically, here are two pseudocode implementations. This
| is what currently happens: i =
| malloc(sizeof(int)) *i = 0 loop:
| <code> *i = *i + 1 goto loop if *i < 10
|
| This is what people expect: secret =
| malloc(sizeof(int)) *secret = 0 loop:
| i = malloc(sizeof(int)) *i = *secret
| <code> *secret = *secret + 1 goto loop
| if *secret < 10
|
| You can see that they are not crazy for picking the first
| implementation; it's less instructions and less code, AND
| the for loop is pretty much exactly implementing what
| you're typing in. It's just so easy to forget what you're
| actually saying that most languages are choosing to do
| something like the second example (though no doubt, not
| allocating 8 bytes of memory for each iteration).
|
| Remember, simple cases work: for i :=
| 0; i < 10; i++ { fmt.Println(i) // 0 1 2 3 4
| ... }
|
| It's the tricky cases that are tricky:
| var is []*int for i := 0; i < 10; i++ {
| is = append(is, &i) } for _, i := range
| is { fmt.Println(*i) // 9 9 9 9 9 ...
| }
|
| If you really think about it, the second example is
| exactly what you're asking for. You declared i into
| existence once. Of course its address isn't going to
| change every iteration.
| hinkley wrote:
| If I understood the example, Java had this same problem.
| I'm wondering if C# does as well.
| mik1998 wrote:
| I don't know much about Go but the design seems very
| intuitive to me. You're doing something like (let ((i
| variable)) (loop (setf i ...) ...body)), which if there
| is a closure in the loop body will capture the variable i
| and also subsequent mutations.
|
| The fix is to make a new variable for each iteration,
| which is less obvious implementation wise but as per the
| post works better if you're enclosing over the loop
| variable.
| jerf wrote:
| jaredpar on the C# team offered the very first comment on the
| Github issue for this proposal:
| https://github.com/golang/go/discussions/56010
|
| I think it played a large part in helping get past the default-
| deny that any language change proposal should have. The other
| big one for me was the scan done over the open source code base
| and the balance of bugs fixed versus created.
| em-bee wrote:
| as soon as i saw mention of c# going through the same thing,
| i realized that this was discussed before:
| https://news.ycombinator.com/item?id=33160236
| nzoschke wrote:
| Thank you Go team and project!
|
| Go continues to be my favorite language and experience to build
| and maintain in.
|
| They got so much right from the start, then have managed to make
| consistent well reasoned, meaningful and safe improvements to the
| language over the years.
|
| It's not perfect, nothing is, but the "cost" of maintaining old
| code is so much lower compared to pretty much every other
| language I have used.
| devjab wrote:
| Go is such a productive language to work with, it's absolutely
| mind blowing how little adoption it has around where I live.
| Well I guess Lunar went from node to java to go, and harvested
| insane benefits from it, but a lot of places have issues moving
| into new languages. Not that I think that you should
| necessarily swap to a new hipster tech, I really don't, but Go
| is really the first language we've worked with that competes
| with Python as far as productivity goes. At least in my
| experience.
|
| We'll likely continue using Typescript as our main language for
| a while since we're a small team and it lets us share resources
| better, but we're definitely keeping an eye on Go.
| rcv wrote:
| I typically develop in Python, C++, and Typescript, and
| recently had to implement some code in Go. So far I've found
| it a pretty unpleasant language to use. It feels pedantic
| when I don't need it to be, and yet I have to deal with
| `interface{}` all over the place. Simple things that would be
| a one-liner Python or TS (or even just an std::algorithm and
| a lambda in C++) feel like pulling teeth to me in Go.
|
| I'd love to hear of any resources that can help me understand
| the Zen of Go, because so far I just don't get it.
| vineyardmike wrote:
| One of the big things that I've found helped is to "stop
| being an architect". Basically defer abstraction more.
|
| People, esp from a Java-esque class based world want class
| inheritance and generics and all that jazz. I've found at
| work like 50% of methods and logic that has some sort of
| generic/superclass/OOP style abstraction feature only ever
| has 1 implemented type. Just use that type and when the
| second one shows up... then try to make some sort of
| abstraction.
|
| For context, I can't remember the last time that I actually
| used "interface{}". Actual interfaces are cheap in go, so
| you can define the interface at use-time and pretty cheaply
| add the methods (or a wrapper) if needed.
|
| If you're actually doing abstract algorithms and stuff
| every day at work... you're in the minority so I don't know
| but all the CRUD type services are pretty ergonomic when
| you realize YAGNI when it comes to those extra
| abstractions.
|
| Edit: also f** one liners. Make it 2 or three lines. It's
| ok.
| badrequest wrote:
| I write Go every day, and can count the number of times per
| year I have to involve an `interface{}` literal on one
| hand. Unless you're doing JSON wrong or working with an API
| that simply doesn't care about returning consistently
| structured data, I can't fathom why you'd be using it "all
| over the place."
| coffeebeqn wrote:
| Me too. We have around a dozen go services and I have
| maybe used or seen interface{} once or twice for a hack.
| Especially after generics. I think the parent comment is
| suffering from poor quality go code. It's like
| complaining about typescript because things in your
| codebase don't have types
| Spiwux wrote:
| You discovered the Zen of Go. There are no magic one
| liners. It's boring, explicit and procedural.
|
| Proponents argue that this forced simplicity enhances
| productivity on a larger organisational scale when you take
| things such as onboarding into account.
|
| I'm not sure if that is true. I also think a senior Python
| / Java / etc resource is going to be more productive than a
| senior Go resource.
| goatlover wrote:
| Go seems like the antithesis to Lisp.
| pharmakom wrote:
| ... so the code ends up being really long then.
| Spiwux wrote:
| Yes, pretty much. It's a pain to write, but easy to read.
| On a larger scale the average engineer likely spends more
| time reading code than writing code
| tuetuopay wrote:
| I don't find go that easy to read. It is so verbose that
| the actual business logic ends up buried in a lot of
| boilerplate code. Maybe I'm bad at reading code, but it
| ends up being a lot of text to read for very little
| information.
|
| Like a one-line list comprehension to transform a
| collection is suddenly four lines of go: allocation, loop
| iteration, and append (don't even start me on the append
| function). I don't care about those housekeeping details.
| Let me read the business logic.
| pharmakom wrote:
| I think that shorter code is easier to read - to a point!
| on balance most code is too long, not too short.
| gtowey wrote:
| Go is the language that's not made for you, it's made to
| make the life of the next guy who has to maintain your code
| easier! :-)
| hagbarth wrote:
| One of the reasons I like Go is that it really doesn't try to
| be a hipster language. It's kinda boring, which is great!
| Philip-J-Fry wrote:
| Boring is good when you want to build things that are
| maintainable by 100s of devs.
|
| Something we have experienced over and over is that devs
| moving from languages like C# or Java just love how easy
| and straight forwarding developing in Go is. They pick it
| up in a week or two, the tool chain is just so simple,
| there's no arguing around what languages features we can
| and can't use.
|
| Almost everyone I've spoke to finds it incredibly
| productive. These people want to be delivering features and
| products and it makes it easy for them to do so.
| VirusNewbie wrote:
| Maybe a 100 devs Go is fine, but it gets to be a
| nightmare as you scale beyond that.
|
| Language abstractions exist to prevent having developers
| build their own ad-hoc abstractions, and you find this
| time and time again in languages like Go. You can read
| the Kubernetes code and see what I mean, they go out of
| their way to work around some of the missing language
| features.
| [deleted]
| kaba0 wrote:
| > They got so much right from the start, then have managed to
| make consistent well reasoned, meaningful and safe improvements
| to the language over the years
|
| In which universe? They have to constantly patch the language
| up and go back on previous assumptions.
| christophilus wrote:
| Fast compiler, simple tooling, baked in fmt, simple cross
| platform compilation, decent standard library, a tendency
| towards good enough performance if doing things the Go way,
| async without function coloring. They got some things right
| and some things wrong. When tossing out orthodoxy, you'll
| tend to get some things wrong. I think a lack of sum types is
| my biggest gripe.
| nzoschke wrote:
| The std library is a big part of the magic. It's so
| shocking to go to JS land and see that there are 10
| different 3rd party libraries to make http requests, all
| with wildly different ergonomics all within one code base
| due to cross dependency heck.
|
| In Go there's pretty much only the http package, and any
| 3rd party packages extend it and have the same ergonomics.
|
| For a while my biggest gripe was package management but
| it's a dream where we are now.
| 13415 wrote:
| That's the _first_ language change that can in theory break
| programs (in practice, it won 't). Everything else was just
| additions to the existing language with full backwards
| compatibility. That's the opposite of constantly patching the
| language up.
| kaba0 wrote:
| You can patch things up without breaking backwards
| compatibility.
|
| But, going on a well-trodden path slower than the pioneers
| is not a big achievement.
| badrequest wrote:
| Please point to when Go has broken backwards
| compatibility.
| kaba0 wrote:
| That's not my point.
|
| Smart men learn from the mistakes of others.
| nzoschke wrote:
| Obviously different perspectives in this thread.
|
| Robert Griesemer, Rob Pike, and Ken Thompson are
| objectively smart men and pioneers and have learned from
| lots of mistakes both they and the industry made.
|
| Go embodied a lot of those learnings out of the gate.
|
| If the bar is to be perfect out of the gate that's
| impossible and I can't think of any language that could
| pretend to be so.
|
| Go was very good out of the gate and has slowly but
| surely evolved to be great.
| vlunkr wrote:
| So how do you think they are "patching things up" more
| than other languages?
| sidewndr46 wrote:
| This nonsense again? Where it is controlled per module,
| effectively making it impossible to review code for correctness
| without checking those controls. This is an anti-feature. If you
| can't be bothered to make a local copy or reference the correct
| variable, the problem is the developer. Not the language.
| RadiozRadioz wrote:
| The problem is the language when its ergonomics coerce most
| developers to assume a construct works in a way it does not.
|
| Programmers are often at fault when a language complex, but I'd
| give them a pass when it's simply counterintuitive.
| ravivooda wrote:
| Similar: https://github.com/ravivooda/gofor
| chen_dev wrote:
| the 'fix' to the example in the README should be obvious, but
| for reference:
|
| - for _, e := range es {
|
| - pumpUp(&e)
|
| }
|
| + for i := range es {
|
| + pumpUp(&es[i])
|
| }
| timrobinson333 wrote:
| I'll be the first to admit I know almost nothing about go, but
| it's surprises me to find we're still inventing languages with
| bobby traps like this, especially bobby traps that were well
| known and understood in other languages at the time.
|
| Actually it surprises me we're still inventing languages where
| local variables can be mutated, which seems to be at the root of
| the problem here
| wrboyce wrote:
| Completely unrelated to the point you're making, but the phrase
| is "booby trap"; I believe it originates from pranks played on
| younger schoolboys in 1600s England (the etymology of booby
| being the Spanish "bobo").
| mixmastamyk wrote:
| > where local variables can be mutated
|
| They aren't local, but belong to the outer scope. The
| misconception in a nutshell.
| tazjin wrote:
| Go has a long list of booby traps like this and prides itself
| on them. From outside of the Go team it looks like a small
| cultural shift might slowly be happening, cleaning up some of
| the obvious mistakes everyone's been telling them about since
| the beginning. Rob Pike retiring and giving up some formal
| power with that probably helps.
| thiht wrote:
| > Go has a long list of booby traps like this
|
| Huh? Where's the list? From the top of my head I think this
| is the only thing that repeatedly bit me, although I'm very
| aware of the behavior of for loop scoping. Linters save me
| nowadays at least.
|
| Are there other things like that in the language that deserve
| a fix? Maybe things to do with json un/marshaling?
| the_gipsy wrote:
| My number one is not having algebraic/sum/union types
| leading to needing zero-values. Which is more like a never
| idling foot gatling gun.
| [deleted]
| wazzaps wrote:
| Copying a mutex by value (thus duplicating the lock,
| causing deadlocks or worse) is far too easy
| icholy wrote:
| `go vet` catches this.
| erik_seaberg wrote:
| Is there a way to declare any type uncopyable? This is
| something I always thought Ada got right.
| icholy wrote:
| There isn't.
| smasher164 wrote:
| you stick this in your struct type
| noCopy struct{} func (*noCopy) Lock() {}
| func (*noCopy) Unlock() {}
| mhh__ wrote:
| I use a tool called "type theory"
| usefulcat wrote:
| Wow, even c++ won't let you do that (without invoking
| undefined behavior, anyway).
| rsc wrote:
| Speaking as the person Rob Pike handed the formal power to (8
| years ago now), I don't think that change has much to do with
| it.
|
| We've known about the problem for a long time. I have notes
| from the run up to Go 1 (circa 2011) where we considered
| making this change, but it didn't seem like a huge problem,
| and we were concerned about breaking old code, so on balance
| it didn't seem worth it.
|
| Two things moved the needle on this particular change:
|
| 1. A few years ago David Chase took the time to make the
| change in the compiler and inventory what it broke in a large
| code base (Google's, but any code base would have worked for
| that purpose). Seeing that real-world data made it clear that
| the problem was more serious than we realized and needed to
| be addressed. That is, it made clear that the positive side
| of the balance was heavier than we thought it was back in
| 2011.
|
| 2. The design of Go modules added a go version line, which we
| can key the change off. That completely avoids breaking any
| old code. That zeroed out the negative side of the balance.
| orblivion wrote:
| What about let's say 5 years from now, someone digs up a Go
| project from 2022, decides to get it up to speed for 2028,
| updates the version line. Is there something that would
| remind them to check for breaking changes, including this
| one? Perhaps the go project initializer could add a comment
| above the version line with a URL with a list of such
| changes. Though, that wouldn't help for this change.
| [deleted]
| flakes wrote:
| I think the key difference here is to consider toleration
| vs adoption. Old code is able to tolerate the changes and
| still work in new ecosystems. There is still work on
| maintainers if they want to actually adopt the features
| themselves. Allowing these two concepts to work together
| is what allows iteratively updating the world, rather
| than requiring big bang introduction of features.
|
| As for validating your software, the answer is the same
| as its always been... tests, tests and more tests.
| smasher164 wrote:
| > we're still inventing languages where local variables can be
| mutated
|
| Local mutability is probably one of the most common uses of
| mutability. A lot of it is using local state to build up a more
| complicated structure, and then getting rid of that state.
| Getting rid of that use-case is just giving up performance.
| [deleted]
| zdimension wrote:
| There is a recurring joke about Go's language design ignoring
| many bits of the general language design knowledge collectively
| acquired through decades of writing new languages. This change
| is an example of why this joke exists.
| kzrdude wrote:
| I have run into this problem in Python too, but not recently. I'm
| not sure if Python has changed or if I just caught on to the
| problem.
|
| This should be enough to show that it still can wind up as a
| problem in Python: funcs = [(lambda: x) for x
| in range(3)] funcs[0]() # outputs: 2
| pulvinar wrote:
| GPT-4 says: The behavior you're observing is due to the late
| binding nature of closures in Python. When you use a lambda
| inside a list comprehension (or any loop), it captures a
| reference to the variable x, not its current value. By the time
| you call funcs[0](), x has already been set to the last value
| in the range, which is 2.
|
| To get the desired behavior, you can pass x as a default
| argument to the lambda: funcs = [(lambda x=x:
| x) for x in range(3)] funcs[0]() # outputs 0
| paulddraper wrote:
| That is correct.
|
| Python used to be worse; it used to share scope _outside the
| list comprehension_.
| DonnyV wrote:
| GO syntax is so hard to look at.
| knodi wrote:
| If you hate readability, sure.
| guessmyname wrote:
| > _GO syntax is so hard to look at._
|
| What do you mean by "hard"?
|
| I find Rust syntax challenging to grasp in a specific way. Rust
| employs numerous symbols and expressions to convey statements,
| which makes reading Rust code a process of constantly
| navigating between different keywords, left and right. I have
| to create a mental map of what certain statements are
| accomplishing before I can truly comprehend the code.
|
| In contrast, I find Go code relatively straightforward,
| especially for those familiar with C-like programming
| languages. This clarity is due to the deliberate verbosity of
| the language, which I personally appreciate, as well as the use
| of early return statements.
|
| But don't get me wrong. I enjoy programming in both Rust and Go
| when they are suitable for the task at hand, but I usually
| spend more time grappling with Rust's syntax than with Go's,
| because I often invest more time in understanding the structure
| and logic of Rust programs compared to their Go counterparts.
| tuetuopay wrote:
| I guess it depends on the way the brain works. I have very
| bad memory, but I prefer the expressiveness of Rust to the
| verbosity of Go. I value much more having the whole context
| on the screen that navigating countless words of boilerplate
| code. I do agree that it gets a bit of getting used to, but I
| find it easier to recognize by eye.
| ShamelessC wrote:
| At least part of the issue for me was that many
| keywords/syntax rules don't match anything I'm familiar with,
| even considering "C-like" languages.
|
| I have similar issues with Rust actually. There's a lot of
| sugar used that you have to grok and that takes some time.
|
| On the other hand Python, C#, Java all stick with a set of
| fairly familiar conventions. In terms of syntax (and only
| syntax), the learning curve is more intense with Go; perhaps
| similar to the initial alienation provided by JavaScript.
|
| My experience has been that once you are being paid to learn
| a language these problems mostly disappear. Alas, no one ever
| paid me to learn Go.
| campbel wrote:
| Won't this end up breaking programs that depend on the current
| behavior?
| [deleted]
| minroot wrote:
| Some time spent with Go gives a strong indication that Go team
| always has backwards compatibility in mind
| jacquesm wrote:
| They do. Go has avoided most of the pitfalls that other
| language eco-systems have fallen for over the years
| (backwards compatibility issues, soft forks masquerading as
| language improvements, re-booting the whole language under
| the same name, aggressively pushing down on other languages
| etc). They've done _remarkably_ well in those respects, and
| should deserve huge credit for it.
| nerdponx wrote:
| Python has the same problem (to the extent that it's actually a
| problem, which you might or might not agree with), and this is
| the #1 reason they won't change it.
| omeid2 wrote:
| I don't know why you're being down voted, but it is actually
| breaking the Go1 compat promise. Which says:
| It is intended that programs written to the Go 1 specification
| will continue to compile and run correctly, unchanged, over the
| lifetime of that specification. At some indefinite point, a Go
| 2 specification may arise, but until that time, Go programs
| that work today should continue to work even as future "point"
| releases of Go 1 arise (Go 1.1, Go 1.2, etc.).
| jacquesm wrote:
| And they do. You can specify the precise logic to use on a
| per-file basis.
| wrs wrote:
| Note the word "programs", not "files". If your program
| doesn't declare go 1.22 in its go.mod, it will continue to
| work (or not work!), unchanged.
| robertlagrant wrote:
| Isn't that "working with future point releases", though? If
| I don't declare 1.22, am I not excluded from that point
| release?
| ben0x539 wrote:
| I assume that if compiling with 1.22 or later, you still
| get all the benefits from that version like other new
| features, bug fixes or perf improvements, just not this
| particular change.
| mrkstu wrote:
| No, the compiler will revert to the original behavior, it
| only adopts the new behavior with the declaration.
| freedomben wrote:
| I upvoted the question to offset one of the downs because I
| agree it's a fair question. However I would guess the
| downvotes are because TFA addressed this issue directly and
| comprehensively, so it's a clear "I didn't read the article"
| indicator :-) Possibly also because the downvoters can't
| imagine a scenario where this would be desirable behavior
| (i.e. it's always a bug)
| colejohnson66 wrote:
| But if it's a bug, then the logic to not compile future
| versions is wrong, IMO. If it's a feature change, then such
| logic would make sense.
| campbel wrote:
| Yeah its fair, I didn't closely read that section.
| Although, I'm not entirely convinced the approach is safe,
| maybe its worth it to fix such a common pitfall.
| [deleted]
| campbel wrote:
| I don't mind.
|
| Yeah, I thought this kind of change wouldn't happen because
| of this promise.
| tgv wrote:
| I think it was downvoted precisely because of that. It's a
| bit of a contentious issue.
| doctor_eval wrote:
| To add to the other comments, in the run-up to go1.21 they
| talked about how they'd analysed a very large corpus of Go code
| to see what would be affected, and it was a very very small
| number.
|
| I remember thinking that the number of people who have created
| inadvertent bugs due to this design (myself included) would be
| significantly greater than the number of people affected by the
| fix.
| omginternets wrote:
| Seems like yes, though hopefully that should be rare.
| matthewmueller wrote:
| It's only enabled for modules that run Go 1.22 and higher
| badrequest wrote:
| I also can't imagine a case where it is useful or even truly
| intended to rely on this behavior.
| ben0x539 wrote:
| Yeah I don't think it's so much "we explicitly rely on this
| behavior, how dare you change this" as "somewhere in our
| mountains of maintenance-mode code that haven't seen the
| sun shine through an editor window in years, this behavior
| cancels out another bug that we never noticed". Tooling
| should be able to detect when code relies on this, but it's
| still gonna cost some non-zero amount of developer effort
| to touch ancient code and safely roll out a new version if
| it needs to be actively addressed.
| rsc wrote:
| If you have tests and they break with
| GOEXPERIMENT=loopvar, then there is a new tool that will
| tell you exactly which loop is causing the breakage.
| That's a post for a few weeks from now.
| omginternets wrote:
| Yeah it's def a code smell ...
| infogulch wrote:
| > To ensure backwards compatibility with existing code, the new
| semantics will only apply in packages contained in modules that
| declare go 1.22 or later in their go.mod files. ... It is also
| possible to use //go:build lines to control the decision on a
| per-file basis.
| sixstringtheory wrote:
| Doesn't that mean that all code written so far can't take up
| newer versions of the Go compiler for any other reason like
| new features/bugfixes/optimizations/etc without a full audit
| of codepaths involving for loops?
| ericpauley wrote:
| No, it does not. Packages can compile using 1.22 and gain
| other benefits without opting into this change.
| sixstringtheory wrote:
| Ah, I didn't see the part about //go:build
| matthewmueller wrote:
| Anyone know how this will impact loop performance?
| tedunangst wrote:
| Nonexistent? It can still reuse the memory if you don't capture
| it. And if you were capturing it "properly" it was already
| making a copy.
| jsmith45 wrote:
| Most commonly no impact. It can require an additional heap
| allocation per iteration if taking the address or capturing in
| a closure, but even in those cases escape analysis may be able
| to determine that the value can remain on the stack because it
| will not remain referenced longer than the current loop
| iteration. If that happens then this change has no impact.
|
| I'm not sure how thorough Go's escape analysis is, but nearly
| all programs that capture the loop variable in a closure and
| are not buggy right now could be shown to have that closure not
| escape by a sufficient thorough escape analysis. On the other
| hand for existing buggy programs, then perf hit is the same as
| assigning a variable and capturing that (the normal fix for the
| bug).
|
| Google saw no statistically significant change in their
| benchmarks or internal applications.
|
| https://github.com/golang/go/wiki/LoopvarExperiment#will-the...
| mongol wrote:
| Is this a common way to fix problems in language syntax? It seems
| unintuitive to me. Now you need to know what version is declared
| in one file to understand behavior in another file. I understand
| they want to fix this but I did not know this way was allowed.
| jjwiseman wrote:
| I know there are much earlier examples, but the earliest warning
| about this behavior I could find in 60 seconds of searching is
| from the comp.lang.lisp FAQ, posted more than 30 years ago, in
| 1992: Mar 21, 1992, 1:00:47 AM Last-
| Modified: Tue Feb 25 17:34:30 1992 by Mark Kantrowitz ;;;
| ****************************************************************
| ;;; Answers to Frequently Asked Questions about Lisp
| *************** ;;;
| ****************************************************************
| ;;; Written by Mark Kantrowitz and Barry Margolin ;;;
| lisp-faq-3.text -- 16886 bytes [...]
| ----------------------------------------------------------------
| [3-9] Closures don't seem to work properly when referring to the
| iteration variable in DOLIST, DOTIMES and DO.
| DOTIMES, DOLIST, and DO all use assignment instead of binding to
| update the value of the iteration variables. So something like
| (dotimes (n 10) (push #'(lambda () (incf n))
| *counters*)) will produce 10 closures over the
| same value of the variable N.
| ----------------------------------------------------------------
___________________________________________________________________
(page generated 2023-09-19 23:00 UTC)