[HN Gopher] Math Foundations from Scratch
___________________________________________________________________
Math Foundations from Scratch
Author : paulpauper
Score : 103 points
Date : 2021-10-19 20:02 UTC (2 hours ago)
(HTM) web link (learnaifromscratch.github.io)
(TXT) w3m dump (learnaifromscratch.github.io)
| jstx1 wrote:
| This is will be inaccessible for anyone who actually starts from
| scratch, it spends too much time on topics that are largely
| irrelevant and it isn't the best resource for anyone who has the
| time to spend on the irrelevant parts.
|
| I don't think there's a good target audience for it.
| sghiassy wrote:
| Know of any resources that do accomplish the goal of starting
| from scratch? I'd be interested
| codesuki wrote:
| I recently started working through Basic Mathematics by Lang.
| It has exercises with solutions to about half of them. It
| starts basic but introduces simple proofs early. Although
| what counts as 'from scratch' depends on the reader I guess.
| The book doesn't define numbers via sets, maybe check the
| table of contents.
| DrPhish wrote:
| Mathematics: From the birth of numbers by Jan gullberg
|
| A worthwhile investment
| [deleted]
| lisper wrote:
| The title of this sounded very promising. Something that actually
| describes math from scratch needs to exist, but this ain't it. It
| runs off the rails from the very first sentence, where it asks
| "What is math?" but doesn't actually answer the question except
| to tell you that it's easy.
|
| I stopped reading when I got to "Numbers are objects you build to
| act like the concept of a number."
|
| No no no no no. No! Math is the act of inventing sets of _rules_
| for _manipulating symbols_ that produce interesting or useful
| results, or exploring the behavior of sets of rules invented by
| others. One of the earliest example of such interesting or useful
| results is rules for manipulating symbols so that the results
| correspond to the behavior of physical objects (like sheep, or
| baskets of grain, or plots of land), and in particular, to their
| _quantity_ so that by manipulating the right symbols according to
| the right rules allowed you to make reliable predictions about
| the behavior of and interactions between quantities of sheep and
| grain and land. The symbols that are manipulated according to
| these rules are called "numerals" and the quantities that they
| correspond to are called "numbers".
|
| But you can invent other rules for other symbols that produce
| other useful and interesting behavior, like "sets" and "vectors"
| and "manifolds" and "fields" and "elliptic curves." But it all
| boils down to inventing sets of rules for manipulating symbols.
| Smaug123 wrote:
| I disagree, and I believe you have made a category error: you
| are mistaking the description of a thing for the thing itself.
| I believe your error is the same as that of the one who asserts
| "the natural numbers are the finite transitive sets". In fact,
| the finite ordinals are only one possible _implementation_ of
| the natural numbers in set theory, and what the natural numbers
| really are is "the abstract notion of the smallest structure
| you can do induction on". Similarly, any given symbolic
| representation of a thing is an implementation of that thing in
| mathematics, but it need not be the thing itself.
|
| Perhaps our best (or even only) hope of automatic mathematics
| verification is to find a symbolic representation of a thing,
| and there are some areas where we have in some sense
| "definitely found the right symbols" which cleave very tightly
| to the thing they represent; and it's probably always true that
| if you can phrase something in the symbolic language, then you
| will be more rigorously constrained to say correct things by
| construction. But when an actual human actually does
| mathematics, they may very well not be manipulating symbols at
| all until they come to the point where they must transmit their
| thoughts to someone else. In fact, in some cases it may even be
| possible to transmit nontrivial mathematical thoughts without
| symbols; one possible method is simply "analogy".
|
| To revisit the example at the start, by the way: If you choose
| the language of category theory, of course, you can formalise
| "the abstract notion of the smallest structure you can do
| induction on" into the symbolic representation whose name is
| "natural numbers object", and thereby claim that once more we
| are in the world of rules for manipulating symbols. But I do
| not believe the implied claim that "unless you think of the
| naturals as their category-theoretic representation, then you
| are not doing mathematics"!
| lisper wrote:
| > I disagree
|
| With what?
|
| > you are mistaking the description of a thing for the thing
| itself
|
| How?
|
| > what the natural numbers really are is "the abstract notion
| of the smallest structure you can do induction on".
|
| And who granted you the authority to define what the natural
| numbers "really are"? The last time I checked, numbers were
| not actually part of physical reality, they were a kind of
| concept or idea. People are free to attach whatever labels
| they like to concepts and ideas. They can even overload terms
| to mean different things in different contexts. So just
| because I use the word "number" in some colloquial context to
| stand for a different idea than you do in some other context
| doesn't make it _wrong_.
| [deleted]
| _448 wrote:
| > One of the earliest example of such interesting or useful
| results is rules for manipulating symbols so that the results
| correspond to the behavior of physical objects (like sheep, or
| baskets of grain, or plots of land)...
|
| Or was it the other way round? First came the objects and then
| people saw patterns in the transactions and then abstracted it
| away with symbols?
|
| One of the reasons children find maths difficult is because we
| tend to describe maths in this reverse order i.e. from symbols
| to physical objects(the way you described above). But
| explaining it in the actual order i.e. from physical objects to
| symbols will make maths accessible to many more kids.
| lisper wrote:
| The evolution of math came by way of things like tally marks
| on sticks and pebbles in pots. The line between symbol and
| physical system is fuzzy.
|
| I think one of the reasons that people find math difficult is
| that the emphasis is on the symbol-manipulation rules and not
| on the reasons that certain sets of symbol-manipulation rules
| are useful. Kids are taught to count "1 2 3 4..." as if these
| symbols were handed down by God rather than being totally
| arbitrary. They should be taught to count "*, **, ***, ****,
| ..." and then, when they get to "***********" or so, they
| should be encouraged to _invent_ shorter ways of writing
| these unwieldly strings.
|
| (Note, BTW, that when you use the "*, **, ***, ****, ..."
| numeral system, addition and multiplication become a whole
| lot easier!)
| bawolff wrote:
| I don't get your objection
|
| "Numbers are objects you build to act like the concept of a
| number."
|
| So, numbers (i read that as a colloquialism for "numerals") are
| symbols that represent the abstract idea of a "number".
|
| You can't have (non-trivial) rules for manipulating symbols
| without symbols, so i don't understand your objection.
| lisper wrote:
| The definition as stated is circular. It has the same
| information content as "A foo is an object that you build to
| act like the concept of a foo."
|
| Also, the _symbols_ that _stand_ for numbers have a name.
| They are called _numerals_.
|
| So: a number is an abstraction of a _quantity_ , which is a
| number plus a unit. "Seven sheep" is a quantity of sheep.
| "Seven bushels of grain" is a quantity of grain. "Seven acres
| of land" is a quantity of land. "Seven" is the abstract
| property that these things have in common, and the symbol "7"
| is the numeral that denotes this property.
| bawolff wrote:
| I don't think its circular, just really informal. The two
| usages of the word "number" are refering to two different
| meanings of the word.
|
| If your original objection was a lack of formality, i'd say
| its a matter of taste, audience and author intent, but
| ultimately a fair objection. But this seems very far afield
| from your original complaint.
| threatofrain wrote:
| I've stated elsewhere, but will repeat here so that
| others are not confused:
|
| "Number" is an informal concept in mathematics. The real
| numbers are a specific concept with multiple
| formalizations. A model of a number system is an assembly
| of rules that behave like the numbers you want.
|
| I don't see the circularity here, and given that they are
| tackling an informal concept of number, I don't see any
| loss of clarity either.
| kmill wrote:
| Ah, I see you're a mathematical formalist. That's one
| philosophy of what math is, but it's not the only one.
|
| What's going on in my mind when I visualize 3-dimensional
| manifolds and manipulate them? Cutting pieces apart,
| reattaching them elsewhere, stereographically reprojecting
| objects embedded in a 3-sphere. I don't see where the symbols
| are exactly here -- it seems like I'm (at least trying to)
| manipulate "actual" mental objects, ones my visual cortex are
| able to help me get glimpses of. Everything I'm doing can be
| turned into symbolic reasoning, but it's a fairly painful
| process. It seems closer to the basis for the philosophy of
| intuitionism, that math derives from mental constructions.
|
| I think it's rather interesting that many creatures (including
| ourselves) have a natural ability to at a glance tell how many
| things there are, up to about five objects or so. Maybe it's
| useful evolutionarily for counting young, or for detecting when
| a berry has gone missing, etc. I think it's also rather
| interesting that many creatures have episodic memory. Sometimes
| it seems to me that the invention of symbolic number derives
| from these two capabilities: through our experience that things
| can happen in sequence, we extend our natural ability to count
| while believing these "numbers" are meaningful. Each number is
| a story that plays out for what's been accumulated.
| lisper wrote:
| The boundary where abstract noodling around with mental
| images becomes math lies precisely at the point where you can
| render those images symbolically. That is the thing that
| distinguishes math from all other forms of human mental
| endeavor.
| bawolff wrote:
| What a weird definition. Which fields of human endeavour
| cannot be rendered symbolically?
|
| If you are taking a very strict view of what it means to
| symbolize something (which i imagine you are if you think
| other fields cant be), then i wonder: do you consider
| geometric proofs math? Is euclid's elements with its prose
| proofs, math?
| lisper wrote:
| > Which fields of human endeavour cannot be rendered
| symbolically?
|
| All of them other than math.
|
| > do you consider geometric proofs math?
|
| Yes, because diagrams are symbols. They happen to be
| rendered in 2-D but they are symbols nonetheless.
|
| What makes something a symbol is that it is subject to a
| convention that allows it to fall into one of a finite
| number of equivalence classes, so that small changes in
| its physical details don't change its function within the
| context of the symbol-manipulation activity. Geometric
| diagrams have this property, so they qualify as symbols.
| kmill wrote:
| I see, so as a low-dimensional topologist I'm not a
| mathematician, thanks!
|
| But more seriously, if you haven't read it already you
| might take a look at Thurston's "On Proof and Progress in
| Mathematics." He had a stunningly accurate intuition, and
| putting what was in his mind into a form that other experts
| could understand and consider to be a proof gave many
| mathematicians a job for quite a long time. So long as you
| had him around, you could query him for any amount of
| detail for why his claims were true, so in that sense he
| had real proofs.
|
| Symbolic proofs are sort of a lowest-common-denominator,
| serializing what's in one's mind to paper and reducing
| things to machine-checkable rules. It's also a rather
| modern idea that this is what math is.
|
| I think it's very much worth considering math to be the
| study of what can be made perfectly obvious, which is
| closer to what Thurston seemed to be doing vs formalism.
| WhitneyLand wrote:
| So spending say 5 years, noodling around with abstract
| mental concepts and ending with symbols that turn out to be
| manipulatable and useful in a mathematical way means, you
| haven't done math until year 6 starts?
|
| I'm trying to see the line as clearly drawn as your comment
| suggests. The exact boundary doesn't seem as simple as,
| stops here / starts here.
| lisper wrote:
| > So spending say 5 years, noodling around with abstract
| mental concepts and ending with symbols that turn out to
| be manipulatable and useful in a mathematical way means,
| you haven't done math until year 6 starts?
|
| Yes, that's right. Until you write down the symbols you
| quite literally haven't done the math.
| dwohnitmok wrote:
| "the quantities that they correspond to are called 'numbers'."
| is rather circular here no? You haven't really defined
| quantities or numbers except in terms of each other and yet at
| least one of these seems important enough to be called a
| mathematical concept that is distinct from a numeral.
|
| Put another way, we clearly can distinguish many different
| numeral systems as referring to the "same thing." We can encode
| that "same thing" symbolically via a set of logical symbols to
| form axioms, but even there the same issue arises. Encoding the
| natural numbers via an embedding of second-order Peano Axioms
| in first-order ZFC or via a straight FOL Peano Axioms
| definition also seems to fall flat, as we still have not been
| able to capture the idea that "these are all really the same
| thing" since these are now different sets of symbols and rules.
| And we haven't even touched the thorny issue of what "encoding"
| and more generally "mapping" really means if everything is just
| symbols and rules.
|
| But perhaps you are fine with infinite egress (which is
| perfectly defensible). The natural numbers have a more
| distressing problem when it comes to Godel's Incompleteness
| Theorems. There is presumably only one "true" set of natural
| numbers in our universe because the natural numbers have
| physical ramifications. That is for any symbolic, FOL statement
| of the natural numbers, we can create a corresponding physical
| machine whose observable behavior is dependent on whether that
| statement is true or false. And yet by Godel's incompleteness
| theorem we can never hope to fully capture the rules and
| symbols that determine these "natural numbers," and yet
| presumably most people would agree that these natural numbers
| exist and are a valid object of mathematical study. So how does
| that fit in? What do we call the subject that studies these
| objects? What do we even call these objects if not mathematical
| numbers?
|
| (As an aside I'm perhaps more of a formalist than I let on, but
| I find the realist side a fun playground to explore in.)
| lisper wrote:
| > "the quantities that they correspond to are called
| 'numbers'." is rather circular here no?
|
| No. "Quantity" is different from "number". A quantity is a
| number plus a unit. "Two" is a number, not a quantity. "Two
| sheep" is a quantity. This is definitional progress because I
| can actually _show_ you two sheep in order to explain to you
| what _that_ means, whereas I cannot show you "two".
|
| > infinite egress
|
| It is not an infinite regress, though explaining why is much
| too long for an HN comment. It also makes an interesting
| little puzzle to figure out where and how the recursion
| bottoms out, but here's a hint: how do you know what a
| "sheep" is?
| dwohnitmok wrote:
| So then of course the question arises, what is "two?" And
| is that a mathematical object? (Shorthand for S(S(0))
| doesn't work either because S(S(0)) is clearly just another
| numeral system, not a number itself, indeed arguably any
| formalism is just a numeral system, rather than the number
| itself) And how does that comport with the reality of
| natural numbers in our universe (i.e. the incompleteness
| argument)?
|
| > how do you know what a "sheep" is?
|
| Right the usual answer is to tackle the map-territory head-
| on and say some of it is essentially "out of scope" but I'm
| curious if you had another idea.
| lisper wrote:
| > what is "two?"
|
| Two is the property shared by all of the things that
| behave the same as the collection of things which contain
| all collections of things which contain all collections
| of things which contain nothing, with respect to a set of
| rules that allow you to make collections of things that
| contain some things and not other things, and check to
| see if a given collection of things contains some things
| or no things.
|
| :-)
|
| > I'm curious if you had another idea.
|
| See:
|
| http://blog.rongarret.info/2015/03/why-some-assumptions-
| are-...
|
| This needs some revision, but it contains the essential
| idea.
| AnimalMuppet wrote:
| That "definition" rubbed me the wrong way, too. A bit
| recursive...
| Smaug123 wrote:
| I am quite happy with that "definition", not least because
| it's obviously not intended to be formal. But anyway, a very
| standard thing for a mathematics lecturer to say is "This is
| the only thing it could be", or "you know what this concept
| is already; we are just making some formal symbols to capture
| it", or words to that effect. It's pedagogically important to
| distinguish between genuinely new things that the reader is
| going to have to put some effort into rearchitecting their
| worldview around, versus things the reader already knows but
| might not have seen in this form before. Since everybody in
| this target audience already knows how to count, the
| treatment of the natural numbers is surely going to be in the
| latter category.
| lisper wrote:
| It's worse than non-formal, it's circular, and hence
| vacuous. It has the same information content as "A foo is
| an object that you build to act like the concept of a foo."
| chobytes wrote:
| Based on my reading you dont seem to fundamentally disagree
| with the author. I think hes just playing with the ambiguity of
| the word number for some effect.
|
| So: (Some mathematical formalism for) numbers are objects you
| build to act like the (informal) concept of a number.
| lisper wrote:
| No! There is nothing informal about the mathematical concept
| of a number. And numbers can be defined without resorting to
| circular definitions that include the word "number" in the
| definition.
|
| It just so happens that the symbols that denote numbers
| behave like certain things in the real world when manipulated
| according to the rules that are customarily associated with
| the symbols. Those things in the real world have properties
| that correspond to the informal idea of numbers. But that has
| absolutely nothing to do with the _math_ (except insofar as
| it 's one of the things that makes math worth doing).
| threatofrain wrote:
| "Number" is an informal concept in mathematics. The _real_
| numbers are a specific concept with multiple
| formalizations.
|
| A model of a number system is an assembly of behaviors,
| rules, or objects which behave like the numbers you want. I
| don't see the circularity here, and given that they are
| tackling an informal concept of number, I don't see any
| loss of clarity either.
| lisper wrote:
| > A number is an informal concept in mathematics.
|
| No, it isn't. You're just wrong about that.
|
| Now, it is true that there are different _kinds_ of
| numbers in math, but each different kind of number is
| formally defined. All of those definitions are
| interrelated, and collectively they define what it means
| to be a number. There is nothing informal about it.
| threatofrain wrote:
| > All of those definitions are interrelated, and
| collectively they define what it means to be a number.
| There is nothing informal about it.
|
| Then you could simply point to the formalism which unites
| various mathematical activity under some universal model
| of number. I will reiterate the claim again -- "number"
| is an informal concept among mathematicians and they are
| fine with that.
| [deleted]
| spekcular wrote:
| This site bills itself as follows:
|
| > This is a draft workshop to build from scratch the basic
| background needed to try introductory books and courses in AI and
| how to optimize their deployments.
|
| I'm not sure learning the things on that page - Peano axioms,
| basic real analysis, etc. - really helps with deploying AI. I
| know plenty of talented graduate students with papers at NeurIPS,
| ICML, and another top conferences who don't know a lick of set
| theory, so certainly it's not necessary, even for highly academic
| work.
|
| Based on my experience and what we know about the psychology of
| learning, I'd recommend finding a motivating ML project or
| application (e.g. point the phone at a sudoku and it overlays the
| solution), and then working backwards to figure out what you need
| to know to implement it. Probably it will not be the Peano
| axioms.
| YeGoblynQueenne wrote:
| Thesee are mathematics necessary for AI, as in "Artificial
| intelligence", not as in "machine learning".
|
| From a cursory look the linked page covers a broad range of
| subjects necessary to understand the last 65 years of AI
| research, not just the 9 years since the deep learning boom
| that most people are familiar with.
|
| That you know many graduate students with published papers in
| major conferences in machine learning who don't have a
| background in AI is ...unfortunate? I guess?
| amautertt wrote:
| Maybe type theory is good to know! I would like to see cutting
| edge NN libraries for Coq or Lean or Optimization ecosystem
| like Scipy or Scikit for Agda.
|
| You develop your algorithm and you prove a bunch of theorems
| about it at the same time, incredibly difficult but totally
| wild!
|
| There's some scattered projects like this
| https://github.com/OUPL/MLCert
| ipnon wrote:
| There are two schools of machine learning education: Math first
| like Andrew Ng and code first like Jeremy Howard. That both of
| these leaders have found academic and professional success
| shows you should approach machine learning in whatever way
| comes more naturally to you. Later on, you can approach it from
| the other angle to improve your understanding.
| spekcular wrote:
| To clarify, I'm not arguing that math isn't important. You
| need your linear algebra, your probability theory, and so on.
| That's non-negotiable. But set theory? Type theory (as
| someone below suggests)? I would avoid unless you're
| separately interested in mathematics for its own sake.
| chobytes wrote:
| Its not immediately practical... but spending a lot time
| learning a science like math in depth makes learning
| various applications very easy later.
|
| I think sometimes educators have to let people be motivated
| by some particular application to teach them something more
| important. Perhaps that is what the author is up to?
| exdsq wrote:
| I'm fairly certain they're not suggesting you need to know
| type theory for ML, but that theorem provers should have
| more ML libraries. My wife does some ML as a postdoc at
| Stanford and I doubt she'd even know what type theory is.
| ABeeSea wrote:
| How do you even do serious linear algebra or probability
| without set theory?
| yarky wrote:
| I agree, however I'm not sure of how far you can get in
| probability theory without set theory. For instance, I
| recall my stats prof defining a lot of sets in class, e.g.
| sigma algebras.
| spekcular wrote:
| Another point of clarification: I'm all for telling
| people what sets are. I'm less warm toward discussing
| things like ZFC, esoteric cardinality issues, and certain
| paradoxes (and large cardinals, forcing, etc.), which is
| typically what mathematicians mean when they say "set
| theory."
| actually_a_dog wrote:
| Right. What most people need to learn is "algebra of
| sets," essentially.
| medo-bear wrote:
| yes but most of the math-based approaches to ML start from
| real (multivariate) analysis, or from probability theory. i
| haven't heard anyone claim that Peano axioms are important.
| as much as I will defend the notion that applied mathematics
| is extremely important for ML research, peano axioms are
| unnecessary and will just cause confusion
| nonameiguess wrote:
| While I would agree you don't really need to know or worry much
| about mathematical foundationalism to grok machine learning,
| anyone writing graduate level papers is going to understand
| _some_ set theory since probability theory relies upon it. Even
| just the high school treatment tends to start out with
| intuitive visualizations of univariate discrete probability
| like defining a probability as the cardinality of an event
| space divided by the cardinality of a sample space, defining
| conjunction and disjunction as set intersection or set union of
| event spaces, defining mutual exclusivity as disjoint event
| spaces, and so on.
|
| That said, there is probably no reason for anyone to dig too
| deep into theory without understanding why it matters first.
| And I don't think it ever matters if you're just trying to
| solve a problem with ML. Even if you want to overlay a solution
| on an image of a Sudoku puzzle, you don't need to understand
| why a particular algorithm works in order to be able to use it.
| Theory only matters if you're either trying to formally prove
| something works or possibly to guide you down more productive
| research paths in developing new algorithms, though honestly
| I'm not even sure that's really true any more. Just gaining a
| ton of experience with various possible techniques might
| generate intuition that is as useful or even more useful than
| actually understanding anything.
|
| I almost hate saying that, as I have a deep personal love of
| math theory and think it is worth studying to me at least for
| its own sake, but this is sort of in the vein of whoever
| invented the curveball almost certainly didn't know much about
| aerodynamics and didn't need to. Discovering something does
| work may happen decades or even centuries before anyone figures
| out why it works.
| k__ wrote:
| I'm currently searching for good math resources.
|
| My girlfriend started studying CS and didn't have math for 15
| years now.
|
| It's quite hard to fill the gaps. Here some trigonometry is
| missing, there some logarithms, etc.
|
| Also, finding a good amount of exercise tasks to work through
| isn't so easy.
| civilized wrote:
| Only a math graduate student could understand this very terse,
| Cliff's Notes presentation of basic math.
|
| And the math grad student wouldn't need it, because they would
| have already learned it at an appropriate pace from much better
| sources.
| ai_ia wrote:
| The problem with these online resources are that you end up
| bookmarking them, but you are most likely never going to come
| back read them. Apologies, if I am generalising.
|
| Learning something is not only reading the course content, that
| is one part of it (of course). But developing a robust and simple
| system where you can CRUD your mental model. If you got your
| note-taking thing figured out, technically you can learn and
| should be able to learn large amounts of material with sustained
| and relatively lesser effort. Once again, apologies for going on
| a tangent.
|
| If anyone is looking to teach themselves CS along with CS Math,
| then we[1] are creating self-paced computer science courses. Our
| course content will be available as free online e-books [2] as
| well as their corresponding (paid) interactive versions and will
| be started getting released mid-November. We have two free
| courses as of now.
|
| Although it doesn't start from scratch and we assume that you
| have got atleast highschool mathematics part covered.
|
| [1]: https://primerlabs.io [2]: https://primerlabs.io/books
| bawolff wrote:
| I actually find i'm the opposite (everyone is different of
| course - to each their own). I can learn from written resources
| fairly well once i get in the flow.
|
| Interactive content or content where i have to do exercises a
| lot break my flow, because i am constantly switching gears.
| [deleted]
| ipnon wrote:
| There is another approach[0] centered on a few key textbooks. The
| goal is to prepare you for fully grokking Deep Learning[1] and
| Elements of Statistical Learning.[2]
|
| [0] https://www.dropbox.com/s/mffzmuo9fvs5j6m/Study_Guide.pdf
|
| [1] https://www.deeplearningbook.org/
|
| [2] https://web.stanford.edu/~hastie/ElemStatLearn/
___________________________________________________________________
(page generated 2021-10-19 23:00 UTC)