undefined

We know what Sir Isaac Newton’s broad opinions were on matters philosophical especially how one must study any phenomenon. What would he have said about Machine Learning?

Wondering what Isaac Newton’s true opinions on Machine Learning would have been are of course impossible but that’s the wonder of public domain.

This post is structured as if Newton were interviewed on Machine Learning and his answers are taken from his two most famous works, Philosophiæ Naturalis Principia Mathematica (or Principles of Natural Philosophy, referred to as Principia)¹ and Opticks², first published 1687 and 1704 respectively. The former is where he first introduced his Theory of Gravitation and the latter is where he presents his Theory of Light. Both books are extremely detailed and thoroughly readable. I recommend them unreservedly.

By Machine Learning I mean the entire methodological system whose underlying belief is that all matters of cognition may be understood by generalizations³ of large data sets. Want computers to identify, say, dogs, throw a few million images of “dogs” at them and it’ll then identify unlabeled images. Additionally, the field’s practitioners claim human cognition functions as merely a generalization to available data. I use the term Machine Learning in this post but it applies equally to the terms Deep Learning, Neural Networks too.

My interviewer asks pretty leading questions which might not seem like an actual interview but remember the man being interviewed is 290 years dead.

Interviewer: First, a general question. What would you say is the goal of scientists and in what manner must they proceed?

Newton: [O]ur purpose is only to trace out the quantity and properties of [any principle] from the phænomena, and to apply what we discover in some simple cases as principles, by which, in a mathematical way, we may estimate the effects thereof in more involved cases; for it would be endless and impossible to bring every particular to direct and immediate observation. [Principia, pg 638]

Interviewer: Machine Learning practitioners often argue that the phenomena they’re studying, such as vision, natural language parsing, are too difficult to study. How would you recommend we approach the study of complex phenomena?

[I] forbore to treat of [complications], because they seem[] of a more difficult Consideration, and [are] not necessary for establishing the Properties… there discoursed of. [Opticks, pg 194]

Interviewer: What is your opinion of using larger and larger sets of data to do, say, image labeling or voice transcription? Do you think it’s a good approach to discover the principles of these actions?

Nature does nothing in vain, and more is in vain when less will serve; for Nature is pleased with simplicity, and affects not the pomp of superfluous causes. [Principia, pg 479]

Interviewer: Machine Learning experts like to often say that nothing in cognition is innate and that everything may be computed as a large association map. Do you agree with this?

How came the Bodies of Animals to be contrived with so much Art, and for what ends were their several Parts? Was the Eye contrived without Skill in Opticks, and the Ear without Knowledge of Sounds? How do the Motions of the Body follow from the Will, and whence is the Instinct in Animals?… Of which things the Images only carried through the Organs of Sense into our little Sensoriums, are there seen and beheld by that which in us perceives and thinks. [Opticks, pg 370]

Interviewer: If I understand you correctly, you’re saying only light falls on our retina but that doesn’t explain how we perceive the light as Images. But how can you attribute a different “Force” (in your words) to each component of cognition?

Eggs grow from insensible Magnitudes, and change into Animals; Tadpoles into Frogs; and Worms into Flies…. And among such various and strange Transmutations, why may not Nature change Bodies into Light, and Light into Bodies?
Have not the small Particles of Bodies certain Powers, Virtues, or Forces, by which they act at a distance… upon one another for producing a great Part of the Phænomena of Nature? For it’s well known, that Bodies act one upon another by the Attractions of Gravity, Magnetism, and Electricity; and these Instances shew the Tenor and Course of Nature, and make it not improbable but that there may be more attractive Powers than these. For Nature is very consonant and conformable to her self. How these… may be perform’d, I do not… consider. [Opticks, pg 376]

Interviewer: But the Machine Learners can equally argue that theirs is a simpler system since it has fewer entities (Forces, as you might put it), and that correlation is the only Force they require as solves all their problems.

To tell us that every Species of Things is endow’d with an occult specifick Quality by which it acts and produces manifest Effects, is to tell us nothing: But to derive two or three general Principles… from Phænomena, and afterwards to tell us how the Properties and Actions of all corporeal Things follow from those manifest Principles, would be a very great step in Philosophy. [Opticks, pg 402]

Interviewer: It seems like you’re a fully materialist scientist. Do you think that all cognition comes from material objects?

The qualities of bodies, which admit neither intension nor remission of degrees, and which are found to belong to all bodies within the reach of our experiments, are to be esteemed the universal qualities of all bodies whatsoever. [Principia, pg 479]

Interviewer: Should scientists gather as much data as they can without a theory or should they propose a theory first and then attempt to generalize?

[A]ll the difficulty of philosophy seems to consist in this — from the phænomena to investigate the forces of nature, and then from these forces to demonstrate… other phænomena…. Then from these forces, by other propositions which are also mathematical, we deduce [more and more phenomena]…. I hope the principles [I have] laid down will afford some light either to this or some truer method of philosophy. [Principia, pg 55]

Interviewer: So would you generalize to all on the basis of experiments? Can you give an example?

The extension, hardness, impenetrability, mobility… of the whole, result from the extension, hardness, impenetrability, mobility… of the parts; and thence we conclude the least particles of all bodies to be also all extended, and hard and impenetrable, and moveable…. And this is the foundation of all philosophy. [Principia, pg 480] [emphasis added]

Interviewer: An example from science, Mr. Newton.

In these computations [of the motion of the moon, its relationship with the earth and sun] I [did] not consider the magnetic attraction of the earth, whose quantity is very small and unknown: if this quantity should ever be found out, and… we should then be enabled to bring this calculation to a greater accuracy. [Principia, pg 567]

Interviewer: People say your methods ignore the full complexity of the object under study. How do you defend against this charge?

[I]n philosophical disquisitions, we ought to abstract from our senses, and consider things themselves, distinct from what are only sensible measures of them…. [B]ecause… [some] parts… cannot be seen, or distinguished from one another by our senses, therefore in their stead we use sensible measures of them. And… instead of absolute[s], we use relative [values]; and that without any inconvenience in common affairs. [Principia, pg 65]

Interviewer: It would seem from your arguments that you attribute to bodies occult properties and that you attribute a different force to explain each phenomenon. Is this an accurate representation?

It seems to me… that… Particles have… only [properties]… accompanied with such passive Laws…, but also that they are moved by certain active Principles, such as is that of Gravity, and that which causes Fermentation, and the Cohesion of Bodies. These Principles I consider, not as occult Qualities, supposed to result from the specifick Forms of Things, but as general Laws of Nature, by which the Things themselves are form’d; their Truth appearing to us by Phænomena, though their Causes be not yet discover’d. For these are manifest Qualities, and their Causes only are occult. 
Things [that] follow from [these] manifest Principles, would be a very great step in Philosophy, though the Causes of those Principles were not yet discover’d: And therefore I scruple not to propose the Principles… they being of very general Extent, and leave their Causes to be found out. [Opticks, pg 402]

Interviewer: On what basis can you say your theory of gravitation is correct?

[O]bservations agree with the theory, so far as they agree with one another… the theory which justly corresponds with a motion so unequable, and through so great a part of the heavens, which observes the same laws with the theory of the planets, and which accurately agrees with accurate astronomical observations, cannot be otherwise than true….
I have given a true representation of the orbit which this comet described [that perfectly matches my theory], and of the tail which it emitted in several places. [Principia, pg 605]

Interviewer: Machine Learning aficionados would argue that their techniques are fairly successful if not perfect. Google Photos correctly tags motorcycles as motorcycles, cars as cars, and no longer tags black people as gorillas. Wouldn’t you say this too is a sign of progress?

[I]n Mathematicks [and] in Natural Philosophy, the Investigation of difficult Things by the Method of Analysis, ought ever to precede the Method of Composition. This Analysis consists in making Experiments and Observations, and in drawing general Conclusions from them by Induction, and admitting of no Objections against the Conclusions, but such as are taken from Experiments, or other certain Truths. [Opticks, pg 405]

Interviewer: But what if an experiment fails?

[I]f at any time… any Exception shall occur from Experiments, it may then begin to be pronounced with such Exceptions as occur. By this way of Analysis we may proceed [for example, in Chemstry] from Compounds to Ingredients, and [in physics] from Motions to the Forces producing them; and in general, from Effects to their Causes, and from particular Causes to more general ones, till the Argument end in the most general.
This is the Method of Analysis: And the Synthesis consists in assuming the Causes discover’d, and establish’d as Principles, and by them explaining the Phænomena proceeding from them, and proving the Explanations. [Opticks, pg 405]

Interviewer: In ML, when they want the computer to recognize, say, wolves, they provide the computer with several images labeled “wolf” and it somewhat works. You’re saying, if I understand you correctly, that this is not the right approach to discover recognition itself. Can you provide an example from your career where you faced a complex scenario and how you simplified it?

Let us imagine a system of lesser bodies revolving about a very great one [that would] be impelled sideways by the force of another vastly greater body situate[d] at a great distance. And because the equal accelerative forces with which the bodies are impelled in parallel directions do not change the situation of the bodies with respect to each other…. Suppose, therefore, all the accelerative attractions made towards the great body; … and then [I made several more assumptions to prove my case and finally my system]… will describe areas proportional to the times, without any errors but those which arise from the distances of the parts, which are by the supposition exceedingly small, and may be diminished at pleasure.
By a like reasoning one may proceed to more compounded cases in infinitum. [Principia, pg 241] [emphasis in original]

Interviewer: By a like reasoning one may proceed to more compounded cases in infinitum. That’s a good note to finish on. Thank you so much, Mr. Newton.

Nature and Nature’s Laws lay hid in night:
God said, “Let Newton be,” and all was light.
 — Alexander Pope (epitaph intended for Newton)

Footnotes

¹ The version I am using is the translation by Andrew Motte, published in 1728. I downloaded the book PDF from https://www.globalgreyebooks.com/philosophiae-naturalis-principia-mathematica-ebook/. I paste below the website maintainer’s words:

It took me about 10 hours to make this book so you could download it within seconds.
However, please remember I am but one woman trying to make a living with this site. So I expect my customers to be customers and help me out a little. Or a lot. You get the drift. Pay me…please.

² I’m using the Fourth Edition of Opticks, published in 1730, two years after Newton’s death in 1728. Available online on Gutenberg at http://www.gutenberg.org/files/33504/33504-h/33504-h.htm.

³ Note that “generalization” is never once defined and always presumed. This is an epistemological problem which ML scientists have, as best as I can tell, never addressed.

The cover image is a composite of 3 images taken from Wikipedia.