<table><tr><th>From:</th><th>Joscha Bach</th></tr><tr><td>Sent:
To:</td><td>Wednesday, March 1, 2017 7:29 PM
Jeffrey Epstein</td></tr><tr><td>Cc:</td><td>Barnaby Marsh</td></tr><tr><td>Subject:</td><td>Re:</td></tr></table>

Thank you, Jeffrey! This is from Noam, right? I would be very interested =n reading the responses of linguists and computational language =odelers to this. May I forward it to a friend at Google X?

Some notes:

> basic assumptions about human language that should I think be =ncontroversial, extensively discussed elsewhere, then turning to a =ample of challenges. A person's language is an internal =ystem, part of human biology, based on configurations of the brain, =eveloping in each person through interaction of specific biological =ndowment (the topic of UG — universal grammar in contemporary =erminology), external environment, and general properties and =rinciples of growth and development that are independent of language. = As far as I understand, there is not yet an agreement among linguists =rt. UG, i.e. how much is innate vs. do humans just converge on the =implest type 3 grammar that is consistent with the constraints they =bserve in their local environment. I think Noam argues that we have =ery specific circuitry for language, whereas the other camp would =uggest that we are general learners, with specific rewards that bias us =owards compositionality and systematicity. OTOH, this might also be =ead as a variant of Noam's "Strong Minimalist ThesisQ=9D (SMT). The controversy will be eventually resolved by progress in building =ystems that learn natural language.

> The acquired system is an "internal language" =I-language), a computational system that yields an infinite array of =ierarchically structured expressions that are interpreted at the =onceptual-intentional CI interface as, in effect, a "language =f thought" (LOT), and that can be externalized to one or =nother sensorymotor system, typically sound. Also relevant are some =onsiderations about evolution of language.

>

> Little is known about the evolution of cognitive faculties, a matter =iscussed in an important article by Richard Lewontin, whose own view =or the prospects was dim

Most folks in cognitive science would probably agree that most cortical =ctivity is devoted to building a generative simulation of the outside =orld by a process of hierarchical learning. These simulations can be =apped on a conceptual manifold, something like an address space of our =ensory motor representations of the world, which we can use to evoke =nd shape our mental simulations. Language is our interface to that =onceptual space, and external language allows us to synchronize =oncepts even in the absence of matching sensory motor representations, =.e. we can build mental simulations of things that we never =xperienced, by interpolating between concepts that address mental =imulations we know.

It seems that Noam's approach is unique in that he focuses =ntirely on language and concepts, while treating the understanding of =he underlying cognitive faculties as hopeless, while many others would =rgue that understanding language without first understanding =re-linguistic mental representations might be impossible.

That said, Noam's characterization of I-language and LOT at the =E24140conceptual-intentional" interface, with an =xternalization through generative mechanisms, is probably a useful =asis, regardless of where individual researchers come from.

> [i] Anatomically modern humans (AMH) appear about 200 thousand years =go.

>

> [ii] The faculty of language FL appears to be a true species property: =hared among human groups (with limited individual differences) and in =11 essential respects, unique to humans. In particular, there is no =eaningful evidence for existence of language prior to AMH.

>

> [iii] Recent genomic studies indicate that some human groups (San =eople) separated from other humans about 150kya. As far as we know, =hey share FL with other human groups.

>

> [iv] The San languages are all and only those that have the curious =roperty of phonetic clicks, and there may be some articulatory =daptation to producing them (See Huijbregts, forthcoming).

Nguni languages have clicks, too, but they seem to have imported them =rom Khoisan.

> [v] The first (very limited) indication of some form of symbolic =ehavior appears at about 75kya. Not long after that, we have rich =vidence of quite extraordinary creative achievements (Lascaux, etc.).

This is consistent with another observation: Modern humans had a =opulation bottleneck of 2000-3000 individuals ca 75000 years ago, which =oincides with the Tonga eruption. This does not necessarily mean that =he volcano killed off almost all hominids, but it increased the =volutionary pressure, and it is possible that our ancestors evolved a =utation that enabled them to outcompete and kill most of the hominid =ompetition (including Neanderthals). What if that mutation is something =hat roughly translates into "symbolic behavior"?

I currently think that much of our civilization might be the result of a series of quite specific mutations. Our ancestors went from 3000 =ndividuals to one million and remained there until they developed =eligions. Religion and other ideologies are based on a need for =onformance to internalized norms, i.e. an innate desire to serve as =art of a system that is larger than the individual's reputation based =roup. They were also based on a shared conceptual space.

Challenge 1 seems mostly to amount to: verify that 1. all human groups =ave language, and 2. there is no grammatical non-human language. One of =he interesting questions might be if dolphins have grammatical =anguage, another one concerns the limits of learning in non-human =rimates. The challenge is completely empirical.

Challenge 2 seems very exciting to me; I read it as: has language =ntrinsically linear order, or is that only imposed by the =equentialization of articulation? Grammatical language has a tree =tructure, and the tree seems to be created probabilistically in the =istener, from a string of discrete symbols. Would natural language be =earnable without the constraints of sequentiality and discreteness?

Challenge 3: do we need externalization to learn and process language? I =ould suspect that an individual can play a language game against itself =ntil it converges on its own language, but it is not clear that humans =re among the class of individuals that can do that from scratch. Most =esearch suggests that there is a critical window in which we must pick =p our first language for perfect fluency, and there seems to be no =vidence of entirely individualistic acquisition/formation of a first =anguage. If that is true, is that a constraint of the way language =earning is implemented in the human brain, or a complexity constraint =ithin language itself?

It seems to be clear that learning a programming language changes the =ay we think, i.e. it provides evidence for a weak version of the Sapir =horf hypothesis. But that is not so much a constraint of =xternalization, but of the semantic structures addressed by the =anguage.

I imagine that pure work in a computer science lab can make some =nteresting progress on challenges 2 and 3.

Challenge 4: I don't understand enough about the context to see =he significance yet; I would think that once we have an SMT model of =anguage formation, we can learn additional operations that perform =perations on the generated

mental representation, based on arbitrary =ignals. This may require us to leave an approach that attempts to =andbox language from general cognition, but why would we want to =onstrain SMT based models by such a sandbox?

Challenge 5: Again, I don't understand enough of the context to =nderstand why probabilistic interpretation cannot fill in the gaps. A =robabilistic model will weight alternatives, and the binary Merge is =he simplest, preferred case?

Challenge 6: The question of the structure of individual lexical items =ight require a perspective that integrates mental representations =eyond language/SMT.

Challenge 7: Do semantic atoms refer to the external world ="referential doctrine")? — This seems to be =uite clearly false; they refer to representations in the neocortex that =re mutable and acquired through learning (structure or reinforcement) =nd inference.

Challenge 8: Noam seems to agree with my take on 7. How are semantic =tems acquired? — This challenge comes down to the general =roblems of learning and perception, i.e. pretty much everything in =ognitive science outside of language! Challenge 8 seems to be designed =y a rocket scientist who specializes in combustion chambers and leaves =11 other parts of getting the rocket to fly as an exercise to their =rad student...

Challenge 9: Noam suggests that meaning must be derived from innate =nformation, and wants to study universals between language to identify =he innate bits. However, it is not clear if they do not stem from the =roperties of mathematics, i.e. there is a limited space of "usefu= simple axiomatic systems" that can be individually explored by =earning systems. Kant attempted to describe this space, identified it =s apriori and synthetic, and listed the basic structural categories =hat we would use to characterize the world. Sowa and a few others have =ade contributions to basic ontologies, and perhaps it is time to =evisit Kant's project?

Challenge 10: Do music, planning, arithmetic stem from language, or do =11 result from a shared innovation of modern hominid brains?— =bviously, different answers in that space might be possible, for =nstance music could be a parasitic byproduct of rewards for discovering =ompositional representations that our brain needs to make us interested =n learning grammar, while basic planning is independent, and complex =lanning needs language for structuring and operating on the conceptual =pace. This makes the question extremely general.

It also gives rise to the more general question of what exactly makes =omo sapiens different from the other chimpanzees. I suspect that our =rains are trained layer by layer, whereby each layer has a time of high =lasticity during its primary training phase, then undergoes synaptic =runing, and has low plasticity later on. The duration of the training =hases is regulated by genetic switches. Increasing the duration will =xtend infancy and childhood (i.e. increase the cost of upbringing), but =ive each layer more training data. Perhaps humans outperform other apes =ecause they get a magnitude more training data before their brains lose =nfant plasticity, which results in dramatically better ability to =eneralize and abstract?

Challenge 11: Rare constructions can be understood by children, and thus =here should be a mechanism to derive them from more simple rules, =espite apparent evidence to the contrary, which should be explained =away].

Challenge 12: Noam suggests that the complexity of most constructions in =he face of "poverty of stimuli" means that (- languages =re 1. very similar, 2. differences result from externalization, 3. =hould therefore stem from UG. He wants this shown, or an alternative.

An alternative explanation might be that the space of possible human =rammars is small enough to allow rapid =volutionary-engineering convergence, UG, and but in polyglots an apriori =ven of the allow mathematics for a complete of human mapping. =rammars. That would not be a property of an

Challenge 13: What small change in a brain could lead to the unique =ognitive abilities of homo sapiens, including language? — There =re a lot of different hypotheses of this, among them what I suggest in =10), and differential

attention/reward for learning compositional =tructures, or several successive modifications in the reward system. I =hink that Noam suspects that the culprit is a new connective pathway, =erhaps somewhat similar to Julian Jayne's Bicameral Mind =ypothesis?

These challenges are extremely inspiring food for thought!

Bests,

Joscha

Am Mar 1, 2017 um 7:01 AM schrieb jeffrey E. <jeevacation@gmail.com>:
> 
> <Challenges Language 2-17.docx>

<?xml version=.0" encoding=TF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version=.0">
<dict>
<key>date-last-viewed</key>
<integer>0</integer>
<key>date-received</key>
<integer>1488396562</integer>
<key>flags</key>
<integer>8590195717</integer>
<key>gmail-label-ids</key>
<array>
<integer>5</integer>
<integer>18</integer>
</array>
<key>remote-id</key>
<string>692309</string>
</dict>
</plist>

4