As I mentioned in the class today, for the longest time, and by that I
mean, until well into late 50's, the conventional scientific wisdom
was that infants come into this world with a "Tabula Rasa" (blank slate) mind, and pick up everything by learning (supervised or
reinforced) and observation. The reigning doctrine was
"behaviorism"--you can condition/reinforce any behavior into any
organism. To behaviorists, children were but cuter (pavlovian) rats, and language acquisition was no different than a acquisition of maze
following skills. B.F. Skinner was the leading exponent of behaviorism
and was, in early fifties, writing book after book expounding on how
behaviorism can explain all sorts of human behavior.
[Skinner was such an ardent behaviorist that there was even an
apocryphal urban legend that said he raised his own daughter in a
"skinner box" to test his behaviorism hypotheses--see
http://www.snopes.com/science/skinner.htm ]
When Skinner came around to applying behaviorism explanations to
language acquisition and wrote the book on "Verbal Behavior", it was expected to further shore up the behaviorism doctrine, and become a
classic. What became a classic instead is a critical scholarly 1959
"review" of the book by a then little-known linguist named Noam
Chomsky (
http://cogprints.ecs.soton.ac.uk/archive/00001148/00/chomsky.htm ).
Chomsky essentially killed the book as well as much of the euphoria of behaviorism by an arugment that has since then come to be known as
the "poverty of stimulus" argument. He said that behaviorism and
stimulus/response reinforcement does not quite explain how it is that children seem to be able to generate sentences that they have never
heard of. In other words, there are not enough examples (hence a
poverty of "stimuli") for the children to learn entire
language--grammar and sentences together (even for children--such as mine--with overly talkative parents ;-) [You note that the argument
that something cannot be learned is being done in terms of the
the inordinate number of examples needed to learn it. As we saw in
the class, difficulty of learning tasks is measured in terms of "sample complexity".]
As an alternative explanation, Chomsky cited his own work on
"generative grammars"--a set of grammar rules that can generate
"grammatically correct" sentences from a language. He said that it must be the case that children come into the world with grammar rules
already in their head. Since the grammars of different world languages
are different, he posited that the children come into this world with
a "universal" grammar. Using the language being spoken around them,
they then set the "parameters" (or knobs, if you will) on their
universal grammar such that it becomes customized to the specific language environment they are in. Once they have the customized
grammar, they then are in the business of learning word sense (or
semantics). "Colorless green ideas sleep furiously" is one of
Chomsky's famous examples, which he uses to show that even small kids
with limited vocabularies can tell automatically whether a sentence is
grammatically correct. [Even here, in learning semantics, children
come into the world with pretty strong biases--including the so-called
"whole object hypothesis". If I point towards a big moving bus and say
"Bus", the kid hypothesizes that the whole big thing is called
bus--not just the wheels, or the hubcaps, or some subset of the
parts. Pretty strong bias, come to think of it--what if I said
"Driver" pointing towards the driver side of the bus?]
Chomsky of course went on to become the demi-god of cognitive science
in general, and mathematical linguistics in particular (and you obviously heard of him in your CSE 355, when you probably learned
about the Chomskian hierarchy of languages--which is in terms of their
grammar complexity). A lot of research has been done since Chomsky's
original work, to shore up the support for the universal grammar
hypothesis. It is so much of an accepted fact (dogma) now that it
(universal grammar) in turn is seen as yet another evidence that all
humans evolved from a common set of ancestors--as against evolving
separately and independently (the "Lucy" theory,
http://www.pbs.org/wgbh/evolution/library/07/1/l_071_01.html ; by the
way Don Johanson, who found Lucy skeleton in Ethiopia, is right here
at ASU--check out http://www.asu.edu/clas/iho/dcj.html ). The basic
argument is that the rank similarity of the human languages cannot be
explained without it. (Of course, there are much stronger arguments for the common ancestor theory--including the fact that we are all
same species--any man from anywhere in the world can mate with any
woman from anywhere in the world and produce healthy offspring).
So that is some of the background on the universal grammar. By the way, note that none of the above says that conditioning will not be
effective in changing ones behavior--you probably saw the recent press
accounts of the infamous Wendell Johnson orphan stuttering experiments
(http://www.jsonline.com/news/nat/jun01/stutter11061001.asp). All
Chomsky's argument says is that conditioning and reinforcement are
only part of the story and cannot alone explain language acquisition; evolution did a whole other part too.
Now for a couple of references. Probably the best-written lay-person
book on human language acquisition is Steven Pinker's "Language
Instinct"( http://www.amazon.com/exec/obidos/tg/detail/-/0060976519/104-1220170-3641559).
A very nice and eminently watchable 3-part PBS series on language
acquisition is "Human Language"
( http://equinoxfilms.home.mindspring.com/HLseries.html).
That is all for now. Now for more important things like Seinfeld rerun
already in progress.
Rao
Epilogue/Postscript: These days of course, a search on Chomsky on the
web is more likely to turn up references to his political views than
his linguistic ones. He is sort of a one-man vocal and loyal
opposition for many of the US government policies. For example, he
wrote one of the first dissenting "non-party line" opinions of the 9/11. Whether I agree with him or not, I am grateful that there is
someone unafraid of speaking his mind--especially in these days of
hyper-patriotism, where FBI thinks it normal to monitor you because
you are against war, and your freedoms are being re-defined as the ones Ashcroft hasn't yet gotten around to excising into the "Patriot
Act".
----------------
"I simply want to tell you that there are some men in this world who
were born to do our unpleasant jobs for us. Your father's one of
them."
[..] "I always thought Maycomb folks were the best folks in the world,
least that's what they seemed like."
"We're the safest folks in the world" said Miss Maudie. "We're so
rarely called on to be Christians, but when we are, we've got men like
Atticus to go for us."
--Miss Maudie and Jem talking about Atticus Finch, as
Scout and Dill look on..
============
I would like to clarify how the points will be awarded for extra credit in Phase 3. You can pick up a maximum of 10 points as extra credit, and it will be graded on a curve. You may attempt any number of extra credit problems from the problem statement, but please remember that points are awarded for a thorough analysis. Doing a little bit of three different extra credit questions will gain you fewer points than doing one problem thoroughly.
Thanks and Regards,
Sushovan De
Hello Dr. Rao,I have a question not from today's class but on the topic "need for smoothing" in classification.Why do we consider that if a coin comes up as Heads for 2 million times out of the 3 million tosses, then it should be a biased coin ?Should we not think otherwise that the coin is not actually biased and is just having a bad day because we know for sure that the probability of Heads/Tails if 0.5.--Shaunak ShahOn Tue, Nov 8, 2011 at 7:10 PM, Subbarao Kambhampati <rao@asu.edu> wrote:
folksI couldn't call on several raised hands today. If you remember your questions, feel free to ask me by email andI will respond.(and if I think the question is of general interest I will post my answer to the blog too)rao
If there is a reserve price, isnt it indicating a true value of item? Like in India the tenders are filled in same way (closed sealed, ignoring corruption for a while). The governments sets the reserve price, people bid and the one who is willing to pay highest price is given the project, with the quotation price he gave.
Shubhendra SinghTo: shubhendra singh <Shubhendra.Singh@asu.edu>
Sent: Tuesday, 8 November 2011 7:28 PM
Subject: Re: questions about today's classI assume you are talking about "reserve price"--i.e., a price below which the you don't want to sell the item.The way to handle reserve prices is to assume that the seller (in the case of search ads, the search enginer). is also bidding--with the minimum (reserve) prices.If the seller wins, then basically that means there are no bids from others that the seller is willing to take (and so basically the item is not sold).RaoOn Tue, Nov 8, 2011 at 7:23 PM, shubhendra singh <Shubhendra.Singh@asu.edu> wrote:In vickrey second auction, is minimum pricing of an item done? I mean for lets say query/keywords, we know the number of hits, we (thinking of we as search engine), we already know how important this query is, Do we decide the minimum quotation for it?
Shubhendra Singh
From: Subbarao Kambhampati <rao@asu.edu>
To: Rao Kambhampati <rao@asu.edu>
Sent: Tuesday, 8 November 2011 7:10 PM
Subject: questions about today's class
folksI couldn't call on several raised hands today. If you remember your questions, feel free to ask me by email andI will respond.(and if I think the question is of general interest I will post my answer to the blog too)rao
In vickrey second auction, is minimum pricing of an item done? I mean for lets say query/keywords, we know the number of hits, we (thinking of we as search engine), we already know how important this query is, Do we decide the minimum quotation for it?
Shubhendra Singh
From: Subbarao Kambhampati <rao@asu.edu>
To: Rao Kambhampati <rao@asu.edu>
Sent: Tuesday, 8 November 2011 7:10 PM
Subject: questions about today's class
folksI couldn't call on several raised hands today. If you remember your questions, feel free to ask me by email andI will respond.(and if I think the question is of general interest I will post my answer to the blog too)rao
HW4 question 1 says:
Show the cluster dissimilarity measure (which is defined as the sum of the similarities of docs from their respective cluster centers).
It should read “the sum of the dissimilarities”.
Thanks to the two of you who pointed this out.
Thanks and Regards,
Sushovan De
Video of the lecture video part 1 (the first 1hr 5min 4gb) and video part 2 (the remaining 10+ min) Efficient computation of pagerank (and how it is important to not represent the M* matrix explicitly given that it is not sparse); doing block-based pagerank iteration to avoid thrashing, the use of asynchronous pagerank iteration to improve convergence. Video of the lecture video part 1 (the first 1hr 5min 4gb) and video part 2 (the remaining 10+ min) Milgram experiment; six-degrees of separation; (uniform) random networks and their properties; realizing that the small world probability increases sharply to 1 right near k=1; where k is the average (expected) degree of the random network. If human networks are (uniform) random, then they will have small-world phenomena (since k, i.e., average number of friends per person, is almost always greater than 1). Trying to confirm whether large-scale social networks are in fact uniform random by comparing their degree distribution to the Poisson degree distribution expected for random networks. Realizing that most real world network degree distributions instead correspond to negative sloping straightlines in log-log space (which means they are of the form P=1/k^r, which is called powerlaws. Discussion of the properties of power-law disributions (which have long tails that fall off only polynomially rather than exponentially). Implications of long tails on everything from probability of existence of such highly-linked sites as google to the ability of making money selling west wing DVDs and iranian classical music CDs on the web. Discussion of generative models which can result in power law distributions over network degrees.Video of the lecture video part 1 (the first 1hr; battery died after that :( ) Attacks vs. disruptions on powerlaw vs. exponential networks; navigation on social networks; applications of social networks; discussion of trust and reputation; trust rank (a page-rank variant); discussion of social search (and aardvark); discussion of othe powerlaws in cse494--zipf's law; heap's law (and even benford's law).| | |
| | |
| | |
| 88.40 | A+ |
| 79.57 | A |
| 62.83 | B |
| | |
| 91.97 | A+ |
| 90.61 | A+ |
| 87.55 | A+ |
| 85.76 | A |
| 85.11 | A |
| 84.34 | A |
| 84.00 | A |
| 83.88 | A |
| 83.85 | A |
| 81.27 | A- |
| 79.92 | B+ |
| 78.76 | B+ |
| 78.63 | B+ |
| 74.76 | B |
| 69.64 | B |
| 69.35 | B |
