Peter Norvig (Director of Research Google)
He went over the last part of his 2007 Singularity Summit talk
Probabilistic First Order Logic (thought he needed it in 2007 but upon review not needed)
Hierarchical Representation and Problem Solving (yes, progress made and needed it)
Learning over the above (yes)
Lots of data (yes, used 10 million videos and photos from Youtube)
Online (yes, this is partial – they loaded it offline and used it online)
Efficiently (yes)
Important research for the automatic image categorization work
Google trained a picture identifying system with 10 million youtube videos
Tens of thousands of nodes. Each node identified a particular thing.
Cats, people, yellow flowers etc..
Sparse Coding (Olhauser and Field 1996)
Deep Belief Networks (Hinton, Osindero and Teh, 2006, 16 pages)
Hinton tutorial on Deep belief networks (100 pages)
Peter Norvig answered questions about AI on Reddit back in 2010
They were able to get the eyes and noses identified on faces, and key features of cars and chairs
Google put together the deep learning team
Andrew Ng
Geoffrey Hinton
Jeff Dean (the goto guy for scaling solutions to warehouse data center)
16000 CPUs, 1000 servers,
1 billion parameters, 200 by 200 pixels (100 times bigger than best prior solution, but 100 to 1000 times less than people)
Trained the picture identifying system with 10 million youtube videos
Tens of thousands of nodes. Each node identified a particular thing.
Cats, people, yellow flowers etc..
They are not duplicating the brain
They applied deep learning to speech recognition and got a 37% error reduction. This is largest error reduction in a number of years and is a lot of progress.
This is hierarchical, learning, unsupervised and using lots of data.
Machine Language translation
They also have machine translation of languages (ie German to English)
Find millions of translated language pairs.
Phrases match to phrases and then use jigsaw puzzle solving techniques
Three submodels are combined
Translation model, target model (expected word counts in the target language), distortion model
Optimize a formula that combines the functions
A Successful Online AI Course
Sebastian Thrun and Peter Norvig
160,000 students in 200 countries
Try to get closer to 1 on 1 tutoring as a goal
Education is the Afghanistan of Technology. Technologies try to conquer education but then fail over 10 years.
Thomas Edison predicted that films used in education would replace books in 10 years.
There needs to be feedback between the teacher and the student and feedback to the student.
Where to go from here
Need to work with more than still images of cats but use full video and other inputs
Computer vision as a field has gone away from 3D to 2D
Many 2D images from many angles are an approximation of 3D
Cannot handle deformable objects like jellyfish
They have plateaued by just looking at words
They now add syntax, clustering words, endings of words, word classes, more hierarchical clustering
Software and algorithmic advances will be more important than hardware for the next 5-10 years at least.
If you liked this article, please give it a quick review on ycombinator or StumbleUpon. Thanks
Brian Wang is a Futurist Thought Leader and a popular Science blogger with 1 million readers per month. His blog Nextbigfuture.com is ranked #1 Science News Blog. It covers many disruptive technology and trends including Space, Robotics, Artificial Intelligence, Medicine, Anti-aging Biotechnology, and Nanotechnology.
Known for identifying cutting edge technologies, he is currently a Co-Founder of a startup and fundraiser for high potential early-stage companies. He is the Head of Research for Allocations for deep technology investments and an Angel Investor at Space Angels.
A frequent speaker at corporations, he has been a TEDx speaker, a Singularity University speaker and guest at numerous interviews for radio and podcasts. He is open to public speaking and advising engagements.