Panel 2: Inflection Points

Speculate on what we most need in 5 years and make radical changes

Candace Sidner

Natural language problems
The most natural way for human beings to communicate
Interfaces will be ubiquitous, how to talk to them?

* Problems in Creating Human-usable Interfaces

• Semantics of NL needs work (and hence KR that works with such semantics)
• Negotiating tasks, beliefs, and meaning
• Speech: models of conversation and prosody
• Body and faces: we face-to-face and our machines can too

* Semantics is hard and hasn't been worked on enough [ref. Jerry Hobbs]

• John shot Bill
  • he died in the street
  • he died in the hospital
  • (implication of time)
• "A jogger was hit on the road last night" vs. "A professor was hit on the road last night"
  • (the assumption of jogging)
• It rained on Saturday. We didn't go hiking.
  • (implication of what we know about the world)

Addressing semantics without KR is doomed to failure (and vice versa)

Semantics imply deeper relations for which we don't have good models(causality). Needs work in both KR and semantics hand-in-hand. Learning semantics is really a challenge

* Negotiation in Interaction

• We negotiate everything
• Beliefs, meanings
• Computers will have different beliefs from us
• Machines have to do tasks for us, so we have to care about negotiation

* Conversational Speech

• The current models of speech recognition are hopeless when it comes to conversation
• Expecting people to adapt their linguistic behavior within the current limits of speech recognition impedes the use of conversation
• Prosody in conversation is essential to understanding what participants say and mean
• Progress is critical

* Embodied Conversational Agents

• Face-to-face conversation
• Gestures: eyes, face, head, hands, body
• Social awareness: relationship to other, social interaction
• Emotional expression in linguistic and facial/body expression
  • Mel the robot penguin
  • Pelachaud's Greta
  • Bickmore's Laura
  • JAST project robot
  • Cassen's Sam the Castleman

When people have social conversation with agents, they react differently

Artificial Intelligence meets Brain Science

Tom Mitchell

* AI meets brain science: An inflection Point for AI?

Progress in studying the brain is accelerating

• new brain imaging methods
• direct cell recordings
• generic profiling and knock-out experiments
• cell-level biology

Brain has already solved AI -- How can AI ride (and drive) the wave of progress in brain science?

fMRI experiment: the active region in the brain -- spatial resolution

Examples to show synergy between AI and Brain Science:

• Oscilloscope to look inside human brain
• Spatial resolution of brain fMRI as a person sees checkerboard. ERP time course with millisecond precision, diagrams of wiring in brains (fluid flow)

* Human Brain Imaging

• fMRI location with millimeter precision
• ERP time course with millisecond precision
• DTI connections tracing millimeter precision

Correspondence between AI and brain science

* Reinforcement learning in the brain

• Reinforcement Learning [ref. Sutton and Barto, 1981, Samual, 1957]
• Dopamine as Reward Signal (light and sugar experiment) choose actions for distant rewards. When humans also get reward and do actions for a contemplated reward. [ref. Schultz et al. 1997] [ref. Seymore et al, 2004]

* Knowledge representation in the brain

• Mean Activation per Voxel for Word Categories
• Where does the brain representation differ between "tools" versus "Dwellings"?
• Space localization different for different words.
• If we can tell the signature, we can ask what will happen if we put the two words into a pattern/sentence?
• [ref. Anderson, Qin, and Sohn, 2002]

AI has contributed to Brain Science, but there will be a shift.

The End of Knowledge Engineering

James Hendler

* mid 90s AI knowledge goals

• DARPA HPKB/RKF
• getting computers to do things, level of abstraction
• master in the area, not the techniques
• HighPerf kbase: how to get subject matter guy to do it than doing by AI person SME can do stuff without AI experiments
• goals: SMEs can build "100k axiom" KBs with minimal help from AI experts

* Early 00's KE goals

• DARPA Agent Markup Language
• AI representation on the web: as annotation to existing resources, a web of KR
• Hundreds of ontologies (100k = 300 ontologies of 300 + classes)
• how to use stuff out there without rebuilding it
• AI rep. on web: as annotation of existing res. Hundreds of ontologies.
• Everyone is an SME on something: who is the expert on your family? on your job? etc.
• things people spend on the web is info on the stuff they care about

* Where we are today

• the semantic web
• survey of 1300 owl ontologies found by crawl
• 19 ontologies with 2000+ classes, 6 ontologies with 10000+ classes, 2 ontologies with 50000+ classes (CYC, NCI)
• Gazillions of bits of "semi-engineered" knowledge
  • flickr: tens of millions of keyword tagged photos
  • wikipedia: thousands of carefully documented subjects (in a hierarchy, with disambiguation)
• With "persistent" URIs
• these things don't have ambiguities ("tank" as armament vs. "tank" as the name of a city)

* Where we are soon

• Linking of web 2.0 and semantic web
  • using informal KE to bootstrap "formal" KE
  • extending formal KE from web 2.0

* Inflection point

• We are moving towards a time here we will be able to have 100000 ontologies, not 100000 axioms
  - available on the web (linked to web resources, data resources, each other, web2.0like annotations)
• And billions of annotated (semi-KE) objects
  - available on the web (linked to web resources, data resources, each other, ontologies)
• The end of the KE bottleneck

* Inflection point challenges

• Many characteristic of the web violate traditional KR assumptions
• But, similar to human intelligence, many characteristics in humans violate fundamental AI assumptions

* Opportunities

This inflection point should help us face up to some long-ignored KR challenges

Inflections in Robotics

Rodey Brooks

Today's deployed robots are navigation machines
When robots can manipulate things, it will be inflection

All types of navigation robots: robots in Iraq/Afghanistan, carries sensors, sends data to human, makes decision

* Levels of Autonomy and "unstructuredness" in Navigation
trains, cars, roombas, manipulation only in structured only, nothing like car or roomba

When things go wrong, things really go wrong

* Mobile robots back in 1977-79

• Stanford Cart
• mobile robots, 20m in 6hrs
• very fast car

* Navigation then and Now

*What changed in Navigation?

• change in mindset
  • path planning (then), perceptual inputs
  • inverse problems
• change in problem
  • model where stuff is not rather than where stuff is
• change in domain
• change in sensors
• change in computer power

* conventional factory "manipulators"

• the grasps and motions are pre-planned
• there is structure to the environment
• there are limits to how much things can change
• there is extraordinary engineering overhead in each application

* changing manipulation

• conventional academic manipulation solves inverse problems
• newer approaches used forward methods
• force sensing in the arms
• many complementary sensors on the hand, lot of computation power
• many orthogonal things many solutions

* touch-based grasping

* safe for human interaction

* two revolutions

• then: large corporate back room operation, automation slow and by specialized engineers
• now: personal machines, office workers, automate their own work and increase their own productivity

* The immediate challenges for AI

• visual object recognition capabilities of a two year old child
• manual dexterity of a six year old child
• language capabilities of a four year old child
• social sophistication of a ten year old child

Possible Inflection Points

Randall Davis

Enabling natural interaction
Building robust systems
Global scale knowledge bases
Human intelligence as a natural phenomenon

* natural interaction in the design context

• noise, disfluency
• 2D non-chronological signal (speech is chronological)
• understand teaching... multimedia understanding of natural interaction, digital blackboard
• draw slowly and talk slowly, you might have a chance
• Intelligence embedded in floors and walls, change idea of desktop system

* multi-modal interaction

• (then) adapt yourself to the system
• (now) natural

* Building Systems [ref. Gerry Sussman]

• ex: biological systems, cities, internet
• how? redundancy of function, degeneracy
• the diseconomies of lean engineering
• degeneracy enables evolution
• the hard part -- how do we do this?

* Human Intelligence is a Natural Phenomenon

• The nature of evolution, evolutionary processes, and outcome
• human intelligence is a natural phenomena, what's true for anatomy might be true for our cognitive process as well
• there won't be "a" singularity
• prediction about the near future (exponential growth of power of machine)
• exponential growth in hardware growth, where is the exponential growth of software?

General Discussion

Moderator: Ronald Brachman

Panelists: Candace Sidner, Rodney Brooks, Randall Davis, James Hendler, Tom Mitchell

Ronald Brachman: Pick a favorite inflection point -- what the world will be, what will actually change dramatically?
Resources is a problem -- anything we can do to make it much quicker to get to the inflection point?

Randall Davis: multimodal understanding & interaction
converse with the whiteboard, if the whitebord can gesture and talk back
the notion of design artifacts, impact on people's productivity
design rational capture
core issue - multiple representations (multiple representation of sketching)
get power from each representation for disambiguation

Rodney Brooks: manipulation in unstructured environment
robots will be in personal life
from 0 to thousands of robots at homes/in the military (exponential growth)
robots will do more than what they can do now
sooner -> more money!
nanotechnology for hand sensors, dense sensors coupling in more work on object recognition for computer vision

James Hendler: more useful things on the web (that are not there yet)
other proof methods
Can't do much to make it faster...
semantics, latent semantics, explicit semantics
right now semantic web comes out of corpora with simple representation
be more involved in these communities

Ronald Brachman: (follow up) how this can change the world, change AI?

James Hendler: makes us pay attention that we've only done this small; can get it up and running much faster
don't tend to talk about KR as bottleneck, because it's hard

Tom Mitchell: computer interfaces
implants in the brain
scientific understanding -- there will be a reason to put the implant
and practical impact will come by these implants.
funding

Ronald Brachman: near future of machine learning?

Tom Mitchell: change to learn about human learning
with brain imaging
lower activation after human learned, but more synchronized
learning will be observable with oscilloscopes in the brain.... dramatic differences in how machine learning works and brain works
make these more observable
think much more about machine learning methods working in the brain (2 different times of learning), should machines do that too?
think away from the local maximum of where we're at

Candace Sidner: human-machine interaction
tell us part of human beings and machines we haven't thought about at all
gestures (we gesture and they gesture right back), make sense socially
funding!

Ronald Brachman: (to the audience) your favorite inflection point comment, what they are, how to take us there, what the world will be?
DARPA grand challenge is a inflection point, more excitement about robot vehicles
new grand challenges for AI (something more intellectual)?

a1: (to Randy) science advances by new instruments
instruments are created outside our field
the software challenge is the instruments we create
how to reach an agreement? (easy for people other fields)
infrastructure -- building a reusable piece of software for different research projects

a2: want a maxima chip in my head, so that I can understand Einstein's equations in my hand... triggering out the appropriate coding problems in enabling this

a3: mental level knowledge -- something we were able to do a decade ago, but I don't see it's been powered up on
it would enable monitoring performance, allow for reinforcement, selective focus, deal multiple perspectives
systems lack motivation, multiple value systems
at the mental we can motivate problem solving of different sorts

a4: published in 1987, but haven't reached the inflection point yet
give it a lot of messages, systems get better
getting the knowledge is too hard
a lot of knowledge is got from reading text
natural language understanding people should focus on text and knowledge problem, how come we're not moving there?

Ronald Brachman: monolithic number for the knowledge for Knowledge? what is the number? seedling program of bootstrapping, learning by reading process

a4: if we know how to do it, just let the computers work

a5: representation for fluid controls
predicate and functions
any representation is an approximation
they have to change their own representation
systems adapt with the time and change the representation?
Enormous tech challenges

a6: unmanageable classified and unclassified information, need analytic power
systems with new ways of interacting, communicating and analysing
tremendous antipathy from analysts who will be replaced
cognition (results vs. enhance your power as an analyst)
think twice whether you want to delegate the decision making (in real time) to your machines
continue the process we can already see, take the trivial and work on knowledge processors
propose machines that have such human interaction
proposals for that will be interesting.

a7: practical inflection point
CAD systems revolutionized hardware exponential growth
have CAD system for robotics -> can build and test the system in a day
will be able to deploy much complex robotic systems

a8: identify technical advances, and economic inflection points
how to provide resources for technical advances

a9: bring together ontology and expert systems
every piece of data has an ontology behind it
make the best use of the ontology

a10: massive amount of scientific data online
in the form that can be incorporated into large scale KB systems
use machine learning technologies to construct new knowledge
closed-loop machine learning system
multi-fluidic genes for scientific discovery with very fast turn-off time
situated scientific discovery -> interested from scientists
transform how scientific labs work

a11: advances in sensing technologies to understand human behavior
record what's going on in our bodies and the physical environment
model human behavior, human cognition and intelligence

a12: a socially acceptable and possible implant -> not for disabled people, but for ordinary people who use medical monitors

a13: machine learning, instinctly hand-coded... trained syntactical operating system, just like the way we train people, as opposed to physically hand-coding

Transcribers: Chih-yu Chao & Varun Aggarwal