1 00:00:00,000 --> 00:00:00,765 Hello. 2 00:00:00,765 --> 00:00:06,300 3 00:00:06,300 --> 00:00:10,470 Welcome to the LCS Distinguished Lecture Series. 4 00:00:10,470 --> 00:00:13,470 For those of you who are not familiar with this series, 5 00:00:13,470 --> 00:00:15,540 I am not Michael Dertouzos. 6 00:00:15,540 --> 00:00:17,640 There are several distinct differences 7 00:00:17,640 --> 00:00:21,090 which I won't go into. 8 00:00:21,090 --> 00:00:25,140 Michael asked me to substitute for him. 9 00:00:25,140 --> 00:00:27,368 I've decided not to give the traditional introduction 10 00:00:27,368 --> 00:00:29,910 and instead to leave some time for our distinguished lecturer 11 00:00:29,910 --> 00:00:32,780 to speak himself. 12 00:00:32,780 --> 00:00:35,760 Our lecturer today is Andries van Dam 13 00:00:35,760 --> 00:00:38,910 from Brown University where he's the T.J. 14 00:00:38,910 --> 00:00:42,150 Watson, Jr., University Professor of Technology 15 00:00:42,150 --> 00:00:46,300 and Education and Professor of Computer Science. 16 00:00:46,300 --> 00:00:49,080 It's certainly an impressive title. 17 00:00:49,080 --> 00:00:52,290 And in fact, he's even an impressive fellow 18 00:00:52,290 --> 00:00:54,480 to go along with the title. 19 00:00:54,480 --> 00:00:57,240 He was an undergraduate at Swarthmore College, 20 00:00:57,240 --> 00:01:00,630 got his doctorate from University of Pennsylvania. 21 00:01:00,630 --> 00:01:03,270 And he tells me it was the second doctorate in the United 22 00:01:03,270 --> 00:01:06,550 States granted in computer science. 23 00:01:06,550 --> 00:01:10,260 So he must be older than he looks. 24 00:01:10,260 --> 00:01:12,390 He left Penn and went to Brown University 25 00:01:12,390 --> 00:01:15,080 as an assistant professor and has been there ever since. 26 00:01:15,080 --> 00:01:18,598 27 00:01:18,598 --> 00:01:20,890 When you started at Brown, you were in the Applied Math 28 00:01:20,890 --> 00:01:22,390 Division, I guess it was called. 29 00:01:22,390 --> 00:01:24,190 Yeah. 30 00:01:24,190 --> 00:01:27,340 Within that, he then started a Computer Science program, 31 00:01:27,340 --> 00:01:30,160 was its first head for three years, 32 00:01:30,160 --> 00:01:32,650 and then co-founded the Computer Science 33 00:01:32,650 --> 00:01:36,720 Department where he was its first chairman for six years. 34 00:01:36,720 --> 00:01:38,540 So he seems to have an incredible stomach 35 00:01:38,540 --> 00:01:41,600 for that sort of thing, though he has escaped it 36 00:01:41,600 --> 00:01:44,480 and is now doing more interesting things. 37 00:01:44,480 --> 00:01:47,540 38 00:01:47,540 --> 00:01:50,510 He's most well known probably for his books 39 00:01:50,510 --> 00:01:53,510 on computer graphics, first Foley and van Dam, 40 00:01:53,510 --> 00:01:57,620 and then Foley, van Dam, Feiner, and Hughes. 41 00:01:57,620 --> 00:02:00,410 I knew Andy before he was a famous person, 42 00:02:00,410 --> 00:02:01,850 even before he was-- 43 00:02:01,850 --> 00:02:03,920 when I was an undergraduate at Brown, 44 00:02:03,920 --> 00:02:08,060 I was early in his tenure there. 45 00:02:08,060 --> 00:02:10,850 In many ways, you have Andy to blame for the fact 46 00:02:10,850 --> 00:02:14,075 that I'm standing today, standing here today. 47 00:02:14,075 --> 00:02:15,920 I'd probably be standing somewhere anyway. 48 00:02:15,920 --> 00:02:22,940 49 00:02:22,940 --> 00:02:26,200 He took under his wing a number of people 50 00:02:26,200 --> 00:02:27,950 who came to Brown thinking they were going 51 00:02:27,950 --> 00:02:30,200 to do something other than computing, 52 00:02:30,200 --> 00:02:32,480 and convinced us that it would be a foolish way 53 00:02:32,480 --> 00:02:34,160 to waste our lives. 54 00:02:34,160 --> 00:02:37,880 And in fact, the only thing that was a lot of fun to do 55 00:02:37,880 --> 00:02:40,040 was to play with computers. 56 00:02:40,040 --> 00:02:46,010 And he certainly persuaded me and several other miscreants, 57 00:02:46,010 --> 00:02:49,140 some of whom have been speakers in this Distinguished Lecture 58 00:02:49,140 --> 00:02:49,640 Series. 59 00:02:49,640 --> 00:02:51,020 So he has a lot to answer for. 60 00:02:51,020 --> 00:02:54,230 61 00:02:54,230 --> 00:02:57,260 He's founder of ACM SIGGRAPH from which he 62 00:02:57,260 --> 00:03:00,230 received the Coons Award in 1991, 63 00:03:00,230 --> 00:03:04,455 and in 1993, won ACM's Karl Karlstrom Teaching Award. 64 00:03:04,455 --> 00:03:06,080 And for those of you who don't know it, 65 00:03:06,080 --> 00:03:08,000 it's probably the most distinguished award 66 00:03:08,000 --> 00:03:10,715 you can get for teaching in the field of computer science. 67 00:03:10,715 --> 00:03:14,310 68 00:03:14,310 --> 00:03:14,810 Let's see. 69 00:03:14,810 --> 00:03:16,518 What else is there to say here about him? 70 00:03:16,518 --> 00:03:19,950 71 00:03:19,950 --> 00:03:21,700 Well, let me read a little story. 72 00:03:21,700 --> 00:03:25,110 This will capture a lot about Andy. 73 00:03:25,110 --> 00:03:27,330 This has to do with-- 74 00:03:27,330 --> 00:03:31,530 in addition to graphics, he had the misfortune 75 00:03:31,530 --> 00:03:35,490 to work on hypertext long before it was a good idea. 76 00:03:35,490 --> 00:03:40,900 In 1969, when I was at Brown, we would use the entire Brown 77 00:03:40,900 --> 00:03:42,900 Computer Center, the thing that did all the data 78 00:03:42,900 --> 00:03:45,450 processing, all the salaries and everything else 79 00:03:45,450 --> 00:03:48,570 from midnight to 8:00 AM to edit a single hypertext 80 00:03:48,570 --> 00:03:52,140 document, usually a rather short hypertext document, 81 00:03:52,140 --> 00:03:54,530 on a 2250 display. 82 00:03:54,530 --> 00:04:00,000 This was text only, but it was in 1969 WYSIWYG editor 83 00:04:00,000 --> 00:04:01,920 several decades ahead of technology, 84 00:04:01,920 --> 00:04:04,530 or at least 10 years ahead of it. 85 00:04:04,530 --> 00:04:07,550 And Andy had a lot of faith in his students, 86 00:04:07,550 --> 00:04:09,060 his undergraduates. 87 00:04:09,060 --> 00:04:13,290 And he would have us do his correspondence for him 88 00:04:13,290 --> 00:04:17,279 from midnight to 8:00 AM on this hypertext system, 89 00:04:17,279 --> 00:04:19,649 just to show it could be used. 90 00:04:19,649 --> 00:04:22,470 So one night, several students are there editing a letter 91 00:04:22,470 --> 00:04:23,828 to the provost. 92 00:04:23,828 --> 00:04:26,370 And I should say Andy's taught me everything I know about how 93 00:04:26,370 --> 00:04:27,412 to deal with the provost. 94 00:04:27,412 --> 00:04:29,670 95 00:04:29,670 --> 00:04:33,780 We produced two versions of that letter, the one Andy gave us 96 00:04:33,780 --> 00:04:35,490 and one in which several [? nots ?] were 97 00:04:35,490 --> 00:04:38,910 inserted at judicious places and removed from other places, 98 00:04:38,910 --> 00:04:42,090 thereby changing the whole sense of the letter. 99 00:04:42,090 --> 00:04:46,140 About 6:00 AM, we finished, left it on Andy's desk-- 100 00:04:46,140 --> 00:04:50,040 the fake later on Andy's desk with the [? nots ?] in it 101 00:04:50,040 --> 00:04:53,310 and out of it, and left. 102 00:04:53,310 --> 00:04:54,660 Thought it was a very good joke. 103 00:04:54,660 --> 00:04:56,700 Came back later in the day. 104 00:04:56,700 --> 00:04:58,630 Andy didn't say a word. 105 00:04:58,630 --> 00:04:59,160 Day went by. 106 00:04:59,160 --> 00:05:01,260 Andy didn't say a word. 107 00:05:01,260 --> 00:05:05,090 Finally, a little bit trepidatiously, people 108 00:05:05,090 --> 00:05:07,340 came up and asked Andy what was going on. 109 00:05:07,340 --> 00:05:08,675 What about the letter? 110 00:05:08,675 --> 00:05:10,550 And he said, oh, I didn't bother to check it. 111 00:05:10,550 --> 00:05:11,330 I just mailed it. 112 00:05:11,330 --> 00:05:15,140 113 00:05:15,140 --> 00:05:18,830 The perpetrators of the would-be hoax immediately whipped out 114 00:05:18,830 --> 00:05:21,200 of Andy's office, spent the entire day 115 00:05:21,200 --> 00:05:24,020 trying to retrieve that letter from various sources 116 00:05:24,020 --> 00:05:27,560 throughout the university mail system. 117 00:05:27,560 --> 00:05:31,490 At 6:00 PM, having missed a whole day of classes, 118 00:05:31,490 --> 00:05:33,270 returned to Andy's office and confessed 119 00:05:33,270 --> 00:05:35,442 the sin, at which point he smiled 120 00:05:35,442 --> 00:05:37,900 and pulled the letter out of his middle drawer of his desk. 121 00:05:37,900 --> 00:05:41,330 122 00:05:41,330 --> 00:05:44,720 So I think that tells a lot about our speaker today. 123 00:05:44,720 --> 00:05:49,580 124 00:05:49,580 --> 00:05:52,000 So as Mike asked me to, I asked Andy 125 00:05:52,000 --> 00:05:56,470 if he had any aphorisms to pass on to the audience. 126 00:05:56,470 --> 00:05:59,950 And he said we should be grateful for the privilege 127 00:05:59,950 --> 00:06:03,580 of being paid to do things that are fun, and if what we do ever 128 00:06:03,580 --> 00:06:06,700 stops being fun we should find another profession. 129 00:06:06,700 --> 00:06:07,750 Andy. 130 00:06:07,750 --> 00:06:11,110 [APPLAUSE] 131 00:06:11,110 --> 00:06:13,030 132 00:06:13,030 --> 00:06:14,440 Thanks, John. 133 00:06:14,440 --> 00:06:17,350 I think that's an Ed Lazowska story that might actually even 134 00:06:17,350 --> 00:06:20,020 be true. 135 00:06:20,020 --> 00:06:22,180 It's a privilege to be back here. 136 00:06:22,180 --> 00:06:24,970 Last time I spoke here, I couldn't speak. 137 00:06:24,970 --> 00:06:29,050 And I sort of hacked and coughed my way through my lecture 138 00:06:29,050 --> 00:06:31,780 and barely made it back to my seat. 139 00:06:31,780 --> 00:06:34,270 I'm in fine voice today, I hope. 140 00:06:34,270 --> 00:06:37,173 And I'd like to say that I got started 141 00:06:37,173 --> 00:06:38,590 in the field about which I'm going 142 00:06:38,590 --> 00:06:43,420 to be speaking today by someone you should all know about. 143 00:06:43,420 --> 00:06:46,100 Some of you are too young to have met him in person. 144 00:06:46,100 --> 00:06:49,000 Some of you knew him, and that's J. C. R. Licklider 145 00:06:49,000 --> 00:06:50,680 whom I believe to be the first person 146 00:06:50,680 --> 00:06:54,040 to have emphasized the importance of user interface. 147 00:06:54,040 --> 00:06:55,970 A great man, a great human being. 148 00:06:55,970 --> 00:06:58,810 And in a sense, I'm giving a lecture in his honor 149 00:06:58,810 --> 00:07:00,650 here today. 150 00:07:00,650 --> 00:07:03,370 So let me tell you what my main thesis is. 151 00:07:03,370 --> 00:07:05,140 You've already read it in the abstract. 152 00:07:05,140 --> 00:07:08,845 It will not surprise you. 153 00:07:08,845 --> 00:07:11,190 I got to learn not to point at the screen. 154 00:07:11,190 --> 00:07:12,330 That's the first trick. 155 00:07:12,330 --> 00:07:15,860 156 00:07:15,860 --> 00:07:17,570 Escaping Flatland. 157 00:07:17,570 --> 00:07:20,070 Our user interfaces are flat today. 158 00:07:20,070 --> 00:07:23,910 And what do we intend to do about that? 159 00:07:23,910 --> 00:07:26,540 So I'm going to motivate why there is something 160 00:07:26,540 --> 00:07:30,320 beyond what we're used to today, talk a little bit about what 161 00:07:30,320 --> 00:07:35,180 it feels like to design 3D user interfaces, 162 00:07:35,180 --> 00:07:37,580 then switch hats a little bit and talk about some 163 00:07:37,580 --> 00:07:40,190 of the implementation software issues, 164 00:07:40,190 --> 00:07:44,190 and then draw the inevitable conclusions. 165 00:07:44,190 --> 00:07:46,440 So let's talk about user interface 166 00:07:46,440 --> 00:07:47,953 and what we would like. 167 00:07:47,953 --> 00:07:49,370 And what we would like, I believe, 168 00:07:49,370 --> 00:07:51,000 is no user interface at all. 169 00:07:51,000 --> 00:07:54,140 In fact, we wouldn't like any of those stinking computers. 170 00:07:54,140 --> 00:07:56,180 We'd like to be able to interact as we 171 00:07:56,180 --> 00:08:00,980 do with our fellow human beings and talk, and emote, 172 00:08:00,980 --> 00:08:04,220 and move about the world and not ever be aware of the fact 173 00:08:04,220 --> 00:08:06,710 that there are computers and that we must control them 174 00:08:06,710 --> 00:08:09,470 or steer them via user interfaces. 175 00:08:09,470 --> 00:08:11,810 The closest thing to a vision about this 176 00:08:11,810 --> 00:08:16,835 was given to us in Stanley Kubrick's immortal film, 2001. 177 00:08:16,835 --> 00:08:19,460 And I just want to refresh your memory with a little, tiny clip 178 00:08:19,460 --> 00:08:20,287 from that. 179 00:08:20,287 --> 00:08:20,954 [VIDEO PLAYBACK] 180 00:08:20,954 --> 00:08:23,096 - [INAUDIBLE] this video provides fully interactive 181 00:08:23,096 --> 00:08:26,570 direct [INAUDIBLE] replacing and modifying-- 182 00:08:26,570 --> 00:08:27,380 Not that one. 183 00:08:27,380 --> 00:08:30,470 Forget that, a miscue here. 184 00:08:30,470 --> 00:08:31,038 185 00:08:31,038 --> 00:08:31,580 - [INAUDIBLE] 186 00:08:31,580 --> 00:08:33,500 [END PLAYBACK] 187 00:08:33,500 --> 00:08:38,240 Somehow, it seems to have rewound itself. 188 00:08:38,240 --> 00:08:40,807 I hate to do this. 189 00:08:40,807 --> 00:08:42,210 [INAUDIBLE] 190 00:08:42,210 --> 00:08:44,430 Yeah. 191 00:08:44,430 --> 00:08:48,940 I left it paused, and it seems to have come unglued. 192 00:08:48,940 --> 00:08:52,530 OK, let me rewind. 193 00:08:52,530 --> 00:08:54,953 Sorry about that. 194 00:08:54,953 --> 00:08:55,620 [VIDEO PLAYBACK] 195 00:08:55,620 --> 00:08:57,260 - [INAUDIBLE] along its length. 196 00:08:57,260 --> 00:08:58,777 We have positioned [INAUDIBLE]. 197 00:08:58,777 --> 00:08:59,360 [END PLAYBACK] 198 00:08:59,360 --> 00:09:02,200 Play, rewind. 199 00:09:02,200 --> 00:09:04,270 Actually, I'll stop first and then rewind. 200 00:09:04,270 --> 00:09:08,689 201 00:09:08,689 --> 00:09:12,090 Ah, the wonders of technology. 202 00:09:12,090 --> 00:09:14,050 [INAUDIBLE] 203 00:09:14,050 --> 00:09:17,480 [LAUGHTER] 204 00:09:17,480 --> 00:09:20,913 205 00:09:20,913 --> 00:09:23,330 So you know, I can't even tell whether it's doing anything 206 00:09:23,330 --> 00:09:25,812 because nothing much is happening. 207 00:09:25,812 --> 00:09:27,740 [INAUDIBLE] 208 00:09:27,740 --> 00:09:31,040 Now I'm hitting the rewind button. 209 00:09:31,040 --> 00:09:31,730 Good. 210 00:09:31,730 --> 00:09:32,870 It did something. 211 00:09:32,870 --> 00:09:34,630 All right, let's try again. 212 00:09:34,630 --> 00:09:42,160 213 00:09:42,160 --> 00:09:44,650 I would like the room to be as dark as possible 214 00:09:44,650 --> 00:09:45,710 when the tape is on. 215 00:09:45,710 --> 00:09:48,115 And you certainly don't need me for video purposes. 216 00:09:48,115 --> 00:09:51,180 217 00:09:51,180 --> 00:09:52,500 All right, here we go. 218 00:09:52,500 --> 00:09:59,890 219 00:09:59,890 --> 00:10:02,016 Oh, nuts. 220 00:10:02,016 --> 00:10:05,362 [LAUGHTER] 221 00:10:05,362 --> 00:10:09,200 222 00:10:09,200 --> 00:10:16,590 See, this interface is modal, and that is a really bad idea. 223 00:10:16,590 --> 00:10:22,590 OK, so we rewind it, we hit stop to get out of the mode. 224 00:10:22,590 --> 00:10:24,150 OK, and then we hit play. 225 00:10:24,150 --> 00:10:27,710 Now that should work, right? 226 00:10:27,710 --> 00:10:28,730 One more time. 227 00:10:28,730 --> 00:10:31,103 228 00:10:31,103 --> 00:10:31,770 [VIDEO PLAYBACK] 229 00:10:31,770 --> 00:10:32,645 - PERSONA is a part-- 230 00:10:32,645 --> 00:10:33,180 No. 231 00:10:33,180 --> 00:10:33,780 Not yet. 232 00:10:33,780 --> 00:10:35,680 - [INAUDIBLE] user interface group at Microsoft Research. 233 00:10:35,680 --> 00:10:36,222 We explored-- 234 00:10:36,222 --> 00:10:41,170 [END PLAYBACK] 235 00:10:41,170 --> 00:10:43,780 Rewind again. 236 00:10:43,780 --> 00:10:45,280 It clicked to the end. 237 00:10:45,280 --> 00:10:48,760 I hit stop, I hit play. 238 00:10:48,760 --> 00:10:49,780 That ought to do it. 239 00:10:49,780 --> 00:10:59,560 240 00:10:59,560 --> 00:11:01,027 [INAUDIBLE] 241 00:11:01,027 --> 00:11:07,460 242 00:11:07,460 --> 00:11:09,487 It can't possibly be worth this, right? 243 00:11:09,487 --> 00:11:10,820 That's what you're all thinking. 244 00:11:10,820 --> 00:11:12,590 You're right, it can't be. 245 00:11:12,590 --> 00:11:14,990 It would have been OK as a 30-second clip, 246 00:11:14,990 --> 00:11:27,580 but I'm going to push fast forward because [LAUGHTER] it 247 00:11:27,580 --> 00:11:29,650 ain't doing anything. 248 00:11:29,650 --> 00:11:32,710 Would somebody like to control this who is more mechanically 249 00:11:32,710 --> 00:11:35,830 apt than I am? 250 00:11:35,830 --> 00:11:37,744 I'm hitting play again. 251 00:11:37,744 --> 00:11:39,640 [INAUDIBLE] 252 00:11:39,640 --> 00:11:42,298 No, that's not the problem. 253 00:11:42,298 --> 00:11:43,930 [INAUDIBLE] 254 00:11:43,930 --> 00:11:46,817 Yeah, all right. 255 00:11:46,817 --> 00:11:48,400 You want to see if you have any better 256 00:11:48,400 --> 00:11:51,230 luck with this, [? Carl? ?] 257 00:11:51,230 --> 00:11:54,880 There's somebody in the control center behind that [INAUDIBLE].. 258 00:11:54,880 --> 00:11:56,860 I've hit play, and it does-- 259 00:11:56,860 --> 00:11:58,090 here we go. 260 00:11:58,090 --> 00:11:58,930 There we go. 261 00:11:58,930 --> 00:12:00,640 All right, long leader maybe. 262 00:12:00,640 --> 00:12:05,912 263 00:12:05,912 --> 00:12:07,370 Boy, am I building up the suspense. 264 00:12:07,370 --> 00:12:10,772 265 00:12:10,772 --> 00:12:11,439 [VIDEO PLAYBACK] 266 00:12:11,439 --> 00:12:13,100 - Knight takes bishop. 267 00:12:13,100 --> 00:12:14,964 Mate. 268 00:12:14,964 --> 00:12:18,230 - Ah. 269 00:12:18,230 --> 00:12:20,092 Yeah, looks like you're right. 270 00:12:20,092 --> 00:12:21,024 I resign. 271 00:12:21,024 --> 00:12:23,720 - Thank you for a very enjoyable game. 272 00:12:23,720 --> 00:12:24,487 - Yeah, thank you. 273 00:12:24,487 --> 00:12:25,070 [END PLAYBACK] 274 00:12:25,070 --> 00:12:27,160 All right, stop. 275 00:12:27,160 --> 00:12:29,750 [? Carl, ?] let's see how well you do at this. 276 00:12:29,750 --> 00:12:31,400 You can't do any worse than I did. 277 00:12:31,400 --> 00:12:34,080 [INAUDIBLE] plug here someplace. 278 00:12:34,080 --> 00:12:35,420 [LAUGHTER] 279 00:12:35,420 --> 00:12:36,033 No, see. 280 00:12:36,033 --> 00:12:36,700 [VIDEO PLAYBACK] 281 00:12:36,700 --> 00:12:38,404 - Several of the projects that we're working on-- 282 00:12:38,404 --> 00:12:38,987 [END PLAYBACK] 283 00:12:38,987 --> 00:12:40,739 [LAUGHTER] 284 00:12:40,739 --> 00:12:43,260 The power works. 285 00:12:43,260 --> 00:12:45,590 No, no, no, no, no. 286 00:12:45,590 --> 00:12:47,192 That won't do. 287 00:12:47,192 --> 00:12:49,040 [LAUGHTER] 288 00:12:49,040 --> 00:12:50,855 OK, maybe I have to be closer. 289 00:12:50,855 --> 00:12:53,360 290 00:12:53,360 --> 00:12:55,100 OK, we'll leave this for now. 291 00:12:55,100 --> 00:12:57,770 We'll switch channels, get the PowerBook back. 292 00:12:57,770 --> 00:12:59,540 That ought to work. 293 00:12:59,540 --> 00:13:00,380 It worked before. 294 00:13:00,380 --> 00:13:03,290 295 00:13:03,290 --> 00:13:05,300 Oh, [INAUDIBLE] disaster. 296 00:13:05,300 --> 00:13:08,630 297 00:13:08,630 --> 00:13:10,310 PowerBook. 298 00:13:10,310 --> 00:13:11,480 Something, all right. 299 00:13:11,480 --> 00:13:14,520 So HAL 9000, what was going on there? 300 00:13:14,520 --> 00:13:17,120 We were talking to the computer, or at least Dave 301 00:13:17,120 --> 00:13:23,000 was, and wasn't aware of a particular interface, conversed 302 00:13:23,000 --> 00:13:25,350 as if dealing with a human being. 303 00:13:25,350 --> 00:13:28,550 Well, that these days is called the idea of an agent, 304 00:13:28,550 --> 00:13:31,130 or some people call it a social interface. 305 00:13:31,130 --> 00:13:32,810 There are folks here, particularly 306 00:13:32,810 --> 00:13:35,240 Pattie Maes in the Media Lab, who 307 00:13:35,240 --> 00:13:40,790 are dealing with the technology and the sociology of having 308 00:13:40,790 --> 00:13:44,090 these kinds of pseudo beings do intelligent things 309 00:13:44,090 --> 00:13:45,570 on your behalf. 310 00:13:45,570 --> 00:13:46,950 It's a very important area. 311 00:13:46,950 --> 00:13:49,680 And I'm not going to talk about it today. 312 00:13:49,680 --> 00:13:51,990 Neither am I, obviously, about the next bullet, 313 00:13:51,990 --> 00:13:54,680 which is what people sometimes refer to as DMA. 314 00:13:54,680 --> 00:13:57,560 315 00:13:57,560 --> 00:14:00,750 I think fortunately, this won't happen in my lifetime. 316 00:14:00,750 --> 00:14:04,580 I can't begin to imagine all the mischief that can be put to. 317 00:14:04,580 --> 00:14:07,130 So I'm going to talk about interfaces that give us 318 00:14:07,130 --> 00:14:09,930 a quality that interface designers like to talk about, 319 00:14:09,930 --> 00:14:11,435 which is transparency. 320 00:14:11,435 --> 00:14:12,560 So what do we mean by that? 321 00:14:12,560 --> 00:14:14,810 Again, it's this notion of not being 322 00:14:14,810 --> 00:14:16,790 cognitively aware of the fact that you 323 00:14:16,790 --> 00:14:18,470 are driving an interface. 324 00:14:18,470 --> 00:14:21,320 When you drive a car, after you've mastered the art, 325 00:14:21,320 --> 00:14:23,690 you are not aware of the fact that you're driving. 326 00:14:23,690 --> 00:14:24,830 Your mind can wander. 327 00:14:24,830 --> 00:14:27,960 You cannot even remember where you've been over the last 10, 328 00:14:27,960 --> 00:14:31,160 20 minutes, because you're on automatic. 329 00:14:31,160 --> 00:14:35,060 And some people get that way even in Emacs. 330 00:14:35,060 --> 00:14:41,420 So that's what we mean by a transparent interface. 331 00:14:41,420 --> 00:14:44,030 If we look at the history of user interface, 332 00:14:44,030 --> 00:14:46,940 we can sort of see four distinct plateaus, 333 00:14:46,940 --> 00:14:49,880 punctuated equilibrium in Gould sense. 334 00:14:49,880 --> 00:14:52,740 Namely things are stable for a long time. 335 00:14:52,740 --> 00:14:55,940 And there is some kind of profound paradigm shift, 336 00:14:55,940 --> 00:14:57,440 and then you do something else. 337 00:14:57,440 --> 00:14:59,420 In the beginning, there was no user interface. 338 00:14:59,420 --> 00:15:01,520 We didn't even talk about the term. 339 00:15:01,520 --> 00:15:05,120 And then we got to time sharing, invented largely here. 340 00:15:05,120 --> 00:15:08,720 And we used glass teletypes with first real teletypes, 341 00:15:08,720 --> 00:15:09,300 of course. 342 00:15:09,300 --> 00:15:11,270 Those of you not old enough to remember them, 343 00:15:11,270 --> 00:15:13,370 they are these archaic mechanical devices 344 00:15:13,370 --> 00:15:15,080 fed by paper tape. 345 00:15:15,080 --> 00:15:17,690 And you could sort of type like this on them. 346 00:15:17,690 --> 00:15:20,420 And you drove your application program 347 00:15:20,420 --> 00:15:24,590 with a command line interface, a shell you would call it today. 348 00:15:24,590 --> 00:15:28,490 And then the big revolution was at Xerox PARC 349 00:15:28,490 --> 00:15:32,060 where bitmap graphics workstations were invented. 350 00:15:32,060 --> 00:15:36,890 And basing a lot on Engelbart's work and even some of the ideas 351 00:15:36,890 --> 00:15:40,760 that first were shown here by Sutherland in Sketchpad, 352 00:15:40,760 --> 00:15:43,190 we got the idea of carving up the screen 353 00:15:43,190 --> 00:15:46,280 into individual foci of attention 354 00:15:46,280 --> 00:15:48,650 doing various tasks in each of these things 355 00:15:48,650 --> 00:15:52,850 called windows, and by the magic of bitmap graphics 356 00:15:52,850 --> 00:15:56,480 came up with icons, and menus, and all those things that today 357 00:15:56,480 --> 00:15:59,210 we call the WIMP interface, the WIMP look and feel, 358 00:15:59,210 --> 00:16:00,740 the messy desktop metaphor. 359 00:16:00,740 --> 00:16:02,060 You're all familiar with it. 360 00:16:02,060 --> 00:16:03,620 I needn't belabor it. 361 00:16:03,620 --> 00:16:05,630 What's happened since the early '70s? 362 00:16:05,630 --> 00:16:08,810 Answer-- essentially, more of the same polishing 363 00:16:08,810 --> 00:16:11,630 third order effects, adding drop shadows, 364 00:16:11,630 --> 00:16:13,370 making things look pretty. 365 00:16:13,370 --> 00:16:16,670 As far as the interface is concerned, nothing much. 366 00:16:16,670 --> 00:16:19,460 From an implementation point of view, a lot has happened. 367 00:16:19,460 --> 00:16:23,420 We have tons and tons of GUI building support software. 368 00:16:23,420 --> 00:16:26,300 And I'll have more to say about that in a moment. 369 00:16:26,300 --> 00:16:29,330 What I'm going to say today is forget that stuff. 370 00:16:29,330 --> 00:16:31,130 It's going to be useful for a long time. 371 00:16:31,130 --> 00:16:33,320 Keep your eye on the future, and the future 372 00:16:33,320 --> 00:16:36,090 lies in post-WIMP interfaces. 373 00:16:36,090 --> 00:16:38,480 So that's what the lecture will be about. 374 00:16:38,480 --> 00:16:41,120 Where are we today before we launch into that? 375 00:16:41,120 --> 00:16:44,330 I claim that finally, user interface technology 376 00:16:44,330 --> 00:16:45,770 has become a discipline. 377 00:16:45,770 --> 00:16:47,930 And we finally recognize that usability 378 00:16:47,930 --> 00:16:50,210 is much more important than functionality, 379 00:16:50,210 --> 00:16:52,430 above a certain threshold of functionality, 380 00:16:52,430 --> 00:16:54,360 as a key to productivity. 381 00:16:54,360 --> 00:16:57,110 It is nothing short of miraculous 382 00:16:57,110 --> 00:17:00,920 that you have preschoolers who can neither read nor write, 383 00:17:00,920 --> 00:17:03,420 yet productively use computers. 384 00:17:03,420 --> 00:17:05,540 And if you find that hard to believe, 385 00:17:05,540 --> 00:17:08,329 think about managers doing the same thing. 386 00:17:08,329 --> 00:17:10,410 [LAUGHTER] 387 00:17:10,410 --> 00:17:16,280 And if you find that miraculous, think about senior faculty 388 00:17:16,280 --> 00:17:18,319 who use computers productively. 389 00:17:18,319 --> 00:17:21,650 That is a revolution. 390 00:17:21,650 --> 00:17:25,010 Why is it possible to do that today? 391 00:17:25,010 --> 00:17:27,960 Because of all of these great interface techniques 392 00:17:27,960 --> 00:17:30,290 and because we are finally paying attention 393 00:17:30,290 --> 00:17:31,430 to the needs of the user. 394 00:17:31,430 --> 00:17:34,520 And that is called user-centered design. 395 00:17:34,520 --> 00:17:37,340 We do usability testing on software products 396 00:17:37,340 --> 00:17:40,760 before we release them on an unsuspecting public. 397 00:17:40,760 --> 00:17:43,460 And the corporations now spend big bucks 398 00:17:43,460 --> 00:17:46,730 building usability labs and testing things and finding out 399 00:17:46,730 --> 00:17:49,790 whether they're going to work before they release them. 400 00:17:49,790 --> 00:17:52,880 We have a professional society, SIGCHI, devoted 401 00:17:52,880 --> 00:17:54,570 to computer-human interaction. 402 00:17:54,570 --> 00:17:56,210 We have textbooks and courses. 403 00:17:56,210 --> 00:17:58,580 And finally, the ultimate proof, we have lawsuits. 404 00:17:58,580 --> 00:18:01,940 We have Apple versus Microsoft, Borland versus Lotus, 405 00:18:01,940 --> 00:18:02,630 and so on. 406 00:18:02,630 --> 00:18:05,370 The field must be real. 407 00:18:05,370 --> 00:18:08,350 So why have these WIMP GUIs stuck around for so long? 408 00:18:08,350 --> 00:18:09,550 Why are they standard? 409 00:18:09,550 --> 00:18:11,650 Well, because it is said that they provide us 410 00:18:11,650 --> 00:18:16,750 with ease of usability, learnability, transferability, 411 00:18:16,750 --> 00:18:17,980 and all those abilities. 412 00:18:17,980 --> 00:18:22,030 X for almost any value of x for your user. 413 00:18:22,030 --> 00:18:25,570 And also because we now have layers of support software 414 00:18:25,570 --> 00:18:29,530 so that we can take advantage of the increasing amounts of RAM 415 00:18:29,530 --> 00:18:31,180 that the manufacturers are providing 416 00:18:31,180 --> 00:18:35,500 us to soak all that up and have something useful to do with 100 417 00:18:35,500 --> 00:18:37,543 MIP processors, which otherwise, well, 418 00:18:37,543 --> 00:18:38,710 what would you do with them? 419 00:18:38,710 --> 00:18:41,930 Nowadays you service the user interface. 420 00:18:41,930 --> 00:18:44,860 So we get ease of implementation and maintainability 421 00:18:44,860 --> 00:18:47,800 because we have a standard way of implementing GUIs. 422 00:18:47,800 --> 00:18:50,260 We have simple tool kits like the X Toolkit. 423 00:18:50,260 --> 00:18:53,590 We have interface builders on top of that and even user 424 00:18:53,590 --> 00:18:55,090 interface management systems. 425 00:18:55,090 --> 00:18:58,540 And that layer cake provides us with lots of support. 426 00:18:58,540 --> 00:19:01,900 And even so, it's still too hard to build user interfaces, 427 00:19:01,900 --> 00:19:03,940 and it takes too much time. 428 00:19:03,940 --> 00:19:07,150 Why do we have limitations in such WIMP GUIs? 429 00:19:07,150 --> 00:19:10,420 Well, the support software doesn't do everything for you. 430 00:19:10,420 --> 00:19:14,050 And in particular, if you look at your stack of motif manuals 431 00:19:14,050 --> 00:19:18,550 about this high literally, more than a foot worth, 432 00:19:18,550 --> 00:19:20,540 you're forced to learn two environments, 433 00:19:20,540 --> 00:19:23,590 not just C and C++ but all the motif conventions 434 00:19:23,590 --> 00:19:24,790 and libraries. 435 00:19:24,790 --> 00:19:26,470 And you're constantly aware of the fact 436 00:19:26,470 --> 00:19:28,180 that you have two environments that you 437 00:19:28,180 --> 00:19:30,400 have to learn to use and to integrate 438 00:19:30,400 --> 00:19:33,070 through the ugly callback mechanism. 439 00:19:33,070 --> 00:19:36,490 Furthermore, the support of higher levels of functionality 440 00:19:36,490 --> 00:19:38,920 is primarily at the level of layout. 441 00:19:38,920 --> 00:19:40,810 Very little support for behavior, 442 00:19:40,810 --> 00:19:41,920 and that's bloody hard. 443 00:19:41,920 --> 00:19:43,330 We shouldn't be surprised. 444 00:19:43,330 --> 00:19:45,183 So it does the part of the job that is easy, 445 00:19:45,183 --> 00:19:46,600 and it does it better than the way 446 00:19:46,600 --> 00:19:48,340 we used to do it 20 years ago. 447 00:19:48,340 --> 00:19:50,770 We've made progress, but not enough. 448 00:19:50,770 --> 00:19:52,450 More importantly, for the post-WIMP 449 00:19:52,450 --> 00:19:55,120 interfaces that I'm going to be talking about, 450 00:19:55,120 --> 00:19:59,110 there is a certain mold that the WIMP GUI casts us 451 00:19:59,110 --> 00:20:01,990 in which is very inimical in my point of view. 452 00:20:01,990 --> 00:20:03,940 And I'll say more about that. 453 00:20:03,940 --> 00:20:07,360 It imposes a sequential dialogue model. 454 00:20:07,360 --> 00:20:08,890 I sit in the neutral state. 455 00:20:08,890 --> 00:20:11,290 I wait for the user to do a single thing. 456 00:20:11,290 --> 00:20:13,720 I react to it, and I go to the next state. 457 00:20:13,720 --> 00:20:16,090 Typical finite state automaton model. 458 00:20:16,090 --> 00:20:17,020 It's deterministic. 459 00:20:17,020 --> 00:20:19,600 You know whether a key has been pressed or not. 460 00:20:19,600 --> 00:20:22,900 It's very difficult to have simultaneity of input. 461 00:20:22,900 --> 00:20:25,240 Imagine how hard it is to have two data 462 00:20:25,240 --> 00:20:29,980 gloves through a standard user interface builder. 463 00:20:29,980 --> 00:20:31,840 It's not set up for that. 464 00:20:31,840 --> 00:20:36,040 It's not set up for two-handed mousing even, two mice. 465 00:20:36,040 --> 00:20:40,120 And in particular, it's not an all natural for virtual reality 466 00:20:40,120 --> 00:20:44,290 of the immersive variety, real virtual reality as it were. 467 00:20:44,290 --> 00:20:47,080 That is, I'm immersed in an environment. 468 00:20:47,080 --> 00:20:48,370 I look around. 469 00:20:48,370 --> 00:20:51,670 It tracks my head, if not other body parts. 470 00:20:51,670 --> 00:20:53,770 And I have the feeling that I'm inside 471 00:20:53,770 --> 00:20:57,070 the scene instead of looking out through a tiny, little glass 472 00:20:57,070 --> 00:21:00,190 window to a tiny world that lies behind there. 473 00:21:00,190 --> 00:21:04,960 So now I'm going to try to run the magic of video again. 474 00:21:04,960 --> 00:21:06,460 Good luck to all of us. 475 00:21:06,460 --> 00:21:09,300 476 00:21:09,300 --> 00:21:12,420 I forgot to roll one video because I got discombobulated. 477 00:21:12,420 --> 00:21:18,390 Let me back up for a moment to the issue of agents and persona 478 00:21:18,390 --> 00:21:22,925 and show you that little bit of Microsoft tape. 479 00:21:22,925 --> 00:21:23,592 [VIDEO PLAYBACK] 480 00:21:23,592 --> 00:21:26,810 - --conversational assistant which interacts with the user 481 00:21:26,810 --> 00:21:29,240 in a natural spoken dialogue. 482 00:21:29,240 --> 00:21:32,660 Our eventual goal is to allow users to express themselves 483 00:21:32,660 --> 00:21:35,360 within a limited domain in whatever way they 484 00:21:35,360 --> 00:21:36,620 find most comfortable. 485 00:21:36,620 --> 00:21:40,380 And our system will be able to respond appropriately. 486 00:21:40,380 --> 00:21:43,430 In addition, our system will be given a rich visual presence 487 00:21:43,430 --> 00:21:46,840 using reactive three-dimensional graphics, expressive audio-- 488 00:21:46,840 --> 00:21:47,390 - What? 489 00:21:47,390 --> 00:21:51,300 --and cinematic techniques. 490 00:21:51,300 --> 00:21:51,910 [INAUDIBLE]? 491 00:21:51,910 --> 00:21:52,410 Madonna? 492 00:21:52,410 --> 00:21:55,794 493 00:21:55,794 --> 00:21:56,294 [CLATTER] 494 00:21:56,294 --> 00:21:59,710 [CRASH] 495 00:21:59,710 --> 00:22:05,066 [WHOOSHING] 496 00:22:05,066 --> 00:22:05,566 [POP] 497 00:22:05,566 --> 00:22:08,020 [MUSICAL NOTES] 498 00:22:08,020 --> 00:22:11,170 - I [INAUDIBLE] 1990. 499 00:22:11,170 --> 00:22:13,780 Would you like to hear something from that? 500 00:22:13,780 --> 00:22:15,940 - Play a couple of songs from that. 501 00:22:15,940 --> 00:22:18,790 502 00:22:18,790 --> 00:22:24,160 - How about "Like a Prayer" and "Justify My Love"? 503 00:22:24,160 --> 00:22:26,900 - Sounds good. 504 00:22:26,900 --> 00:22:27,609 - OK. 505 00:22:27,609 --> 00:22:28,192 [END PLAYBACK] 506 00:22:28,192 --> 00:22:30,160 [LAUGHTER] 507 00:22:30,160 --> 00:22:32,420 Now, it's a research project. 508 00:22:32,420 --> 00:22:34,370 It's not yet ready for prime time. 509 00:22:34,370 --> 00:22:39,910 But don't think of social agents like Bob as the answer. 510 00:22:39,910 --> 00:22:42,730 We're much more interested in this kind of technology. 511 00:22:42,730 --> 00:22:46,157 Anyhow, agents, another story for another day. 512 00:22:46,157 --> 00:22:47,740 And I'm going to talk for a little bit 513 00:22:47,740 --> 00:22:50,260 about virtual reality just so we're calibrated 514 00:22:50,260 --> 00:22:51,950 on what I mean by that term. 515 00:22:51,950 --> 00:22:56,064 So here's my first example of that. 516 00:22:56,064 --> 00:22:59,930 May we douse the lights again, please? 517 00:22:59,930 --> 00:23:04,178 This is the NASA virtual wind tunnel project. 518 00:23:04,178 --> 00:23:05,666 [VIDEO PLAYBACK] 519 00:23:05,666 --> 00:23:09,138 [MUSIC PLAYING] 520 00:23:09,138 --> 00:23:10,626 521 00:23:10,626 --> 00:23:14,140 - Before the first flight of the space shuttle, 522 00:23:14,140 --> 00:23:18,640 models were tested using wind tunnels and computers. 523 00:23:18,640 --> 00:23:21,490 The virtual wind tunnel project combines advantages of both 524 00:23:21,490 --> 00:23:24,130 by placing a scientist directly inside a computer-generated 525 00:23:24,130 --> 00:23:25,052 flow field. 526 00:23:25,052 --> 00:23:28,150 527 00:23:28,150 --> 00:23:31,870 CRTs present computer-generated wide-field stereo images 528 00:23:31,870 --> 00:23:35,480 from the user's point of view. 529 00:23:35,480 --> 00:23:37,290 [END PLAYBACK] 530 00:23:37,290 --> 00:23:41,370 OK, so you're inside, and you're wearing kinky black clothing. 531 00:23:41,370 --> 00:23:43,470 And you're gesturing hypnotically. 532 00:23:43,470 --> 00:23:49,260 That's sort of a non-scientific definition of VR. 533 00:23:49,260 --> 00:23:51,570 It doesn't have to be that way, of course. 534 00:23:51,570 --> 00:23:57,510 And see channel two. 535 00:23:57,510 --> 00:23:59,610 What we're looking for is stuff that might even 536 00:23:59,610 --> 00:24:03,580 work pretty comfortably in the privacy of your office, 537 00:24:03,580 --> 00:24:06,510 and so desktop virtual reality or fish tank 538 00:24:06,510 --> 00:24:09,450 virtual reality where you're wearing stereo glasses. 539 00:24:09,450 --> 00:24:13,170 It's being head tracked, and you may have some gear on your hand 540 00:24:13,170 --> 00:24:15,360 with which you can grab and manipulate objects. 541 00:24:15,360 --> 00:24:19,980 That's another kind of probably more accessible, cheaper, 542 00:24:19,980 --> 00:24:22,620 and safer kinds of VR. 543 00:24:22,620 --> 00:24:25,140 Now, why am I concerned about VR and using it 544 00:24:25,140 --> 00:24:29,130 as a driving application, in Fred Brooks's 545 00:24:29,130 --> 00:24:33,390 terminology, for talking about post-WIMP interfaces? 546 00:24:33,390 --> 00:24:36,150 Can we pull the lights down just a little? 547 00:24:36,150 --> 00:24:37,530 You really don't need to see me. 548 00:24:37,530 --> 00:24:41,160 And I'd like to have the screen show up better. 549 00:24:41,160 --> 00:24:43,950 What we have here is a little trip 550 00:24:43,950 --> 00:24:47,130 in virtual reality by our heroine. 551 00:24:47,130 --> 00:24:50,040 And the reason I thought this was amusing 552 00:24:50,040 --> 00:24:53,310 was that she still has a Macintosh style pull-down 553 00:24:53,310 --> 00:24:55,530 menu hanging from the menu bar. 554 00:24:55,530 --> 00:24:57,600 And I want you to think about whether you 555 00:24:57,600 --> 00:25:00,930 want to be walking around in some kind of virtual reality, 556 00:25:00,930 --> 00:25:04,740 but still use this 20-year-old WIMP pull-down menu paradigm. 557 00:25:04,740 --> 00:25:07,720 I don't think that fits very well. 558 00:25:07,720 --> 00:25:11,460 So what is the characterisation of post-WIMP interfaces? 559 00:25:11,460 --> 00:25:15,240 Multiple channels going full bore simultaneously 560 00:25:15,240 --> 00:25:17,370 and probably multiple participants. 561 00:25:17,370 --> 00:25:20,370 It's very important for us to raise our sights 562 00:25:20,370 --> 00:25:24,990 and not think about computing as what I call a solitary vice. 563 00:25:24,990 --> 00:25:27,480 That is something that we do by ourselves. 564 00:25:27,480 --> 00:25:29,430 Computing, more and more in the future, 565 00:25:29,430 --> 00:25:32,580 is going to be a group activity, things where 566 00:25:32,580 --> 00:25:38,730 we work collaboratively in teams together, possibly not located 567 00:25:38,730 --> 00:25:43,560 in space and time, but through the medium of the computer, 568 00:25:43,560 --> 00:25:46,470 working together nonetheless-- multiple participants 569 00:25:46,470 --> 00:25:50,580 in shared virtual worlds, and not just for tank battle 570 00:25:50,580 --> 00:25:52,110 simulations. 571 00:25:52,110 --> 00:25:54,990 When we are walking around in the simulated world, 572 00:25:54,990 --> 00:25:58,830 gesturing with our hands, moving our heads, talking and so 573 00:25:58,830 --> 00:26:00,540 on all at the same time, we don't 574 00:26:00,540 --> 00:26:03,060 have the deterministic key press which 575 00:26:03,060 --> 00:26:05,740 makes WIMP interface decoding so much easier. 576 00:26:05,740 --> 00:26:06,750 It's probabilistic. 577 00:26:06,750 --> 00:26:09,990 We have to be able to backtrack when we make a wrong guess. 578 00:26:09,990 --> 00:26:12,150 We have to be able to disambiguate. 579 00:26:12,150 --> 00:26:13,770 We have to deal with objects that 580 00:26:13,770 --> 00:26:15,570 have their own independent behavior, that 581 00:26:15,570 --> 00:26:20,670 don't sit still as they do in PowerPoint on the screen. 582 00:26:20,670 --> 00:26:24,930 So Negropontes put that their demo from the '70s 583 00:26:24,930 --> 00:26:29,570 is exactly a good paradigm for what we're talking about. 584 00:26:29,570 --> 00:26:32,790 So I'm walking around in the simulated world. 585 00:26:32,790 --> 00:26:34,800 There are objects flying around and doing 586 00:26:34,800 --> 00:26:36,810 their own autonomous behavior. 587 00:26:36,810 --> 00:26:41,790 And I say, put that, as I reach out and gesture and speak, 588 00:26:41,790 --> 00:26:42,300 there. 589 00:26:42,300 --> 00:26:44,310 Put that over there. 590 00:26:44,310 --> 00:26:45,270 What is that? 591 00:26:45,270 --> 00:26:46,290 What is there? 592 00:26:46,290 --> 00:26:47,190 What is noun? 593 00:26:47,190 --> 00:26:48,330 What is verb? 594 00:26:48,330 --> 00:26:50,920 All of that has to be disambiguated. 595 00:26:50,920 --> 00:26:53,130 And that's a very, very hard job. 596 00:26:53,130 --> 00:26:56,820 And WIMP interfaces give us no handle on that. 597 00:26:56,820 --> 00:26:59,130 Why am I so interested in WIMP interfaces? 598 00:26:59,130 --> 00:27:00,933 Not just they're sexy, they're neat, 599 00:27:00,933 --> 00:27:02,100 but they're going to happen. 600 00:27:02,100 --> 00:27:04,680 They're going to come upon us for the usual push 601 00:27:04,680 --> 00:27:06,030 and pull reasons. 602 00:27:06,030 --> 00:27:09,960 Pushing us is the technology, personal digital assistants, 603 00:27:09,960 --> 00:27:12,600 the immense 3D graphics power that we're 604 00:27:12,600 --> 00:27:16,410 going to have for almost free in our settop and game boxes, 605 00:27:16,410 --> 00:27:18,900 more than we get in most high-end workstations 606 00:27:18,900 --> 00:27:20,010 for that matter. 607 00:27:20,010 --> 00:27:22,020 And the fact is that in '95, we'll 608 00:27:22,020 --> 00:27:25,920 see commodity 3D accelerators for several hundred dollars 609 00:27:25,920 --> 00:27:27,480 that all of a sudden will open up 610 00:27:27,480 --> 00:27:30,660 the technology in the same way that bitmap graphics 611 00:27:30,660 --> 00:27:33,960 accelerators got us out of the ASCII world. 612 00:27:33,960 --> 00:27:36,150 Virtual reality, particularly for games, 613 00:27:36,150 --> 00:27:37,920 is going to play a big part. 614 00:27:37,920 --> 00:27:41,190 What kind of applications are people putting together? 615 00:27:41,190 --> 00:27:42,510 Games especially. 616 00:27:42,510 --> 00:27:45,060 They are going to be driving our industry. 617 00:27:45,060 --> 00:27:48,270 I hate the games personally, but they are 618 00:27:48,270 --> 00:27:49,860 what is driving our industry. 619 00:27:49,860 --> 00:27:53,400 And 3D games are far more compelling than 2D ones. 620 00:27:53,400 --> 00:27:56,760 Interactive television, if that can ever be done economically. 621 00:27:56,760 --> 00:28:00,240 Online shopping, not on a 2D representation 622 00:28:00,240 --> 00:28:01,830 but in a virtual mall where you'll 623 00:28:01,830 --> 00:28:04,620 be able to walk in a store, and open up 624 00:28:04,620 --> 00:28:07,470 the cabinets and the showcases, and look at the contents, 625 00:28:07,470 --> 00:28:10,470 and spent lots of money, the developers hope. 626 00:28:10,470 --> 00:28:12,210 Virtual teleconferencing where we're 627 00:28:12,210 --> 00:28:16,230 doing group work together, and we're gesturing about objects 628 00:28:16,230 --> 00:28:18,030 and how they ought to be modified. 629 00:28:18,030 --> 00:28:22,290 Scientific visualization where we're inside our vortices 630 00:28:22,290 --> 00:28:24,810 as in this wind tunnel example. 631 00:28:24,810 --> 00:28:28,110 And all kinds of trainers, starting with flight simulators 632 00:28:28,110 --> 00:28:28,620 and so on. 633 00:28:28,620 --> 00:28:31,230 Those are among the ones. 634 00:28:31,230 --> 00:28:32,760 Well, why is 3D so hot? 635 00:28:32,760 --> 00:28:34,260 What are the potential benefits? 636 00:28:34,260 --> 00:28:35,890 Emphasis on the word "potential," 637 00:28:35,890 --> 00:28:37,690 haven't been realized yet. 638 00:28:37,690 --> 00:28:41,170 And also realize that it's extremely task dependent. 639 00:28:41,170 --> 00:28:42,910 Some tasks will not work well. 640 00:28:42,910 --> 00:28:46,420 I don't want to do my bloody word processing in 3D. 641 00:28:46,420 --> 00:28:50,200 I'll do it on my flat screen, thank you very much. 642 00:28:50,200 --> 00:28:54,520 But other things that require physical movement, for example, 643 00:28:54,520 --> 00:28:58,750 or estimations of size are much better done not just in 3D 644 00:28:58,750 --> 00:29:01,070 but in VR, is the claim. 645 00:29:01,070 --> 00:29:03,850 So I want to be able to leverage the natural abilities that I've 646 00:29:03,850 --> 00:29:06,670 developed all my life walking around, reaching out 647 00:29:06,670 --> 00:29:09,100 for objects, moving things and so on. 648 00:29:09,100 --> 00:29:12,580 In particular, I want to be able to reduce the physical as well 649 00:29:12,580 --> 00:29:16,510 as the cognitive distance between me and the task 650 00:29:16,510 --> 00:29:17,760 that I'm trying to execute. 651 00:29:17,760 --> 00:29:21,100 And I'm going to explain that via a bit more of videotape. 652 00:29:21,100 --> 00:29:25,440 653 00:29:25,440 --> 00:29:26,540 Lights down, please. 654 00:29:26,540 --> 00:29:36,893 655 00:29:36,893 --> 00:29:39,851 [VIDEO PLAYBACK] 656 00:29:39,851 --> 00:29:42,200 - The deformation operators are powerful, 657 00:29:42,200 --> 00:29:44,360 high-level tools for manipulating the geometry 658 00:29:44,360 --> 00:29:45,440 of objects. 659 00:29:45,440 --> 00:29:47,360 However, users struggle to control 660 00:29:47,360 --> 00:29:49,280 as many as 14 different parameters 661 00:29:49,280 --> 00:29:51,470 in this 2D array of sliders. 662 00:29:51,470 --> 00:29:54,590 These one-dimensional sliders have a number of disadvantages. 663 00:29:54,590 --> 00:29:57,110 First, they consume a great deal of screen real estate. 664 00:29:57,110 --> 00:29:59,765 Second, they artificially separate related parameters 665 00:29:59,765 --> 00:30:02,140 from each other, such as the three components of a vector 666 00:30:02,140 --> 00:30:03,650 or point. 667 00:30:03,650 --> 00:30:05,960 Lastly, they remove all controls for parameters 668 00:30:05,960 --> 00:30:08,270 from the 3D view of the object, increasing 669 00:30:08,270 --> 00:30:09,890 the distance between user intention 670 00:30:09,890 --> 00:30:11,550 and their actual effects. 671 00:30:11,550 --> 00:30:13,130 We can see how difficult it is here 672 00:30:13,130 --> 00:30:14,840 to relate the positions of the sliders 673 00:30:14,840 --> 00:30:17,045 to the actual deformations performed on the object. 674 00:30:17,045 --> 00:30:33,060 675 00:30:33,060 --> 00:30:35,530 Our original rack combines the twist, taper, 676 00:30:35,530 --> 00:30:38,470 and bend operations in a single widget. 677 00:30:38,470 --> 00:30:40,900 With three handles, we control 10 degrees of freedom 678 00:30:40,900 --> 00:30:42,910 with one and two-way constraints. 679 00:30:42,910 --> 00:30:44,590 The angles and positions of the bars 680 00:30:44,590 --> 00:30:46,930 control and indicate the parameter values 681 00:30:46,930 --> 00:30:48,460 of the deformations. 682 00:30:48,460 --> 00:30:51,070 The handles are modeled to look like actual handles 683 00:30:51,070 --> 00:30:52,992 so that they invite twisting or moving, 684 00:30:52,992 --> 00:30:55,450 but they do not indicate what their functions are until one 685 00:30:55,450 --> 00:30:56,786 actually pulls on them. 686 00:30:56,786 --> 00:31:13,320 687 00:31:13,320 --> 00:31:15,870 By introducing a suggestive visual representation 688 00:31:15,870 --> 00:31:17,760 for each of the handles, the rack widget 689 00:31:17,760 --> 00:31:19,740 becomes more self disclosing. 690 00:31:19,740 --> 00:31:21,710 For instance, the twisted shape of the twist 691 00:31:21,710 --> 00:31:23,760 handle indicates its function. 692 00:31:23,760 --> 00:31:25,147 By pulling on the twisted handle, 693 00:31:25,147 --> 00:31:26,730 the object twists in direct proportion 694 00:31:26,730 --> 00:31:29,190 to the angle through which the handle is rotated. 695 00:31:29,190 --> 00:31:31,020 When we push down on the taper handle, 696 00:31:31,020 --> 00:31:33,990 the object is made to taper down to the height of the bar. 697 00:31:33,990 --> 00:31:35,820 When we pull on the bent handle, the object 698 00:31:35,820 --> 00:31:37,395 bends in the same direction. 699 00:31:37,395 --> 00:31:40,420 All of the operations occur in the range between the two twist 700 00:31:40,420 --> 00:31:41,880 and taper handles. 701 00:31:41,880 --> 00:31:43,560 The rack itself can be manipulated 702 00:31:43,560 --> 00:31:48,372 to change the axis along which the deformation is applied. 703 00:31:48,372 --> 00:31:49,870 [END PLAYBACK] 704 00:31:49,870 --> 00:31:54,040 OK, so the idea is simply that you 705 00:31:54,040 --> 00:31:57,730 act as if you're in the virtual world with a set of tools 706 00:31:57,730 --> 00:32:01,690 at your disposal, a vice, a set of pliers, whatever. 707 00:32:01,690 --> 00:32:04,780 And those tools are part of the three-dimensional scene, 708 00:32:04,780 --> 00:32:07,600 as opposed to what I think of as the television model where 709 00:32:07,600 --> 00:32:10,600 I have some sliders mounted elsewhere on the screen. 710 00:32:10,600 --> 00:32:12,760 And I'm pulling on the slider while I'm 711 00:32:12,760 --> 00:32:14,020 peering into the world. 712 00:32:14,020 --> 00:32:17,570 That's indirect and has greater cognitive distance. 713 00:32:17,570 --> 00:32:21,730 So what we want to do also as much as possible 714 00:32:21,730 --> 00:32:24,820 is to shift from cognition, where I'm explicitly 715 00:32:24,820 --> 00:32:28,570 aware of reasoning and thinking my way through a problem, 716 00:32:28,570 --> 00:32:31,870 to having the perceptual system that I'm comfortable using 717 00:32:31,870 --> 00:32:34,210 in the 3D world take over. 718 00:32:34,210 --> 00:32:37,210 So it's very important that we computer scientists, 719 00:32:37,210 --> 00:32:40,570 as we think about these matters, learn from our colleagues 720 00:32:40,570 --> 00:32:43,990 in perceptual psychology about what cues are important 721 00:32:43,990 --> 00:32:47,500 and how human beings perform tasks in the real world. 722 00:32:47,500 --> 00:32:50,110 That wasn't necessary for 2D GUIs. 723 00:32:50,110 --> 00:32:53,200 We won't be able to do it properly in the 3D world 724 00:32:53,200 --> 00:32:54,790 if we don't learn from these folks. 725 00:32:54,790 --> 00:32:57,760 I'll have more to say about that in a moment, too. 726 00:32:57,760 --> 00:33:00,050 What I'm going to show you next is to say, 727 00:33:00,050 --> 00:33:02,110 well, I talk about tools in the 3D world, 728 00:33:02,110 --> 00:33:04,270 and that was pretty easy to understand 729 00:33:04,270 --> 00:33:07,750 in the case of trying to mold this piece of clay 730 00:33:07,750 --> 00:33:09,310 with appropriate tools. 731 00:33:09,310 --> 00:33:12,280 But can we extend this and say, what about information 732 00:33:12,280 --> 00:33:14,950 visualization where the information intrinsically 733 00:33:14,950 --> 00:33:16,720 doesn't have to be geometrical? 734 00:33:16,720 --> 00:33:19,900 So I'm now going to roll you a little clip from Xerox Palo 735 00:33:19,900 --> 00:33:24,310 Alto Research Center, Stu Card, George Robertson, et al. 736 00:33:24,310 --> 00:33:27,490 And these folks have spun off some of the work 737 00:33:27,490 --> 00:33:32,620 that you're about to see to their XSoft company. 738 00:33:32,620 --> 00:33:35,590 And they're making use not just of three dimensions but also 739 00:33:35,590 --> 00:33:39,910 of time, animation to say we have greater information 740 00:33:39,910 --> 00:33:41,170 density. 741 00:33:41,170 --> 00:33:45,180 Actually, before I do that, let me show you 742 00:33:45,180 --> 00:33:49,230 an alternative which some people may say they actually prefer. 743 00:33:49,230 --> 00:33:51,990 This is Ben Shneiderman's treemap. 744 00:33:51,990 --> 00:33:55,230 And he can show an amazing amount of information 745 00:33:55,230 --> 00:33:57,490 in a very small amount of real estate. 746 00:33:57,490 --> 00:33:59,970 This is a file system. 747 00:33:59,970 --> 00:34:03,930 And size is proportional to the number of bytes. 748 00:34:03,930 --> 00:34:06,600 And any time you see a rectangular container, 749 00:34:06,600 --> 00:34:11,340 you see its contained entities, and that of course nests. 750 00:34:11,340 --> 00:34:13,440 So this is a nice static overview, 751 00:34:13,440 --> 00:34:15,670 and you can definitely learn a lot from it. 752 00:34:15,670 --> 00:34:18,989 But the folks at PARC bet that by making things three 753 00:34:18,989 --> 00:34:21,690 dimensional and by using the time dimension, 754 00:34:21,690 --> 00:34:24,540 they can give you more in the same amount of time 755 00:34:24,540 --> 00:34:27,179 and give you a higher level of understanding. 756 00:34:27,179 --> 00:34:29,909 So let me show you that. 757 00:34:29,909 --> 00:34:40,113 758 00:34:40,113 --> 00:34:40,780 [VIDEO PLAYBACK] 759 00:34:40,780 --> 00:34:43,030 - This visualization, called the cone tree, 760 00:34:43,030 --> 00:34:46,090 shows part of a Unix directory hierarchy. 761 00:34:46,090 --> 00:34:48,610 Each node is a directory. 762 00:34:48,610 --> 00:34:50,770 When a directory is selected, the tree 763 00:34:50,770 --> 00:34:52,719 rotates through the shortest path 764 00:34:52,719 --> 00:34:55,630 to bring the selected path to the front. 765 00:34:55,630 --> 00:35:00,430 The selected path is highlighted with the directory name. 766 00:35:00,430 --> 00:35:02,530 3D is used to make more effective use 767 00:35:02,530 --> 00:35:04,900 of available screen space. 768 00:35:04,900 --> 00:35:08,470 A 2D layout of the same tree would not fit on the screen. 769 00:35:08,470 --> 00:35:11,410 The user would have to scroll to 2D the layout 770 00:35:11,410 --> 00:35:14,530 and wouldn't have a view of the whole tree. 771 00:35:14,530 --> 00:35:16,540 Interactive animation is used to shift 772 00:35:16,540 --> 00:35:18,130 some of the user's cognitive load 773 00:35:18,130 --> 00:35:20,830 to the human perceptual system. 774 00:35:20,830 --> 00:35:23,140 You can see this by looking at a selection done 775 00:35:23,140 --> 00:35:25,150 without animation. 776 00:35:25,150 --> 00:35:28,120 It takes several seconds to re-assimilate the relationships 777 00:35:28,120 --> 00:35:30,730 between the parts. 778 00:35:30,730 --> 00:35:32,980 If we walk into the next room, we 779 00:35:32,980 --> 00:35:35,710 see an alternative layout which allows text 780 00:35:35,710 --> 00:35:39,580 to be displayed for all nodes. 781 00:35:39,580 --> 00:35:42,970 This visualization is called the cam tree. 782 00:35:42,970 --> 00:35:45,430 As before, selection rotates the tree 783 00:35:45,430 --> 00:35:48,460 to move the selected path up to the front. 784 00:35:48,460 --> 00:35:51,670 The perspective wall is used to lay out 2D information 785 00:35:51,670 --> 00:35:54,490 onto a 3D segmented wall. 786 00:35:54,490 --> 00:35:57,220 The front segment is used for viewing details. 787 00:35:57,220 --> 00:36:00,260 And the side segments provide context. 788 00:36:00,260 --> 00:36:02,740 In this case, each node represents a file 789 00:36:02,740 --> 00:36:04,600 with its horizontal position determined 790 00:36:04,600 --> 00:36:07,690 by its date of last modification. 791 00:36:07,690 --> 00:36:12,010 Vertical position is used to indicate the class of the file. 792 00:36:12,010 --> 00:36:14,320 When a node is selected, the wall 793 00:36:14,320 --> 00:36:16,900 moves to bring the selected node to the center 794 00:36:16,900 --> 00:36:18,610 of the front section. 795 00:36:18,610 --> 00:36:21,340 This is the 3D rooms overview. 796 00:36:21,340 --> 00:36:24,010 Like a rooms overview, it shows all the rooms 797 00:36:24,010 --> 00:36:25,700 that are available. 798 00:36:25,700 --> 00:36:28,180 However, it is also active. 799 00:36:28,180 --> 00:36:30,260 Everything that you can do in a room 800 00:36:30,260 --> 00:36:32,262 you can do from the overview. 801 00:36:32,262 --> 00:36:34,070 [END PLAYBACK] 802 00:36:34,070 --> 00:36:36,010 So we could look at that last image 803 00:36:36,010 --> 00:36:40,330 as essentially a 3D desktop which is tiled. 804 00:36:40,330 --> 00:36:42,430 And of course, you can zoom in and blow up 805 00:36:42,430 --> 00:36:50,300 the window in the same way that you do that on the 2D version. 806 00:36:50,300 --> 00:36:53,970 OK, so switch back here. 807 00:36:53,970 --> 00:36:56,700 808 00:36:56,700 --> 00:37:00,050 So another benefit is that we have many more things 809 00:37:00,050 --> 00:37:02,510 that we can play with, many more special effects. 810 00:37:02,510 --> 00:37:05,940 We have an incredibly larger number of things, 811 00:37:05,940 --> 00:37:09,090 including texture mapping, lighting, and so on. 812 00:37:09,090 --> 00:37:12,740 But that complexity is both a feature and a bug. 813 00:37:12,740 --> 00:37:16,070 It takes a lot more design knowledge and implementation 814 00:37:16,070 --> 00:37:19,590 ability to be able to implement 3D user interfaces. 815 00:37:19,590 --> 00:37:22,730 So let me say now, whatever I said about 3D 816 00:37:22,730 --> 00:37:25,490 is true to the max when we consider 817 00:37:25,490 --> 00:37:29,300 virtual reality and its potential benefits 818 00:37:29,300 --> 00:37:31,310 as well as its current problems. 819 00:37:31,310 --> 00:37:35,210 It's especially good, the claim is, for problems where 820 00:37:35,210 --> 00:37:37,920 I have to make size or precision judgments, 821 00:37:37,920 --> 00:37:41,180 which I'm very comfortable doing in the real world 822 00:37:41,180 --> 00:37:45,120 because my whole system has been trained since birth to do that. 823 00:37:45,120 --> 00:37:47,720 So in particular, if I want to do a walkthrough 824 00:37:47,720 --> 00:37:49,760 through a new building, I can better 825 00:37:49,760 --> 00:37:52,970 simulate that by walking around on a treadmill 826 00:37:52,970 --> 00:37:56,030 or just standing in place and looking around 827 00:37:56,030 --> 00:37:57,800 than looking at a little scale model, 828 00:37:57,800 --> 00:38:00,170 because things will appear more like they will 829 00:38:00,170 --> 00:38:03,540 when the building is finished in that virtual reality. 830 00:38:03,540 --> 00:38:04,250 This is a claim. 831 00:38:04,250 --> 00:38:06,080 No usability studies have been done 832 00:38:06,080 --> 00:38:08,600 to prove that that is true for a mass of people. 833 00:38:08,600 --> 00:38:10,640 But I certainly buy into it. 834 00:38:10,640 --> 00:38:13,880 The Boeing 77 was extensively studied 835 00:38:13,880 --> 00:38:17,210 from a reachability point of view in a VR rig 836 00:38:17,210 --> 00:38:20,630 where you want to know standing in the hall, in the aisle, 837 00:38:20,630 --> 00:38:24,740 what can you reach in the way of a cupboard upstairs? 838 00:38:24,740 --> 00:38:27,440 Or for a maintenance person, how does he or she 839 00:38:27,440 --> 00:38:30,920 have to contort him or herself to get at something? 840 00:38:30,920 --> 00:38:33,410 These are things that you can simulate in a VR trainer 841 00:38:33,410 --> 00:38:36,800 that a desktop simulation is utterly useless for. 842 00:38:36,800 --> 00:38:40,490 It's also good when we want to involve multiple channels, 843 00:38:40,490 --> 00:38:43,100 and that's a hallmark of VR, and do 844 00:38:43,100 --> 00:38:46,880 things that are pretty natural like making small size 845 00:38:46,880 --> 00:38:47,780 adjustments. 846 00:38:47,780 --> 00:38:49,790 And one of the most beautiful examples 847 00:38:49,790 --> 00:38:52,910 of this that I've seen that's actually used in production 848 00:38:52,910 --> 00:38:56,240 is a joint project between University of North Carolina 849 00:38:56,240 --> 00:38:58,340 and one of the California universities, 850 00:38:58,340 --> 00:38:59,930 called the nanoManipulator, where 851 00:38:59,930 --> 00:39:03,410 they are steering a scanning tunneling microscope 852 00:39:03,410 --> 00:39:09,140 and are literally moving atoms around, and feeling the forces, 853 00:39:09,140 --> 00:39:12,920 and seeing things as the scan is being done in real time, 854 00:39:12,920 --> 00:39:16,730 and get to use their proprioceptic system, 855 00:39:16,730 --> 00:39:20,300 their eyes, their muscles, their skeleton, everything 856 00:39:20,300 --> 00:39:23,915 to feel their way around in a trial and error sort of way. 857 00:39:23,915 --> 00:39:25,790 You can imagine that there are a lot of folks 858 00:39:25,790 --> 00:39:29,600 interested in surgical training for much the same reasons. 859 00:39:29,600 --> 00:39:32,070 So I think this is terrific. 860 00:39:32,070 --> 00:39:34,730 Going to show you a raft of examples and then, 861 00:39:34,730 --> 00:39:38,340 having finished motivation, get into some more substance. 862 00:39:38,340 --> 00:39:41,720 So here's an overview of some of the things 863 00:39:41,720 --> 00:39:44,130 I'll show you very quickly. 864 00:39:44,130 --> 00:39:48,230 So this is 3D faked to the max. 865 00:39:48,230 --> 00:39:53,130 This is a beautiful look, but it's standard functionality. 866 00:39:53,130 --> 00:39:56,600 And I think this is as far as we should go. 867 00:39:56,600 --> 00:39:59,480 No more obsessing about 3D interfaces 868 00:39:59,480 --> 00:40:01,520 which aren't really 3D. 869 00:40:01,520 --> 00:40:03,804 Microsoft Bob, enough said. 870 00:40:03,804 --> 00:40:06,530 [LAUGHTER] 871 00:40:06,530 --> 00:40:08,450 Some people actually like it. 872 00:40:08,450 --> 00:40:11,390 I think it would drive me stark raving bonkers 873 00:40:11,390 --> 00:40:13,590 within five minutes. 874 00:40:13,590 --> 00:40:15,810 But it will get better over time. 875 00:40:15,810 --> 00:40:19,640 And it's a very interesting point of departure. 876 00:40:19,640 --> 00:40:22,310 I want to show you Arc Workspace for a moment 877 00:40:22,310 --> 00:40:25,970 because this is a fine example of faked 3D interface. 878 00:40:25,970 --> 00:40:27,560 This is 3D wannabee. 879 00:40:27,560 --> 00:40:29,870 The picture looks 3D, but it isn't 3D. 880 00:40:29,870 --> 00:40:31,700 It's just static 2D. 881 00:40:31,700 --> 00:40:39,450 So switch sources and play. 882 00:40:39,450 --> 00:40:41,250 By the way, feel free to interrupt, 883 00:40:41,250 --> 00:40:44,535 hector, badger, ask questions any time. 884 00:40:44,535 --> 00:40:46,410 It's warm in here, and I'm worried you're all 885 00:40:46,410 --> 00:40:48,690 going to fall asleep on me. 886 00:40:48,690 --> 00:40:51,180 So here is this fake 3D room. 887 00:40:51,180 --> 00:40:54,030 I can get an explanation of what I'm looking at. 888 00:40:54,030 --> 00:40:57,300 I can pick up these tools, the calculator, the drawing 889 00:40:57,300 --> 00:41:01,110 program, the paint brush, and so on. 890 00:41:01,110 --> 00:41:04,290 And, of course, when I need any real information, 891 00:41:04,290 --> 00:41:07,416 you pop up a standard 2D dialog box. 892 00:41:07,416 --> 00:41:09,240 [LAUGHTER] 893 00:41:09,240 --> 00:41:11,610 And you obscure the room, which means 894 00:41:11,610 --> 00:41:15,610 you lose all your spatial memory to a first approximation. 895 00:41:15,610 --> 00:41:20,010 So it's pretty, and it's more appealing, I admit, 896 00:41:20,010 --> 00:41:21,750 for at least the first 15 minutes. 897 00:41:21,750 --> 00:41:23,310 I wouldn't want to use it because it 898 00:41:23,310 --> 00:41:26,280 takes too many pixels, and pixels are scarce. 899 00:41:26,280 --> 00:41:29,100 And I'd rather put more icons on the desk. 900 00:41:29,100 --> 00:41:32,220 But if this moved, and if you could take advantage 901 00:41:32,220 --> 00:41:33,900 of the three, dimensionality that 902 00:41:33,900 --> 00:41:35,550 would be a horse of a different color. 903 00:41:35,550 --> 00:41:38,760 904 00:41:38,760 --> 00:41:42,450 So a little on this I think goes a long way. 905 00:41:42,450 --> 00:41:44,570 I have less than a minute of it. 906 00:41:44,570 --> 00:41:47,620 So here's a project planner that I pulled off the wall. 907 00:41:47,620 --> 00:41:50,580 I can get information in detail. 908 00:41:50,580 --> 00:41:52,510 Here is a file. 909 00:41:52,510 --> 00:41:55,290 I don't know what it is. 910 00:41:55,290 --> 00:41:57,210 Oops. 911 00:41:57,210 --> 00:41:58,830 Still got those problems. 912 00:41:58,830 --> 00:42:01,480 3D interface doesn't save you from that. 913 00:42:01,480 --> 00:42:05,070 And here I can map which icons I want to associate with which 914 00:42:05,070 --> 00:42:07,110 applications. 915 00:42:07,110 --> 00:42:13,080 So a simple, little 3D fake interface. 916 00:42:13,080 --> 00:42:15,430 What makes it fake and not real? 917 00:42:15,430 --> 00:42:17,080 Because I can't move around. 918 00:42:17,080 --> 00:42:19,900 None of the objects are really manipulatable. 919 00:42:19,900 --> 00:42:23,140 They don't have any autonomous behavior. 920 00:42:23,140 --> 00:42:27,693 It's just a different way of portraying 2D icons. 921 00:42:27,693 --> 00:42:30,531 Is it qualitatively different from the cone tree, 922 00:42:30,531 --> 00:42:31,480 for example? 923 00:42:31,480 --> 00:42:35,380 Yes, the cone tree is animated. 924 00:42:35,380 --> 00:42:39,130 It's animated in ways that are responsive to what I'm doing. 925 00:42:39,130 --> 00:42:41,110 Here all you do is you can open things, 926 00:42:41,110 --> 00:42:42,910 and you could put them back on the desktop 927 00:42:42,910 --> 00:42:45,045 and do a minimum amount of rearrangement. 928 00:42:45,045 --> 00:42:46,420 You can change your point of view 929 00:42:46,420 --> 00:42:49,015 arbitrarily in the 3D world in which the cone 930 00:42:49,015 --> 00:42:50,450 tree is embedded. 931 00:42:50,450 --> 00:42:51,680 So it is real 3D. 932 00:42:51,680 --> 00:42:56,600 There's a 3D model, whereas this was just a bitmap. 933 00:42:56,600 --> 00:43:00,770 Another example that I think is finding some significant amount 934 00:43:00,770 --> 00:43:05,480 of commercial use at this point is the Magic Cap interface, 935 00:43:05,480 --> 00:43:10,430 which is not very pretty in the sense 936 00:43:10,430 --> 00:43:12,200 that the previous one was pretty. 937 00:43:12,200 --> 00:43:14,210 It uses a minimum number of pixels. 938 00:43:14,210 --> 00:43:18,140 And I think it's OK as a mnemonic device-- 939 00:43:18,140 --> 00:43:21,260 the desk, the library, controls, standard icons. 940 00:43:21,260 --> 00:43:23,090 Here is the same idea of the desktop 941 00:43:23,090 --> 00:43:24,588 with all the usual artifacts. 942 00:43:24,588 --> 00:43:26,630 One of the things that's cute is you can actually 943 00:43:26,630 --> 00:43:28,550 type on this keyboard. 944 00:43:28,550 --> 00:43:31,070 And it's small, but it works. 945 00:43:31,070 --> 00:43:34,310 And you can dial the telephone with an infrared link. 946 00:43:34,310 --> 00:43:37,050 And I think it's OK. 947 00:43:37,050 --> 00:43:38,900 And here are some standard apps that you 948 00:43:38,900 --> 00:43:42,890 can fire up that you can launch by walking into the store. 949 00:43:42,890 --> 00:43:45,440 And I think if this helps people get familiar 950 00:43:45,440 --> 00:43:48,260 with the technology, fine. 951 00:43:48,260 --> 00:43:50,750 Another thing that probably many of you are familiar with 952 00:43:50,750 --> 00:43:55,190 is the Myst phenomenon, gorgeously pre-rendered images 953 00:43:55,190 --> 00:43:57,337 with a very minimum amount of movement 954 00:43:57,337 --> 00:43:58,670 that you can take through there. 955 00:43:58,670 --> 00:44:01,520 It's really kind of a hypercard application. 956 00:44:01,520 --> 00:44:05,210 Or something that has more possibilities, which 957 00:44:05,210 --> 00:44:08,300 is Apple QuickTime VR, which of course has 958 00:44:08,300 --> 00:44:11,210 zero to do with virtual reality, except that they 959 00:44:11,210 --> 00:44:12,920 like the title. 960 00:44:12,920 --> 00:44:14,750 Virtual reality, from my point of view, 961 00:44:14,750 --> 00:44:18,320 is this immersive experience with objects 962 00:44:18,320 --> 00:44:21,050 that are not static, that have autonomous 963 00:44:21,050 --> 00:44:24,650 behaviors, whereas this is just a very clever way 964 00:44:24,650 --> 00:44:28,790 to duplicate the work that the Negroponte group did, again, 965 00:44:28,790 --> 00:44:31,340 in the '70s with the Aspen walkthrough 966 00:44:31,340 --> 00:44:35,780 where they shot a whole bunch of pictures of Aspen 967 00:44:35,780 --> 00:44:37,730 and then let you essentially put them together 968 00:44:37,730 --> 00:44:40,370 quickly one next to the other to simulate 969 00:44:40,370 --> 00:44:41,660 driving through the town. 970 00:44:41,660 --> 00:44:44,210 Here what they do is they take panoramic views 971 00:44:44,210 --> 00:44:49,160 from a number of nodes and then have a very clever image 972 00:44:49,160 --> 00:44:53,510 transformation program that lets you map one frame into the next 973 00:44:53,510 --> 00:44:56,180 and give you a feeling of being in the room. 974 00:44:56,180 --> 00:44:58,140 So let me roll that for you. 975 00:44:58,140 --> 00:45:00,200 I think this is quite a neat technique. 976 00:45:00,200 --> 00:45:02,630 It's a good hack, and I think it will catch on. 977 00:45:02,630 --> 00:45:05,250 978 00:45:05,250 --> 00:45:06,210 Who needs to go? 979 00:45:06,210 --> 00:45:09,490 980 00:45:09,490 --> 00:45:11,550 Feel free to leave any time you need to. 981 00:45:11,550 --> 00:45:16,940 982 00:45:16,940 --> 00:45:22,450 So here we are in the Starship Enterprise and the main control 983 00:45:22,450 --> 00:45:23,500 room. 984 00:45:23,500 --> 00:45:28,210 And the nodes on the right-hand side in that bird's eye view 985 00:45:28,210 --> 00:45:31,330 tell you where all the camera positions were. 986 00:45:31,330 --> 00:45:33,970 And you can sort of teleport yourself 987 00:45:33,970 --> 00:45:36,620 from one node to another. 988 00:45:36,620 --> 00:45:40,690 And you can see you can pan, and zoom, and change 989 00:45:40,690 --> 00:45:42,970 your point of view, zoom in on something, 990 00:45:42,970 --> 00:45:45,490 get a pretty crude bitmap at first. 991 00:45:45,490 --> 00:45:50,930 And then over time, you'll get a better bitmap. 992 00:45:50,930 --> 00:45:52,652 So it has levels of detail. 993 00:45:52,652 --> 00:45:54,860 And it's rendering, which is quite a common technique 994 00:45:54,860 --> 00:45:56,010 these days. 995 00:45:56,010 --> 00:46:00,020 So for static scenes, I think this is a pretty decent way 996 00:46:00,020 --> 00:46:00,940 of doing things. 997 00:46:00,940 --> 00:46:03,980 You can either pre-render or actually take photographs. 998 00:46:03,980 --> 00:46:06,570 It's the same technique inside the program. 999 00:46:06,570 --> 00:46:07,110 Andy. 1000 00:46:07,110 --> 00:46:07,610 Yes. 1001 00:46:07,610 --> 00:46:11,146 What you can't do here is you can't walk around [INAUDIBLE] 1002 00:46:11,146 --> 00:46:12,040 and take a look. 1003 00:46:12,040 --> 00:46:12,790 Absolutely not. 1004 00:46:12,790 --> 00:46:14,710 There is no occlusion relation. 1005 00:46:14,710 --> 00:46:19,490 These are fixed in time. 1006 00:46:19,490 --> 00:46:23,140 So this is not 3D, and it's certainly not VR. 1007 00:46:23,140 --> 00:46:27,640 But it's a very neat hack that I think has applicability. 1008 00:46:27,640 --> 00:46:32,440 1009 00:46:32,440 --> 00:46:37,600 So we'll stop that, switch channels, 1010 00:46:37,600 --> 00:46:43,570 and talk about actually another development which is 1011 00:46:43,570 --> 00:46:45,572 perhaps only a month or so old. 1012 00:46:45,572 --> 00:46:46,780 We're not seeing it yet, huh? 1013 00:46:46,780 --> 00:46:47,290 Thank you. 1014 00:46:47,290 --> 00:46:51,180 1015 00:46:51,180 --> 00:46:54,600 Again, uses virtual reality for advertising purposes. 1016 00:46:54,600 --> 00:46:57,430 Has nothing to do as yet with virtual reality. 1017 00:46:57,430 --> 00:46:59,100 It's essentially being able to share 1018 00:46:59,100 --> 00:47:01,080 display files over the network. 1019 00:47:01,080 --> 00:47:03,180 Display files are graphics commands 1020 00:47:03,180 --> 00:47:06,420 that paint polygons and lines and so on. 1021 00:47:06,420 --> 00:47:10,170 And this particular combination, how can it miss? 1022 00:47:10,170 --> 00:47:14,160 Virtual reality, World Wide Web, HTML. 1023 00:47:14,160 --> 00:47:15,780 It's a winner, right? 1024 00:47:15,780 --> 00:47:17,500 Never mind whether it's any good or not. 1025 00:47:17,500 --> 00:47:20,610 I think it is just good enough to be above threshold, 1026 00:47:20,610 --> 00:47:22,990 and a lot of people are going to have fun with this. 1027 00:47:22,990 --> 00:47:25,980 I think it will even find commercial applicability. 1028 00:47:25,980 --> 00:47:29,940 Right now, it's not exclusively based on 1029 00:47:29,940 --> 00:47:33,540 but takes good advantage of SGI's Open Inventor class 1030 00:47:33,540 --> 00:47:36,750 library, which is kind of an object-oriented display list 1031 00:47:36,750 --> 00:47:39,940 package, if you will forgive me that shortcut. 1032 00:47:39,940 --> 00:47:43,230 So let's go look at a tiny clip of that 1033 00:47:43,230 --> 00:47:44,730 so you can see what that looks like. 1034 00:47:44,730 --> 00:47:52,400 1035 00:47:52,400 --> 00:47:55,670 How many of you have seen it already, just out of curiosity? 1036 00:47:55,670 --> 00:47:56,250 Not so many. 1037 00:47:56,250 --> 00:47:56,750 Good. 1038 00:47:56,750 --> 00:47:59,660 I have something new to show you. 1039 00:47:59,660 --> 00:48:01,040 A and play. 1040 00:48:01,040 --> 00:48:06,630 1041 00:48:06,630 --> 00:48:11,790 So here is a standard look. 1042 00:48:11,790 --> 00:48:13,980 And now you see we have this little golf cart 1043 00:48:13,980 --> 00:48:17,790 controller at the very bottom here with which I'm rotating, 1044 00:48:17,790 --> 00:48:21,150 or I can change my angle of view. 1045 00:48:21,150 --> 00:48:24,420 And I hope you notice the texture just popped in. 1046 00:48:24,420 --> 00:48:26,770 It took a while to transmit over the net, 1047 00:48:26,770 --> 00:48:28,200 but it finally made it. 1048 00:48:28,200 --> 00:48:31,020 So levels of detail again, very important to get 1049 00:48:31,020 --> 00:48:32,550 real-time response. 1050 00:48:32,550 --> 00:48:34,920 Once we're here, we use whatever power of machine 1051 00:48:34,920 --> 00:48:38,670 we have to work of this inventor display file. 1052 00:48:38,670 --> 00:48:41,700 And this is now running on an SGI in our lab. 1053 00:48:41,700 --> 00:48:44,940 And in this radiosity room, we get pretty decent response 1054 00:48:44,940 --> 00:48:45,510 time. 1055 00:48:45,510 --> 00:48:49,170 So the little doohickey on the right-hand side 1056 00:48:49,170 --> 00:48:54,420 of that telescope-like thing changes my pitch. 1057 00:48:54,420 --> 00:48:57,330 And then I can change my angle. 1058 00:48:57,330 --> 00:49:01,830 And I can also lift myself on the floor. 1059 00:49:01,830 --> 00:49:04,340 So it has appropriate viewing controls. 1060 00:49:04,340 --> 00:49:06,550 They don't take very long to master. 1061 00:49:06,550 --> 00:49:10,230 So this lets me do a share of 3D models across the network. 1062 00:49:10,230 --> 00:49:12,860 And that's a good thing. 1063 00:49:12,860 --> 00:49:13,990 It's primitive now. 1064 00:49:13,990 --> 00:49:18,080 It will get better over time, is the hope. 1065 00:49:18,080 --> 00:49:20,730 Back to the slides. 1066 00:49:20,730 --> 00:49:23,730 Now I want to talk about some real 3D widgets which 1067 00:49:23,730 --> 00:49:25,890 imitate things that I see in the real world, 1068 00:49:25,890 --> 00:49:28,270 like my control knobs on radios. 1069 00:49:28,270 --> 00:49:28,770 Yes. 1070 00:49:28,770 --> 00:49:30,710 What method would [INAUDIBLE]? 1071 00:49:30,710 --> 00:49:34,110 Would you do it [INAUDIBLE]? 1072 00:49:34,110 --> 00:49:39,480 As I understand it, how would you share this 3D space? 1073 00:49:39,480 --> 00:49:44,220 They're still working on sort of unifying the MU 1074 00:49:44,220 --> 00:49:48,120 stream of collaboration with this VRML 1075 00:49:48,120 --> 00:49:51,180 so that you'll be able to have shared experiences 1076 00:49:51,180 --> 00:49:55,470 and be able to have multiple players each controlling 1077 00:49:55,470 --> 00:49:58,877 their version of the scene and seeing avatars that represent 1078 00:49:58,877 --> 00:49:59,460 other players. 1079 00:49:59,460 --> 00:50:03,150 1080 00:50:03,150 --> 00:50:05,970 OK, so the world of real widgets, 1081 00:50:05,970 --> 00:50:08,940 how do we imitate those? 1082 00:50:08,940 --> 00:50:11,430 First of all, what do I mean by 3D user interface? 1083 00:50:11,430 --> 00:50:13,110 It's a collection of 3D widget. 1084 00:50:13,110 --> 00:50:13,980 What is a widget? 1085 00:50:13,980 --> 00:50:18,300 In the X/Motif sense, it has geometry, and it has behavior. 1086 00:50:18,300 --> 00:50:21,030 A 3D user interface, in my book, therefore 1087 00:50:21,030 --> 00:50:24,240 is not a user interface for 3D applications. 1088 00:50:24,240 --> 00:50:26,850 We have many CAD/CAM applications, 1089 00:50:26,850 --> 00:50:30,930 for example, that have panels and panels of motif controls. 1090 00:50:30,930 --> 00:50:34,260 That's the television model where you separate the user 1091 00:50:34,260 --> 00:50:36,990 interface from the world that you're trying to manipulate. 1092 00:50:36,990 --> 00:50:39,810 My controls are in the scene itself. 1093 00:50:39,810 --> 00:50:42,120 And they are made, as we will see in a moment, 1094 00:50:42,120 --> 00:50:46,140 by the same system that makes the application objects. 1095 00:50:46,140 --> 00:50:49,210 And I'm going to run through these examples in a moment. 1096 00:50:49,210 --> 00:50:53,400 Now, being inside the 3D scene for some people 1097 00:50:53,400 --> 00:50:57,300 is achieved already when they're playing a video game like Doom 1098 00:50:57,300 --> 00:50:58,440 or Marathon. 1099 00:50:58,440 --> 00:51:00,235 They sit in front of the screen. 1100 00:51:00,235 --> 00:51:01,110 They are close to it. 1101 00:51:01,110 --> 00:51:02,340 They feel surrounded. 1102 00:51:02,340 --> 00:51:05,940 And the story sucks them in to the 3D space, 1103 00:51:05,940 --> 00:51:07,470 and they feel that they're in it. 1104 00:51:07,470 --> 00:51:10,830 They say that they are immersed, even though they really 1105 00:51:10,830 --> 00:51:13,740 aren't-- for example, they can't turn their heads and see 1106 00:51:13,740 --> 00:51:14,880 what's behind-- 1107 00:51:14,880 --> 00:51:18,690 but that they feel as if they are there. 1108 00:51:18,690 --> 00:51:21,000 And so an important point that I want to make 1109 00:51:21,000 --> 00:51:24,120 is that we talk about technological differentiation 1110 00:51:24,120 --> 00:51:26,680 between what is real VR and what is not. 1111 00:51:26,680 --> 00:51:28,830 But the fact is that, first and foremost, 1112 00:51:28,830 --> 00:51:31,410 it's the experience that the user reports. 1113 00:51:31,410 --> 00:51:34,770 And Doom comes a lot closer to what we really 1114 00:51:34,770 --> 00:51:37,350 want in the future than our traditional computer 1115 00:51:37,350 --> 00:51:38,170 applications. 1116 00:51:38,170 --> 00:51:40,590 And one of the main reasons that it does 1117 00:51:40,590 --> 00:51:43,470 is that it is incredibly responsive. 1118 00:51:43,470 --> 00:51:46,200 It generates a hell of a lot of frames per second. 1119 00:51:46,200 --> 00:51:47,700 And it does so in a guaranteed way 1120 00:51:47,700 --> 00:51:50,880 because they've really optimized for that. 1121 00:51:50,880 --> 00:51:52,980 And our normal applications don't do that. 1122 00:51:52,980 --> 00:51:55,360 I'll have more to say about that. 1123 00:51:55,360 --> 00:51:59,110 So technology and the sensory experience. 1124 00:51:59,110 --> 00:52:03,120 And for real VR, as I showed you in the previous clip, 1125 00:52:03,120 --> 00:52:06,000 the visual channel wants to have head tracking. 1126 00:52:06,000 --> 00:52:08,850 So I really have the feeling that the system is being 1127 00:52:08,850 --> 00:52:10,500 responsive to me and that the world is 1128 00:52:10,500 --> 00:52:13,980 changing just as the real world does when I turn my head. 1129 00:52:13,980 --> 00:52:18,900 It has wide field of view so that I really 1130 00:52:18,900 --> 00:52:20,790 feel that I'm inside. 1131 00:52:20,790 --> 00:52:26,220 And stereoscopic view for depth perception helps a lot. 1132 00:52:26,220 --> 00:52:28,110 Since I walk around in a world that 1133 00:52:28,110 --> 00:52:32,220 has auditory sensations brought to me, I like to hear sound. 1134 00:52:32,220 --> 00:52:34,080 If balls bounce, I want to hear them. 1135 00:52:34,080 --> 00:52:37,920 If things somehow collide, I want to hear that. 1136 00:52:37,920 --> 00:52:40,110 So spatially generated sound is important. 1137 00:52:40,110 --> 00:52:43,800 And increasingly, haptic or tactile feedback 1138 00:52:43,800 --> 00:52:45,000 is becoming important. 1139 00:52:45,000 --> 00:52:46,560 The nanoManipulator, I mentioned. 1140 00:52:46,560 --> 00:52:49,020 Surgical training obviously demand it. 1141 00:52:49,020 --> 00:52:52,170 A lot is going on at MIT and multiple labs. 1142 00:52:52,170 --> 00:52:56,190 I saw this afternoon Ken Salisbury's Phantom device. 1143 00:52:56,190 --> 00:52:58,200 His thesis is that it's better to have 1144 00:52:58,200 --> 00:53:00,420 a limited but very high-resolution 1145 00:53:00,420 --> 00:53:03,750 experience, low lag, high resolution 1146 00:53:03,750 --> 00:53:06,180 than to have something that, for example, involves lots 1147 00:53:06,180 --> 00:53:09,120 of fingers and hands but reports back very crudely 1148 00:53:09,120 --> 00:53:10,050 with lots of delay. 1149 00:53:10,050 --> 00:53:14,010 And I encourage you to go see this nifty device. 1150 00:53:14,010 --> 00:53:16,410 It's quite evocative. 1151 00:53:16,410 --> 00:53:19,710 Well, we don't just have virtual reality where you're immersed, 1152 00:53:19,710 --> 00:53:22,620 and the computer generates all your sensorium. 1153 00:53:22,620 --> 00:53:25,110 I think what will be especially important in the future is 1154 00:53:25,110 --> 00:53:28,290 a variation on the theme, which is augmented reality, where 1155 00:53:28,290 --> 00:53:30,360 we superimpose the real world and the computer 1156 00:53:30,360 --> 00:53:31,830 generated world. 1157 00:53:31,830 --> 00:53:36,360 And an example of that is this thought experiment 1158 00:53:36,360 --> 00:53:38,250 from University of North Carolina, where 1159 00:53:38,250 --> 00:53:39,720 you see the airplane. 1160 00:53:39,720 --> 00:53:41,640 You see parts of it exploded. 1161 00:53:41,640 --> 00:53:45,240 You see documentation in the hypermedia sort of way. 1162 00:53:45,240 --> 00:53:47,520 And the real and the virtual world are combined. 1163 00:53:47,520 --> 00:53:51,060 Here's an early experiment in Columbia University 1164 00:53:51,060 --> 00:53:55,710 by Steve Feiner, KARMA, knowledge-based system 1165 00:53:55,710 --> 00:53:58,080 in which you can see the insides of the copier 1166 00:53:58,080 --> 00:54:00,150 as you get directions on how to repair it, 1167 00:54:00,150 --> 00:54:01,800 kind of X-ray vision. 1168 00:54:01,800 --> 00:54:04,410 And you see that through monocular vision 1169 00:54:04,410 --> 00:54:07,110 superimposed on the real world. 1170 00:54:07,110 --> 00:54:11,520 We are in this whole field in this stage of the game. 1171 00:54:11,520 --> 00:54:15,570 Anybody recognize what this is? 1172 00:54:15,570 --> 00:54:18,120 Early '40s Link Trainer. 1173 00:54:18,120 --> 00:54:21,000 It looks like a toy, sort of like a soapbox derby 1174 00:54:21,000 --> 00:54:21,670 kind of thing. 1175 00:54:21,670 --> 00:54:24,120 And it was a toy, but it saved lives 1176 00:54:24,120 --> 00:54:26,670 because it was the best kind of training that World War II 1177 00:54:26,670 --> 00:54:28,830 pilots could get at the time. 1178 00:54:28,830 --> 00:54:32,220 And it sufficed unto the purpose. 1179 00:54:32,220 --> 00:54:33,300 Not fancy. 1180 00:54:33,300 --> 00:54:36,240 I think we're not even at that stage yet, but we will be, 1181 00:54:36,240 --> 00:54:38,590 and you got to keep the faith. 1182 00:54:38,590 --> 00:54:40,170 So that was the motivation. 1183 00:54:40,170 --> 00:54:43,020 That was supposed to take 20 minutes. 1184 00:54:43,020 --> 00:54:43,950 Oh, well. 1185 00:54:43,950 --> 00:54:45,180 I'm going to be cutting some. 1186 00:54:45,180 --> 00:54:49,470 And any time you have to leave, feel free to walk out. 1187 00:54:49,470 --> 00:54:51,900 Having tried to persuade you with some examples 1188 00:54:51,900 --> 00:54:55,530 that 3D is coming and that these kinds of immersive interfaces 1189 00:54:55,530 --> 00:54:57,750 are on the way, what I want to talk about now 1190 00:54:57,750 --> 00:55:01,560 for about 15 minutes is, from an interface designers 1191 00:55:01,560 --> 00:55:03,740 perspective, how does this new world feel? 1192 00:55:03,740 --> 00:55:05,610 What are the kinds of things we worry about? 1193 00:55:05,610 --> 00:55:08,530 And then we'll talk about implementation. 1194 00:55:08,530 --> 00:55:11,970 First, the devices suck. 1195 00:55:11,970 --> 00:55:14,910 They are completely inadequate unto the task. 1196 00:55:14,910 --> 00:55:16,230 They are uncomfortable. 1197 00:55:16,230 --> 00:55:17,370 They weigh too much. 1198 00:55:17,370 --> 00:55:18,900 They're unhygienic. 1199 00:55:18,900 --> 00:55:23,010 They are infinite latency. 1200 00:55:23,010 --> 00:55:26,460 They are miserably low resolution. 1201 00:55:26,460 --> 00:55:30,390 And we're supposed to perform a multitude of tasks, which 1202 00:55:30,390 --> 00:55:32,340 in the real world we do with specialized tool, 1203 00:55:32,340 --> 00:55:33,840 with one general purpose tool. 1204 00:55:33,840 --> 00:55:35,310 One size does not fit all. 1205 00:55:35,310 --> 00:55:38,110 The mouse even for 2D doesn't do it all. 1206 00:55:38,110 --> 00:55:40,230 And in 3D, the problem is much worse. 1207 00:55:40,230 --> 00:55:42,810 And in 3D VR, it's much more. 1208 00:55:42,810 --> 00:55:44,920 I'm not going to talk about this anymore. 1209 00:55:44,920 --> 00:55:47,460 Instead, I'm going to talk about the next two points 1210 00:55:47,460 --> 00:55:50,650 in some more detail. 1211 00:55:50,650 --> 00:55:52,890 How do we do 3D user interface design? 1212 00:55:52,890 --> 00:55:55,390 Much the way we do 2D user interface design. 1213 00:55:55,390 --> 00:55:57,510 So I'm going to gloss over this very quickly 1214 00:55:57,510 --> 00:55:59,790 in the interest of time and let you read it, 1215 00:55:59,790 --> 00:56:02,640 and simply keep saying, whatever it was in 2D, 1216 00:56:02,640 --> 00:56:04,390 it's more so in 3D. 1217 00:56:04,390 --> 00:56:06,130 So we start with task analysis. 1218 00:56:06,130 --> 00:56:07,380 What are we trying to do? 1219 00:56:07,380 --> 00:56:10,140 Then we have to map from the problem domain 1220 00:56:10,140 --> 00:56:12,870 to a particular 3D scenario. 1221 00:56:12,870 --> 00:56:14,880 How are we going to go about doing this task? 1222 00:56:14,880 --> 00:56:16,890 What are the metaphors we're going to choose? 1223 00:56:16,890 --> 00:56:19,830 How much 3D do we in fact want? 1224 00:56:19,830 --> 00:56:21,510 You could say, well, put somebody 1225 00:56:21,510 --> 00:56:23,490 inside the head-mounted display. 1226 00:56:23,490 --> 00:56:25,980 No one has ever worn a head-mounted display 1227 00:56:25,980 --> 00:56:27,932 for more than about 10 minutes. 1228 00:56:27,932 --> 00:56:29,890 And if you tried to do it for longer than that, 1229 00:56:29,890 --> 00:56:32,850 you would get the most amazing muscle and eye and everything 1230 00:56:32,850 --> 00:56:35,670 else strains that you could imagine. 1231 00:56:35,670 --> 00:56:38,160 Well, these little glasses are a lot better. 1232 00:56:38,160 --> 00:56:40,830 But even still, they are not all that comfortable. 1233 00:56:40,830 --> 00:56:43,890 So you better have good reason for wanting to be immersed. 1234 00:56:43,890 --> 00:56:46,680 And it's easy to say, for intrinsically 3D problems, 1235 00:56:46,680 --> 00:56:48,000 we'll go 3D. 1236 00:56:48,000 --> 00:56:50,430 For information visualization problems, 1237 00:56:50,430 --> 00:56:52,150 looking at large amounts of data, 1238 00:56:52,150 --> 00:56:54,000 the case may be a lot harder. 1239 00:56:54,000 --> 00:56:57,090 Then we got to find suitable devices. 1240 00:56:57,090 --> 00:56:58,680 We got to talk about whether we want 1241 00:56:58,680 --> 00:57:00,460 a highly-schematic representation 1242 00:57:00,460 --> 00:57:02,250 or something that looks as realistic 1243 00:57:02,250 --> 00:57:06,270 as possible for which we have to pay a lot of compute cycles. 1244 00:57:06,270 --> 00:57:08,640 Do we want to directly manipulate, reach in there 1245 00:57:08,640 --> 00:57:10,890 with our data gloves where we have 1246 00:57:10,890 --> 00:57:14,100 to gesture very, well, evocatively 1247 00:57:14,100 --> 00:57:16,110 and suggestively but imprecisely? 1248 00:57:16,110 --> 00:57:18,600 Or do we want some specialized tools 1249 00:57:18,600 --> 00:57:21,900 with limited degrees of freedom that we can control more? 1250 00:57:21,900 --> 00:57:23,850 That's a choice you make. 1251 00:57:23,850 --> 00:57:27,480 You can design a special purpose widget like the rack widget. 1252 00:57:27,480 --> 00:57:30,450 And you, of course, want to involve other channels 1253 00:57:30,450 --> 00:57:33,900 as much as possible and learn from what people have learned 1254 00:57:33,900 --> 00:57:38,640 from in the past, namely the 2D world and the real 3D world. 1255 00:57:38,640 --> 00:57:41,310 You have to learn a lot about perceptive psychology, 1256 00:57:41,310 --> 00:57:42,670 as I mentioned. 1257 00:57:42,670 --> 00:57:47,230 And then there are all these other motherhood principles. 1258 00:57:47,230 --> 00:57:50,880 And I'm not going to take time to elaborate on them, 1259 00:57:50,880 --> 00:57:57,290 except to say all these issues are compounded in 3D. 1260 00:57:57,290 --> 00:58:01,910 Mapping from devices to tasks to interaction techniques. 1261 00:58:01,910 --> 00:58:04,730 And be aware of the lowest common denominator phenomena. 1262 00:58:04,730 --> 00:58:07,530 The mouse is in a sense a lowest common denominator. 1263 00:58:07,530 --> 00:58:09,500 And as great a device as it is, it 1264 00:58:09,500 --> 00:58:12,650 has restricted our ability to do things, even in 2D, 1265 00:58:12,650 --> 00:58:14,405 because it isn't perfect for every task. 1266 00:58:14,405 --> 00:58:18,690 1267 00:58:18,690 --> 00:58:20,490 I'll skip over that point. 1268 00:58:20,490 --> 00:58:21,990 Let me show you some more examples 1269 00:58:21,990 --> 00:58:26,070 because it's much easier on you than looking at these text 1270 00:58:26,070 --> 00:58:27,250 slides. 1271 00:58:27,250 --> 00:58:29,580 So what I'm going to show now is some examples 1272 00:58:29,580 --> 00:58:34,720 of 3D widgets built with 3D construction tools. 1273 00:58:34,720 --> 00:58:36,730 So not the television model. 1274 00:58:36,730 --> 00:58:38,490 I'm going to start by showing you 1275 00:58:38,490 --> 00:58:43,320 some by now fairly classic widgets that the folks at SGI 1276 00:58:43,320 --> 00:58:44,550 did. 1277 00:58:44,550 --> 00:58:48,510 We started with those, and then other people looked at them 1278 00:58:48,510 --> 00:58:51,660 and said, yeah, those make sense. 1279 00:58:51,660 --> 00:58:57,790 And they're by now kind of standard in the industry, 1280 00:58:57,790 --> 00:58:59,790 although there are many variations on the theme. 1281 00:58:59,790 --> 00:59:12,150 1282 00:59:12,150 --> 00:59:17,360 So those arrows indicate that there are textural annotations. 1283 00:59:17,360 --> 00:59:22,310 I want to zoom in and put various kinds of tools 1284 00:59:22,310 --> 00:59:23,450 on that stop sign. 1285 00:59:23,450 --> 00:59:24,530 That was rotation. 1286 00:59:24,530 --> 00:59:26,390 This is another kind of rotation. 1287 00:59:26,390 --> 00:59:29,510 Here is a translation and a scale box. 1288 00:59:29,510 --> 00:59:32,100 I pull at the handles in a moment. 1289 00:59:32,100 --> 00:59:33,860 And then I will be able to change 1290 00:59:33,860 --> 00:59:37,670 its size all in the scene, no menus, no dialog 1291 00:59:37,670 --> 00:59:39,618 boxes, no dials. 1292 00:59:39,618 --> 00:59:41,330 [INAUDIBLE] with the mouse or 3D? 1293 00:59:41,330 --> 00:59:43,340 This is all done with the mouse, yeah. 1294 00:59:43,340 --> 00:59:45,350 They're still very mouse-centric. 1295 00:59:45,350 --> 00:59:49,300 And here we're going to type a little textual annotation. 1296 00:59:49,300 --> 00:59:52,340 I don't particularly care for those handles. 1297 00:59:52,340 --> 00:59:54,280 I think they're kind of intrusive and ugly, 1298 00:59:54,280 --> 00:59:56,800 but it's a way of solving a hard problem. 1299 00:59:56,800 --> 00:59:59,230 I don't know what a better way of doing it would be. 1300 00:59:59,230 --> 01:00:04,410 The T there stands for this is annotated with text. 1301 01:00:04,410 --> 01:00:08,550 So translation, rotation, scale, and textual annotation, 1302 01:00:08,550 --> 01:00:10,050 everybody has got those. 1303 01:00:10,050 --> 01:00:23,820 1304 01:00:23,820 --> 01:00:28,080 Next I'm going to show you some examples of our widgets. 1305 01:00:28,080 --> 01:00:30,040 We have all of those kinds of things. 1306 01:00:30,040 --> 01:00:32,670 But now we have a new idea, which is 1307 01:00:32,670 --> 01:00:35,220 application-specific widgets. 1308 01:00:35,220 --> 01:00:39,120 So these widgets can be used for essentially any task. 1309 01:00:39,120 --> 01:00:42,210 And now what we're going to do is say special purpose tools 1310 01:00:42,210 --> 01:00:44,430 for special purpose jobs. 1311 01:00:44,430 --> 01:00:47,040 And these widgets again are in the scene. 1312 01:00:47,040 --> 01:00:59,440 1313 01:00:59,440 --> 01:01:00,950 [VIDEO PLAYBACK] 1314 01:01:00,950 --> 01:01:03,290 - We have been developing 3D user interfaces 1315 01:01:03,290 --> 01:01:05,540 for exploratory scientific visualization 1316 01:01:05,540 --> 01:01:08,240 of computational fluid dynamics data sets. 1317 01:01:08,240 --> 01:01:09,890 Built on top of the [? OTIS ?] system, 1318 01:01:09,890 --> 01:01:11,690 the 3D widgets presented in this video 1319 01:01:11,690 --> 01:01:14,360 provide fully interactive direct manipulation techniques 1320 01:01:14,360 --> 01:01:16,700 for placing and modifying parameters of commonly 1321 01:01:16,700 --> 01:01:19,070 used visualization techniques. 1322 01:01:19,070 --> 01:01:21,530 To demonstrate the 3D widgets we have developed, 1323 01:01:21,530 --> 01:01:23,030 we will be using a data set computer 1324 01:01:23,030 --> 01:01:25,460 at NASA Ames of airflow past the body of the space 1325 01:01:25,460 --> 01:01:27,470 shuttle in which the velocity is computed 1326 01:01:27,470 --> 01:01:30,482 at each point on a curvilinear grid. 1327 01:01:30,482 --> 01:01:31,940 We have implemented a set of probes 1328 01:01:31,940 --> 01:01:34,460 which generate tufts, streamlines, cutting planes, 1329 01:01:34,460 --> 01:01:36,830 and isosurfaces at their sample points 1330 01:01:36,830 --> 01:01:38,540 and provide interfaces for modifying 1331 01:01:38,540 --> 01:01:40,590 some relevant parameters. 1332 01:01:40,590 --> 01:01:43,280 First, we show a vector probe which samples the vector field 1333 01:01:43,280 --> 01:01:44,210 at a point. 1334 01:01:44,210 --> 01:01:45,830 One visualization technique which 1335 01:01:45,830 --> 01:01:47,960 can be attached to a probe is a streamline. 1336 01:01:47,960 --> 01:01:50,450 The path is colored by velocity from blue to red 1337 01:01:50,450 --> 01:01:52,580 with increasing speed. 1338 01:01:52,580 --> 01:01:55,310 We can transform the point probe into a line, otherwise known 1339 01:01:55,310 --> 01:01:57,530 as a rake, which generates a set of streamlines 1340 01:01:57,530 --> 01:01:58,830 along its length. 1341 01:01:58,830 --> 01:02:01,412 We can position the rake, rotate it, scale it, 1342 01:02:01,412 --> 01:02:02,870 or change the number of streamlines 1343 01:02:02,870 --> 01:02:05,665 on the rake using simple 3D widgets. 1344 01:02:05,665 --> 01:02:07,415 This could be extended to multidimensional 1345 01:02:07,415 --> 01:02:10,460 isosurface probes as well, though the usefulness 1346 01:02:10,460 --> 01:02:12,218 of this technique is uncertain. 1347 01:02:12,218 --> 01:02:15,113 1348 01:02:15,113 --> 01:02:17,280 We have implemented a number of different interfaces 1349 01:02:17,280 --> 01:02:20,400 for specifying the position of visualization tools. 1350 01:02:20,400 --> 01:02:23,070 As we have seen, screen-aligned translation 1351 01:02:23,070 --> 01:02:25,957 is simple but doesn't allow one to translate objects out 1352 01:02:25,957 --> 01:02:26,790 of the screen plane. 1353 01:02:26,790 --> 01:02:32,730 1354 01:02:32,730 --> 01:02:35,100 The shadow on the floor plane is a useful depth cue 1355 01:02:35,100 --> 01:02:36,960 and a widget in its own right. 1356 01:02:36,960 --> 01:02:39,570 Translating an object's interactive shadow widget 1357 01:02:39,570 --> 01:02:42,210 also translates the object itself in the plane parallel 1358 01:02:42,210 --> 01:02:43,890 to the shadow plane. 1359 01:02:43,890 --> 01:02:46,320 Using this technique, we can move the streamline quickly 1360 01:02:46,320 --> 01:02:48,833 to the tip of the wing. 1361 01:02:48,833 --> 01:02:51,000 The shadow widgets have some disadvantages, however, 1362 01:02:51,000 --> 01:02:53,430 when there are large amounts of geometry in the scene. 1363 01:02:53,430 --> 01:02:56,190 Here the shadow body sometimes obscures the streamlined shadow 1364 01:02:56,190 --> 01:02:57,720 on the right wall. 1365 01:02:57,720 --> 01:03:00,000 Another technique called object handles provides 1366 01:03:00,000 --> 01:03:01,950 an alternative direct manipulation interface 1367 01:03:01,950 --> 01:03:05,280 to constrain translation using significantly less geometry 1368 01:03:05,280 --> 01:03:07,260 than the interactive shadows. 1369 01:03:07,260 --> 01:03:08,910 Using this technique, we do not have 1370 01:03:08,910 --> 01:03:11,460 to worry as much about important widgets becoming obscured 1371 01:03:11,460 --> 01:03:13,230 by other geometry in the scene. 1372 01:03:13,230 --> 01:03:15,330 If a selected object is visible, the handles 1373 01:03:15,330 --> 01:03:17,220 will also be visible. 1374 01:03:17,220 --> 01:03:20,040 Here we see a slight variation of the object handles widget. 1375 01:03:20,040 --> 01:03:22,440 Instead of becoming aligned with the Cartesian coordinate 1376 01:03:22,440 --> 01:03:25,350 system, these handles display the basis of the curvilinear 1377 01:03:25,350 --> 01:03:28,650 computation grid at the point closest to the selected object. 1378 01:03:28,650 --> 01:03:30,250 By translating the selected object, 1379 01:03:30,250 --> 01:03:32,460 we can see the local structure of the grid. 1380 01:03:32,460 --> 01:03:34,500 Because the grid is non-linear, the handles 1381 01:03:34,500 --> 01:03:36,240 are not straight lines. 1382 01:03:36,240 --> 01:03:38,580 Using this technique, it is straightforward to translate 1383 01:03:38,580 --> 01:03:40,800 the streamline along any of the lines in the grid, 1384 01:03:40,800 --> 01:03:42,045 including the line which corresponds 1385 01:03:42,045 --> 01:03:42,900 to the [INAUDIBLE]-- 1386 01:03:42,900 --> 01:03:44,940 [END PLAYBACK] 1387 01:03:44,940 --> 01:03:47,220 OK, next thing I'm going to show you 1388 01:03:47,220 --> 01:03:50,520 is an experiment to try to make use of a 6 degree 1389 01:03:50,520 --> 01:03:52,703 of freedom flying mouse from Logitech. 1390 01:03:52,703 --> 01:03:53,370 [VIDEO PLAYBACK] 1391 01:03:53,370 --> 01:03:55,380 --introduce a new user interface metaphor 1392 01:03:55,380 --> 01:03:58,050 for immersive VR, the virtual tricorder. 1393 01:03:58,050 --> 01:04:00,540 The virtual tricorder offers a visual and tactile interface 1394 01:04:00,540 --> 01:04:02,850 that is programmable engineer form and that 1395 01:04:02,850 --> 01:04:04,560 takes advantage of the user's familiarity 1396 01:04:04,560 --> 01:04:05,795 with real-world tools. 1397 01:04:05,795 --> 01:04:08,560 It visually duplicates the six degrees of freedom input device 1398 01:04:08,560 --> 01:04:10,363 such is the Logitech flying mouse 1399 01:04:10,363 --> 01:04:12,780 and met all translations and rotations of the input device 1400 01:04:12,780 --> 01:04:14,610 one to one to the visual model. 1401 01:04:14,610 --> 01:04:16,140 Desktop setup is for illustration 1402 01:04:16,140 --> 01:04:18,520 only since the virtual tricorder applies strictly 1403 01:04:18,520 --> 01:04:19,800 to immersive VR. 1404 01:04:19,800 --> 01:04:22,718 Effects-based boom services are a head-mounted display. 1405 01:04:22,718 --> 01:04:25,260 We demonstrate four interaction techniques for the tricorder. 1406 01:04:25,260 --> 01:04:27,090 First, anchored menus or 2D menus 1407 01:04:27,090 --> 01:04:28,950 anchored to the front of the tricorder. 1408 01:04:28,950 --> 01:04:31,410 To see the menu better, the user pulls the tricorder closer 1409 01:04:31,410 --> 01:04:32,720 to his face. 1410 01:04:32,720 --> 01:04:38,600 By pressing the labeled buttons, he chooses and selects items. 1411 01:04:38,600 --> 01:04:41,120 Second, zero-gravity grappling lets the user 1412 01:04:41,120 --> 01:04:45,560 grab arbitrary points in space and push or pull himself 1413 01:04:45,560 --> 01:04:48,120 away or towards them. 1414 01:04:48,120 --> 01:04:52,750 So this is like the hand in 2D 1415 01:04:52,750 --> 01:04:54,500 - To move more than an arm's length, 1416 01:04:54,500 --> 01:04:57,520 he uses the ratcheting motion or throws himself 1417 01:04:57,520 --> 01:04:59,380 in the design direction. 1418 01:04:59,380 --> 01:05:01,360 He then uses a built-in jet thruster 1419 01:05:01,360 --> 01:05:03,253 to correct the resulting flight path. 1420 01:05:03,253 --> 01:05:05,427 [LAUGHTER] 1421 01:05:05,427 --> 01:05:07,120 Thus, he smoothly integrated the scene 1422 01:05:07,120 --> 01:05:12,520 at hand applying [INAUDIBLE] aviation techniques. 1423 01:05:12,520 --> 01:05:15,450 Third, we demonstrate selecting and positioning objects. 1424 01:05:15,450 --> 01:05:17,580 An object selection with a semi-transparent cone 1425 01:05:17,580 --> 01:05:19,770 projects from the front of the tricorder. 1426 01:05:19,770 --> 01:05:21,990 To each selection, the user adjusts its width 1427 01:05:21,990 --> 01:05:24,600 interactively. 1428 01:05:24,600 --> 01:05:26,820 The object it highlights here, a built-in spotlight 1429 01:05:26,820 --> 01:05:28,800 at the cone's apex is selected when the user 1430 01:05:28,800 --> 01:05:31,440 presses the Select button. 1431 01:05:31,440 --> 01:05:33,120 Once an object is selected, the user 1432 01:05:33,120 --> 01:05:35,400 transmits it by pressing the right-most button 1433 01:05:35,400 --> 01:05:37,290 and transmitting the tricorder. 1434 01:05:37,290 --> 01:05:40,460 Or he rotates the object by pressing the middle button. 1435 01:05:40,460 --> 01:05:42,210 The tricorder's orientation is transmitted 1436 01:05:42,210 --> 01:05:46,070 to modify the object's orientation. 1437 01:05:46,070 --> 01:05:47,930 Fourth and finally, we use the tricorder 1438 01:05:47,930 --> 01:05:49,737 as a visualization tool. 1439 01:05:49,737 --> 01:05:51,945 A wireframe lens renders the scene in wireframe mode. 1440 01:05:51,945 --> 01:05:54,470 1441 01:05:54,470 --> 01:05:56,150 By controlling eye to lens distance, 1442 01:05:56,150 --> 01:05:58,010 the user also controls how much of the scene 1443 01:05:58,010 --> 01:06:00,215 is displayed through the lens and thus rendered 1444 01:06:00,215 --> 01:06:00,990 in wireframe mode. 1445 01:06:00,990 --> 01:06:04,180 1446 01:06:04,180 --> 01:06:06,910 The magnifying glass magnifies the scene. 1447 01:06:06,910 --> 01:06:09,145 Once more, the user controls how much of the scene 1448 01:06:09,145 --> 01:06:12,580 is magnified by varying the eye to lens distance. 1449 01:06:12,580 --> 01:06:15,470 The user can also interactively change the glass magnification 1450 01:06:15,470 --> 01:06:15,970 factor. 1451 01:06:15,970 --> 01:06:18,540 1452 01:06:18,540 --> 01:06:21,788 And he can even change the tool into a magnifying lens. 1453 01:06:21,788 --> 01:06:23,192 [END PLAYBACK] 1454 01:06:23,192 --> 01:06:27,410 1455 01:06:27,410 --> 01:06:30,530 OK, I think that's a child's garden of widgets. 1456 01:06:30,530 --> 01:06:33,500 Let me summarize the design issues before briefly moving on 1457 01:06:33,500 --> 01:06:34,610 to software. 1458 01:06:34,610 --> 01:06:39,110 So I said 3D design space is hugely larger than 2D 1459 01:06:39,110 --> 01:06:42,470 and much, much less well explored. 1460 01:06:42,470 --> 01:06:44,300 And there's a much higher learning curve 1461 01:06:44,300 --> 01:06:45,800 for getting into the game. 1462 01:06:45,800 --> 01:06:48,290 Essentially, we're faced with still creating the design 1463 01:06:48,290 --> 01:06:49,160 discipline. 1464 01:06:49,160 --> 01:06:51,380 When PARC started in the early '70S, 1465 01:06:51,380 --> 01:06:56,300 there was no design discipline for bitmap graphics-based GUIs. 1466 01:06:56,300 --> 01:06:57,500 And they invented it. 1467 01:06:57,500 --> 01:07:01,040 It was polished by Apple and then polished some more 1468 01:07:01,040 --> 01:07:01,790 by other folks. 1469 01:07:01,790 --> 01:07:04,860 That's what we need to do in 3D. 1470 01:07:04,860 --> 01:07:07,760 So now I'm going to talk very briefly about software issues. 1471 01:07:07,760 --> 01:07:10,710 I know that 5 o'clock is essentially the drop dead time. 1472 01:07:10,710 --> 01:07:13,200 And I'll work to that. 1473 01:07:13,200 --> 01:07:16,300 This is the list of things I'm going to cover ever so quickly. 1474 01:07:16,300 --> 01:07:17,320 So let me get to it. 1475 01:07:17,320 --> 01:07:20,040 First of all, the development methodology, I claim, 1476 01:07:20,040 --> 01:07:22,980 will be quite different for 3D than in 2D. 1477 01:07:22,980 --> 01:07:26,160 In 2D, we still use a kind of software engineering waterfall 1478 01:07:26,160 --> 01:07:28,380 model where we can go pretty much top down 1479 01:07:28,380 --> 01:07:30,540 through all the standard phases. 1480 01:07:30,540 --> 01:07:33,630 In 3D, we don't know enough to be able to do that yet. 1481 01:07:33,630 --> 01:07:37,885 And so it's much easier to have somebody quickly prototype 1482 01:07:37,885 --> 01:07:39,510 something and say, "Here's what I like. 1483 01:07:39,510 --> 01:07:41,820 Here's what I don't like," and play with it 1484 01:07:41,820 --> 01:07:43,703 than to go through a specification 1485 01:07:43,703 --> 01:07:45,370 and see whether you're going to like it. 1486 01:07:45,370 --> 01:07:49,530 So this says, it's essential to have a rapid prototyping 1487 01:07:49,530 --> 01:07:52,680 environment which has an interpreted or visual language 1488 01:07:52,680 --> 01:07:53,740 or both. 1489 01:07:53,740 --> 01:07:56,910 So we're going to sit in this design loop refining it. 1490 01:07:56,910 --> 01:08:00,660 And some of those rack widgets had 20 or 30 variations 1491 01:08:00,660 --> 01:08:02,430 before we were happy with them. 1492 01:08:02,430 --> 01:08:06,250 And some of them took as little as half an hour to do. 1493 01:08:06,250 --> 01:08:07,030 Why is that? 1494 01:08:07,030 --> 01:08:08,790 Because we are building the widgets 1495 01:08:08,790 --> 01:08:10,830 in exactly the same environment as which 1496 01:08:10,830 --> 01:08:12,480 we build the application objects. 1497 01:08:12,480 --> 01:08:13,800 Uses the same system. 1498 01:08:13,800 --> 01:08:15,450 We don't have this problem of having 1499 01:08:15,450 --> 01:08:18,510 the motif hierarchy over here and the application 1500 01:08:18,510 --> 01:08:20,460 hierarchy over here and trying to link them 1501 01:08:20,460 --> 01:08:22,290 through callback registration. 1502 01:08:22,290 --> 01:08:24,450 So I want to reset the clock. 1503 01:08:24,450 --> 01:08:27,359 I want to blur the distinction between what is user interface 1504 01:08:27,359 --> 01:08:29,310 and application and take advantage 1505 01:08:29,310 --> 01:08:33,149 of the semantic feedback that we get only from the application. 1506 01:08:33,149 --> 01:08:36,029 For the programmer, this means things are easier to learn 1507 01:08:36,029 --> 01:08:39,160 and easier to use because it's the same machinery. 1508 01:08:39,160 --> 01:08:41,729 And we can do faster prototyping. 1509 01:08:41,729 --> 01:08:44,149 We also can get higher bandwidth communication 1510 01:08:44,149 --> 01:08:46,649 between the parts that are user interface and the parts that 1511 01:08:46,649 --> 01:08:48,607 are application object, because, in fact, there 1512 01:08:48,607 --> 01:08:49,710 is no distinction. 1513 01:08:49,710 --> 01:08:52,830 You don't have this barrier of the mass of spaghetti 1514 01:08:52,830 --> 01:08:55,529 that the callbacks generate. 1515 01:08:55,529 --> 01:08:58,380 Now, that is not to say that user interface design is 1516 01:08:58,380 --> 01:08:59,880 the same as application design. 1517 01:08:59,880 --> 01:09:02,040 We still need that design discipline, 1518 01:09:02,040 --> 01:09:04,950 and modularity is still useful. 1519 01:09:04,950 --> 01:09:07,890 We have barely begun to understand, particularly 1520 01:09:07,890 --> 01:09:11,069 we graphics bigots, how to integrate the other media 1521 01:09:11,069 --> 01:09:13,008 into our graphics presentation. 1522 01:09:13,008 --> 01:09:14,550 Most of the time when you see things, 1523 01:09:14,550 --> 01:09:16,830 you're only seeing graphics channels. 1524 01:09:16,830 --> 01:09:19,770 But when you have bouncing balls or Peedy the talking parrot, 1525 01:09:19,770 --> 01:09:23,670 you have to worry deeply about sound synchronization, 1526 01:09:23,670 --> 01:09:26,880 beak syncing as they called it at Microsoft. 1527 01:09:26,880 --> 01:09:30,390 And that's difficult, and there are remarkably few tools 1528 01:09:30,390 --> 01:09:32,700 for doing that. 1529 01:09:32,700 --> 01:09:34,620 This brings me to a topic that we're 1530 01:09:34,620 --> 01:09:36,240 studying in my research group. 1531 01:09:36,240 --> 01:09:39,720 I have a PhD dissertation in progress 1532 01:09:39,720 --> 01:09:41,580 on this, something that we've been calling 1533 01:09:41,580 --> 01:09:43,290 time-critical computing. 1534 01:09:43,290 --> 01:09:45,540 In normal computing, you design algorithms. 1535 01:09:45,540 --> 01:09:48,600 And the outcome is you have a correct algorithm. 1536 01:09:48,600 --> 01:09:51,090 It produces the results that you want. 1537 01:09:51,090 --> 01:09:53,560 In virtual reality, in particular, 1538 01:09:53,560 --> 01:09:57,780 a result on time given according to certain time guarantees 1539 01:09:57,780 --> 01:10:01,590 is worth more than a better answer, but too late. 1540 01:10:01,590 --> 01:10:02,100 Why? 1541 01:10:02,100 --> 01:10:04,800 Well, for one thing, you're going to get motion sick. 1542 01:10:04,800 --> 01:10:08,430 If latency is too high and if you don't have adequate refresh 1543 01:10:08,430 --> 01:10:10,680 rate, the illusion is going to break. 1544 01:10:10,680 --> 01:10:14,220 Very unpleasant psychophysical effects 1545 01:10:14,220 --> 01:10:16,800 if you don't deliver in time. 1546 01:10:16,800 --> 01:10:20,610 So timeliness in this domain is more important than quality. 1547 01:10:20,610 --> 01:10:23,970 This involves research at two different levels, 1548 01:10:23,970 --> 01:10:25,470 operating system research. 1549 01:10:25,470 --> 01:10:28,320 I was talking to Professor [? Castle ?] earlier today 1550 01:10:28,320 --> 01:10:31,650 about his risk style operating system in which it should 1551 01:10:31,650 --> 01:10:35,010 be a lot easier to design a soft real time 1552 01:10:35,010 --> 01:10:39,550 library for the kinds of effects that we need here. 1553 01:10:39,550 --> 01:10:43,140 But we also need to turn algorithm research on its head 1554 01:10:43,140 --> 01:10:46,980 and say, let's have classes of what the AI folks sometimes 1555 01:10:46,980 --> 01:10:49,800 call any time or interruptible algorithms. 1556 01:10:49,800 --> 01:10:51,990 Well, you tell the algorithm how much time it has, 1557 01:10:51,990 --> 01:10:53,820 and it computes to that time. 1558 01:10:53,820 --> 01:10:56,880 Or conversely, it's always got partial results 1559 01:10:56,880 --> 01:10:58,020 that are meaningful. 1560 01:10:58,020 --> 01:11:01,260 And the more time you give it, the better the result. 1561 01:11:01,260 --> 01:11:04,470 In rendering, 3D rendering, that's 1562 01:11:04,470 --> 01:11:06,000 called progressive refinement. 1563 01:11:06,000 --> 01:11:09,450 We had another PhD dissertation for collision detection 1564 01:11:09,450 --> 01:11:12,840 where it used a three or actually 1565 01:11:12,840 --> 01:11:15,540 four-dimensional computational geometry problem 1566 01:11:15,540 --> 01:11:17,820 to put bounds around objects that 1567 01:11:17,820 --> 01:11:21,840 were going to collide, and look for potential intersections 1568 01:11:21,840 --> 01:11:24,900 and very rapidly give a approximate answer. 1569 01:11:24,900 --> 01:11:28,350 And then given more time, we'd give you a more precise answer. 1570 01:11:28,350 --> 01:11:30,450 Plenty good enough for government work, 1571 01:11:30,450 --> 01:11:33,900 and much better than taking non-real time amounts of time 1572 01:11:33,900 --> 01:11:35,590 to get the right answer. 1573 01:11:35,590 --> 01:11:39,300 So we can do these kinds of interruptible algorithms 1574 01:11:39,300 --> 01:11:42,900 only if we are informed about what makes sense 1575 01:11:42,900 --> 01:11:46,830 for a particular task in terms of the perceptual cues 1576 01:11:46,830 --> 01:11:48,840 that we need to perform the task. 1577 01:11:48,840 --> 01:11:50,940 Computer scientists don't talk about tasks. 1578 01:11:50,940 --> 01:11:52,650 They talk about algorithms. 1579 01:11:52,650 --> 01:11:54,667 In this game, you've got to talk about tasks, 1580 01:11:54,667 --> 01:11:56,250 you've got to talk about users, you've 1581 01:11:56,250 --> 01:11:58,050 got to talk about perceptual cues. 1582 01:11:58,050 --> 01:12:00,180 Are shadows important or not? 1583 01:12:00,180 --> 01:12:03,420 Is beautifully photo realistically rendered 1584 01:12:03,420 --> 01:12:04,550 essential or not? 1585 01:12:04,550 --> 01:12:07,910 Or could you make do with wireframe or faceted shading? 1586 01:12:07,910 --> 01:12:10,110 How accurate does the simulation have to be? 1587 01:12:10,110 --> 01:12:12,500 If you want to see a pool table simulation, 1588 01:12:12,500 --> 01:12:14,750 must the balls move smoothly, or must you 1589 01:12:14,750 --> 01:12:17,270 be guaranteed that at certain time, collisions happen? 1590 01:12:17,270 --> 01:12:20,750 And in between, you could stand some amount of jerkiness? 1591 01:12:20,750 --> 01:12:24,800 It's all a question of the perceptions and the tasks. 1592 01:12:24,800 --> 01:12:28,280 And we have to become sensitive to that. 1593 01:12:28,280 --> 01:12:31,190 Another major problem is that we're trying to build these 1594 01:12:31,190 --> 01:12:35,270 systems in antiquated 1970 styles programming language, 1595 01:12:35,270 --> 01:12:37,970 C or C++. 1596 01:12:37,970 --> 01:12:42,290 Devising algorithms in this new world is hard enough. 1597 01:12:42,290 --> 01:12:44,570 And we don't really have the tools 1598 01:12:44,570 --> 01:12:49,850 at our disposal to think of the new algorithms in the new ways. 1599 01:12:49,850 --> 01:12:52,160 And I claim it's kind of a Whorfian hypothesis. 1600 01:12:52,160 --> 01:12:55,580 We haven't been exposed to these better languages. 1601 01:12:55,580 --> 01:12:57,770 The actor model is about the only thing 1602 01:12:57,770 --> 01:13:03,140 that I saw many years ago that kind of broke the paradigm 1603 01:13:03,140 --> 01:13:05,840 and got us into a new way of thinking. 1604 01:13:05,840 --> 01:13:07,890 What do we need in this new world? 1605 01:13:07,890 --> 01:13:11,180 Well, first of all, again, not a solitary activity, 1606 01:13:11,180 --> 01:13:14,540 multiple participants, what programming language 1607 01:13:14,540 --> 01:13:18,680 knows for multiple participants, an active world with autonomous 1608 01:13:18,680 --> 01:13:21,650 objects whose behavior is time-varying, 1609 01:13:21,650 --> 01:13:24,230 lots of channels coming in simultaneously 1610 01:13:24,230 --> 01:13:25,640 and continuously. 1611 01:13:25,640 --> 01:13:28,040 And no programming language knows anything 1612 01:13:28,040 --> 01:13:29,900 about any of these concepts. 1613 01:13:29,900 --> 01:13:32,690 In particular, the object models are broken. 1614 01:13:32,690 --> 01:13:39,110 C++ style static class instance kind of model does not work 1615 01:13:39,110 --> 01:13:41,810 because our objects have to be able to change their behavior 1616 01:13:41,810 --> 01:13:44,490 over time, to acquire new traits. 1617 01:13:44,490 --> 01:13:49,730 So we are much more impressed with a prototype delegation 1618 01:13:49,730 --> 01:13:52,610 kind of model which allows for dynamic behavior. 1619 01:13:52,610 --> 01:13:54,290 We need multi-methods. 1620 01:13:54,290 --> 01:13:58,430 It's not reasonable to say all of the behavior of an object 1621 01:13:58,430 --> 01:14:00,480 is the property of that object. 1622 01:14:00,480 --> 01:14:02,390 It's the way objects interact that 1623 01:14:02,390 --> 01:14:06,050 determines the behavior of the individual interacting objects. 1624 01:14:06,050 --> 01:14:08,540 Again, most programming languages, even 1625 01:14:08,540 --> 01:14:10,760 object languages, don't support that. 1626 01:14:10,760 --> 01:14:15,200 And multi-methods is one way of getting the message 1627 01:14:15,200 --> 01:14:19,220 to be in a sense decoded by the multiple objects involved 1628 01:14:19,220 --> 01:14:21,950 in sending and receiving that message. 1629 01:14:21,950 --> 01:14:24,560 Distributed processing. 1630 01:14:24,560 --> 01:14:26,930 MIT is one of the hotbeds of this. 1631 01:14:26,930 --> 01:14:28,670 And they've certainly invented languages, 1632 01:14:28,670 --> 01:14:30,740 but no language in common use knows 1633 01:14:30,740 --> 01:14:32,240 anything about distribution. 1634 01:14:32,240 --> 01:14:34,310 That's left to libraries. 1635 01:14:34,310 --> 01:14:37,860 Concurrency, synchronization, and so on. 1636 01:14:37,860 --> 01:14:39,445 So we have lots of needs. 1637 01:14:39,445 --> 01:14:40,820 But if we throw them all together 1638 01:14:40,820 --> 01:14:43,160 into a single object-oriented language, 1639 01:14:43,160 --> 01:14:46,040 we'll wind up once again with something like PL/I. 1640 01:14:46,040 --> 01:14:48,350 So we can't have a kitchen sink approach. 1641 01:14:48,350 --> 01:14:53,240 And I have another PhD student working on a very simple object 1642 01:14:53,240 --> 01:14:56,990 models that can be grown to have the properties that you need, 1643 01:14:56,990 --> 01:14:59,540 which might be application-dependent. 1644 01:14:59,540 --> 01:15:04,820 The system we currently work in is implemented in C and C++. 1645 01:15:04,820 --> 01:15:08,870 It supports a LISP-ish style interpretive language. 1646 01:15:08,870 --> 01:15:11,570 And it also has an interaction tool kit 1647 01:15:11,570 --> 01:15:14,390 with which we can do some amount of visual programming 1648 01:15:14,390 --> 01:15:16,310 in a kind of dataflow-ish way. 1649 01:15:16,310 --> 01:15:18,560 And I want to finish up with that. 1650 01:15:18,560 --> 01:15:20,660 Essentially what we noticed is that there 1651 01:15:20,660 --> 01:15:24,020 are a bunch of things that get recycled over and over again-- 1652 01:15:24,020 --> 01:15:27,020 points, lines, planes, polygons, cylinders, 1653 01:15:27,020 --> 01:15:29,240 and so on, and constrained behaviors. 1654 01:15:29,240 --> 01:15:31,340 Things slide relative to each other. 1655 01:15:31,340 --> 01:15:33,870 Things rotate relative to other things. 1656 01:15:33,870 --> 01:15:36,660 So affine transformations come up a lot. 1657 01:15:36,660 --> 01:15:40,550 So we said, let's encapsulate those simple behaviors 1658 01:15:40,550 --> 01:15:44,610 and the geometry and try to program this stuff visually. 1659 01:15:44,610 --> 01:15:47,090 So I'm going to show you how we set up 1660 01:15:47,090 --> 01:15:52,010 one-way dependencies into multi-way constraint networks. 1661 01:15:52,010 --> 01:15:56,090 And then our system continuously does constraint maintenance 1662 01:15:56,090 --> 01:15:56,930 in some sense. 1663 01:15:56,930 --> 01:15:59,870 And you will see that it isn't all that peppy. 1664 01:15:59,870 --> 01:16:02,300 The system isn't good enough, and I 1665 01:16:02,300 --> 01:16:06,560 would say it was a decent first effort and no more. 1666 01:16:06,560 --> 01:16:08,750 So let me show that to you now. 1667 01:16:08,750 --> 01:16:11,670 1668 01:16:11,670 --> 01:16:15,240 Actually, I need to switch tapes. 1669 01:16:15,240 --> 01:16:16,395 Let's see. 1670 01:16:16,395 --> 01:16:19,335 I think I have to use the thing to eject. 1671 01:16:19,335 --> 01:16:23,800 1672 01:16:23,800 --> 01:16:25,630 Now I don't know where I am on this tape 1673 01:16:25,630 --> 01:16:27,370 because this is the one that was rewound. 1674 01:16:27,370 --> 01:16:30,400 So we're going to go through the same nonsense, I'm afraid, 1675 01:16:30,400 --> 01:16:31,308 that we did earlier. 1676 01:16:31,308 --> 01:16:31,975 [VIDEO PLAYBACK] 1677 01:16:31,975 --> 01:16:33,610 - [INAUDIBLE] widgets and interactive illustrations. 1678 01:16:33,610 --> 01:16:34,347 Oh, perfect. 1679 01:16:34,347 --> 01:16:36,430 - [INAUDIBLE] provides [INAUDIBLE] with the direct 1680 01:16:36,430 --> 01:16:37,510 manipulation of 3D primitives-- 1681 01:16:37,510 --> 01:16:38,260 This is the first [INAUDIBLE]-- 1682 01:16:38,260 --> 01:16:39,580 --which can be linked through a visual interface 1683 01:16:39,580 --> 01:16:42,100 to create objects with complex constrained behavior. 1684 01:16:42,100 --> 01:16:45,975 All demonstrations in this video were recorded in real time. 1685 01:16:45,975 --> 01:16:47,350 The basic tool kit primitives are 1686 01:16:47,350 --> 01:16:49,990 based on a coordinate system metaphor of points, vectors, 1687 01:16:49,990 --> 01:16:51,707 planes, and 3D volumes. 1688 01:16:51,707 --> 01:16:54,040 Each primitive has a set of characteristic traits called 1689 01:16:54,040 --> 01:16:55,480 slots that define the primitive's 1690 01:16:55,480 --> 01:16:58,660 interactive behavior. 1691 01:16:58,660 --> 01:17:01,030 The point primitive represented by the small sphere 1692 01:17:01,030 --> 01:17:02,830 has one slot, its position slot. 1693 01:17:02,830 --> 01:17:05,490 1694 01:17:05,490 --> 01:17:07,680 The vector primitive represented by the arrow 1695 01:17:07,680 --> 01:17:10,050 has three slots for its position, direction, 1696 01:17:10,050 --> 01:17:11,730 and length. 1697 01:17:11,730 --> 01:17:14,152 The plane primitive represented by flat sheet 1698 01:17:14,152 --> 01:17:18,420 has slots for its normal orientation, center, and scale. 1699 01:17:18,420 --> 01:17:20,220 3D volume primitives, like this cube, 1700 01:17:20,220 --> 01:17:21,990 have slots similar to the plane primitive, 1701 01:17:21,990 --> 01:17:24,570 but also contains slots for scaling in the third dimension 1702 01:17:24,570 --> 01:17:27,450 and for non-geometric attributes such as color, transparency, 1703 01:17:27,450 --> 01:17:29,520 and visibility. 1704 01:17:29,520 --> 01:17:32,130 To build up objects with complex interactive behavior, 1705 01:17:32,130 --> 01:17:33,990 we specified persistent relationships 1706 01:17:33,990 --> 01:17:36,210 between a slot of one primitive and one or more slots 1707 01:17:36,210 --> 01:17:37,290 of another primitive. 1708 01:17:37,290 --> 01:17:39,420 We call this process linking. 1709 01:17:39,420 --> 01:17:41,370 The constraints imposed by the linking process 1710 01:17:41,370 --> 01:17:44,370 are established and maintained by the toolkit. 1711 01:17:44,370 --> 01:17:46,590 For example, to link the vector's position slot 1712 01:17:46,590 --> 01:17:48,450 to a point, we first select the vector 1713 01:17:48,450 --> 01:17:49,890 and then select the point. 1714 01:17:49,890 --> 01:17:52,515 The pending length is then illustrated by the large arrow. 1715 01:17:52,515 --> 01:17:54,840 The motif window shows us the available slots 1716 01:17:54,840 --> 01:17:57,450 of the vector and the possible linking behaviors. 1717 01:17:57,450 --> 01:17:59,220 The tool kit selects a slot and a behavior 1718 01:17:59,220 --> 01:18:01,800 by default. In this case, the default slot 1719 01:18:01,800 --> 01:18:02,730 is the position slot. 1720 01:18:02,730 --> 01:18:04,560 And the default behavior positions the base 1721 01:18:04,560 --> 01:18:06,240 of the vector at the point. 1722 01:18:06,240 --> 01:18:09,060 We confirm the pending link by clicking on the glue bottle. 1723 01:18:09,060 --> 01:18:11,602 The constraint is established, and the vector's position slot 1724 01:18:11,602 --> 01:18:12,810 is linked to the point. 1725 01:18:12,810 --> 01:18:14,100 This link is bidirectional. 1726 01:18:14,100 --> 01:18:16,225 We can translate the point, and the vector follows, 1727 01:18:16,225 --> 01:18:18,270 or we translate the vector, and the point moves. 1728 01:18:18,270 --> 01:18:19,770 The other slots of the vector remain 1729 01:18:19,770 --> 01:18:22,710 free to be manipulated or constrained. 1730 01:18:22,710 --> 01:18:24,090 We can continue this construction 1731 01:18:24,090 --> 01:18:25,840 to build up the behavior of a line segment 1732 01:18:25,840 --> 01:18:27,900 by first linking the vector's directional slot 1733 01:18:27,900 --> 01:18:29,340 to a second point. 1734 01:18:29,340 --> 01:18:31,375 Since the position slot is already constrained, 1735 01:18:31,375 --> 01:18:33,000 the tool kit selects the direction slot 1736 01:18:33,000 --> 01:18:37,900 by default. Notice the vector always points from the first 1737 01:18:37,900 --> 01:18:38,790 to the second point. 1738 01:18:38,790 --> 01:18:41,550 1739 01:18:41,550 --> 01:18:43,620 Next, we link the vector's length slot 1740 01:18:43,620 --> 01:18:45,250 to the same second point. 1741 01:18:45,250 --> 01:18:49,950 Now the vector resizes in order to span between the two points. 1742 01:18:49,950 --> 01:18:51,960 To constrain a point to lie along this line, 1743 01:18:51,960 --> 01:18:54,450 we link a point's position slot to the vector. 1744 01:18:54,450 --> 01:18:56,700 Notice there are several options presented in the menu 1745 01:18:56,700 --> 01:18:58,627 for the linking operation. 1746 01:18:58,627 --> 01:19:00,960 The default linking behavior constrains the point simply 1747 01:19:00,960 --> 01:19:03,120 to lie along the line. 1748 01:19:03,120 --> 01:19:05,160 If we decide we're unhappy with this behavior, 1749 01:19:05,160 --> 01:19:07,002 we can undo it and apply a new constraint. 1750 01:19:07,002 --> 01:19:09,210 By clicking on the point while holding the Shift key, 1751 01:19:09,210 --> 01:19:10,500 we remove the [INAUDIBLE]-- 1752 01:19:10,500 --> 01:19:14,070 I'll just fast forward and go to an application. 1753 01:19:14,070 --> 01:19:16,810 1754 01:19:16,810 --> 01:19:22,561 This shows encapsulation of the-- 1755 01:19:22,561 --> 01:19:24,510 - We are modeling the portion of a bicycle 1756 01:19:24,510 --> 01:19:26,490 where the head tube is connected to the rest of the bicycle 1757 01:19:26,490 --> 01:19:27,053 frame. 1758 01:19:27,053 --> 01:19:29,220 We want to make the joints between these two smooth. 1759 01:19:29,220 --> 01:19:29,400 [INAUDIBLE]-- 1760 01:19:29,400 --> 01:19:29,935 This is an application. 1761 01:19:29,935 --> 01:19:31,560 -- to investigate this particular problem came from 1762 01:19:31,560 --> 01:19:32,700 a bicycle designer. 1763 01:19:32,700 --> 01:19:35,283 We want to design a blend region that joins the tubes smoothly 1764 01:19:35,283 --> 01:19:36,100 and attractively. 1765 01:19:36,100 --> 01:19:37,920 The specification of the blend range in between two 1766 01:19:37,920 --> 01:19:39,600 of the tubes is somewhat complex. 1767 01:19:39,600 --> 01:19:41,017 The boundaries of the blend region 1768 01:19:41,017 --> 01:19:43,680 must be specified as curves on the tube services to be joined. 1769 01:19:43,680 --> 01:19:45,555 We see two such curves, known as rail curves, 1770 01:19:45,555 --> 01:19:46,682 shown here in white. 1771 01:19:46,682 --> 01:19:48,390 We examine the two blend regions together 1772 01:19:48,390 --> 01:19:50,460 and see that the result does not have a nice shape. 1773 01:19:50,460 --> 01:19:51,750 Instead of the flat area on the head 1774 01:19:51,750 --> 01:19:54,120 tube between these regions, we would like a smooth shape 1775 01:19:54,120 --> 01:19:55,687 that unifies both blends. 1776 01:19:55,687 --> 01:19:57,645 To accomplish this, we will alter the head tube 1777 01:19:57,645 --> 01:19:58,650 by warping it. 1778 01:19:58,650 --> 01:20:01,050 This will create a smooth saddle between the blends. 1779 01:20:01,050 --> 01:20:03,300 We use a previously constructed encapsulated warp tool 1780 01:20:03,300 --> 01:20:05,560 to accomplish this operation. 1781 01:20:05,560 --> 01:20:07,820 The warp tool consists of a center of warp, direction 1782 01:20:07,820 --> 01:20:10,710 and magnitude of warp, and a sphere of influence. 1783 01:20:10,710 --> 01:20:12,990 Because of the required symmetry of the construction, 1784 01:20:12,990 --> 01:20:15,232 we need to be able to constrain the warp interaction. 1785 01:20:15,232 --> 01:20:17,190 To do this, we constrain the center of the warp 1786 01:20:17,190 --> 01:20:19,000 to a previously constructed access line 1787 01:20:19,000 --> 01:20:20,500 through the center of the head tube. 1788 01:20:20,500 --> 01:20:23,010 1789 01:20:23,010 --> 01:20:25,010 Then we constrain the direction of the warp tool 1790 01:20:25,010 --> 01:20:27,050 to lie in a plane that contains the center lines of each 1791 01:20:27,050 --> 01:20:28,010 of the bicycle tubes. 1792 01:20:28,010 --> 01:20:33,297 1793 01:20:33,297 --> 01:20:34,880 Once the warp tool is in approximately 1794 01:20:34,880 --> 01:20:37,625 the right position, we use the toolkit to link the head tube 1795 01:20:37,625 --> 01:20:38,510 to the warp tool. 1796 01:20:38,510 --> 01:20:45,555 1797 01:20:45,555 --> 01:20:47,180 We can interactively warp the head tube 1798 01:20:47,180 --> 01:20:49,687 by adjusting the direction, center, and radius of the warp 1799 01:20:49,687 --> 01:20:51,770 with the parameters encapsulated in the warp tool. 1800 01:20:51,770 --> 01:20:57,300 1801 01:20:57,300 --> 01:20:58,965 Once the desired warp is achieved, 1802 01:20:58,965 --> 01:21:01,590 we freeze the warp by unlinking the surface from the warp tool. 1803 01:21:01,590 --> 01:21:03,418 [END PLAYBACK] 1804 01:21:03,418 --> 01:21:05,710 1805 01:21:05,710 --> 01:21:07,900 So that was an exercise that used 1806 01:21:07,900 --> 01:21:12,610 Utah Alpha 1 geometric modeler and our widgets. 1807 01:21:12,610 --> 01:21:14,980 And that was all real time that you saw. 1808 01:21:14,980 --> 01:21:17,320 It's still a toy exercise. 1809 01:21:17,320 --> 01:21:20,740 It's not ready for use by a production designer. 1810 01:21:20,740 --> 01:21:23,410 But it was the first time that the Utah folks 1811 01:21:23,410 --> 01:21:26,908 had more than an Emacs interface to their parameters. 1812 01:21:26,908 --> 01:21:28,450 It was the first time that they could 1813 01:21:28,450 --> 01:21:31,360 see the effect of various kinds of parameters 1814 01:21:31,360 --> 01:21:34,300 on the actual surface that they were warping. 1815 01:21:34,300 --> 01:21:37,300 So let me wind up here now. 1816 01:21:37,300 --> 01:21:38,425 My last slide. 1817 01:21:38,425 --> 01:21:44,220 1818 01:21:44,220 --> 01:21:45,510 3D is happening. 1819 01:21:45,510 --> 01:21:48,360 3D is going to be here sooner than we think. 1820 01:21:48,360 --> 01:21:52,260 It's going to drown what we've seen in the last 20 years 1821 01:21:52,260 --> 01:21:56,400 in terms of the slow, steady rise of the WIMP interface 1822 01:21:56,400 --> 01:21:59,220 because it's going to be driven not by research labs the way 1823 01:21:59,220 --> 01:22:01,560 the first revolution in user interface design 1824 01:22:01,560 --> 01:22:04,980 was, but by commodity folks such as the game 1825 01:22:04,980 --> 01:22:07,830 people, the interactive television people. 1826 01:22:07,830 --> 01:22:12,430 And we research lab folks still have plenty of work to do. 1827 01:22:12,430 --> 01:22:15,960 But we're going to be trying hard to follow the wave. 1828 01:22:15,960 --> 01:22:17,910 We won't be leading it. 1829 01:22:17,910 --> 01:22:22,350 The current technology is wholly insufficient unto the task. 1830 01:22:22,350 --> 01:22:23,760 It's not fast enough. 1831 01:22:23,760 --> 01:22:25,510 We don't have control over it. 1832 01:22:25,510 --> 01:22:26,940 It isn't time-critical. 1833 01:22:26,940 --> 01:22:28,710 The hardware doesn't work. 1834 01:22:28,710 --> 01:22:31,950 There are lots of things in a university environment 1835 01:22:31,950 --> 01:22:35,490 for people to pay attention to, not to mention commoditizing it 1836 01:22:35,490 --> 01:22:37,770 after we find out how to do it. 1837 01:22:37,770 --> 01:22:42,100 3D is still very much a solution looking for a problem, 1838 01:22:42,100 --> 01:22:43,980 except in the games area. 1839 01:22:43,980 --> 01:22:46,350 We have this implicit belief that it's 1840 01:22:46,350 --> 01:22:47,730 going to be wonderful. 1841 01:22:47,730 --> 01:22:50,370 Some of us, who have been in the 3D interactive graphics 1842 01:22:50,370 --> 01:22:53,130 game all of our lives, have been saying, it's coming. 1843 01:22:53,130 --> 01:22:55,170 It's coming any time now. 1844 01:22:55,170 --> 01:22:57,690 And '95 is the time when it's really coming. 1845 01:22:57,690 --> 01:23:02,160 But what we can't answer, except for intrinsically 3D problems, 1846 01:23:02,160 --> 01:23:03,240 what is it good for? 1847 01:23:03,240 --> 01:23:05,550 Is it cost effective and so on? 1848 01:23:05,550 --> 01:23:08,280 But it's going to happen because it looks good, 1849 01:23:08,280 --> 01:23:10,020 it makes people feel good in the same way 1850 01:23:10,020 --> 01:23:11,940 that you couldn't justify color. 1851 01:23:11,940 --> 01:23:14,460 And all the people trying to predict how fast color would 1852 01:23:14,460 --> 01:23:16,230 come up were wrong. 1853 01:23:16,230 --> 01:23:19,020 Black and white is sufficient unto the purpose, 1854 01:23:19,020 --> 01:23:21,000 but people want color in the same way 1855 01:23:21,000 --> 01:23:23,760 people will want 3D because of the glitz factor, 1856 01:23:23,760 --> 01:23:24,570 if nothing else. 1857 01:23:24,570 --> 01:23:28,170 And it's only because we will design the right kinds of 3D 1858 01:23:28,170 --> 01:23:30,630 user interfaces that they will actually say, 1859 01:23:30,630 --> 01:23:33,240 and it's useful for solving problems. 1860 01:23:33,240 --> 01:23:36,780 It's terra incognito, and I invite 1861 01:23:36,780 --> 01:23:40,080 you to join us on this quest. 1862 01:23:40,080 --> 01:23:41,572 Thank you. 1863 01:23:41,572 --> 01:23:50,901 [APPLAUSE] 1864 01:23:50,901 --> 01:23:55,330 1865 01:23:55,330 --> 01:23:58,060 As John remembers, I'm unfortunately 1866 01:23:58,060 --> 01:24:00,070 capable of staying forever. 1867 01:24:00,070 --> 01:24:04,030 So you leave, because I'm not going to be leaving. 1868 01:24:04,030 --> 01:24:07,130 I'm happy to stick around and answer questions. 1869 01:24:07,130 --> 01:24:07,630 Yeah. 1870 01:24:07,630 --> 01:24:13,119 1871 01:24:13,119 --> 01:24:16,612 On your conclusion slide, you said that the hardware 1872 01:24:16,612 --> 01:24:17,610 isn't fast enough. 1873 01:24:17,610 --> 01:24:20,110 What's your prognosis [INAUDIBLE]?? 1874 01:24:20,110 --> 01:24:21,880 Let's wait for people to clear. 1875 01:24:21,880 --> 01:24:24,620 1876 01:24:24,620 --> 01:24:25,120 Mike. 1877 01:24:25,120 --> 01:24:26,750 I want to fix the screen. 1878 01:24:26,750 --> 01:24:29,350 Yeah. 1879 01:24:29,350 --> 01:24:30,720 Good animation, right? 1880 01:24:30,720 --> 01:24:32,280 No. 1881 01:24:32,280 --> 01:24:33,810 It works in my PowerBook. 1882 01:24:33,810 --> 01:24:35,790 It's the connection to the video. 1883 01:24:35,790 --> 01:24:47,909 1884 01:24:47,909 --> 01:24:49,492 What's your schedule? 1885 01:24:49,492 --> 01:24:51,470 I'm going to dinner with Michael and stuff. 1886 01:24:51,470 --> 01:24:51,800 Finally I made it here. 1887 01:24:51,800 --> 01:24:52,400 Come along. 1888 01:24:52,400 --> 01:24:55,880 [INTERPOSING VOICES] 1889 01:24:55,880 --> 01:24:58,220 It was so sunny outside that, when I came in, 1890 01:24:58,220 --> 01:24:59,480 I was totally blind. 1891 01:24:59,480 --> 01:25:01,200 I couldn't [INAUDIBLE]. 1892 01:25:01,200 --> 01:25:02,595 I was stumbling upstairs. 1893 01:25:02,595 --> 01:25:16,100 1894 01:25:16,100 --> 01:25:19,460 OK, [? Miguel, ?] you get the first question. 1895 01:25:19,460 --> 01:25:20,012 OK. 1896 01:25:20,012 --> 01:25:22,220 On your last slide, you mentioned that the hardware's 1897 01:25:22,220 --> 01:25:23,155 not fast enough. 1898 01:25:23,155 --> 01:25:25,772 So my question is, what's your prognosis on that? 1899 01:25:25,772 --> 01:25:27,230 Is it ever going to be fast enough? 1900 01:25:27,230 --> 01:25:28,970 OK, I'll repeat the question. 1901 01:25:28,970 --> 01:25:30,750 Will the hardware ever be good enough? 1902 01:25:30,750 --> 01:25:34,350 Well, the standard academic's answer is of course not. 1903 01:25:34,350 --> 01:25:36,890 But you get certain threshold at which 1904 01:25:36,890 --> 01:25:41,220 a certain amount of adequacy is clearly recognized. 1905 01:25:41,220 --> 01:25:43,430 For example, I think for ordinary 2D 1906 01:25:43,430 --> 01:25:47,090 desktop applications, hardware to a first approximation 1907 01:25:47,090 --> 01:25:48,110 is good enough. 1908 01:25:48,110 --> 01:25:49,790 The mouse works well enough, the screen 1909 01:25:49,790 --> 01:25:52,550 is good enough, to CPU power, the amount of memory. 1910 01:25:52,550 --> 01:25:54,980 You always want more, but you can get useful work 1911 01:25:54,980 --> 01:25:56,420 done with what you have now. 1912 01:25:56,420 --> 01:26:01,190 That is not yet the case, except for the half a million 1913 01:26:01,190 --> 01:26:04,910 to a million dollar acquisition at the high end of 3D graphics, 1914 01:26:04,910 --> 01:26:06,650 let alone for virtual reality. 1915 01:26:06,650 --> 01:26:09,440 And for virtual reality, you need all of that processing 1916 01:26:09,440 --> 01:26:11,810 power, you need all of the rendering power, 1917 01:26:11,810 --> 01:26:15,320 and you need far better device technology than we have today. 1918 01:26:15,320 --> 01:26:18,140 There are people working on it, and things will get better. 1919 01:26:18,140 --> 01:26:21,680 But don't expect any miracles in a year or so. 1920 01:26:21,680 --> 01:26:24,950 Henry Fuchs, my colleague at University of North Carolina, 1921 01:26:24,950 --> 01:26:30,500 dreams of flip down sunglasses, that light weight 1922 01:26:30,500 --> 01:26:34,910 and with high resolution, with which you then see either 1923 01:26:34,910 --> 01:26:37,010 VR or augmented reality. 1924 01:26:37,010 --> 01:26:41,300 And he has his hands on some very good LCDs 1925 01:26:41,300 --> 01:26:43,550 that they use in view finder cameras. 1926 01:26:43,550 --> 01:26:45,000 They're very small. 1927 01:26:45,000 --> 01:26:49,250 They can be viewfinders of video cameras. 1928 01:26:49,250 --> 01:26:50,060 They're very small. 1929 01:26:50,060 --> 01:26:53,900 And he believes that with that form factor, 1930 01:26:53,900 --> 01:26:56,180 within a few years, we'll have these kinds 1931 01:26:56,180 --> 01:26:57,920 of flip down sunglasses. 1932 01:26:57,920 --> 01:26:59,240 And that will help a lot. 1933 01:26:59,240 --> 01:27:00,980 Doesn't help with haptic feedback, which 1934 01:27:00,980 --> 01:27:05,750 is actively being worked on both at UNC and in this University. 1935 01:27:05,750 --> 01:27:09,410 Sound recognition is still a bad problem, 1936 01:27:09,410 --> 01:27:11,510 although again this place is a world 1937 01:27:11,510 --> 01:27:14,780 center in that particular research area. 1938 01:27:14,780 --> 01:27:18,440 In time, that will come as will spatially-located sound. 1939 01:27:18,440 --> 01:27:20,240 We have early examples. 1940 01:27:20,240 --> 01:27:24,930 And for small, selected demo applications, it's good enough. 1941 01:27:24,930 --> 01:27:26,540 So I think by the turn of the century, 1942 01:27:26,540 --> 01:27:30,120 things will look a lot different. 1943 01:27:30,120 --> 01:27:32,940 It will move a lot faster than 2D graphics moved. 1944 01:27:32,940 --> 01:27:35,160 That took a long time to take hold. 1945 01:27:35,160 --> 01:27:39,360 It wasn't until Macintosh, which was more than 10 years 1946 01:27:39,360 --> 01:27:42,930 after Xerox demonstrated the concept amply and had 1947 01:27:42,930 --> 01:27:46,047 commercial products in the marketplace. 1948 01:27:46,047 --> 01:27:47,755 Paul, do you want to tell us why it's not 1949 01:27:47,755 --> 01:27:50,200 the fault of the LSI designers? 1950 01:27:50,200 --> 01:27:53,380 Oh, you'll get all the hardware support you need. 1951 01:27:53,380 --> 01:27:55,700 Just wait a couple years. 1952 01:27:55,700 --> 01:27:59,680 My question is you've drawn a distinction between 2D 1953 01:27:59,680 --> 01:28:00,700 interfaces-- 1954 01:28:00,700 --> 01:28:02,710 and you call them 3D, but I call them 1955 01:28:02,710 --> 01:28:05,440 greater than or equal to 3D. 1956 01:28:05,440 --> 01:28:07,480 And in particular, some of these involve 1957 01:28:07,480 --> 01:28:09,700 time in an essential way, and some did not. 1958 01:28:09,700 --> 01:28:11,410 Is there a useful distinction that 1959 01:28:11,410 --> 01:28:16,120 can be made between truly 3D and 4D? 1960 01:28:16,120 --> 01:28:18,730 We've often called our system a 4D modeler. 1961 01:28:18,730 --> 01:28:20,560 Did everybody hear the question? 1962 01:28:20,560 --> 01:28:23,920 3D plus time, because in our descriptions, 1963 01:28:23,920 --> 01:28:26,720 time is a first-class citizen. 1964 01:28:26,720 --> 01:28:29,740 So any attribute can be time-varying. 1965 01:28:29,740 --> 01:28:31,270 That includes geometry. 1966 01:28:31,270 --> 01:28:32,650 That includes behaviors. 1967 01:28:32,650 --> 01:28:35,900 That includes constrained dependencies, any of these. 1968 01:28:35,900 --> 01:28:39,040 So I would say our system is a 4D system. 1969 01:28:39,040 --> 01:28:43,660 Now, even if you only have a 3D description, if you go in there 1970 01:28:43,660 --> 01:28:45,760 and you do the editing by hand, you 1971 01:28:45,760 --> 01:28:47,840 can get it to change over time. 1972 01:28:47,840 --> 01:28:50,530 And in any 3D system, you at least 1973 01:28:50,530 --> 01:28:53,710 have the time dependency that you 1974 01:28:53,710 --> 01:28:55,720 get from navigating through the scene 1975 01:28:55,720 --> 01:28:59,020 in a three-dimensional way and shifting your point of view. 1976 01:28:59,020 --> 01:29:02,360 The camera is always traveling over time. 1977 01:29:02,360 --> 01:29:03,670 So-- 1978 01:29:03,670 --> 01:29:05,650 I think you've weaseled out of the question. 1979 01:29:05,650 --> 01:29:06,240 Possibly. 1980 01:29:06,240 --> 01:29:06,907 I'm an academic. 1981 01:29:06,907 --> 01:29:10,390 And with all [INAUDIBLE],, a 2D interface, which 1982 01:29:10,390 --> 01:29:12,610 is able to change under your direction, 1983 01:29:12,610 --> 01:29:16,460 and you mouse over it, you would almost call a 3D one. 1984 01:29:16,460 --> 01:29:19,180 And fine. 1985 01:29:19,180 --> 01:29:22,160 If yours really supports time as a first-class object, 1986 01:29:22,160 --> 01:29:23,960 isn't there an intermediate stage, 1987 01:29:23,960 --> 01:29:26,980 which doesn't, which is actually three-dimensional, 1988 01:29:26,980 --> 01:29:29,710 but which is therefore much simpler 1989 01:29:29,710 --> 01:29:34,270 and might be further in, further closer to realization 1990 01:29:34,270 --> 01:29:35,720 and might be useful? 1991 01:29:35,720 --> 01:29:37,390 Yeah, sure. 1992 01:29:37,390 --> 01:29:40,930 If you take a standard 3D graphics library, 1993 01:29:40,930 --> 01:29:43,970 and it does not have support for time, 1994 01:29:43,970 --> 01:29:46,240 but it does support viewing controls 1995 01:29:46,240 --> 01:29:49,510 and does support your editing of that description 1996 01:29:49,510 --> 01:29:53,950 so that you can by hand animate things, that's a way station. 1997 01:29:53,950 --> 01:29:58,490 That's 3D plus by hand support for time variance. 1998 01:29:58,490 --> 01:30:01,180 And that's what we've used all these years 1999 01:30:01,180 --> 01:30:02,950 to do our 3D graphics. 2000 01:30:02,950 --> 01:30:05,830 So it's a well-known way of doing things. 2001 01:30:05,830 --> 01:30:10,000 And you don't see 2D time-varying interfaces. 2002 01:30:10,000 --> 01:30:13,270 About the only thing you do is, when a window expands, 2003 01:30:13,270 --> 01:30:14,890 you see a little, tiny animation. 2004 01:30:14,890 --> 01:30:16,570 And when it contracts, it does. 2005 01:30:16,570 --> 01:30:20,560 Very little use has been made of animation for 2D interfaces. 2006 01:30:20,560 --> 01:30:23,230 People haven't been particularly creative there. 2007 01:30:23,230 --> 01:30:25,600 But for 3D, since you're constantly 2008 01:30:25,600 --> 01:30:27,850 moving your point of view, it's more natural. 2009 01:30:27,850 --> 01:30:30,130 And you tend to have more autonomous objects 2010 01:30:30,130 --> 01:30:31,810 in these kinds of applications. 2011 01:30:31,810 --> 01:30:35,230 Your 2D desktop applications, the entities 2012 01:30:35,230 --> 01:30:38,590 basically sit there on the screen and are static. 2013 01:30:38,590 --> 01:30:42,250 In 3D, much less so. 2014 01:30:42,250 --> 01:30:43,865 All right, take one more question. 2015 01:30:43,865 --> 01:30:46,115 Do I have someone who wants to ask the final question? 2016 01:30:46,115 --> 01:30:48,545 2017 01:30:48,545 --> 01:30:49,940 [INAUDIBLE] 2018 01:30:49,940 --> 01:30:55,040 Last week, Silicon Graphics had the demonstration of WebFORCE. 2019 01:30:55,040 --> 01:31:00,480 And they mentioned quickly that they either have available 2020 01:31:00,480 --> 01:31:02,480 or will put out shortly, in a week 2021 01:31:02,480 --> 01:31:04,295 or two, 3D on the internet. 2022 01:31:04,295 --> 01:31:06,427 And I'm not sure what that they meant. 2023 01:31:06,427 --> 01:31:07,260 Do you have an idea? 2024 01:31:07,260 --> 01:31:11,000 VRML, I'm quite sure. 2025 01:31:11,000 --> 01:31:14,000 So essentially being able to send inventor files 2026 01:31:14,000 --> 01:31:14,810 over the internet. 2027 01:31:14,810 --> 01:31:17,760 They also accept other kinds of file formats. 2028 01:31:17,760 --> 01:31:20,930 And that's a perfectly reasonable way station. 2029 01:31:20,930 --> 01:31:22,610 Nobody argues that it's the best, 2030 01:31:22,610 --> 01:31:24,990 but it's a de facto standard. 2031 01:31:24,990 --> 01:31:25,745 Many-- 2032 01:31:25,745 --> 01:31:29,390 Do you see it having a good commercial application? 2033 01:31:29,390 --> 01:31:32,450 Yeah, I think people will be building applications 2034 01:31:32,450 --> 01:31:36,620 on top of VRML because it combines various features 2035 01:31:36,620 --> 01:31:38,810 that they're used to in the 2D world 2036 01:31:38,810 --> 01:31:43,040 with 3D geometry description. 2037 01:31:43,040 --> 01:31:45,920 So, yeah, I think we'll see some use of that. 2038 01:31:45,920 --> 01:31:50,840 The fact is that the WWW community now 2039 01:31:50,840 --> 01:31:53,960 has the power to set standards. 2040 01:31:53,960 --> 01:31:57,650 When they say, we subscribe to XYZ-- 2041 01:31:57,650 --> 01:32:00,660 I won't say X because that's already overloaded-- 2042 01:32:00,660 --> 01:32:05,060 when they say, XYZ, people stop questioning 2043 01:32:05,060 --> 01:32:06,410 whether it's any good or not. 2044 01:32:06,410 --> 01:32:07,160 They don't argue. 2045 01:32:07,160 --> 01:32:11,010 They say, fine, we'll write to that de facto standard. 2046 01:32:11,010 --> 01:32:15,430 So that's why I think VRML is going to take off. 2047 01:32:15,430 --> 01:32:18,820 It has some amount of papal blessing bestowed on it. 2048 01:32:18,820 --> 01:32:21,707 [APPLAUSE] 2049 01:32:21,707 --> 01:32:22,207