1
00:00:13,906 --> 00:00:14,859
OK
2
00:00:16,817 --> 00:00:20,744
Camera is rolling
This is lecture 14
3
00:00:21,340 --> 00:00:26,423
Starting a new chapter:
chapter about orthogonality
4
00:00:26,458 --> 00:00:30,953
What it means for
vectors to be orthogonal
5
00:00:31,469 --> 00:00:35,342
What it means for subspaces
to be orthogonal
6
00:00:35,377 --> 00:00:38,289
What it means for basis
to be orthogonal
7
00:00:39,520 --> 00:00:44,892
So this is 90-degree chapter
8
00:00:46,553 --> 00:00:47,886
So what does that mean?
9
00:00:48,845 --> 00:00:52,583
Let me jump to subspaces
10
00:00:53,097 --> 00:00:56,845
because I've drawn here
the big picture
11
00:00:56,880 --> 00:01:00,974
This is the 1806 picture here
12
00:01:01,976 --> 00:01:05,640
And...Hold it down, guys!
13
00:01:06,938 --> 00:01:11,952
So this is the picture and we know
a lot about that picture already
14
00:01:12,650 --> 00:01:16,139
We know the dimension of every subspace
15
00:01:16,458 --> 00:01:23,883
We know that these dimensions
are r and n-r
16
00:01:24,673 --> 00:01:29,330
We know these dimensions
are r and m-r
17
00:01:30,199 --> 00:01:38,950
What I want to show now is what this
figure is saying that the angle...
18
00:01:38,985 --> 00:01:42,173
the picture is like just...
19
00:01:42,174 --> 00:01:44,143
My...attempt to draw...
20
00:01:44,178 --> 00:01:46,441
what I'm going to say that
21
00:01:47,299 --> 00:01:51,722
the angle between these
subspaces is 90 degrees
22
00:01:52,454 --> 00:01:56,144
And the angle between
these subspaces is 90 degrees
23
00:01:56,179 --> 00:01:57,702
I've to say what that means
24
00:01:57,737 --> 00:02:01,853
What does that mean for
a subspace to be orthogonal?
25
00:02:03,125 --> 00:02:08,585
But I hope you appreciate
the beauty of this picture
26
00:02:09,204 --> 00:02:15,761
...that those subspaces are
going to come out to be orthogonal
27
00:02:16,207 --> 00:02:19,746
...those two and also those two
28
00:02:20,467 --> 00:02:23,905
So that's like one point...
29
00:02:25,842 --> 00:02:30,831
one important point to step forward
in understanding those subspaces
30
00:02:31,232 --> 00:02:33,405
We knew what each subspace was like
31
00:02:33,440 --> 00:02:35,570
We could compute basis for them
32
00:02:35,605 --> 00:02:39,463
Now we know more
or we will in a few minutes
33
00:02:40,300 --> 00:02:42,031
OK, I have to say
34
00:02:42,451 --> 00:02:46,718
first of all what it means for
two vectors to be orthogonal
35
00:02:47,136 --> 00:02:48,925
So let me start with that:
36
00:02:51,874 --> 00:02:53,454
orthogonal vectors
37
00:02:59,438 --> 00:03:04,192
The word orthogonal is just
another word for perpendicular
38
00:03:04,735 --> 00:03:07,594
It means that in n-dimensional space
39
00:03:07,629 --> 00:03:11,673
the angle between those vectors
is 90 degrees
40
00:03:12,286 --> 00:03:15,240
It means that they form
a right triangle
41
00:03:15,594 --> 00:03:19,026
It even means that...
42
00:03:19,061 --> 00:03:25,857
going way back to the Greek
that this triangle:
43
00:03:25,892 --> 00:03:35,424
a vector x, a vector y
and a vector x+y
44
00:03:35,459 --> 00:03:39,011
...of course that would
be the hypotenuse
45
00:03:39,677 --> 00:03:47,682
So what was that the Greek figured out
and its need is the fact that...the
46
00:03:48,443 --> 00:03:51,945
So, these are orthogonal
this is right angle
47
00:03:52,394 --> 00:03:56,635
If...So let me put the great
name down: Pythogaros
48
00:04:00,037 --> 00:04:02,520
I am looking for...
What am I looking for?
49
00:04:02,961 --> 00:04:05,295
I'm looking for the condition
50
00:04:06,631 --> 00:04:10,107
if you give me two vectors
how do I know if they're orthogonal
51
00:04:10,994 --> 00:04:13,885
How can I tell two
perpendicular vectors
52
00:04:13,760 --> 00:04:16,339
And actually you probably
know the answer
53
00:04:16,370 --> 00:04:17,795
Let me write the answer down
54
00:04:18,223 --> 00:04:22,000
Orthogonal vectors...
what's the test for orthogonality?
55
00:04:22,825 --> 00:04:31,303
I take the dot product
which I tend to write as x transpose y
56
00:04:32,567 --> 00:04:35,263
So that's a row times a column
57
00:04:36,011 --> 00:04:39,329
And that matrix multiplication
just gives me
58
00:04:39,364 --> 00:04:45,111
the right thing that x1y1+x2y2
and so on
59
00:04:45,731 --> 00:04:54,491
So these vectors are orthogonal
if this result x transpose y is 0
60
00:04:54,942 --> 00:04:57,471
That's the test, ok.
61
00:04:57,506 --> 00:05:01,900
Can I connect that two are the same?
62
00:05:03,116 --> 00:05:05,140
I mean it's like amazing
63
00:05:05,175 --> 00:05:07,960
It's just beautiful that ...
here we have...
64
00:05:08,368 --> 00:05:11,996
If it were n-dimension
we've got a couple of vectors
65
00:05:12,031 --> 00:05:14,085
We want to know
the angle between them
66
00:05:14,559 --> 00:05:17,019
and the right thing to look at is
67
00:05:17,517 --> 00:05:21,345
the simplest thing that you can imagin:
the dot product
68
00:05:21,380 --> 00:05:23,390
Ok, now why...
69
00:05:23,425 --> 00:05:26,828
So I'm answering the question now
why…
70
00:05:26,829 --> 00:05:28,902
Let's add some...
71
00:05:28,952 --> 00:05:36,799
Let's add some justification
to this fact, the fact of test... Ok
72
00:05:37,592 --> 00:05:40,799
So Pythagoras would say
we've got a right triangle...
73
00:05:40,834 --> 00:05:45,392
...if that line squared plus that line
squared equals that line squared
74
00:05:45,427 --> 00:05:57,136
OK, can I write it as
x^2+ y^2 equals (x+y)^2 ?
75
00:05:57,545 --> 00:06:00,685
Now don't! Please don't think
this is always true
76
00:06:01,671 --> 00:06:05,501
This is only gonna be true in this
77
00:06:05,945 --> 00:06:10,184
It's gonna be equivalent
to orthogonality
78
00:06:10,219 --> 00:06:13,511
For other triangles of course
it's not true
79
00:06:13,512 --> 00:06:15,356
For other triangles, it's not
80
00:06:15,391 --> 00:06:21,650
But for a right triangle, somehow
that fact should connect to that fact
81
00:06:21,685 --> 00:06:23,615
Can we just make that connection?
82
00:06:26,084 --> 00:06:30,784
What's the connection between
this test for orthogonality
83
00:06:30,819 --> 00:06:33,746
and this statement of orthogonality?
84
00:06:33,781 --> 00:06:37,784
Well I guess I have to say
what is the length square?
85
00:06:38,405 --> 00:06:44,438
So let's continue on the board
underneath with that equation
86
00:06:45,244 --> 00:06:49,789
Give me another way to express
the length squared of a vector
87
00:06:50,566 --> 00:06:54,768
Let me just give you a vector:
the vector (1,2,3)
88
00:06:55,543 --> 00:06:57,947
That's in 3-dimensions
89
00:06:58,670 --> 00:07:04,217
What's the length squared
of the vector x equal (1,2,3)?
90
00:07:06,306 --> 00:07:08,489
So how do you find the length squared?
91
00:07:09,409 --> 00:07:11,435
Well really you just …
92
00:07:11,470 --> 00:07:17,313
you want the length of that vector that
goes along line up to an out 3...
93
00:07:17,880 --> 00:07:22,815
and will come back to that
right triangle stuff
94
00:07:22,850 --> 00:07:24,743
The length squared is ...
95
00:07:24,778 --> 00:07:29,209
This is exactly x transpose x
96
00:07:31,831 --> 00:07:35,043
Whenever I see x transpose x
I know I've got...
97
00:07:35,631 --> 00:07:37,512
a number that's positive
98
00:07:38,026 --> 00:07:43,297
It's a length squared
unless x happens to be the zero vector
99
00:07:43,332 --> 00:07:46,432
That's the one case
where the length is zero
100
00:07:47,357 --> 00:07:54,783
So this is just
x1^2 + x2^2 + ... + xn^2
101
00:07:54,818 --> 00:07:57,405
So one... in the example I gave you
102
00:07:57,440 --> 00:08:00,724
what was the length squared
of that vector (1,2,3)?
103
00:08:02,169 --> 00:08:06,649
So you square those you get
1, 4, 9... you add... you get 14
104
00:08:07,341 --> 00:08:10,725
So the vector (1,2,3) has length 14
105
00:08:11,378 --> 00:08:15,148
So let me just put down the vector here
106
00:08:16,022 --> 00:08:19,262
Let x be the vector (1,2,3)
107
00:08:21,879 --> 00:08:25,329
Let me cook up a vector
that's orthogonal to it
108
00:08:28,269 --> 00:08:30,775
So what's the vector that
the orthogonal tell...
109
00:08:30,775 --> 00:08:37,224
Let's write down here x^2 =14
110
00:08:38,472 --> 00:08:41,224
Let me cook up a vector
that's orthogonal to it
111
00:08:42,082 --> 00:08:43,029
We'll get...
112
00:08:47,978 --> 00:08:51,183
Right? Those two vectors
are orthogonal
113
00:08:51,693 --> 00:08:58,581
The length of y^2 is 5
114
00:08:59,079 --> 00:09:09,195
And x+y is (3,1,3)
115
00:09:09,230 --> 00:09:17,277
And the length of
this squared is 19
116
00:09:18,257 --> 00:09:20,535
And sure enough…
117
00:09:21,873 --> 00:09:23,786
I haven't proved anything
118
00:09:23,821 --> 00:09:29,838
I just like check to see that my...
119
00:09:29,873 --> 00:09:34,823
My test x transpose y equals 0
which is true, right?
120
00:09:34,858 --> 00:09:37,885
Everybody see that
x transpose y is zero here?
121
00:09:38,354 --> 00:09:40,704
That's maybe the main point
122
00:09:40,739 --> 00:09:45,362
You should get really quick at feeling
x transpose y
123
00:09:45,397 --> 00:09:48,974
So it's just this plus this plus this
and that's zero
124
00:09:49,009 --> 00:09:56,438
And sure enough that clips
with 14 + 5 agree with 19
125
00:09:56,473 --> 00:09:59,198
Now let me just do that in letters
126
00:09:59,756 --> 00:10:10,436
So that's y transpose y
and this is (x+y) transpose (x+y)
127
00:10:11,463 --> 00:10:12,228
Ok
128
00:10:12,751 --> 00:10:14,536
So I'm looking again
129
00:10:14,571 --> 00:10:16,758
This is always true, I repeat
130
00:10:16,793 --> 00:10:20,394
This is gonna be true
when we have a right angle
131
00:10:21,078 --> 00:10:22,512
And let's just...
132
00:10:22,547 --> 00:10:26,191
Well, of course, I'm just gonna
simplify this stuff here
133
00:10:26,226 --> 00:10:32,125
There's an x transpose x there
and there's a y transpose y there
134
00:10:32,701 --> 00:10:41,694
And there's x transpose y
and there's a y transpose x
135
00:10:44,398 --> 00:10:48,517
I knew I could do that
simplification because...
136
00:10:49,471 --> 00:10:53,316
I'm just doing matrix multiplication
and I'm just following the rule
137
00:10:53,351 --> 00:10:59,179
Ok, So x transpose x is canceled
y transpose y is canceled
138
00:10:59,214 --> 00:11:00,908
and what about these, guys
139
00:11:00,944 --> 00:11:02,888
What can you tell me about...
140
00:11:04,845 --> 00:11:09,125
the inner product of x with y
and the inner product of y with x?
141
00:11:09,922 --> 00:11:11,817
Is there any difference?
142
00:11:12,942 --> 00:11:20,447
I think while we are doing real vectors
which is all we're doing now
143
00:11:20,465 --> 00:11:22,927
there isn't the difference
So there's no difference
144
00:11:23,106 --> 00:11:27,264
If I take x transpose y
that'll give me 0
145
00:11:27,299 --> 00:11:30,620
If I took y transpose x
I would have the same
146
00:11:30,655 --> 00:11:35,616
x1 y1 , x2 y2 and x3 y3
147
00:11:36,186 --> 00:11:37,430
It would be the same
148
00:11:37,465 --> 00:11:40,629
So this is the same as that
149
00:11:40,664 --> 00:11:41,635
This is really out...
150
00:11:41,670 --> 00:11:44,118
I knock that guy out
and say 2 of these
151
00:11:47,290 --> 00:11:55,989
So actually this equation boil down
to this thing being 0. Right?
152
00:11:56,327 --> 00:11:57,975
Everything else cancel
153
00:11:58,010 --> 00:12:00,926
And this equation boils
down to that one
154
00:12:00,961 --> 00:12:02,511
So that's really all I wanted
155
00:12:03,749 --> 00:12:06,490
I just wanted to check that
156
00:12:07,163 --> 00:12:14,470
Pythagoras for a right triangle
led me to this...
157
00:12:14,505 --> 00:12:17,600
of course I cancel the 2 now
no problem...
158
00:12:17,635 --> 00:12:21,458
to x transpose y equals 0 as the test
159
00:12:22,138 --> 00:12:23,851
Very nice, Ok
160
00:12:24,567 --> 00:12:25,939
You knew it was coming
161
00:12:26,485 --> 00:12:30,104
The dot product of
orthogonal vectors is 0
162
00:12:30,139 --> 00:12:32,379
In this I just want to say
163
00:12:32,414 --> 00:12:36,104
that's really indeed that
it comes out so well
164
00:12:36,139 --> 00:12:37,139
All right
165
00:12:38,543 --> 00:12:39,960
Now what about...
166
00:12:39,995 --> 00:12:41,592
so now I know
if two vectors...
167
00:12:41,627 --> 00:12:44,198
what it means when two
vectors are orthogonal
168
00:12:44,233 --> 00:12:48,423
By the way, what about if one
of these guys is zero vector?
169
00:12:49,504 --> 00:12:54,244
Suppose X is zero vector
and Y is whatever
170
00:12:54,889 --> 00:12:57,013
are they orthogonal?
171
00:12:58,732 --> 00:12:59,432
Sure!
172
00:13:00,726 --> 00:13:04,396
In maths, the one thing about
maths is that you follow the rule
173
00:13:05,647 --> 00:13:06,901
So you suppose...
174
00:13:06,936 --> 00:13:10,487
if X is zero vector
you suppose to take
175
00:13:10,522 --> 00:13:15,303
the zero vector dotted with Y
and of course, you always get zero
176
00:13:15,796 --> 00:13:19,589
So just so we're all sure:
177
00:13:19,987 --> 00:13:22,796
the zero vector is orthogonal
to everybody
178
00:13:24,546 --> 00:13:30,014
But what I want to...
what I now want to...
179
00:13:32,107 --> 00:13:34,229
to think about is subspace...
180
00:13:35,833 --> 00:13:39,121
What does it mean for me to say
181
00:13:39,631 --> 00:13:43,472
that some subspace is
orthogonal to some other subspace?
182
00:13:44,160 --> 00:13:47,254
So Ok, now I got to write this down
so...
183
00:13:47,254 --> 00:14:02,320
to finding definition of subspace S
is orthogonal to subspace
184
00:14:04,107 --> 00:14:05,808
Let's say T
185
00:14:06,807 --> 00:14:08,388
So we've got a couple of subspaces
186
00:14:10,999 --> 00:14:14,687
And what should it mean for
those guys to be orthogonal?
187
00:14:15,412 --> 00:14:19,462
So what's the naturally extension from
188
00:14:19,497 --> 00:14:23,504
for orthogonal vectors
to orthogonal subspaces?
189
00:14:23,539 --> 00:14:26,826
Well, and in particular
190
00:14:29,618 --> 00:14:33,791
Let's think of some orthogonal
subspaces like...
191
00:14:35,776 --> 00:14:36,910
this wall
192
00:14:38,454 --> 00:14:39,978
Let's say in 3-dimensions
193
00:14:41,109 --> 00:14:44,555
So the blackboard extended
to infinity, right?
194
00:14:44,590 --> 00:14:49,514
There's a subspace, a plane
a 2-dimensional subspace
195
00:14:51,527 --> 00:14:54,028
It's a little something but…
Anyway
196
00:14:54,063 --> 00:14:56,749
It's...think of it as a subspace
197
00:14:58,847 --> 00:15:01,890
Let me take the floor
as another subspace
198
00:15:05,505 --> 00:15:06,538
Again
199
00:15:08,388 --> 00:15:11,996
it's not....
great subspace
200
00:15:12,031 --> 00:15:16,769
It might be only doubted as...
like so so, but
201
00:15:18,450 --> 00:15:21,987
And I'll put the origin right here
202
00:15:22,697 --> 00:15:26,309
So the origin of the world
is right there
203
00:15:26,942 --> 00:15:27,892
Ok
204
00:15:30,528 --> 00:15:35,451
Thereby giving linear algebra is
proper importance in this
205
00:15:35,486 --> 00:15:39,106
OK, so there's one subspace
There's another one on the floor
206
00:15:39,647 --> 00:15:40,844
And
207
00:15:42,864 --> 00:15:44,292
Are they orthogonal?
208
00:15:45,825 --> 00:15:49,107
What does it mean for two
subspaces to be orthogonal?
209
00:15:49,142 --> 00:15:51,429
And in that special case
are they orthogonal?
210
00:15:51,464 --> 00:15:54,381
All right.
Let's finish this sentence
211
00:15:55,201 --> 00:15:56,532
What does it mean? Means...
212
00:16:01,466 --> 00:16:04,217
You have to know
what we are talking about this
213
00:16:04,252 --> 00:16:08,265
So what would be a reasonable
idea of orthogonal?
214
00:16:09,154 --> 00:16:12,292
Or let me put
the right thing up
215
00:16:12,916 --> 00:16:16,160
It means that every vector in S
216
00:16:16,692 --> 00:16:27,619
Every vector in S is orthogonal to ...
217
00:16:30,142 --> 00:16:32,567
to...
what am I gonna say?
218
00:16:33,724 --> 00:16:35,815
Every vector in T
219
00:16:41,792 --> 00:16:43,042
That's...
220
00:16:44,799 --> 00:16:46,030
That's a reasonable
221
00:16:46,065 --> 00:16:48,653
and it's a good and it's
a right definition
222
00:16:48,688 --> 00:16:50,835
for two subspaces to be orthogonal
223
00:16:50,870 --> 00:16:53,888
But I just want you to see
hey, what does that mean?
224
00:16:54,619 --> 00:17:00,527
So I'll answer the question
about the blackboard and the floor
225
00:17:01,310 --> 00:17:05,652
Are those two subspaces..
they are 2-dimensional, right?
226
00:17:05,687 --> 00:17:07,614
…and were in R3...
227
00:17:08,723 --> 00:17:14,362
It's like the x-z plane
or something and x-y plane
228
00:17:18,994 --> 00:17:20,659
Are they orthogonal?
229
00:17:21,387 --> 00:17:25,560
Is every vector in the blackboard
orthogonal to every vector in the floor?
230
00:17:26,317 --> 00:17:28,699
So I've already put the origin
right there
231
00:17:31,552 --> 00:17:34,279
Yes or no?
How to take a vote?
232
00:17:36,052 --> 00:17:38,336
Can we get some guess to the note?
233
00:17:39,233 --> 00:17:41,299
No is the answer. That's not.
234
00:17:41,334 --> 00:17:45,639
You can tell me a vector
in the blackboard
235
00:17:45,674 --> 00:17:48,978
and a vector in the floor
that are not orthogonal
236
00:17:55,042 --> 00:17:58,475
Well, you can tell me quite
a few I guess
237
00:17:59,250 --> 00:18:05,392
Maybe I can take some 45
degrees guy in the blackboard
238
00:18:05,759 --> 00:18:09,200
And something in the floor...
239
00:18:10,836 --> 00:18:13,016
they aren't 90 degrees right?
240
00:18:13,051 --> 00:18:14,490
In fact even more...
241
00:18:14,525 --> 00:18:16,126
you could tell me a vector
242
00:18:17,713 --> 00:18:21,949
that's in both the blackboard
plane and the floor plane
243
00:18:21,984 --> 00:18:24,439
so it's certainly not orthogonal
to itself
244
00:18:24,474 --> 00:18:29,465
So for sure, those two planes
are not orthogonal
245
00:18:30,173 --> 00:18:31,294
What would that be?
246
00:18:31,329 --> 00:18:35,663
So what's the vector that's
in both of those planes?
247
00:18:36,678 --> 00:18:40,062
This guy running along the crap there
248
00:18:40,097 --> 00:18:43,082
were the intersects, the intersects...
249
00:18:44,076 --> 00:18:49,571
It's two subspaces meet
some vector, well then
250
00:18:49,606 --> 00:18:51,969
for sure they not orthogonal
because...
251
00:18:53,536 --> 00:18:56,892
that vector is in one
and it's in the other
252
00:18:56,927 --> 00:19:00,162
and it's not orthogonal
to itself unless it's zero
253
00:19:00,197 --> 00:19:03,657
So the only, I mean
orthogonal...
254
00:19:04,871 --> 00:19:08,494
if permit to say these
subspaces are orthogonal...
255
00:19:08,529 --> 00:19:09,355
First of all
256
00:19:09,390 --> 00:19:18,681
I'm certainly saying that they
don't interset in any nonzero vector
257
00:19:18,716 --> 00:19:23,832
But also I mean more than that
not intersecting isn't good enough
258
00:19:23,867 --> 00:19:25,409
So give me an example
259
00:19:26,137 --> 00:19:28,276
Oh, let's say in the plane...
260
00:19:29,537 --> 00:19:31,464
Oh, well
261
00:19:31,499 --> 00:19:34,552
When do we have orthogonal
subspaces in the plane?
262
00:19:34,807 --> 00:19:39,334
Tell me in the plane so we don't get
any different subspaces in the plane
263
00:19:39,369 --> 00:19:43,583
What do we get in the plane
as possible some spaces?
264
00:19:45,306 --> 00:19:48,313
The zero vector, real small
265
00:19:49,275 --> 00:19:53,608
align through the origin
or the whole plane?
266
00:19:53,643 --> 00:19:59,111
OK, now so when is the line
through the origin
267
00:19:59,146 --> 00:20:01,446
orthogonal to the whole plane?
268
00:20:04,532 --> 00:20:06,300
Never! right? Never!
269
00:20:06,797 --> 00:20:11,435
When is the line through the origin
orthogonal to the zero subspace?
270
00:20:13,966 --> 00:20:15,271
Always, right!
271
00:20:15,860 --> 00:20:18,869
When is the line through the origin
orthogonal to
272
00:20:18,904 --> 00:20:21,105
a different line through the origin?
Well
273
00:20:21,140 --> 00:20:24,947
That's the case that we
all have a clear picture of...
274
00:20:24,982 --> 00:20:29,897
the two lines have to meet
it 90 degrees, they have only the...
275
00:20:29,932 --> 00:20:33,838
So that's like a simple
case I'm talking about:
276
00:20:33,873 --> 00:20:36,702
There's one subspace
there's the other subspace
277
00:20:36,737 --> 00:20:40,865
They only meet at zero
and they're orthogonal, OK.
278
00:20:41,591 --> 00:20:42,273
Now
279
00:20:43,128 --> 00:20:44,920
so we now know what it means
280
00:20:44,955 --> 00:20:48,147
for two subspaces to be orthogonal
281
00:20:48,182 --> 00:20:49,812
and now I want to say
282
00:20:49,847 --> 00:20:53,394
this is true
...for the row space and null space...
283
00:20:53,429 --> 00:20:58,074
ok, so let's meet fact
284
00:20:58,109 --> 00:21:10,376
So row space is orthogonal
to the nullspace
285
00:21:10,411 --> 00:21:12,491
Now how did I come up with that?
286
00:21:16,945 --> 00:21:19,686
But you see the line descript
287
00:21:19,721 --> 00:21:23,984
that means that these subspaces
are just the right thing...
288
00:21:24,019 --> 00:21:31,676
they are just cutting the whole space up
into two perpendicular subspaces
289
00:21:32,374 --> 00:21:33,617
OK, so why?
290
00:21:38,067 --> 00:21:42,392
Well we're now gonna come work with
291
00:21:42,802 --> 00:21:51,700
all I know is , the nullspace has
vector that's all, Ax=0
292
00:21:51,701 --> 00:21:54,599
So this is the guy X
293
00:21:55,407 --> 00:21:57,324
X is in the nullspace
294
00:21:58,679 --> 00:22:00,270
Then Ax=0
295
00:22:02,055 --> 00:22:06,766
Why is it orthogonal
to the rows of A?
296
00:22:08,000 --> 00:22:12,303
If I write down Ax=0 which is
all I know about the nullspace
297
00:22:13,125 --> 00:22:13,989
Then
298
00:22:14,912 --> 00:22:18,010
I guess I want to see
that's telling me
299
00:22:18,045 --> 00:22:20,554
just as the equation right there
is telling me
300
00:22:22,008 --> 00:22:24,317
that the rows of A
let me write it out
301
00:22:24,352 --> 00:22:27,501
There's row1 of A
302
00:22:29,059 --> 00:22:36,759
Row2...row n of A...
that's A
303
00:22:37,188 --> 00:22:42,506
And it's multiplying X and
it's producing zero
304
00:22:44,828 --> 00:22:45,981
OK
305
00:22:46,016 --> 00:22:49,171
Written out that way, you see it
306
00:22:52,288 --> 00:22:54,944
So I'm saying that
a vector in the row space
307
00:22:54,979 --> 00:22:58,992
is perpendicular to this guy
x in the nullspace
308
00:22:59,027 --> 00:23:00,713
And you see why?
309
00:23:02,347 --> 00:23:08,690
Cause this equation is telling you
that row 1 of A multiplying...
310
00:23:08,725 --> 00:23:11,051
that's a dot product right?
311
00:23:11,086 --> 00:23:14,973
Row 1 of A dot product with
this x is producing this zero
312
00:23:16,581 --> 00:23:19,893
So x is orthogonal to the first row
313
00:23:20,580 --> 00:23:22,492
and to the second row
314
00:23:22,527 --> 00:23:25,594
Row 2 of A, x, is giving that zero
315
00:23:25,629 --> 00:23:28,539
Row m of A and x is giving that zero
316
00:23:28,574 --> 00:23:29,683
So x is...
317
00:23:30,100 --> 00:23:34,472
The equation is telling me that
x is orthogonal to all the rows
318
00:23:36,378 --> 00:23:38,309
Right? It's just sitting there
319
00:23:38,727 --> 00:23:39,845
As all we...
320
00:23:39,880 --> 00:23:41,222
It has to be sitting there
321
00:23:41,222 --> 00:23:44,588
coz we didn't know anything
more about the nullspace than this
322
00:23:44,588 --> 00:23:50,857
And now I guess to be totally
complete here
323
00:23:51,552 --> 00:23:56,522
I've now checked that x is orthogonal
to each separate row
324
00:23:57,077 --> 00:24:01,022
But what else strictly thinking
do I have to do?
325
00:24:04,778 --> 00:24:10,292
To show that some spaces is orthogonal
I have to take this x in all spaces
326
00:24:10,327 --> 00:24:15,696
and show that it's orthogonal to
every vector in the row space
327
00:24:15,731 --> 00:24:17,301
every vector in the row space
328
00:24:17,336 --> 00:24:19,747
So what else to think in the row space?
329
00:24:20,417 --> 00:24:23,854
This row is in the row space
That row is in the row space
330
00:24:24,331 --> 00:24:25,969
They are all there
331
00:24:26,004 --> 00:24:28,473
But it's not the whole row space
right?
332
00:24:28,508 --> 00:24:30,724
We just would like to remember
what does that mean
333
00:24:30,759 --> 00:24:33,591
what does that word space telling us?
334
00:24:34,788 --> 00:24:41,488
And what else is in the row space
besides the rows?
335
00:24:42,962 --> 00:24:46,463
All there come it?
336
00:24:46,864 --> 00:24:49,877
So I really have to check it
that sure enough
337
00:24:49,912 --> 00:24:55,118
if x is perpendicular to row1, row2
all the different separate rows
338
00:24:55,153 --> 00:24:59,379
then also, x is perpendicular to
a combination of the rows
339
00:24:59,414 --> 00:25:01,962
And that's just matrix
multiplication again
340
00:25:01,997 --> 00:25:07,992
You know I have row1
transpose x is zero
341
00:25:08,807 --> 00:25:12,483
So on row 2 transpose x is zero
342
00:25:16,224 --> 00:25:20,327
So I may tell you some multiply
that by some c1
343
00:25:20,362 --> 00:25:22,154
this by some c2
344
00:25:22,189 --> 00:25:23,552
I still have zeros
345
00:25:23,587 --> 00:25:25,459
I may tell you to add
346
00:25:25,821 --> 00:25:29,562
so I have c1 row 1
so all these...
347
00:25:29,597 --> 00:25:39,700
I put that together that big parenthesis
to see c1 row1 + c2 row2 and so on...
348
00:25:40,149 --> 00:25:44,100
transpose x is zero
Right?
349
00:25:44,135 --> 00:25:46,578
I just add it to zero
We've got zero
350
00:25:46,613 --> 00:25:49,589
And I just add these
following the rules
351
00:25:51,900 --> 00:25:56,855
No big deal, the whole point
was right sitting in that
352
00:25:57,288 --> 00:25:58,101
Ok
353
00:25:59,928 --> 00:26:00,834
So
354
00:26:01,615 --> 00:26:07,465
If I come back to this figure now
I'm like the happier person
355
00:26:07,877 --> 00:26:10,922
Cause I have this...
356
00:26:12,267 --> 00:26:16,043
now see how those
subspaces are oriented
357
00:26:16,078 --> 00:26:20,611
And these subspaces are
also oriented, well, actually
358
00:26:21,408 --> 00:26:24,612
Why is that orthogonality?
359
00:26:25,416 --> 00:26:30,834
Well it's the same statement for
A transpose as that one was for A
360
00:26:30,869 --> 00:26:33,192
So I won't take kind of
provement again
361
00:26:33,227 --> 00:26:37,981
because we've checked it
for every matrix
362
00:26:38,016 --> 00:26:41,484
and A transpose is just
as good a matrix as A
363
00:26:41,519 --> 00:26:44,585
So we are orthogonal over there
364
00:26:44,881 --> 00:26:50,645
So we really have carved up...
this...
365
00:26:51,582 --> 00:26:58,081
This was like carving up m-dimensional
space in the two subspaces
366
00:26:59,459 --> 00:27:03,332
And this one was carving up
n-dimensional space
367
00:27:05,017 --> 00:27:07,006
in the two subspaces
368
00:27:07,351 --> 00:27:08,097
And
369
00:27:09,914 --> 00:27:13,501
Well one more thing here
one more important thing
370
00:27:16,167 --> 00:27:20,081
Let me move in the 3-dimensions
371
00:27:21,204 --> 00:27:30,004
Tell me a couple of orthogonal
subspaces in 3-dimensions that....
372
00:27:30,537 --> 00:27:34,743
somehow don't carve up
the whole spaces that lap there
373
00:27:35,867 --> 00:27:39,675
I'm thinking of a couple
of orthogonal lines
374
00:27:40,442 --> 00:27:44,113
If I...Suppose I mean 3-dimensions
R3
375
00:27:44,732 --> 00:27:50,839
And I have one line, one dimensional
subspace and a perpendicular one
376
00:27:51,429 --> 00:27:56,098
Could those be the row
space and the nullspace?
377
00:27:56,572 --> 00:27:59,592
Could those be the row
space and the nullspace?
378
00:27:59,627 --> 00:28:02,276
Could I be in 3-dimensions...
379
00:28:04,123 --> 00:28:10,536
And have a row space that's a line
and a nullspace that's a line?
380
00:28:11,675 --> 00:28:12,426
No.
381
00:28:12,981 --> 00:28:14,012
Why not?
382
00:28:15,683 --> 00:28:18,047
Because its dimensions
aren't right, right ?
383
00:28:18,082 --> 00:28:19,557
The dimensions are no good
384
00:28:19,592 --> 00:28:24,701
The dimensions here r and n-r
they add up to 3
385
00:28:24,736 --> 00:28:25,945
They add up to n
386
00:28:27,129 --> 00:28:31,863
If I take...
Follow that example
387
00:28:32,567 --> 00:28:36,950
If the row space is 1-dimensional
388
00:28:36,985 --> 00:28:41,855
Suppose A is...
what's a good...I mean in R3...
389
00:28:41,890 --> 00:28:44,239
I want a 1-dimensional row space
390
00:28:44,274 --> 00:28:47,703
let me take 1, 2, 5
2, 4, 10
391
00:28:49,353 --> 00:28:51,760
What's the dimension of
that row space?
392
00:28:54,379 --> 00:28:55,165
One?
393
00:28:55,768 --> 00:28:58,601
What's the dimension of the nullspace?
394
00:29:01,342 --> 00:29:04,985
What does the nullspace
look like in that case?
395
00:29:05,020 --> 00:29:07,199
The row space is a line
Right?
396
00:29:07,234 --> 00:29:10,119
One dimensional
just a line through 1, 2, 5
397
00:29:12,122 --> 00:29:14,672
Symmetrically what does
the row space look like?
398
00:29:16,895 --> 00:29:17,930
It's a ...
399
00:29:19,744 --> 00:29:21,212
What's the dimension?
400
00:29:21,247 --> 00:29:25,238
So here n is 3
401
00:29:25,984 --> 00:29:27,728
The rank is 1
402
00:29:28,595 --> 00:29:32,673
So the dimension of the nullspace...
403
00:29:32,708 --> 00:29:40,134
I'm looking at this X...x1, x2, x3...
to get zero
404
00:29:42,345 --> 00:29:45,765
So the dimension of the nullspace is
405
00:29:46,158 --> 00:29:47,631
We all know it
406
00:29:48,358 --> 00:29:49,233
2
407
00:29:49,268 --> 00:29:50,629
Right, it's a plane
408
00:29:51,467 --> 00:29:55,172
And now actually we know
we'd see better what plane it is
409
00:29:55,647 --> 00:29:57,271
What plane is it?
410
00:29:58,203 --> 00:30:04,281
It's the plane that
perpendicular to (1, 2, 5)
411
00:30:05,928 --> 00:30:07,435
Right we've now seen...
412
00:30:07,470 --> 00:30:10,578
in fact 2, 4, 10 actually
didn't have any effect at all
413
00:30:10,613 --> 00:30:12,529
Just ignore that
414
00:30:13,005 --> 00:30:16,009
That didn't change the row space
or the nullspace
415
00:30:18,164 --> 00:30:21,720
I'll just make that one equation
416
00:30:22,402 --> 00:30:24,118
Yes, ok, sure
417
00:30:24,153 --> 00:30:27,182
That's the easiest with one equation
418
00:30:28,046 --> 00:30:29,111
three unknown
419
00:30:30,513 --> 00:30:32,438
And...
420
00:30:34,305 --> 00:30:36,109
I want to ask
421
00:30:38,786 --> 00:30:41,270
what does the equation give me?
422
00:30:41,305 --> 00:30:42,941
It gives me the nullspace
423
00:30:43,535 --> 00:30:49,383
and you would have said that
in September it gives you a plane
424
00:30:49,776 --> 00:30:52,545
and we were completely right
425
00:30:52,580 --> 00:30:54,190
And the plane it gives you
426
00:30:54,225 --> 00:30:57,812
the normal vector, you remember
in calculus it was
427
00:30:58,266 --> 00:31:00,768
the normal vector called n
428
00:31:00,803 --> 00:31:04,445
Well there is it: (1,2,5)
OK
429
00:31:05,867 --> 00:31:06,635
So
430
00:31:09,159 --> 00:31:14,052
What's the point I want to make here?
431
00:31:14,705 --> 00:31:21,998
I want to emphasize that not only
are the... let me write it in words
432
00:31:25,698 --> 00:31:28,543
So I want to write that nullspace
433
00:31:31,382 --> 00:31:41,350
and a row space are orthogonal
434
00:31:43,035 --> 00:31:49,336
that does need the fact
with just check Ax=0
435
00:31:49,371 --> 00:31:51,829
But now I want to say
436
00:31:51,864 --> 00:31:55,738
I wanna say more
because it's a little more bit true
437
00:31:57,244 --> 00:32:00,353
Their dimension adds to
the whole space
438
00:32:01,260 --> 00:32:03,457
So that's like a little
extra information
439
00:32:04,042 --> 00:32:09,470
It's not like that I couldn't have
a line and a line in 3-dimensions
440
00:32:09,505 --> 00:32:12,688
Those don't add up...
one and one don't add to three
441
00:32:13,348 --> 00:32:24,209
So I use the word orthogonal
complements in Rn
442
00:32:24,244 --> 00:32:28,520
And I give this word "complements"
is that
443
00:32:30,255 --> 00:32:34,933
the orthogonal complement
of a row space
444
00:32:35,535 --> 00:32:40,201
contains not just some vectors
that are orthogonal to it but all
445
00:32:40,604 --> 00:32:42,387
So what does that mean?
446
00:32:42,422 --> 00:32:49,665
That means that the
nullspace contains all...
447
00:32:51,538 --> 00:32:58,995
not just some but all vectors that
are perpendicular to the row space
448
00:33:00,215 --> 00:33:01,407
OK
449
00:33:03,959 --> 00:33:09,586
Really what I have done
in the past of this lecture is just
450
00:33:10,483 --> 00:33:17,411
… notice some nice geometry
that we didn't pick up before
451
00:33:17,446 --> 00:33:20,905
because we didn't discuss
perpendicular vectors before
452
00:33:21,286 --> 00:33:25,368
But with all sitting there
and now we've picked it up that
453
00:33:25,403 --> 00:33:27,953
these vectors are orthogonal
complements
454
00:33:27,988 --> 00:33:30,308
And I guess I can call this
455
00:33:31,222 --> 00:33:35,772
part one of the fundamental
theorem of linear algebra
456
00:33:35,807 --> 00:33:42,030
The fundamental theorem of linear
algebra is about these four subspaces
457
00:33:43,087 --> 00:33:44,929
So part one is about their dimension
458
00:33:44,964 --> 00:33:47,004
Maybe I should call it part two now
459
00:33:47,958 --> 00:33:50,252
Their dimensions we've got...
460
00:33:50,729 --> 00:33:54,564
now we're getting their
orthogonality, that's part two
461
00:33:55,297 --> 00:34:02,855
And part three will be about
matrix form, orthogonal matrix
462
00:34:03,548 --> 00:34:07,821
So that's coming up, OK
463
00:34:07,856 --> 00:34:17,050
So I'm happy with that
geometry right now
464
00:34:17,085 --> 00:34:22,604
OK, ok, now what's my next
going with this chapter?
465
00:34:22,639 --> 00:34:25,464
Here's the main problem of
this chapter
466
00:34:26,417 --> 00:34:29,433
The main problem of this chapter is ...
467
00:34:29,468 --> 00:34:35,110
so this is coming ...
coming abstraction…
468
00:34:40,040 --> 00:34:44,303
This is the very last chapter
that's about Ax=b
469
00:34:50,229 --> 00:34:55,076
I would like just solve
that set of an equation
470
00:34:55,786 --> 00:34:57,793
When there's no solutions?
471
00:35:01,171 --> 00:35:03,637
You may say what a rediculous
thing to do
472
00:35:03,672 --> 00:35:06,948
But I have to say that
it's done all the time
473
00:35:06,983 --> 00:35:08,962
In fact it has to be done
474
00:35:08,997 --> 00:35:10,221
You get...
475
00:35:10,256 --> 00:35:16,152
so the problem is solve the
best possible
476
00:35:16,187 --> 00:35:18,712
"Solve", I put a quote
477
00:35:20,628 --> 00:35:28,276
Ax=b
when there is no solution
478
00:35:31,325 --> 00:35:35,200
And of course what does it mean?
b isn't in the column space
479
00:35:36,152 --> 00:35:38,255
And it's quite typical
480
00:35:38,290 --> 00:35:42,420
if this matrix A is rectangular
if I ...
481
00:35:42,455 --> 00:35:44,817
maybe I have m equations
482
00:35:44,852 --> 00:35:48,016
and that's bigger than the number
I've known
483
00:35:48,912 --> 00:35:57,090
Then for sure the rank is not m...
the rank couldn't be m now
484
00:35:57,125 --> 00:36:00,329
So there will be a lot right in size
with no solution
485
00:36:00,364 --> 00:36:01,210
But...
486
00:36:03,415 --> 00:36:05,494
here's an example
487
00:36:07,146 --> 00:36:11,409
Some satellite is buzzling along
you measure its position
488
00:36:13,093 --> 00:36:15,028
you make a thousand measurements
489
00:36:16,940 --> 00:36:20,503
So that give you a thousand equations
for the...
490
00:36:20,538 --> 00:36:24,601
for the parameters
that gives the position
491
00:36:25,203 --> 00:36:28,993
But to write the thousand parameters
maybe six
492
00:36:30,124 --> 00:36:32,462
Or you're measuring...
493
00:36:33,166 --> 00:36:34,767
you're doing questionares
494
00:36:36,406 --> 00:36:39,133
You are measuring...
495
00:36:42,771 --> 00:36:44,556
you're taking postscript
496
00:36:44,591 --> 00:36:47,034
you're measuring somebody's postscript
497
00:36:47,069 --> 00:36:50,136
Ok, just one unknown…
the postscript
498
00:36:51,801 --> 00:36:53,993
so you measured one…
Ok fine
499
00:36:54,028 --> 00:36:58,358
But if you really want to know it
you measure it multiple times
500
00:36:59,141 --> 00:37:02,550
But then the measuments
have no indent
501
00:37:02,585 --> 00:37:06,857
So the problem is that
in many many problems
502
00:37:06,892 --> 00:37:09,233
we've got too many equations
503
00:37:09,669 --> 00:37:13,153
And they've got no aid
in the right hand size
504
00:37:13,188 --> 00:37:15,726
So Ax=b
505
00:37:16,603 --> 00:37:18,924
I can't expect to solve it exactly
right?
506
00:37:18,959 --> 00:37:23,352
I don't even know the errors
there's in the measurement
507
00:37:23,819 --> 00:37:25,316
The same in b
508
00:37:25,710 --> 00:37:27,696
But there's information too
509
00:37:28,180 --> 00:37:31,815
There's a lot of information
about x in there
510
00:37:31,850 --> 00:37:36,684
And what I want to do is like
separate the noise, the junk
511
00:37:37,550 --> 00:37:39,689
from the information
512
00:37:40,384 --> 00:37:45,481
And so this is a straightforward
linear algebra problem
513
00:37:45,516 --> 00:37:48,619
How do I solve...
what's the best solution?
514
00:37:48,654 --> 00:37:52,949
OK, now let me...
515
00:37:55,200 --> 00:38:01,905
I want to say so that's like describe
the problem in an algebraic way
516
00:38:01,940 --> 00:38:03,651
I've got this equation
517
00:38:04,206 --> 00:38:05,998
I'm looking for the best solution
518
00:38:06,033 --> 00:38:09,227
Well one way to find it is ...
one way to start...
519
00:38:09,262 --> 00:38:11,559
one way to find a solution is
520
00:38:12,017 --> 00:38:16,551
throw away equations
521
00:38:16,586 --> 00:38:19,951
until you've got a nice squared
invertible system and solve that
522
00:38:21,969 --> 00:38:23,972
That's not satisfactory
523
00:38:25,325 --> 00:38:28,781
There's no reason...
and these measurements...
524
00:38:28,816 --> 00:38:32,282
to say, these measurements are perfect
and these measurements are useless
525
00:38:33,206 --> 00:38:37,266
We want to use all the measurements to
get the best information
526
00:38:37,301 --> 00:38:38,998
to get the maximum information
527
00:38:39,033 --> 00:38:41,134
But how? OK.
528
00:38:41,820 --> 00:38:46,604
Let me anticipate a matrix
that's gonna show up
529
00:38:47,132 --> 00:38:50,086
This A is typically rectangular
530
00:38:50,913 --> 00:38:53,700
but a matrix that shows up
531
00:38:53,735 --> 00:38:58,880
whenever we have in chapter 3 was
all about rectangular matrices
532
00:38:59,589 --> 00:39:04,521
And we know whether this is solvable
you could do an elimination on it, right?
533
00:39:05,292 --> 00:39:07,646
But I'm thinking, hey
you're doing elimination
534
00:39:07,681 --> 00:39:13,798
and you get equation
0=other nonzeros
535
00:39:13,833 --> 00:39:17,577
I'm thinking we really...
elimination is going to fail
536
00:39:17,981 --> 00:39:22,899
So that's our question:
elimination will get us down to...
537
00:39:23,566 --> 00:39:25,901
it will tell us
if there's a solution or not
538
00:39:25,936 --> 00:39:28,549
But I'm now thinking
Not
539
00:39:29,069 --> 00:39:31,170
Ok, so what are we going to do?
540
00:39:31,205 --> 00:39:35,752
All right. I want to tell
you that jump ahead
541
00:39:35,787 --> 00:39:39,927
does the matrix that
will play a key role?
542
00:39:39,962 --> 00:39:45,793
So this is the matrix that you want to
understand for this chapter 4
543
00:39:46,166 --> 00:39:49,459
And it's the matrix A transpose A?
544
00:39:55,187 --> 00:39:56,263
What...
545
00:39:57,144 --> 00:39:59,763
Tell me something about that matrix
546
00:40:00,549 --> 00:40:04,876
So A is this n by m matrix
rectangular
547
00:40:05,613 --> 00:40:10,132
But now I'm saying that a good matrix
that shows up in the end
548
00:40:10,574 --> 00:40:12,398
is A transpose A
549
00:40:12,433 --> 00:40:14,927
So tell me something about that
550
00:40:16,168 --> 00:40:20,455
Is it.. yeah..tell me what's the first
thing in it all about A transpose A?
551
00:40:21,315 --> 00:40:23,238
It's squared, right?
552
00:40:24,121 --> 00:40:26,313
Squared, because this is m by n
553
00:40:26,939 --> 00:40:28,357
And this is n by m
554
00:40:28,962 --> 00:40:32,199
So this is the result n by n
Good
555
00:40:32,234 --> 00:40:34,505
Squared, what else?
556
00:40:35,389 --> 00:40:37,009
It's symmetric, good
557
00:40:37,044 --> 00:40:38,373
It's symmetric
558
00:40:44,964 --> 00:40:47,006
Of course you remember
how to do that
559
00:40:47,041 --> 00:40:49,232
We transpose that matrix...
560
00:40:50,249 --> 00:40:51,786
let's transpose it...
561
00:40:51,821 --> 00:41:00,427
A transpose A by transpose it
then that comes first transpose...
562
00:41:00,462 --> 00:41:08,012
this comes second transpose
and then transpose in twice is...
563
00:41:08,836 --> 00:41:10,970
This brings it back to the same ...
564
00:41:11,005 --> 00:41:13,173
So it's symmetric
Good
565
00:41:13,707 --> 00:41:14,809
Now
566
00:41:16,345 --> 00:41:19,422
we now know how to
ask more about a matrix
567
00:41:21,178 --> 00:41:25,819
I'm interested in
is it invertible?
568
00:41:27,409 --> 00:41:29,688
If not, what's its nullspace?
569
00:41:31,248 --> 00:41:33,019
So I want to know about..
570
00:41:33,054 --> 00:41:34,472
Cause you're gonna see...
571
00:41:34,507 --> 00:41:36,541
Well let me leave an ...
572
00:41:38,177 --> 00:41:40,765
well I shouldn't do this but I will
573
00:41:40,800 --> 00:41:44,145
Let me tell you what equation to solve
574
00:41:45,953 --> 00:41:47,929
when you can't solve that one
575
00:41:49,984 --> 00:41:56,689
The good equation comes from
multiplying both sides by trans(A)
576
00:41:57,260 --> 00:42:02,385
So the good equation
that you get to is this one
577
00:42:02,420 --> 00:42:08,793
A transpose Ax equals A transpose b
578
00:42:12,512 --> 00:42:15,173
That would be the central
equation in the chapter
579
00:42:16,038 --> 00:42:20,271
So I think why not tell it to you
why not admit it right away
580
00:42:20,970 --> 00:42:22,044
OK
581
00:42:22,079 --> 00:42:24,684
I should really give x...
582
00:42:28,151 --> 00:42:31,847
I would sort of indicate this x...
583
00:42:31,882 --> 00:42:37,261
I mean this x was the solution
to that equation
584
00:42:37,296 --> 00:42:39,594
if it existed but probably didn't
585
00:42:41,093 --> 00:42:45,903
Now let me give this
a different name: x ^
586
00:42:46,819 --> 00:42:51,389
Because I'm hoping
this one will have a solution
587
00:42:53,338 --> 00:42:56,457
And I'm saying that
it's my best solution
588
00:42:57,178 --> 00:42:59,648
I'd have to say what is best mean
589
00:42:59,683 --> 00:43:03,206
But that's gonna be my plan
590
00:43:03,241 --> 00:43:07,614
I'm gonna say that a best
solution solve this equation
591
00:43:07,649 --> 00:43:11,315
so you see right away
why I'm so interested in this matrix:
592
00:43:11,350 --> 00:43:15,803
A transpose A
and in it, invertibility
593
00:43:16,523 --> 00:43:19,970
OK, now when is it invertible?
594
00:43:21,529 --> 00:43:22,638
Ok
595
00:43:22,673 --> 00:43:26,942
Let me take a case
let me just do an example
596
00:43:26,943 --> 00:43:28,409
And then
597
00:43:29,906 --> 00:43:32,462
I'll just pick a matrix here
598
00:43:33,337 --> 00:43:36,419
Just so we see what A transpos A
looks like
599
00:43:36,454 --> 00:43:42,952
So let me take a matrix
[1, 1, 1, 2, 1, 5]
600
00:43:42,987 --> 00:43:45,134
Just invent a matrix
601
00:43:45,169 --> 00:43:46,437
So there's a matrix A
602
00:43:48,061 --> 00:43:54,059
Notice that it has m equals
3 rows and n equals 2 columns
603
00:43:56,122 --> 00:43:57,686
Its rank is...
604
00:43:59,313 --> 00:44:03,551
the rank of that matrix is 2, right?
605
00:44:03,586 --> 00:44:05,379
Yes, the columns are independent
606
00:44:06,169 --> 00:44:10,917
Does Ax=b...
if I look at Ax=b...
607
00:44:10,952 --> 00:44:17,586
so x is just [x1 x2]
and b is [b1 b2 b3]
608
00:44:20,448 --> 00:44:23,130
Do I expect just all Ax equal b?
609
00:44:23,165 --> 00:44:25,398
What...No way, right?
610
00:44:25,433 --> 00:44:28,729
I mean linear algebra is great but...
611
00:44:28,764 --> 00:44:33,088
solving three equations with only
two unknowns, usually we can't do it
612
00:44:33,843 --> 00:44:38,098
We can only solve that
if this vector b is what...
613
00:44:40,527 --> 00:44:49,581
I can solve that equation if that vector
[b1 b2 b3] is in the column space...
614
00:44:49,616 --> 00:44:52,577
if it's a combination of those
columns then fine
615
00:44:52,612 --> 00:44:54,307
But usually it won't be
616
00:44:55,092 --> 00:44:57,520
The combinations just fill up a plane
617
00:44:57,555 --> 00:44:59,900
and most vectors are not on that plane
618
00:45:00,738 --> 00:45:02,814
So what I'm saying is
619
00:45:02,849 --> 00:45:08,115
that I'm going to work with
the matrix A transpose A
620
00:45:08,150 --> 00:45:13,646
and I just want to figure out
in this example what A transpose A is
621
00:45:15,788 --> 00:45:17,259
So it's 2 by 2
622
00:45:17,294 --> 00:45:21,049
The first entry is 3 and
the next entry is 8
623
00:45:21,851 --> 00:45:23,446
This entry is...
624
00:45:26,021 --> 00:45:27,326
What's the entry...
625
00:45:28,974 --> 00:45:30,544
8, for sure
626
00:45:30,959 --> 00:45:32,487
We do it...has to be...
627
00:45:32,522 --> 00:45:38,600
And this entry is...
628
00:45:40,009 --> 00:45:41,635
30, is that right?
629
00:45:43,633 --> 00:45:48,582
And is that matrix invertible?
There's an A transpose A
630
00:45:49,201 --> 00:45:50,793
And it is invertible, right?
631
00:45:50,828 --> 00:45:55,656
3*8 is not a multiple of 8*30
and it's invertible
632
00:45:55,691 --> 00:45:59,142
And that's the normal..
that's what I expect
633
00:45:59,177 --> 00:46:04,046
So this is what I want to show
634
00:46:04,041 --> 00:46:07,165
So here's the key point
635
00:46:08,001 --> 00:46:11,912
The nullspace of A transpose A
636
00:46:12,809 --> 00:46:15,267
it's not gonna be always invertible
637
00:46:17,497 --> 00:46:20,535
Tell me a matrix...
yes...
638
00:46:20,570 --> 00:46:21,692
now I have to say that
639
00:46:21,727 --> 00:46:24,650
I can't say that A transpose A
is always invertible
640
00:46:24,685 --> 00:46:27,680
Cause that's asking too much
641
00:46:27,715 --> 00:46:30,777
I mean what could a matrix A be
for example
642
00:46:30,812 --> 00:46:34,139
so that A transpose A
was not invertible?
643
00:46:35,369 --> 00:46:37,900
Well even could be the zero matrix
644
00:46:37,935 --> 00:46:39,990
I mean that's like an extreme case
645
00:46:40,436 --> 00:46:44,982
Suppose I make this range...
646
00:46:46,710 --> 00:46:48,941
suppose I change to that A
647
00:46:54,308 --> 00:46:58,460
Now I figure out again
and I get, what do I get?
648
00:47:02,085 --> 00:47:03,243
I get 9
649
00:47:04,232 --> 00:47:05,768
I get 9 of course
650
00:47:05,803 --> 00:47:10,028
Then here I get ..
what's that entry?
651
00:47:10,894 --> 00:47:11,747
What is that?
652
00:47:15,062 --> 00:47:18,357
And is that matrix invertible?
No.
653
00:47:20,274 --> 00:47:22,766
And why... I'll do...
654
00:47:22,801 --> 00:47:24,371
it would be invertible any way
655
00:47:24,793 --> 00:47:30,485
Cause this matrix only has rank 1
656
00:47:31,208 --> 00:47:34,490
And if I have a product of
matrices of rank 1
657
00:47:34,525 --> 00:47:37,569
the product is not going to
have a rank bigger than 1
658
00:47:38,542 --> 00:47:42,239
So I'm not surprised that the
answer only has rank 1
659
00:47:42,274 --> 00:47:46,275
and that's what always happen
660
00:47:47,229 --> 00:47:51,562
That the rank of A transpose A
comes up equal to rank of A
661
00:47:52,216 --> 00:47:59,872
So the nullspace of
equals a nullspace of A
662
00:47:59,907 --> 00:48:07,937
The rank of A transpose A
equals the rank of A
663
00:48:10,234 --> 00:48:12,321
So let's..
664
00:48:14,594 --> 00:48:19,676
I'll tell you as soon as
I can why that's true
665
00:48:19,711 --> 00:48:27,013
But let's draw from that
was the fact that I want
666
00:48:27,048 --> 00:48:28,693
This tells me that
667
00:48:29,272 --> 00:48:34,323
this squared symmetric matrix
is invertible if…
668
00:48:35,081 --> 00:48:37,371
So here's my conclusion
669
00:48:37,406 --> 00:48:44,326
A transpose A is invertible if...
670
00:48:44,361 --> 00:48:46,539
exactly if...
671
00:48:50,892 --> 00:48:55,154
this nullspace is
only get to zero vector...
672
00:48:55,951 --> 00:49:00,145
which means the columns
of A are independent
673
00:49:00,180 --> 00:49:09,846
So A transpose A is invertible exactly
if A has independent columns
674
00:49:12,028 --> 00:49:20,995
That's the fact that I need
about A transpose A
675
00:49:21,030 --> 00:49:26,979
Then you'll see next time how
A transpose A enters everything
676
00:49:27,014 --> 00:49:30,433
Next lecture is actually a crucial one
677
00:49:30,468 --> 00:49:37,114
Here I'm preparing for it by
getting us thinking about A transpose A
678
00:49:37,149 --> 00:49:40,634
and its rank is the same as
the rank of A
679
00:49:40,669 --> 00:49:42,708
and we can decide
whether it's invertible
680
00:49:42,743 --> 00:49:44,903
OK, see you Friday
681
00:49:45,783 --> 00:49:46,670
Thanks
Last Modified 3/30/08 4:39 AM
|
