Matrix Multiplication as Linear Combination

Matrix-Vector Multiplication

Suppose we have a matrix ARm×n and a column vector bRn. Then

Ab=b1a:,1++bna:,n.
matrix right-multiplied by a column vector

A matrix right-multiplied by a column vector gives a column vector, which is the linear combination of its columns.

Suppose we have a row vector cR1×m. Then

cA=c1a1++cmam.
matrix left-multiplied by a row vector

A matrix left-multiplied by a row vector gives a row vector, which is the linear combination of its rows.

Matrix-Matrix Multiplication

Suppose we have a matrix ARm×n, and BRn×r. We can see this multiplication in two ways:

AB=A[b:,1b:,r]=[Ab:,1Ab:,r]=[a1an]B=[a1BanB].
matrix-matrix multiplication

An m×n matrix right-multiplied by another n×r matrix gives an m×r matrix, where the j-th column is the linear combination of the columns of the multiplied matrix, weighted by the j-th column of the multiplying matrix.

An n×r matrix left-multiplied by another m×n matrix gives an m×r matrix, where the i-th row is the linear combination of the rows of the multiplied matrix, weighted by the i-th row of the multiplying matrix.

In one word, AB can be seen as the stacked linear combinations of columns of A, or as the stacked linear combination of rows of B.

Rank of Matrix Product

The rank of matrix A is the dimensionality of the linear space spanned by A's columns. Therefore, as AB's columns are the linear combination of A's columns, the dimensionality of the linear space spanned by them should be no more than rank(A):

rank(AB)rank(A);

similarly we have

rank(AB)rank(B).

Therefore

rank(AB)min{rank(A),rank(B)}.