The Kronecker product has all kinds of interesting mathematical applications, but is also often extremely handy when creating certain kinds of patterned matrices in R or Matlab. Let’s say you are interested in measuring the smoothness of a function ), along the
and
direction. One measure is given by
and
. Let’s say that you sample
along a regular grid. The derivatives can be approximated by a discrete differentation operator. If the sampled
is stored as a matrix F (image-like storage), then 2nd order discrete differentiation along the columns can be done via:
D is a bunch of shifted copies of the operator . The product
corresponds to
, differentiated column-wise. Similarly,
will be the row-differentiated version of
. What we are interested in computing is the energy of
and
, which amounts to nothing more than taking each element, squaring it and taking the sum.
What if is not stored as a matrix, but as a vector
? The operator
is mathematical notation for taking a matrix and turning it into a column vector by stacking the columns one below the other, i.e. what happens when you do
in Matlab, or
in R. We need an operator E that does the same thing to
that D was doing to F, i.e. discrete differentiation along the
axis.
Enters the Kronecker product, and this wonderful equality you’ll find for example in the Matrix cookbook:
stands for the Kronecker product here.
Taking ,
, and
, we obtain:
is the operator we were looking for. This is extremely handy because such operators crop up a lot in Gauss-Markov random fields and penalised splines methods, and this gives a very fast way of creating the right precision matrices. More on Kronecker products in this article by C.F. van Loan.