The Kronecker product has all kinds of interesting mathematical applications, but is also often extremely handy when creating certain kinds of patterned matrices in R or Matlab. Let’s say you are interested in measuring the smoothness of a function ), along the and direction. One measure is given by and . Let’s say that you sample along a regular grid. The derivatives can be approximated by a discrete differentation operator. If the sampled is stored as a matrix F (image-like storage), then 2nd order discrete differentiation along the columns can be done via:
D is a bunch of shifted copies of the operator . The product corresponds to , differentiated column-wise. Similarly, will be the row-differentiated version of . What we are interested in computing is the energy of and , which amounts to nothing more than taking each element, squaring it and taking the sum.
What if is not stored as a matrix, but as a vector ? The operator is mathematical notation for taking a matrix and turning it into a column vector by stacking the columns one below the other, i.e. what happens when you do in Matlab, or in R. We need an operator E that does the same thing to that D was doing to F, i.e. discrete differentiation along the axis.
Enters the Kronecker product, and this wonderful equality you’ll find for example in the Matrix cookbook:
stands for the Kronecker product here.
Taking , , and , we obtain:
is the operator we were looking for. This is extremely handy because such operators crop up a lot in Gauss-Markov random fields and penalised splines methods, and this gives a very fast way of creating the right precision matrices. More on Kronecker products in this article by C.F. van Loan.