This special subset of Empirical Orthogonal Functions (EOF)
can be determined by performing a large Principal Component Analysis of
all the residuals of a large regression model
after having first orthonormalized them separately within every
block k (k = 1, 2,..., K).
This computational procedure was originally developed for computing
so-called Canonical Correlations i.e. the correlations between Common Factors
of only two blocks (K = 2).
It was described (in Finnish only, sorry) in report No. 1/1969 of
the Computing Centre of the University of Jyväskylä under title
"KANONISEN ANALYYSIN LASKENNOISTA" by
Lange (1969a):

page 1,
page 2,
page 3,
page 4,
page 5,
page 6,
page 7,
page 8,
page 9,
page 10,
page 11,
page 12,
page 13,
page 14,
page 15,
page 16,...
from file
Lange1969a.zip .

If the large covariace matrix of all the residuals is singular then the number K of the blocks must be reduced. This can usually be achieved by clustering of the blocks into a smaller number of blocks using results e.g. from a Discriminant Analysis, see report No. 2/1969 of the Computing Centre of the University of Jyväskylä under title "EROTTELUSTA JA LUOKITTELUSTA" Lange1969b.zip.