User Guide

  1. Select a NetCDF file with the data with gaps to be filled.

  2. Optionally, select a NetCDF file with a land-see mask. If no mask is selected, all pixels are assumed to be ocean.

  3. Set the maximum number of EOF modes to compute. Only the optimal number of EOFs + 3 will be computed for robust optimum detection. This number cannot be higher than the temporal size of the data minus 5. Start high and adjust down for subsequent reconstructions. If the max number of modes have been used, the algorithm probably needs more information.

  4. Set the minimum number of EOF modes to compute. 1 is recommended.

  5. Set the maximum size of the Krylov subspace. This number must be smaller or equal to the temporal size of teh data, and must be higher or equal to the maximum number of EOF modes plus 5.

  6. Optionally, choose to reconstruct the entire image, and not only the missing data.

  7. Optionally, save the left and right EOFs to file.

  8. Optionally, normalize the input matrix for the multivariate case.

  9. Optionally, choose a time vector for the diffusion of hte covariance matrix.

    1. The time file should also be in NetCDF file format, and specified as the relative increment in time between the individual images. For example [1,2,3,4,…] if all data is 1 day apart; [1,2,4,…] if day 3 is missing, …
    2. Optionally, set the strength of the filter.
    3. Optionally, set the number of iterations for the filter.
  10. Set the advance functions, if you know what you are doing:

    1. You can set the maximum number of iterations for each EOF calculation. Use large umber and only increase if this number is reached for each EOF.
    2. You can change the threshold, for which EOF iterations stop, once rms(successive reconstruction) / std(existing data) is below this threshold.
    3. You can set the threshold for the Lanczos convergence. 1e-8 is a good number here.
  11. Optionally, choose a folder for all results.

Once the processing is done, the the log tab in the widget shows as summary of the process. This summary can also be found in the file log.txt:

Numerical data read
You entered the values:
number of EOF modes you want to compute        5
maximal size for the Krylov subspace          10

You asked not to normalise the input matrices
You asked not to write the left and right EOFs used for the reconstruction

Now some statistics about your data:

Number of mask land points:                                    38315
       Dimension of file  1:                   301  x   201  x    10
                       Mean:                                   18.80
         Standard deviation:                                    0.63

Size of the matrix used in DINEOF:                    22186 x     10

Missing data:                    100636   out of    221860 ( 45.36%)

Number of cross validation points                               2258

Time (in seconds) for 1 EOF mode calculation in DINEOF    0.0156

# EOF modes asked:   5        Convergence level required: 0.1E-02

EOF mode    Expected Error    Iterations made   Convergence achieved
________    ______________    _______________   ____________________

     1              0.4966                300             0.1237E-02
     2              0.4755                 96             0.9935E-03
     3              0.5418                205             0.9989E-03
     4              0.7587                300             0.1262E-02
     5              0.8973                234             0.9967E-03

Minimum reached in cross-validation
Number of optimal EOF modes:            2

Make last reconstruction, including data put aside for cross-validation

     2              0.4755                 96             0.9935E-03

DINEOF finished!

number of eigenvalues retained for the reconstruction   2
        expected error calculated by cross-validation   0.4755
           total time (in seconds) in lanczos process   11.3438

SCE filled matrix (valex eliminated) = 173496.98260663458

mean    18.818932248247631
Standard deviation  0.88403924852814730       0.63180127935943464        18.796893489562994


The actual result, will be saved in the output folder, in a NetCDF file wit the same name as your original data.