User Guide¶
Select a NetCDF file with the data with gaps to be filled.
Optionally, select a NetCDF file with a land-see mask. If no mask is selected, all pixels are assumed to be ocean.
Set the maximum number of EOF modes to compute. Only the optimal number of EOFs + 3 will be computed for robust optimum detection. This number cannot be higher than the temporal size of the data minus 5. Start high and adjust down for subsequent reconstructions. If the max number of modes have been used, the algorithm probably needs more information.
Set the minimum number of EOF modes to compute. 1 is recommended.
Set the maximum size of the Krylov subspace. This number must be smaller or equal to the temporal size of teh data, and must be higher or equal to the maximum number of EOF modes plus 5.
Optionally, choose to reconstruct the entire image, and not only the missing data.
Optionally, save the left and right EOFs to file.
Optionally, normalize the input matrix for the multivariate case.
Optionally, choose a time vector for the diffusion of hte covariance matrix.
- The time file should also be in NetCDF file format, and specified as the relative increment in time between the individual images. For example [1,2,3,4,…] if all data is 1 day apart; [1,2,4,…] if day 3 is missing, …
- Optionally, set the strength of the filter.
- Optionally, set the number of iterations for the filter.
Set the advance functions, if you know what you are doing:
- You can set the maximum number of iterations for each EOF calculation. Use large umber and only increase if this number is reached for each EOF.
- You can change the threshold, for which EOF iterations stop, once rms(successive reconstruction) / std(existing data) is below this threshold.
- You can set the threshold for the Lanczos convergence. 1e-8 is a good number here.
Optionally, choose a folder for all results.
Once the processing is done, the the log tab in the widget shows as summary of the process. This summary can also be found in the file log.txt:
********************************************************************
Numerical data read
You entered the values:
number of EOF modes you want to compute 5
maximal size for the Krylov subspace 10
You asked not to normalise the input matrices
You asked not to write the left and right EOFs used for the reconstruction
...
********************************************************************
Now some statistics about your data:
Number of mask land points: 38315
Dimension of file 1: 301 x 201 x 10
Mean: 18.80
Standard deviation: 0.63
Size of the matrix used in DINEOF: 22186 x 10
Missing data: 100636 out of 221860 ( 45.36%)
Number of cross validation points 2258
********************************************************************
Time (in seconds) for 1 EOF mode calculation in DINEOF 0.0156
# EOF modes asked: 5 Convergence level required: 0.1E-02
EOF mode Expected Error Iterations made Convergence achieved
________ ______________ _______________ ____________________
1 0.4966 300 0.1237E-02
2 0.4755 96 0.9935E-03
3 0.5418 205 0.9989E-03
4 0.7587 300 0.1262E-02
5 0.8973 234 0.9967E-03
Minimum reached in cross-validation
Number of optimal EOF modes: 2
Make last reconstruction, including data put aside for cross-validation
2 0.4755 96 0.9935E-03
DINEOF finished!
number of eigenvalues retained for the reconstruction 2
expected error calculated by cross-validation 0.4755
total time (in seconds) in lanczos process 11.3438
SCE filled matrix (valex eliminated) = 173496.98260663458
mean 18.818932248247631
Standard deviation 0.88403924852814730 0.63180127935943464 18.796893489562994
...done!
The actual result, will be saved in the output folder, in a NetCDF file wit the same name as your original data.