MCTK User\'s Guide

The MODIS Conversion Toolkit (MCTK) User’s Guide Devin Alan White Technical Support Engineer / Instructor ITT Visual Information Solutions dwhite@ittvis.com The MODIS Conversion Toolkit User’s Guide 1 Table of Contents I. Introduction………………………………………………………………………………. 2 II. Installation………………………………………………………………………………... 2 III. Converting MODIS data interactively in ENVI using MCTK…………………….. 3 A. Level 1A Uncalibrated Radiance…………………………………………………. 4 B. Level 1B Calibrated Radiance…………………………………………………….. 5 C. Level 2 Swath………………………………………………………………………… 6 D. Level 2/3/4 Grid……………………………………………………………………... 7 IV. Converting MODIS data programmatically in ENVI using MCTK……………….. 10 A. API calling syntax………………………………………………………………… 11 B. Examples…………………………………………………………………………… 15 V. Supported MODIS products…………………………………………………………… 19 VI. MODIS Conversion Toolkit Processing Workflow Diagram…………………….. 25 The MODIS Conversion Toolkit User’s Guide 2 I. Introduction The MODIS Conversion Toolkit (MCTK) is an HDF file conversion and projection utility for all known MODIS data products. The plugin works on all operating systems that can run IDL and ENVI. A complete list of supported products can be found in Section V. MCTK allows you to take a cafeteria-style approach to working with MODIS data. Through an intuitive user interface, you can extract and project only the data you need. Full programmatic access to MCTK is also available (see Section IV for details). II. Installation To install MCTK, place the “modis_conversion_toolkit.sav” file in your ENVI save_add folder. The location of this folder will vary by operating system: Windows: c:\rsi\idl63\products\envi43\save_add UNIX\Linux: /usr/local/rsi/idl_6.3/products/envi_4.3/save_add Mac: /applications/rsi/idl_6.3/products/envi_4.3/save_add If the plugin is installed correctly, there should be a MODIS Conversion Toolkit button in the ENVI menu system under FileÆOpen External FileÆEOS the next time ENVI is started (Figure 1). Figure 1. The plugin creates a button in the EOS group in ENVI, allowing for easy access to data sets. The MODIS Conversion Toolkit User’s Guide 3 III. Converting MODIS data interactively in ENVI using MCTK To bring a MODIS file into MCTK, launch the tool from the Open External File menu and in the resulting window, click on Input HDF, and select the file you wish to input. By design, a file filter is preset so that only files that begin with “M” and have an “.hdf” suffix will be displayed in the file selection dialog. Only one file can be selected at a time for conversion. It is crucial that the name of your input file follows standard MODIS naming convention. The tool uses the first part of the file name (e.g., MOD02HKM) to determine the product type and appropriate processing options. The best way to ensure compliance with the naming convention is to retain the filenames assigned by your data provider. When a valid MODIS product is supplied to the tool, the various fields in the window will update (Figure 2). Directly below the Input HDF button is a field that will display metadata related to the inputted file. Georeferencing options for converted data will appear directly below the metadata viewer, when appropriate. The Processing Options area on the right side will provide you with an interactive list of what can be done with the inputted data. This list is different for every product. If georeferencing options are available for the inputted product, and one of them is chosen, the window will expand to include a section that gives you control over several parameters including projection type, resampling method, density of the triangulation grid (warp points), bow tie correction, and pixel size (when appropriate). Figure 2. A “blank” instance of MCTK. The various fields and sections will update with information and options that are relevant to the supplied MODIS product. To ensure that you have the most recent version of MCTK, check the displayed Build Date. To select where converted data should be placed, click on the Output Path button and select a folder. The folder where the input file resides is chosen by default. You must also provide a rootname for the outputted data. The string value you choose will be used as the first part of longer filenames automatically built by the tool during the conversion process. Since multiple The MODIS Conversion Toolkit User’s Guide 4 files can be generated by a single conversion process, it is useful to choose an intuitive and descriptive rootname so data from the same input file can be easily grouped afterwards. A. Level 1A Uncalibrated Radiance When a Level 1A Uncalibrated Radiance file is provided as input, you will be presented with several processing options. On the right side, you will see the four datasets stored inside of the HDF that contain image data at three spatial resolutions (250m, 500m, and 1000m). MCTK recognizes the fact that there are multiple resolutions within the file and displays them in the metadata window under Pixel Size. You can choose to output any combination of the available datasets, but each will be placed in a separate file due to how they are processed by the tool. Each output file will contain appropriate band number designations and wavelength values. After clicking on Process, you will see two ENVI progress bars: one related to the conversion of the current dataset and one related to the overall conversion process. The tool also recognizes that Level 1A data does not contain georeferencing information (indicated by the Native Projection being set to None in the metadata viewer). However, if you have access to the related Geolocation product (MOD03/MYD03), you can use it to project the Level 1A data. To do so, select Projected or Standard and Projected under Output Type and then click on Input Geolocation File. You will be prompted to supply an appropriate file. A filter is provided to aid you in finding the correct product and the current Level 1A filename will show up in the title bar of the file selection dialog. In order to proceed, the second, third and fourth sections of the Level 1A filename and geolocation filename have to match. For example, a Level 1A file with the name “MYD01.A2006007.0300.005.2007078081622.hdf” requires a geolocation file with the name “MYD03.A2006007.0300.005.2006125230829.hdf.” If the supplied filenames do not match, the projection options section will be greyed out and you will not be allowed to proceed with the conversion. If the filenames do match, the most appropriate UTM zone for the supplied data is automatically calculated and made available (Figure 3). Standard resampling methods (Nearest Neighbor, Bilinear, and Cubic Convolution) are available, as is Bow Tie Correction. If the latter is chosen, the former set of methods become inaccessible since only one can be used during projection. The number of warp points to use is set to 50 x 50 by default, which results in 2500 GCPs distributed evenly across the entire scene. While this should be more than enough to carry out an accurate triangulation warp, you can choose to use more or less points. This option is also not available when Bow Tie Correction is chosen due to differences in how this method uses the supplied geolocation file. Output pixel size is automatically determined based on product type and dataset. For projections that do not use meters, the meter-based pixel size is automatically converted to one that is appropriate. You also have the option of providing your own background value to use during the projection process. The default value is 0. If a value less than zero is provided, zero is used instead. Filenames for converted data For Level 1A files, two types of filenames are possible. They are constructed as follows: • Rootname + Dataset Name + Raw_DN.img (unprojected data) • Rootname + Dataset Name + Raw_DN_georef.img (projected data) A Note on Output Type Because MCTK is a conversion utility, you will always receive at least one ENVI format file as output. If you choose to output projected data using the Projected or Standard and Projected The MODIS Conversion Toolkit User’s Guide 5 option, two sets of output files will be generated. The first set is unprojected (Standard) and is used to create the second set (Projected). The difference between the two options is that when you choose Projected, the first set of files (Standard) is deleted once the projection process is complete. A Note on Using Bow Tie Correction with 250m Data In order to use Bow Tie Correction with 250m data, you must change your Image Tile Size in the ENVI Preferences dialog (under Miscellaneous) to a value of at least 5.0 MB. If this is not done, you will likely receive an error message stating that the interpolation method did not have enough data to successfully carry out the correction. Figure 3. MCTK with a Level 1A file supplied as input, along with a matching geolocation file. B. Level 1B Calibrated Radiance When a Level 1B Calibrated Radiance file is provided as input, you will be presented with several processing options that will vary depending on the product you are working with. In all cases, you will have the option to output projected data using the internal 1km-resolution geolocation fields in a fashion similar to that for Level 1A data (see above discussion). You do not need to supply a matching MOD03/MYD03 geolocation file. For a MOD02QKM file there is only one possible dataset to select for output, but multiple datasets are available for MOD02HKM, MOD021KM, and MOD02SSH. Output will always be consolidated into one file (before projection takes place) due to the fact that the spatial resolutions are the same for each dataset within a file. Appropriate band numbers and wavelengths will be attached to the output file as well. If you want to use Bow Tie Correction on MOD02QKM data, please refer to the above note regarding that correction method and 250m data. Apart from georeferencing and dataset selection options, MCTK also gives you the ability to choose which type of calibration to The MODIS Conversion Toolkit User’s Guide 6 perform on the stored data: radiance & emissivity or TOA reflectance & emissivity (Figure 4). Radiance is returned in units of W/m2/μm/sr, reflectance is returned as unitless values between 0.0 and 1.0. Calibration for emissivity data is only carried out on 1km products that contain an emissive dataset. You also have the option of providing your own background value to use during the projection process, which can be specified as an integer or floating point. The default value is NaN (not a number). Filenames for converted data For Level 1B files, two types of filenames are possible. They are constructed as follows: • Rootname + Calibration Method + .img (unprojected data) • Rootname + Calibration Method + georef.img (projected data) Figure 4. MCTK with a Level 1B file supplied as input (MOD021KM). Multiple datasets have been selected for georeferenced output, Reflectance / Emissivity has been chosen as the calibration method, and Bow Tie Correction has been turned on. C. Level 2 Swath When a Level 2 Swath file is provided as input, the available output options will be very similar to those for Level 1B files. However, there are a few important exceptions related to available datasets and converted output. As with Level 1A/B files, you will be presented with a list of datasets contained within the file that can be processed. The list is determined by scanning all datasets present in the file and determining which ones contain data that are stored in at least two dimensions (Figure 5). The dimensionality of each dataset appears to the left of the dataset name in the list (2D or 3D). If scale and offset factors are present for a particular dataset, they are automatically applied during the conversion process. Depending on the datasets chosen The MODIS Conversion Toolkit User’s Guide 7 and the number of spatial resolutions present, you may receive multiple output files. Datasets are grouped by spatial resolution and dimensionality prior to processing. For example, if you are working with MOD05_L2 (Precipitable Water), datasets exist at both 1000m and 5000m spatial resolutions and in two and three dimensions. All two dimensional data at 1000m are grouped into one multiband output file, where each band name is the name of an individual dataset. Each three dimensional dataset at 1000m receives its own output file. The process is then repeated for 5000m datasets. The filenames automatically generated by MCTK will reflect this process. You also have the option of providing your own background value to use during the projection process, which can be specified as an integer or floating point. The default value is NaN (not a number). Filenames for converted data For Level 2 Swath files, two types of filenames are possible. They are constructed as follows: • Rootname + Swath + Dimensionality + Resolution Index + Output File Number + .img (unprojected data) • Rootname + Swath + Dimensionality + Resolution Index + Output File Number + georef.img (projected data) where Dimensionality will be 2D or 3D, Resolution Index is an incremented value from 1 to the total number of spatial resolutions present (starting with the highest resolution), and Output File Number is an incremented value from 1 to the total number of files outputted for a particular dimensionality within a particular spatial resolution. There is currently no way within MCTK to tell which datasets will map to a particular spatial resolution prior to starting the conversion process. Figure 5. MCTK with a Level 2 Swath file supplied as input. Multiple datasets with different dimensionalities and spatial resolutions have been selected for processing. The MODIS Conversion Toolkit User’s Guide 8 D. Level 2/3/4 Grid When a Level 2G, Level 3 or Level 4 Grid file is provided as input, you may encounter an additional dialog which will prompt you to select which grid to process (Figure 6). While the occurrence of multiple grids within a file is rare, it can happen. Figure 6. When inputting a file that contains more than one grid, you will automatically be prompted to select which one to process. Once a grid is selected, the output options are almost identical to those for Level 2 Swath data. There are two major differences. First, grid data are already projected, so Standard output will use the native map information stored in the HDF file (usually Sinusoidal or Geographic Lat/Lon, but EASE Grid is also supported for NSIDC datasets). As a result of this fact, the usual Projected option is listed as Reprojected and gives you the chance to put your data in another map projection. This is a common practice with Sinusoidal data, which is generally put into Geographic Lat/Lon or UTM. When the Reprojected option is chosen, the map projection output options will show up with Geographic Lat/Lon chosen by default (Figure 7). A default output pixel size (in degrees) is also automatically calculated in the appropriate units, based on the input pixel size for the product (usually meters). Note that if the grid’s native projection is Geographic Lat/Lon, that projection is still the default Reprojection option. Choosing to reproject using the default values in this case will essentially create the same output as choosing Standard. A useful thing to do in this situation, since most Geographic Lat/Lon data is global in scale, would be to choose a larger output pixel size, which in essence shrinks the size of the resulting output file but retains the same projection as the original input. You also have the option of providing your own background value to use during the reprojection process, which can be specified as an integer or floating point. The default value is NaN (not a number). The MODIS Conversion Toolkit User’s Guide 9 Figure 7. MCTK with a Level 4 Grid file supplied as input. Multiple datasets with different dimensionalities have been selected for processing and the reprojection option is set for Geographic Lat/Lon. Filenames for converted data For Level 2G, Level 3 and Level 4 Grid files, four types of filenames are possible. They are constructed as follows: • Rootname + Grid + Dimensionality + Output File Number + .img (unprojected 2D or 3D data) • Rootname + Swath + Dimensionality + Output File Number + reprojected.img (projected 2D or 3D data) • Rootname + Grid + Dimensionality + Dataset Index + Dimension Index + .img (unprojected 4D data) • Rootname + Swath + Dimensionality + Dataset Index + Dimension Index + reprojected.img (projected 4D data) where Dimensionality will be 2D, 3D, or 4D; Dimension Index is an incremented value from 1 to the total number of “bands” in the fourth dimension (each one maps to a 3D data cube); Output File Number is an incremented value from 1 to the total number of files outputted for 2D and 3D datasets; and Dataset Index is an incremented value from 1 to the total number of 4D datasets chosen for output. The naming convention for 4D output is potentially headache-inducing due to the complexities of the HDF file structure for that type of data, so an example would be as follows. If you inputted a MOD43B3 file, selected both the 4D “Albedo” dataset and 3D “Albedo_Quality” dataset for conversion, provided “albedo” as your rootname, and opted to reproject the converted data, the resulting files would look like this: albedo_Grid_4D_1_1_reprojected.img (first 4D dataset, first 3D cube of data) albedo_Grid_4D_1_2_reprojected.img (first 4D dataset, second 3D cube of data) The MODIS Conversion Toolkit User’s Guide 10 albedo_Grid_3D_1_reprojected.img (first 3D dataset) Since ENVI cannot handle 4D datasets, they are broken down into more manageable 3D cubes. In this example, the original 4D dataset had dimensions of 1200x1200x10x2, so two 3D datasets were generated—each one consisting of a 1200x1200x10 cube. A 4D dataset with dimensions of 1200x1200x10x20 would result in 20 3D cubes (and 20 output files). IV. Converting MODIS data programmatically in ENVI using MCTK All of MCTK’s functionality is programmatically available. The interactive version is simply a way to gather information about the inputted file and assist the user with selecting the data they want to output and how the output should be handled. Once all of this information is collected by the widget, it is handed off to the same processing routine discussed here (CONVERT_MODIS_DATA). The processing routine can be called like any other IDL procedure as long as convert_modis_data.sav is in your ENVI save_add folder. There are many keywords associated with the procedure, but not all of them are required for every type of MODIS data. Consult the keyword definitions and example programs on the following pages to determine which ones you will need to use. Multiple file conversion must be carried out one file at a time, so running CONVERT_MODIS_DATA on multiple files will require the construction of a FOR loop that passes each file of interest into the routine sequentially. The MODIS Conversion Toolkit User’s Guide 11 CONVERT_MODIS_DATA Syntax CONVERT_MODIS_DATA [,IN_FILE=string] [,OUT_PATH=string] [,OUT_ROOT=string] [,/L1A] [,/L1B] [,/MOD14] [,/HIGHER_PRODUCT] [,/SWATH] [,/GRID] [,SWT_NAME=string] [,GD_NAME=string] [,SD_NAMES=string array]