Processing and Assessment of Spectrometric, Stereoscopic Imagery Collected Using a Lightweight UAV Spectral Camera for Precision Agriculture

Abstract

Imaging using lightweight, unmanned airborne vehicles (UAVs) is one of the most rapidly developing fields in remote sensing technology. The new, tunable, Fabry-Perot interferometer-based (FPI) spectral camera, which weighs less than 700 g, makes it possible to collect spectrometric image blocks with stereoscopic overlaps using light-weight UAV platforms. This new technology is highly relevant, because it opens up new possibilities for measuring and monitoring the environment, which is becoming increasingly important for many environmental challenges. Our objectives were to investigate the processing and use of this new type of image data in precision agriculture. We developed the entire processing chain from raw images up to georeferenced reflectance images, digital surface models and biomass estimates. The processing integrates photogrammetric and quantitative remote sensing approaches. We carried out an empirical assessment using FPI spectral imagery collected at an agricultural wheat test site in the summer of 2012. Poor weather conditions during the campaign complicated the data processing, but this is one of the challenges that are faced in operational applications. The results indicated that the camera performed consistently and that the data processing was consistent, as well. During the agricultural experiments, promising results were obtained for biomass estimation when the spectral data was used and when an appropriate radiometric correction was applied to the data. Our results showed that the new FPI technology has a great potential in precision agriculture and indicated many possible future research topics.

1. Introduction

Modern airborne imaging technology based on unmanned airborne vehicles (UAVs) offers unprecedented possibilities for measuring our environment. For many applications, UAV-based airborne methods offer the possibility for cost-efficient data collection with the desired spatial and temporal resolutions. An important advantage of UAV-based technology is that the remote sensing data can be collected even under poor imaging conditions, that is, under cloud cover, which makes it truly operational in a wide range of environmental measuring applications.

We focus here on lightweight systems, which is one of the most rapidly growing fields in UAV technology. The systems are quite competitive in local area applications and especially if repetitive data collection or a rapid response is needed.

An appropriate sensor is a fundamental component of a UAV imaging system. The first operational, civil, lightweight UAV imaging systems typically used commercial video cameras or customer still cameras operating in selected three wide-bandwidth bands in red, green, blue and/or near-infrared spectral regions [1–3]. The recent sensor developments tailored for operation from UAVs offer enhanced possibilities for remote sensing applications in terms of better image quality, multi-spectral, hyper-spectral and thermal imaging [4–10] and laser scanning [11–13].

One interesting new sensor is a lightweight spectral camera developed by the VTT Technical Research Center of Finland (VTT). The camera is based on a piezo-actuated, Fabry-Perot interferometer (FPI) with an adjustable air gap [6,14]. This technology makes it possible to manufacture a lightweight spectral imager that can provide flexibly selectable spectral bands in a wavelength range of 400–1,000 nm. Furthermore, because the sensor produces images in a frame format, 3D information can be extracted if the images are collected with stereoscopic overlaps. In comparison to pushbroom imaging [7,9], the advantages of frame imaging include the possibility to collect image blocks with stereoscopic overlaps and the geometric and radiometric constraints provided by the rigid rectangular image geometry and multiple overlapping images. We think that this is important in particular for UAV applications, which typically utilize images collected under dynamic, vibrating and turbulent conditions.

Conventional photogrammetric and remote sensing processing methods are not directly applicable for typical, small-format UAV imagery, because they have been developed for more stable data and images with a much larger spatial extent than what can be obtained with typical UAV imaging systems. With UAV-based, small-format frame imaging, a large number—hundreds or even thousands—of overlapping images are needed to cover the desired object area. Systems are often operated under suboptimal conditions, such as below full or partial cloud cover. Despite the challenging conditions, the images must be processed accurately so that object characteristics can be interpreted on a quantitative geometric and radiometric basis using the data.

Precision agriculture is one of the potential applications for hyperspectral UAV imaging [1,2,4,6,9,15–18]. In precision agriculture, the major objectives are to enable efficient use of resources, protection of the environment and documentation of applied management treatments by applying machine guidance and site-specific seeding, fertilization and plant protection. The expectation is that UAVs might provide an efficient remote sensing tool for these tasks [18]. A review by Zhang and Kovacs [16] showed that research is needed on many topics in order to develop efficient UAV-based methods for precision agriculture. In this study, we will demonstrate the use of the FPI spectral camera in a biomass estimation process for wheat crops; biomass is one of the central biophysical parameters to be estimated in precision agriculture [17].

The objectives of this investigation were to investigate a complete processing methodology for the FPI spectral imagery, as well as to demonstrate its potential in a biomass estimation process for precision agriculture. We depict a method for FPI image data processing in Section 2. We describe the test setup used for the empirical investigation in Section 3. We present the empirical results in Section 4 and discuss them in more detail in Section 5.

2. A Method for Processing FPI Spectral Data Cubes

2.1. An FPI-Based Spectral Camera

The FPI-based spectral camera developed by the VTT provides a new way to collect spectrometric image blocks. The imager is based on the use of multiple orders of the Fabry-Perot interferometer together with the different spectral sensitivities of the red, green and blue pixels of the image sensor. With this arrangement, it is possible to capture three wavelength bands with a single exposure. When the FPI is placed in front of the sensor, the spectral sensitivity of each pixel is a function of the interferometer air gap. By changing the air gap, it is possible to acquire a new set of wavelengths. With smaller air gaps, it is also possible to capture only one or two wavelengths in each image. Separate short-pass and long-pass filters are needed to cut out unwanted transmissions at unused orders of the Fabry-Perot interferometer. During a flight, a predefined sequence of air gap values is applied using the FPI camera to reconstruct the spectrum for each pixel in the image (see more details in Section 2.3.3). Often, 24 different air gap values are used, and by these means, it is possible to collect 24 freely selectable spectral bands in a single flight, while the rest of the bands (0–48) are not independent. The desired spectral bands can be selected with a spectral step of 1 nm. This technology provides spectral data cubes with a rectangular image format, but each band in the data cube exposed to a different air gap value has a slightly different position and orientation. The principles of the FPI spectral camera have been described by Saari et al. [6] and Mäkynen et al. [14]. Results presented by Nackaerts et al. [19] demonstrated the feasibility of the sensor concept. Honkavaara et al. [20] developed the first photogrammetric processing line for the FPI spectral camera, and Pölönen et al. [21] carried out the first performance assessments in precision agriculture using the 2011 prototype.

The 2012 prototype was used in this investigation. It is equipped with custom optics with a focal length of 10.9 mm and an f-number of less than 3.0. The camera has a CMOSIS CMV4000 Complementary Metal Oxide Semiconductor (CMOS) red, green and blue (RGB) image sensor with an electronic shutter; the infrared cut filter has been removed from the image sensor. The sensor has a 2,048 × 2,048 pixel resolution, a pixel size of 5.5 μm and a radiometric resolution of 12 bits. In practical applications, the sensor is used in the two-times binned mode, while only part of the sensor area is used. This provides an image size of 1,024 × 648 pixels with a pixel size of 11 μm. The field of view (FOV) is ±18° in the flight direction, ±27° in the cross-flight direction and ±31° at the format corner. Application-based filters can be used, for example 500–900, 450–700, 600–1,000 or 400–500 nm filters. The spectral resolution range is 10–40 nm at the full width at half maximum (FWHM), and it is dependent on the FPI air gap value, as well as the filter selection.

Table 1. Spectral imager specifications for 2011 and 2012 prototypes. FOV, field of view; FWHM, the full width at half maximum; FPI, Fabry-Perot interferometer.

Parameter	Prototype 2011	Prototype 2012
Horizontal and vertical FOV (°)	>36, >26	>50, >37
Nominal focal length (mm)	9.3	10.9
Wavelength range (nm)	400–900	400–900
Spectral resolution at FWHM (nm); depending on the selection of the FPI air gap value	9–45	10–40
Spectral step (nm): adjustable by controlling the air gap of the FPI	<1	<1
f-number	<7	<3
Pixel size (μm); no binning/default binning	2.2/8.8	5.5/11
Maximum spectral image size (pixels)	2,592 × 1,944	2,048 × 2,048
Spectral image size with default binning (pixels)	640 × 480	1,024 × 648
Camera dimensions (mm)	65 × 65 × 130	<80 × 92 × 150
Weight (g); including battery, GPS receiver, downwelling irradiance sensors and cabling	<420	<700

shows the differences between the 2011 prototype and the 2012 prototype. Many of the parameters were improved for the 2012 prototype. The most significant improvement was the improvement of the f-number in order to improve the signal-to-noise ratio (SNR). This was achieved by improving the lens system and allowing for greater FPI ray angles (10° in comparison to 4°). The blurring of images was reduced by changing the rolling shutter to an electronic shutter.

The sensor setup is shown in Figure 1. The entire imaging system includes the FPI spectral camera, a 32 GB compact flash memory card, irradiance sensors for measuring downwelling and upwelling irradiance (the irradiance sensor for measuring upwelling irradiance is not operational in the current setup), a GPS receiver and a lithium polymer (LiPo) battery. The system weighs less than 700 g. With this setup, more than 1,000 data cubes, each with up to 48 bands, can be collected in the two-times binned mode within a single flight.

2.2. FPI Spectral Camera Data Processing

The FPI spectral camera data can be processed in a similar manner as small-format frame imagery, but some sensor-specific processing is also required. Figure 2 shows a general data processing chain for FPI spectral image data. The processing chain includes data collection, FPI spectral data cube generation, image orientation, digital surface model (DSM) calculations, radiometric model calculations and output product generation. When developing the data processing chain for the FPI spectral camera, our objective has been to integrate the sensor-specific processing steps into our existing processing line based on commercially available photogrammetric and remote sensing software.

Data collection constitutes the first phase in the imaging chain. The central parameters that need to be set are the spectral sensitivities of the bands and the integration time. The spectral sensitivities are selected based on the requirements of the application. The integration time should be selected so that the images are not overexposed in relation to the bright objects and that there are good dynamics for the dark objects. Furthermore, the flight speed and flying altitude will impact the data quality (Section 2.3.3).

The pre-processing phase requires that the sensor imagery be accurately corrected radiometrically based on laboratory calibrations [7,8,14]. Correcting the spectral smile and mismatch of different spectral bands requires sensor-specific processing, which are described in more detail in Section 2.3.

The objective of geometric processing is to obtain non-distorted 3D data for a desired coordinate system. The geometric processing steps involve determining the image orientations and DSM generation. An important recent advancement in photogrammetric processing is the new generation of image matching methods for DSM measurement, which is an important advantage for the image analysis process when using UAVs [20,22–27].

The objective of radiometric processing is to provide information about the object’s reflectivity [28]. With this process, all of the disturbances related to the imaging system, atmospheric influences and view/illumination-related factors need to be compensated for. We describe our approaches to radiometric processing in Section 2.4.

The outputs resulting from the above-mentioned process include georeferenced reflectance image mosaics and 3D products, such as point clouds, point clouds with reflectivity information, DSMs, object models and stereomodels for visual evaluations [20].

2.3. FPI Spectral Data Cube Generation

A crucial step in the data processing chain is the construction of radiometrically and geometrically consistent spectral data cubes. The process includes three major phases: radiometric image corrections based on laboratory calibrations, spectral smile corrections and band matching.

2.3.1. Radiometric Correction Based on Laboratory Calibration

The first steps in the data processing chain involve dark signal compensation and applying the sensor calibration information to the images with the usual corrections for photon response nonuniformity (PRNU) (including the CMOS array nonuniformity and lens falloff corrections). These parameters are determined in the VTT’s calibration laboratory. Dark signal compensation is carried out using a dark image collected before the image data collection. A Bayer-matrix reconstruction is carried out to obtain three-band images. The process has been described by Mäkynen et al. [14], and it will not be emphasized in this investigation.

2.3.2. Correction of the Spectral Smile

Ideally, the optics of the FPI spectral camera are designed so that light rays go through the FPI as a collimated beam for a specific pixel of the image. In order to improve the f-number, this requirement was compromised, and a maximum FPI ray angle of 10° was allowed. This will cause a shift in the central wavelength of the FPI’s spectral peak (a spectral smile effect). For the most part, the peak wavelength (λ0) is linearly dependent on the cosine of the ray angle (θ) at the image plane:

$$\mathrm{\lambda }0=2\cdot \text{d}\cdot \text{cos}(\mathrm{\theta })/\text{m},\text{m}=1,2,\dots ,$$

(1)

where d is the air gap between the Fabry-Perot interferometer mirrors and m is the interference order at which the FPI is used.

There are different approaches to handling the spectral smile:

• Correcting images: Our assumption is that we can calculate smile-corrected images so that the corrected spectrums can be resampled from two spectrally (with a difference in peak wavelength preferably less than 10 nm) and temporally (with a spatial displacement less than 20 pixels) adjacent image bands.
• Using the central areas of the images: When images are collected with a minimum of 60% forward and side overlaps and when the most nadir parts of images are used, the smile effect is less than 5 nm and can be ignored in most applications (when the FWHM is 10–40 nm).
• Resampling a “super spectrum” for each object point of the overlapping images providing variable central wavelengths. The entire spectrum can be utilized in the applications.

For typical remote sensing applications and software, approaches 1 and 2, are the most functional, because they can be carried out in a separate step during the preprocessing phase. In order to utilize the entire extent of the images, approaches 1 or 3 are required. The VTT has developed a method for correcting the spectral smile by using two spatially and spectrally adjacent bands (approach 1). A correlation-based matching using shifts in the row and column directions is first carried out on the images to accurately align two of the bands. The corrected image is then formed by interpolating the desired wavelength from the adjacent bands. This simple approach is considered sufficient, because the bands that need to be matched are spectrally and spatially close to one another.

2.3.3. Band Matching

The bands of a single data cube image do not overlap perfectly, due to the imaging principle of the FPI spectral camera (Section 2.1). This makes it challenging to determine the orientation of the bands, because, in principle, all of the bands and images have a different orientation.

Exposing and transferring a single band to a synchronous dynamic random-access memory (SDRAM) takes approximately 75 ms, which is the time difference between two temporally adjacent bands. Recording a single cube, for example, with 24 air gaps, takes a total of 1,800 ms. The resulting spatial shift is dependent on the flight speed of the UAV, and the shift in pixel coordinates is determined by the flying height of the system. For example, with a flight speed of 5 m·s⁻¹ and a continuous data collection mode (not stopping during image exposure), the horizontal transition of the UAV in the flight direction between temporally adjacent bands is 0.375 m, whereas it is 9 m for the entire data cube; with a flying altitude of 150 m (ground sample distance (GSD), 15 cm), the spatial displacements are thus 2.5 and 60 pixels, respectively. The UAV is also swinging and vibrating during the time of the data cube exposure, which generates nondeterministic positional and rotational mismatches for the bands. Due to these small random movements, the a priori information has to be improved by using data-based analysis.

Two different approaches for processing the FPI image data are as follows:

• Determining the orientations of and georeferencing the individual bands separately. With this approach, there are number-of-bands (typically 20–48) image blocks that need to be processed. We used this approach in our investigation with the 2011 camera prototype, where we processed five bands [20,29].
• Sampling the bands of individual data cubes in relation to the geometry of a reference band. Orientations are determined for the image block with reference band images, and this orientation information is then applied to all other bands. We studied this approach for this investigation.

The principle aspect of approach 2 involves using one of the bands as the reference band and matching all of the other bands to it. In order to transform a band to match the geometry of the reference band, a geometric image transformation must be carried out. The parameters of the transformation are determined by utilizing tie points that are automatically measured between the bands via image matching. There are several challenges in this matching process. A scene provides different digital numbers (DN) in different spectral bands according to the spectral responses of the different objects, which complicates the image matching process. Another challenge is that, with dynamic UAV imaging, the overlaps between the different bands can be quite small. With 3D objects, a further complication is that a simple 2D modeling of the object’s geometry is not likely to provide accurate results; with homogeneous objects, the images might not contain sufficient features for matching, and repetitive features, such as those seen in crops, can cause false matches.

Our current implementation combines the above two approaches. We use a few reference bands and then match several adjacent bands to these bands. By using several reference bands, we try to improve the accuracy and robustness of the image transformations. For example, we can select reference bands for each major wavelength region (blue, green, red, near-infrared (NIR)) or we could select reference bands to maximize the spatial adjacency of the bands to be matched together. We also avoid georeferencing all layers separately, which could slow down the processing in general and be too laborious in the case of challenging objects that require manual interactions. For the first implementation, we employed feature-based matching (FBM) for the band pairs using point features extracted via the Förstner operator [30]. Finally, an image transformation is carried out using an appropriate geometric model; in our system, an affine or a projective model is used.

2.4. Radiometric Correction of Frame Image Block Data

Our experience is that UAV remote sensing data often need to be collected under sub-optimal illumination conditions, such as varying degrees of cloudiness. The possibility to collect data below cloud cover and under difficult conditions is also one of the major advantages of the UAV technology. We have developed two approaches for the radiometric processing of UAV image blocks in order to produce homogeneous data for non-homogeneous input data.

The first approach is an image-information-based radiometric block adjustment method [20,29]. The basic principle of the approach is to use the gray values (digital numbers (DNs)) of the radiometric tie points in the overlapping images as observations and to determine the parameters of the radiometric model indirectly via the least squares principle. Currently, we use the following model for a gray value (DN):

$$D{N}_{jk}=a_{rel\_j}\cdot (a_{abs}\cdot R_{jk}({\theta }_i,{\theta }_r,\phi )+b_{abs})+b_{rel\_j}$$

(2)

where R_jk(θ_i,θ_r,φ) is the bi-directional reflectance factor (BRF) of the object point, k, in image j; θ_i and θ_r are the illumination and reflected light (observation) zenith angles, φ_i and φ_r are the azimuth angles, respectively, and φ = φ_r− φ_i is the relative azimuth angle; a_abs and b_abs are the parameters for the empirical line model for transforming the reflectance to DN and a_{rel_j} and b_{rel_j} are relative correction parameters with respect to the reference image.

The impact of view/illumination geometry on the object reflectance is an important challenge in passive imaging. Our approach is to use a model-based, multiplicative, anisotropy factor. A reflectance, R_jk(θ_i,θ_r,φ), of the object point, k, can be given as follows:

$$R_{jk}({\theta }_i,{\theta }_r,\phi )=R_k(0,0,\phi )\cdot {\rho }_{\mathit{modeled}}({\theta }_i,{\theta }_r,\phi )/{\rho }_{\mathit{modeled}}(0,0,\phi )$$

(3)

where R_k(0,0,φ) is the reflectance factor at the nadir viewing geometry, ρ_modeled(θ_i,θ_r,φ) is the bi-directional reflectance distribution function (BRDF) for the specific view/illumination geometry and ρ_modeled(0,0,φ) is the modeled reflectance in a desired nadir viewing geometry (see, also, [31]). We use the simple BRDF model developed by Walthall et al. [32]. The final BRF with coefficients a’ and b’ for the desired reflected geometry of θ_r= 0 is:

$$R_{jk}({\theta }_i,{\theta }_r,\phi )=R_k(0,0,\phi )(a\text{'}{\theta }_r^2+b\text{'}{\theta }_r\mathit{cos}\hspace{0.17em}\phi +1)$$

(4)

In the radiometric model, the absolute reflectance transformation parameters eliminate atmospheric influences. The relative additive and multiplicative parameters eliminate differences between the images that are mainly due to illumination changes and sensor instability. The BRDF model takes care of the view/illumination geometry-related issues. The numbers for the parameters are as follows: absolute calibration: 2; relative calibration: (number of images − 1) × 2; BRDF model: 2 (if the same model is used for the entire object); nadir reflectance of radiometric tie points: number of radiometric tie points. The parameters that are used depend on the conditions during imaging. This model includes many simplifications, but it can be extended using physical parameters. For this approach, a minimum of two reflectance reference targets are needed. With the current implementation, the parameters are determined separately for each band. During the image correction phase, R_k(0,0,φ) is calculated based on Equations (2) and (4).

A second alternative is to utilize the irradiance measurements collected during the flight campaign, as described by Hakala et al. [33]. In this case, the relative adjustment of the images is obtained by selecting one reference image and calculating the relative multiplicative correction factors, C_j(λ), with respect to it:

$$C_j(\lambda )=E_{ref}(\lambda )/E_j(\lambda )$$

(5)

where E_j(λ) and E_ref(λ) are the spectral irradiance measurements during the acquisition of image j and reference image ref. When multiplying the DNs of image j using C_j(λ), the radiometric differences caused by changes in the irradiance are eliminated. Our analysis, presented in another article [33], showed that this model is functional under certain conditions. If the assumptions are not valid, the accuracy of the correction will be reduced.

3. Empirical Investigation

3.1. Test Area

An empirical campaign was carried out at the MTT Agrifood Research Finland (MTT) agricultural test site in Vihti (60°25′21″N, 24°22′28″E) (Figure 3). The test area, a 76 m by 385 m (2.9 ha) patch of land, has a rather flat topography with a terrain height variation of 11 m and maximum slopes of 3.5°. The area consisted of test plots that contained both wheat and barley. The seed and fertilizer amounts were varied to cause a large range of variation in the vegetation; the applied values were determined by applying farming knowledge as the average input for the existing conditions. The seeds were applied at various rates ranging from (0.5 × standard density)/m² to (1.5 × standard density)/m² using a Junkkari combi drill. The amount of nitrogen fertilizer varied in the range of 0 kg·ha⁻¹ to 180 kg·ha⁻¹ for the different varieties of wheat and barley, which consisted of wheat Anniina, wheat Kruunu, barley Voitto and barley Saana. Every fourth fertilization stripe contained spraying tracks. Figure 3 shows the spring fertilization plan and seeding plan for the agricultural test area. The image on the left gives information about the spring fertilization plan, while the image on the right shows the seeding plan.

Altogether, there were 50 vegetation samples with a size of 1 m × 1 m (Figure 4a (left)). The growing season measurements included the dry biomass, the wetness percentage and the nitrogen content. The samples had quite evenly distributed biomass variation in the range of 500–2,700 kg·ha⁻¹; in addition, we took three samples in non-vegetated areas (Figure 4b).

We used three 5 m by 5 m reflectance reference tarps (P05, P20 and P30, with a nominal reflectance of 0.05, 0.2 and 0.3, respectively) to determine the transformation between the DNs and the reflectance. We used the reflectance values that were measured in the laboratory using the ASD Field Spec Pro FR spectroradiometer (Analytical Spectral Devices Inc., Boulder, CO, USA), and the measurements were normalized to a calibrated white, 30 cm by 30 cm Labsphere Spectralon reference standard. In situ reflectance measurements were not carried out, due to extremely varying imaging conditions.

There were altogether 11 targeted XYZ ground control points (GCPs), the coordinates of which were measured using the virtual reference station real time kinematic GPS (VRS-RTK) method (with a relative accuracy of 2 cm) and 13 natural XYZ GCPs, which were measured based on a previous UAV orthophoto and DSM (with a relative accuracy of 20 cm) (Figure 4).

We had a reference DSM with a point interval of 10 cm and an approximated height accuracy of 20 cm. It was produced by automatic image matching using higher spatial resolution UAV imagery (GSD 5 cm; green, red and NIR bands with a spectral bandwidth of 50 nm at the FWHM), which was collected on the same day as the FPI spectral camera imagery. This DSM was used as the reference to evaluate the height accuracy of the DSM measured using the FPI spectral camera imagery. We used the national airborne laser scanning (ALS) DSM as the bare ground season information. By calculating the difference between the DSM provided using the imagery with vegetation and the ALS DSM, we could estimate the height of the crop. The minimum point density of the national ALS data is half a point per square meter, while the elevation accuracy of the points in well-defined surfaces is 15 cm and the horizontal accuracy is 60 cm, making this data an appropriate reference surface for an agricultural application.

3.2. Flight Campaigns

An image block was collected with the FPI spectral camera using a single-rotor UAV helicopter based on Mikado Logo 600 mechanics with a 5 kg payload capacity [34]. A preprogrammed flight path was flown autonomously using autopilot DJI ACE Waypoint. The FPI spectral camera was fixed to the bottom of the UAV (Figure 5a). The FPI spectral camera was equipped with a 500–900 nm filter, which is considered appropriate for an agricultural application. The camera was operated in free running mode and took spectral data cubes at the given intervals; the integration time was 5 ms. A data cube with 24 different air gap values and 42 bands was collected within a time frame of 1,800 ms.

The flight was carried out at a flying altitude of 140 m, which provided a GSD of 14.4 cm; the flying speed was 3.5 m·s⁻¹. The block used in this investigation consisted of five image strips and a total of 80 images; the forward and side overlaps were 78% and 67%, respectively (Figure 4a). We also used the central flight line (strip 3) for several detailed studies. The image footprint was 92 m by 145 m.

The campaign was carried out between 10:39 and 10:50 in the morning, local time (Coordinated Universal Time (UTC) + 3 hours). The solar elevation and azimuth angles were 43° and 125°, respectively. The wind speed was 5 m·s⁻¹, and the temperature was 19 °C. During the campaign, the illumination conditions were extremely poor with fluctuating levels of cloudiness (Figure 5). The non-optimal illumination conditions are a realistic situation for precision agriculture, where the timing of the data collection is critical; in the summer of 2012, there were no better imaging conditions during the time window for precision agriculture.

We carried out in situ irradiance measurements during the flight. In the UAV, we used an irradiance sensor based on the Intersil ISL29004 photodetector [35] to measure wide-bandwidth (400–1,000 nm) relative irradiance. On the ground, we measured spectral irradiance (W·m⁻²·nm⁻¹) at a spectral range of 350–2,500 nm in the central part of the test area using an ASD spectroradiometer that was equipped with 180° cosine collector irradiance optics for viewing the entire hemisphere of the sky. The changes in illumination in the images are clearly visible in Figure 5b; the level of irradiance was more than two-times higher at end of the flight than during the central part of the flight. We utilized the irradiance measurements in a relative mode. We provide details on these measurements in another publication [33].

3.3. Data Processing

We processed the data as described in Section 2.

First, radiometric pre-processing and spectral smile corrections were carried out. It was possible to generate 30 spectral bands with the smile correction (

Table 2. The central peak wavelengths and FWHMs for raw data cubes and the smile-corrected, final data cubes.

Raw Data, 42 Bands

Central peak wavelength (nm): 506.8, 507.4, 507.9, 508.4, 510.2, 515.4, 523.3, 533.0, 541.3, 544.1, 550.5, 559.6, 569.7, 581.3, 588.6, 591.3, 596.7, 601.7, 606.7, 613.8, 629.5, 643.1, 649.7, 657.2, 672.6, 687.3, 703.2, 715.7, 722.7, 738.8, 752.7, 766.9, 783.2, 798.1, 809.5, 811.1, 826.4, 840.6, 855.2, 869.9, 884.5, 895.4

FWHM (nm): 14.7, 22.1, 15.2, 16.7, 19.7, 23.8, 25.5, 24.9, 22.7, 12.7, 23.9, 23.0, 27.2, 21.4, 18.3, 41.1, 22.1, 44.0, 21.4, 41.5, 41.1, 35.3, 12.9, 40.4, 36.5, 38.3, 33.5, 29.9, 32.7, 32.8, 27.6, 31.8, 32.1, 25.9, 14.7, 28.2, 29.5, 26.5, 28.3, 28.4, 26.4, 22.3

Smile Corrected Data, 30 Bands

Central peak wavelength (nm): 511.8, 517.9, 526.6, 535.5, 544.2, 553.3, 562.5, 573.1, 582.7, 590.6, 595.2, 599.5, 606.2, 620.0, 634.4, 648.0, 662.5, 716.8, 728.2, 742.9, 757.0, 772.1, 787.5, 801.6, 815.7, 830.3, 844.4, 859.0, 873.9, 887.3

FWHM (nm): 19.7, 23.8, 25.5, 24.9, 22.7, 23.9, 23.0, 27.2, 21.4, 18.3, 41.1, 22.1, 44.0, 41.5, 41.1, 35.3, 40.4, 29.9, 32.7, 32.8, 27.6, 31.8, 32.1, 25.9, 28.2, 29.5, 26.5, 28.3, 28.4, 26.4

; Figure 6). The bandwidths were 18–44 nm at FWHM. Table 2 shows the central peak wavelengths and FWHMs for the raw data cubes and for the final corrected data cubes.

For the dataset, the movement of the sensor in the flight direction when collecting a single spectral data cube at a recording time of 1.8 s was approximately 6 m (40 pixels), while the spatial displacement between temporally adjacent bands was 0.26 m (1.7 pixels). During the band matching phase, we selected bands in each principle wavelength area (band 7: 535.5 nm, 24.9 nm; band 16: 606.2 nm, 44.0 nm; band 29: 787.5 nm, 32.1 nm) as reference bands. The temporal differences between the bands matched the reference bands were ranging from −0.4 to 0.6 s, while the computational horizontal movement in the flight direction was less than 1.8 m (12 pixels). We used an affine transformation and nearest neighbor interpolation for the geometric image transformation done as part of the band matching process. The reference bands were selected so that they had a good SNR and the temporal difference between the bands that needed to be matched was as small as possible; the reference bands were collected using different air gap values.

The geometric processing phase consisted of determining the orientations of the images and calculating the DSMs. We carried out this phase using a Bae Systems SOCET SET photogrammetric workstation [20,26]. The self-calibrating bundle block adjustment method was applied to determine the orientations of three reference bands. Because the direct orientation information provided by the UAV system was of a poor quality, some interaction was required to determine the approximate orientations of the images; the subsequent tie point measurement was fully automatic. The GCPs were measured interactively.

The DSM was generated via image matching using Next Generation Automated Terrain Extraction software (NGATE), which is a part of the SOCET SET software; this process has been described in more detail by Rosnell and Honkavaara [26]. The DSMs were generated using a triangulated irregular network (TIN) format with the NGATE software with a point interval of 20 cm. This format attempts to provide terrain height data at approximately 20 cm point intervals, but it does not interpolate the points in cases of failure in the final DSM.

We used different radiometric models during the image processing phase, as shown in

Table 3. Processing options for the radiometric image processing phase. BRDF, bi-directional reflectance distribution function.

Calculation Case (id)	Dataset	Parameters
No correction (no corr)	Full, Strip 3	aabs, babs
Relative radiometric correction using wide-bandwidth irradiance measured in UAV (uav)	Full	Cj, aabs, babs
Relative radiometric correction using spectral irradiance measured on the ground (ground)	Full	Cj(λ), aabs, babs
Radiometric block adjustment with relative multiplicative correction (BA: relA)	Full	arel_j, aabs, babs
Radiometric block adjustment with BRDF and relative additive correction (BA: relB, BRDF)	Strip 3	brel_j, a’, b’, aabs, babs

. Relative correction parameters are crucial for the dataset, due to variations in solar illumination. In principle, BRDF parameters are not necessary because of cloudy weather, but we tested this option anyway. For the radiometric block adjustment, we used a grid of radiometric tie points with a point interval of 10 m, while the average DNs (to be used in Equation (2)) were calculated using 4.5 m by 4.5 m image windows. The DNs were scaled to the range of 0–1 during the processing phase. After making the radiometric block adjustment, the absolute calibration parameters (a_abs, b_abs) were determined using the empirical line method. In the quality assessment, we used the average variation coefficients (standard deviation of gray values divided by the average gray value) for each radiometric tie point having multiple observations as a measure of the homogeneity of the data. Details on the radiometric correction method are provided in several publications [20,29,33].

We calculated orthophoto mosaics with a GSD of 0.20 m using the image orientations, DSM and radiometric model. We obtained the reflectance values using the most nadir method; hence, we used the DN from the image where the projection of the ground point was closest to the image nadir point to calculate reflectance (Equations (2) and (4)).

3.4. Biomass Estimation Using Spectrometric Data

The motivation for conducting these field tests stemmed from precision agriculture. The idea is to produce precise maps for nitrogen fertilization [16,17]. If we know the biomass and nitrogen content in the field, then, based on this knowledge, as well as information about the soil structure and harvest history of the field, it is possible to generate fertilization plans. We evaluated the performance of the dataset for biomass estimation using a k-nearest neighbor (KNN, k = 9) estimator, which is a supervised machine learning technique [36]. We calculated the spectral features as an average reflectance in 1 m × 1 m areas, and we used all of the spectral channels from the spectral data cubes as a feature space. The biomass estimate was obtained as an average of the k spectrally nearest samples. We used the leave-one-out cross validation technique to assess the performance of the KNN estimator. In this method, a single observation from the original sample is used as the validation data, and the remaining observations are used as the training data. This process is repeated such that each observation in the sample (a total of 53) is used once as the validation data. Finally, we calculated a normalized root-mean-square error (NRMSE) using the individual errors.

4. Results

4.1. Image Quality

Visually, the image quality appeared to be good. Examples of the image quality can be seen in Figure 7. This figure was divided into four parts: Parts 1–3 (from left) are individual bands 7 (535.5 nm, 24.9 nm), 16 (606.2 nm, 44.0 nm) and 29 (787.5 nm, 32.1 nm), respectively, while the last one part is the three-band, un-matched band composite (7, 16 and 29), which shows the mismatch between the bands in the original data. In comparison to the datasets obtained using the 2011 prototype [20,29], the improvement in the f-number from <7 to <3 and the change from a rolling shutter to an electronic shutter is apparent in the image quality in the form of reduced noise and reduced blurring.

We estimated the signal-to-noise ratio (SNR) as a ratio of the average signal to the standard deviation in a small image window when using a tarpaulin with a nominal reflectance of 0.3 (Figure 8). This is not an accurate estimate of the SNR, because it was also influenced by the nonuniformity of the tarpaulin; still, it can be used as an indicative value, because the relationships between bands are realistic. The SNR was typically around 80, but a reduction appeared in some of the green bands (wavelength 550–600 nm), as well as in the NIR bands (wavelength >800 nm). The behavior was as expected. In the NIR band, the quantum efficiency of the CMOS image sensor decreases as the wavelengths increases. The decreasing SNR in some green bands was a result of the low transmission at those bands, due to the edge filter, which limits the spectral bandwidth and decreases the transmission.

4.2. Band Matching

The assessment of the band matching results indicated that the matching was successful. The numbers of tie points were 10–806, mostly >100, and the median was 276. The standard deviations of the shift parameters were 1.5, 0.9 and 0.8 pixels for the green, red and NIR reference bands, respectively.

The average shifts of the bands matched to the green, red and NIR reference bands are shown in Figure 9. They were between five and −15 pixels in the flight direction and less than three pixels in the direction perpendicular to the flight direction. The maximum shifts were up to 25 pixels in the flight direction and 43 pixels in the cross-flight direction. The results presented in Figure 9 show that in the flight direction, the measured shift values were quite close to the expected values that were calculated based on the time difference between the reference band and the matched band. In the direction perpendicular to the flight, the shifts are shown as a function of the time difference between the bands. The average values showed minor systematic drift, which was larger as the time difference between the bands became larger; this is likely due to the possible minor drift of the camera’s x-axis with respect to the flying direction.

These results show that the band matching results are consistent with our expectations. Our conclusion is that the method developed here for band matching was appropriate for the experimental testing done in this investigation. It can be improved upon in many ways if required for future applications. Efficient, automated quality control procedures can be developed by evaluating the matching and transformation statistics and the number of successful tie points and by comparing the estimated transformation parameters to the expected values. The matching methods can be further improved, as well.

4.3. Geometric Processing

4.3.1. Orientations

We determined the orientations of the reference bands in the green, red and near-infrared spectral regions (bands 7, 16 and 29). Block statistics are provided in

Table 4. Statistics on the block adjustments for reference bands 7, 16 and 29: standard error of unit weight (σ₀), root-mean-square error (RMSE) values for the standard deviations of the perspective center positions (X0, Y0, Z0) and image rotations (ω, φ, κ) and the RMSEs for the residuals of the GCPs; the number of GCPs (N; targeted, natural). Central peak wavelengths and FWHMs of the reference bands: 7 (535.5 nm, 24.9 nm), 16 (606.2 nm, 44.0 nm) and 29 (787.5 nm, 32.1 nm).

Band	σ0	RMSE Positions (m)			RMSE Rotations (°)			RMSE GCPs (m)			N
	(pixels)	X0	Y0	Z0	ω	φ	κ	X	Y	Z
7	0.68	0.28	0.27	0.06	0.105	0.107	0.019	0.08	0.10	0.05	9,10
16	0.73	0.26	0.25	0.06	0.094	0.100	0.018	0.06	0.13	0.05	11,13
29	0.49	0.24	0.24	0.06	0.091	0.093	0.016	0.14	0.13	0.03	10,13

. The standard deviations of the unit weight were 0.5–0.7 pixels, and they were best for the near-infrared band (band 29); the precision estimates for the orientation parameters also indicated better results for band 29. The better results for band 29 might be due to the higher intensity values in the NIR band (that provided a better SNR), which could provide better quality for the automatic tie point measurements (the fields are very dark on the green and red bands). On the other hand, the root-mean-square errors (RMSEs) of the GCPs indicated the poorest performance for band 29, which was probably due to the difficulties in measuring the GCPs for this band (the likely reason for this is that the properties of the paint used for the GCPs were not ideal in the NIR spectral region). The RMSEs for the targeted GCPs were on the level of one pixel, and they were expected to be a representative estimator of the georeferencing accuracy.

For the self-calibrating bundle block adjustment, we estimated the principal point of autocollimation (x0, y0) and the radial distortion parameter, k₁ (the radial distortion correction (dr) for radial distance r from the image center is dr = k₁r³) (

Table 5. Estimated coordinates for the principal point of autocollimation (x0, y0) and radial distortion (k1) and their standard deviations for reference bands 7, 16 and 29. Central peak wavelengths and FWHMs of the reference bands: 7 (535.5 nm, 24.9 nm), 16 (606.2 nm, 44.0 nm), and 29 (787.5 nm, 32.1 nm).

Band	Parameters			Standard Deviation
	x0 (mm)	y0 (mm)	k1 (mm·mm−3)	x0 (mm)	y0 (mm)	k1 (mm·mm−3)
7	0.270	−0.023	0.00251	0.003	0.006	1.26E-05
16	0.242	−0.091	0.00252	0.002	0.005	1.07E-05
29	0.208	−0.132	0.00251	0.002	0.005	1.06E-05

). The k₁ parameter values were similar in the different bands, which was consistent with our expectations. This property is favorable for the band matching process. The estimated values for the principal point of autocollimation varied in the different bands; this was likely due to the non-optimality of the block for this task. The coordinates of the principal point correlated with the coordinates of the perspective center. More detailed investigations of the camera calibration should be carried out in a laboratory using suitable imaging geometry.

The quality of the block adjustment was in accordance with our expectations. These results also indicated that the orientations of individual bands can be determined using state-of-the-art photogrammetric methods at commercial photogrammetric workstations.

4.3.2. DSM

Figure 10a (left) shows a DSM obtained via automatic image matching using band 29 of the FPI image block. The matching quality was poor for tractor tracks, which appeared either as matching failures (missing points) or as outliers (high points), but otherwise, we obtained a relatively good point cloud. We also tested how to generate DSMs using the green and red reference bands, but the matching quality was poor, likely due to the lower SNR with the dark object (the fields are dark in the green and red bands). The reference DSM extracted using the higher spatial resolution image block collected on the same day is shown in Figure 10b (left). This DSM has a higher quality, but some minor matching failures in the tractor tracks appeared with this DSM, as well. The height RMSE of the FPI spectral camera DSM was 35 cm at the GCPs; likewise, the comparison of the FPI DSM to the reference DSM at the vegetation sample locations provided an RMSE of 35 cm.

We evaluated the potential for using vegetation heights taken from the DSM during the biomass estimation. We obtained the estimate for the crop height by calculating the difference between the FPI spectral camera DSM and the DSM based on airborne laser scanning (ALS DSM), which was collected in springtime on bare ground (Figure 10a (middle)). The estimated height of the vegetation was between zero and 1.4 m; the average height was 0.74 m, and the standard deviation was 0.36 m. The linear regression between the dry biomass values and the vegetation height did not show any correlation (the R² value of regression was zero) (Figure 10a (right)). In the more accurate reference DSM (Figure 10b), the corresponding values were as follows: a vegetation height of 0.44 to 0.99 m, an average height of 0.76 m and a standard deviation of 0.12 m; the R² value for the linear regression of dry biomass and the vegetation height was 0.36. With both DSMs, the vegetation height was slightly overestimated. It was possible to visually identify the areas without vegetation using both DSMs. Using the reference DSM, the areas with 0% fertilization could also be visually detected; this was not possible with the FPI DSM. The FPI DSM was not of sufficient quality to reveal differences in the vegetation heights, which was an expected result, due to the relatively high height deviations in the DSM. With both DSMs, it is expected that the DSM is reliable only within the area surrounded by the GCPs.

The reason for the worse DSM quality when using the FPI spectral camera could be due to the lower SNR and narrower bands, as well as to the fact that the matching software and the parameters might not have been ideal for this type of data; we tested several parameter options using the NGATE software, but they did not improve the results. One possible improvement could be to use more ideal band combinations during the matching phase. Furthermore, the matching algorithm could be further tuned, and different matching methods should be studied. The quality of the DSM can be considered promising and sufficient for image mosaic generation if the matching failures can be interpolated and filtered at a sufficient level of quality.

4.3.3. Image Mosaics

We calculated the orthophoto mosaics using the orientation information and the DSM. The RMSE when using the GCPs as checkpoints was less than 0.2 m for the x and y coordinates.

4.4. Radiometric Processing

We conducted radiometric processing for the entire block and for strip 3. In addition to the new results for strip 3, we reproduced relevant parts of the results from a previous study [33] with the full block in order to evaluate the impact of radiometric correction on the final application. Significant radiometric differences appeared within the image mosaics, due to the extremely variable illumination conditions. This was apparent especially for the full block (Figure 11b), while strip 3 was of a more uniform quality (Figure 11a). The radiometric correction greatly improved the homogeneity of the data (Figure 11c–f). The detail of a corrected mosaic (Figure 11g) shows the good quality of the data.

We used the average coefficient of variation for the radiometric tie points as the indicator of the radiometric homogeneity of the block (Figure 12) (the correction methods are described in Table 3). The coefficients of variation values for the full block (Figure 12a) were 0.14–0.18 when a radiometric correction was not used and 0.05–0.12 when a radiometric correction was applied. The best results of 0.05–0.08 were obtained with the relative block adjustment (BA: relA). For strip 3 (Figure 12b), the values were 0.06–0.08 when a radiometric block adjustment was not performed. With radiometric correction, we obtained the lowest variation coefficients, approximately 0.02, for NIR bands; the variation coefficients were better than 0.03 for the green bands and better than 0.04 for the red bands (BA: relB, BRDF). The better values for the single strip are most likely due to the fact that less variability needed to be adjusted for the single strip. The correction based on the spectral irradiance measurement on the ground (ground) appeared to provide better homogeneity than the UAV-based correction (uav). The image-based correction provided the best homogeneity, but there were some drift effects, which appeared as a slight tendency of the block to brighten in the north-east direction. The results from the previous study [33] also showed that all of the correction approaches—UAV irradiance (uav), ground irradiance (ground) and radiometric block adjustment (BA: relA)—provided relatively similar estimates of the illumination variations.

Six sample spectral profiles are shown in Figure 13 for different radiometric processing cases. The reflectances were taken as the median value in a 1 m by 1 m image window. The greatest differences between the samples appeared in the reflectance values in the NIR spectral region. In all cases, the NIR reflectance of samples with low biomass values (<1,000 kg·ha⁻¹) were clearly lower than the NIR reflectance of samples with higher biomass. With the radiometric corrections, the high (>2,500 kg·ha⁻¹) and medium (1,500–2,000 kg·ha⁻¹) biomass samples could be separated (Figure 13b–e), which was not possible in the data without radiometric corrections (Figure 13a). There were also some concerns with the spectral profiles. The green reflectance peaks were not distinct, especially in cases without radiometric correction (Figure 13a), which indicated that the radiometric correction was not perfectly accurate in these cases. Furthermore, the reflectance values were low in the green and red reflectance regions, typically lower than the dark reflectance target used in radiometric calibration (Figure 13f); this means that the reflectance at green and red spectral regions was extrapolated. Radiometric calibration is very challenging at low reflectance, because small errors in calibration will cause relatively large errors in reflectance values.

We did not have reference spectrums of the vegetation samples, so it was not possible to evaluate the absolute reflectance accuracy. The reflectance spectra of the three reflectance tarpaulins indicated that the spectra measured in images fitted well with the reference spectra measured in the laboratory (Figure 13f).

The results of the radiometric correction were promising. In the future, it would be of interest to develop an approach that integrates the image-based information and the external irradiance measurements. Special attention must be paid to situations in which the illumination conditions change between the flight lines from sunny to diffuse (sun behind a cloud). Furthermore, a means for correcting the shadows and topographic effects should be integrated into the method.

4.5. Biomass Estimation Using Spectrometric Information from the FPI Spectral Camera

We tested the performance of the reflectance output data from the FPI spectral camera in a biomass estimation process using a KNN estimator. Furthermore, we calculated Normalized Difference Vegetation Index (NDVI) (NDVI = (NIR _815.7 − R₆₄₈)/(NIR _815.7 + R₆₄₈)). Figure 14 shows the biomass estimation and NDVI statistics for different radiometric processing options.

In the biomass estimate maps, the areas that have no plants were quite visible in all cases (leftmost plots in Figure 14). The strips with 0% fertilization could be identified in most cases. The continuous lower biomass pattern in the middle of the area in the corrected dataset corresponds to a slight downhill terrain slope, which flattens towards the north and moves the valuable nutrients there. If a radiometric correction had not been carried out, the radiometric differences caused by the changes in the illumination were clearly visible in the mosaic and distorted the biomass estimates (Figure 14a). In the case of the correction based on the irradiance measurement with the UAV (Figure 14c), some strip-related artifacts (high biomass values in strips 2 and 4) appeared; we also identified these inaccuracies in the correction parameters in our recent study [33] and concluded that these inaccuracies were likely due to some shadowing effects of the irradiance sensor in image strips 1, 3 and 5 (solvable in future campaigns). The largest differences between the estimates with a correction based on the ground irradiance measurement (Figure 14d) and the image-based correction (Figure 14e) appeared in the south-west part of the mosaic, which had lower biomass estimates, and in the north-east part of the mosaic as higher biomass estimates for the image-based method. The results for the two image-based corrections were quite similar (Figure 14b,e). It is possible that the correction based on image information could have some drifts, which appeared as mosaic brightening towards the north-east direction.

For strip 3, the NRMSEs of biomass estimates were 26.2% for uncorrected data and 20.4% for radiometrically corrected data (Figure 14, second column). For the full block, the NRMSEs were 24.4%, 17.3% and 15.5% for a radiometric correction based on wide-bandwidth irradiance measured using an UAV, spectral irradiance measured on the ground and a relative block adjustment, respectively. The radiometric correction of the full block data with a relative multiplicative correction (BA: relA) thus provided the highest degree of accuracy, while the correction based on the ground irradiance measurement (ground) was very close to this result. The results clearly showed the great impact of radiometric correction on the biomass estimation. The resulting biomass estimate maps were feasible when appropriate corrections were applied and the numerical values supported the visual results. The KNN estimator was not quite ideal for estimating continuous variables, but we considered that it was feasible for our purposes of validating the data processing; our training data also was suitable for this estimator (Figure 4b).

The NDVI maps appeared to be realistic, showing the areas with and without vegetation (Figure 14, third column). The R² values of the linear regression of the dry biomass and NDVI were 0.57–0.58 for most of the cases (Figure 14, right column). Strip 3 with radiometric correction (BA: relB, BRDF) was an exception (Figure 14b); it provided a lower R² of 0.38, and also, the NDVI map appeared to be biased, showing relatively high NDVI values at the non-vegetated areas in the upper part of the strip. These observations were consistent with the biomass estimation results and analysis.

5. Discussion

Developing quantitative, lightweight UAV remote sensing applications is becoming ever more important, because this technology is increasingly needed in various environmental measurement and monitoring applications. In this study, we presented a complete processing chain for a novel, lightweight, spectrometric imaging technology based on a Fabry-Perot interferometer (FPI) in an agricultural application.

In our previous investigations [20,21,29], we performed the first set of analyses with the FPI spectral camera 2011 prototype using five selected spectral bands. In this investigation, we processed the data using the improved 2012 prototype sensor. First, we developed a method to process all of the bands. We investigated the orientation process and calculated the digital surface models (DSM) by automatic image matching using FPI spectral camera data. We also evaluated the impacts of different radiometric correction approaches during a supervised biomass estimation process. It was important to carry out the entire processing chain in order to identify major bottlenecks and develop the methods further. The results were quite promising; they indicated that the current sensor is already operational and that the processing can be carried out quantitatively and also be highly automated. All sensor-specific processing steps could be implemented as independent steps in our existing processing environment based on commercial photogrammetric and remote sensing software; this is an important issue for companies planning to use the FPI spectral camera in their operational work.

The challenging part of processing the FPI data is that the bands in the spectral data cube are collected with a small time delay. Our approach was to select a few reference bands and determine the exterior orientations for them. We transformed the rest of the bands to match the geometry of the reference bands and, then, applied the orientation parameters of the reference bands to these bands. While the method for band matching proved to be operational, it can still be further improved. In the dataset, the estimated error was on the level of 1–2 pixels (15–30 cm). This level of accuracy is sufficient for most remote sensing applications, and we expect further improvements in the future because of improving sensors and processing methods. Difficulties or reduced accuracy are to be expected for objects with extensive height differences if large spatial differences exist between the bands (fast vehicle) and in cases where the objects are homogeneous (water areas); carefully designed flight parameters and band matching processes are needed to obtain good accuracy. The most optimal approach for producing the best accuracy could be to georeference the individual bands separately. This is a software issue: the software used in this investigation was not ideal for this approach. The FPI sensor provides many alternative ways for processing the data, but in this study, we concentrated on methods that could easily be integrated with our existing photogrammetric and remote sensing environment.

Geometric processing of the frame imaging sensors is a quite mature technology, even though methods are being improved constantly, for instance, to improve the reliability of processing very small format sensors operating in a highly dynamic environment, such as UAV imaging. Our processing required a certain amount of interaction during the block initialization phase; approaches for improving this include a better direct georeferencing solution [11,38] or applying some recently presented ordering methods to determine the approximate orientations of the images, such as the structure from the motion technique (e.g., [39–41]). Because rigorous integrated global navigation satellite and inertial measurement unit (GNSS/IMU) orientation systems for direct georeferencing are still quite expensive and heavy for light and low-cost systems, photogrammetry-based methods should be developed, so that they can operate at an optimum level. For image matching, SNR is a critical image quality indicator. Accurate GNSS data will also improve the georeferencing accuracy and eliminate or reduce the need for GCPs, which has been demonstrated in previous investigations [26,39]. While our geometric accuracy results were quite good in comparison to recent results obtained using hyperspectral sensors [7], they were not as good as those obtained using higher spatial resolution, wide-bandwidth sensors [26,39].

The quality of the point clouds extracted from the FPI spectral camera imagery was poorer than what many recently published results have indicated [20,23–27,29]. In these studies, wide-bandwidth, high dynamic range, high spatial resolution sensors were used. Because the spatial resolution of the spectral data is expected to be lower than what can be obtained with commercial wide-bandwidth small-format cameras, a functional approach would be to integrate a high spatial resolution sensor with an FPI spectral camera in order to obtain high-quality 3D information, as suggested in our previous study [20]. However, the lower quality DSM provided by the FPI spectral camera is also useful when processing and analyzing the data.

In the case of UAV imaging, radiometric processing is a relatively unexplored topic. Radiometric sensor correction is needed for a quantitative remote sensing processing line, and we also applied these methods to our processing line [7–9,14]. In this study, we considered the available methods to be accurate, but in future studies, reliable quality criteria should be developed for the sensor pre-processing phase. Traditional atmospheric correction methods based on radiative transfer have been developed for pushbroom imaging systems [31,42–45], and similar approaches have also been applied to UAV-based hyperspectral imaging systems [7,9]. Recently, approaches have been established for making radiometric block adjustments and for generating reflectance images for block data with rectangular images collected using stable, large-format digital photogrammetric cameras [42,46–48]. For UAV remote sensing applications using rectangular images, simple balancing approaches are typically used [3,5]; and empirical line-based approaches are popular [4]. Our objective is to develop a physically-based method for the atmospheric correction of frame images, one that includes a radiometric block adjustment utilizing radiometric tie points and utilizes in situ irradiance measurements in UAV and/or on the ground, but we are still applying many simplifications to the method [20,29,33]. While the radiometric processing proved to be quite complicated, due to the variability in the illumination conditions, we also found that both radiometric block adjustment and in situ irradiance measurement-based methods greatly improved the data quality. In the future, it will be of interest to integrate these methods [33]. The investigated and developed processing methods are useful for airborne UAV frame format imagery in general. Further investigations are still needed in order to develop accurate radiometric correction methods for high-resolution, multi-overlap frame image data collected under variable conditions. In the future, there will be a need to thoroughly consider the reflectance output products resulting from UAV remote sensing [28]. The quantitative radiometry is expected to improve the performance of the remote sensing application in general and, furthermore, will enable the use of rigorous radiative transfer modeling-based methods in the analysis of object characteristics; this would be advantageous for UAV-based precision agriculture, as the need for site-specific training data would be eliminated; the importance of the accurate radiometric processing and atmospheric correction is highlighted also in agricultural applications with global and regional focus [49].

Recently, researchers have conducted experiments with UAV imaging systems with hyperspectral scanners using the pushbroom principle [7,9]. In comparison to those systems, the FPI spectral camera collects less spectral bands that are not as narrow (10–40 nm in comparison to 1–10 nm). The advantages of the FPI spectral camera include its light weight and the fact that a direct orientation solution requiring expensive GNSS/IMU equipment is not needed, as well as the fact that it offers the possibility to conduct stereoscopic measurements and multi-angular reflectance measurements. All innovations developed for frame geometry images can be directly utilized when processing FPI spectral camera images; these techniques are expected to develop further due to the invasion of computer vision technologies in personal mobile equipment. We expect that the FPI spectral camera concept could provide more robust and cost-efficient applications than systems based on the pushbroom principle and that data collection can be optimized by using carefully selected spectral bands for each application. For many applications, we expect that the FPI spectral camera will be used as one component in an integrated sensor system; in the agricultural application, the important sensors that need to be integrated are a high spatial resolution, wide-bandwidth camera that provides more accurate DSMs, as well as a thermal camera [4,9].

We demonstrated the use of FPI data for estimating crop biomass in order to validate the data processing phase. Our results when using radiometrically corrected data and a supervised classification method provided at best a 15.5% normalized root-mean-square-error (NRMSE) during the biomass estimation process, which is in line with the results presented in the existing literature [15]; the NRMSE was 26.3% for the radiometrically uncorrected data. We assume that the results can be improved upon in many ways, such as if we were to use spectral band selection, spectral indices or multivariate statistics for the feature extraction [15]. The results when using the new sensor data and the entire data cube were better than the results from the previous year, which were obtained using the 2011 prototype sensor [20,21]. We will emphasize the optimization of the estimation process in our future investigations [4,15–17]. Integrating the vegetation heights into the estimation process is also an interesting option [21].

In the future, it will be important to develop an operational concept for precision agriculture using UAV technology [6,20]. In this operational concept, one of the crucial steps will be to quantify the geometric and radiometric properties required for the UAV remote sensing data, which has also been emphasized by Zhang and Kovacs [16]. Further legislation also needs to be developed; this is an important factor influencing the way in which UAV technology is used in practical applications, as discussed by Watts et al. [50].

6. Conclusions

Rapidly developing lightweight unmanned airborne vehicle (UAV) sensor technology provides new possibilities for environmental measurement and monitoring applications. We investigated the processing and performance of a new Fabry-Perot interferometer (FPI)-based spectral camera weighing less than 700 g that can be operated from lightweight UAV platforms. By collecting frame-format images in a block structure, spectrometric, stereoscopic data can be obtained. We developed and assessed an end-to-end processing chain for the FPI spectral camera data, together with image preprocessing; spectral data cube generation, image orientation, digital surface model (DSM) extraction, radiometric correction and supervised biomass estimation.

Our results provided new knowledge about high-resolution, passive UAV remote sensing. The pre-processing provided consistent results, and the orientations of the images could be calculated using self-calibrating bundle block adjustment using regular photogrammetric software. The quality of the DSM provided by automatic image matching was not as high as what was obtained with a wider spectral bandwidth, higher spatial resolution camera. The estimated root-mean-square-error was 40 cm in height and 20 cm in horizontal coordinates, for output image mosaics, and a DSM with ground sample distances of 20 cm, for image data collected using a flying altitude of 140 m. The varying illumination conditions caused great radiometric differences between the images; our radiometric correction methods reduced the variation of grey values in overlapping images from 14%–18% to 6%–8%. The supervised estimation of biomass provided a normalized root-mean-square-error of 15.5% at best. Data quality is an important factor influencing the performance of a remote sensing application; our results showed that signal-to-noise-ratio (SNR) and the radiometric uniformity amongst individual images forming the image mosaics impacted the biomass estimation quality. These results proved that a lightweight imaging sensor that is based on the sequential exposure of different bands can provide spectrometric, stereoscopic data. Furthermore, the results validated that useful spectrometric, stereoscopic data can be collected using lightweight sensors under highly variable illumination conditions, with fluctuating cloudiness, which is a typical operating environment for these systems.

The results showed that all FPI technology-related processing steps (image preprocessing and spectral data cube generation) can be taken care of in separate steps and that the rest of the processing can be carried out using regular photogrammetric and remote sensing software. The fact that images can be processed using regular software is an important aspect for users integrating the FPI spectral camera into their operational workflows. For radiometric processing, we have developed new quantitative methods that are suited for frame format images collected in variable illumination and atmospheric conditions.

This was the first quantitative experiment with FPI camera-type technology covering the entire remote sensing processing chain. Our emphasis in developing analysis tools for extremely challenging illumination conditions represents a new approach in hyperspectral remote sensing. Our results confirmed the operability of the FPI camera in UAV remote sensing and the high potential of lightweight UAV remote sensing in general.

We identified many aspects that should be improved in our processing line. These are also recommendations for method development universally. In general, there is a fundamental need to develop reliable methods for the geometric and radiometric processing of huge numbers of small, overlapping images. Another important conclusion is that it will be crucial to develop all-weather processing technology in order to take full advantage of this new technology and to make this technology operational in practical applications. We expect that there will be a great demand for these methods in the near future.

We note that in the future, it will be of great importance to develop reliable error propagation for all phases of the process to enable quantitative applications for these data. Numerical tolerances and criteria will be required by the user community as soon as the UAV-based remote sensing business increases. Further development of the quality indicators that were presented in this investigation is necessary.