The Wall: The Earth in True Natural Color from Real-Time Geostationary Satellite Imagery

Abstract

We present “The Wall”, the first web-based platform that animates the Earth in true natural color and close to real-time. The living planet is displayed both during day and night with a pixel resolution of approximately 1 km and a time frequency of 10 min. The automatic processing chains use the synchronized measurements provided by three geostationary satellites: the METEOSAT Second Generation (MSG2), Himawari-8, and GOES-16. A Rayleigh scattering correction is applied, and a cloud of artificial neural networks, chosen to render “true natural color” RBG composites, is used to recreate the missing daytime bands in the visible spectrum. The reconstruction methodology is validated by means of the TERRA/AQUA “Moderate Resolution Imaging Spectroradiometer” (MODIS) instrument reflectance values. “The Wall” is a dynamic broadcasting platform from which the scientific community and the public can trace local and Earth-wide phenomena and assess their impact on the globe.

1. Introduction

For many years geostationary satellites have surrounded and observed the Earth. In the last decade, their resolution has improved to 1 km at nadir. At the same time, real-time reception of imagery has become increasingly practical through satellite downlinks and online-accessible archives. However, little effort has been devoted to the creation of images showing the whole Earth in the splendor of its natural color. One of the main reasons is that often one or more components of the visible spectrum is not measured by the sensors on-board the satellites. In this application, we have selected three geostationary satellites to tile the entire globe: (1) The MSG2 “Spinning Enhanced Visible and Infrared Imager” (MSG2 SEVIRI) instrument managed by EUMETSAT was launched in 2002, and offers a resolution of 3 km at nadir. It covers Europe, Africa, and the Middle East. However, the blue and green visible bands (0.4 and 0.5 μm) are missing and make this satellite almost blind over the spectral range of human vision. (2) The NASA GOES-16 series “Advanced Baseline Imager” (ABI) [1] instrument covers the American continent. Launched in 2016, and managed by NOAA, it has a resolution of 1 km at nadir but it lacks the visible green band. (3) The Himawari-8 [2,3] “Advanced Himawari Imager” (AHI) instrument managed by the Japan Meteorological Agency (JMA) was launched in 2014. This instrument captures all three visible bands with a resolution of 1 km at nadir and covers the Western Pacific region. These three instruments offer detailed and quantitative monitoring of a wide array of surface and atmospheric parameters [1,4]. Although the tracking of atmospheric and land-surface phenomena usually requires the interrogation and combination of multiple spectral bands, we think it is also desirable for the human eye to view the spectral data in natural color RGB composites to help the analysis of the image components.

True color renderings of geostationary data can be assessed from the comparison with the orbiting satellite imagers that hold three visible reflectance bands at approximately 0.4, 0.5 and 0.6 μm, hereafter called RGB visible bands. In this work, we will use Moderate Resolution Imaging Spectroradiometer (MODIS) as a reference.

Today, several RGB composite terminologies are used: “Natural color RGB”, “Natural color RGB (enhance)”, “True color reproduction image”, and “GeoColor”, as displayed in Figure 1, all trying to mimic with more or less success what we think is “True natural color” corresponding to the human vision. Himawari-8 AHI uses the visible bands at 0.4697, 0.5094 and 0.6363 μm to directly build RGB images called “true color reproduction” images [3,5] (see inset (C) of Figure 1). Both Himawari-8 AHI and MSG2 SEVIRI create so-called “natural color RGB” composites [6] (see inset (A) of Figure 1) using the bands at 1.6, 0.8 and 0.6 μm, in which the clouds appear blue-green. Alternatively, the clouds are replaced by white levels, leading to the “natural color RGB (Enhance)” composite (see inset (B) of Figure 1). The best RGB rendering is the ”GeoColor” approach for GEOS-16 (see inset (D) of Figure 1), for which the missing green band is reconstructed either from correlations to adjacent bands or from a lookup table constructed from Himawari-8 AHI green band [7,8].

The project is named “The Wall” as a huge billboard broadcasting over the globe the best possible color rendering of geostationary satellite images and animations to the public and to the scientific community. It is thus necessary to create “true natural color” images for the three complementary satellites (GOES-16 ABI, MSG2 SEVIRI, and Himawari-8 AHI), using visible bands which, in cases where they are lacking on sensors like MSG2 SEVIRI and GOES-16 ABI, have to be reconstructed. In the published NASCube (North African Sandstorm Survey) project and web platform [9], we have demonstrated that artificial neural networks (ANNs) are very powerful to reconstruct reflectance signals of the MSG2 SEVIRI instrument. ANNs were also used to track sandstorms over the Sahara and Middle East and to estimate their optical thicknesses. We propose hereafter to apply this ANN methodology to predict the missing green band for the GOES-16 ABI. The reconstruction will be validated by comparison with the NASA MODIS products.

2. Methodology to Process the Geostationary Satellite Data

The aim of the study is to present a homogeneous visualization of the main atmospheric and terrestrial phenomena during day and night times, using data acquired simultaneously from three instruments with measurement characteristics that differ in terms of wavelengths, pixel sizes, and acquisition time delays. To perform this task, we have collected the level 1B data products, and calibrated and geolocated radiances expressed in radiance units W sr⁻¹ m⁻² over the Full Disk (FD).

We first present the specifications of the spectral bands used for each satellite (see Section 2.1). Section 2.2 explains the construction of the pseudocolor composites in night-time from several thermal infrared bands. Section 2.3 details the atmospheric corrections applied to the used spectral visible and near-infrared (NIR) bands. It also focuses on the key algorithms designed and implemented to reconstruct the missing visible bands for GOES-16 ABI and MSG2 SEVIRI, and finally build “true natural color” composite images.

2.1. Spectral Bands and Satellite Specifications of the Level 1B Data Products

2.1.1. Satellites with 1 km Resolution at Nadir

Both Himawari-8 AHI and GOES-16 ABI have a 1 km resolution at nadir and an acquisition delay time of 10 min. Himawari-8 AHI monitors the West Pacific ((144.418°W, 64.903°E)/(75.348°S, 75.348°N)). It is managed by the Japan Meteorological Agency (JMA) with on the on-board instrument AHI imager. The data is distributed by the National Institute of Information and Communications Technology (NICT) Science Cloud [10,11]. We have processed 6 of the 16 available bands, referenced by their central wavelength: 0.4697, 0.5094, 0.6363, 3.8823, 8.5878, and 9.6329 μm. The first three map the visible range as displayed in Figure 2.

To map the American continent, we have used the GOES-16 ABI-EAST region ((156.299°W, 6.299°E)/(81.328°S, 81.328°N)) which is managed by the National Environmental Satellite, Data, and Information Service (NESDIS) with the on-board ABI and distributed by the Google Cloud Platform [12]. We have processed 6 of the 16 bands, referenced by their central wavelengths: 0.4702, 0.6365, 0.8639, 3.8905, 8.4444 and 9.6072 μm. Note that the green visible band is missing in this instrument.

2.1.2. MSG2 SEVIRI Data with 3 km Resolution at Nadir

The MSG2 geostationary platform, equipped with MSG2 SEVIRI instrument, provides observations of Europe, Africa, Middle East, etc. ((81.236°W, 81.236°E)/(81.144°S, 81.144°N)), every 15 min since 2004. The satellite is managed by the European Organization for the Exploitation of Meteorological Satellites (EUMETSAT). We have processed 6 out of the 12 distributed spectral channels, referenced by their central wavelengths: 0.6, 0.8, 1.6, 3.9, 8.7 and 9.7 μm. Note that both the blue and green visible bands are missing as indicated in Figure 2.

2.2. Night-Time RGB Composite

The RGB rendering of night-time images is not straightforward. One option is to render them in gray levels. In “The Wall”, we currently offer two empirical algorithms designed with the primary aim of keeping a visual day/night white color transition of the clouds for the NIR bands shown in Figure 3. The final night composite is sufficiently sensitive to show in the video animations the propagation of the thermal changes over the Earth surface. The first algorithm is taken from that used for the NASCube project [9] and is directly applied to GOES-16 ABI. The second algorithm is a linear combination of bands applied to Himawari-8 AHI images. The results are close enough to offer a smooth visualization over the whole globe.

The three MSG2 SEVIRI (3.9, 8.7 and 9.7 μm) and GOES-16 ABI (3.8905, 8.4444 and 9.6072 μm) radiances are used to build the RGB (0; 255) Pseudo-Night Composites ($PNC$) for each instrument acquisition. The final image is directly computed by means of a linear-log transformation of the corresponding radiance values ${\rho }_i$:

$$\begin{array}{c}\hfill PNC\left({\lambda }_i\right)=256\times \frac{log\left(10000\times \left(a_i-b_i{\rho }_i\right)\right)-{min}_i}{\left({max}_i-{min}_i\right)}.\end{array}$$

(1)

For Himawari-8 AHI, the RGB $PNC$ is constructed from the following three bands (3.8823, 8.5878 and 9.6329 μm), and the final image is computed using a linear transformation of the corresponding radiance values:

$$\begin{array}{c}\hfill PNC\left({\lambda }_i\right)=256\times \left(a_i-b_i{\rho }_i\right)\end{array}$$

(2)

The $a_i$, $b_i$, $mi{n}_i$, and $ma{x}_i$ parameters are listed in

Table 1. Parameters for the linear-log and linear transformations to the Pseudo Night Composites for MSG2 SEVIRI, GOES-16 ABI, and Himawari-8 AHI satellites.

${\mathit{\lambda }}_{\mathit{i}}$ (μm)	PNC Channel	${\mathit{a}}_{\mathit{i}}$	${\mathit{b}}_{\mathit{i}}$	${\mathit{m}\mathit{i}\mathit{n}}_{\mathit{i}}$	${\mathit{m}\mathit{a}\mathit{x}}_{\mathit{i}}$
	MSG2 SEVIRI
3.88	R	0.004975	1.308458	7.5	9.5
8.58	G	0.002165	1.246753	7.7	9.5
9.63	B	0.004202	1.899160	7.5	9.5
	GOES-16 ABI
3.89	R	0.000338	1.007768	7.5	9.5
8.44	G	0.000501	1.046115	7.7	9.5
9.60	B	0.001000	1.176000	7.5	9.5
	Himawari-8 AHI
3.9	R	−1.269036	0.842010	N/A
8.7	G	−0.119018	1.118907	N/A
9.7	B	−0.181937	1.250030	N/A

. They were empirically adjusted to yield the best contrast for the resulting night images, as illustrated in Figure 4. Figure 5 and Figure 6 illustrate the ability of the night composite to highlight the ground hot spots. In Figure 5, the active bushfires that raged in early 2020 in Australia are visible as dark spots on the night RGB image. In Figure 6, the activity of the Klyuchevskaya volcano in the Kamchatka Peninsula appears as a blue hot spot.

2.3. Daytime RGB Composite

Daytime reflectance values are rendered with a “True Natural Color RGB Composite” ($TNCC$) that is built from visible reflectance data. For MSG2 SEVIRI and GOES-16 ABI, we had to reconstruct the missing visible bands with a cloud of ANNs, which is applied to the spectral bands that have been corrected for atmospheric effects as described below.

2.3.1. Atmospheric Corrections

To obtain natural colors from the recorded reflectance values, the minimum atmospheric correction required is the compensation for the Rayleigh scattering contribution, to which a correction for ozone aerosols was added according to the 6S radiative transfer code (6sV2.1) [13]. The corrections are identical to that applied for the NASCube study [9], but we list them here for the sake of completeness. Let $L_m$ (W sr⁻¹ m⁻²) be the Top-Of-Atmosphere (TOA) radiance in band m and ${\rho }_m$ the TOA reflectance, calculated as

$$\begin{array}{c}\hfill {\rho }_m=\pi L_m/{\mu }_s{E}_s,\end{array}$$

(3)

with ${\mu }_s=cos{\theta }_s$, where ${\theta }_s$ is the solar zenith angle, and with $E_s$ the solar flux at the top of the atmosphere. The Rayleigh and ozone corrections assuming a Lambert’s law of the surface reflectance defines ${\rho }_m$ as

$$\begin{array}{c}\hfill {\rho }_m=T_{O_3}\left[{\rho }_{atm}+\frac{T\left({\mu }_s\right)T\left({\mu }_v\right){\rho }_s}{1-{\rho }_s·S_{alb}}\right],\end{array}$$

(4)

where ${\rho }_S$ stands for the surface reflectance, $S_{alb}$ is the spherical albedo of the atmosphere, and ${\mu }_v=cos{\theta }_v$, with ${\theta }_v$ the view zenith angle as defined in the 6S reference manual [14,15]. In Equation (4), $T_{O_3}$ accounts for the light absorption on the direct downward and upward paths through the stratospheric ozone layer:

$$\begin{array}{c}\hfill T_{O_3}=exp\left(-A_{O_3}·{\mu }_{O_3}·\left(\frac{1}{cos\theta s}+\frac{1}{cos\theta v}\right)\right),\end{array}$$

(5)

where ${\mu }_{O_3}$ is the ozone absorber amount for each satellite wavelength, and $A_{O_3}$ is the ozone absorption coefficient by Shettle et al. [16], tabulated in steps of 200 cm⁻¹ between 13.000 and 24.200 cm⁻¹ and by steps of 500 cm⁻¹ between 27.500 and 50.000 cm⁻¹.

The other terms of Equation (4) correspond to atmospheric effects and are estimated assuming both a black surface and a pure molecular atmosphere. ${\rho }_{atm}$ is the atmospheric reflectance that corresponds to the light directly scattered by the atmosphere. It is computed by the analytical expressions [14] that reproduce Chandrasekhar isotropic scattering values [17]. $T\left({\mu }_s\right)$ and $T\left({\mu }_v\right)$ are the transmission fluxes of the atmosphere on the path between the sun and the surface, and, respectively, between the surface and the sensor. Both are estimated from the Delta–Eddington approximation [18], such as

$$\begin{array}{c}\hfill T\left(\mu \right)=\frac{\left(2/3+\mu \right)+\left(2/3-\mu \right)·e^{-\tau /\mu }}{\left(4/3+\tau \right)},\end{array}$$

(6)

where $\mu$ is the cosine of the solar and/or observational zenith angle and $\tau$ is the optical thickness. For a pure molecular atmosphere and for some observation conditions (${\mu }_s$, ${\mu }_v$, and ${\rho }_S$), numerical figures of ${\rho }_m$ were derived directly (independently) from the “successive orders of scattering” code of Lenoble et al. [19]. By reporting these contributions to the left-hand side of Equation (4), $S_{alb}$ can be derived, leading to $S_{alb}$ = 0.0497, 0.0198, and 0.0012 for wavelengths 0.615, 0.810 and 1.64 μm, respectively. Assuming that $\left({\rho }_s·S_{alb}\right)\ll 1$ in Equation (4), the surface reflectance can be computed as

$$\begin{array}{c}\hfill {\displaystyle {\rho }_s^{*}=\frac{\left({\rho }_m{T}_{O_3}-{\rho }_{atm}\right)}{T\left({\mu }_s\right)T\left({\mu }_v\right)},}\end{array}$$

(7)

from where, the surface reflectance is computed as

$$\begin{array}{c}\hfill {\rho }_s=\frac{{\rho }_s^{*}}{1+S_{alb}·{\rho }_s^{*}}.\end{array}$$

(8)

2.3.2. RGB Visible Composites

Once all three visible bands are fetched or reconstructed, the daytime RGB composite $TNCC$ can be built by log-transforming the 0.4, 0.5 and 0.6 μm (0; 255) RGB visible bands colors:

$$\begin{array}{c}\hfill TNCC\left({\lambda }_i\right)=256\times \frac{log\left(10000\times \left(a_i-b_i{\rho }_i\right)\right)-{min}_i}{\left({max}_i-{min}_i\right)},\end{array}$$

(9)

with $mi{n}_i$ and $ma{x}_i$ values in the range of 5.8 to 9.6, respectively.

2.3.3. Reconstruction of the Missing Green Band for GOES-16 ABI

The details of the ANN methodology used to reconstruct the missing blue and green bands (0.4 and 0.5 μm) of the MSG2 SEVIRI sensors are given in Ref. [9]. The use of an ANN is clearly superior to any lookup table, or other band correlation algorithm, as our true natural color composites for MSG2 SEVIRI images appear more natural than the MSG Natural Color RGB (Enhance)-distributed product (see Figure 1), the reference being MODIS imagery as discussed later in the paper.

Fine-tuning the parameters of an ANN (Stuttgart Neural Network Simulator (SNNS) [20]) to faithfully reconstruct GOES-16 ABI green reflectances turned out to be a tricky process. We have designed an ANN that takes as training inputs the three MODIS 0.469, 0.645 and 0.858 μm (spectrally close to the GOES-16 ABI bands, 0.47, 0.636 and 0.863 μm, as seen from Figure 2), and the green MODIS band at 0.555 μm as target values. This ANN is a fully connected feedforward network [21]. The ANN topology we present yields the best green retrieval out of many tests we carried out. It consists of two hidden layers with 8 nodes each (See Figure 7). The training data set is composed of a selection of about thirty MODIS granules for which we hand-selected representative areas such as water, ocean colors, deserts, rocks, white and bright surfaces, etc. Once trained, the network was tested on various MODIS granules and the training ANN data set was enlarged with those showing inaccurate—poor correlation with respect to MODIS real green reflectances—reconstructed green values.

From our recent experience with the design of ANNs for aerosol optical depth retrievals and with the reconstruction of the lacking ASTER blue band [9,22], we have demonstrated that the training and accuracy of ANNs are improved if applied to specific classes of reflectance values. Thus, our new algorithm uses MODIS values that are classified according to the empirically chosen values x defining two main groups:

$$\begin{array}{c}\hfill x={\rho }_{MODIS}\left(0.469\right)+\frac{2}{3}\left({\rho }_{MODIS}\left(0.469\right)+{\rho }_{MODIS}\left(0.645\right)+{\rho }_{MODIS}\left(0.858\right)\right),\end{array}$$

(10)

the first one corresponding to water, dark areas, and vegetation ($x<0.2$), and the second one to desert or cloudy areas ($x>0.2$). Figure 8 is an example of the pixel classification over one MODIS granule, the first group being the unmasked pixels and the second group appearing in blue.

The histograms of the two groups of all MODIS input values are shown in Figure 9. Each of the two histograms is split to generate subclasses of pixel values defining the inputs of ANNs. The determination of the number of data by classes (bins) was carried out in stages by monitoring both the convergence of the networks and the quality of the green retrieval over the MODIS granules composing the training data sets. It soon became clear that the first group requires 1536 networks and converges with an average number of 6000 pixels per bin and the second group requires 1256 networks and converges with an average number of 3000 pixels per bin. For each bin of the histograms of Figure 9, a dedicated ANN with the architecture defined before (3 inputs, 2 hidden layers with 8 nodes each, and 1 output value, see Figure 7), is designed. Altogether, an ensemble of 2792 ANNs is trained.

3. Results and Discussion

3.1. Validation of the ANN Green Reconstruction for GOES-16

To assess the quality of the ANN green reconstruction for GOES-16, we have considered a MODIS granule far away in time from the ANN training data set shown in Figure 8, and we focus on the red-selected area. The correlation plot drawn on the left-hand side of Figure 10 shows the excellent correlation between the native MODIS green reflectance values and the ones predicted by the cloud of ANNs over the full granule. To further illustrate the accuracy of the green reconstruction, we also present, on the same figure, the plots of the green reflectance values over the red transect. The green (original) and black (ANNs retrieval) curves are in nearly perfect agreement, illustrating that the 1 MODIS resolution is preserved.

In Figure 11, we have drawn the correlation plots between the GOES-16 ABI and MODIS red, green, and blue reflectance values for two reprojected images over the same region of South America (Chile/Argentina) taken within few minutes. As the images do not perfectly spatially overlap, we note some spreading in the correlations for the red and blue bands that directly mirror into the spreading of the correlation of the green reflectance values. In conclusion, both Figure 10 and Figure 11 fully validate the ANN-based green reconstruction methodology for GOES-16 ABI.

Figure 12 and Figure 13 allow to compare the same scenes, in natural color components, as rendered by MODIS, GOES-16 ABI with our ANN reconstructed green channel, and with the GeoColor RGB composite. As MODIS has three visible wavelengths, it must approach the reality. These figures convincingly illustrate that our RGB composite faithfully reproduces the MODIS RGB colors. The matching is nearly perfect over rocks and desert areas, while we may pinpoint differences over vegetated areas, due to the lack of Bidirectional Reflectance Distribution Function (BRDF) correction effects.

3.2. Description of “The Wall” Processing Chain and Web Visualization Platform

Table 2. An incomplete list of web-based platforms for geostationary satellite imagery.

GOES-16 satellite

GOES-16 imagery

https://www.goes-r.gov/multimedia/dataAndImageryImagesGoes-16.html#GOES-EastOperational

NOAA GOES image viewer

https://www.star.nesdis.noaa.gov/GOES/fulldisk.php?sat=G17

GOES-16 and Himawari-8 satellites

SSEC Geostationary Satellite Imagery

https://www.ssec.wisc.edu/data/geo/#/animation

Himawari-8 satellite

NICT

https://himawari8.nict.go.jp/

JAXA EORC

https://www.eorc.jaxa.jp/ptree/index.html

Meteorological Satellite Center of JMA

https://www.data.jma.go.jp/mscweb/data/himawari/index.html

MSG2 SEVIRI

ICARE

https://www.icare.univ-lille.fr/data-access/browse-images/geostationary-satellites/

METEOSAT

http://www.meteo-spatiale.fr/src/images_en_direct.php

NASCube

http://nascube.univ-lille1.fr/

lists the current web-based platforms that present imagery of the MSG2 SEVIRI, Himawari-8, and GOES-16 geostationary satellites. “The Wall” is the first platform to present synchronized views of these three satellites. It acquires the data of the three geostationary satellites as they are made available on the distribution platforms. The Level1B data are processed according to the flowchart in Figure 14 within a few minutes to generate all the animated global mosaics and subsets. The acquisition step uses simultaneously on three Linux-based computing nodes with 24 cores each, while the data processing is carried out on a computing node with 32 cores and 96 GB of RAM. “The Wall” web platform (http://earth2day.com/TheWall) homepage presents a global animation of the Earth in plate-Carree projection, in which the views of the three satellites are alive, as illustrated by the snapshot in Figure 15. The central panel is surrounded by focuses on specific Earth regions. In addition, a gallery (see

Table 3. List of events presented in “The Wall” gallery.

Events and hyperlinks to

http://earth2day.com/TheWall/Gallery/gallery.html

A huge sandstorm crossing the Atlantic in June 2020

http://earth2day.com/TheWall/Gallery/Movies/SANDSTORM20200611140020200620160016F.html

Huge fires in Canada in June 2019

http://earth2day.com/TheWall/Gallery/Movies/NORTHAMERIC2201905280000201906032355.html

One km accuracy image showing multiple fires in Amazonia in August 2019

http://earth2day.com/TheWall/Gallery/Movies/AMAZONI1km201908110000201908172300.html

Wild fires in the South East of Australia in January 2020

http://earth2day.com/TheWall/Gallery/Movies/VICT_OSW202001010000202001080700.html

Huge sand storm covering Australia in January 2020

http://earth2day.com/TheWall/Gallery/Movies/AUSTRALI2202001100600202001130600.html

Huge sand storm starting in Mongolia and crossing China toward Japan in May 2020

http://earth2day.com/TheWall/Gallery/Movies/MONGOLIASANDSTORM202005110600202005130540.html

) shows exceptional events and will be updated periodically. “The Wall” gives quick access to various areas of the globe, where the user can admire the dancing clouds over the equatorial regions, measure the strength of frequent immense desert sandstorms, the extent of the large forest fires, and the path of tropical storms. Being able to see the Earth alive on a wall gives us all a vision of both its finite size and its fragility.

4. Conclusions and Perspectives, and “The Wall” Website

“The Wall” is the first attempt to provide an operational platform to visualize in nearly real-time all the atmospheric events occurring on our planet (±70.0°). The platform can be easily tuned to the needs of the user and can be extended to include additional algorithms, such as BRDF corrections. The ANN-based reconstruction of the missing visible bands can be implemented in a straightforward manner to postprocess any image from the geostationary archives. The system has been running for several months with minimal human maintenance. We are eagerly waiting for the launch of the next generation of METEOSAT (MTG-I) by the end of 2021, which has an onboard "Flexible Combined Imager (FCI)” instrument. This will make “The Wall” homogeneous in spatial resolution and will allow a complete synchronization of its animations.