-
Notifications
You must be signed in to change notification settings - Fork 0
/
week6.qmd
196 lines (115 loc) · 12 KB
/
week6.qmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
# Week 6 Introduction to Google Earth Engine {.unnumbered}
This is a very exciting week that I could not physically attend since I was at the CUSP Data Dive (Datathon), GEE is something that interested me before selecting this module. This week we are introduced to the basic concepts of GEE and JavaScript.
## JavaScript and GEE Basics
Google Earth Engine is a cloud platform developed by Google for scientific data analysis. [@reis2020primer] GEE can handle large datasets in a relatively faster than many other traditional commercial software, its mature integration with JavaScript also enables new users to master the tool suite with a shallower learning curve.[@cardille2022cloud] JavaScript is the language used in GEE code editor, it is an essential tool for front-end development, enabling developers to implement complex and interactive websites. JavaScript also interacts with HTML and CSS to style and build the user interface. I have also summarised some useful introduction materials for learning GEE [here in the term summary](summary.qmd).
Here is a basic block to code a function:
``` js
function sayHi(name) {
alert("Hi," + name + "!");
}
sayHi(" guys, "); // Pops up "Hi, guys, !"
```
## GEE Case Study Bristol
First, I gathered air quality data over Bristol. But the resolution from Sentinel 5P was not ideal, thus I zoomed out in order to find places with high NO2 levels, in the end, even London is included. At this resolution, it is obvious that we will need more image processing and better resolution to have fine-grained data on urban air quality
### Air Quality, SAVI, ARVI, NDVI and NDWI
In this introductory study of GEE, I have examined the NO2 from Sentinel 5P and calculated NDVI, NDWI, SAVI and ARVI for Bristol. Time from the data used are Sentinel 2 and Sentinel 5P (for NO2) from 2021-Jan-01 to 2023-Dec-30, I made a date slider for the user to drag to select which data from a 30-day window. Rarely, the selection won't be valid, that is because the filter that I coded for the cloud is very low 5, and the Sentinel 2 images tend to be more cloudy for Bristol, please refresh the webpage and try another month. Most of the time it is working, due to the confined time-frame, I will be optimising this app and its UI in the future. For now, NO2 is not bounded, as the Sentinel 5P data is low-resolution compared to Sentinel 2, thus NDVI, NDWI, SAVI and ARVI are bounded within the Bristol image, where as NO2 layer is not.
### Putting Everything Together
Here is the [link to the lookup app](https://ee-mengyu-derek-ding.projects.earthengine.app/view/brsitol-ndvindwiarvisavi-lookup) for Bristol, it is also embedded as follows:
<p align="left">
<iframe title="gee_app_01" width="680" height="700" src="https://ee-mengyu-derek-ding.projects.earthengine.app/view/brsitol-ndvindwiarvisavi-lookup" title="GEE App Bristol Look-up" frameborder="0">
</iframe>
</p>
For the legend in this GEE app, I have modified and derived the code from: [Open Geo Blog](https://mygeoblog.com/2016/12/09/add-a-legend-to-to-your-gee-map/). The colour code gradients for the four metrics are gathered from this [website](https://www.color-hex.com/). The boundary for the app is Greater Bristol (or the Global Administrative Unit (GAUL) Layer Level 2 that contains Bristol) which is larger than the shapefile used to clip for the city boundary in the week 6 practical or the following screenshots.
For week 6, the author downloaded the [Bristol City Boundary](https://opendata.bristol.gov.uk/datasets/bcc::bristol-boundary/explore) shape file, uploaded to the assets and clipped out images with only Bristol. This is smaller compared to the GAUL level 2 boundary used in the GEE app.
::: {.image-container style="display: grid; grid-template-columns: repeat(2,1fr); grid-template-rows: repeat(2,1fr); gap:5px;"}
<img src="images/wk6/wk6_ndvi.png" alt="NDVI for Bristol" height="310px" width="340px"/>
<img src="images/wk6/wk6_ndwi.png" alt="NDWI for Bristol" height="310px" width="340px"/>
<img src="images/wk6/wk6_savi.png" alt="SAVI for Bristol" height="310px" width="340px"/>
<img src="images/wk6/wk6_arvi.png" alt="ARVI for Bristol" height="310px" width="340px"/>
:::
<p align="center" class="grid-caption">
NDVI, NDWI, SAVI and ARVI bounded by the Bristol Boundary
</p>
## Principle Component Analysis
GEE can run PCA for satellite data, but the browser froze for a few minutes before the results are ready.
::: {.image-container style="display: grid; grid-template-columns: repeat(2,1fr); grid-template-rows: repeat(4,1fr); gap:5px;"}
::: image-item
{width="340" height="310"}
<p align="center" class="caption">
Bristol Boundary
</p>
:::
::: image-item
{width="340" height="310"}
<p align="center" class="caption">
Bristol Clip
</p>
:::
::: image-item
{width="340" height="310"}
<p align="center" class="caption">
GLCM
</p>
:::
::: image-item
{width="340" height="310"}
<p align="center" class="caption">
P1 and P2
</p>
:::
::: image-item
{width="340" height="230"}
<p align="center" class="caption">
PC1
</p>
:::
::: image-item
{width="341"}
<p align="center" class="caption">
PC2
</p>
:::
::: image-item
{width="340"}
<p align="center" class="caption">
PC5
</p>
:::
::: image-item
{width="340" height="230"}
<p align="center" class="caption">
PC6
</p>
:::
:::
PC1 explains 77.17% of the variance in the dataset. With my visual assessment of the GLCM and PCA results, it is obvious PC1 and PC2 captures high NDWI and highly building dense areas repectively, whereas GLCM captures high-reflectance areas, it performs well on flat rooftops, making it suitable for identifying buildings.[@christaki2022building] Such results are essential as the input of more complex models, such as classification models with dimensionalities reduced to PC1, PC2 and PC3 (together they explain 95.88% of variances). However, PC4 to PC6, or the other PCs on condition that they are still meaningful and not rounded to 0, can provide crucial insights to anomalies and noises.
| PC | Percentage Variance Explained |
|------------|:-----------------------------:|
| PC1 | 77.17 |
| PC2 | 11.40 |
| PC3 | 7.31 |
| PC4 | 2.41 |
| PC5 | 1.58 |
| PC6 | 0.11 |
| PC7 | 0.03 |
| PC8 - PC20 | 0.00 |
: PCA Variance Results
## More on GEE and Literature Review
Google Earth Engine (GEE) offers a wide range of Earth data for researchers to use with easy access, this has led to a surge in publication using GEE. As a ECE major, I first explored the how GEE can support a vast user group scattered around the planet, and then looked into the recent trend in the scientific community with GEE.
### GEE Capabilities and Trend in Scientific Research
The [system architecture](https://www.sciencedirect.com/science/article/pii/S0034425717302900?via%3Dihub) of GEE is well designed to offer services at a large scale. This design allows the GEE to host massive amount of data while being scalable. Of course, being the birth-company for the famous MapReduce, it is obvious that hadnling massive amount of data is what Google does best. GEE offers a seamless integration with machine learning, whether it is using the API from python or code editor, the ML-workflow is well-integrated.
Due to its advantages, [more members from the scientific research](https://link.springer.com/article/10.1007/s12145-023-01035-2#Fig6)community have been adapting GEE into their research in recent years.[@pham2023trends] GEE can provide geo-spatial analysis at a large-scale for free and its ample image series datasets are attractive features that drive a growing usage from the researchers. Furthermore, GEE can meet the needs of urban researcher since the nature or urban science research requires multi-faceted data and spatiotemporal data at large scale. GEE's abundant city bird-view images in with sufficient temporal richness, encourages urban researcher to utilise it in conjunction with other urban data, sparking novel research findings about our growing metropolitan environment.
<div class="image-container" style="display: grid; grid-template-columns: repeat(2,1fr); gap:5px;">
{width="340" height="290"}
{width="340" height="290"}
`</div><p align="center" class="grid-caption"> System architecture of GEE (left)`{=html} [@gorelick2017google] `and Annual number of articles published by the top ten journals`{=html} [@pham2023trends]
</p>
### GEE Applications with GLCM
In the context of remote sensing for cities, GEE is employed in a diverse range of new applications and publications.[@amani2020google] During the practical, I dived into the GLCM (Grey-Level Co-occurrence Matrix) and its applications in identifying potentials buildings in remote sensing datasets, this enabled me to have a wider range of methodology design for both my future research and the group assessment.
There are many unique publications regarding GLCM, in the aspects of the theoretical construct of GLCM methods, @lan2018study proposed a method to optimse the multi-scale parameters for the GLCM texture window. On the application side, researchers developed methods that combine the GLCM with SNIC (Simple Non-Iterative Clustering) to process the data for classification input in LULC (Land Use and Land Cover) research. [@tassi2020object]
Other researchers have extended the GLCM methods to process multi-layered data and combined the modified GLCM with SVM (Support Vector Machine) to analyse collapsed buildings.[@moya20193d] These new developments highlight the innovative usage and adaptation of GLCM for remote sensing research, however, it also indicates that such numeric methods should be handled with caution as its limitations may hinder the analysis accuracy if proper fine-tuning are not performed.
## Reflection
Employing Google Earth Engine (GEE) certainly rekindles memories of my extensive collaborative efforts on diverse projects in Melbourne. During those projects, I relied on Nearmap (which was really high-res!) to coordinate with the design team and worked closely alongside engineers and GIS specialists. Together, we organised design modifications and participated the digital transformation of the construction processes. Unlike those days when my role was more supervisory and collaborative, I now find myself directly engaged with GEE, utilizing its powerful remote sensing capabilities for urban analysis and environmental applications.
Although GEE offers a powerful tool suite for urban research, there may be certain limitations. First, it fosters a level of vendor lock-in where extensive use of GEE can lead to dependence on Google's infrastructure, terms and conditions, posing risks to the research workflow. Furthermore, GEE is challenging to work with offline. Researchers with poor internet connection or working on remote sites may struggle to gain access to GEE. Data sharing with GEE can be more complicated than other platforms. Sharing custom-made shapefiles might turn out to be a complex process due to GEE's data management policies and permissions. Finally but not the least, GEE can be difficult for non-coders to learn, what is available to it JavaScript library can be occasionally counter-intuitive.
Despite GEE's limitations, its versatile and lightweight nature, combined with its openness and collection of data, still weighs out its limitations. Researching with GEE on urban topics is still promising as the platform itself is robust and powerful for large datasets, besides, the GEE community is open and vibrant, there are many [resources](summary.qmd) and people for turn to for collaboration and assistance.