-
-
Notifications
You must be signed in to change notification settings - Fork 474
/
osm2pgsql-replication.1
395 lines (327 loc) · 9.85 KB
/
osm2pgsql-replication.1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
.TH "OSM2PGSQL-REPLICATION" "1" "2.0.0" "" ""
.SH NAME
osm2pgsql-replication \- osm2pgsql database updater
.SH SYNOPSIS
.B osm2pgsql-replication
[-h] {init,update,status} ...
.SH DESCRIPTION
Update an osm2pgsql database with changes from a OSM replication server.
.br
.br
This tool initialises the updating process by looking at the import file
.br
or the newest object in the database. The state is then saved in a table
.br
in the database. Subsequent runs download newly available data and apply
.br
it to the database.
.br
.br
See the help of the \(cqinit\(cq and \(cqupdate\(cq command for more information on
.br
how to use osm2pgsql\-replication.
.SH OPTIONS
.SS
\fBSub-commands\fR
.TP
\fBosm2pgsql-replication\fR \fI\,init\/\fR
Initialise the replication process.
.TP
\fBosm2pgsql-replication\fR \fI\,update\/\fR
Download newly available data and apply it to the database.
.TP
\fBosm2pgsql-replication\fR \fI\,status\/\fR
Print information about the current replication status, optionally as JSON.
.SH OPTIONS 'osm2pgsql-replication init'
usage: osm2pgsql-replication init [-h] [-q] [-v] [-d DB] [-U NAME] [-H HOST]
[-P PORT] [-p PREFIX]
[--middle-schema SCHEMA] [--schema SCHEMA]
[--osm-file FILE | --server URL]
[--start-at TIME]
Initialise the replication process.
.br
.br
This function sets the replication service to use and determines from
.br
which date to apply updates. You must call this function at least once
.br
to set up the replication process. It can safely be called again later
.br
to change the replication servers or to roll back the update process and
.br
reapply updates.
.br
.br
There are different methods available for initialisation. When no
.br
additional parameters are given, the data is initialised from the data
.br
in the database. If the data was imported from a file with replication
.br
information and the properties table is available (for osm2pgsql >= 1.9)
.br
then the replication from the file is used. Otherwise the minutely
.br
update service from openstreetmap.org is used as the default replication
.br
service. The start date is either taken from the database timestamp
.br
(for osm2pgsql >= 1.9) or determined from the newest way in the database
.br
by querying the OSM API about its creation date.
.br
.br
The replication service can be changed with the \(cq\-\-server\(cq parameter.
.br
To use a different start date, add \(cq\-\-start\-at\(cq with an absolute
.br
ISO timestamp (e.g. 2007\-08\-20T12:21:53Z). When the program determines the
.br
start date from the database timestamp or way creation date, then it
.br
subtracts another 3 hours by default to ensure that all new changes are
.br
available. To change this rollback period, use \(cq\-\-start\-at\(cq with the
.br
number of minutes to rollback. This rollback mode can also be used to
.br
force initialisation to use the database date and ignore the date
.br
from the replication information in the file.
.br
.br
The initialisation process can also use replication information from
.br
an OSM file directly and ignore all other date information.
.br
Use the command \(cqosm2pgsql-replication \-\-osm\-file <filename>\(cq for this.
.TP
\fB\-q\fR, \fB\-\-quiet\fR
Print only error messages
.TP
\fB\-v\fR, \fB\-\-verbose\fR
Increase verboseness of output
.TP
\fB\-d\fR DB, \fB\-\-database\fR DB
Name of PostgreSQL database to connect to or conninfo string
.TP
\fB\-U\fR NAME, \fB\-\-username\fR NAME
PostgreSQL user name
.TP
\fB\-H\fR HOST, \fB\-\-host\fR HOST
Database server host name or socket location
.TP
\fB\-P\fR PORT, \fB\-\-port\fR PORT
Database server port
.TP
\fB\-p\fR PREFIX, \fB\-\-prefix\fR PREFIX
Prefix for table names (default 'planet_osm')
.TP
\fB\-\-middle\-schema\fR SCHEMA
Name of the schema to store the table for the replication state in
.TP
\fB\-\-schema\fR SCHEMA
Name of the schema for the database
.TP
\fB\-\-osm\-file\fR FILE
Get replication information from the given file.
.TP
\fB\-\-server\fR URL
Use replication server at the given URL
.TP
\fB\-\-start\-at\fR TIME
Time when to start replication. When an absolute timestamp (in ISO format) is given, it will be used. If a number is given, then replication starts the number of minutes before the known date of the database.
.SH OPTIONS 'osm2pgsql-replication update'
usage: osm2pgsql-replication update update [options] [-- param [param ...]]
Download newly available data and apply it to the database.
.br
.br
The data is downloaded in chunks of \(cq\-\-max\-diff\-size\(cq MB. Each chunk is
.br
saved in a temporary file and imported with osm2pgsql from there. The
.br
temporary file is normally deleted afterwards unless you state an explicit
.br
location with \(cq\-\-diff\-file\(cq. Once the database is up to date with the
.br
replication source, the update process exits with 0.
.br
.br
Any additional arguments to osm2pgsql need to be given after \(cq\-\-\(cq. Database
.br
and the prefix parameter are handed through to osm2pgsql. They do not need
.br
to be repeated. \(cq\-\-append\(cq and \(cq\-\-slim\(cq will always be added as well.
.br
.br
Use the \(cq\-\-post\-processing\(cq parameter to execute a script after osm2pgsql has
.br
run successfully. If the updates consists of multiple runs because the
.br
maximum size of downloaded data was reached, then the script is executed
.br
each time that osm2pgsql has run. When the post\-processing fails, then
.br
the entire update run is considered a failure and the replication information
.br
is not updated. That means that when 'update' is run the next time it will
.br
recommence with downloading the diffs again and reapplying them to the
.br
database. This is usually safe. The script receives two parameters:
.br
the sequence ID and timestamp of the last successful run. The timestamp
.br
may be missing in the rare case that the replication service stops responding
.br
after the updates have been downloaded.
.TP
\fBparam\fR
Extra parameters to hand in to osm2pgsql.
.TP
\fB\-\-diff\-file\fR FILE
File to save changes before they are applied to osm2pgsql.
.TP
\fB\-\-max\-diff\-size\fR \fI\,MAX_DIFF_SIZE\/\fR
Maximum data to load in MB (default: 500MB)
.TP
\fB\-\-osm2pgsql\-cmd\fR \fI\,OSM2PGSQL_CMD\/\fR
Path to osm2pgsql command
.TP
\fB\-\-once\fR
Run updates only once, even when more data is available.
.TP
\fB\-\-post\-processing\fR SCRIPT
Post\-processing script to run after each execution of osm2pgsql.
.TP
\fB\-q\fR, \fB\-\-quiet\fR
Print only error messages
.TP
\fB\-v\fR, \fB\-\-verbose\fR
Increase verboseness of output
.TP
\fB\-d\fR DB, \fB\-\-database\fR DB
Name of PostgreSQL database to connect to or conninfo string
.TP
\fB\-U\fR NAME, \fB\-\-username\fR NAME
PostgreSQL user name
.TP
\fB\-H\fR HOST, \fB\-\-host\fR HOST
Database server host name or socket location
.TP
\fB\-P\fR PORT, \fB\-\-port\fR PORT
Database server port
.TP
\fB\-p\fR PREFIX, \fB\-\-prefix\fR PREFIX
Prefix for table names (default 'planet_osm')
.TP
\fB\-\-middle\-schema\fR SCHEMA
Name of the schema to store the table for the replication state in
.TP
\fB\-\-schema\fR SCHEMA
Name of the schema for the database
.SH OPTIONS 'osm2pgsql-replication status'
usage: osm2pgsql-replication status [-h] [-q] [-v] [-d DB] [-U NAME] [-H HOST]
[-P PORT] [-p PREFIX]
[--middle-schema SCHEMA] [--schema SCHEMA]
[--json]
Print information about the current replication status, optionally as JSON.
.br
.br
Sample output:
.br
.br
2021\-08\-17 15:20:28 [INFO]: Using replication service 'https://planet.openstreetmap.org/replication/minute', which is at sequence 4675115 ( 2021\-08\-17T13:19:43Z )
.br
2021\-08\-17 15:20:28 [INFO]: Replication server's most recent data is <1 minute old
.br
2021\-08\-17 15:20:28 [INFO]: Local database is 8288 sequences behind the server, i.e. 5 day(s) 20 hour(s) 58 minute(s)
.br
2021\-08\-17 15:20:28 [INFO]: Local database's most recent data is 5 day(s) 20 hour(s) 59 minute(s) old
.br
.br
.br
With the \(cq\-\-json\(cq option, the status is printed as a json object.
.br
.br
{
.br
"server": {
.br
"base_url": "https://planet.openstreetmap.org/replication/minute",
.br
"sequence": 4675116,
.br
"timestamp": "2021\-08\-17T13:20:43Z",
.br
"age_sec": 27
.br
},
.br
"local": {
.br
"sequence": 4666827,
.br
"timestamp": "2021\-08\-11T16:21:09Z",
.br
"age_sec": 507601
.br
},
.br
"status": 0
.br
}
.br
.br
.br
\(cqstatus\(cq is 0 if there were no problems getting the status. 1 & 2 for
.br
improperly set up replication. 3 for network issues. If status ≠ 0, then
.br
the \(cqerror\(cq key is an error message (as string). \(cqstatus\(cq is used as the
.br
exit code.
.br
.br
\(cqserver\(cq is the replication server's current status. \(cqsequence\(cq is it's
.br
sequence number, \(cqtimestamp\(cq the time of that, and 'age_sec' the age of the
.br
data in seconds.
.br
.br
\(cqlocal\(cq is the status of your server.
.TP
\fB\-\-json\fR
Output status as json.
.TP
\fB\-q\fR, \fB\-\-quiet\fR
Print only error messages
.TP
\fB\-v\fR, \fB\-\-verbose\fR
Increase verboseness of output
.TP
\fB\-d\fR DB, \fB\-\-database\fR DB
Name of PostgreSQL database to connect to or conninfo string
.TP
\fB\-U\fR NAME, \fB\-\-username\fR NAME
PostgreSQL user name
.TP
\fB\-H\fR HOST, \fB\-\-host\fR HOST
Database server host name or socket location
.TP
\fB\-P\fR PORT, \fB\-\-port\fR PORT
Database server port
.TP
\fB\-p\fR PREFIX, \fB\-\-prefix\fR PREFIX
Prefix for table names (default 'planet_osm')
.TP
\fB\-\-middle\-schema\fR SCHEMA
Name of the schema to store the table for the replication state in
.TP
\fB\-\-schema\fR SCHEMA
Name of the schema for the database
.SH SEE ALSO
* osm2pgsql website (https://osm2pgsql.org)
.br
* osm2pgsql manual (https://osm2pgsql.org/doc/manual.html)