But like many things, we need more people coding, and more community support for those who are already coding.

@Andy Allan: as you’re mentioning this point: the pull request to remove those remaining slow parts in the map call is already out there, waiting to be reviewed, merged and deployed: https://github.com/zerebubuth/openstreetmap-cgimap/pull/142

@Komяpa : you’re still very welcomed to rerun your test setup to see how this change affects runtime, see this comment

Looks like you hit one of the hotkeys. Check the data menu

• Set building shape to circle ALT-Z
• Set building shape to rectangle ALT-R

I put together a small Overpass query to find such instances with exactly 1 from, 1 to element and at least 3 way members (that’s close enough for the purpose of this analysis):

http://overpass-turbo.eu/s/xjH

Caveat: query uses not yet released feature count_by_role (will be part of 0.7.55 release)

Actually I didn’t think my mass edits falled in “automated” category mentioned by the code of conduct you quoted.

If you take a look at the scope of said Automated Edits code of conduct, you’ll notice that your blog post exactly matches point 4 in the scope section:

“use of find-and-replace functionality using a standard editor such as JOSM or finding using services such as Overpass API and changing without reviewing cases individually”

It’s really not necessary to remove the trailing „m“ in Osm data for overpass turbo to work. Instead you should convert the relevant ele tags to a number. Roland wrote a blog post about this topic which explains all the details!

http://dev.overpass-api.de/blog/numbers.html

Given how much discussion there was on data privacy issues with regards to Pascal Neis’ HDYC tool (with the result of requiring people to logon and only providing for one user at a time), it’s really intimidating that you manually analyzed the mapping behavior of each and every single top 2000 contributor (!) and put it up out there in public as “Notes” column.

I’m pretty sure things like “sadly deceased Nov. 2016” for sure don’t belong in such a list. Also, I find comments like “badly overnoded import of Fukuoka woods” or “very badly inefficient waterways” not that helpful.

Any chance you remove this information altogether?

@tyr_asd: afaik, JOSM changeset viewer is essentially another frontend for OSMCha, the data source is the same, i.e. geohackers aws feed.

This is the logical way to get it, but isn’t usable from a script because the internal structure of the zip file isn’t easily predicted.

Maybe using a Github tarball and stripping the leading directory would do?

mkdir -p openstreetmap-carto && 
     curl -L https://api.github.com/repos/gravitystorm/openstreetmap-carto/tarball/v4.6.0 |
     tar xzf - -C openstreetmap-carto --strip=1

https://github.com/openstreetmap/openstreetmap-website/issues/1543 has some addntl details on the large number of anonymous notes last year.

I noticed that your demo loads vector tiles from tiles.mapbox.com in addition to the raster tiles from tile.openstreetmap.org (although you won’t see them). And then there’s the Mapbox API key, which I wouldn’t expect for such a demo.

Is this bounty is still relevant in its current form?

As we have found out in the meantime, the proper solution would be much more complex than just exposing the queue depth as a configurable parameter.

Anyone considering this topic should take a very close look at the detailed analysis posted by @apmon first: https://github.com/openstreetmap/mod_tile/pull/152#issuecomment-348805403

FYI: There’s some follow up discussion to this blog post on the osmf-talk mailing list: https://lists.openstreetmap.org/pipermail/osmf-talk/2017-December/thread.html

As a CoC would for sure affect all mailing lists, forums, etc. at some point in time, please consider moving this discussion to a place where everyone can participate, not only OSMF members. The talk mailing list might be a good start. Thank you.

Talking about efficiency. Recent measurements on a performance optimized branch indicate that a full day worth of queries can in fact be processed in a 25h timeframe on just ~ 2 CPU cores. This includes minutely updates, hourly area updates and gzip compression on the webserver.

Yet, the production instance at overpass-api.de frequently uses 6 out of 8 CPU cores, sometimes even 7 out of 8 cores according to munin. So, 4-5 cores are busy doing some work, which adds no real value. Even worse, users are getting “Too many queries” or “Timeout” error messages, although more resources could be made available for more productive purposes. It’s really only a matter of addressing the respective Github issues.

Most disturbing figure in this survey: less than 20% use OSM because they believe it’s the best map in their area (7 out of 37).

Schneller als achavi

… falls das Changeset im Amazon S3 Cache drin ist, ansonsten wird exakt dieselbe Query ausgeführt wie in Achavi, allerdings auf der Overpass Instanz von Mapbox. Da ist nicht ganz so viel los ;)

A new Endless Achavi demo is up, see this post for details.

It’s giving a different perspective to the question: is caching really worth it? Should we rather spend more time improving Overpass performance instead?

JOSM indeed supports downloading along a gpx track, but it would do so by downloading all data in a number of smaller bboxes rather than the data you’re only interested in.

Well, I guess this needs to be merged into the official Overpass API branch first. So far, this is only in my own personal branch and not part of the official release.

Zum Cachen der Changesets: so etwas gab es in ähnlicher Form schon vor 3 Jahren in Overpass API mit den Augmented Diffs, allerdings wurde es mit Version 0.7.50 eingestellt. Aus den Release Notes von damals:

“The Augmented Diffs are redesigned to be always generated on the fly. This is because the Augmented Diffs have been piling up on the server to almost a terabyte of data and we are running out of space.”

Die vorgestellte Lösung fängt aktuell irgendwo im Frühjahr 2017 an, noch ältere (oder auch sehr junge) Changesets müssen also nachwievor über Overpass angefragt werden (Mapbox hat eine eigene Instanz). Insgesamt hat das Cachen aber den großen Vorteil, Last von der Hauptinstanz zu nehmen. Wieviel Platz das Cachen aktuell aber wirklich beansprucht, ist leider nicht überliefert.

Das Thema ist insgesamt etwas umfangreicher und wird auch auf verschiedenen Github-Issues diskutiert (Overpass API, OSM main page und Achavi). Es gibt auch immer noch neue Ideen dazu, und Cachen ist vielleicht auch nur ein möglicher Ansatz. :)

@Adityo: why did you create a new thread for this very same topic on GIS SE. Please keep it all at one location.