I took a look at the issues you posted in the Overpass repo on Github. The incorrect messaging in shutdown.sh got a quick fix but that script is due for much more attention soon.

And I noted the issue you had with applying global diffs to the extract database. I think there’s a lock file issue lurking in there that needs more investigation. But that wouldn’t address the bigger issue of diffs needing to match the extract.

If you bump into any other configuration or operational issues, I’d be happy to help with those too!

Nice write-up! I haven’t tried to get Overpass working with smaller extracts, so it’s good to see how that worked out.

I have been actively working on the updated scripts in my fork on Github. At the moment, that means that fetch_osc.sh got the most attention. Other scripts will get more work soon, starting with apply_osc_to_db.sh.

If you happen to bump into any issues with the scripts, I’d love to hear about them!

Thanks for writing up these comments on features that don’t have precise boundaries!

I think about this as well in the context of natural=desert, which has unfortunately been conflated with natural=sand. It’s true that there are some sandy deserts, but most of the world’s deserts are rocky rather than sandy. And even deserts have scrub. Sadly, the wiki entry for natural=desert isn’t helping this.

Deserts are important places and have rough boundaries defined by cultural, biological, and geological distinctions. And like oceans or many other large natural features, these boundaries are indistinct and subject to different interpretations.

In spite of the fact that the wiki says natural=desert applies only to areas, there are more deserts mapped as nodes than as areas. And I imagine that many of those areas are either conflated with natural=sand or have issues with the verifiability of the borders.

It would be nice to be able to map deserts as nodes with some indication of the size of the area. The sqkm tag is a possibility, but I see the issues with it as well. Just as it’s hard to get consensus on the boundaries of features like this, it is hard to get consensus on the precise area as well. Does it matter if the Mojave Desert is 65,000 or 130,000 sq km? Maybe that doesn’t make too much difference to a renderer, but it could lead to endless “corrections” to the tag.

It seems like mapping large, poorly defined areas is an old problem in OSM. If we do find a solution, it would improve mapping for many different types of features.

I’m not sure what you’re asking @lindsaywoods219. The idea of using a small, inexpensive computer to run Overpass is to have something convenient for local usage and not to burden the larger public Overpass servers with lots of speculative queries.

In general, this is not a solution for a large scale or high-reliability Overpass system. Query response times are OK for small volumes but a small system like this does not handle large volumes of queries well. And there are still some reliability issues. For example, a power outage or other unplanned shutdown can still result in corrupted data that needs to be manually repaired. Often, that means starting from scratch with a fresh database copy.

If anyone is curious about the withdrawn and historical GNIS records that are currently mapped in OSM, I have Overpass queries that will find them.

Withdrawn GNIS records in OSM - Overpass Query

Historical GNIS records in OSM - Overpass Query

Any attempt to correct these features should be done with care and should take several things into consideration.

• Many GNIS records have been withdrawn because they were duplicates of other records. Additional research is necessary to identify and apply the correct GNIS Feature ID.
• Withdrawn and historical features may exist in some form that can still be mapped in OSM (e.g., as abandoned: or historical=yes). Additional research is necessary to confirm the current state of the feature.
• Where withdrawn or historical features still exist in some form, they may have been altered or repurposed in a way that the former GNIS data no longer applies. Additional research is necessary to confirm the current attributes of the features.
• There are some false positives where GNIS withdrew records for airports that still exist and may still be in use. (GNIS no longer maintains records for airports.)
• Some features were marked as historical before GNIS archived several feature classes in 2021. These features will appear to have been withdrawn from the current GNIS data set (but have been identified as “historical” in the query above). Any review of features in the archived classes must use the archived GNIS data set.

That’s pretty cool! Thanks for doing the analysis!

Given the scale of the data, the overlap between categories at one standard deviation is remarkably small.

This would be a good topic to post on the OpenStreetMap Community Forum to see if we can get some consensus on how to map mountains whose names differ from the names of their summits.