Extremely long response time with graph_from_place OSMNX
Question:
I’m trying to download the map of Mexico to avoid querying using save_graphml and avoiding long response times in the graph_from_place, but I’ve already left this code running for almost six hours and absolutely nothing happens.
import osmnx as ox
ox.config(use_cache=True, log_console=True)
G = ox.graph_from_place('Mexico', network_type = 'drive', simplify=False)
G = ox.add_edge_speeds(G)
G = ox.add_edge_travel_times(G)
ox.save_graphml(G, '/var/www/html/repmexico.graphml')
print("Success!!!")
Today I am trying to run the code on a server with 74GB of RAM and (Intel xeon x5570) X2
(I know that due to the stipulated area the time is long, but what I wanted to know is if there is an alternative to this procedure or if there is a way to optimize so that the creation of the map is a little faster or if there is another way to load maps to route with osmnx and networkx without using queries to servers)
Answers:
I’ve already left this code running for almost six hours and absolutely nothing happens.
A lot has been happening! Don’t believe me? You ran ox.config(log_console=True), so look at your terminal and watch what’s happening while it runs. You’ll see a line like "2021-10-14 13:05:39 Requesting data within polygon from API in 1827 request(s)"… so you are making 1,827 requests to the Overpass server and the server is asking you to pause for rate limiting between many of those requests.
I know that due to the stipulated area the time is long, but what I wanted to know is if there is an alternative to this procedure or if there is a way to optimize so that the creation of the map is a little faster or if there is another way to load maps to route with osmnx and networkx without using queries to servers
Yes. This answer provides more details. There are tradeoffs between 1) model precision vs 2) area size vs 3) memory/speed. For faster modeling, you can load the network data from a .osm XML file instead of having to make numerous calls to the Overpass API. I’d also recommend using a custom_filter as described in the linked answer. OSMnx by default divides your query area into 50km x 50km pieces, then queries Overpass for each piece one a time to not exceed the server’s per-query memory limits. You can configure this max_query_area_size parameter, as well as the server memory allocation, if you prefer to use OSMnx’s API querying functions rather than its from-file functionality.
I’m trying to download the map of Mexico to avoid querying using save_graphml and avoiding long response times in the graph_from_place, but I’ve already left this code running for almost six hours and absolutely nothing happens.
import osmnx as ox
ox.config(use_cache=True, log_console=True)
G = ox.graph_from_place('Mexico', network_type = 'drive', simplify=False)
G = ox.add_edge_speeds(G)
G = ox.add_edge_travel_times(G)
ox.save_graphml(G, '/var/www/html/repmexico.graphml')
print("Success!!!")
Today I am trying to run the code on a server with 74GB of RAM and (Intel xeon x5570) X2
(I know that due to the stipulated area the time is long, but what I wanted to know is if there is an alternative to this procedure or if there is a way to optimize so that the creation of the map is a little faster or if there is another way to load maps to route with osmnx and networkx without using queries to servers)
I’ve already left this code running for almost six hours and absolutely nothing happens.
A lot has been happening! Don’t believe me? You ran ox.config(log_console=True), so look at your terminal and watch what’s happening while it runs. You’ll see a line like "2021-10-14 13:05:39 Requesting data within polygon from API in 1827 request(s)"… so you are making 1,827 requests to the Overpass server and the server is asking you to pause for rate limiting between many of those requests.
I know that due to the stipulated area the time is long, but what I wanted to know is if there is an alternative to this procedure or if there is a way to optimize so that the creation of the map is a little faster or if there is another way to load maps to route with osmnx and networkx without using queries to servers
Yes. This answer provides more details. There are tradeoffs between 1) model precision vs 2) area size vs 3) memory/speed. For faster modeling, you can load the network data from a .osm XML file instead of having to make numerous calls to the Overpass API. I’d also recommend using a custom_filter as described in the linked answer. OSMnx by default divides your query area into 50km x 50km pieces, then queries Overpass for each piece one a time to not exceed the server’s per-query memory limits. You can configure this max_query_area_size parameter, as well as the server memory allocation, if you prefer to use OSMnx’s API querying functions rather than its from-file functionality.