Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integration tests for Geofabrik Overpass API #959

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

nataliejschultz
Copy link
Contributor

Addressing e-mission/e-mission-docs#1036

As of now, there are three tests in the TestOverpass.py module:

  1. test_overpass, which compares a call to the free version of overpass with a call to the geofabrik version
  2. test_get_stops_near, which passes a set of coordinates (as a list inside a dictionary) to get_stops_near in match_stops.py.
  3. test_get_predicted_transit_mode, which passes two sets of coordinates to get_predicted_transit_mode in match_stops.py.

Adding TestOverpass.py
Workflow file runs the shell script.

Shell script sets up emission environment and runs TestOverpass.py.
Adding commands to workflow to get a sense of why it's not running.
Running with the correct path this time (I think)
Adding runner permissions for setup script.
Didn't set correct path to file to add permissions.
@nataliejschultz
Copy link
Contributor Author

Currently, tests 2 and 3 only call the free version of the API to make calls. There are a few ways that come to mind to change this:

  1. Add hard-coded query to geofabrik as seen in test 1 and compare with the free call (not preferred)
  2. Add a similar block of code to match_stops.py as we did for nominatim.py using environment variables:
try:
    NOMINATIM_QUERY_URL = os.environ.get("NOMINATIM_QUERY_URL")
    logging.info(f"NOMINATIM_QUERY_URL: {NOMINATIM_QUERY_URL}")
    print("Nominatim Query URL Configured:", NOMINATIM_QUERY_URL)

    if NOMINATIM_QUERY_URL is None:
        raise Exception("Nominatim query url not configured")
except:
    print("Nominatim URL not configured, place decoding must happen on the client")
  1. Add a file called overpass_server.json with the geofabrik query URL and work with the current method of setting the query URL in match_stops.py
try:
    config_file = open('conf/net/ext_service/overpass_server.json')
except:
    print("overpass not configured, falling back to default overleaf.de")
    config_file = open('conf/net/ext_service/overpass_server.json.sample')
config_data = json.load(config_file)
url = config_data["url"]

I think the answer is a combination of 2 and 3, but I'd like some feedback from @shankari on my tests before deciding what to do

@nataliejschultz nataliejschultz marked this pull request as ready for review February 12, 2024 22:03
@shankari
Copy link
Contributor

shankari commented Mar 4, 2024

@nataliejschultz high-level question: why do we need to keep the configuration in a file? There is a single variable in the file.
Recall that for the nominatim use case, we switched the config from a file to an environment variable. I would suggest using the same pattern here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Review done; Changes requested
2 participants