Pickle fails to load Pyproj objects

[alexstaravoitau]

I'm working on adding Pyproj to my app that embeds its own Python environment, and it looks like Pyproj doesn't work well with pickle. Not sure if this is a known issue — I've tried searching, but didn't find anything (apologies if this has been mentioned somewhere).

Here is a couple of examples that reproduce the issue, where I create Pyproj objects and try to pickle and unpickle them. Let me know if these need to be treated as separate issues, I wasn't sure so figured I'd include both here for now:

Example 1:

from pyproj import CRS
import pickle

crs = CRS.from_cf(
    {
        "grid_mapping_name": "transverse_mercator",
        "semi_major_axis": 6377563.396,
        "inverse_flattening": 299.3249646,
        "longitude_of_prime_meridian": 0.0,
        "latitude_of_projection_origin": 49.0,
        "longitude_of_central_meridian": -2.0,
        "scale_factor_at_central_meridian": 0.9996012717,
        "false_easting": 400000.0,
        "false_northing": -100000.0,
    }
)

with open('temp.pickle', 'wb') as f:
    pickle.dump(crs, f)

with open('temp.pickle', 'rb') as f:
    crs_unpickled = pickle.load(f)

print(crs == crs_unpickled)

Example 2:

from pyproj.crs import VerticalCRS
from pyproj.crs.coordinate_system import VerticalCS
import pickle

vertcrs = VerticalCRS(
    name="NAVD88 height",
    datum="North American Vertical Datum 1988",
    vertical_cs=VerticalCS(),
    geoid_model="GEOID12B",
)

with open('temp.pickle', 'wb') as f:
    pickle.dump(vertcrs, f)

with open('temp.pickle', 'rb') as f:
    vertcrs_unpickled = pickle.load(f)

print(vertcrs == vertcrs_unpickled)

Problem description

Both examples throw errors while loading the object from pickle file.

Example 1 throws multiple CRSErrors ("Invalid coordinate operation name", "Invalid coordinate operation string"):

---------------------------------------------------------------------------
CRSError                                  Traceback (most recent call last)
pyproj/_crs.pyx in pyproj._crs.CoordinateOperation.from_string()

pyproj/_crs.pyx in pyproj._crs.CoordinateOperation.from_name()

CRSError: Invalid coordinate operation name: {"$schema": "https://proj.org/schemas/v0.2/projjson.schema.json", "type": "ProjectedCRS", "name": "undefined", "base_crs": {"$schema": "https://proj.org/schemas/v0.2/projjson.schema.json", "type": "GeographicCRS", "name": "undefined", "datum": {"type": "GeodeticReferenceFrame", "name": "undefined", "ellipsoid": {"name": "undefined", "semi_major_axis": 6377563.396, "inverse_flattening": 299.3249646}, "prime_meridian": {"name": "undefined", "longitude": 0}}, "coordinate_system": {"subtype": "ellipsoidal", "axis": [{"name": "Longitude", "abbreviation": "lon", "direction": "east", "unit": "degree"}, {"name": "Latitude", "abbreviation": "lat", "direction": "north", "unit": "degree"}]}}, "conversion": {"$schema": "https://proj.org/schemas/v0.2/projjson.schema.json", "type": "Conversion", "name": "unknown", "method": {"name": "Transverse Mercator", "id": {"authority": "EPSG", "code": 9807}}, "parameters": [{"name": "Latitude of natural origin", "value": 49, "unit": "degree", "id": {"authority": "EPSG", "code": 8801}}, {"name": "Longitude of natural origin", "value": -2, "unit": "degree", "id": {"authority": "EPSG", "code": 8802}}, {"name": "Scale factor at natural origin", "value": 0.9996012717, "unit": "unity", "id": {"authority": "EPSG", "code": 8805}}, {"name": "False easting", "value": 400000, "unit": "metre", "id": {"authority": "EPSG", "code": 8806}}, {"name": "False northing", "value": -100000, "unit": "metre", "id": {"authority": "EPSG", "code": 8807}}]}, "coordinate_system": {"$schema": "https://proj.org/schemas/v0.2/projjson.schema.json", "type": "CoordinateSystem", "subtype": "Cartesian", "axis": [{"name": "Easting", "abbreviation": "E", "direction": "east", "unit": "metre"}, {"name": "Northing", "abbreviation": "N", "direction": "north", "unit": "metre"}]}}

During handling of the above exception, another exception occurred:

CRSError                                  Traceback (most recent call last)
<ipython-input-9-e94676834313> in <module>
     20 
     21 with open('temp.pickle', 'rb') as f:
---> 22     crs_unpickled = pickle.load(f)
     23 
     24 crs_unpickled

~/anaconda3/lib/python3.7/site-packages/pyproj/crs/crs.py in __init__(self, conversion, name, cartesian_cs, geodetic_crs)
   1603             ).to_json_dict(),
   1604             "conversion": CoordinateOperation.from_user_input(
-> 1605                 conversion
   1606             ).to_json_dict(),
   1607             "coordinate_system": CoordinateSystem.from_user_input(

pyproj/_crs.pyx in pyproj._crs._CRSParts.from_user_input()

pyproj/_crs.pyx in pyproj._crs.CoordinateOperation.from_string()

pyproj/_crs.pyx in pyproj._crs.CoordinateOperation.from_string()

pyproj/_crs.pyx in pyproj._crs.CoordinateOperation._from_string()

CRSError: Invalid coordinate operation string: {"$schema": "https://proj.org/schemas/v0.2/projjson.schema.json", "type": "ProjectedCRS", "name": "undefined", "base_crs": {"$schema": "https://proj.org/schemas/v0.2/projjson.schema.json", "type": "GeographicCRS", "name": "undefined", "datum": {"type": "GeodeticReferenceFrame", "name": "undefined", "ellipsoid": {"name": "undefined", "semi_major_axis": 6377563.396, "inverse_flattening": 299.3249646}, "prime_meridian": {"name": "undefined", "longitude": 0}}, "coordinate_system": {"subtype": "ellipsoidal", "axis": [{"name": "Longitude", "abbreviation": "lon", "direction": "east", "unit": "degree"}, {"name": "Latitude", "abbreviation": "lat", "direction": "north", "unit": "degree"}]}}, "conversion": {"$schema": "https://proj.org/schemas/v0.2/projjson.schema.json", "type": "Conversion", "name": "unknown", "method": {"name": "Transverse Mercator", "id": {"authority": "EPSG", "code": 9807}}, "parameters": [{"name": "Latitude of natural origin", "value": 49, "unit": "degree", "id": {"authority": "EPSG", "code": 8801}}, {"name": "Longitude of natural origin", "value": -2, "unit": "degree", "id": {"authority": "EPSG", "code": 8802}}, {"name": "Scale factor at natural origin", "value": 0.9996012717, "unit": "unity", "id": {"authority": "EPSG", "code": 8805}}, {"name": "False easting", "value": 400000, "unit": "metre", "id": {"authority": "EPSG", "code": 8806}}, {"name": "False northing", "value": -100000, "unit": "metre", "id": {"authority": "EPSG", "code": 8807}}]}, "coordinate_system": {"$schema": "https://proj.org/schemas/v0.2/projjson.schema.json", "type": "CoordinateSystem", "subtype": "Cartesian", "axis": [{"name": "Easting", "abbreviation": "E", "direction": "east", "unit": "metre"}, {"name": "Northing", "abbreviation": "N", "direction": "north", "unit": "metre"}]}}

Example 2 throws a TypeError:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-8-398f1c3491c5> in <module>
     14 
     15 with open('temp.pickle', 'rb') as f:
---> 16     vertcrs_unpickled = pickle.load(f)
     17 
     18 vertcrs_unpickled

TypeError: __init__() missing 1 required positional argument: 'datum'

Expected Output

Both examples are expected to load their pickled objects successfully.

Environment Information

I'm running these examples in a Jupyter notebook on macOS, if that's of any importance.

pyproj info:
    pyproj: 3.1.0
      PROJ: 8.0.1
  data dir: .../anaconda3/lib/python3.7/site-packages/pyproj/proj_dir/share/proj
user_data_dir: .../Library/Application Support/proj

System:
    python: 3.7.3 (default, Mar 27 2019, 16:54:48)  [Clang 4.0.1 (tags/RELEASE_401/final)]
executable: .../anaconda3/bin/python
   machine: Darwin-20.6.0-x86_64-i386-64bit

Python deps:
   certifi: 2019.06.16
       pip: 19.1.1
setuptools: 57.0.0
    Cython: 0.29.12

Installation method

Installed from pip.

[alexstaravoitau]

Wow that was very quick! Thank you @snowman2. 🙏