flattening not working - string indices must be integers

[fekle]

Hi there,

as promised on Slack, I'm opening an issue here regarding the issue with flattening.

When using the tap without flattening enabled, everything works great, but most of the properties are buried in a json field in the resulting database table, as expected. In order to improve performance etc when using the data, flattening while extracting would be great.

The documentation for this tap mentions the options flattening_enabled and flattening_max_depth.
I tried enabling flattening with flattening_enabled, and setting flattening_max_depth to various values such as 1, 3, 9 etc - but always get the same error as shown below.

I dug into the errors a bit further, and the problem seems to be related to this tap's schema - specifically this: https://github.com/MeltanoLabs/tap-klaviyo/blob/d049862b4d8d3426481c6739f48ec98a06dda09c/tap_klaviyo/schemas/profiles.json#L175C19-L189.

When commenting out this part of the schmema (I set up the tap source locally), flattening works.
As I'm not familiar with the workings of the meltano SDK, I'm not sure on how to proceed here - but I'm sure someone with more experience should be able to find the issue :-)

This is the log output when invoking the tap with flattening enabled:

> meltano invoke tap-klaviyo                                                                                              15:37:11 
2023-05-30T13:37:15.794503Z [info     ] Environment 'dev' is active
Need help fixing this problem? Visit http://melta.no/ for troubleshooting steps, or to
join our friendly Slack community.

Catalog discovery failed: command ['/REDCACTED/.meltano/extractors/tap-klaviyo/venv/bin/tap-klaviyo', '--config', '/REDCACTED/.meltano/run/tap-klaviyo/tap.6aa80ce5-ba97-46d2-8691-3cdc6da9e962.config.json', '--discover'] returned 1 with stderr:
 2023-05-30 15:37:18,372 | INFO     | tap-klaviyo          | Added 'listperson' as child stream to 'lists'
Traceback (most recent call last):
  File "/REDCACTED/.meltano/extractors/tap-klaviyo/venv/bin/tap-klaviyo", line 8, in <module>
    sys.exit(TapKlaviyo.cli())
  File "/REDCACTED/.meltano/extractors/tap-klaviyo/venv/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/REDCACTED/.meltano/extractors/tap-klaviyo/venv/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/REDCACTED/.meltano/extractors/tap-klaviyo/venv/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/REDCACTED/.meltano/extractors/tap-klaviyo/venv/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/REDCACTED/.meltano/extractors/tap-klaviyo/venv/lib/python3.9/site-packages/singer_sdk/tap_base.py", line 516, in cli
    tap = cls(  # type: ignore[operator]
  File "/REDCACTED/.meltano/extractors/tap-klaviyo/venv/lib/python3.9/site-packages/singer_sdk/tap_base.py", line 103, in __init__
    self.mapper.register_raw_streams_from_catalog(self.catalog)
  File "/REDCACTED/.meltano/extractors/tap-klaviyo/venv/lib/python3.9/site-packages/singer_sdk/mapper.py", line 628, in register_raw_streams_from_catalog
    self.register_raw_stream_schema(
  File "/REDCACTED/.meltano/extractors/tap-klaviyo/venv/lib/python3.9/site-packages/singer_sdk/mapper.py", line 671, in register_raw_stream_schema
    self.default_mapper_type(
  File "/REDCACTED/.meltano/extractors/tap-klaviyo/venv/lib/python3.9/site-packages/singer_sdk/mapper.py", line 93, in __init__
    self.transformed_schema = self.flatten_schema(self.transformed_schema)
  File "/REDCACTED/.meltano/extractors/tap-klaviyo/venv/lib/python3.9/site-packages/singer_sdk/mapper.py", line 141, in flatten_schema
    return flatten_schema(
  File "/REDCACTED/.meltano/extractors/tap-klaviyo/venv/lib/python3.9/site-packages/singer_sdk/helpers/_flattening.py", line 205, in flatten_schema
    new_schema["properties"] = _flatten_schema(
  File "/REDCACTED/.meltano/extractors/tap-klaviyo/venv/lib/python3.9/site-packages/singer_sdk/helpers/_flattening.py", line 244, in _flatten_schema
    _flatten_schema(
  File "/REDCACTED/.meltano/extractors/tap-klaviyo/venv/lib/python3.9/site-packages/singer_sdk/helpers/_flattening.py", line 255, in _flatten_schema
    if list(v.values())[0][0]["type"] == "string":
TypeError: string indices must be integers

I'm using Postgres as the target, so i can use the tap without flattening and using jsonb selectors - so this is not a big deal, but I think it should either be fixed, or flattening removed from the documentation.
Thanks for the great tap!