Error: Error transforming Elasticsearch response for index

[p-appmap]

The problem

I have an index in Elasticsearch with a join field type and after a long time trying to make the facets work with the fields that are inside the inner_hits object and failing, i started considering the possibility of using the proxy feature of searchkit but i'm not having any luck either, i'm getting an error saying: Error transforming Elasticsearch response for index

Environment

• Dependencies:

"dependencies": {
    "@adobe/react-spectrum": "^3.33.1",
    "@searchkit/api": "4.11.0",
    "@searchkit/instantsearch-client": "4.12.0",
    "@types/node": "20.3.1",
    "@types/react": "18.2.13",
    "@types/react-dom": "18.2.6",
    "instantsearch.css": "^8.0.0",
    "next": "13.4.7",
    "react": "18.2.0",
    "react-dom": "18.2.0",
    "react-instantsearch": "^7.2.0",
    "react-instantsearch-dom": "^6.40.4",
    "react-instantsearch-router-nextjs": "^7.5.5",
    "searchkit": "4.11.0",
    "typescript": "5.0.2"
  }

• ElasticSearch/OpenSearch version: 8.12.0 (Using Elastic Cloud)

Code To Reproduce Issue

This is the index mapping:

PUT /search-tk-simple-nested-ii/_mapping
{
  "properties": {
    "join_field": {
      "type": "join",
      "relations": {
        "product": "product_by_pdv" 
      }
    },
    "ref_id": {"type": "keyword"},
    "ref_id_p": {"type": "keyword"},
    "price_list": {"type": "float"},
    "rank": {"type": "integer"},
    "combo_components": {
      "type": "nested",
      "properties": {
        "refIdComp": {"type": "keyword"}
      }
    }
  }
}

This is the configuration that i'm using to run the query:

const apiClient = Client(
  {
    connection: {
      host: 'http://localhost:8080',
    },
    search_settings: {
      highlight_attributes: ['name'], // Usar highlight para atributos que no tienen muchos chars
      snippet_attributes: ['child_category:200'], // Usar snippet para atributos grandes (como una descripcion)
      search_attributes: [
        { field: 'name', weight: 3 },
        { field: 'child_category', weight: 2 },
      ],
      result_attributes: ['name', 'parent_category', "child_category", "ref_id", "price_list", "rank"], 
      facet_attributes: [
        { attribute: 'child_category', field: 'child_category.enum', type: 'string' },
        { attribute: 'price_list', field: 'price_list', type: 'numeric',
        },
        { attribute: 'rank', field: 'rank', type: 'numeric' },
      ],
      filter_attributes: [
        {
          attribute: 'parent_category',
          field: 'parent_category.keyword',
          type: 'string'
        }
      ],
    }
  },
  { debug: true }

And this is the response i'm sending from my middleware:

I realize that the aggregations' field names are not the ones that searchkit expect, but taking that into account i also tried removing all the filters and facets from the query config and still gets the same error Error transforming Elasticsearch response for index

{
    "took": "6",
    "responses": [
        {
            "_shards": {
                "total": 2,
                "failed": 0,
                "successful": 2,
                "skipped": 0
            },
            "hits": {
                "hits": [
                    {
                        "_index": "search-tk-simple-nested-ii",
                        "inner_hits": {
                            "product_by_pdv": {
                                "hits": {
                                    "hits": [
                                        {
                                            "_routing": "1",
                                            "_index": "search-tk-simple-nested-ii",
                                            "_source": {
                                                "price_list": 520.52,
                                                "id_pdv": 2,
                                                "rank": 1,
                                                "join_field": {
                                                    "parent": "1",
                                                    "name": "product_by_pdv"
                                                }
                                            },
                                            "_id": "3",
                                            "_score": 1
                                        }
                                    ],
                                    "total": {
                                        "value": 1,
                                        "relation": "eq"
                                    },
                                    "max_score": 1
                                }
                            }
                        },
                        "_source": {
                            "ref_id": "ARC-1010301-UN-1",
                            "price_list": 520.52,
                            "id_pdv": null,
                            "parent_category": "Alimentos",
                            "name": "Ketchup Doypack 250g",
                            "rank": 1,
                            "child_category": "AlmacÚn"
                        },
                        "_id": "1",
                        "_score": 1
                    },
                    {
                        "_index": "search-tk-simple-nested-ii",
                        "inner_hits": {
                            "product_by_pdv": {
                                "hits": {
                                    "hits": [
                                        {
                                            "_routing": "4",
                                            "_index": "search-tk-simple-nested-ii",
                                            "_source": {
                                                "price_list": 43.52,
                                                "id_pdv": 2,
                                                "rank": 2,
                                                "join_field": {
                                                    "parent": "4",
                                                    "name": "product_by_pdv"
                                                }
                                            },
                                            "_id": "5",
                                            "_score": 1
                                        }
                                    ],
                                    "total": {
                                        "value": 1,
                                        "relation": "eq"
                                    },
                                    "max_score": 1
                                }
                            }
                        },
                        "_source": {
                            "ref_id": "ARC-1010302-UN-1",
                            "price_list": 43.52,
                            "id_pdv": null,
                            "parent_category": "Alimentos",
                            "name": "Ketchup Picante 250g",
                            "rank": 2,
                            "child_category": "AlmacÚn"
                        },
                        "_id": "4",
                        "_score": 1
                    }
                ],
                "total": {
                    "value": 2,
                    "relation": "eq"
                },
                "max_score": 1
            },
            "took": 6,
            "timed_out": false,
            "aggregations": {
                "children#to-product_by_pdv": {
                    "doc_count": 4,
                    "stats#rank$_stats": {
                        "min": 1,
                        "avg": 1.5,
                        "max": 2,
                        "count": 4,
                        "sum": 6
                    },
                    "stats#price_list$_stats": {
                        "min": 43.52000045776367,
                        "avg": 198.27000331878662,
                        "max": 520.52001953125,
                        "count": 4,
                        "sum": 793.0800132751465
                    },
                    "lterms#rank$_entries": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "doc_count": 2,
                                "key": 1
                            },
                            {
                                "doc_count": 2,
                                "key": 2
                            }
                        ]
                    },
                    "dterms#price_list$_entries": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "doc_count": 1,
                                "key": 43.52000045776367
                            },
                            {
                                "doc_count": 1,
                                "key": 101.5199966430664
                            },
                            {
                                "doc_count": 1,
                                "key": 127.5199966430664
                            },
                            {
                                "doc_count": 1,
                                "key": 520.52001953125
                            }
                        ]
                    }
                }
            },
            "status": 200
        }
    ]
}

I'd be really greatfull if you could give me any guidance on what might be the posible issue here. I assume I might be doing something wrong, but I just can't find anything.

What is the error trying to say to me? What would be the proper structure that searchkit expects on the response so that it can correctly transform the response for the index?

Any help would be much appreciated!
Thanks in advance!

[joemcelroy]

Hey there, sorry took a little longer to respond.

So looks like your index uses join or nested here (https://www.elastic.co/guide/en/elasticsearch/reference/current/joining-queries.html). Searchkit supports nested filtering / faceting (see docs). Haven't tested but likely doesn't support joining.

Could you share me the schema of your index and take a look at the documentation?

Could you also share the stack trace too?

Im also reachable on discord too.