SolrCore Initialization Failures

[apotek]

I am following the steps in the README.

1. ddev get ddev/ddev-solr
2. ddev restart
3. "Once up and running, access Solr's UI within your browser by opening http://<projectname>.ddev.site:8983"

So, I go to http://<my project>.ddev.site:8983 and log in and see the following:

SolrCore Initialization Failures

dev: org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: Please check your logs for more information

So I check my logs:

Log4j2

Time (Local)	Level	Core	Logger	Message
4/22/2024, 10:18:14 PM	WARN false		ZkController	Contents of zookeeper /security.json are world-readable; consider setting up ACLs as described in https://solr.apache.org/guide/zookeeper-access-control.html
4/22/2024, 10:18:14 PM	WARN false		CoreContainer	Solr authentication is enabled,​ but SSL is off. Consider enabling SSL to protect user credentials and data with encryption.
4/22/2024, 10:18:14 PM	ERROR false	x:dev	ZkController
4/22/2024, 10:18:14 PM	ERROR false	CoreContainer	Error waiting for SolrCore to be loaded on startup

Consequences

1. I also note that no cores or collections have been initialized on startup (according to the solr UI). I was expecting that one would be created in the docker compose build steps.

2. Sure enough, just as I feared, when I configure my connection to solr in Drupal with:

Server Name: solr
Backend: Solr
Solr Connector: Solr Cloud with Basic Auth
Solr node: solr
Solr port: 8983
Solr Path: /
Default solr collection: mycollection

http basic auth: (username and password as specified).

Save

I get the following message:

Warning message
Server solr is reachable but the configured collection is not available.

I check the solr admin page and I see that solr, as I feared, does not see my collection. I forge ahead...

On the server's "view" page click the Upload Configset button and check the "Upload (and overwrite)

Ok. I follow these instructions. Number of shards 1, etc, And I click the upload button.

I get the following error:

Error message
The configs of the essential Solr field types are missing or broken for server "solr".

So it looks like the container is failing to be initialized with something essential pre-requisite setups.

When I ssh in to the container, I see indeed that no collection has been created on the file system:

$ ls -al /var/solr/data
total 24
drwxrwx--- 4 solr solr 4096 Apr 23 02:52 .
drwxrwx--- 4 solr root 4096 Apr 23 02:12 ..
drwxr-xr-x 2 solr solr 4096 Apr 23 02:18 filestore
-rw-r--r-- 1 solr solr 2427 Jan 12 17:42 solr.xml
drwxr-xr-x 2 solr solr 4096 Apr 23 02:18 userfiles
-rw-r--r-- 1 solr solr 1048 Jan 12 17:42 zoo.cfg

Viewing the solr server I set up, I see that I can connect, but the collection is not available. "The Solr collection could not be accessed. Further data is therefore unavailable."

You can see also in the search_api_solr module output that the module is making a very odd request to solr:8983/solr/benz/admin/luke??? I have no idea why it would be requesting such a path.

I have made no changes to anything other than the ones specified in the README. No customizations to the image, or to .ddev/docker-compose.solr.yaml

Changing connection from auto detect to 8.x doesn't help.

Trying to make it work

Using the solr admin UI, I created a collection, using the same name as the failed attempt from the search_api_solr module.

That succeeded, and when I visited the server "view" page again, I get a message saying that the module can see the collection:

The Solr collection could be accessed (latency: 11.675165725708 ms).

Ok Great! Time to upload the configset!

I click the Upload configset button and say yes to overwrite the _default config set for my collection.

And then I get this error in search_api_solr:

Error message
The configs of the essential Solr field types are missing or broken for server "solr".
You are using an incompatible Solr schema. Please follow the instructions described in the README.md file for setting up Solr.

Looking in the logs on the solr server, I see:

solrconfig.xml: is no longer supported,​ use solr.xml:/metrics/reporter section instead

I don't know how to interpret this but it sure seems like there is an incompatibility between the config set search_api_solr wants to generate and what solr-cloud is expecting.

Well, let's try with drush:

% ddev drush --numShards=1 search-api-solr:upload-configset solr

In SolrConfigSetController.php line 336:

The configs of the essential Solr field types are missing or broken for server "solr".

Failed to run drush --numShards=1 search-api-solr:upload-configset solr: exit status 1

I'm stumped. I tore down and removed my ddev setup, purged my volumes and docker images and tried again. Same result.

[apotek]

Just wiped everything, including docker volumes and images and went through all the steps in the README, exactly as specified.

I think this is the source of all the problems. When I attempt to upload my config set (either via drush or via the UI), the following error message is returned:

The configs of the essential Solr field types are missing or broken for server "solr".

I interpret this to mean, given that other users seem to be having success with this ddev add-on, that something has changed in the solr docker container and ddev-solr needs to supply or run a setup command to install the essential solr field types.

% docker --version
Docker version 25.0.3, build 4debf41
% ddev --version
ddev version v1.22.7

search_api_solr module version: 4.3.2
search_api_solr_admin module is enabled.

[rfay]

You're doing ddev drush @adobe ? Doesn't that mean that this entire problem is happening on the remote server and has nothing to do with this add-on?

[apotek]

You're doing ddev drush @adobe ? Doesn't that mean that this entire problem is happening on the remote server and has nothing to do with this add-on?

Thanks @rfay, but no, "adobe" is just my drush alias for the drupal site in the ddev project I am working on. Nothing is happening with a remote server. I have removed it from the error description in order to remove confusion, but I do wonder if this add on has been tested in multi-site setups. The old ddev-drupal9-solr add works fine in multi-site mode.

One thing I appreciate about this add-on (ddev-solr) is that it doesn't force you to name your "core" (now collection) any one specific thing. But the cost of this seems to be that since nothing is pre-built/pre-configured on the server, that there are more problems in getting to a non-error state. When I ran into the error with the missing basic fields config, I wondered if it could be a change in the upstream docker container ... that this add on used to work but now is assuming certain defaults and configuration that no longer holds.

[rfay]

The old ddev-drupal9-solr add works fine in multi-site mode

I guess there's nothing wrong with using that one.

I do wonder if this add on has been tested in multi-site setups

I have not heard of anyone using ddev-solr with multi-site, but don't know why that would actually be relevant.

this add on used to work but now is assuming certain defaults and configuration that no longer holds

There have been no non-trivial changes to this add-on in 10 months, see https://github.com/ddev/ddev-solr/commits/main/

Please try it in a limited test environment and see if you can sort out what's going on. (Create a trivial project using a quickstart, extend it to a multisite after you have the add-on working, etc.)

[apotek]

Some interesting reading that might have some connection to the issues encountered here:

• DDEV Solr Cloud Instructions do not work: no collection created
• entity type needs to be installed - Cache, Request Dispatcher, Request Handler

[rfay]

Was https://www.drupal.org/project/search_api_solr/issues/3104189 actually the issue for you?

[apotek]

Was https://www.drupal.org/project/search_api_solr/issues/3104189 actually the issue for you?

Not sure. I am waiting for another time window where I can dig deeper and hopefully come up with some useful knowledge to help others.

[mkalkbrenner]

@apotek try to follow the README with a fresh installation of drupal. It seems that you messed up your Solr configs in Drupal.

If that works we can take a closer look.

[mkalkbrenner]

no further feedback

[apotek]

no further feedback

Sorry for the lack of feedback. I didn't have time budget to keep troubleshooting this add-on. We ended up having to use https://github.com/ddev/ddev-drupal-solr in order to get things working. If I ever get the budget to revisit this, I will add more info, but it looks doubtful.