Upgrading Apache Solr 8 to 9: Reindexing, Java 11, and the Relevance Regression Nobody Budgets For

Where things stand

Apache Solr 8 reached end of life on 25 October 2024, with 8.11.4 as its final release. Upstream maintains 9.x and has shipped Solr 10. Nothing is backported below 9, and Solr's advisory history is dominated by remote code execution reached through request handlers — so an unpatched Solr is not a theoretical exposure.

The three hard requirements

From Solr's own Major Changes in Solr 9 notes:

• Java 11 is the minimum. Solr 9 requires Java 11 and is also tested with Java 17. Estates still running Solr 8 on Java 8 have a JVM migration inside their Solr migration.
• Reindexing is strongly recommended if the collection was created in Solr 8. Solr's wording is that you should always fully reindex after a major version upgrade.
• Reindexing is mandatory if the collection was created on Solr 7 or older. This catches more estates than expected: a collection created on 7 and carried forward through in-place upgrades is still a Solr 7 collection as far as the index format is concerned.

That third point deserves a check before anything else, because it changes the project from “upgrade” to “rebuild”:

# List the collections
curl -s 'http://solr:8983/solr/admin/collections?action=CLUSTERSTATUS&wt=json' \
  | jq '.cluster.collections | keys'

# Lucene version that wrote the oldest segment in a core
curl -s 'http://solr:8983/solr/mycore/admin/segments?wt=json' \
  | jq -r '.segments[].version' | sort -u

If the oldest segment version starts with a 7, you are reindexing. Plan for it rather than discovering it.

What else moved

• Modules. Functionality that shipped as contribs in 8.x became modules in 9.x, enabled explicitly. Anything relying on a contrib being on the classpath by default needs configuring.
• <lib/> directives are deprecated and scheduled for removal in Solr 10. If your solrconfig.xml loads jars this way, fix it during this upgrade rather than the next.
• The Java Security Manager. Solr used it as an extra layer against unintended file, network and process access. Java has removed Security Manager support from Java 24, so Solr disables the feature on those JVMs — a quiet loss of defence in depth worth knowing about when you choose a JVM.
• SolrJ dependencies. Client applications may need dependency changes depending on which SolrClient implementation they use.

The part that takes the time: relevance

Every item above is mechanical. This one is not. Between major versions, Lucene's scoring, analysers and query parser behaviour shift, and result ordering shifts with them. Nothing errors. No health check fails. Search simply returns slightly different results in a slightly different order, and the first person to notice is a user, or a merchandiser, or whoever owns conversion.

A Solr major upgrade therefore needs a relevance regression, and it needs an owner who can sign it off:

1. Capture a query set from production — real queries, weighted by frequency, including the long tail and the zero-result queries.
2. Record the top results for each on Solr 8. This is your baseline.
3. Build the Solr 9 cluster in parallel and reindex into it. A parallel cluster is what makes both the comparison and the rollback possible.
4. Replay the query set and diff. Rank-biased overlap or a plain top-ten intersection is enough to find the queries that moved.
5. Triage the differences. Most are neutral. Some are improvements. A few are regressions traceable to a specific analyser or parser change, and those are the ones to fix before cutover.

Teams that skip this do not save the time. They spend it afterwards, under pressure, working backwards from a complaint.

A sequence that works

1. Establish which collections require a mandatory reindex.
2. Move to Java 11 or 17 on Solr 8.11 first, so the JVM change is isolated from the Solr change.
3. Audit solrconfig.xml for contribs, <lib/> directives and deprecated handlers.
4. Stand up Solr 9 in parallel. Reindex from source, not from the old index.
5. Run the relevance regression and get sign-off.
6. Cut traffic over behind an alias, keeping Solr 8 warm until you are confident.

Note the dependency on step four: reindexing from source assumes you can still regenerate the documents. If the only copy of some content is the Solr index itself, that is a separate problem to solve first, and a more urgent one than the upgrade.

SolrCloud: check the ensemble too

SolrCloud coordinates through ZooKeeper, and that ensemble is usually older than the Solr cluster it serves. ZooKeeper 3.7, 3.6 and 3.5 are all past end of life. Upgrading Solr on top of an unmaintained ensemble moves the exposure rather than removing it.

If the upgrade is a quarter away

It usually is — a reindex plus a relevance cycle plus a sign-off is rarely less. Solr 8 is unpatched for that whole period. OSSeva ships patched Solr 6, 7 and 8 builds that keep the Lucene index format unchanged, so they drop onto the existing data directory with no reindex, and the upgrade can be done properly rather than quickly.

Related

• Apache Solr 8 end of life · Solr 7
• Apache Solr extended support
• Apache ZooKeeper extended support