Can one use Elastic Search as a service rather than installing it locally

Given the existence of Solr as a Service, such as Hosted Apache Solr from such a Drupal community stalwart as Jeff Geerling, i figured there surely had to be also hosted Elastic Search that would be easy to use with Drupal.

It seems… not?

Articles about Elastic Search and Drupal that only mention downloading and installing Elastic Search yourself:

There are of course plenty of hosted elasticsearch options, including elastic.co, bonsai.io, and qbox.io. And there’s the big issue in the libre software and SaaS world of Amazon Web Services “stealing” elastic search (making tons of money on it without contributing to development) and the reactions to that.

OK wow exactly what i expected does exist, it’s just… not easily found by searching for any of the obvious words (Elastic search, service, Drupal, hosted) that i searched for. But searching specificalyl for Qbox and Drupal brought me to this blog post highlighting the Elasticsearch Connector module which got me to take another look at that module, which despite the obvious name i’d sort of glossed over.

https://www.drupal.org/project/elasticsearch_connector

Anyhow the developers coming to us for technical expertise, the back and forth about why Elasticsearch at all:

Mayfirst:

We do have our own Elasticsearch setup, but it’s not available for member use and would not be suitable to run on a shared server. Both elasticsearch and kibana are real resource hogs and are overkill for 99% of our users so I doubt we will be making that available.

what context there is for the request? Is it to improve Drupal’s search? Or are there other reasons?

elasticsearch is a real beast to maintain and keep up with (for example, logstash is no longer the preferred method to transfer logs - there’s now the “beats” - e.g. filebeat). Here’s a good article on why you might not want to use elasticsearch: https://medium.com/@KyleCondon/elasticsearch-for-the-curious-or-what-i-learned-processing-reddit-data-d643952146f1

You’d mentioned some analytics… I’m not sure in what sense you meant there, but there are certainly web analytics modules for Drupal that don’t require an Elastic Search instance. We should perhaps clarify that requirement.

I’m curious how many entities you’d estimate we will be indexing in total? While Solr doesn’t have the most whiz-bang Web2.0 UI but it does use the same back end for searching (Lucene), and it integrates clealy with Drupal.

What sort of other things might you expect to be asking from Elastic? Aside from helping index things/providing improved search functionality.

The goal is to improve Drupal’s search and give us more control on faceting etc. I did a quick check with Drupal’s native search and had a few cringey moments.

I don’t know much about what it takes to maintain ELK. I know they have frequent updates and releases. That may be too much at some point, obviously. What I know about Beats is that they are specialized, lightweight, and real time processors. I never worked with them or got one to work for me.

I will have to check Drupal’s analytics modules but I think Kibana is pretty exceptional in being able to aggregate data.

My main concern is the size of the content we may have to deal with, but I suppose we can put this concern aside for the time being and got back to that question when we need to.

I’d never worked with solr before. So I will have to learn how to use it and learn it fast. it seems.

Solr it is then. Let’s get this operation out the door first, then we’ll see how things are going.