Laravel and Elasticsearch

What is Elasticsearch?

Elasticsearch is a document-oriented database consisting of clusters, indexes and documents. It’s used for a fast full-text search and data analysis.

A more detailed info can be found on the official docs.

HTTP protocol can be used to communicate with Elasticsearch which makes it easier to interact using the API since you don’t need anything else but curl.

First step is installing Elasticsearch on your server or local machine following the instruction.

Start working with Elasticsearch & Laravel

For a search to work correctly and the data be actualized from the Laravel side we need to implement the following:

  • Create indexes and mappings
  • Adding documents
  • Updating documents
  • Deleting documents
  • Search

Let’s start!

We’ll be using the official Elasticsearch package for PHP

Search architecture

To start with, you need to decide what information you’re going to search and how’s going to be stored.

Why is that important?

The search engine should contain only the relevant information that search needs to be performed on. It doesn’t need to store anything unrelated to the search itself to avoid unnecessary database growth and overloading.

For example, if there’s a product that should be possible to be found by the category name or a tag, we should store that information in the document of the product itself:

{
    "name" : "my name",
    "id" : 7,
    “category_name”: “my first category”,
    “tags” : [“my first tag”, “my second tag”]
  }

Elasticsearch stores data as documents which are stored inside indexes.

Prior to version 6 there Elasticsearch also had such entities as Types which were often incorrectly used and brought sparsity of data within one index.

Types were marked as deprecated since version 6 and were completely removed in v. 7. You can find more info on Types here.

Elasticsearch provides multiple data types and flexible configuration options for each of them. Please check the official docs for more details.

Test Laravel + Elasticsearch Application

Please note that the app architecture is not covered in this article, instead we are going through the basic principles of interaction between Elasticsearch and Laravel.

We’ll perform most of the operations in the terminal using Commands and Tinker tool.

In this test app we will store and search for the products.

Preparation

Let’s create a model and migration first.

php artisan make:model Product -m

Fill the table rows

Schema::create('products', function (Blueprint $table) {
    $table->id();
    $table->string('name');
    $table->text('description');
    $table->double('price');
    $table->timestamps();
  });

Perform a migration

php artisan migrate

Now, we create a config file to store the connection details

return [
    'hosts'    => ['elasticsearchhost:9200'],
  ];

Next, let’s add a command to create the products index

php artisan make:command CreateProudctIndexCommand
class CreateProductIndexCommand extends Command
  {
    /**
     * The name and signature of the console command.
     *
     * @var string
     */
    protected $signature = 'elasticsearch:create-product-index';

    /**
     * @var \Elasticsearch\Client
     */
    private $client;

    /**
     * Create a new command instance.
     *
     * @return void
     */
    public function __construct(\Elasticsearch\ClientBuilder $builder)
    {
        parent::__construct();

        $this->client = $builder::fromConfig(
            Config::get('elasticsearch')
        );
    }

    /**
     * Execute the console command.
     *
     * @return int
     */
    public function handle()
    {
        $params = [
            'index' => 'products',
            'body'  => [
                'mappings' => [
                    'properties' => [
                        'id'        => [
                            'type' => 'long',
                        ],
                        'name'        => [
                            'type' => 'keyword',
                        ],
                        'description' => [
                            'type' => 'text',
                        ],
                        'price'       => [
                            'type' => 'double',
                        ],
                        'created_at'  => [
                            'type' => 'date',
                        ],
                        'updated_at'  => [
                            'type' => 'date',
                        ],
                    ],
                ],
            ],
        ];

        $this->client->indices()->create($params);
    }
  }

Execute

php artisan elasticsearch:create-product-index

and check that the product index has been created

curl elasticsearchhost:9200/products?pretty
{
    "products" : {
      "aliases" : { },
      "mappings" : {
        "properties" : {
          "created_at" : {
            "type" : "date"
          },
          "description" : {
            "type" : "text"
          },
          "id" : {
            "type" : "long"
          },
          "name" : {
            "type" : "keyword"
          },
          "price" : {
            "type" : "double"
          },
          "updated_at" : {
            "type" : "date"
          }
        }
      },
      "settings" : {
        "index" : {
          "number_of_shards" : "1",
          "blocks" : {
            "read_only_allow_delete" : "true"
          },
          "provided_name" : "products",
          "creation_date" : "1604291853222",
          "number_of_replicas" : "1",
          "uuid" : "Xdk4ptlKTKWOJugUAltCkQ",
          "version" : {
            "created" : "7090299"
          }
        }
      }
    }
  }

Let’s create a small trait to check the mode events, that way we’ll be able to update data in Elasticsearch when the event fires.

<?php

  namespace App\Traits;

  use Illuminate\Support\Facades\Config;

  trait Searchable
  {
    /** @var \Elasticsearch\Client */
    private static $elastic_client;

    /**
     * Get model index in Elasticsearch
     *
     * @return string
     */
    abstract public static function getElasticIndexName(): string;

    public static function bootSearchable()
    {
        /** Subscribe to created event and add new document to Elasticsearch */
        static::created(function ($model) {
            static::getElasticClient()->index([
                'index' => static::getElasticIndexName(),
                'id'    => $model->id,
                'body'  => $model->toSearchableArray(),
            ]);
        });

        /** Subscribe to update event and update document to Elasticsearch */
        static::updated(function ($model) {
            static::getElasticClient()->update([
                'index' => static::getElasticIndexName(),
                'id'    => $model->id,
                'body'  => [
                    'doc' => $model->toSearchableArray(),
                ],
            ]);
        });

        /** Subscribe to delete event and delete document from Elasticsearch */
        static::deleted(function ($model) {
            static::getElasticClient()->delete([
                'index' => static::getElasticIndexName(),
                'id'    => $model->id,
            ]);
        });
    }

    /**
     * Search document by query
     * 
     * @param array $query
     *                    
     * @return array
     */
    public static function search(array $query)
    {
        return static::getElasticClient()->search([
            'index' => static::getElasticIndexName(),
            'body'  => $query,
        ]);
    }

    /**
     * Get model searchable data
     *
     * @return array
     */
    public function toSearchableArray(): array
    {
        return $this->toArray();
    }

    /**
     * Get Elasticsearch Client
     *
     * @return \Elasticsearch\Client
     */
    private static function getElasticClient()
    {
        if (!static::$elastic_client) {
            return static::$elastic_client = \Elasticsearch\ClientBuilder::fromConfig(
                Config::get('elasticsearch')
            );
        }

        return static::$elastic_client;
    }
}

Let's add a trait to the Product model

class Product extends Model
  {
    use HasFactory;
    use Searchable;

    protected $fillable = [
        'name',
        'description',
        'price'
    ];

    public static function getElasticIndexName()
    {
        return 'products';
    }
  }

Working with data

Now, we’ll add some products

php artisan tinker
$p1 = new Product([
    'name' => 'my name', 
    'description' => 'my description', 
    'price' => 20
  ]);
  $p1->save();

And check if everything went fine

curl elastic:9200/products/_doc/1?pretty
{
    "_index" : "products",
    "_type" : "_doc",
    "_id" : "1",
    "_version" : 1,
    "_seq_no" : 1,
    "_primary_term" : 1,
    "found" : true,
    "_source" : {
      "name" : "my name",
      "description" : "my description",
      "price" : 20,
      "updated_at" : "2020-09-02T05:05:29.000000Z",
      "created_at" : "2020-09-02T05:05:29.000000Z",
      "id" : 1
    }
  }

Great! A product has been created and a new document has been added to Elasticsearch.

Let’s try to update now. Once again, we use Tinker.

Product::find(1)->update(['name' => 'my second name']);
curl elastic:9200/products/_doc/1?pretty
{
    "_index" : "products",
    "_type" : "_doc",
    "_id" : "8",
    "_version" : 2,
    "_seq_no" : 1,
    "_primary_term" : 1,
    "found" : true,
    "_source" : {
      "name" : "my second name",
      "description" : "my description",
      "price" : 20.0,
      "updated_at" : "2020-11-02T05:25:11.000000Z",
      "created_at" : "2020-11-02T05:23:06.000000Z",
      "id" : 8
    }
  }

It works! Let’s try to find our product now:

Product::search([
  'query' => [
      'match' => [
        'name' => 'my second name',
      ],
    ],
  ]);

And get a response

[
     "took" => 0,
     "timed_out" => false,
     "_shards" => [
       "total" => 1,
       "successful" => 1,
       "skipped" => 0,
       "failed" => 0,
     ],
     "hits" => [
       "total" => [
         "value" => 1,
         "relation" => "eq",
       ],
       "max_score" => 0.18232156,
       "hits" => [
         [
           "_index" => "products",
           "_type" => "_doc",
           "_id" => "10",
           "_score" => 0.18232156,
           "_source" => [
             "name" => "my name",
             "description" => "my description",
             "price" => 20,
             "updated_at" => "2020-11-02T05:44:11.000000Z",
             "created_at" => "2020-11-02T05:44:11.000000Z",
             "id" => 10,
           ],
         ],
       ],
     ],
   ]
  • hits.total - Total number of documents found
  • hits.hits - Documents found

A more detailed info and description of all products can be found in the docs.

Now, let’s deleted the product

Product::find(1)->delete();
curl elastic:9200/products/_doc/1?pretty
{
    "_index" : "products",
    "_type" : "_doc",
    "_id" : "1",
    "found" : false
  }

Conclusion

We’ve shown basic principles of interaction between Laravel and Elasticsearch to search and add documents.

To learn Elasticsearch in detail, we highly recommend to start with the official documentation

And the following tutorial. Although it was written quite long ago, it’s still relevant

Also, there are several existing Laravel packages that can be used to simplify the interaction with Elasticsearch. This way you wouldn’t need to write the routine functions handling index addition, search and indexation.

Here’re some of the existing packages:

  • https://github.com/babenkoivan/scout-elasticsearch-driver
  • https://github.com/matchish/laravel-scout-elasticsearch
  • https://github.com/cviebrock/laravel-elasticsearch
Let's Talk
What’s next?
  1. We’ll get in touch in 1 business day
  2. We’ll sign the NDA if required and get on a 30 minute call to discuss the project
  3. Based on that call and requirements, we’ll prepare a quote