In a previous tutorial, we discussed how to use one of Rust's Elasticsearch clients, rs-es, to interact with Elasticsearch via REST API. Now, we'll take a look at the other Rust Elasticsearch client, elastic

Elastic, like rs-es, is idiomatic Elasticsearch, but unlike rs-es, it is not idiomatic Rust. Strong typing over document types and query responses is prioritized over providing a comprehensive mapping of the Query DSL into Rust constructs. The elastic project aims to be equally usable for developers with and without Rust experience.

Structurally, the elastic crate combines several other crates which can also be used independently depending on the user's needs. The first of these is elastic-reqwest, a synchronous implementation of the Elasticsearch REST API based on Rust's reqwest library. Elastic-reqwest serves as the HTTP backend for the elastic crate itself. 

Second is elastic-requests, a strongly-typed implementation of Elasticsearch's REST API. Third is elastic-responses, which integrates with elastic-reqwest and facilitates handling Elasticsearch search responses by creating iterators for search results. Finally, elastic-types allows custom definitions of Elasticsearch types as Rust structures. It uses serde, which we encountered in the prior Rust elasticsearch tutorial, for serialization.

Tutorial

First, we add the relevant dependencies to our Cargo.toml. Since the elastic crate is frequently updated, we provide the URL of its GitHub repository rather than a crate version number to stay as up-to-date as possible with the latest revisions. Elastic-derive provides compile-time code generation for Elasticsearch types. The json-str crate simplifies building JSON strings in Rust. Finally, HTTP library hyper is linked to provide username and password information in our client's headers, thus avoiding 401 errors.

[dependencies]
elastic = { git = "https://github.com/elastic-rs/elastic" }
elastic_derive = "*"
serde_json = "1.0"
serde = "1.0"
serde_derive = "1.0"
json_str = "*"
hyper = "*"

Here are the corresponding import statements at the beginning of our main.rs. Note that the tutorial assumes that your Qbox username and password are saved for security purposes as system environment variables called $QBOX_USERNAME and $QBOX_PASSWORD. We import std::env, the module in Rust's standard library that can inspect system information, to extract information about these variables.

#[macro_use]
extern crate elastic_derive;
#[macro_use]
extern crate serde_derive;
extern crate serde;
extern crate hyper;
extern crate elastic;
#[macro_use]
extern crate json_str;
use elastic::error::*;
use elastic::prelude::*;
use elastic::http::header::Authorization;
use hyper::header::Basic;
use std::env;

Now, we'll create our custom document type as a Rust struct. In addition to deriving Serialize and Deserialize, as required by serde, we also need to derive ElasticType on our struct, which will allow us to derive its mapping.

We can specify the document's fields from the supported datatypes listed here. These will be mapped to corresponding Elasticsearch types. For example, our id field will be mapped as an integer field in Elastic, our title field will be mapped as a text field with a keyword subfield, and our timestamp field will be mapped as a 'date' field with an epoch_millis format.

#[derive(Debug, Serialize, Deserialize, ElasticType)]
struct QboxType {
    id: i32,
    title: String,
    timestamp: Date<DefaultDateFormat>,
}

Now, in our main() function, we can define our ClientBuilder and its request parameters. The "pretty" parameter will pretty-print any JSON output. The Authorization header specifies username and password for our Qbox host, as described above.

let builder = ClientBuilder::new()
.base_url("YOUR_QBOX_URL_GOES_HERE")
.params(|p| p
    .url_param("pretty", true)
    .header(Authorization(
     Basic {
         username: env::var("QBOX_USERNAME").unwrap().to_owned(),
         password: Some(env::var("QBOX_PASSWORD").unwrap().to_owned())
     }
 )));

Now, we will create a client from our ClientBuilder. As in rs-es, functions on our Client correspond to Elasticsearch APIs. These functions return a RequestBuilder struct, with the details of the RequestBuilder being specific to the operation being performed on the Client. For example, an IndexRequestBuilder is the builder for a Client.index_document request.

let client = builder.build().unwrap();

Then, we create a QboxType document to index:

let doc = QboxType {
    id: 1,
    title: String::from("Qbox tutorial"),
    timestamp: Date::now(),
};

Let's create some helper functions before proceeding. First, sample_index() will create an index which we can later use for our document.

fn sample_index() -> Index<'static> {
    Index::from("qbox_tutorial_index")
}

Second, put_index() will create an index for our QboxType document using sample_index() and elastic'screate_index functionPut_mapping() is also an elastic function, corresponding to Elasticsearch mapping as described in the Elasticsearch docs.

fn put_index(client: &Client) {
    client.create_index(sample_index()).send().unwrap();
    client.put_mapping::<QboxType>(sample_index()).send().unwrap();
}

Third, put_doc() indexes the document created.

fn put_doc(client: &Client, doc: QboxType) {
    client
        .index_document(sample_index(), id(doc.id), doc)
        .params(|p| p.url_param("refresh", true))
        .send()
        .unwrap();
}

Finally, we'll create a function, ensure_indexed(), to make sure information is only sent to our cluster if it hasn't been sent already. If our document is found, we simply print "document already indexed" and the document information. If the index is found, but not the document, we'll map and index the document using put_doc(). If there's no index, we'll create and index the document using put_index() and put_doc().

fn ensure_indexed(client: &Client, doc: QboxType) {
    let get_res = client
        .get_document::<QboxType>(sample_index(), id(doc.id))
        .send();
    match get_res {
        Ok(GetResponse { source: Some(doc), .. }) => {
            println!("document already indexed: {:?}", doc);
        }
        Ok(_) => {
            println!("indexing doc");
            put_doc(client, doc);
        }
        Err(Error(ErrorKind::Api(ApiError::IndexNotFound { .. }), _)) => {
            println!("creating index and doc");
            put_index(client);
            put_doc(client, doc);
        }
        Err(e) => panic!(e),
    }
}

Returning now to our new client and new QboxType document titled "Qbox tutorial", we'll call ensure_indexed():

ensure_indexed(&client, doc);

Now, we'll create a function called search() which searches our document for the query "Qbox tutorial".

fn search(client: &Client) -> SearchResponse<QboxType> {
    client
        .search()
        .index(sample_index())
        .body(json_str!({
                query: {
                    query_string: {
                        query: "Qbox tutorial"
                    }
                }
          }))
        .send()
        .unwrap()
}
let res = search(&client);

Finally, we'll print the result of our search:

println!("{:?}", res);

Output

We'll run our program twice to check that ensure_indexed is working as expected. On the first run, we see our index and document are created and the search query is successful.

creating index and doc
SearchResponseOf { took: 17, timed_out: false, shards: Shards { total: 4, successful: 4, failed: 0 }, hits: Hits { total: 1, max_score: Some(0.5753642), hits: [Hit { index: "qbox_tutorial_index", ty: "qboxtype", version: None, score: Some(0.5753642), source: Some(QboxType { id: 1, title: "Qbox tutorial", timestamp: Date { value: 2017-08-02T21:03:38.280Z, _t: PhantomData } }), routing: None }] }, aggregations: None, status: None }

On the second run, our search is still successful, but index and document are not created, since they already exist.

document already indexed: QboxType { id: 1, title: "Qbox tutorial", timestamp: Date { value: 2017-08-02T21:03:38.280Z, _t: PhantomData } }
SearchResponseOf { took: 6, timed_out: false, shards: Shards { total: 4, successful: 4, failed: 0 }, hits: Hits { total: 1, max_score: Some(0.5753642), hits: [Hit { index: "qbox_tutorial_index", ty: "qboxtype", version: None, score: Some(0.5753642), source: Some(QboxType { id: 1, title: "Qbox tutorial", timestamp: Date { value: 2017-08-02T21:03:38.280Z, _t: PhantomData } }), routing: None }] }, aggregations: None, status: None }

Conclusion

In conclusion, elastic is a comprehensive Rust Elasticsearch client which can be readily used for interacting with your Qbox cluster. Elastic is under active development and the lead developer is extremely responsive to user questions and issues. For further resources, please check out the elastic docs.

comments powered by Disqus