We recently began hosting the Solr search infrastructure for a Sitecore client who was looking for increased search performance and scalability. Naturally, they selected a Solr solution. Our client, a large Australian government entity, would see frequent high bursts of traffic and search activity. Their Sitecore deployment is hosted on Microsoft’s Azure Platform as a Service (PaaS). They needed a highly available and resilient Solr setup on Azure to deal with their traffic spikes.
In addition to the bump in performance and scalability, they were also looking to accelerate their Sitecore search deployment, testing, and release cycle times. Our client wanted to accelerate Solr app development by integrating their automated testing routines to ensure quality releases and minimized cycle times. They wanted to utilize APIs to programmatically create Solr search clusters, utilize them for their automated testing runs, and then tear them down. This has numerous benefits over creating these deployments manually each time OR creating the deployments and letting them sit idle in between releases. In this article, we’ll talk about what we did to help our client meet their goals.
Setting up a search cluster using open-source software such as Solr from scratch can be painful. Some of the things that an engineer would have to do to set up a cluster include:
Doing the initial setup can easily take an entire day or more, even for experienced engineers. And once the system is set up, there is another matter of maintaining the cluster, attending to issues like OS patch upgrades, disk failures, node crashes, etc.
Fortunately, software such as SearchStax can ease the pain of setting up a cluster by leveraging modern automation and cloud technology. With SearchStax, a cluster can be setup in minutes by filling out some simple forms and pushing a button.
Point-and-click cluster creation is an amazing feat of modern computation. However, it turns out that automation techniques can be leveraged even further. By providing a provisioning API, a cluster can be set up by an automation job, requiring zero human intervention.
Why would one want to set up a cluster via an API? The primary use case for programmatic cluster creation is for integration of testing automation. While production search environments are long-lived, testing environments tend to be of an ephemeral nature: a cluster is created, tests are run, and the cluster is torn down. Of course, semi-permanent dedicated clusters can be provisioned for test environments. This has several downsides:
SearchStax provides a REST API that makes it easy to integrate cluster provisioning with an automated testing pipeline. In the next section, we walk through a typical sequence of API calls that such a job might perform.
The first step in the process is to obtain credentials from SearchStax. Given a username and password, a token is retrieved
curl -X POST \
https://app.searchstax.com/api/rest/v1/obtain-auth-token/ \
-d 'username=admin&password=s3cr3t'
{
"token": "b09c"
}
One might want to view the deployments that are currently up and running. This is accomplished via the list-deployments API call. Note that we pass in the token in the Authorization header.
curl -X GET \
https://app.searchstax.com/api/rest/v2/account/MyAccount/deployment/ \
-H 'Authorization: Token b09c'
{
"count": 1,
"next": null,
"previous": null,
"results": [
{
"name": "mydeployment1",
"uid": "ss123456",
"application": "Solr",
"application_version": "6.6.2",
"tier": "Silver",
"http_endpoint": "https://ss123456-ap-southeast-2-aws.searchstax.com/solr/",
"status": "Running",
"provision_state": "Done",
"termination_lock": true,
"plan_type": "DedicatedDeployment",
"plan": "DN1",
"region_id": "ap-southeast-2",
"cloud_provider": "Amazon Web Services",
"cloud_provider_id": "aws",
"num_additional_app_nodes": 0,
"deployment_type": "Dedicated Node",
"num_nodes_default": 1,
"num_additional_zookeeper_nodes": 0,
"servers": [
"ss123456-1"
]
}
]
}
Now for the main event. We request a new cluster, passing parameters specifying the cluster name, the Solr version, the cloud provider, the region, whether a termination lock is enabled, and some other parameters.
curl -X POST \
https://app.searchstax.com/api/rest/v2/account/MyAccount/deployment/ \
-H 'Authorization: Token b09c' \
-d 'name=MyDeployment2&application_version=7.1.0&plan_type=DedicatedDeployment&cloud_provider_id=aws&plan=DN1®ion_id=us-west-1&termination_lock=true&application=Solr'
For a response, we receive an HTTP 201 CREATED status code
Notice that in our creation request, we set termination_lock to true. This protects against accidental deletion of the cluster. To enable deletion, we need to set the termination_lock to false.
curl -X POST \
https://app.searchstax.com/api/rest/v2/account/MyAccount/deployment/ \
-H 'Authorization: Token b09c' \
-d 'name=MyDeployment2&application_version=7.1.0&plan_type=DedicatedDeployment&cloud_provider_id=aws&plan=DN1®ion_id=us-west-1&termination_lock=true&application=Solr'
For the response, we receive an HTTP 200 OK status code.
Once we are through using the cluster, we will want to tear it down using the Delete API. This can be done as follows:
curl -X DELETE \
https://app.searchstax.com/api/rest/v2/account/VicRoads2/deployment/ss677906/ \
-H 'Authorization: Token b09c'
For the response, we receive an HTTP 200 OK status code.
APIs for provisioning search clusters provide an invaluable tool for automating search related tests. In this article, we described how cloud and automation technology provide a powerful engine for creating clusters, while provisioning APIs can further accelerate high-quality releases of search applications.
Learn how to build a headless Drupal site with React and SearchStax Studio.
It’s easier than ever to build customized site search with SearchStax Studio. Learn how to build Vue search apps for
In this blog, we delve into Headless Content Management Systems, their role in content storage and rendering, and various rendering
Copyrights © SearchStax Inc.2014-2023. All Rights Reserved.
close
close
