{"id":1438,"date":"2015-01-17T13:51:49","date_gmt":"2015-01-17T10:21:49","guid":{"rendered":"http:\/\/vua.nadiran.com\/?p=1438"},"modified":"2015-01-19T14:02:54","modified_gmt":"2015-01-19T10:32:54","slug":"1438","status":"publish","type":"post","link":"https:\/\/vua.nadiran.com\/?p=1438","title":{"rendered":"ElasticWho?"},"content":{"rendered":"<div style=\"direction: ltr; align: left;\"><span style=\"font-size: 1.5em; line-height: 19px;\">ElasticWho?<\/span><\/p>\n<section>ElasticSearch is a flexible and powerful open source, distributed real-time search and analytics engine.<\/p>\n<p>&nbsp;<\/p>\n<\/section>\n<section>\n<h2>Features<\/h2>\n<ul>\n<li>Real time analytics<\/li>\n<li>Distributed<\/li>\n<li>High availability<\/li>\n<li>Multi tenant architecture<\/li>\n<li>Full text<\/li>\n<li>Document oriented<\/li>\n<li>Schema free<\/li>\n<li>RESTful API<\/li>\n<li>Per-operation persistence<\/li>\n<\/ul>\n<\/section>\n<section>\n<h2>Distributed<\/h2>\n<p>Start small and scale horizontally out of the box. For more capacity, just add more nodes and let the cluster reorganize itself.<\/p>\n<\/section>\n<section>\n<h2>High Availability<\/h2>\n<p><a href=\"http:\/\/vua.nadiran.com\/wp-content\/uploads\/highly-available.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter\" alt=\"highly-available\" src=\"http:\/\/vua.nadiran.com\/wp-content\/uploads\/highly-available.png\" width=\"462\" height=\"200\" \/><\/a><\/p>\n<p>ElasticSearch clusters detect and remove failed nodes, and reorganize themselves.<\/p>\n<\/section>\n<section>\n<h2>Multi Tenancy<\/h2>\n<pre><code data-trim=\"\">\r\n$ curl -XPUT http:\/\/localhost:9200\/people\r\n\r\n$ curl -XPUT http:\/\/localhost:9200\/gems\r\n\r\n$ curl -XPUT http:\/\/localhost:9200\/gems\/document\/pry-0.5.9\r\n\r\n$ curl -XGET http:\/\/localhost:9200\/gems\/document\/pry-0.5.9\r\n\t\t\t\t\t\t<\/code><\/pre>\n<p>A cluster can host multiple indices which can be queried independently, or as a group.<\/p>\n<\/section>\n<section>\n<h2>Document Oriented<\/h2>\n<pre><code data-trim=\"\">\r\n{\r\n    \"_id\": \"pry-0.5.9\", \r\n    \"_index\": \"gems\", \r\n    \"_source\": {\r\n        \"authors\": [\r\n            \"John Mair (banisterfiend)\"\r\n        ], \r\n        \"autorequire\": null, \r\n        \"bindir\": \"bin\", \r\n        \"cert_chain\": [], \r\n        \"date\": \"Sun Feb 20 11:00:00 UTC 2011\", \r\n        \"default_executable\": null, \r\n        \"description\": \"attach an irb-like session to any object at runtime\", \r\n        \"email\": \"jrmair@gmail.com\"\r\n    }\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<p>Store complex real world entities in Elasticsearch as structured JSON documents.<\/p>\n<\/section>\n<section>\n<h2>RESTful API<\/h2>\n<p>Almost any operation can be performed using a simple RESTful interface using\u00a0<strong>JSON<\/strong>\u00a0over\u00a0<strong>HTTP<\/strong>.<\/p>\n<ul>\n<li>curl -X GET<\/li>\n<li>curl -X PUT<\/li>\n<li>curl -X POST<\/li>\n<li>curl -X DELETE<\/li>\n<\/ul>\n<\/section>\n<section>\n<h2>Apache Lucene<\/h2>\n<p>ElasticSearch is built on top of Apache Lucene. Lucene is a high performance, full-featured Information Retrieval library, written in Java.<\/p>\n<p><a href=\"http:\/\/vua.nadiran.com\/wp-content\/uploads\/highly-available.png\">\u00a0<\/a><\/p>\n<\/section>\n<section>\n<h2>ElasticSearch Terminology<\/h2>\n<\/section>\n<section>\n<h2>Document<\/h2>\n<p>$ curl -XGET http:\/\/localhost:9200\/gems\/document\/pry-0.5.9<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n    \"_id\": \"pry-0.5.9\", \r\n    \"_index\": \"gems\", \r\n    \"_source\": {\r\n        \"authors\": [\r\n            \"John Mair (banisterfiend)\"\r\n        ], \r\n        \"autorequire\": null, \r\n        \"bindir\": \"bin\", \r\n        \"cert_chain\": [], \r\n        \"date\": \"Sun Feb 20 11:00:00 UTC 2011\", \r\n        \"default_executable\": null, \r\n        \"description\": \"attach an irb-like session to any object at runtime\", \r\n        \"email\": \"jrmair@gmail.com\", \r\n        \"executables\": [\r\n            \"pry\"\r\n        ], \r\n        \"extensions\": [], \r\n        \"extra_rdoc_files\": [], \r\n        \"files\": [\r\n            \"lib\/pry\/commands.rb\", \r\n            \"lib\/pry\/command_base.rb\", \r\n            \"lib\/pry\/completion.rb\", \r\n            \"lib\/pry\/core_extensions.rb\", \r\n            \"lib\/pry\/hooks.rb\", \r\n            \"lib\/pry\/print.rb\", \r\n            \"lib\/pry\/prompts.rb\", \r\n            \"lib\/pry\/pry_class.rb\", \r\n            \"lib\/pry\/pry_instance.rb\", \r\n            \"lib\/pry\/version.rb\", \r\n            \"lib\/pry.rb\", \r\n            \"examples\/example_basic.rb\", \r\n            \"examples\/example_commands.rb\", \r\n            \"examples\/example_command_override.rb\", \r\n            \"examples\/example_hooks.rb\", \r\n            \"examples\/example_image_edit.rb\", \r\n            \"examples\/example_input.rb\", \r\n            \"examples\/example_input2.rb\", \r\n            \"examples\/example_output.rb\", \r\n            \"examples\/example_print.rb\", \r\n            \"examples\/example_prompt.rb\", \r\n            \"test\/test.rb\", \r\n            \"test\/test_helper.rb\", \r\n            \"CHANGELOG\", \r\n            \"LICENSE\", \r\n            \"README.markdown\", \r\n            \"Rakefile\", \r\n            \".gemtest\", \r\n            \"bin\/pry\"\r\n        ], \r\n        \"has_rdoc\": true, \r\n        \"homepage\": \"http:\/\/banisterfiend.wordpress.com\", \r\n        \"id\": \"pry-0.5.9\", \r\n        \"licenses\": [], \r\n        \"name\": \"pry\", \r\n        \"platform\": \"ruby\", \r\n        \"post_install_message\": null, \r\n        \"rdoc_options\": [], \r\n        \"require_paths\": [\r\n            \"lib\"\r\n        ], \r\n        \"requirements\": [], \r\n        \"rubyforge_project\": null, \r\n        \"rubygems_version\": \"1.5.2\", \r\n        \"signing_key\": null, \r\n        \"specification_version\": 3, \r\n        \"summary\": \"attach an irb-like session to any object at runtime\", \r\n        \"test_files\": [], \r\n        \"version\": {\r\n            \"prerelease\": null, \r\n            \"version\": \"0.5.9\"\r\n        }\r\n    }, \r\n    \"_type\": \"document\", \r\n    \"_version\": 1, \r\n    \"exists\": true\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<p>In ElasticSearch, everything is stored as a Document. Document can be addressed and retrieved by querying their attributes.<\/p>\n<\/section>\n<section>\n<h2>Document Types<\/h2>\n<p>Lets us specify document properties, so we can differentiate the objects.<\/p>\n<\/section>\n<section>\n<h2>Shard<\/h2>\n<p>Each Shard is a separate\u00a0<em>native<\/em>\u00a0Lucene Index. Lets us overcome RAM limitations, hard disk capacity.<\/p>\n<\/section>\n<section>\n<h2>Replica<\/h2>\n<p>An exact copy of primary Shard. Helps in setting up HA, increases query throughput.<\/p>\n<\/section>\n<section>\n<h2>Index<\/h2>\n<p>ElasticSearch stores its data in logical Indices. Think of a table, collection or a database.<\/p>\n<p><img decoding=\"async\" alt=\"ElasticSearch Index\" src=\"gem-index-head.png\" \/><\/p>\n<p>An Index has atleast 1 primary Shard, and 0 or more Replicas.<\/p>\n<\/section>\n<section>\n<h2>Cluster<\/h2>\n<p>A collection of cooperating ElasticSearch nodes. Gives better availability and performance via Index Sharding and Replicas.<\/p>\n<\/section>\n<section>\n<h2>ElasticSearch Workshop<\/h2>\n<\/section>\n<section>\n<h2>Download and start<\/h2>\n<p>Download ElasticSearch from\u00a0<a href=\"http:\/\/www.elasticsearch.org\/download\">http:\/\/www.elasticsearch.org\/download<\/a><\/p>\n<pre><code data-trim=\"\">\r\n\t\t\t\t\t\t\t# service elasticsearch start\r\n\t\t\t\t\t<\/code><\/pre>\n<pre><code data-trim=\"\">\r\n\t\t\t\t\t\t\t# \/etc\/init.d\/elasticsearch start\r\n\t\t\t\t\t<\/code><\/pre>\n<pre><code data-trim=\"\">\r\n\t\t\t\t\t\t\t# .\/bin\/elasticsearch -f\r\n\t\t\t\t\t<\/code><\/pre>\n<\/section>\n<section>\n<h2>ElasticSearch Plugins<\/h2>\n<p>A site plugin to view contents of ElasticSearch cluster.<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n# cd \/usr\/share\/elasticsearch\r\n# .\/bin\/plugin -install mobz\/elasticsearch-head\r\n\t\t\t\t\t<\/code><\/pre>\n<pre><code data-trim=\"\">\r\n# cd \/opt\/elasticsearch-0.90.2\r\n# .\/bin\/plugin -install mobz\/elasticsearch-head\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>Restart ElasticSearch. Plugins are detected and loaded on service startup.<\/p>\n<\/section>\n<section>\n<h2>elasticsearch-head<\/h2>\n<p><img decoding=\"async\" alt=\"\" src=\"elasticsearch-head.png\" \/><\/p>\n<\/section>\n<section>\n<h2>RESTful interface<\/h2>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n$ curl -XGET 'http:\/\/localhost:9200\/'\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n  \"ok\" : true,\r\n  \"status\" : 200,\r\n  \"name\" : \"Drake, Frank\",\r\n  \"version\" : {\r\n    \"number\" : \"0.90.2\",\r\n    \"snapshot_build\" : false,\r\n    \"lucene_version\" : \"4.3.1\"\r\n  },\r\n  \"tagline\" : \"You Know, for Search\"\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<\/section>\n<section>\n<h2>Create Index<\/h2>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n$ curl -XPUT 'http:\/\/localhost:9200\/gems'\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n  \"ok\":true,\r\n  \"acknowledged\":true\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<\/section>\n<section>\n<h2>Cluster status<\/h2>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n$ curl -XGET 'localhost:9200\/_status'\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\"ok\":true,\"_shards\":{\"total\":20,\"successful\":10,\"failed\":0},\r\n\"indices\":{\"gems\":{\"index\":{\"primary_size\":\"495b\",\"primary_size_in_bytes\":495,\r\n\"size\":\"495b\",\"size_in_bytes\":495},\"translog\":{\"operations\":0},\r\n\"docs\":{\"num_docs\":0,\"max_doc\":0,\"deleted_docs\":0},\"merges\":\r\n{\"current\":0,\"current_docs\":0,\"current_size\":\"0b\",\"current_size_in_bytes\":0,\r\n\"total\":0,\"total_time\":\"0s\",\"total_time_in_millis\":0,\"total_docs\":0,\r\n\"total_size\":\"0b\",\"total_size_in_bytes\":0},\r\n...\r\n...\r\n...\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<\/section>\n<section>\n<h2>Pretty Output<\/h2>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n$ curl -XGET 'localhost:9200\/_status?pretty'\r\n\t\t\t\t\t<\/code><\/pre>\n<pre><code data-trim=\"\">\r\n$ curl -XGET 'localhost:9200\/_status' | python -mjson.tool\r\n\t\t\t\t\t<\/code><\/pre>\n<pre><code data-trim=\"\">\r\n$ curl -XGET 'localhost:9200\/_status' | json_reformat\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n    \"ok\": true,\r\n    \"_shards\": {\r\n        \"total\": 20,\r\n        \"successful\": 10,\r\n        \"failed\": 0\r\n    },\r\n    \"indices\": {\r\n        \"gems\": {\r\n            \"index\": {\r\n                \"primary_size\": \"495b\",\r\n                \"primary_size_in_bytes\": 495,\r\n                \"size\": \"495b\",\r\n                \"size_in_bytes\": 495\r\n            },\r\n...\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<\/section>\n<section>\n<h2>Delete Index<\/h2>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n$ curl -XDELETE 'http:\/\/localhost:9200\/gems'\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n  \"ok\":true,\r\n  \"acknowledged\":true\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<\/section>\n<section>\n<h2>Create custom Index<\/h2>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n    \"settings\" : {\r\n        \"index\" : {\r\n            \"number_of_shards\" : 6,\r\n            \"number_of_replicas\" : 0\r\n        }\r\n    }\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<pre><code data-trim=\"\">\r\n$ curl -XPUT 'http:\/\/localhost:9200\/gems' -d @body.json\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n  \"ok\":true,\r\n  \"acknowledged\":true\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<\/section>\n<section>\n<h2>Index a document<\/h2>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n  \"name\": \"pry\", \r\n  \"platform\": \"ruby\", \r\n  \"rubygems_version\": \"1.5.2\", \r\n  \"description\": \"attach an irb-like session to any object at runtime\", \r\n  \"email\": \"anurag@example.com\", \r\n  \"has_rdoc\": true, \r\n  \"homepage\": \"http:\/\/banisterfiend.wordpress.com\"\r\n}\r\n\r\n\t\t\t\t\t<\/code><\/pre>\n<pre><code data-trim=\"\">\r\n$ curl -XPOST 'http:\/\/localhost:9200\/gems\/test\/' -d @body.json\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n  \"ok\":true,\r\n  \"_index\":\"gems\",\r\n  \"_type\":\"test\",\r\n  \"_id\":\"lsJgxiwET6eg\",\r\n  \"_version\":1\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<\/section>\n<section>\n<h2>Get document<\/h2>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n$ curl -XGET 'http:\/\/localhost:9200\/gems\/test\/lsJgxiwET6eg' | python -mjson.tool\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n    \"_id\": \"lsJgxiwET6eg\", \r\n    \"_index\": \"gems\", \r\n    \"_source\": {\r\n        \"description\": \"attach an irb-like session to any object at runtime\", \r\n        \"email\": \"anurag@example.com\", \r\n        \"has_rdoc\": true, \r\n        \"homepage\": \"http:\/\/banisterfiend.wordpress.com\", \r\n        \"name\": \"pry\", \r\n        \"platform\": \"ruby\", \r\n        \"rubygems_version\": \"1.5.2\"\r\n    }, \r\n    \"_type\": \"test\", \r\n    \"_version\": 1, \r\n    \"exists\": true\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<\/section>\n<section>\n<h2>Index another document<\/h2>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n  \"name\": \"grit\", \r\n  \"platform\": \"jruby\", \r\n  \"rubygems_version\": \"2.5.0\", \r\n  \"description\": \"Ruby library for extracting information from a git repository.\", \r\n  \"email\": \"mojombo@github.com\", \r\n  \"has_rdoc\": false,\r\n  \"homepage\": \"http:\/\/github.com\/mojombo\/grit\"\r\n}\r\n\r\n\t\t\t\t\t<\/code><\/pre>\n<pre><code data-trim=\"\">\r\n$ curl -XPOST 'http:\/\/localhost:9200\/gems\/test\/' -d @body.json\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n  \"ok\":true,\r\n  \"_index\":\"gems\",\r\n  \"_type\":\"test\",\r\n  \"_id\":\"ijUOHi2cQc2\",\r\n  \"_version\":1\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<\/section>\n<section>\n<h2>Custom Document IDs<\/h2>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n  \"name\": \"grit\", \r\n  \"platform\": \"jruby\", \r\n  \"rubygems_version\": \"2.5.1\", \r\n  \"description\": \"Ruby library for extracting information from a git repository.\", \r\n  \"email\": \"mojombo@github.com\", \r\n  \"has_rdoc\": false,\r\n  \"homepage\": \"http:\/\/github.com\/mojombo\/grit\"\r\n}\r\n\r\n\t\t\t\t\t<\/code><\/pre>\n<pre><code data-trim=\"\">\r\n$ curl -XPUT 'http:\/\/localhost:9200\/gems\/test\/grit-2.5.1' -d @body.json\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n  \"ok\":true,\r\n  \"_index\":\"gems\",\r\n  \"_type\":\"test\",\r\n  \"_id\":\"grit-2.5.1\",\r\n  \"_version\":1\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>IDs are unique across Index. Composed of DocumentType and ID.<\/p>\n<\/section>\n<section>\n<h2>Document Versions<\/h2>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n$ curl -XPUT 'http:\/\/localhost:9200\/gems\/test\/grit-2.5.1' -d @body.json\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n  \"ok\":true,\r\n  \"_index\":\"gems\",\r\n  \"_type\":\"test\",\r\n  \"_id\":\"grit-2.5.1\",\r\n  \"_version\":2\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<\/section>\n<section>\n<h2>Searching Documents<\/h2>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n  \"query\": {\r\n    \"term\": {\"name\": \"pry\"}\r\n  }\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n$ curl -XPOST http:\/\/localhost:9200\/gems\/_search -d @body.json | python -mjson.tool\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n  \"_shards\": {\r\n    \"failed\": 0, \r\n    \"successful\": 6, \r\n    \"total\": 6\r\n  },\r\n  \"hits\": {\r\n    \"hits\": [\r\n      {\r\n        \"_id\": \"MWkKgzsMRgK\", \r\n        \"_index\": \"gems\", \r\n        \"_score\": 1.4054651, \r\n        \"_source\": {\r\n          \"description\": \"attach an irb-like session to any object at runtime\", \r\n          \"email\": \"anurag@example.com\", \r\n          \"has_rdoc\": true, \r\n          \"homepage\": \"http:\/\/banisterfiend.wordpress.com\", \r\n          \"name\": \"pry\", \r\n          \"platform\": \"ruby\", \r\n          \"rubygems_version\": \"1.5.2\"\r\n        }, \r\n        \"_type\": \"test\"\r\n      }\r\n    ], \r\n    \"max_score\": 1.4054651, \r\n    \"total\": 1\r\n  }, \r\n  \"timed_out\": false, \r\n  \"took\": 2\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<\/section>\n<section>\n<h2>Counting Documents<\/h2>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n  \"term\": {\"name\": \"pry\"}\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n$ curl -XGET http:\/\/localhost:9200\/gems\/test\/_count -d @body.json\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n    \"_shards\": {\r\n        \"failed\": 0, \r\n        \"successful\": 6, \r\n        \"total\": 6\r\n    }, \r\n    \"count\": 1\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<\/section>\n<section>\n<h2>Update a Document<\/h2>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n  \"doc\": {\r\n   \"platform\": \"macruby\" \r\n  }\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n$ curl -XPOST http:\/\/localhost:9200\/gems\/test\/grit-2.5.1\/_update -d @body.json\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n  \"ok\":true,\r\n  \"_index\":\"gems\",\r\n  \"_type\":\"test\",\r\n  \"_id\":\"grit-2.5.1\",\r\n  \"_version\":4\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>The partial document is merged using simple recursive merge.<\/p>\n<\/section>\n<section>\n<h2>Update via Script<\/h2>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n    \"script\" : \"ctx._source.platform = vm_name\",\r\n    \"params\" : {\r\n        \"vm_name\" : \"rubinius\"\r\n    }\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n$ curl -XPOST http:\/\/localhost:9200\/gems\/test\/grit-2.5.1\/_update -d @body.json\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n  \"ok\":true,\r\n  \"_index\":\"gems\",\r\n  \"_type\":\"test\",\r\n  \"_id\":\"grit-2.5.1\",\r\n  \"_version\":5\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<\/section>\n<section>\n<h2>Delete Document<\/h2>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n$ curl -XDELETE 'http:\/\/localhost:9200\/gems\/test\/grit-2.5.1'\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n  \"ok\":true, \r\n  \"found\":true,\r\n  \"_index\":\"gems\",\r\n  \"_type\":\"test\",\r\n  \"_id\":\"grit-2.5.1\",\r\n  \"_version\":6\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<\/section>\n<section>\n<h2>Put Mapping<\/h2>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n  \"gem\" : {\r\n    \"properties\" : {\r\n      \"name\" :        {\"type\" : \"string\", \"index\": \"not_analyzed\"},\r\n      \"platform\" :    {\"type\" : \"string\", \"index\": \"not_analyzed\"},\r\n      \"rubygems_version\" : {\"type\" : \"string\", \"index\": \"not_analyzed\"},\r\n      \"description\" : {\"type\" : \"string\", \"store\" : \"yes\"},\r\n      \"has_rdoc\" :    {\"type\" : \"boolean\"}      \r\n    }\r\n  }\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n$ curl -XPUT 'http:\/\/localhost:9200\/gems\/gem\/_mapping' -d @body.json\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n$ curl -XGET 'http:\/\/localhost:9200\/gems\/_mapping' | python -mjson.tool\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<\/section>\n<section>\n<h2>Index Document with Mapping<\/h2>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n  \"name\": \"grit\", \r\n  \"platform\": \"ruby\", \r\n  \"rubygems_version\": \"2.5.1\", \r\n  \"description\": \"Ruby library for extracting information from a git repository.\", \r\n  \"email\": \"mojombo@github.com\", \r\n  \"has_rdoc\": false,\r\n  \"homepage\": \"http:\/\/github.com\/mojombo\/grit\"\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<pre><code data-trim=\"\">\r\n$ curl -XPUT 'http:\/\/localhost:9200\/gems\/gem\/grit-2.5.1' -d @body.json\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n  \"ok\":true,\r\n  \"_index\":\"gems\",\r\n  \"_type\":\"gem\",\r\n  \"_id\":\"grit-2.5.1\",\r\n  \"_version\":1\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<\/section>\n<section>\n<h2>Matching documents<\/h2>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n  \"query\": {\r\n    \"match\" : {\r\n        \"description\" : \"git repository\"\r\n    }\r\n  }\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n$ curl -XPOST http:\/\/localhost:9200\/gems\/gem\/_search -d @body.json\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<\/section>\n<section>\n<h2>Highlighting<\/h2>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n  \"query\": {\r\n    \"match\" : {\r\n        \"description\" : \"git repository\"\r\n    }\r\n  },\r\n  \"highlight\" : {\r\n        \"fields\" : {\r\n            \"description\" : {}\r\n        }\r\n    }\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n$ curl -XPOST http:\/\/localhost:9200\/gems\/gem\/_search -d @body.json\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n\"highlight\": {\r\n  \"description\": [\r\n    \"Ruby library for extracting information from a <em>git<\/em> <em>repository<\/em>.\"\r\n  ]\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<\/section>\n<section>\n<h2>Search Facets<\/h2>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n{\r\n  \"query\": { \"match_all\" : {} },\r\n  \"facets\" : {\r\n    \"gem_names\" : {\r\n      \"terms\" : { \"field\": \"name\" }\r\n    }\r\n  }\r\n}\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n$ curl -XPOST http:\/\/localhost:9200\/gems\/_search -d @body.json\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n...\r\n  \"facets\": {\r\n    \"gem_names\": {\r\n      \"_type\": \"terms\", \r\n      \"missing\": 0, \r\n      \"other\": 0, \r\n      \"terms\": [\r\n        {\r\n          \"count\": 2, \r\n          \"term\": \"pry\"\r\n        }, \r\n        {\r\n          \"count\": 2, \r\n          \"term\": \"grit\"\r\n        }, \r\n        {\r\n          \"count\": 1, \r\n          \"term\": \"abc\"\r\n        }\r\n      ], \r\n      \"total\": 5\r\n    }\r\n  },\r\n  \"hits\": {\r\n    \"hits\": [\r\n...\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<\/section>\n<section>\n<h2>(Lab)<\/h2>\n<h2>Analyzing Aadhaar&#8217;s Datasets<\/h2>\n<\/section>\n<section>\n<h2>Download Public Dataset<\/h2>\n<p>Download from Aadhaar Public Data Portal at\u00a0<a href=\"https:\/\/data.uidai.gov.in\/uiddatacatalog\/getDatsetInfo.do?dataset=UIDAI-ENR-DETAIL\">https:\/\/data.uidai.gov.in<\/a><\/p>\n<\/section>\n<section>\n<h2>Download Tools<\/h2>\n<p>$ git clone https:\/\/github.com\/gnurag\/aadhaar<\/p>\n<\/section>\n<section>\n<h2>Prepare Data &amp; Configure<\/h2>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n# gem install yajl-ruby tire activesupport\r\n\r\n$ git clone https:\/\/github.com\/gnurag\/aadhaar\r\n$ cd aadhaar\/data\r\n$ unzip UIDAI-ENR-DETAIL-20121001.zip\r\n$ cd ..\/bin\r\n$ vi aadhaar.rb\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<\/section>\n<section>\n<h2>Configuration<\/h2>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\nAADHAAR_DATA_DIR = \"\/path\/to\/aadhaar\/data\"\r\nES_URL           = \"http:\/\/localhost:9200\"\r\nES_INDEX         = 'aadhaar'\r\nES_TYPE          = \"UID\"\r\nBATCH_SIZE       = 1000\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<\/section>\n<section>\n<h2>Index<\/h2>\n<p>$ ruby aadhaar.rb<\/p>\n<\/section>\n<section>\n<h2>Running Examples<\/h2>\n<p>$ curl -XPOST http:\/\/localhost:9200\/aadhaar\/UID\/_search -d @template.json | python -mjson.tool<\/p>\n<\/section>\n<section>\n<h2>Additional Notes<\/h2>\n<\/section>\n<section>\n<h2>Index Aliases<\/h2>\n<p>Group multiple Indexes, and query them together.<\/p>\n<pre><code data-trim=\"\">\r\ncurl -XPOST 'http:\/\/localhost:9200\/_aliases' -d '\r\n{\r\n    \"actions\" : [\r\n        { \"add\" : { \"index\" : \"index1\", \"alias\" : \"master-alias\" } }\r\n        { \"add\" : { \"index\" : \"index2\", \"alias\" : \"master-alias\" } }\r\n    ]\r\n}'\r\n\t\t\t\t\t<\/code><\/pre>\n<pre><code data-trim=\"\">\r\ncurl -XPOST 'http:\/\/localhost:9200\/_aliases' -d '\r\n{\r\n    \"actions\" : [\r\n        { \"remove\" : { \"index\" : \"index2\", \"alias\" : \"master-alias\" } }\r\n    ]\r\n}'\r\n\t\t\t\t\t<\/code><\/pre>\n<\/section>\n<section>\n<h2>Document Routing<\/h2>\n<p>Control which Shard the document will be placed and queried from.<\/p>\n<\/section>\n<section>\n<h2>Parents &amp; Children<\/h2>\n<p>&nbsp;<\/p>\n<pre><code data-trim=\"\">\r\n$ curl -XPUT http:\/\/localhost:9200\/gems\/gem\/roxml?parent=rexml -d '{\r\n    \"tag\" : \"something\"\r\n}'\r\n\t\t\t\t\t<\/code><\/pre>\n<p>&nbsp;<\/p>\n<\/section>\n<section>\n<h2>Custom Analyzers<\/h2>\n<p>&nbsp;<\/p>\n<\/section>\n<section>\n<h2>Boosting Search Results<\/h2>\n<p>&nbsp;<\/p>\n<\/section>\n<section>\n<h2>ElasticSearch Ecosystem<\/h2>\n<p>A wide range of site plugins, analyzers, river plugins available from the community.<\/p>\n<\/section>\n<section>\n<h1>THE END<\/h1>\n<\/section>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>ElasticWho? ElasticSearch is a flexible and powerful open source, distributed real-time search and analytics engine. &nbsp; Features Real time analytics Distributed High availability Multi tenant architecture Full text Document oriented Schema free RESTful API Per-operation persistence Distributed Start small and scale horizontally out of the box. For more capacity, just add more nodes and let <a href='https:\/\/vua.nadiran.com\/?p=1438' class='excerpt-more'>[&#8230;]<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"link","meta":{"footnotes":""},"categories":[34],"tags":[37],"class_list":["post-1438","post","type-post","status-publish","format-link","hentry","category-34","tag-elasticsearch","post_format-post-format-link","category-34-id","post-seq-1","post-parity-odd","meta-position-corners","fix"],"_links":{"self":[{"href":"https:\/\/vua.nadiran.com\/index.php?rest_route=\/wp\/v2\/posts\/1438"}],"collection":[{"href":"https:\/\/vua.nadiran.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/vua.nadiran.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/vua.nadiran.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/vua.nadiran.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1438"}],"version-history":[{"count":4,"href":"https:\/\/vua.nadiran.com\/index.php?rest_route=\/wp\/v2\/posts\/1438\/revisions"}],"predecessor-version":[{"id":1440,"href":"https:\/\/vua.nadiran.com\/index.php?rest_route=\/wp\/v2\/posts\/1438\/revisions\/1440"}],"wp:attachment":[{"href":"https:\/\/vua.nadiran.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1438"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/vua.nadiran.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1438"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/vua.nadiran.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1438"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}