search-indexer-engines.html 14 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197
  1. <!DOCTYPE html>
  2. <html lang="en">
  3. <head>
  4. <meta charset="utf-8" />
  5. <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  6. <meta name="viewport" content="width=device-width, initial-scale=1">
  7. <title>Query your local search engines &#8212; Searx Documentation (Searx-1.1.0.tex)</title>
  8. <link rel="stylesheet" type="text/css" href="../_static/pygments.css" />
  9. <link rel="stylesheet" type="text/css" href="../_static/searx.css" />
  10. <link rel="stylesheet" type="text/css" href="../_static/tabs.css" />
  11. <script data-url_root="../" id="documentation_options" src="../_static/documentation_options.js"></script>
  12. <script src="../_static/jquery.js"></script>
  13. <script src="../_static/underscore.js"></script>
  14. <script src="../_static/_sphinx_javascript_frameworks_compat.js"></script>
  15. <script src="../_static/doctools.js"></script>
  16. <script src="../_static/sphinx_highlight.js"></script>
  17. <link rel="index" title="Index" href="../genindex.html" />
  18. <link rel="search" title="Search" href="../search.html" />
  19. <link rel="next" title="Running shell commands to fetch results" href="command-line-engines.html" />
  20. <link rel="prev" title="Query SQL servers" href="sql-engines.html" />
  21. </head><body>
  22. <div class="related" role="navigation" aria-label="related navigation">
  23. <h3>Navigation</h3>
  24. <ul>
  25. <li class="right" style="margin-right: 10px">
  26. <a href="../genindex.html" title="General Index"
  27. accesskey="I">index</a></li>
  28. <li class="right" >
  29. <a href="../py-modindex.html" title="Python Module Index"
  30. >modules</a> |</li>
  31. <li class="right" >
  32. <a href="command-line-engines.html" title="Running shell commands to fetch results"
  33. accesskey="N">next</a> |</li>
  34. <li class="right" >
  35. <a href="sql-engines.html" title="Query SQL servers"
  36. accesskey="P">previous</a> |</li>
  37. <li class="nav-item nav-item-0"><a href="../index.html">Searx Documentation (Searx-1.1.0.tex)</a> &#187;</li>
  38. <li class="nav-item nav-item-1"><a href="index.html" accesskey="U">Blog</a> &#187;</li>
  39. <li class="nav-item nav-item-this"><a href="">Query your local search engines</a></li>
  40. </ul>
  41. </div>
  42. <div class="document">
  43. <div class="documentwrapper">
  44. <div class="bodywrapper">
  45. <div class="body" role="main">
  46. <section id="query-your-local-search-engines">
  47. <h1>Query your local search engines<a class="headerlink" href="#query-your-local-search-engines" title="Permalink to this heading">¶</a></h1>
  48. <p>From now on, searx lets you to query your locally running search engines. The following
  49. ones are supported now:</p>
  50. <ul class="simple">
  51. <li><p><a class="reference external" href="https://www.elastic.co/elasticsearch/">Elasticsearch</a></p></li>
  52. <li><p><a class="reference external" href="https://www.meilisearch.com/">Meilisearch</a></p></li>
  53. <li><p><a class="reference external" href="https://solr.apache.org/">Solr</a></p></li>
  54. </ul>
  55. <p>All of the engines above are added to <code class="docutils literal notranslate"><span class="pre">settings.yml</span></code> just commented out, as you have to
  56. <code class="docutils literal notranslate"><span class="pre">base_url</span></code> for all them.</p>
  57. <p>Please note that if you are not using HTTPS to access these engines, you have to enable
  58. HTTP requests by setting <code class="docutils literal notranslate"><span class="pre">enable_http</span></code> to <code class="docutils literal notranslate"><span class="pre">True</span></code>.</p>
  59. <p>Furthermore, if you do not want to expose these engines on a public instance, you can
  60. still add them and limit the access by setting <code class="docutils literal notranslate"><span class="pre">tokens</span></code> as described in the <a class="reference external" href="private-engines.html#private-engines">blog post about
  61. private engines</a>.</p>
  62. <section id="configuring-searx-for-search-engines">
  63. <h2>Configuring searx for search engines<a class="headerlink" href="#configuring-searx-for-search-engines" title="Permalink to this heading">¶</a></h2>
  64. <p>Each search engine is powerful, capable of full-text search.</p>
  65. <section id="elasticsearch">
  66. <h3>Elasticsearch<a class="headerlink" href="#elasticsearch" title="Permalink to this heading">¶</a></h3>
  67. <p>Elasticsearch supports numerous ways to query the data it is storing. At the moment
  68. the engine supports the most popular search methods: <code class="docutils literal notranslate"><span class="pre">match</span></code>, <code class="docutils literal notranslate"><span class="pre">simple_query_string</span></code>, <code class="docutils literal notranslate"><span class="pre">term</span></code> and <code class="docutils literal notranslate"><span class="pre">terms</span></code>.</p>
  69. <p>If none of the methods fit your use case, you can select <code class="docutils literal notranslate"><span class="pre">custom</span></code> query type and provide the JSON payload
  70. searx has to submit to Elasticsearch in <code class="docutils literal notranslate"><span class="pre">custom_query_json</span></code>.</p>
  71. <p>The following is an example configuration for an Elasticsearch instance with authentication
  72. configured to read from <code class="docutils literal notranslate"><span class="pre">my-index</span></code> index.</p>
  73. <div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">elasticsearch</span><span class="w"></span>
  74. <span class="w"> </span><span class="nt">shortcut </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">es</span><span class="w"></span>
  75. <span class="w"> </span><span class="nt">engine </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">elasticsearch</span><span class="w"></span>
  76. <span class="w"> </span><span class="nt">base_url </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">http://localhost:9200</span><span class="w"></span>
  77. <span class="w"> </span><span class="nt">username </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">elastic</span><span class="w"></span>
  78. <span class="w"> </span><span class="nt">password </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">changeme</span><span class="w"></span>
  79. <span class="w"> </span><span class="nt">index </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">my-index</span><span class="w"></span>
  80. <span class="w"> </span><span class="nt">query_type </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">match</span><span class="w"></span>
  81. <span class="w"> </span><span class="nt">enable_http </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">True</span><span class="w"></span>
  82. </pre></div>
  83. </div>
  84. </section>
  85. <section id="meilisearch">
  86. <h3>Meilisearch<a class="headerlink" href="#meilisearch" title="Permalink to this heading">¶</a></h3>
  87. <p>This search engine is aimed at individuals and small companies. It is designed for
  88. small-scale (less than 10 million documents) data collections. E.g. it is great for storing
  89. web pages you have visited and searching in the contents later.</p>
  90. <p>The engine supports faceted search, so you can search in a subset of documents of the collection.
  91. Furthermore, you can search in Meilisearch instances that require authentication by setting <code class="docutils literal notranslate"><span class="pre">auth_token</span></code>.</p>
  92. <p>Here is a simple example to query a Meilisearch instance:</p>
  93. <div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">meilisearch</span><span class="w"></span>
  94. <span class="w"> </span><span class="nt">engine </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">meilisearch</span><span class="w"></span>
  95. <span class="w"> </span><span class="nt">shortcut</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">mes</span><span class="w"></span>
  96. <span class="w"> </span><span class="nt">base_url </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">http://localhost:7700</span><span class="w"></span>
  97. <span class="w"> </span><span class="nt">index </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">my-index</span><span class="w"></span>
  98. <span class="w"> </span><span class="nt">enable_http</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">True</span><span class="w"></span>
  99. </pre></div>
  100. </div>
  101. </section>
  102. <section id="solr">
  103. <h3>Solr<a class="headerlink" href="#solr" title="Permalink to this heading">¶</a></h3>
  104. <p>Solr is a popular search engine based on Lucene, just like Elasticsearch.
  105. But instead of searching in indices, you can search in collections.</p>
  106. <p>This is an example configuration for searching in the collection <code class="docutils literal notranslate"><span class="pre">my-collection</span></code> and get
  107. the results in ascending order.</p>
  108. <div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">solr</span><span class="w"></span>
  109. <span class="w"> </span><span class="nt">engine </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">solr</span><span class="w"></span>
  110. <span class="w"> </span><span class="nt">shortcut </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">slr</span><span class="w"></span>
  111. <span class="w"> </span><span class="nt">base_url </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">http://localhost:8983</span><span class="w"></span>
  112. <span class="w"> </span><span class="nt">collection </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">my-collection</span><span class="w"></span>
  113. <span class="w"> </span><span class="nt">sort </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">asc</span><span class="w"></span>
  114. <span class="w"> </span><span class="nt">enable_http </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">True</span><span class="w"></span>
  115. </pre></div>
  116. </div>
  117. </section>
  118. </section>
  119. <section id="next-steps">
  120. <h2>Next steps<a class="headerlink" href="#next-steps" title="Permalink to this heading">¶</a></h2>
  121. <p>The next step is to add support for various SQL databases.</p>
  122. </section>
  123. <section id="acknowledgement">
  124. <h2>Acknowledgement<a class="headerlink" href="#acknowledgement" title="Permalink to this heading">¶</a></h2>
  125. <p>This development was sponsored by <a class="reference external" href="https://nlnet.nl/discovery">Search and Discovery Fund</a> of <a class="reference external" href="https://nlnet.nl/">NLnet Foundation</a> .</p>
  126. <div class="line-block">
  127. <div class="line">Happy hacking.</div>
  128. <div class="line">kvch // 2021.04.07 23:16</div>
  129. </div>
  130. </section>
  131. </section>
  132. <div class="clearer"></div>
  133. </div>
  134. </div>
  135. </div>
  136. <span id="sidebar-top"></span>
  137. <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
  138. <div class="sphinxsidebarwrapper">
  139. <p class="logo"><a href="../index.html">
  140. <img class="logo" src="../_static/searx_logo_small.png" alt="Logo"/>
  141. </a></p>
  142. <h3>Project Links</h3>
  143. <ul>
  144. <li><a href="https://searx.github.io/searx/blog/index.html">Blog</a>
  145. <li><a href="https://github.com/searx/searx">Source</a>
  146. <li><a href="https://github.com/searx/searx/wiki">Wiki</a>
  147. <li><a href="https://twitter.com/Searx_engine">Twitter</a>
  148. <li><a href="https://github.com/searx/searx/issues">Issue Tracker</a>
  149. </ul><h3>Navigation</h3>
  150. <ul>
  151. <li><a href="../index.html">Overview</a>
  152. <ul>
  153. <li><a href="index.html">Blog</a>
  154. <ul>
  155. <li>Previous: <a href="sql-engines.html" title="previous chapter">Query SQL servers</a>
  156. <li>Next: <a href="command-line-engines.html" title="next chapter">Running shell commands to fetch results</a></ul>
  157. </li>
  158. </ul>
  159. </li>
  160. </ul>
  161. <div id="searchbox" style="display: none" role="search">
  162. <h3 id="searchlabel">Quick search</h3>
  163. <div class="searchformwrapper">
  164. <form class="search" action="../search.html" method="get">
  165. <input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
  166. <input type="submit" value="Go" />
  167. </form>
  168. </div>
  169. </div>
  170. <script>document.getElementById('searchbox').style.display = "block"</script>
  171. </div>
  172. </div>
  173. <div class="clearer"></div>
  174. </div>
  175. <div class="footer" role="contentinfo">
  176. &#169; Copyright 2015-2022, Adam Tauber, Noémi Ványi.
  177. Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> 5.3.0.
  178. </div>
  179. <script src="../_static/version_warning_offset.js"></script>
  180. </body>
  181. </html>