Use Lucene.Net in for faster data search
Introduction
This tutorial explains how to use Lucene.Net to achieve faster search responses in ASP.NET or MVC or C# applications.
Why Lucene.Net?
All business or consumer applications require data search and it can be implemented using SQL LIKE clause or SQL Full-Text Search. But we found that it is much faster to search data when we use Lucene.Net library.
Lucene is a full-text search library which uses inverted index for search due to which it is able to achieve fast search responses. You can use Lucene.Net library to create indexes and search to retrieve the data which can be sorted by relevance or by a sort field.
“Lucene.Net is a port of the Lucene search engine library, written in C# and targeted at .NET runtime users. The Lucene search library is based on an inverted index.”
Read more about Lucene.Net from Apache Lucene.Net or Lucene.Net GitHub.
Let’s start with the coding :)
Setup C# Lucene Project
Before you start writing your first example using Lucene.Net library, you have to make sure that you have setup your Lucene environment properly.
Let’s start by creating a new project with name - “LuceneConsoleApplication”. Open VS and New Project -> Visual C# -> Windows -> Console Application.
Add "Lucene.Net" references using NuGet Package Manager in your project.
Add below key in App.config file. This is the path where we will store the Lucene indexes. It will create "LuceneIndexes" in bin directory. We can even pull the data from SQL Server database and create Lucence indexes. In this tutorial and sample given below, we have created a list using C# code.
<add key="LuceneIndexStoragePath" value="LuceneIndexes" />
In code given below, the StartLuceneIndexCreateProcess() method will create Lucene indexes.
class Program
{
static void Main(string[] args)
{
try
{
LuceneIndexer startIndexer = new LuceneIndexer();
Console.WriteLine("Starting Lucene index creation process for search...\r\n");
Console.WriteLine("Lucene index has been created from the following list:");
// This method will create Lucene indexes according to your given list items.
startIndexer.CreateLuceneIndexes();
Console.WriteLine("");
// You can check CFX and CFS file in folder "..\LuceneConsoleApplication\App_Data".
// Open the file in Notepad or any other text editor and you can see list of items.
Console.WriteLine("Lucene Index creation successful!\r\n");
GetSearchLucene getSearchLucene = new GetSearchLucene();
Console.WriteLine("Enter a name to search:");
// Get input string from user
string inputParam = Console.ReadLine();
List result = getSearchLucene.GetSearchLuceneText(inputParam);
Console.WriteLine("");
Console.WriteLine("You searched for:");
if (result.Count > 0)
{
foreach (var item in result)
{
Console.WriteLine(item.Actor);
}
}
else
{
Console.WriteLine("Sorry! No matching records found.");
}
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
}
}
File #1 - Program.cs – Main() method
Create Indexes for Search
Now, let’s add list of items to CreateDocument() method.
Directory represents the storage location of the indexes. Write your desired directory path in the App.config file. Here we will create our index of our list items. User given input will searched with "Actor" key field.
private void StartLuceneIndexCreateProcess()
{
string luceneIndexStoragePath = @ConfigurationManager.AppSettings["LuceneIndexStoragePath"];
bool folderExists = System.IO.Directory.Exists(luceneIndexStoragePath);
if (!folderExists)
System.IO.Directory.CreateDirectory(luceneIndexStoragePath);
analyzer = new Lucene.Net.Analysis.Standard.StandardAnalyzer(Lucene.Net.Util.Version.LUCENE_30);
Lucene.Net.Store.Directory directory = Lucene.Net.Store.FSDirectory.Open(new System.IO.DirectoryInfo(luceneIndexStoragePath));
writer = new Lucene.Net.Index.IndexWriter(directory, analyzer, true, Lucene.Net.Index.IndexWriter.MaxFieldLength.LIMITED);
try
{
// We will populate below list to create Lucene index.
List actorsList = new List();
actorsList.Add("Johnny Depp");
actorsList.Add("Robert Downey Jr.");
actorsList.Add("Johnny Depp");
actorsList.Add("Tom Cruise");
actorsList.Add("Brad Pitt");
actorsList.Add("Tom Hanks");
actorsList.Add("Denzel Washington");
actorsList.Add("Russell Crowe");
actorsList.Add("Kate Winslet");
actorsList.Add("Christian Bale");
actorsList.Add("Hugh Jackman");
actorsList.Add("Will Smith");
actorsList.Add("Sean Connery");
foreach (var item in actorsList)
{
Console.WriteLine(item);
writer.AddDocument(CreateDocument(item.ToString()));
}
}
catch
{
Lucene.Net.Index.IndexWriter.Unlock(directory);
throw;
}
finally
{
writer.Optimize();
analyzer.Close();
writer.Dispose();
analyzer.Dispose();
}
}
private Lucene.Net.Documents.Document CreateDocument(string actorName)
{
try
{
Lucene.Net.Documents.Document doc = new Lucene.Net.Documents.Document();
doc.Add(new Lucene.Net.Documents.Field("actors", actorName, Lucene.Net.Documents.Field.Store.YES, Lucene.Net.Documents.Field.Index.ANALYZED));
return doc;
}
catch
{
throw;
}
}
File #2 – LuceneIndexer.cs - Contains code to create Lucene indexes
Search
Provided input will pass to GetSearchLuceneText() method. GetLuceneIndexPath() method will get the path of stored Lucene index i.e "bin\Debug\LuceneIndexes" folder. Input given by user will search in "LuceneIndexes" folder using Search() method.
public class GetSearchLucene
{
public List GetSearchLuceneText(string actorName)
{
LuceneSearcher searchIndex = new LuceneSearcher(GetLuceneIndexPath().FullName);
List results = new List();
if (actorName != string.Empty)
{
results = searchIndex.Search(actorName, "actors").ToList();
}
return results;
}
// Get directory path for Lucene index creation
public DirectoryInfo GetLuceneIndexPath()
{
return new DirectoryInfo(ConfigurationManager.AppSettings["LuceneIndexStoragePath"]);
}
}
File #3 - GetSearchLucene.cs - Contains code to invoke search on Lucene indexes
Search method
IndexSearcher class acts as a core component which searcher indexes created during indexing process. Lucene directory will point to location where indexes are to be stored. Initialize the QueryParser object created with a standard analyzer having version information and index name on which this query is to run.
public List Search(string searchText, string searchField)
{
List output = new List();
string searchQuery = string.Empty;
try
{
if (string.IsNullOrEmpty(searchText.Trim()))
{
throw new Exception("You forgot to enter something to search for...");
}
searchQuery = searchText;
}
catch
{
throw;
}
try
{
// Open the IndexReader with readOnly=true.
// This makes a big difference when multiple threads are sharing the same reader,
// as it removes certain sources of thread contention.
searcher = new Lucene.Net.Search.IndexSearcher(_directory, true);
analyzer = new Lucene.Net.Analysis.Standard.StandardAnalyzer(Lucene.Net.Util.Version.LUCENE_30);
output = (GetSearchResultByField(searchQuery, searchField, searcher, analyzer));
}
catch
{
throw;
}
finally
{
analyzer.Close();
analyzer.Dispose();
searcher.Dispose();
}
return output;
}
File #4.1 – LuceneSearcher.cs - Contain code to search on Lucene indexes
Mapping With Lucene
User given input will be mapped with search field and user input search will be fetched from document object.
private Lucene.Net.Search.Query ParseQuery(string searchQuery, Lucene.Net.QueryParsers.QueryParser parser)
{
Lucene.Net.Search.Query query;
try
{
query = parser.Parse(searchQuery.ToLower().Trim() + "*");
}
catch (Lucene.Net.QueryParsers.ParseException)
{
query = parser.Parse(Lucene.Net.QueryParsers.QueryParser.Escape(searchQuery.Trim()));
throw;
}
return query;
}
private IEnumerable MapLuceneDataToIDList(IEnumerable hits, Lucene.Net.Search.IndexSearcher searcher)
{
return hits.Select(hit => MapLuceneData(searcher.Doc(hit.Doc))).ToList();
}
// Mapping Lucene data
private LuceneData MapLuceneData(Lucene.Net.Documents.Document doc)
{
return new LuceneData
{
Actor = (doc.Get("actors"))
};
}
File #4.2 – LuceneSearcher.cs - Contains code to prase and map Lucene search data
You can download our sample LuceneConsoleApplication project from GitHub.
Happy Coding :)