Expect machine learning to reshape search in the enterprise

August 14, 2017 Lucidworks

Machine learning is becoming a staple of enterprise search applications, helping companies wrangle big data and make sense of it for their customers.

It’s being integrated into systems across many industries including financial services, healthcare, customer service and retail, to name a few.

Each of these industries has unique needs, but they share one particular challenge: They need to manage large amounts of structured and unstructured data.

Machine learning is a subset of artificial intelligence (AI) which looks at patterns to predict and understand future, similar data inputs. Although there’s a lot of hype around AI’s potential, enterprise search hasn’t yet achieved a truly intelligent system.

However, machine learning components already have real-world applications in enterprise search tools: helping stock traders recognize trends, retail customers find product information on a website, and employees pull data at a service center.

Smarter enterprise search enables employees and business leaders to more quickly and accurately retrieve information and enable a better user experience.

Understanding Queries

One of the biggest challenges facing enterprise search is better matching the right content with the right query. Part of this means understanding the “intent” of a query.

Does the user want a document, an answer, an opinion, or a fact? For example, note the difference between: “How do I invest in our company’s 401k?” vs. “What are the best options for my 401k plan?” The first could likely be answered by a company document or contact information for an HR employee. The latter requires supporting evidence and will likely be opinion-based.

Machine learning can aggregate and digest queries and signals across channels — listening to what people are talking about in Slack, email, and other collaboration tools — and customize search results based on past actions or actions of similar users. Instead of keyword searches that churn out irrelevant results, people want more sophisticated analysis of queries.

This high quality user experience is driving the evolution of the enterprise search industry. Using a subset of machine learning called natural language processing, the user is able to ask a question in a conversational manner and enterprise search applications can predict which data sources are most relevant.

Google is already doing this by sending their queries to RankBrain so the system can learn more about the semantics of search and teach itself how to better give users the answers it thinks they want.

Delivering Content Supported by Big Data

Another important application of machine learning to enterprise search is natural language generation. The sheer volume of unstructured data and its multiple sources make it difficult to extract useful insights or analysis.

This is frustrating considering the value of big data lies in a company’s ability to leverage it for insights about customers and users. Using natural language generation, enterprises can mine unstructured data to generate reports and summarize business intelligence insights.

All in all, machine learning has barely scratched the surface in enterprise search; there hasn’t been enough feedback from users to deploy it at maximum capacity.

However, the returns for early adopters will be compounded — the more user experience data these companies can gather, the more quickly they can adjust and improve search applications.

Companies have begun integrating machine learning components into their search platforms to make big data more valuable than it’s ever been. Employees can efficiently access what they’re looking for, customers’ high Google-like expectations are satisfied, and insights from “humanized” data are easily accessible for non-experts.

2017 could be the year that enterprises properly wrangle big data, with machine learning powering it all.

By Grant Ingersoll, CTO of Lucidworks

Join 30,000+ people who read the weekly 🤖Machine Learnings🤖newsletter to understand how AI will impact the way they work.


Expect machine learning to reshape search in the enterprise was originally published in Machine Learnings on Medium, where people are continuing the conversation by highlighting and responding to this story.

Previous Article
Instagram eliminating trolls with machine learning
Instagram eliminating trolls with machine learning

SourceAwesome, not awesome.#Awesome“Hate crimes in America have historically been difficult to track since ...

Next Article
It’s Time for a New Interface
It’s Time for a New Interface

What will the world look like when our primary means of interaction with technology is voice, and no longer...