Here’s how semantic search will transform capital markets

The world is becoming increasingly connected and the information growth rate continues to skyrocket with no signs of slowing down.

As per Reuters, global data will grow to approximately 35 zettabytes in 2020 from its current levels of 8 zettabytes i.e. approximately 35% CAGR.

Mobile technologies, social media and ever evolving internet business models have fueled this growth so far and Internet of Things (IoT) threatens to take this to unimaginable levels.

Innovations like universal search, semantic search, knowledge graphs, Ok Google/Cortana/Siri are steps in the right direction to enable a future where search will transform into a smart machine that uses "connected knowledge" to answer complex and specific questions.

Semantic search and value addition of an intelligent search engine is going to benefit trading firms in a big way too.

As the markets are also becoming connected and competitive, and are undergo digital transformation, firms are looking for search solutions that allow them to absorb, traverse, and find relevant information through volumes of structured and unstructured data quickly to generate alpha.

For investment managers and portfolio managers, a search platform that goes across investment research, market data, company fundamentals, news, social media and portfolio structure, and enables varied set of queries, is a holy-grail.

From their perspective, it would be great if search platforms could support free-form queries like 'news published in last one week about suppliers of companies in technology sector showing more than 5% quarter-over-quarter growth sorted by news sentiment'. Ideal situation is to have less than 10 search results, all relevant.

This is quite challenging to achieve as this involves interpreting free-form query, expanding it to multiple queries across datasets, and then linking, filtering, ranking the results.

And this is where semantic technologies offer solution to this challenge.

Solution involves building domain-centric Ontology, knowledge about the domain i.e. the concepts, linguistic elements like synonyms, facts, relationships among concepts etc.

For instance, 'market risk' is synonymous to terms like 'systemic risk' and 'value-at-risk' (VAR) and terms like 'variance-covariance', 'monte-carlo' are linked to 'market risk' as these are measures for such a risk.

Domain knowledge is stored in a flexible structure as "subject-predicate-object". In the phrase 'Samsung is a supplier of Apple', Samsung is the subject, 'is a supplier of' is the predicate and 'Apple' is the object.

Each of these elements is uniquely identified. Once data is prepared in this manner, relationships among subjects/objects can be used to prompt users to build complex queries.

In addition, natural language processing can be employed to transform a free-form query into queries that platform could understand. While, the first approach is more accurate as platform-assisted query building rules out misinterpretation, the second approach could also be refined over time using machine learning techniques.

Machine learning techniques can also be employed to continuously evolve effectiveness of the platform from query-response, ontology evolution and entity extraction standpoints.

The platform could be further evolved to enable natural language analytical queries like 'compare Google and Apple using 5-day moving average over last 4 weeks' and support subsequent complex iterative analysis.

Combined with natural language processing abilities, and speech-to-text and gestures support, the users interface for these searches can be extremely simplified.

This means that the user is either applying a few available filters or speaking out their financial query and receiving a response just as Ironman would receive from Jarvis in the Marvel universe.

While the value of semantic search platforms is easy to establish in the context of investment research, there are other related areas where it can help players in the financial ecosystem.

Trade surveillance for exchanges and regulators, tracking regulatory compliance for brokerages and investment firms, and legal contracts management/case management for legal entities are some of these areas.

Considering these abilities of semantic search platforms, I feel Semantic search platform will prove to be a powerful tool in CIO’s armory and market players could look at tapping business opportunities which were unexplored so far due to lack of ability to sift through huge amounts of structured and unstructured data.

(The author of this article is Nitin Agrawal, Director Technology, Sapient Global Markets)

(Image: Thinkstock)