From Headlines to Reality: Building Effective Business Solutions with LLMs

Judging based on the weekly headlines in the news, Large Language Models (LLMs) can do just about anything… But how do you translate ‘anything’ into something that actually makes an impact on your business?

At Agent Factory, we’ve been designing and building business solutions that leverage language models for over a decade. With the introduction of LLMs, we’ve come to appreciate how tasks that required a lot of R&D efforts in the past have become easy to implement once you have an LLM at your disposal. With that said, don’t let shiny demos fool you. As convincing as they seem, your actual implementation is rarely as trivial. Keep in mind that demos are well crafted to achieve a Wow effect quickly and tend to skip all the common pitfalls that your implementation will likely need to cope with. More often than not, we hear customers saying “I saw this demo yesterday…” or “I tried this tool…” convinced that what they are asking for is trivial and can be implemented in a couple of days (and, of course, cost close to nothing). But the devil is in the details and once you start the conversation and dive a little deeper, the complexity is revealed. The key to a successful implementation is understanding this complexity and navigating it while minimizing risk and optimizing business impact.

The goal of this post is to help you set your expectations as you approach your first/next LLM-based project by exposing some of the complexity involved in building them. Starting off with the key question: What should you build?

Common Business Scenarios for LLMs

While it seems like the options are limitless, we find that for many businesses integrating LLMs means one of these two scenarios:

Making Data More Accessible

Making your data more accessible to users is always a challenge. Businesses produce data at a growing rate, yet our search and retrieval solutions (read: search box) forces users to ‘understand’ and ‘speak’ the specific dialect of the data repository in order to be effective in finding the information they need. This leads to having only a handful of ‘expert’ employees that can find the answers to key questions. Needless to say, it does not fully unlock the potential value of this data.

With LLMs, Natural Language information retrieval becomes accessible at a much lower cost than ever before. The promise is that users will be able to ask their questions the same way they would ask a co-worker for help, while receiving well formulated (and grounded) answers. This time around, all users can leverage this technology to access data and unlock much more of its potential.

Business Process Automation

Many organizations rely on business workflows that require processing documents. While the wave of digitalization reduced dramatically the number of cases in which actual paper documents are processed, it did not remove the need to process the documents that arrive digitally. Examples range from processing purchase orders from customers to extracting information from hospitalization summaries and many in between.

Automation of such workflows has been extremely costly and required on-going maintenance since it lacked robustness. In addition, even when automation was built, more commonly than not, these workflows remain dependent on humans to some extent limiting their potential business value. Humans are typically required when the tasks are too complex to automate (e.g. reading and reasoning over free-text forms) or certain expertise is required (e.g. law/business/engineering/medical/you-name-it know-how is required to process the text).

With LLMs, a new level of robustness can be achieved at a much lower upfront cost and the dependency on humans in the loop can be reduced or even completely removed. LLMs come we wide and depth knowledge and can be easily extended with more specific knowledge required for the task (i.e. context window).

Sounds Promising, But What Are the Pitfalls?

In the next couple of blog posts, we will dive deeper into the key aspects you need to consider when you start planning your organization LLM integration.

In the meanwhile, feel free to reach out if you want to explore ways we can assist you on this journey.