Enhance your workflows: The power of MuleSoft AI Chain (MAC)

AI is like a brain in a jar, without the spine that connects to the rest of the world, it’s worthless. This project provides a bridge – the MuleSoft AI Chain connector – to connect AI services with all MuleSoft components. Although the project is still in the development phase, it promises quite interesting and useful tools in the near future. Some of them are connectors to NLP libraries, Deep Learning models, and language translations, as well as connectors for providing web crawling capabilities.

“AI engines are like a brain in a jar— capable of providing insights but lacking the nervous system needed to automate processes, commands, or updates within an organization, making implementation difficult.”

– Founder of MuleSoft

This article focuses on the custom connector interacting with the AI agents. The project was made on top of the langChain4j project to enable easy access to MuleSoft components. The project is known as the MuleSoft AI Chain or MAC for short.

TL;DR

The MuleSoft AI Chain Project connects AI services with MuleSoft components. This way, it enables functionalities of sentiment analysis, chat prompts, image generation, document analysis, and embeds stores for querying stored documents. It supports multiple large language models, is open-source, and can be deployed via CloudHub or on-premise, making setup straightforward and flexible.

Although still in development, the project promises significant potential for enhancing customer support, market research, and content creation. Key concerns include data leakage, inconsistent AI responses, and response time issues. Despite these, it offers powerful tools that can transform tasks from hours to minutes, inviting contributions and direct communication with developers.

Good to know before setting up the MAC connector:

MAC supports the following LLMs (Large Language Models) configurations:

---

How to set up the connector?

The setup of the project is straightforward and can be found via this link: mac-project.ai

While setting up the connector you may or may not run into the following issues:

---

Sentiment Analyzer

Chat prompt

Chat answer operation with memory

Checkout the examples with chat history:

Images

With the MAC connector, you can generate images based on your input as well as analyze them and generate information based on the images.

Read images

With this operation, you can analyze images to detect and count specific objects, such as cars in a parking lot or products on a shelf. It also allows you to extract text from documents, receipts, invoices, or handwritten notes, and effectively handle text in various languages and alphabets, including Cyrillic and Arabic!

INPUT: Prompt about the image and URL of the image -> OUTPUT: AI-generated answer based on the prompt

Some interesting use cases that we tested:

Rag Load Document

With this operation, you can analyze documents and extract relevant information. The AI model extracts and uses relevant information to provide an answer or respond to the request. Information is retrieved from an in-memory embedding store that works in the background. RAG is short for Retrieval-Augmented Generation.

INPUT:

1. User request/command (prompt)
2. File Path to the document
3. File Type –
   • pdf
   • text – Any type of text file (JSON, XML, txt, CSV, etc.)
   • URI – Only single URL supported

OUTPUT: AI-generated answer based on the prompt and document provided

Checkout this example with a specification for a thermostat:

Use Cases:

• Enhancing chatbot responses – By retrieving relevant documents, the chatbot can provide more accurate and informative answers to user queries.
• Content creation – Writers and journalists can use RAG to gather background information and ensure their articles are well-informed and comprehensive.
• Customer support – RAG can assist customer service agents by quickly retrieving relevant troubleshooting guides or policies to resolve customer issues efficiently.
• Research – You can access information from research papers or documents based on specific queries.

Embeddings using documents

One of the cool features MAC provides is the ability to add documents to a store. This way, you can ask for information and the AI will search for this information inside all documents so you can ‘talk’ with your stored documents. That can be quite helpful to reduce the amount of wrong information the AI gives you.

I’ll explain how this works using 3 MAC operations.

Embedding new store

The first step is to create your embedding store. This can easily be done by using the “embedding new store” operation and providing a store name. This store name will be reused in the other operation. So in practice, you’ll probably put this in one of your config files.

Embedding add document to store

In this test, I used the following URL to retrieve some information about Cloud Hub 2.0 https://docs.mulesoft.com/cloudhub-2/ch2-deploy.

To be able to extract information from one of your documents you’ll need to store those documents as well. Currently, you can store 3 types of documents:

So, what you need to do to add the document to the store is use the “embedding add document to store” function?
In this function, you’ll need to use the same store name as the one you’ve just created. As well as the context file of the document. You also have to choose the file type you’re using.

Embedding get info from store

Finally, you can ask AI for the information you want. In this example I asked ”Which replica statuses exist in CloudHub and when do they occur?”. To be able to ask the information you’ll need to use the “embedding get info from store” function. You’ll need to add the name of the store where the document containing your information is stored together with the question (data) you want to ask.

Once we do this request we’ll receive something like this. This way, we are able to talk with our documents.

Use cases to use this function are for example:

• Knowledge base Initialization: Storing all of our Knowledge base documents into one store and create a chatbot which we can ask for information that exist in those documents without the need to go through all of them on our own.
• Data Segmentation: Setting up new stores for different types of data or documents for better organisation.
• Custom Embedding Stores: Creating specialized embedding stores tailored to specific applications or use cases, such as industry-specific document storage.

Agent Define Prompt Template

Another function within the MAC connector is the agent-defined prompt template. This function can be used to basically tell what the AI needs to do. This is done by using 3 input fields.

I tested this by trying to let the agent create some kind of passport based on a written text. In case not all information is given the complete field is false.

Please be aware that the agent will require extensive training, and you will need to fine-tune the template to minimize errors. In this scenario, the agent will simply be gathering some information to populate all the necessary fields.

Example use cases for this function are:

• Customer Service Agents: Enhance customer service by providing case summaries, case classifications, summarizing large datasets, and more.
• Sales Operation Agents: Aid sales teams in writing sales emails, summarizing cases for specific accounts, assessing the probability of closing deals, and more.
• Marketing Agents: Assist marketing teams in generating product descriptions, creating newsletters, planning social media campaigns, and more.

AI tools

The tool’s operation can be used for querying APIs autonomously with the capabilities of AI services. Querying is done using natural language. The concept of tools applies to any asset presented in the Anypoint Exchange. The collection of these ‘tools’ is provided via a .json file.

Example of a config.json (you can name it as you like) file:

[
    {
        "action":"Execute GET requests for API endpoints.",
        "url": "https://anypoint.mulesoft.com/mocking/api/v1/sources/exchange/assets/7b99cead-a984-497b-9e6c-c16a3b4dcb76/employee-api/1.0.1/m/employees",
        "name": "Show accounts",
        "method": "GET",
        "headers": "",
        "example-payload":"{}",
        "query": [
            "Get all accounts",
            "Show all accounts"
            "Show accounts for the country Switzerland", 
            "How many accounts do we have", 
            "Show a list of all accounts"
        ],
        "description": "Get all information about all accounts. This action applies whenever users' intent is 'accounts', 'workforce', 'get accounts'. Use the headers to perform the request."
    }
]

In this example we used the public API from the official documentation in order to see how it works on a larger dataset:

Combining the query request to the API and a query with a generic question is not possible:

Conclusion

Although the project is rather new, it offers interesting features with limitless potential. These AI services are very flexible and can be integrated into various systems. In the examples above, we demonstrated just the basic use cases of these operations but it’s up to you how you will use them! These tools can reduce tasks that once took hours to just a few minutes. One of the benefits of this project is that it’s open-source, allowing you to contribute easily and communicate directly with the developers about any issues you encounter.

Wow! But…

Some of the risks and downsides of using it:

• During data processing or transmission, there is a risk of data being intercepted or unintentionally exposed to unauthorized parties. In the case of document embedding you have possession of your storage which causes only the functions using that storage to use that data.
• When asking questions to AI, differences between answers for the same questions can occur within the same integration environment and that’s something to keep in mind.
• While testing the functions of the MuleSoft AI Chain connector, we noticed 2 to 10-second response times. This may cause timeouts & other performance issues.

Additional Materials

Want to know more? Or implement this yourself? Reach out and let’s have a chat (GPT)!