In our last blog post, we introduced your new AI Country Risk Assistant and explained how it can help you quickly and easily interrogate our quantitative risk scores. Now, we’ve given the Assistant a powerful new feature: the ability to answer questions about the contents of thousands of publicly available, long-form reports from over 50 reputable sources. 

Country risk research typically begins with a need for specific information, which might be prompted by a third-party request or the requirements of a longer research report. In turn, this will trigger a task or question, which might be as simple as “What are the underlying risks to Algeria’s external position?” or “What information is available about beneficial ownership registration?”; or more advanced, like “Compare the climate change vulnerabilities associated with Indonesia and Thailand”. 

Let’s look at a relatively straightforward example. Suppose you want to answer a research question about Kazakhstan’s climate-related risks. In the past, you’d probably search the web for reports that are likely to contain the information you need, such as the IMF’s Article IV report on Kazakhstan. Then, you’d download the PDFs, search them for relevant keywords, and write a summary of what you find. 

Now, let’s see how your improved AI Country Risk Assistant can take much of the legwork out of answering such a research question.

You’re probably familiar with CountryRisk.io’s Reports page, where you can search and filter all of the reports available on the platform by country, topic, source, or date. Now, many of these reports feature a new icon indicating that the AI Country Risk Assistant can answer questions about their contents. To give you as much control as possible, you choose which reports the AI Country Risk Assistant is allowed to consider before asking your question. 

For simplicity, we selected only the Article IV Kazakhstan report for this example:

Once you’ve selected the report(s) you want the AI Country Risk Assistant to look at, you can ask your question. In this case, we want to know what the Kazakhstan report has to say about climate change risks. So, we might ask:

Then, just hit Enter to get an answer based on the contents of the Article IV report: 

To verify the answer or find its source, click the Source button beneath the question field. This will reveal the original passages upon which the Assistant based its answer, along with links to their source documents: 

You can also see everything you’ve already asked about the topic at hand. So, if you didn’t get everything you need from the first answer, you can continue the conversation by asking follow-up questions:

Once you have what you need, you can copy any and all of the answers before pasting them into your report, presentation, or email.

We’ve yet to resolve the tradeoff between the amount of knowledge our custom LLM is capable of drawing upon and the inclusion of a full conversation history as part of that knowledge base. Until we can find a way to give the AI Country Risk Assistant a better conversational memory, then, it will answer each question in isolation.

Even so, we think you’ll agree this new feature is already powerful enough make answering many research questions significantly more efficient, so you can spend more of your time on valuable, creative work.

So, how does this feature actually work? The illustration below shows the workflow.

As you might have guessed, the most important component of the workflow is the knowledge base. Indeed, you might already have some sort of knowledge base on a shared drive consisting of all the reports you thought might be useful for future research. The problem is that shared drives tend to quickly turn into disorganised PDF graveyards that don’t give up their secrets easily. After all, maintaining a well-structured knowledge base is hard work. 

But it’s also something at which we at CountryRisk.io are well practiced. We already maintain a custom knowledge base consisting of an augmented version of our risk scores, which is what enables the LLM underlying the AI Country Risk Assistant to provide consistently high-quality answers to questions about those scores. Now, the Assistant can tap into a second custom knowledge base consisting of around 200,000 long form reports from more than 50 reputable sources we find valuable for country risk analysis. These include, among many, IMF Article IV country reports and FATF country assessments. All of these reports are technically available to anyone for free; but in practice, locating and acquiring them from their original sources can involve a lot of legwork.

The Assistant interrogates this custom report database using Retrieval-Augmented Generation (RAG), which combines the strengths of extractive (i.e. return specific text from the custom knowledge base) and generative (i.e. generate a custom answer based on available information via the LLM) question answering models. This means we don’t need to worry about keeping the reports organised—we can simply dump them into the knowledge base so you can find and choose the best content for your question via the search tool.

That said, the knowledge base is “custom” in that we must first do some data pre-processing on each report. As we’ve seen, LLMs have limited contextual memories, and some reports run to over 200 pages. Moreover, the more contextual information the LLM has to review, the greater the performance, financial, and environmental costs associated with each query. Until these technical issues are resolved, then, we must make each report as easy for the LLM to digest as possible.

As with any data analytics project, this process is complex enough to fill a blog post of its own. In short, though, we begin by converting the PDF to plain text and using sentence cleaning to strip out any non-content elements, like tables of contents, bibliographies, and footnotes. Then, we split the resulting ream of unstructured content into topic-specific chunks that the LLM can digest. This process can be fuzzy: although we aim for chunks of 10 sentences, we might need to experiment with different chunk lengths or using semantic breaks for splitting wherever a report devotes more or fewer than 10 sentences to a particular topic. 

Once we’ve entered the “chunkified” report into the custom knowledge base, the LLM needs to be able to determine which chunks are relevant to the question by comparing their semantic similarities. To summarise another involved process, we use word embedding—the representation of words as vectors, in which semantically similar words are closer in the vector space than semantically dissimilar words—generated via OpenAI’s text-embedding-ada-002 model, which we then store alongside the corresponding chunks in a vector database.

So, when you ask a question, the AI Country Risk Assistant converts it into a vector and retrieves the top n chunks with the highest semantic similarity until it reaches a length of 1,000 tokens across all chunks. The Assistant then hands the question and retrieved chunks to the LLM for a completion request, along with two instructions: first, the LLM can base its response only on information contained in the retrieved chunks; and second, that it should respond as if it were an analyst tasked with assessing country risks.

The AI Country Risk Assistant is a powerful tool that can help you answer specific questions from colleagues, write your own reports, or conduct a sovereign rating analysis more quickly and easily, especially when it comes to assessing qualitative risks like effectiveness of fiscal rule or strength of banking sector regulation. 

If you’re an enterprise client, you can even add proprietary or in-house reports to the existing repository to create your own bespoke knowledge base, so you and your colleagues can use the Assistant to get instant answers about any topic that’s salient for your organisation—which could come in handy at your next credit risk committee meeting. 

If you’d like a one-on-one demo of how you can use your own reports to create a custom knowledge base or you have ideas for improvements, you can send us an email at contact@countryrisk.io.

And now, for your next trick…: A new ability for your AI Country Risk Assistant

In this webinar, CountryRisk.io’s managing partner, Bernhard Obenhuber, discussed our latest product, your AI assistant: what it is, what it does, and what it means for you and your workflow. 

Bernhard also recapped our journey in creating this very practical and convenient tool. In the Q&A, he furthered detailed our suite of products -- to be unrolled over the coming weeks -- related to AI.

We thank Paula Haunit, Chief of Staff and Head of Customer Success at Oradian, for moderating the webinar. Paula  has worked with both co-founders of CountryRisk.io at Deutsche Bank Research, and is a seasoned economist and country risk expert.

Webinar: Your AI Country Risk Assistant: Accelerating your productivity, delivering value

In March this year, we wrote about the research and development we’ve been doing to explore how natural language generation tools might support better, more efficient country risk analysis. 

Now, after around six months of work by the CountryRisk.io team, we’re ready to make the first fruits of our labour available to our community. 

What we’ve been working on: a quick recap

At CountryRisk.io, we use our risk scoring methodology to approximate sovereigns’ default risk. If you’ve been with us a while, you’re probably already familiar with that methodology; otherwise, you’ll find it here. (Link). 

As a second step, we’ve been exploring how we can use natural language generation to convert data into text snippets, as well as adding details to numerical data. And, in tandem, we’ve been working to incorporate OpenAI’s large language model (LLM) into our platform, so our members can do so for themselves.

This feature is now available for our standard risk scores, which cover sovereign credit risk, ESG sovereign risk, AML, and supply chain country risk. When you select any country’s risk score, a new field will appear below the charts into which you can enter your question.

And, in case you could use some inspiration, we’ve pre-loaded the field with some common questions. 

You can be creative and provide prompts like "Draft an email to the credit risk committee with a summary for Angola".

Or for instance "I need python code for making a line chart for the sovereign risk score for Angola". Your country risk assistant cannot do everything but a lot. 

When using this new field, it’s important to remember that we’ve “instructed” the LLM to consider only the information we’ve provided. So, it can tell you all about Tunisia’s sovereign credit risk, but it won’t tell you which of the country’s sushi restaurants is best. While this restricts the amount of knowledge our LLM is able to talk about, it also significantly reduces the risk of hallucination without compromising its ability to answer the kinds of questions our community want to ask. 

Give it a try and let us know what you think.

Country risk information is needed in many different situations and formats, as well as by people with different levels of country risk expertise. This often means that economists must re-package the same content over and over again—a multi-page country report or presentation here, a statement for senior management there—all of which require additional time commitments.

As former country risk analysts, we’ve experienced the pain of losing all that time to such busy-work. Now, though, our community can use CountryRisk.io’s newly integrated LLM to start automating some of these rote tasks, so you have more time to actually use your hard-won expertise to add value through difficult, creative thought. 

We’ve started small with this first-generation version of our new LLM tool, and we’ve yet to unleash its full potential in terms of the amount of contextual data to which it has access. So, our focus now is on releasing more updates while integrating the tool elsewhere on the CountryRisk.io Insights platform. 

For instance, we’re currently working on adding the ability to use the LLM to interrogate long-form documents. This would enable you to ask questions like, “How about climate change in Brazil?”, which would prompt the LLM to review the latest IMF Article IV reports and return a summary based on all relevant paragraphs throughout these documents. We also plan to integrate the assistant into the sovereign rating model, so you can ask about a specific aspect of sovereign risk (e.g. quality of public debt management) or get help with writing a rating rationale and outlook. 

Finally, we’re working on a feature that will enable you to define your organisation’s specific exposure profile using criteria such as proportion of sales or portfolio investment exposure by country. This will unlock the ability to ask questions like, “Which countries contribute most to our country risk exposure?” or, “What are our organisation’s primary risk factors?”, and receive a response in natural language. We believe this exciting, unique feature has the potential to make our community members’ workflows significantly more efficient.

The changing role of economists and country risk analysts

It’s an exciting time to be working in our field. As the ongoing development of LLMs unlocks the possibility of automating an increasingly wide range of humdrum tasks, we’ll be able to use the time we save to produce better, more considered analyses. 

But we also believe that the nature of these professions and, in turn, the expertise they require, will change. As LLMs become more capable, analysts will have to focus more on orchestrating the entire process of knowledge generation and distribution:

With so much relevant data and information out there, it’ll be increasingly important to be aware of alternative data sources—not just the go-to ones. You’ll also need the technical skill to access and work with such data, as well as the experience required to assess their quality. And, since LLMs can interrogate long-form texts, analysts will increasingly need to regard such documents as sources of quality data alongside sources of traditional numeric or time-series data—something that, for many of us, could take some getting used to.

Here, we’re talking about the identification and collation of knowledge from data, especially those that enable an assessment for a particular research question (e.g. likelihood of sovereign default or recession). Knowledge, then, can take the form of numerical data (e.g., GDP growth: 1.5%) or informed statements (e.g. “We believe that GDP growth will be lower in the next decade compared to the past due to declining productivity growth and a shrinking labour pool”). And, since LLMs are capable of quickly producing such statements by themselves, natural language generation via heuristics is set to become an increasingly crucial area of knowledge creation. However, it will also be important to maintain our focus not just on quantity of knowledge created, but also on quality. Even an inexperienced economist can generate plenty of new data and insights, and LLMs will make doing so easier than ever. But a good economist will be able to identify the data points that are most relevant to every research topic, as well as how to transform them and then translate the results into code.

Of course, dumping the knowledge you create onto a shared drive is the easiest way to store it. But doing so will also make it difficult for anyone else to find and use that knowledge. And, should you decide to go elsewhere, your organisation may end up misplacing it entirely. To maximise the potential value of the knowledge you create, then, you must be able to make available to everyone in your organisation via a quality knowledge management system.  

If you can do the first three steps well, this last one should be a piece of cake. Self-service bots and their APIs will increasingly be powered by our knowledge so other departments or even clients can easily integrate it into their workflows and dashboards, including the generation of text snippets for standardised reports. This will give you more time to create compelling presentations, reports, and other bespoke work capable of feeding value back into the overall cycle. 

All of this amounts to a new era for country risk; one in which the very best analysts will be differentiated by a combination of extensive domain knowledge, confidence to form a view of the world, and mastery of the techniques and possibilities associated with machine learning.

If you’re interested in receiving a one-on-one demo of this new feature, learning about how you can apply the abilities it unlocks to your own data and risk scores, or have ideas for improvements, you can send us an email at contact@countryrisk.io.

Country risk analysis needn’t be protracted or complex. In this blog post, we introduce our CountryRisk.io AI Assistant. Our approach drives productivity and delivers value through independence, transparency, domain expertise and technology.

Imagine producing a country risk report in just minutes...

Webinar: Your AI Country Risk Assistant: Accelerating your productivity, delivering value

Jenny Asuncion
Sep 20, 2023

And now, for your next trick…: A new ability for your AI Country Risk Assistant

Imagine producing a country risk report in just minutes...

Written by:
Jenny Asuncion

Webinar: Your AI Country Risk Assistant: Accelerating your productivity, delivering value

Jenny AsuncionSep 20, 2023

And now, for your next trick…: A new ability for your AI Country Risk Assistant

Imagine producing a country risk report in just minutes...

Written by: Jenny Asuncion

Jenny Asuncion
Sep 20, 2023

Written by:
Jenny Asuncion