FinGPT: Revolutionizing Open-source Finance with Data-Centric Approach

Name: Matt Popovic

Updated on 8/17/2023

Why FinGPT?

The financial landscape is highly dynamic, making it challenging to keep up with the constant changes. Traditional methods of retraining LLMs using a mixed dataset of finance and general data sources can be costly and time-consuming. For instance, BloombergGPT, another LLM, requires approximately 1.3 million GPU hours for retraining, costing around $5 million. This makes it impractical to retrain an LLM model every month or every week.

FinGPT presents a more accessible alternative. It prioritizes lightweight adaptation, leveraging the strengths of some of the best available open-source LLMs. These models are then fed with financial data and fine-tuned for financial language modeling. The cost of adaptation falls significantly, estimated at less than $300 per training, making FinGPT a cost-effective solution.

How FinGPT Works:

As we continue our exploration of FinGPT, let's delve deeper into its technical aspects. FinGPT is built on the foundation of open-source LLMs, which are fine-tuned with financial data for financial language modeling. This process is made possible through the use of reinforcement learning from human feedback (RLHF), a technology that allows the model to learn individual preferences.

One of the critical aspects of FinGPT is its focus on open-sourced internet-scale financial data. This allows for timely updates (monthly or weekly) using an automatic data curation pipeline. Unlike BloombergGPT, which has privileged data access and APIs, FinGPT is more accessible and prioritizes lightweight adaptation.
Another one of the key strengths of FinGPT is its ability to adapt swiftly to new data. This is made possible through its lightweight adaptation approach, which significantly reduces the cost of training. This makes FinGPT not only a powerful tool in the financial industry but also a cost-effective solution.
FinGPT also prioritizes the internet-scale financial data. This is achieved through an automatic data curation pipeline, which allows for timely updates. This feature sets FinGPT apart from other LLMs, making it a more accessible alternative.

The key technology behind FinGPT is "RLHF (Reinforcement learning from human feedback)". This technology, which is missing in BloombergGPT, enables an LLM model to learn individual preferences such as risk-aversion level, investing habits, and personalized robo-advisor. This is the "secret" ingredient of ChatGPT and GPT4, making FinGPT a powerful tool in the financial industry.

FinGPT Demos: A Detailed Walkthrough

FinGPT provides several demos to showcase its capabilities and offer a practical guide for users. These demos are designed to demonstrate how to train your own FinGPT model in different financial markets. Let's delve into the details of these demos.

FinGPT V1: Chinese Financial Market with ChatGLM and LoRA

The first demo, FinGPT V1, focuses on the Chinese Financial Market. It utilizes ChatGLM and LoRA (Low-Rank Adaptation) to train the FinGPT model. Here's a step-by-step guide on how to use it:

Set up the environment: Before starting, ensure that you have the necessary packages installed. You can do this by running the command pip install -r requirements.txt in your terminal.
Import the necessary modules: Import the necessary modules for training the model. This includes FinGPT, ChatGLM, and LoRA among others.

from fingpt import FinGPT
from chatglm import ChatGLM
from lora import LoRA

Initialize the model: Initialize the FinGPT model with the desired parameters.

model = FinGPT(
    num_tokens=50257,
    dim=768,
    depth=12,
    heads=12,
)

Train the model: Train the model using the ChatGLM and LoRA.

model.train(ChatGLM, LoRA)

Save the model: After training, save the model for future use.

model.save("fingpt_v1.pth")

FinGPT V2: American Financial Market with LLaMA and LoRA

The second demo, FinGPT V2, focuses on the American Financial Market. It uses LLaMA and LoRA for training the FinGPT model. Here's a step-by-step guide on how to use it:

Set up the environment: As with the first demo, ensure that you have the necessary packages installed.
Import the necessary modules: Import the necessary modules for training the model. This includes FinGPT, LLaMA, and LoRA among others.

from fingpt import FinGPT
from llama import LLaMA
from lora import LoRA

Initialize the model: Initialize the FinGPT model with the desired parameters.

model = FinGPT(
    num_tokens=50257,
    dim=768,
    depth=12,
    heads=12,
)

Train the model: Train the model using the LLaMA and LoRA.

model.train(LLaMA, LoRA)

Save the model: After training, save the model for future use.

model.save("fingpt_v2.pth")

These demos provide a practical guide on how to train your own FinGPT model. By following these steps, you can leverage the power of FinGPT to analyze and predict financial markets.

Conclusion

We hope this article has provided you with a comprehensive understanding of FinGPT and its potential to revolutionize the financial industry. Stay tuned for more insights into the world of open-source finance!

FAQs

As we conclude our exploration of FinGPT, let's address some frequently asked questions about this revolutionary tool.

What is FinGPT?

FinGPT is a data-centric tool developed by the AI4Finance Foundation. It democratizes financial data for large language models (LLMs), providing an open-source solution for open finance.

How does FinGPT work?

FinGPT works by leveraging the strengths of some of the best available open-source LLMs. These models are fed with financial data and fine-tuned for financial language modeling. The key technology behind FinGPT is RLHF (Reinforcement learning from human feedback), which enables the model to learn individual preferences.

What sets FinGPT apart from other LLMs?

FinGPT stands out due to its focus on democratizing internet-scale financial data and its unique RLHF technology. It also offers a cost-effective solution, with the cost of adaptation estimated at less than $300 per training.