AI LLM (opensource on local Server) for Financial Data Analysis
Budget: €250 – €750 EUR
I'm seeking an experienced AI LLM Consultant with a proven track record in developing and implementing Artificial Intelligence solutions based on Large Language Models (LLMs).
The goal is to create a locally deployable LLM system (on-premise), PRE-TRAINED in the financial domain (Mistral, FinBERT, Bloom, FinGPT, BloombergGPT, ...), capable of learning from internal documentation (Intranet) and prioritizing this internal information.
The environment is Windows Server 2019 with the ability to run virtual machines.
Ensure the system operates fully offline, independent of any external cloud services.
Responsibilities:
- Evaluate and select the most suitable open-source LLM for a Windows server environment and specific financial needs (DeepSeek is preferred, then Mistral 7B).
- Design and implement the architecture for on-premise LLM installation, ensuring complete independence from external cloud services or other AI LLM APIs.
- Develop and configure the system for ingesting and indexing documentary sources from our Intranet.
- Implement mechanisms to PRIORITIZE information from the Intranet over the LLM's pre-trained knowledge base (LoRA/QLoRA).
At the end of project:
- Provide technical support and training to our internal team for managing and further training the system.
- Document the entire architecture and implementation processes.
Essential Technical Requirements:
- Familiarity with Windows Server environment and managing hardware resources for AI.
- Deep understanding of Large Language Models (LLMs) and their architectures.
- Proven experience with open-source LLMs (e.g., Llama, Falcon, Mistral, etc.).
- Hands-on experience in installing and configuring on-premise LLMs on Windows servers.
- Familiarity with fine-tuning techniques, RAG (Retrieval Augmented Generation), and continuous learning.
Desirable Requirements:
- Previous experience in AI projects within the financial sector.
----- Would you like to bid on this project? -----
To best evaluate your application and technical approach, please include a dedicated section addressing the following points:
- Minimum Hardware Estimation: Provide a detailed estimate of the minimum hardware requirements (CPU, RAM, GPU, storage) for a Windows server capable of hosting the proposed LLM system, considering both training and inference needs with our internal documentation.
- Required Software: List the essential software components (operating systems, runtimes, libraries, specific tools) that would be necessary to implement and operate the on-premise LLM system.
- Proposed Architecture Design: Present a preliminary design of the architecture to be implemented, specifying the main components (logical units, databases, applications, virtual machines, etc.). Please note that considerations for High Availability (HA), Disaster Recovery (DR), or Load Balancing are not required.
- A detailed activity plan (day/task) as part of your proposal.
----- How will it be used? -----
Mainly via API/WebService: a web application (ASP.NET) will collect requests from the connected user and request answers from the AI service.
However, a web interface (such as ChatGPT, Claude, ...) is required to make impromptu requests.
----- Costs and quality -----
Please pay attention on environment performances.
Please, no ask me «what is your budget for this project?»
Bonuses provided at the end of the project for compliance with the timing and quality of the software.
No upfront. No payment before successful completion of all tests.
Unnecessary images, files, libraries, ... must be removed.
----- Collaboration -----
The consulting engagement will commence in March and will be conducted full-remote.
Participation in daily update meetings is mandatory and non-negotiable. Failure to adhere to this requirement will result in immediate project disengagement, without exception.
The goal is to create a locally deployable LLM system (on-premise), PRE-TRAINED in the financial domain (Mistral, FinBERT, Bloom, FinGPT, BloombergGPT, ...), capable of learning from internal documentation (Intranet) and prioritizing this internal information.
The environment is Windows Server 2019 with the ability to run virtual machines.
Ensure the system operates fully offline, independent of any external cloud services.
Responsibilities:
- Evaluate and select the most suitable open-source LLM for a Windows server environment and specific financial needs (DeepSeek is preferred, then Mistral 7B).
- Design and implement the architecture for on-premise LLM installation, ensuring complete independence from external cloud services or other AI LLM APIs.
- Develop and configure the system for ingesting and indexing documentary sources from our Intranet.
- Implement mechanisms to PRIORITIZE information from the Intranet over the LLM's pre-trained knowledge base (LoRA/QLoRA).
At the end of project:
- Provide technical support and training to our internal team for managing and further training the system.
- Document the entire architecture and implementation processes.
Essential Technical Requirements:
- Familiarity with Windows Server environment and managing hardware resources for AI.
- Deep understanding of Large Language Models (LLMs) and their architectures.
- Proven experience with open-source LLMs (e.g., Llama, Falcon, Mistral, etc.).
- Hands-on experience in installing and configuring on-premise LLMs on Windows servers.
- Familiarity with fine-tuning techniques, RAG (Retrieval Augmented Generation), and continuous learning.
Desirable Requirements:
- Previous experience in AI projects within the financial sector.
----- Would you like to bid on this project? -----
To best evaluate your application and technical approach, please include a dedicated section addressing the following points:
- Minimum Hardware Estimation: Provide a detailed estimate of the minimum hardware requirements (CPU, RAM, GPU, storage) for a Windows server capable of hosting the proposed LLM system, considering both training and inference needs with our internal documentation.
- Required Software: List the essential software components (operating systems, runtimes, libraries, specific tools) that would be necessary to implement and operate the on-premise LLM system.
- Proposed Architecture Design: Present a preliminary design of the architecture to be implemented, specifying the main components (logical units, databases, applications, virtual machines, etc.). Please note that considerations for High Availability (HA), Disaster Recovery (DR), or Load Balancing are not required.
- A detailed activity plan (day/task) as part of your proposal.
----- How will it be used? -----
Mainly via API/WebService: a web application (ASP.NET) will collect requests from the connected user and request answers from the AI service.
However, a web interface (such as ChatGPT, Claude, ...) is required to make impromptu requests.
----- Costs and quality -----
Please pay attention on environment performances.
Please, no ask me «what is your budget for this project?»
Bonuses provided at the end of the project for compliance with the timing and quality of the software.
No upfront. No payment before successful completion of all tests.
Unnecessary images, files, libraries, ... must be removed.
----- Collaboration -----
The consulting engagement will commence in March and will be conducted full-remote.
Participation in daily update meetings is mandatory and non-negotiable. Failure to adhere to this requirement will result in immediate project disengagement, without exception.