Can an LLM Replace a FinTech Manager? Comprehensive Guide to Develop a GPU-Free AI Tool for Corporate Analysis

[BLING models, LLM, RAG | Originally published on this Medium post via the Towards Data Science publication]

Introduction

It's been no more than a year now, where GPT stardust ✨ covered almost any sector globally. More and more experts, from any field, crave to utilise Large Language Models (LLM) in order to optimise their workflow. Evidently, the corporate world could not be absent from this new trend's safari. The future promises unprecedented possibilities, yet wrapped in the suited… cost.

Scope

This project aims to demonstrate an end-to-end solution for leveraging LLMs, in a way that mitigates the privacy and cost concerns. We will utilise LLMWare, an open-source framework for industrial-grade enterprise LLM apps development, the Retrieval Augmented Generation (RAG) method, and the BLING - a newly introduced collection of open-source small models, solely run on CPU.

Authors and Acknowledgement

[It stands as an independent analysis in an effort to enhance my ability to communicate results, reason about data statistically and stay motivated to continuously implement newly aquired skills & capabilities, so as to enrich my portfolio of data science-oriented projects]

• @makispl for concept & implementation.
• @MPlegas 𝕏
• @gerasimos_plegas Medium