HelferChat

To the project

TL;DR

A retrieval‑augmented chat assistant focused on Austrian law, running end‑to‑end encrypted on a stack of TypeScript (Next.js 15), Langchain, PostgreSQL + pgvector, and entirely open‑source embedding models.

Background

The AI space is a wild west regarding privacy and data safety. Most of the popular chatbots are hosted on servers in the US, which means that your data is stored on servers that are not subject to European data protection laws. This is a big problem for Austrian companies and individuals who want to keep their data safe and private. The goal of this project is to create a chat assistant that is hosted in Austria or at least in the EU and is subject to Austrian data protection laws. Focusing on Austrian companies also helps us tweak the model to be more relevant for the Austrian market.

Tech Stack

  • Next.js: The frontend is built with Next.js, later versions might differentiate between marketing and app pages. Next.js is great for data heavy marketing sites but in my opinion not the best choice for the chat app. That is also why OpenAI switched to Remix for their chat app.
  • Langchain: The chat assistant is built with Langchain. Initially it was build as RunnableChains but in later versions I switched to Langgraph as the graph structure is more suitable for routing and handling different modalities of prompts.
  • PostgreSQL: The database is built with PostgreSQL and pgvector for vector storage. When starting a project I always try to not fall for the latest hype as most of it is flaky and not production ready. Also having an open source database is crucial for self hosting.
  • Models: For the MVP I used OpenAI's o4-mini model but the plan is to switch to open source models when the app itself is stable.
  • Supabase: The backend is built with Supabase which makes it easy to self host and scale. It is also a great choice for the chat app as it is easy to use and has a lot of features that are useful for chat apps.
  • TypeScript: The entire stack is built with TypeScript which makes it easy to refactor and document as you go. I am a big fan of TypeScript and some even call me a TypeScript Ninja. I am not sure if that is a compliment or an insult but I take it as a compliment.

Architecture

The app is a fully SaaS app that focuses on teams but also enables individuals to use the app. The application is built with a modular architecture which makes it easy to add new features and change existing ones. I like to work feature based which means that each feature is encapsulated in its own module kind of like a microservices but without the additional overhead. Working in modules also makes it easy to test and debug the application as each module can be tested and debugged independently. And when things grow too big I can always split them into microservices.

I also like to work with feature flags to enable and disable features as I go. I wrote a blog post about it a while ago if you are interested. The idea is to build the application in a way that it is easy to add new features and change existing ones. This is especially important for chat applications as they are constantly evolving and changing and if the team is small. The modular architecture makes it easy to add new features and change existing ones without breaking the entire application.

RAG

The chat assistant is built with Langchain and uses a retrieval-augmented generation (RAG) approach. Langchain is for me at least somehow confusing as every time I look at it, it has changed. There is a good book on it but it is kind of outdated already and the javascript documentation is unfortunately not as good as the python one. Here is the book if you are interested: Learning Langchain.

There are also a lot of research papers written about RAG and how to tweak it for different use cases but as we say in Austria: "Probieren geht über Studieren" ("Trying is better than studying"). I will make sure to keep this page updated with the latest techniques I found useful!

To be continued...