Exactly 1 year ago I created PrivacyGPT. It caused me issues. LOTS of issues. So I re-wrote it in Flask over a weekend. I’ve never used Flask prior.

Click me if you wanna check it out right away!

Background:

So last year I made Privacy GPT, a GPT wrapper using a vector database consisting of my custom knowledge base, a.k.a raw text of the KSA law, which can be used to ask questions particular to Data Privacy laws in the Middle East (I’m a Data Privacy professional, so this was a legit use case as Data Privacy is a relatively new concept in the Middle East and we’re still seeing updates and legislations coming in).

It was written in Python. When I initially worked on it, my intention was to play around with OpenAI’s LLMs and build an MVP showcasing a custom knowledge bot. I quickly found out having the text of law for multiple jurisdictions in a prompt template was probably the worst way to go about this.

To combat this, I used Qdrant, a persitent vector database with that splits documents into chunks and combined with langchain’s retieval QA function, the OpenAI API is called only on the ‘retrieved’ (as in relevant) bits. There is NO prompt template used in PrivacyGPT, the query is passed to OpenAI API on the Retrieval based Question Answering (QA) chain created during input which searches the indexed vector store with created with the text of the laws, and then the LLM (OpenAI’s GPT here) helps generate a coherent response on the narrowed down data. This approach is significantly faster AND cheaper, as the info passed onto the API doesn’t contain all the text from the law.

And so I had my neat little Python app that I could use to show everyone my I too worked with AI. Great. Except for one problem…..how do I host and show this to anyone in the world? After some intense googling, I found streamlit.io.

Streamlit’s Community Cloud platform lets you deploy your python code from Github right away for the masses to see and gives you a URL on their domain. I thought this seemed solid and would go on to modify my code to include the streamlit library to include some markdown and text input for the app. Boy, was I wrong thinking it would be that simple.

The app worked. Well not exactly, the UI loaded up when I self hosted on EC2, but crashed constantly on any input. So I found Streamlit Community Cloud, and the app deployed successfully with it working as intended.

WELL, you could see it in action IF it was running and not asleep. Unless you paid for streamlit, you couldn’t use custom domains and I had to iframe it on my domain, and my app specifically had an issue of having to be manually rebooted from my streamlit dashboard everytime it went to sleep. Yes, sleep, another con of hosting it for free. The biggest one as my app would go to sleep every few hours and it was never up when I needed people to see it, or you were a random visitor. I thought I was clever trying to use a CRON job to keep my app awake, but yeah, that didn’t work.

It was truly annoying but I never thought about alternatives or fixing it and the months went by, life happens. Until few weeks ago when I realised the app would not even reboot as some dependencies (cough cough…langchain) were deprecated, OpenAI API replaced User API keys with Project API keys, and it was almost an year since the app was up. And then I thought maybe it was time.

The Solution:

Flask. Light, easy, Dependency agnostic, no strings attached…like a weekend fling.

Going with Flask was a no brainer, I wanted to keep things lightweight and host the whole application on my VPS alongside other applications, I would use my current nginx setup and have the hosted web-app point to a custom subdomain.

Now, there were two parts to this that made sense at the time. First, fix the app and get it running. Next, rewrite the whole thing in Flask, because what could possibly go wrong.

To keep it short, fixing the app required updating the deprecated libraries that were causing errors. Here’s a brief:

• Qdrant function was moved from langchain.vectorstores to langchain_community.vectorstores.
• OpenAIEmbeddings was moved from langchain.embeddings.openai to langchain_openai.

And soon enough, the app was running. First step, DONE.

Now, rewriting the code for Flask. Full disclosure, I used help from Claude 3.5 (Sonnet) to create the render_template index.html code required used for displaying the web page made using tailwind.css and basic JS to handle the input and text response logic. For app. py containing the main application code, 2 major changes were done to accommodate Flask.

1) Set app routes to display the index.html on load & another route for the primary function handling the request and response from index.html to the app backend

2) Using the Jsonify function, convert function response to JSON before returning it, where on the webpage a script shall unhide the answer div after overwriting it with the generated response.

The frontend component is quite straightforward, a form captures user input, sends the request via POST to the Flask backend, and a little JS was used to dynamically update page with the JSON response and unhide the div it was contained in. All resources are served from the Flask static folder in my VPS.

The app was deployed on my VPS using Gunicorn, very quickly my favourite Python web server gateway.

And that’s how my frustration with Streamlit made me rebuild PrivacyGPT in Flask, hosted on my own VPS, no downtime, no sleep, no iframes, custom domain, proxied through a CDN so much better security and load times. It’s great.

Check it out at privacybotweb.shanen.works !