Decided to setup another AI model to my local. The last time I had different AI models was back when my workstation is on Windows 10 Pro. These days since I got into Linux, the only running LLM which I would say is pretty impressive for its small size and capabilities is Microsoft’s Phi4-Mini but I’m planning to setup a much bigger Qwen AI model from Alibaba. As long as it can run based on my hardware specifications and as long as it is open source which means it’s source code is open for public to scrutinize then I wouldn’t worry about data being sent to China.
The reason for deciding on another AI model in my local is to protect my intellectual properties. I realized that I could not just rely on the frontier models when it comes to security because in the end of a breach, it is my intellectual property that will be at risk.
The cybersecurity product will not be built using a frontier model due to the risk of breach of confidentiality. For things like secrets, the best way is to run a local LLM that is open source. This is also one of the reasons why companies are investing on their own AI infrastructure to run AI models with higher parameters. The main goal is to protect their intellectual property and second is to skip paying for tokens.
This site will hold short thoughts, daily observations, technical discoveries, project progress, and records that do not require a full article.