After setting up my new graphics card, I’ve begun some initial local AI tinkering. So far, the results have been interesting with some successes and challenges along the way.

I started by testing a large image model to generate some images. The results look significantly better than what I was able to achieve before, with about the same processing speed. This is a nice improvement in quality without sacrificing performance.

The large voice model produces very pleasant output and sounds natural. However, I encountered some issues with cloning voices. There were also a lot of compatibility problems with my older Ubuntu 20.04 WSL setup, which I’ll need to address.

Despite the compatibility issues, I was able to get a complete STT/LLM/TTS pipeline working. The system can transcribe videos and process the audio through speech-to-text, language model processing, and then text-to-speech output.

Even with some optimization and tweaking, there’s still a 3-5 second delay in the processing. While not exactly real-time, it’s a functional solution that demonstrates the potential of local AI processing.

I’m planning to continue experimenting with different models and configurations to improve performance and compatibility. The current setup shows promise for practical local AI applications, even with the limitations.