The Fix That Let Us Run the Biggest Open Models Overnight
We hit a wall trying to run the latest open-source LLMs like Qwen3, Llama 4 and Deepseek R1, even on our multi-GPU AWS instances.
At Basis Set, our engineering team is always chasing faster ways to experiment with the newest models. As models become larger, hardware limitations are one of the biggest hurdles, specifically GPU memory. For large models (72B parameters+), the memory demands are simply too immense.
As a small team, deploying these models is often infeasible due to resource intensity. Frustrated by memory constraints limiting our model performance, we were eager to find a smarter, more scalable solution.
The Use Case: Reddit Sentiment Analyzer
We built a tool internally to track sentiment and proactively surface reactions across community conversations within Reddit threads. Initially, we used OpenAI’s GPT-4.1, but it came with two problems:
- We couldn’t easily swap in newer models for comparison
- The cost was adding up fast
We started looking for alternatives.
Enter Parasail + Open Source Models
We turned to Parasail for model hosting. It gave us a plug-and-play way to run the latest open-source LLMs like the new Qwen3 model they released last week and the Llama4 model released last month with the click of a button. We were eager to integrate these models into our AI systems and Parasail was the first provider to launch both of these new models.
Our first test was to see if we could replace OpenAI’s GPT 4.1 with Qwen3, because of recent developer chatter, to see what impact it would have on output quality on our Reddit sentiment analyzer. We built this tool to track public sentiment and community reactions across specific Reddit threads.
Using Parasail we were able to implement Qwen3 to test against GPT 4.1 and other popular models. We found that Qwen3 achieved comparable results to GPT 4.1 at a fraction of the cost.
Pricing Comparison
Input
$2.00 / 1M tokens
$0.10 / 1M tokens
Output
$8.00 / 1M tokens
$0.50 / 1M tokens
Why This Matters for Builders
As the market for large AI models matures, we expect continued dramatic swings in both performance and costs of new models. Switching models and running experiments is a bottleneck. What worked for us was decoupling infra from model experimentation. Tools like Parasail make that easy. Now we can swap models in near real-time while evaluating quality and cost
What Are You Using?
If you’re building with LLMs and hitting the same GPU or infra walls, we’d love to hear from you. We’re constantly testing new tools and workflows and always looking to exchange tips with fellow builders. Shoot us a note at bsvtech@basisset.ventures
Check out our tool outputs:
GPT 4.1
Cross-Subreddit Trends This Week
Concerns Over AI's Impact on Employment
AI Hallucination and Reliability
Advancements in AI and Open Source Models
Deepfakes and Trust Issues
Monetization and Accessibility of Open Source Projects
AI in Personal and Creative Applications
r/singularity
Woopsie daisie
Key Takeaways
Positive Insights
- Several users correctly note that LLMs reflect patterns in training data and tend to respond affirmatively to leading prompts, highlighting the importance of prompt context.
- Research is cited (arxiv.org/pdf/2501.11120) showing that LLMs may sometimes demonstrate awareness of behavioral patterns introduced by fine-tuning, sparking nuanced discussion about model self-awareness.
- Some participants draw meaningful analogies (e.g., LLM responses as reflections of the echo chamber nature of social media, or likening AI limitations to human metacognition) to deepen understanding of AI outputs.
- A few users emphasize the need for critical engagement with both AI outputs and the online discourse (e.g., Reddit as an echo chamber itself), raising awareness about the broader context of AI-generated content.
- There is recognition that biases can be mitigated or manipulated through system prompts, RLHF (reinforcement learning from human feedback), and fine-tuning, hinting at the complexities of model alignment.
Concerns & Criticisms
- Many express concern that users completely misunderstand LLMs and over-interpret their outputs as revelations of internal model intent or training regime.
- Cynicism about political bias in AI is prevalent; some argue that both training data and model guardrails inevitably introduce leanings, making neutrality elusive.
- There are significant worries about the reliability of LLM outputs on controversial or politicized topics, especially when prompts are loaded or leading.
- Critics point out that potential viral marketing or manipulative use of Grok and other models could sway users or public opinion, especially around politically sensitive times (e.g., elections).
- A recurring criticism is that the culture of Reddit (and by extension, the data used for training) does not represent a neutral or comprehensive view, potentially skewing any LLM trained on such data.
Qwen3
Cross-Subreddit Trends This Week
AI's Impact on Jobs
AI Agents and LLMs
Community and Open Source Contribution
Deepfakes and Misinformation
Open Source AI Projects
Monopolies and Tech Control
r/singularity
Woopsie daisie
Key Takeaways
Positive Insights
- Acknowledgment that LLMs reflect patterns in training data rather than personal knowledge, highlighting their statistical nature.
- Discussion of how LLMs can be influenced by system prompts or user input, showing adaptability in responses.
- Recognition of the challenges in verifying AI claims, with users emphasizing the importance of critical thinking and cross-checking information.
- Insights into the 'Gell-Mann effect' and research suggesting LLMs may form implicit biases during training, even if they don't explicitly know their training details.
Concerns & Criticisms
- Widespread skepticism about LLMs' ability to accurately describe their training or intentions, with many calling their responses hallucinations.
- Concerns about political bias in AI outputs, with debates over whether biases stem from training data, system prompts, or user interpretation.
- Criticism of the 'liberal bias' narrative, arguing it conflates ideology with reality and overlooks the complexity of training data sources.
- Fears about AI being manipulated by external actors (e.g., Elon Musk) to serve specific agendas, undermining trust in their neutrality.
Try a demo of our tool! Input a specific reddit post url with a few comments and check out the AI summarization newsletter.
