VC Consolidation, AI Distillation, and Who’s Hiring in Venture

Good Morning! Welcome to the Thursday update. We’ve got consolidation, distillation, and jobs today.

Distilling Models: Smarter, Smaller, and Maybe Stolen?

DeepSeek followed the release of their R1 model with an image model, Janus-Pro-7B, but skepticism around DeepSeek's training claims escalated to full accusations of model distillation.

What is Model Distillation?

Model distillation is the process of taking a large AI model and creating a smaller, more efficient version that retains most of its intelligence. Instead of training from scratch on raw data, the smaller (student) model learns directly from the larger (teacher) model’s outputs — or uses a large model to create probabilistic distributions for a classification. So instead of training on books and random Reddit threads, the distilled model gets to train on pairs of prompts and completions or ready-made probabilities of “correct” completions, which is way better.

The big model has already done the heavy lifting—understanding complex tasks, reasoning, and making decisions — the distilled model just needs to mimic that knowledge efficiently, and can often do so with smaller parameters. After training, it can generate nearly the same outputs as the teacher model but requires significantly fewer computational resources.

OpenAI does this internally for practical reasons: smaller models run faster, use less power, and cost less to deploy, like them distilling GPT-4 to GPT-4-turbo. OpenAI even offers it as a service. But if another company distills an AI model trained by OpenAI—especially without permission—it can look a lot like stealing.

DeepSeek Caught Red-Handed?

It’s hard to say this definitively. OpenAI says they found suspicious activity and evidence that DeepSeek is using its models in this way. If DeepSeek DID train on OpenAI’s outputs, this would be a textbook example of "model extraction". It will be hard to prove, but there might be signs, like OpenAI model quirks appearing in DeepSeek’s responses, or R1 thinking it is made by OpenAI (which people are reportedly seeing).

Frontier and Safety Concerns

We often hear about AI companies moat (or lack thereof). One of the biggest moats in AI is not just the raw model but its thinking process. Advanced techniques like Chain of Thought (CoT) reasoning allow models to break down complex problems, improving accuracy and reducing hallucinations.

If another company manages to extract CoT reasoning, this could significantly narrow the performance gap between AI leaders and challengers. It also raises concerns about safety, as smaller, unregulated AI models might lack the alignment, security, and deployment measures agreed to by frontier labs.

Is This Illegal?

Right now, it’s a legal and ethical grey area, where the law hasn’t really caught up (people are comparing it to OpenAI’s own alleged training on copyrighted material). It’s certainly against OpenAI’s terms of service. OpenAI disallows this practice because it works — it’s an effective way to shortcut AI development. If regulators don’t step in, expect more companies to attempt the same strategy.

Woof, ok, that’s enough AI and DeepSeek talk for this week. But also, do you think DeepSeek used distillation for its model?

Toolbox

GP Ryan Hoover successfully transitioned from founding Product Hunt to running a thriving venture fund. Here’s an example of a successful pitch deck from Weekend Fund’s third fundraise. View the Deck:

Weekend Fund Deck

Thanks for reading. We hope everyone has a wonderful weekend.

A short message from a sponsor:

Get your news where Silicon Valley gets its news 📰

The best investors need the information that matters, fast.

That’s why a lot of them (including investors from a16z, Bessemer, Founders Fund, and Sequoia) trust this free newsletter.

It’s a five minute-read every morning, and it gives readers the information they need ASAP so they can spend less time scrolling and more time doing.

Subscribe for free