Recommended Words
Starting a business is really not easy. Just looking at Li Mu’s experiences over the past year makes one sweat buckets.
Original Text
This article is converted by SimpRead 简悦, original source www.bilibili.com
When I was in Amazon for five years, I was already thinking about entrepreneurship, but it was delayed by the pandemic. By the seventh and a half year, I felt too restless and resigned. Looking back now, if there’s something you must try in your life, start early. Because once you really start, you’ll find there are too many new things to learn and always regret not starting sooner.
Name: The Origin of BosonAI
Before starting my business, I worked on a series of projects named Gluon. In quantum physics, Gluon is a boson that binds quarks together, symbolizing that this project initially was a joint effort between Amazon and Microsoft. The project manager came up with the name on a whim, but naming is hard for programmers; we constantly debate about file names and variable names daily. Eventually, the new company simply named itself after bosons (Boson). I hope everyone can get the joke that “Bosons and Fermions make up the world” and have a knowing smile. But I didn’t expect many people to misread it as Boston.
“I’m in Boston now, let’s meet up sometime?” “Huh? I’m in the Bay Area 
”
Funding: Lead investor ran away a day before signing
At the end of 2022, I had two ideas about using large language models (LLM) for productivity tools. I happened to meet Zhang Yiming and consulted him. After discussion, he asked: Why not make the LLM itself? My immediate reaction was hesitation: our team at Amazon worked on this for years, requiring tens of thousands of GPUs, and there were all sorts of difficulties. Yiming chuckled and said: these are just short-term problems; you need a long-term vision.
My strength is that I take advice, so I really started to build an LLM. We assembled a founding team responsible for data, pre-training, fine-tuning, and architecture, and then went fundraising. Luckily, we quickly secured seed investment. But the money wasn’t enough to buy GPUs, so we had to raise a second round. The lead of this round was a very large institution. After months of documentation and negotiation, the day before signing, the lead said no to investment, causing several co-investors to withdraw. I’m very grateful to the remaining investors who still completed the round, securing our entry ticket to build LLMs.
Reflecting now, back then, while the capital market enthusiasm still existed, we could have continued fundraising. Maybe we would have had a billion cash on hand like other competitors. At the time, I worried that too much fundraising would make exit difficult or inflate the valuation too high. Now I think entrepreneurship is about overturning destiny—what kind of fallback plan is that?
Machines: The first to try something new
With money in hand, we went to buy GPUs. Every vendor said H100 delivery would take more than a year. On a whim, I emailed Jensen Huang (老黄). He replied immediately saying he would look into it. An hour later, the CEO of Supermicro called. We paid extra and got priority. Twenty days later we got the machines. We were proud to be the early crab eaters (early adopters).
Eating crabs (early trials) brought existential doubts with all kinds of unimaginable bugs. For example, unstable power supply to GPUs fixed by patches engineers made to the BIOS; improper fiber optic cutting angles causing communication instability; Nvidia’s recommended network topology was suboptimal, so we designed a new plan that Nvidia later adopted. I don’t understand why, as a relatively small buyer with less than a thousand cards, we encountered these issues. Did big buyers not have them? Why do they need our debugging?
We also rented as many H100s, encountering similar bugs. GPUs failed daily, sometimes we wondered if we were the only early adopters in the cloud. Later, seeing Llama 3’s technical report saying their H100 training had hundreds of interruptions, I deeply empathized with the pain behind their words.
Comparing building our own cluster and renting GPUs, the costs over three years are about the same. Renting is convenient and worry-free. Building has two advantages: first, if Nvidia’s technology continues to lead in three years, it can control prices so that GPUs retain value 
. Second, self-built data storage costs less. Storage must be close to GPUs; whether big or small cloud providers, storage prices are high. But a single model training may use several terabytes for checkpoints, with training data storage starting at 10 petabytes. Using AWS S3, 10PB costs around $2 million a year. Building our own on that money could store 100PB.
Business: Grateful to clients, break-even in the first year
We were very fortunate to break even in income and expenses in the first year. Our expenses were mainly manpower and computing power. Thanks to OpenAI’s financial power and Nvidia’s leading technology, these costs were significant . Our income came from customized models for large clients. Those companies early on with LLMs usually had very decisive CEOs who were not intimidated by high costs and decisively pushed internal teams to try new technologies. We’re very grateful clients gave us breathing room; otherwise, I’d still be running around investors for months.
More companies will try LLMs next, whether to upgrade their own products or reduce costs and increase efficiency. The reason is that technical costs are decreasing, and industry leaders (like our clients) will gradually release LLM-based products that drive industry competition.
We’re also watching LLM implementation in to-C. Last wave top projects like c.ai and Perplexity are still exploring business models, but about a dozen native LLM application companies already have decent income. We provided models to a role-playing startup focusing on deep players; their income and expenditure broke even too, impressive. Models continue evolving with more modalities (voice, music, images, video) being integrated, promising more imaginative applications coming soon.
Overall, the industry and capital remain impatient. Several companies over a billion RMB in funding but only a year old chose to exit. Technology to product is a long journey, and spending 2-3 years is normal. Considering emerging user demands, it may take even longer. We focus on navigating the fog and remain optimistic about the future.
Technology: Four stages of LLM cognition
Our understanding of LLMs went through four stages. First stage: from Bert to GPT-3; the feeling was new architecture and big data, doable. At Amazon, we were among the first doing large-scale training and product deployment.
Second stage: at the start of entrepreneurship when GPT-4 came out, we were shocked. Much due to its technical secrecy. Rumor said training a model once cost 100 million RMB, labeling several tens of millions. Many investors asked me how much it cost to replicate GPT-4; I said 300-400 million for sure. Later, one investor truly invested hundreds of millions in us.
Third stage: first six months after starting up. We couldn’t build GPT-4, so we decided to start from specific problems. We looked for clients in gaming, education, sales, finance, insurance. We trained models based on specific demands. At first, there were no good open-source models, so we trained from scratch. Later many great models emerged, reducing our costs. Then we designed evaluation methods and labeling for business scenarios, checked model failures, and improved selectively.
At the end of 2023, we were thrilled to find our Photon (a kind of Boson) series models outperforming GPT-4 in client applications. Customized models have one-tenth inference cost compared to API calls. Though APIs became cheaper, our technology also improved, maintaining the 1/10 cost. Also, QPS and latency are better controlled. The understanding at this stage: for specific applications, we can outperform the best model on the market.
Fourth stage: second six months of entrepreneurship. Although clients got models per contracts, it wasn’t ideal; GPT-4 was still insufficient. Early this year, we found that training models for single applications made it difficult to leap further. Looking back, if AGI means reaching average human level, clients want professional-level expertise. Games need professional planners and actors, education needs gold medal teachers, sales gold medal salespeople, finance and insurance advanced analysts. This is AGI plus domain expertise. Though we respected AGI, it felt unavoidable.
Early this year, we designed the Higgs (God particle, a kind of Boson) series models. They focus on general ability while excelling in some capabilities. We chose role-playing: playing virtual characters, teachers, salespeople, analysts, etc. By mid-2024, we iterated to V2, which on the Arena-Hard and AlpacaEval 2.0 tests for general capability matched top models. On domain knowledge MMLU-Pro test, it was close too.
Higgs-V2 is based on Llama 3 base with full post-training. We can’t spend big on data annotation like Meta, so V2 surpasses Llama3 Instruct mainly due to algorithmic innovation.
We also created a role-playing evaluation dataset including persona-based and scenario-based acting. Embarrassingly, our model ranked first on our own leaderboard. But model training did not use evaluation data. Since this dataset was meant for internal use to truly reflect model ability, we avoided overfitting. Our evaluation team wanted to publish a technical report, so it was released. Interestingly, role-playing test samples came from c.ai, but their model ranked at the bottom.
The understanding at the fourth stage is that good vertical models also need strong general abilities like reasoning and instruction following. In the long run, both general and vertical models will move towards AGI. Vertical models can specialize more, scoring high in their domain but average generally, so R&D cost is slightly lower and approach somewhat different.
What about the fifth stage? It’s still ongoing and hopefully we will share soon.
Vision: Human companionship
Embarrassingly, we focus on technology, customizing for clients, and then slowly think about our own vision. We look at what clients want, what we want, and what may be needed in the future. For me personally, years ago I dreamed of a robot nanny to help me take care of and accompany my kids because this is difficult and I don’t fully understand kids’ cognition and thoughts. I hope to have a powerful virtual assistant at work to invent new things together. When I’m old, I also want interesting robots to accompany me.
My prediction is that production tools will become more advanced, enabling an individual to accomplish what once took a team, making people more independent but also lonelier as everyone pursues their own things.
Combining all this, we set our vision as “intelligent agents for human companionship.” An agent with high emotional intelligence and sharp intellect. In human terms, it would be a professional team: if you want it to play with you, it would be planner + actor; to exercise, encourager + coach; to study, an instructor who can explain what you don’t understand. The advantage of a model is it can accompany you long-term, truly understanding you and being “sincerely for you.”
But currently the technology is still far from this vision. Today’s tech can only chat a bit. Many chats are not great, content is lacking, IQ and EQ sometimes offline. These are problems to solve now. If anyone is developing overseas applications in this area, please contact us.
Team: Challenging tasks require a team
Only after starting a business did I truly realize the importance of a team. In big companies, I thought I was a screw, team members were screws, even the team itself was just a screw. But a startup team is a vehicle. Small car, but it can run, carry loads, turn nimbly, go everywhere. When the company was founded, MiHoYo’s Cai came by, saw everyone in one room, and sighed, “Small teams are really nice.”
Of course, there are drawbacks, always monitoring fuel and cautious on rough roads not to shake the vehicle apart. Every member is important; no redundancy. If one is weak, it’s like a flat tire. People are precious; losing one is losing a tire.
Previously, I would choose projects I could lead development on. But these problems were not very challenging. Entrepreneurship chose a big problem to solve, relying entirely on the team. Although this article uses many “I"s, the work is all a team effort. Without them, I probably would have switched careers to teaching.
Personal pursuit: fame or fortune?
So far, I have always followed my inner voice to make decisions: after working, I went for a PhD, created videos, then started a business. Entrepreneurship requires strong motivation to overcome endless difficulties. This requires deeper analysis of one’s motivation.
Motivation comes from either desire or fear. Ten years ago, I might have been more passionate about fame and fortune, but now at this age, the marginal utility of money is low, and emotional value from fame is small. My deep motivation comes from fear that life may be meaningless. Not to mention the vast universe, but throughout human history, one person is like a grain of sand. Suddenly appearing, quickly disappearing. One hundred billion people have lived on Earth; most won’t leave a trace in history. I don’t even know most of the thousands of names in my family genealogy.
So what is the meaning of one person’s existence? I was once depressed as a child because I couldn’t figure this out. So subconsciously, I wanted to create value to gain meaning. I choose to “strive forward” to improve my creative value; I choose to record long videos and write textbooks to create educational value; I choose to write summaries of PhD, work, and entrepreneurship recounting struggles and difficulties to create case value; I choose entrepreneurship to unite many people’s power and create greater value.
Afterword
Last year, walking with Su Hua at Stanford, he patted my shoulder and asked, “Tell me honestly, why do you want to start a business?” At that time I responded casually: “Just wanted to try something different.” Su Hua smiled.
Now I understand because he experienced the ups and downs of entrepreneurship. If I were to answer today, I’d say: “I was just crazy.” But I’m glad I didn’t realize how hard it would be, so I dove in headfirst. Otherwise, you might have seen a “ten-year reflection at work.” I think today’s story is more interesting.
Salute to all entrepreneurs.
(Finally, a little advertisement for our company’s recruitment (Bay Area and Vancouver) https://jobs.lever.co/bosonai. Also, if anyone is working on overseas applications, please contact us at api@boson.ai)