LLM Studies (Part 2) — Understanding Fine-tuning

Finetuning LLMs is no doubt the hottest topic of early 2023. Thx to the release of LLaMA, within a couple of months so many high-impact fine-tuning projects emerged: Alpaca, Dolly, Vicuna… (apparently exhausting the list thx to Facebook’s unique taste for this species). It’s worth noting that the incumbent “fine-tuning” is fundamentally different from theContinue reading “LLM Studies (Part 2) — Understanding Fine-tuning”

LLM Studies (Part 1) — Pre-training Feasibility

This a series of posts discussing technical details of LLM. As early practitioner of LLM, I would like to share a series of technical reports to the community as my work progresses.For LLM pre-training, I want to focus on “feasibility” in this blog post. Summarizing the training recipe from published papers is easy, in factContinue reading “LLM Studies (Part 1) — Pre-training Feasibility”

Model Productionization – Large Scale Machine Learning Platform Design

What is “model productionization” and how is different from traditional software’s “productionization” process? Revisit Traditional Software Development Let’s look at 2 popular software platform architectures: Micro-Service Oriented Architecture The principle is to split “monolithic” design into singular function “micro-services”. Each micro-service has simple, well defined input/output specs, and with good network infrastructures such design canContinue reading “Model Productionization – Large Scale Machine Learning Platform Design”

A Daily Life of AI Agent

Feb 16, 2021 I’d like to start our discussion from examine a human’s daily life. We all know that human brain has evolved tens thousands of years ever since the debut of homo-sapien – to a large degree marks the debut of human intelligence. But besides just human animals are also considered as intelligent, thisContinue reading “A Daily Life of AI Agent”

How An AI Scientist Train His Puppy

Aug 02, 2020 Training a puppy is a fun experience. Our little pokie is turning 1 yr old, and turning into a cutie-pie. This thoughts emerged while training my own dog. There’re many examples that reflected the fundamental difference between how human learn vs how animal. One of the key observations is the lack ofContinue reading “How An AI Scientist Train His Puppy”