https://www.henrypan.com/blog Project Repository: https://github.com/workofart/harness-training So I recently wanted to see whether an AI agent could self-improve a harness to solve terminal bench tasks. To align on the definitions, “harness” means the system (e.g. Claude Code, Codex, ChatGPT web interface etc…) wrapping around the model (e.g. GPT-5.5, Claude Opus 4.7 etc…) that... Sat, 18 Jul 2026 23:55:00 -0400 https://www.henrypan.com/blog/2026-07-18-harness-training/ https://www.henrypan.com/blog/2026-07-18-harness-training/ Project Repository: https://github.com/workofart/harness-experiment Update (July 2026): The follow-up, Training Self-Improving AI Agent Harnesses, turns these lessons into a deterministic training framework that allows for model-agnostic and task-environment-agnostic capability improvements. So I recently wanted to see whether an AI agent could self-improve a harness to solve terminal bench tasks. To align... Mon, 25 May 2026 22:55:00 -0400 https://www.henrypan.com/blog/2026-05-25-self-improvement-harness/ https://www.henrypan.com/blog/2026-05-25-self-improvement-harness/ 1. The Familiar Training Step 2. Why Call loss.backward() At All 3. Why An Update Actually Helps 4. Values Remember Where They Came From 5. What One Operation Does During Backward 6. How loss.backward() Walks the Whole Graph 7. So We Have All the Gradients in the Graph. What’s Next?... Sat, 14 Mar 2026 18:32:00 -0400 https://www.henrypan.com/blog/2026-03-14-how-deep-learning-library-enables-learning/ https://www.henrypan.com/blog/2026-03-14-how-deep-learning-library-enables-learning/ Motivation and Goals Abstraction layers of Machine Learning Libraries Comparing this project to PyTorch for the same functionality Efficiency vs Learning Technical Design Tensor Class Tensor-level Operations Function class Computational Graph Tensor.backward() NN Module Functional Module Optimizer Additional Thoughts Project link: https://github.com/workofart/ml-by-hand I recently started working on a project called... Thu, 06 Feb 2025 03:23:00 -0500 https://www.henrypan.com/blog/2025-02-06-ml-by-hand/ https://www.henrypan.com/blog/2025-02-06-ml-by-hand/ Video games were very attractive to me when I was kid. This is because the possibilities in the virtual world is vastly greater than the real world, as least during my teenage years where most of my life was filled with school work that can’t be related to in the... Mon, 15 Nov 2021 00:23:00 -0500 https://www.henrypan.com/blog/2021-11-15-peaking-into-the-real-game/ https://www.henrypan.com/blog/2021-11-15-peaking-into-the-real-game/ Code 1. Goal 2. Environment State Action Reward Termination Condition 3. Approach 3.1 Reward Function Design 3.2 Learning Algorithm 3.3 Hyperparameters 4. Experiment & Findings 4.1 Training Evaluation 4.2 Optimal Policy Discussion Experiment - Play two copies of the same agent against each other, by following the trained policy Experiment... Fri, 06 Dec 2019 16:00:00 -0500 https://www.henrypan.com/blog/2019-12-06-tic-tac-toe-selfplay/ https://www.henrypan.com/blog/2019-12-06-tic-tac-toe-selfplay/ Code 1. Goal 2. Environment State: Action: Reward: Terminal Condition: Solved Condition: 3. Approach 3.1 Algorithm Comparison 3.2 Problem Parameterization 3.3 Policy Key consideration: 3.4 Neural Network Architecture 3.5 Hyperparameters 4. Experiment & Findings 4.1 Performance 4.2 Training Duration 4.3 Neural Network Complexity 4.4 Discount Rate 5. Next Steps Code... Tue, 03 Dec 2019 16:00:00 -0500 https://www.henrypan.com/blog/2019-12-03-acrobot/ https://www.henrypan.com/blog/2019-12-03-acrobot/ Code 1. Goal 2. Environment State Actions Reward Starting State Episode Termination Solved Condition 3. Approach 3.1 Discretization 3.2 Exploration vs Exploitation 3.3 Gaussian Policy 3.4 Linear Value Function 3.5 Training 3.6 Hyperparameters 4. Experiment & Findings 5. Next Steps Code Here 1. Goal The problem setting is to solve... Tue, 05 Nov 2019 16:00:00 -0500 https://www.henrypan.com/blog/2019-11-05-pendulum/ https://www.henrypan.com/blog/2019-11-05-pendulum/ Code 1. Goal 2. Environment State Actions Reward Starting State Episode Termination Solved Requirements 3. Approach 3.1 Discretization 3.2 Exploration vs Exploitation 3.3 Training 3.4 Hyperparameters 4. Experiment & Findings 4.1 Introducing baseline to reduce variance 4.2 Discrete vs Continuous Actions 4.3 Performance 5. Next Steps Code Here 1. Goal... Mon, 04 Nov 2019 18:00:00 -0500 https://www.henrypan.com/blog/2019-11-04-mountain-car/ https://www.henrypan.com/blog/2019-11-04-mountain-car/ 1. Perception 1.1 Current Player Position Green Circle Player Name 1.2 Stars (Reward) Player Stars Team Stars 2. Planning 2.1 Agent 2.2 Brain 3. Error Analysis 4. Challenges & Future Steps First and foremost, I must say, perception is harder than planning. This part, I will be attempting to apply... Thu, 25 Apr 2019 01:00:00 -0400 https://www.henrypan.com/blog/2019-04-25-Brawlstars-RL/ https://www.henrypan.com/blog/2019-04-25-Brawlstars-RL/ 1. Motivation 2. Goals 3. Starting Point 3. Related Work 4. Project Focus/Scope 5. Showcase 6. Supervised Learning 6.1 Creating training data 6.2 Features 6.2.1 Raw Pixels as Features 6.2.2 Using MobileNet as the feature extractor 6.3 Action Determination 6.3.1 AlexNet (Convolutional Neural Network) 6.3.2 Long short-term memory 7. Challenges... Sat, 20 Apr 2019 01:00:00 -0400 https://www.henrypan.com/blog/2019-04-20-Brawlstars-AI/ https://www.henrypan.com/blog/2019-04-20-Brawlstars-AI/ Policy Gradient Problem Setting Agent Environment Reward Technical Details Policy Network Design Training Key Considerations Key Challenges Challenge 1 Challenge 2 Results Next Steps This is the first post that’s part of the series for teaching an agent to trade. I will evaluate different reinforcement learning (RL) approaches and share... Thu, 04 Apr 2019 01:55:00 -0400 https://www.henrypan.com/blog/2019-04-04-pg-trading/ https://www.henrypan.com/blog/2019-04-04-pg-trading/ Previous Knowledge Required Goals Showcase Training Set Test Set Input Data Neural Network Architecture (All 3 Versions) Version 1 Version 2 Version 3 Previous Knowledge Required Understand what is a neural network (NN) and how it works conceptually. Python Basic understanding of what derivatives/gradients are Goals In this tutorial, I... Wed, 20 Mar 2019 11:50:00 -0400 https://www.henrypan.com/blog/2019-03-20-ml-tut-price-prediction/ https://www.henrypan.com/blog/2019-03-20-ml-tut-price-prediction/ The A3C method in Reinforcement Learning (RL) combines both a critic’s value function (how good a state is) and an actor’s policy (a set of action probability for a given state). I promise this explanation doesn’t not contain greek letters or calculus. It only contains English alphabets and subtraction in... Wed, 27 Feb 2019 12:00:00 -0500 https://www.henrypan.com/blog/2019-02-27-a3c-rl-layments-explanation/ https://www.henrypan.com/blog/2019-02-27-a3c-rl-layments-explanation/ I’ve been bugged by the native state management system in React that I finally had to take a stab at Redux. Here are some notes I took along the way to understand what Redux is and why we need it. First off, why do we need Redux when we already... Sat, 26 Jan 2019 12:00:00 -0500 https://www.henrypan.com/blog/2019-01-26-react-redux-intro/ https://www.henrypan.com/blog/2019-01-26-react-redux-intro/ This post is adapted from my answer on Reddit. I think there are various career paths that fall under “Data Science/Machine Learning”. This doesn’t mean there are 4 roles, it’s just how I see this industry; nevertheless, the more skills you possess, the better. Below are my personal thoughts based... Thu, 10 Jan 2019 12:00:00 -0500 https://www.henrypan.com/blog/2019-01-10-career-paths-in-ml/ https://www.henrypan.com/blog/2019-01-10-career-paths-in-ml/ It’s been a while since my last post. There’s just too much going on in my life that I didn’t get time to write down. Anyways, here goes nothing. The past 1.5 year has been really, really rough without any exaggeration. After switching majors, I’ve learnt a lot of new... Thu, 03 Dec 2015 08:46:00 -0500 https://www.henrypan.com/blog/2015-12-03-looking-back-planning-forward/ https://www.henrypan.com/blog/2015-12-03-looking-back-planning-forward/ When choosing a career path, the most important thing is to do the research and plan ahead. When we are talking about doing research for career cruising, we need to understand its industry. Its past, present, future. You don’t just wanna look at the present condition, because who knows what... Tue, 24 Dec 2013 09:52:00 -0500 https://www.henrypan.com/blog/2013-12-24-a-few-thoughts-on-choosing-a-career-path/ https://www.henrypan.com/blog/2013-12-24-a-few-thoughts-on-choosing-a-career-path/ How I ever wonder, how does so many people have so many different answers to a simple “1+1” problem. To put the question into perspective, we have all heard of the “Standard Deviation” term from basic statistics, if not, this measures the variability of the sample or population data points.... Sat, 16 Nov 2013 08:11:00 -0500 https://www.henrypan.com/blog/2013-11-16-different-life-experiences-bring-different-perspectives/ https://www.henrypan.com/blog/2013-11-16-different-life-experiences-bring-different-perspectives/ It has been approximately 24 days since classes have started and I have gotten used to the work and life style in business school, the case method and socializing all the way. Most of the courses here are taught in the form of case studies, where the learning, supposedly comes... Wed, 25 Sep 2013 04:11:00 -0400 https://www.henrypan.com/blog/2013-09-25-finding-the-right-route/ https://www.henrypan.com/blog/2013-09-25-finding-the-right-route/