Urgent.News

What's breaking now, across hundreds of outlets.

Tech

Moving from ML/quant work into robotics: a concrete first week, measured

Part of a public week-one log of moving from ML/quant work into robotics: what was actually run, what it cost, and the honest next steps. The scripting was done with an AI coding agent; every number below was measured on one CPU-only Linux box (Python 3.12, MuJoCo 4.0.1, PyTorch 2.15 CPU wheels). Why MuJoCo first, not VLA The robotics job market talks about VLA and foundation models, but the…

This week-long log chronicles the author's transition from machine learning and quantitative analysis roles into robotics research. The primary tool utilized throughout this first week was MuJoCo, a physics-based simulation environment that can be run on a CPU-only Linux system.

On Day 0, the author set up a locked-down Linux environment with no GPU capabilities, using the PEP 668 system Python. They installed necessary MuJoCo CPU wheels and noted the cost of roughly an hour spent troubleshooting errors.

Day 1 focused on establishing a rigorous measurement process before any training commenced. The author timed a single step execution for the HalfCheetah environment at approximately 50 microseconds, indicating that the computational bottleneck lies in gradient updates rather than raw physics calculations. Specifically, a Soft Actor-Critic (SAC) update batch of 256 samples took about 6 milliseconds. The author emphasizes the importance of quantifying these metrics before committing to more resource-intensive tasks.

On Day 1, a sanity check was conducted on the Pendulum environment using the full pipeline. This included squashed Gaussian noise, twin critics, and learned temperature adjustment. The results showed return values ranging around -110 to -170 over 100,000 steps on a CPU with both one and two gradient updates per step. Any subsequent reinforcement learning code being developed should meet or exceed these benchmarks before progressing to more compute-heavy tasks.

Days 1-2 were dedicated to reproducing results from a benchmark paper using HalfCheetah. With one million steps and a single random seed, the training took 109 minutes wall clock time on a 16-core CPU. The final evaluation scored 9,180 points (with a best of 9,914), surpassing the paper's reference of approximately 6,000 for the same computational budget. All evaluation data, logs, plots, and comparison tables were generated programmatically, allowing readers to independently verify the results.

On Day 2, the author documented the reproduction effort in a README file, clearly delineating what was actually reproduced versus what was not claimed. This README serves as a portfolio artifact that interviewers can validate, emphasizing the reproducibility aspect which is often overlooked in robotics portfolios. By providing logs, evaluation CSV files, plotting scripts, and a generated comparison table, the author demonstrates a commitment to transparency and verifiability.

Throughout the week, all outputs were recorded and MIT-licensed, with the code, training logs, and comparison tables made available through an MIT license on the sac-mujoco-repro repository. For those who prefer not to run the 1M-step training themselves, a packaged version containing all the above artifacts can be purchased for $1 from their store. The author argues that this modest price serves to collect feedback on the experiment and does not prevent access to the free repository.

Rewritten by urgent.news; may contain errors — check the original.

Source: dev.to

Read the original at dev.to →

Report a problem with this story

More in Tech

Why PDF previews go blank on high-density screens

A PDF preview can look fine on a laptop and go blank on a high-density phone. The file may be small on disk, yet the viewer allocates a large canvas for every rendered page.

  • High-density screens cause PDF previews to appear blank due to memory constraints.
  • Large canvas for each rendered page multiplies dimensions based on device's pixel ratio.
  • Setting per-page budget for canvas dimensions can prevent excessive memory consumption.

More from Sunday 22 November →