Skip to main content

How LLMs Are Trained After Pretraining: SFT, Reward Models, and RL Without the Alphabet Soup

A practical map of LLM post-training: how SFT, reward models, RL (PPO, GRPO), DPO, and RLVR fit together, and why a reward model is not RL.

hackernoon.com ·1d ago

How LLMs Are Trained After Pretraining: SFT, Reward Models, and RL Without the Alphabet Soup

We haven't written up this one. HackerNoon has the full story — the link below goes straight to it.

Read the original at hackernoon.com →

More in AI

AI in schools brings back pen and paper

AI in schools brings back pen and paper

A child sitting down to complete homework in 2026 has more assistance within reach than any previous generation. A chatbot can explain a maths problem, suggest an essay structure, summarise a chapter…

1d ago ·gulfnews.com

Moving past the chatbox: The hidden risks of agentic AI and MCP in enterprise infrastructure

Moving past the chatbox: The hidden risks of agentic AI and MCP in enterprise infrastructure

In Singapore, Hong Kong, and across the APAC region, the corporate adoption of Generative AI has completed its initial trial phase.

1d ago ·e27.co

Hyperscaler AI borrowing binge shakes up foreign credit markets

Hyperscaler AI borrowing binge shakes up foreign credit markets

Flood of bond issuance by likes of Amazon and Alphabet has pushed up costs in Canadian dollars, Swiss francs and sterling

1d ago ·ft.com

A recent controversy surrounding an artificial intelligence (AI) system's grading for an English essay test has sparked concerns over fairness and accuracy in automated evaluations. The issue came to light when a student took the College Scholastic Ability Test (CSAT) mock exam, specifically the English section, and received an unexpectedly high score for an essay response that was later found to contain several instances of specific phrases or "buzzwords" that are often favored in evaluations. The student expressed frustration upon learning that despite their teacher grading the same essay as a C, the AI system awarded it an A. This discrepancy has raised questions about the reliability of AI in assessing student performance, particularly when it appears to prioritize certain keywords over the logical coherence or overall quality of the response. Further investigation revealed that the AI system seemed to give more weight to the presence of certain "buzzwords" rather than the content or logical structure of the essay. This approach has been criticized for potentially rewarding students who are adept at identifying and incorporating favored phrases into their work, rather than those who demonstrate a deeper understanding or more nuanced expression of the subject matter. The Korea Institute of Curriculum and Evaluation (KICE), the organization responsible for developing and grading the CSAT, has acknowledged the issue and is taking steps to address concerns over the AI grading system. KICE officials have stated that they are working to refine the AI evaluation algorithm to ensure that it assesses responses based on a more balanced and comprehensive set of criteria, rather than relying heavily on specific keywords. The incident highlights the challenges and limitations of using AI in educational assessments, particularly in evaluating subjective or creative tasks such as essay writing. As educators and policymakers continue to integrate AI into various aspects of education, ensuring the fairness, accuracy, and reliability of AI systems in grading and evaluation will be crucial. The goal is to create a system that supports and fairly assesses student learning, without inadvertently encouraging superficial strategies or disadvantaging students who approach problems differently.

1d ago ·hani.co.kr

Chinese chipmaker SMIC increases prices on strong AI demand

Chinese chipmaker SMIC increases prices on strong AI demand

Co-CEO Zhao Haijun said on an earnings call that SMIC raised prices following negotiations with customers in the first quarter, and that it would charge more for wafers processed in the third quarter.

1d ago ·economictimes.indiatimes.com

Korea's ICT exports up 140% in July on sustained AI chip demand

Korea's ICT exports up 140% in July on sustained AI chip demand

Korea's exports of information and communication technology (ICT) products jumped 140.6 percent in July from a year earlier, driven by sustained global investments into artificial intelligence (AI)…

1d ago ·koreatimes.co.kr

More from Friday 14 August →