ReAct (Reason + Act)

1 · What it is

ReAct is a way of prompting a language model so it takes turns. A prompt is the text you give the model to start it off. With ReAct, the model thinks, does one thing, looks at what came back, then thinks again. Researchers from Princeton University and Google’s Brain team presented it at ICLR 2023, a major AI conference.

Some steps are actions that touch the outside world, like a search. Others are thoughts: sentences of reasoning that touch nothing, like notes to itself.

The paper used a big model, PaLM-540B, left “frozen” (its inner settings unchanged). The prompt showed it a few worked examples: six for HotpotQA, a question set, and three for FEVER, a fact-checking set.

The model had three moves: search, lookup (like Ctrl+F on a page) and finish.

In FEVER, the model labels a claim as supported, refuted, or not enough information. Two claims can differ by one small detail, so correct facts matter.

In the figure, the model never guesses the years. It reads 1844 and 1989 from search results, then compares them.

Why not just act, or just think?

The paper tested ReAct against simpler set-ups. “Act” is ReAct with the thoughts taken out, so the model only issues actions and reads results. “Chain-of-thought” (CoT) is the opposite: thoughts only, with no actions and no results from outside.

ReAct beat Act on both HotpotQA and FEVER. Thinking helps the model choose better actions. It helps most with the final answer.

It also beat Act on ALFWorld, a text game, and WebShop, a shopping site.

Against chain-of-thought, the picture is mixed. ReAct won on FEVER. On HotpotQA it came out slightly behind. Chain-of-thought lays out its reasoning better, but it often makes up facts. ReAct sticks to facts it looked up. But its fixed rhythm makes it less flexible.

Mixing the two

Each method is strong where the other is weak, so the authors combined them. They used self-consistency: ask chain-of-thought 21 times and keep the most common answer. Then simple rules switch methods:

• ReAct first, then CoT. If ReAct has not answered within a set number of steps (7 for HotpotQA, 5 for FEVER), fall back to chain-of-thought.
• CoT first, then ReAct. If the most common chain-of-thought answer shows up in fewer than half the samples, the model’s own knowledge may be shaky, so fall back to ReAct and search.

These mixes gave the best prompting results. ReAct-then-CoT was best on HotpotQA, and CoT-then-ReAct was best on FEVER. Both matched self-consistency’s 21 samples using only 3 to 5.

Where you see it now

The same loop now sits inside agent libraries. Hugging Face’s smolagents docs call ReAct the main way people build agents today. Every smolagents agent is built on one shared class that packages the ReAct loop.

OpenAI’s Agents SDK (a toolkit for building agents) runs a similar loop. It runs the tools the model asks for. It adds the results. Then it calls the model again. It stops at a limit called max_turns. LangChain’s docs put it simply: an agent keeps using tools, round after round, until its job is finished.

Anthropic stresses getting real results, such as tool outputs, at every step. It also notes a common stopping rule to stay in control: cap how many times the loop may run.