Computer Use
Computer use is the capability of an AI agent to operate a computer like a person: it reads the screen through screenshots, moves the cursor, clicks, types and scrolls in any software, including programs that expose no API.
No jargon, no math
In plain words, what is it?
The agent works in a loop of screenshots. It captures the screen, the model reads the image the way you do, decides the next interface action (click this, type that, scroll), and a controller executes it through the operating system's accessibility or input layer. Then a new screenshot confirms what happened and the loop restarts. This is how you automate the software nobody can automate: the 2009 ERP, the customs portal, the supplier's booking site, none of which have an API. It breaks three ways: brittle layouts, because a moved button breaks the flow; latency, because every step is a round-trip of screenshot, model, action; and risk, because an agent that can click “save” can just as easily click “send”, so permissions and logs are not optional.
- 01
See. Capture the screen and read it like a person.
- 02
Decide. Choose the next interface action: click, type, scroll.
- 03
Act. Execute it through the input layer.
- 04
Verify. A new screenshot confirms the action landed.
- 05
Repeat or stop. Loop until the task is done or a permission stops it.
Terms that live nearby.
AI Agent
An AI agent is software that pursues a goal on your behalf: it reads the situation, plans the next step, acts through tools (email, calendars, databases, APIs) and checks the result before continuing, instead of answering one question and stopping.
ReadTool Use
Tool use is the capability of an AI model to do more than write text: it can call external tools, such as searching a database, sending an email or running a calculation, by emitting a structured request that software executes for real.
ReadAgent Loop
The agent loop is the cycle an AI agent repeats while pursuing a goal: observe the current state, plan the next action, act through a tool, check the result, then continue, re-plan or stop.
Read§ 04
Want this running in your stack?
We design and ship AI systems with permissions, logs and human approval. It starts with a 30-minute scoping call.