ScreenAgent
Overview
Open‑source VLM agent to control computer GUIs via mouse/keyboard planning and execution.
Best For Professions:
ScreenAgent is an open‑source Vision Language Model agent that interacts with real computer screens via screenshot observation and mouse/keyboard actions, following a planning‑execution‑reflection loop. It supports multi‑step GUI tasks, dataset collection, and achieves positioning accuracy comparable to GPT‑4V.
Autonomy level
81%
Reasoning: ScreenAgent demonstrates high autonomy through its three-phase operational framework: planning, acting, and reflecting. It enables continuous interaction with computer environments without human intervention by autonomously assessing execution status and adjusting actions in real-time. The reflection phase allows self-evaluation of action outcomes,...
Comparisons
Custom Comparisons
Some of the use cases of ScreenAgent:
- Automating desktop tasks with LLM-based GUI control.
- Building VLM agents that plan, act and reflect over screen state.
- Collecting and leveraging GUI interaction datasets.
- Researching multi-step visual task execution.
Loading Community Opinions...
Pricing model:
Code access:
Popularity level: 44%
Tags:
Make it work for you
Describe the job. Get an AI worker you can actually message.
We create the setup, keep it running after your laptop closes, and save its memory. Test in the browser, then add Telegram, WhatsApp, or Slack.
Runs without your laptopBrowser + messaging appsBackups and clonesMemory survives restarts
Plans start at $29/month. Cancel anytime.
Hosted agent
OpenClaw or Hermes
Browser
WhatsApp
Telegram
Slack
“I checked the inbox, handled the routine messages, and sent you the one question that needs a decision.”
Create an AI worker that keeps running after this tab closes.
Open Agent FactoryDid you find this page useful?
Not useful
Could be better
Neutral
Useful
Loved it!