文档Docs

屏幕求助：扫码和验证码发到手机上Screen help: QR codes and verification codes on your phone

自动化卡在二维码、验证码或人机验证时，Agent J 把二维码或整屏截图发到你配对好的手机上，你扫码、点完成或回一句答案，它接着干。When automation stops at a QR code, a verification code or a human check, Agent J puts the code or a full screenshot on your paired phone; you scan, tap Done or type the answer, and the work goes on.

Agent 在浏览器里干活，难免会卡在需要人确认的地方：登录要扫码、填验证码、证明自己不是机器人。你不在电脑前也没关系——Agent J 会把画面发到你配对的手机上。

页面上有二维码：Agent J 把二维码单独裁出来，放大发给你，你直接扫。
别的情况（验证码、人机验证、找不到二维码）：Agent J 发一张高清截图，可以是浏览器页面，也可以是整个电脑屏幕。你看图扫码、点完成，或者把要填的内容写回去。

Agent J 专用浏览器与登录保活 · 手机上怎么用 · 命令参考

手机上会看到什么

手机上弹出一张求助卡，标题写着「扫码求助」或「屏幕求助」，后面注明是 Agent 发的，还是你自己的哪个自动化发的。

中间一张大图，下面是倒计时。二维码默认有效 2 分钟，截图默认 5 分钟。
点一下截图，全屏放大；再点一下，还原。
同一台手机扫不了自己的屏幕。可以用另一台设备显示这张卡，或者长按图片保存到相册，再去对应 App 里识别。
二维码卡：扫完点「我扫好了」。
截图卡：照着画面处理好，点「已完成」。要填验证码这类内容，写在下面的输入框里，点「发送」。
现在不想管，点「先不处理」。Agent 会知道你没处理，停下来告诉你卡在哪里。

卡片会同时出现在你所有配对好的遥控器上。在一台上处理完，其他几台的也会一起关掉。如果有多张卡片，一次只显示一张，下面会写着后面还有几张。

如果 Agent 在盯着页面，你扫完码、页面一跳转，卡片会自己关掉，不用再点。

Agent 怎么用它

这是 Agent J 自带的能力，不用安装。主 Agent 带着 agentj-screen-help 技能，自动化卡在扫码、验证码、人机验证时会自己用。

你也可以直接跟它说，比如：卡在扫码就把码发我手机，或者 把屏幕截给我看看，我告诉你验证码。

Agent 拿到结果后，会自己再看一眼页面：登录成功了才继续。二维码过期了，它会刷新页面再发一次；连续两次没成，就停下来告诉你。

命令

这些命令一般由 Agent 来跑，你知道有这回事就行。

把浏览器当前页面上的二维码裁出来发到手机，页面一变就自动关卡：

agentj qr push --text "公众号后台登录要扫码" --wait-until page-change

只截某个标签页上的某一块（--tab 填网址里的一段文字，--selector 填页面元素）：

agentj qr push --tab weixin --selector ".login-qrcode" --wait-until answered

发一张截图问你验证码是什么，等你回答：

agentj screen ask --question "验证码是什么？"

截整个电脑屏幕，只要你点完成：

agentj screen ask --screen --question "请帮我点一下允许" --reply done

一次最多等 170 秒。没等到结果会返回 pending 和卡片编号，用下面两条继续等，或者撤回：

agentj screen wait <id>

agentj screen cancel <id>

结果以 JSON 打印出来：done 是你点了完成，answered 是你写了答案（在 answer 里，只交出一次），page_changed 是页面已经往下走了，cancelled 是你点了「先不处理」，expired 是过期了。只有 shown 为 true 才说明卡片真的到了手机上。

用在你自己的脚本里

不经过 Agent 也能用。你自己的 cron、systemd 定时任务在这台电脑上跑到要扫码的地方，同样调用这两条命令，加上 --from 写个名字，卡片上就会显示是哪个自动化发的：

agentj qr push login.png --from nightly-upload --text "后台登录要扫码" --wait-until answered

名字只能用字母、数字和 . _ -。
只收图片：PNG、JPEG 或 WebP，每张不超过 1.4 MB。只有 PNG 会自动裁出二维码，其他格式原样发送。
每个来源每分钟最多几张，同时最多 4 张卡开着。
只有这台电脑上、和 Agent J 同一个用户跑的程序才能发。

本地找不到二维码时

二维码在这台电脑上找、在这台电脑上裁，不花钱，图片也不出电脑。裁好后会再识别一遍，跟原图对得上才用裁好的，对不上就发整张图。

想让本地找不到时再试一次视觉模型，可以给 agentj qr push 加上 --vision，并在运行这条命令的环境里设好三个变量：AGENTJ_VISION_URL、AGENTJ_VISION_MODEL、AGENTJ_VISION_API_KEY（OpenAI 兼容的接口，用你自己的账号）。默认关闭；只有本地什么也没认出来时才会调用，用了这一步，这张图会发给你配置的那个模型服务。

支持哪些电脑

裁二维码：Linux（x86_64、aarch64）和 macOS。其他电脑上不裁，整张图发过去。
截整个屏幕（--screen）要有桌面：macOS 用系统自带的 screencapture；Linux 在 Wayland 下用 grim，在 X11 下用 import、scrot 或 gnome-screenshot。没有图形界面的服务器截不了整屏，改用浏览器页面或一张图片。

隐私

截图里可能有你的私人内容。Agent J 这样处理：

只通过端到端加密发到你自己配对的遥控器。不发好友，不发给外部机器人，不走 Telegram，也不走邮件。
不进聊天记录，不存手机，Agent J 主机上也只存在内存里。卡片一结束（处理完、撤回、过期、点「全部停下」或重启）就删掉。
日志里只记编号、类型、大小和结果，不记图片、问题、答案，也不记二维码里的内容。
别在这张卡里填密码。看起来像密码或密钥的答案，手机会拒绝发送。要把密钥交给 Agent，用手机上的密钥卡（agentj secret request）。

技术细节

命令通过本机的 Agent J 通道（同一用户才能连）交给 Agent J 主机端程序，再走端到端加密送到手机。
二维码用 zxing-cpp 在本机识别；裁剪范围是最大的那个二维码，四周补上白边，再解码一次，结果一致才用。
卡片有效期可以用 --ttl 改：二维码最长 10 分钟，截图最长 15 分钟。
退出码：0 已发出 / 已完成 / 页面已变，3 被取消，4 还在等，5 没送到或已过期，2 参数或图片不对，1 Agent J 没在运行。

An Agent working in a browser sooner or later reaches a step only a person can pass: a sign-in QR code, a verification code, a check that you are not a robot. You do not need to be at the computer — Agent J sends that screen to the phone you paired.

A QR code on the page: Agent J cuts the code out, enlarges it and sends it to you. You scan it.
Anything else (a verification code, a human check, a code it could not find): Agent J sends a high-resolution screenshot — of the browser page or of the whole computer screen. You scan from it, tap done, or type what needs to be filled in.

Agent J browser and sign-ins · Using the phone · Command reference

What the phone shows

A help card opens on the phone. Its top line says "Scan request" or "Screen help" and whether it comes from the Agent or from one of your own automations.

The picture fills the middle, with a countdown below it. A QR code stays valid for 2 minutes by default, a screenshot for 5 minutes.
Tap a screenshot to see it full screen; tap again to go back.
A phone cannot scan its own screen: show the card on another device, or long-press the picture to save it and pick it in the scanning app.
QR card: scan, then tap "I scanned it".
Screenshot card: deal with what it shows and tap "Done". If something must be typed in (a verification code), write it in the box below and tap "Send".
Not now? Tap "Not now". The Agent learns you skipped it, stops and tells you where it is stuck.

The card shows on every remote you paired; once you deal with it on one, it closes on the others too. When several cards are waiting, you see one at a time with a note of how many follow.

If the Agent is watching the page, the card closes by itself as soon as the page moves on after your scan.

How the Agent uses it

This is built into Agent J; there is nothing to install. The main Agent has the agentj-screen-help skill and uses it on its own whenever automation is stuck on a QR code, a verification code or a human check.

You can also just ask, for example: if you get stuck on a QR code, send it to my phone, or show me the screen and I will tell you the code.

After your answer the Agent looks at the page again and carries on only if the sign-in really worked. If the code expired, it reloads the page and sends it once more; after a second miss it stops and tells you.

Commands

The Agent usually runs these; it is enough to know they exist.

Cut the QR code out of the browser's current page, send it, and close the card when the page changes:

agentj qr push --text "Sign-in needs a scan" --wait-until page-change

Only one part of one tab (--tab is a piece of the address, --selector a page element):

agentj qr push --tab weixin --selector ".login-qrcode" --wait-until answered

Send a screenshot asking for the verification code, and wait for your answer:

agentj screen ask --question "What is the verification code?"

The whole computer screen, asking only for a tap on done:

agentj screen ask --screen --question "Please click Allow for me" --reply done

One call waits at most 170 seconds. If there is no outcome yet it returns pending with the card's id; keep waiting, or withdraw the card:

agentj screen wait <id>

agentj screen cancel <id>

The outcome is printed as JSON: done (you tapped done), answered (you typed an answer; it is in answer and handed over once), page_changed (the page moved on), cancelled (you tapped "Not now"), expired. Only shown set to true means the card actually reached the phone.

In your own scripts

No Agent needed. When one of your own cron jobs or systemd timers on this computer reaches a sign-in, it can call the same commands. Add --from with a name and the card shows which automation sent it:

agentj qr push login.png --from nightly-upload --text "Admin sign-in needs a scan" --wait-until answered

Names use letters, digits and . _ - only.
Pictures only: PNG, JPEG or WebP, up to 1.4 MB each. Only PNG pictures get the QR code cut out; other formats go as they are.
A few cards a minute per source, at most 4 open at once.
Only programs on this computer, running as the same user as Agent J, can send.

When the code is not found locally

The QR code is found and cut out on this computer: it costs nothing and the picture stays here. The cut is read again and used only if it gives the same result as the original; otherwise the whole picture goes.

To give a vision model a try when the local search finds nothing, add --vision to agentj qr push and set three variables in the environment that runs the command: AGENTJ_VISION_URL, AGENTJ_VISION_MODEL and AGENTJ_VISION_API_KEY (an OpenAI-compatible service on your own account). It is off by default and only called when nothing was recognised locally; when it is used, that picture goes to the model service you configured.

Which computers

QR cut-out: Linux (x86_64, aarch64) and macOS. Elsewhere the whole picture is sent uncut.
Whole-screen capture (--screen) needs a desktop: macOS uses its own screencapture; Linux uses grim on Wayland, and import, scrot or gnome-screenshot on X11. A server without a desktop cannot capture its screen — use the browser page or a picture file instead.

Privacy

A screenshot may show private things. This is how Agent J handles it:

It travels only end-to-end encrypted to your own paired remotes. Never to friends, public bots, Telegram or e-mail.
It never enters chat history and is not saved on the phone; on the Agent J host it is kept in memory only. It is deleted when the card ends (dealt with, withdrawn, expired, "Stop everything" or a restart).
Logs record only the id, type, size and outcome — never the picture, the question, the answer or what the QR code contains.
Never type a password into this card. An answer that looks like a password or key is refused on the phone. To hand a key to the Agent, use the phone's key card (agentj secret request).

Technical details

The commands reach the agentj program on the Agent J host over a local channel open only to the same user; from there the card goes end-to-end encrypted to the phone.
QR codes are found locally with zxing-cpp; the cut is the largest code plus a fresh white margin, decoded again and used only when the result matches.
--ttl changes how long a card stays: up to 10 minutes for a QR code, 15 minutes for a screenshot.
Exit codes: 0 sent / done / page changed, 3 cancelled, 4 still waiting, 5 not shown or expired, 2 bad arguments or picture, 1 Agent J is not running.

这一页的 Markdown 原文This page as Markdown: zh.md · en.md