HTTP API

optics serve runs a local HTTP server that lets any language or tool drive a device through Optics. You start a session, send keywords one at a time, and read screenshots, UI trees and events back.

For a guided introduction, see Drive Optics over HTTP. The machine-readable OpenAPI document is at https://optics-framework.org/openapi.json.

Start the server#

optics serve [--host HOST] [--port PORT] [--workers N]

The server prints its own log lines and Optics' debug logs to the terminal. Interactive docs are at http://127.0.0.1:8000/docs while it runs.

The server doesn't read config.yaml or your project's test files. Each session gets its engines from the request. Sessions also don't write screenshots or page sources to disk, and they log at DEBUG.

Endpoints#

SessionConfig#

The body of POST /v1/sessions/start. Every field is optional, but a session needs a driver and at least one element source.

Engine names are the same as in config.yaml. LLM credentials never go in the request; set them in the server's environment.

ExecuteRequest#

The body of POST /v1/sessions/{id}/action.

The session has no elements file, so pass locators directly, or define variables first with keywords such as Evaluate.

Responses#

A successful action returns:

{
  "execution_id": "4f1c2b7e-...",
  "status": "SUCCESS",
  "data": { "result": "Trail Runner 2" }
}

• When the keyword returns a value that isn't an object, data is {"result": <value>}.
• When the keyword returns an object, data is that object. This is why /source and /screen_elements put their fields directly in data.
• When self-heal recovers a step, data is {"result", "healed": true, "heal_summary", "suggested_steps"}. suggested_steps lists the keywords and parameters that fixed it.

Unlike optics execute, the HTTP API returns keyword results, so Get Text, Assert Equality and Execute Script values come back to you.

Errors#

• The code field can carry a Code. prefix, as in Code.E0402. It's the same code. See Error codes.
• DELETE /stop on an unknown session returns 500 with Session termination failed: 404: Session not found.
• If closing the app fails during DELETE /stop, the session isn't removed. Call it again.

Streams#

Both stream endpoints use server-sent events (SSE): the connection stays open and each event is a data: line of JSON.

**/events** sends one event when each keyword starts, passes or fails, including the automatic launch_app at start, close_and_terminate_app at stop, and the GET helpers such as /screenshot.

data: {"execution_id": "4f1c...", "status": "RUNNING", "message": "Starting keyword: press_element"}
data: {"execution_id": "4f1c...", "status": "SUCCESS", "message": "Keyword press_element executed successfully"}

• Statuses are RUNNING, SUCCESS and FAIL, plus ERROR or CANCELLED when the stream itself stops.
• After 15 seconds without events, it sends {"execution_id": "heartbeat", "status": "HEARTBEAT", ...}.
• Events wait in a queue until someone reads them. With several clients, each event goes to only one of them, so connect one client per session.

**/workspace/stream** sends the screen whenever it changes.

Each event is {"screenshot", "elements", "screenshotFailed"}, plus source when requested. It's sent only when the screen or elements change. Otherwise a {"type": "heartbeat"} arrives every 15 seconds, and errors arrive as {"type": "error", "message"}. The stream ends when the session ends.

Walkthrough#

This drives the Birch app on an Android emulator. You need an Appium server on port 4723 and the appium extra.

1. 1

   Start the server#

   Copy

   $ optics serve --port 8000

   Leave it running and open a second terminal.

2. 2

   Start a session#

   Copy

   $ curl -s -X POST http://127.0.0.1:8000/v1/sessions/start \
   >   -H 'Content-Type: application/json' \
   >   -d '{"driver_sources": [{"appium": {"url": "http://127.0.0.1:4723",
   >          "capabilities": {"platformName": "Android", "automationName": "UiAutomator2",
   >            "deviceName": "emulator-5554", "appPackage": "com.birch.shop",
   >            "appActivity": "com.birch.shop.MainActivity"}}}],
   >        "elements_sources": ["appium_find_element", "appium_page_source", "appium_screenshot"],
   >        "text_detection": [], "image_detection": []}'
   {"session_id":"8d0e7c55-...","driver_id":"...","status":"created"}

   The app launches. Save the session id:

   Copy

   $ SID=8d0e7c55-...

3. 3

   Watch events (optional)#

   In a third terminal:

   Copy

   $ curl -N http://127.0.0.1:8000/v1/sessions/$SID/events

4. 4

   Run keywords#

   Copy

   $ curl -s -X POST http://127.0.0.1:8000/v1/sessions/$SID/action \
   >   -H 'Content-Type: application/json' \
   >   -d '{"mode": "keyword", "keyword": "Press Element", "params": [["//*[@resource-id=\"com.birch.shop:id/search\"]", "text=Search"]]}'
   $ curl -s -X POST http://127.0.0.1:8000/v1/sessions/$SID/action \
   >   -H 'Content-Type: application/json' \
   >   -d '{"mode": "keyword", "keyword": "Enter Text", "params": {"element": "text=Search", "text": "Trail Runner 2"}}'
   $ curl -s -X POST http://127.0.0.1:8000/v1/sessions/$SID/action \
   >   -H 'Content-Type: application/json' \
   >   -d '{"mode": "keyword", "keyword": "Assert Presence", "params": ["text=Trail Runner 2", "20"]}'

   The first call passes a list for element, so Optics tries the XPath and then the text.

5. 5

   Save a screenshot#

   Copy

   $ curl -s http://127.0.0.1:8000/v1/sessions/$SID/screenshot \
   >   | python3 -c 'import sys,json,base64; open("screen.png","wb").write(base64.b64decode(json.load(sys.stdin)["data"]["result"]))'

6. 6

   End the session#

   Copy

   $ curl -s -X DELETE http://127.0.0.1:8000/v1/sessions/$SID/stop
   {"status":"terminated"}

   Optics writes junit_output.xml for the session to execution_output/ in the project path, or in the folder where the server runs.

Limits#

• Sessions live in memory. Restarting the server ends them, and optics live and optics mcp can't see them.
• There's no record or save endpoint. Write the steps you ran into project files to replay them with optics execute.
• Error detection doesn't run over HTTP.
• With the ble driver, the automatic launch_app at session start fails, because BLE can't launch apps.